(06:34 UTC — Mar 17) Resolved – We’re marking this as resolved as we are seeing everything settle down. We have also verified that a majority of backups completed, and we told Jahy to continue where it left off. We apologize for any downtime that occurred within the last hour.
(06:18 UTC — Mar 17) Monitoring – We continue to monitor all affected services and hope to see full recovery by 2:30 AM ET.
(06:13 UTC — Mar 17) Update – Waiting for internet to fully reset. Not sure what’s going on but power surges do suck. One thing we really wish to invest in, is a powerful enough backup UPS that will keep most of the equipment online so things recover faster and have time to properly shutdown. Unfortunately, Jahy was in a middle of a backup cycle and wasn’t able to complete it. We’ll attempt backups again.
(06:05 UTC — Mar 17) Update – Firewall became degraded but recovered. Mow we are dealing with internet not obtaining.
(06:01 UTC — Mar 17) Update – Network infrastructure is taking a bit longer than usual to reinitialize.
(05:57 UTC — Mar 17) Investigating – We are aware of a system wide outage. updates will be provided soon.