Network Outage (RESOLVED)

  • Tuesday, 28th May, 2019
  • 09:38am

Unfortunately at this time, we are experiencing a major outage on our primary cloud manager. We are currently investigating the issue and will provide an update and ETA as soon as possible. Our data center is working diligently to get the equipment back up ASAP.

We sincerely appreciate your business and patience. We were/are actively planning on moving all clients to a more stable solution. While our Ceph Cluster has proven to have many advantages and I still think its great technology, its complexity also has some disadvantages and when it fails, it fails hard. Uptime and performance are paramount to any solution and we will be migrating to a more common local solution in the near future. This will allow for quicker recoveries and more isolated outages (that we obviously want to avoid in general).

2:12 PM CST - All Hardware is back online, but central storage array is refusing connections.

4:05 PM CST - 2 x SSD's replaced in storage array and currently rebuilding. All systems have been booted, but are extremely slow during the array recovery process.

6:09 PM CST - All systems are fully functional at this time.

« Back