Degraded Performance
Incident Report for Crucial Hosting
Resolved
All services are now operational.
Posted Sep 05, 2019 - 18:39 AEST
Monitoring
Servers operational and in the process of completing database checks
sh18-eqx3-syd.ha-node.net
sh20-eqx3-syd.ha-node.net
rs36-eqx3-syd.ha-node.net

Servers now operational
sh11-eqx3-syd.ha-node.net
sh12-eqx3-syd.ha-node.net
sh17-eqx3-syd.ha-node.net
sh21-eqx3-syd.ha-node.net
sh27-eqx3-syd.ha-node.net
sh28-eqx3-syd.ha-node.net
rs25-eqx3-syd.ha-node.net
rs31-eqx3-syd.ha-node.net
rs37-eqx3-syd.ha-node.net
Posted Sep 05, 2019 - 17:23 AEST
Update
Servers currently undergoing filesystem checks
sh12-eqx3-syd.ha-node.net
sh28-eqx3-syd.ha-node.net
rs36-eqx3-syd.ha-node.net

Servers operational and in the process of completing database checks
sh12-eqx3-syd.ha-node.net
sh17-eqx3-syd.ha-node.net
sh18-eqx3-syd.ha-node.net
sh20-eqx3-syd.ha-node.net
sh21-eqx3-syd.ha-node.net

Servers now operational
sh11-eqx3-syd.ha-node.net
sh27-eqx3-syd.ha-node.net
rs25-eqx3-syd.ha-node.net
rs31-eqx3-syd.ha-node.net
rs37-eqx3-syd.ha-node.net
Posted Sep 05, 2019 - 17:03 AEST
Update
Additional servers have been identified as affected:

Web Hosting:
sh12-eqx3-syd.ha-node.net
sh18-eqx3-syd.ha-node.net
sh20-eqx3-syd.ha-node.net
sh21-eqx3-syd.ha-node.net

Reseller Hosting:
rs25-eqx3-syd.ha-node.net

The affected servers are being rebooted and having filesystem checks run.
Posted Sep 05, 2019 - 16:39 AEST
Update
We are in the process of running file system and database checks across the affected servers.

The server rs31-eqx3-syd.ha-node.net has just been rebooted.

We will provide further updates when there is a significant change in the situation.
Posted Sep 05, 2019 - 15:43 AEST
Identified
An issue has been detected with the underlying storage which houses these servers. Technicians are currently working to resolve this.
Posted Sep 05, 2019 - 15:28 AEST
Update
We have identified that the following hosting services are affected:

Web Hosting:
sh27-eqx3-syd.ha-node.net
sh28-eqx3-syd.ha-node.net

Reseller Hosting:
rs31-eqx3-syd.ha-node.net
rs32-eqx3-syd.ha-node.net
rs36-eqx3-syd.ha-node.net
rs37-eqx3-syd.ha-node.net
Posted Sep 05, 2019 - 15:04 AEST
Investigating
We are currently investigating a issue on some shared and reseller services. This is impacting connectivity to some services on our infrastructure.

We are treating this issue as priority one and are actively investigating.

We will update you on our progress in 15 minutes.
Posted Sep 05, 2019 - 14:54 AEST
This incident affected: Web Hosting Services and Reseller Hosting Services.