VPS Hypervisor Down - s655
Incident Report for Crucial Hosting
Resolved
The root-cause has been identified as a combination of a faulty RAID card and some faulty memory modules. After systematic testing and replacement of all server components we are confident in the stability of this HyperVisor.
Posted Apr 14, 2016 - 16:01 AEST
Update
All VMs are now booted. We will continue to monitor the server.
Posted Apr 14, 2016 - 07:02 AEST
Monitoring
VMs are now being booted.
Posted Apr 14, 2016 - 06:56 AEST
Identified
Memory has been replaced and server is being rebooted.
Posted Apr 14, 2016 - 06:41 AEST
Investigating
Be advised our internal monitoring has indicated that the VPS Hypervisor s655 is down or unavailable again. As a result your VPS service that resides on this node may not be accessible or connectible.

We are investigating this and further updates will follow within the next 60 minutes.
Posted Apr 14, 2016 - 05:43 AEST
Monitoring
Server is operational and spare memory is currently being tested as a precaution.
Posted Apr 14, 2016 - 03:45 AEST
Identified
We appear to have a bad batch of RAM. A couple of the memory modules in the replacements have failed. These have now been replaced and the server is operational again. We are currently testing another batch in case it reoccurs.
Posted Apr 14, 2016 - 02:51 AEST
Update
We have found that one of the replacement hardware components is causing issues, we are reconfiguring the hardware for this particular component.
Posted Apr 14, 2016 - 02:09 AEST
Update
Testing has revealed additional issues, we are working to resolve the issues found.
Posted Apr 14, 2016 - 01:48 AEST
Update
We have replaced several more hardware components and are performing additional testing before bringing this online.
Posted Apr 14, 2016 - 01:28 AEST
Investigating
Be advised our internal monitoring has indicated that a VPS Hypervisor is down or unavailable. As a result your VPS service that resides on this node may not be accessible or connectible.

Following the previous hardware changes, we will be replacing the last components that have not been swapped. We are actively working on this and further updates will follow in the next 60 minutes.

This is being addressed as a matter of priority.
Posted Apr 14, 2016 - 00:36 AEST