Major Incident Report

Major Incident Report

This page details the findings of our investigation into major service outages (MSOs). Thankfully MSOs occur extremely rarely and when they do occur we go through a detailed investigation and planning session to ensure that they cannot happen again.

If you would like this report e-mailed to you please contact us at feedback@tagadab.com

 

Major Incident Report - 1st December 2013

On Sunday 1st December one of our datacentres suffered a total loss of power, starting at approximately 16.40.  Power was restored at 17.30.

The exact cause of this incident is still not fully understood and we realise that after two weeks, we should be able to provide you with a full report on the matter.  We are working very closely with the datacentre to get more information on the exact cause of this power failure.  Our top priority is ensuring that we can provide you with a full report on the cause of this power failure as soon as possible.

Based on initial conversations we’ve had with the datacentre, it seems a component in the UPS failed, which might have caused the total failure in the UPS.  As you know, measures are put in place to prevent this from happening.  Each cabinet in the datacentre has two independent power supplies and typically the failure of a single component in the UPS cannot affect both.  In addition, any UPS failure should result in the mains power being supplied directly (bypass mode) but for some reason this did not happen.

The issue has now been referred to the manufacturer of the UPS in question and we are still waiting for their full explanation.  Again, we will share that with you as soon as we receive it.

In the meantime, a new UPS system upgrade is being planned.  This is not a result of this incident but was an upgrade planned several months ago to facilitate increased capacity.

We realise this might have impacted the services we provide you and we apologise for any inconvenience.  Once we receive a full report from the data centre, we will share it with you.