Dublin 3 - HV17 - RFO

Scheduled on 24/05/2019 05:00:00 Status Resolved Fault / Issue Estimated finish 24/05/2019 15:00:00

Dear Dediserve Dublin users,

You may have noticed a period of service unavailability on Friday 24th of May, for which we sincerely apologize. Below is the formal RFO for the incident, please rest assured we are taking all reasonable measures to make sure this does not re-occur, thank you for your understanding.

What was the cause?:
A power surge caused a trip on the power protection unit for host #17, attempts were made to recover from the trip, including re-seating the blade and clearing NVRAM to remove the error, unfortunately these were unsuccessful indicating a damaged motherboard and we were forced to replace the host in its entirety.

What was the fix?:
Initially on-site engineers were instructed to re-seat the blade, in the hopes it would clear the power trip, following those unsuccessful attempts the NVRAM for the unit was manually cleared, finally we arranged for a whole-host replacement to avoid an even more protracted maintenance period.

What was the impact?:
From 5am VMs located on Hypervisor #17 specifically were unavailable, until the time of fix at 3pm. We apologise for the time it took to correct this problem, the multi-step diagnosis, coupled with the time it took to get engineers on-site prolonged the works.

Will the issue re-occur?:
Following replacement of the host all VMs were recovered successfully with no data loss, there is no reason that this issue should re-occur.

As always, if you have any questions, please let us know.

Related servers / services

Dubai Cloud Essential maintenance

Scheduled on 22/05/2019 23:00:00 Status Scheduled Estimated finish 23/05/2019 04:00:00

Dell Unity (Dubai-VNX01) OS and Firmware Upgrade

As part of our commitment to quality and continual improvement, we will be upgrading the Dell Unity (Dubai-VNX01) Operating Environment (OE) to the latest version. In addition to this, the disk firmware on the arrays will also be upgraded.

What will happen?

Each service processor and associated disks of the Unity system will be upgraded one at a time by downloading the software to the Unity system and invoking the upgrade service task, the disk firmware is completely non-disruptive and also does not require any restart.

When will the work be carried out?

The maintenance work will begin at 23:00 (GST) on Wednesday 22nd May 2019 and last until 04:00 (GST) on Thursday 23rd May 2019.

Who will be completing the work?

Dell EMC will establish a remote session and perform the upgrades with assistance from our engineering team.

Will I experience downtime?

As with any upgrade or maintenance work, there are risks present. To mitigate these risks the Dell Unity system is an active/active system and the partner service processor will take over the tasks of the upgraded service processor during the reboot that is required once the firmware has been applied.

All storage paths to each storage processor are also served via redundant storage fabrics. Dell EMC have also established the readiness for the upgrades and will perform a health check before commencing. In the unlikely event of a firmware upgrade failure the firmware will not be installed and the system will remain in its current state.

If you have any further questions or concerns regarding this work, please get in touch.

Related servers / services

Tokyo Cloud Essential maintenance

Scheduled on 06/05/2019 23:00:00 Status Completed Maintenance Estimated finish 07/05/2019 03:00:00

Dear Dediserve users,

Please be aware that at 23:00PM JST on Tuesday 6th May 2019 we will be performing upgrades to essential hardware in our Tokyo cloud, this should not cause any downtime, but as with all maintenance, there is a risk involved. Full information can be found below:

Dell Unity (TY8-UNITY-01) OS and Firmware Upgrade

As part of our commitment to quality and continual improvement, we will be upgrading the Dell Unity (TY8-UNITY-01) Operating Environment (OE) to the latest version. In addition to this, the disk firmware on the arrays will also be upgraded.

What will happen?

Each service processor and the associated disks of the Unity system will be upgraded one at a time by downloading the software to the Unity system and invoking the upgrade service task.

When will the work be carried out?

The maintenance work will begin at 23:00 (JST) on Monday 6th May 2019 and last until 03:00 (JST) on Tuesday 7th May 2019.

Who will be completing the work?

Dell EMC will establish a remote session and perform the upgrades with assistance from our Engineering team.

Will I experience downtime?

As with any upgrade or maintenance work, there are risks present. To mitigate these risks the Dell Unity system is an active/active system and the partner service processor will take over the tasks of the upgraded service processor during the reboot that is required once the firmware has been applied.

All storage paths to each storage processor are also served via redundant storage fabrics. Dell EMC have also established the readiness for the upgrades and will perform a health check before commencing. In the unlikely event of a firmware upgrade failure the firmware will not be installed and the system will remain in its current state.

Related servers / services