Software error complicates Amazon's data center recovery

The storage snapshot management system incorrectly deleted data but it should be recovered

Amazon Web Services' efforts to restore service following a power outage at its Dublin data center were complicated further on Monday by an error in the EBS (Elastic Block Storage) software, the company said.

As a result of the software error, the EBS snapshot management system incorrectly thought some of the blocks were no longer being used and deleted them, Amazon wrote on its Service Health Dashboard at 3:11 PM PDT.

The company has addressed the error to prevent it from recurring and also disabled all of the snapshots that contain these missing blocks. Amazon will send e-mails to affected customers as soon as it has a new copy of their snapshots available, which can then be used to recover the data, it said.

At 10:01 PM PDT, Amazon said that it had recovered all of the EBS volumes and EC2 (Elastic Compute Cloud) instances that it was able to verify were fully consistent at the time of the power outage.

However, the company was unable to verify whether or not there were any in-flight writes that did not get consistently saved to some EBS volumes. To remedy that, it has started creating recovery snapshots for the affected volumes. As they become available, the snapshots will be added to users' accounts, according to Amazon. The process will be time consuming and may take up to 24 hours to fully complete, it said.

The availability problems started after lightening struck a transformer, sparking an explosion and fire which caused the power outage at 10:41 AM PDT on Sunday.

The last few days have not been very merry for Amazon Web Services. Besides the issues in Dublin, Amazon also suffered a brief outage in the U.S. as a result of network connectivity issues.

Send news tips and comments to mikael_ricknas@idg.com

Tags Amazon Web Servicesamazon.comweb servicesinternetcloud computingSoftware as a serviceInfrastructure services

Recommended

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Mikael Ricknäs

IDG News Service

Comments

Comments are now closed.

Most Popular Reviews

Follow Us

Best Deals on GoodGearGuide

Shopping.com

Latest News Articles

Resources

GGG Evaluation Team

Kathy Cassidy

STYLISTIC Q702

First impression on unpacking the Q702 test unit was the solid feel and clean, minimalist styling.

Anthony Grifoni

STYLISTIC Q572

For work use, Microsoft Word and Excel programs pre-installed on the device are adequate for preparing short documents.

Steph Mundell

LIFEBOOK UH574

The Fujitsu LifeBook UH574 allowed for great mobility without being obnoxiously heavy or clunky. Its twelve hours of battery life did not disappoint.

Andrew Mitsi

STYLISTIC Q702

The screen was particularly good. It is bright and visible from most angles, however heat is an issue, particularly around the Windows button on the front, and on the back where the battery housing is located.

Simon Harriott

STYLISTIC Q702

My first impression after unboxing the Q702 is that it is a nice looking unit. Styling is somewhat minimalist but very effective. The tablet part, once detached, has a nice weight, and no buttons or switches are located in awkward or intrusive positions.

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?