When The Cloud Shuts Down

S3 Is Down” is never something any IT manager expected to hear but, on February 28, that is exactly what happened, turning the day into an interesting–and unplanned–test of our core infrastructure.

Over the past several years, we have moved more and more of our critical GTP and SKUpics infrastructure to AWS which has enabled us to rapidly scale and offer high-availability services. Fortunately we still prioritize on-site and redundant assets meaning that our companies can weather a storm, even one as big as “the cloud” shutting down.

While some customer services were affected for a few hours, GTP’s on-site customer-facing systems across the country remained functional throughout the outage and handled the Amazon failure in stride, properly caching and recovering following the AWS restoration.

At SKUpics, we were able to lean on our software’s proprietary “offline mode,” a capability designed to allow us to be fully operational in difficult shooting environments where network and power–or “the cloud”–are not always present.

Between our support team monitoring and responding to the event in Hershey, PA and our fault-tolerant on-site infrastructure, “The Day S3 Went Down” will be little more than a blip on our radar. Because of information we learned from how our systems responded to the outage, we’ll be able to mitigate future disruptions even further, making for an even stronger GTP and SKUpics.

The cloud may have shut down, but it’s always sunny at GTP and SKUpics!

Be sure to check out the full postmortem of the S3 event from Amazon.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s