Rackspace Hosted Exchange Incident: Backup Your Cloud Data

On December 2nd, 2022, Rackspace posted an alert on their status page depicting an issue with their hosted exchange environment, which alluded to connectivity issues.  End users also took to social media to check if other users were having issues as well.

From the status page:

02:49 AM EST

12/02/22

We are investigating an issue that is affecting our Hosted Exchange environments. More details will be posted as they become available.

There was much speculation during the day on what the incident was, most people figured encryption and ransomware, which were correct.  Later, Rackspace announced that they recommended users should move their email environments over to Office 365 in which they would be providing Exchange Plan 1 licenses until further notice and provided a knowledge base article on how to accomplish this.

https://docs.rackspace.com/support/how-to/how-to-set-up-O365-via-your-cloud-office-control-panel

Now, this would mean that any new email would be delivered to Office 365, but they provided no information pertaining to historical data.

For days, users continued to take to social media platforms to discuss issues with Rackspace support and migration paths away from Rackspace hosted exchange.  It wasn’t until 12/7 where Rackspace finally discussed that the issue was due to a ransomware attack but was incredibly vague in any other details.  On 1/5/23, Rackspace provided more details from the forensic investigation on how the attack occurred, discussed how no other Rackspace services were affected by this, but most importantly mentioned that they do not have any plans on bringing back their hosted exchange environment.

Backup Your Cloud Data

One big takeaway from this incident is to backup your cloud data.  Most of the cloud providers do not backup your data, they just provide the software and its availability.  It is the customers responsibility to backup and validate that data in the event it needs to be restored.