At the end of August Amazon beta-launched the first rentable virtual datacenter based on the Xen hypervisor: the Elastic Cloud Computing (EC2).
It’s possibly the largest Xen implementation available to customers and is a notable opportunity for Xen development team (and several commercial companies surrounding it) to demonstrate reliability of the project.
Unfortunately after just 1 month since launch Amazon is hit by the a severe security warning on the system (not depending on Xen in itself): data leakage.
Samuel T. Cossette detailed on his blog how it’s easy access other EC2 customers data, after they released used Xen virtual machines:
…If you plan to give it a try (and you definitely should), don’t forget to use an encrypted partition or wipe your instances’ hard drives, since Amazon won’t do it for you. In fact, when you terminate an instance, Amazon simply shuts the machine down. Then, if the same physical machine is allocated to somebody else, a hamster goes to that machine, powers it up, formats the hard drives and reinstalls a brand new operating system. Herein lies the problem – the hard drives are only repartitioned and formated, not initialized. This means that all the data is still physically on the hard drives, even though it is not readily accessible!
I have looked at a couple of hard drives and found some sensible data in the form of “private” source code, OpenVPN (complete with key and certificate) configurations, S3 access keys, EC2 keys and certificates, logins and passwords to domain name administration interfaces, etc. It was easy to find out who the owners were – they ranged from individuals to profitable startups (according to their pictures on flickr).
Who’s to blame? Of course, Amazon could (and should) do something about the clean-up process, but organizations storing their intellectual property in plain text is also somewhat of a questionable practice…
Read the whole article at source.
At the moment of writing the author already reported Amazon solved the problem, filling used partitions with random data, anyway this incident helped pointing the attention to some usually uncovered aspects of provisioning.
The more we’ll go towards fully automated virtual infrastructures, the more we’ll need severe (and scalable) controls on the provisioning phase: authorization for requesting, deploying and using new VMs, and strict control on usage and re-allocation of physical resources (RAM and disk space).