What is Xen, and why is it cool?

Quoting from Nathan Torkington at O’Reilly Developers Weblogs:

I got into Xen a few weeks ago, and I’m loving that they’re getting lots of attention. But most folks don’t know what they are and why their product is so cool. Allow me to shed light on the matter …
Xen is like the Mach microkernel, where you can have multiple operating systems running at once and a thin kernel handles switching between them and managing device access. This thin layer in Xen is called the hypervisor, and is analogous to the Mach microkernel. It provides an idealized hardware layer that you port your OS to, and in return you get the ability to run multiple operating system instances at once (e.g., run two copies of Redhat’s latest, one copy of the Novell Desktop, and an OpenBSD), freeze and restore snapshots of a running OS, and more.

What you can’t do with Xen is run Windows on it–that’s always going to be VMware’s niche (at least until Intel’s VM technology becomes ubiquitous). But Xen makes a whole lot of situations possible that are slow or impossible at the moment. Two applications that are working well for Xen: testing and server load balancing. If you’re working on your app and want to test it on a staging server, it’s no fun to reboot, or negotiate time on a shared staging server, and it’s way less fun to rebuild if your app hoses the staging server. The Xen way, you run your development OS and your staging OS on your machine at the same time and switch from one to the other when you need to. If the staging server gets borked, you delete that running OS and reload from a saved stable snapshot.

In the server room, it’s often easier and more secure to manage a single service on a running machine. The more ways into a box, the less defensible it is and the more risk for damage and service downtime if the box is compromised. So run Xen and use one OS instance per service. If a service is compromised, only that service is compromised. If you experience high load, say due to Slashdotting, you can easily reconfigure machines to run different services. (You can rdist the snapshot of an OS running that service and then bring it up on however many machines you need).

The potential for Xen is great. We’re going to feature them at OSCON because their technology is just so cool. Lots of companies like RedHat and HP are very interested in what Xen makes possible, because the hypervisor enables things that seemed like wishful fantasy a few years ago. I loved my time meeting with one of the company’s founders and playing with Xen–they’re very smart engineers with their heads screwed on right. There’s obviously a lot of work to be done in making Xen friendlier to install, getting more tools around the administration of Xen, etc., so the interest and involvement of companies with big budgets is a good thing. They’ll help move Xen from the research lab where it was born to data centers and developer desktops where it can be ubiquitous and useful.

So look for lots of action from Xen. I expect the next versions of Novell, HP, etc.’s offerings will feature Xen support (either standard or as an alternate kernel shipped with the distro). I hope there’ll be a great distro like Ubuntu or Gentoo offering a Xen install as well as a solo install. This will give everyone a painless way to do some very cool things and open the door for even cooler things down the line.