Now that the SPEC has finally released the first industry standard benchmark for hardware and OS virtualization platforms, customers may believe that there’s no more need for the VMmark proprietary framework that VMware released in July 2007.
VMware has a different opinion and last week announced the public beta of VMmark 2.0.
While SPECvirt_sc2010 and VMmark 1.x measure the performance of a single virtualization host, VMmark 2.0 has been designed to benchmark a whole virtual data center.
This implies measuring complex operations like manual and automated (or DRS-initated) vMotion, Storage vMotion, as well as virtual machines cloning and deployment.
On top of that, VMmark 2.0 also features more resource-intensive workloads, including:
- a multi-tier OLTP workload (DVD Store 2) consisting of a 4-vCPU database VM and three 2-vCPU webserver VMs driving a bursty load profile
- a multi-tier social networking workload (OLIO) consisting of a 4-vCPU web server and a 2-vCPU database server
- a mail server workload (Microsoft Exchange2007) consisting of a 4-vCPU mailserver
- a standby server with just one vCPU
For the large majority of customers, benchmarking entire virtual infrastructures may sound completely useless at today. There are even doubts that the market really needs to benchmark a single virtualization host.
But going forward, the design of VMmark 2.0 seems a good start to benchmark fully automated Infrastructure-as-a-Service (IaaS) cloud computing platform, which already seems a concrete need looking at the efforts of companies like CloudHarmony, Compuware and even Web Hosting Talk.
Of course, until VMware drops its restrictions on VMmark’s EULA, nobody will be really able to use it.