SPEC releases the first standard benchmark for hardware and OS virtualization – UPDATED

Posted by virtualization.info Staff   |   Thursday, July 15th, 2010   |  

Finally, the Standard Performance Evaluation Corporation (SPEC) released the first industry standard benchmark for hardware virtualization and OS virtualization: SPECvirt_sc2010
The standard body is working on this since November 2006.

The benchmark designed to simulate the activity of three typical workloads: a web server, a Java application server and a mail server.
To do so it leverages existing SPEC benchmarks, modified to measure performance in a virtualization environment: SPECweb2005, SPECjAppServer2004 and SPECmail2008.

SPECvirt_sc2010 adopts the same approach used by the VMware benchmark VMmark, the tiles, to measure scalability: the framework deploys additional tiles until overall throughput reaches a peak and all virtual machines continue to meet required quality of service (QoS) criteria.

The web server workload is made by three tiers: a file server, an infrastructure server and of course the web server. They are hosted in two virtual machines (file server and web server are together).
The application server workload is made by two tiers: a backed database server and of course the application server. They are hosted in two VMs.
The mail server workload is made by just a single tier: the IMAP mail server, which is hosted inside a single VM.
A sixth, additional workload called SPECpoll has been created and put into an idle virtual machines. It sends and acknowledges network pings against the idle server in 100% load phase to measure its responsiveness and to all VMs in the 0% load phase (active idle) during power-enabled runs.

SPECvirt_sc2010_Tile

The benchmark is also able to measure power consumption and power/performance relationship.

It’s not available for free: it costs $3,000 with discounts for qualified non-profit and educational institutions.

Customers may believe that the availability of this standard finally closes the endless discussions about the value of VMmark, originally introduced more than three years ago, the VMware’s restriction in using it to compare vSphere against other products, and the quality of independent alternatives like the Project Virtual Reality Check (VRC) or the Anandtech vApus Mark.

Unfortunately, it may not be the case yet. The SPEC virtualization subcommittee that developed it in fact includes the following members and contributors: AMD, Dell, Fujitsu Siemens Computers, HP, IBM, Intel, Microsoft, Parallels, Red Hat, Sun Microsystems, Trigence, Unisys, and VMware.
For some reasons anyway, the press announcement issued by the SPEC only includes a subset of the names: AMD, Dell, Fujitsu, HP, IBM, Intel, Oracle, Red Hat, Unisys and VMware.

Microsoft is not there. And Citrix is not even a member of the subcommittee. It’s unclear if the two support this benchmark or not.
If they don’t, then that customers will get only some benefits: now will be able to use SPECvirt_sc2010 to compare multiple hypervisors, but until all players will validate the benchmark, the results will always be questionable.

Update: IBM is the first and only OEM that already published some SPECvirt_sc2010 scores, and they are quite interesting because the benchmark has been used to measure the performance of KVM.

The System x x3650 M3 achieved an overall performance score of 1169 @ 72 VMs served by Red Hat Enterprise Linux (RHEL) 5.5 and KVM.

The x3650 M3 was configured with the Intel Xeon Processor X5680 (3.33GHz with 256KB L2 cache per
core and 12MB L3 cache per processor—2 chips/12 cores/6 cores per chip), 144GB of memory and ninety-six 73GB disk drives.



blog comments powered by Disqus


virtualization.info Newest articles
Convirture opens a UK based office to explore EMEA market

September 15th, 2014

Convirture the company, founded in 2006 that provides an open source management console to manage virtualization platforms,  announced to have opened a UK- based office which will help the company…

Release: VMware ThinApp 5.1

September 12th, 2014

VMware has released version 5.1 of its application virtualization product ThinApp. Version 5.1 is the follow up of version 5.0 which was released in November last year. The update provides…

Brian Stevens former Red Hat CTO joins Google as VP of Cloud Platforms

September 12th, 2014

Two weeks ago we were wondering about Brian Stevens, who resigned as Red Hat’s CTO, asking ourselves if this could be seen as a “tremendous opportunity” for both.
Red Hat…

HP Acquires Eucalyptus

September 12th, 2014

Yesterday HP announced to have reached a definitive agreement to acquire Eucalyptus, provider of an open source Cloud Solution competing with OpenStack, CloudStack and OpenNebula.
This acquisition will put…

Release: VMware ESXi and vCenter Server 5.5 Update 2

September 12th, 2014

This week VMware released version 5.5 Update 2 of two of its flagship products, ESXi and vCenter Server, updates that were announced at VMworld 2014 in San Francisco.
Among the…

Release: Red Hat Satellite 6

September 11th, 2014

Satellite is Red Hat’s server life-cycle management based on the community project Spacewalk, it basically provides provision, patch, configuration and subscription management across Red Hat’s ecosystem.
Yesterday the Raleigh company…

Release: Oracle VM VirtualBox 4.3.16

September 11th, 2014

Oracle has released a new version of its virtualization platform VM Virtualbox. Version 4.3.16 is considered a maintenance release which can be installed on top of version 4.3. The update…

Microsoft releases limited preview of Migration Accelerator for Azure

September 8th, 2014

Microsoft last week announced a limited preview of Migration Accelerator (MA) for Azure. MA can seamlessly migrate physical, VMware, Amazon Web Services (AWS) and Hyper-V workloads into Azure, automating discovery…

Amazon CFO will retire in summer 2015

September 5th, 2014

On September 3, Amazon announced that Chief Financial Officer Thomas Szkutak plans to retire in June 2015 after about 13 years in the business, beginning in October 2002, where he…

Release: VMware Fusion 7

September 3rd, 2014

Following the technology preview announced in May VMware has released version 7 of its desktop virtualization platform for OS X: Fusion.
This new release, that comes in the normal and…

VMworld 2014 Wrap-Up: The open source side

September 2nd, 2014

As we mentioned last week, VMware used the stage of VMworld 2014 to “remind” its audience how deeply the company is involved in the open source community.
The company took…

Nutanix raises $140 million in Series E funding

September 1st, 2014

Nutanix, provider of web-scale IT infrastructure to medium and large enterprises with its software-driven Virtual Computing Platform, announced to have raised $140 million in a Series E funding over…

VMworld 2014 Wrap-Up: End User Computing Announcements

September 1st, 2014

The first day at VMworld 2014 was all about the new vCloud Air catalog and VMware’s “we are all open source folks” strategy but progressing in the event has also…

VMworld 2014 Wrap-Up: VMware vCloud Air Announcements

September 1st, 2014

Just before the beginning of VMworld 2014 VMware announced the rebranding of its owned and operated public Infrastructure as a Service (IaaS) service vCloud Hybrid Service (vCHS) into the more…

 
Monthly Archive