SPEC releases the first standard benchmark for hardware and OS virtualization – UPDATED

Posted by virtualization.info Staff   |   Thursday, July 15th, 2010   |  

Finally, the Standard Performance Evaluation Corporation (SPEC) released the first industry standard benchmark for hardware virtualization and OS virtualization: SPECvirt_sc2010
The standard body is working on this since November 2006.

The benchmark designed to simulate the activity of three typical workloads: a web server, a Java application server and a mail server.
To do so it leverages existing SPEC benchmarks, modified to measure performance in a virtualization environment: SPECweb2005, SPECjAppServer2004 and SPECmail2008.

SPECvirt_sc2010 adopts the same approach used by the VMware benchmark VMmark, the tiles, to measure scalability: the framework deploys additional tiles until overall throughput reaches a peak and all virtual machines continue to meet required quality of service (QoS) criteria.

The web server workload is made by three tiers: a file server, an infrastructure server and of course the web server. They are hosted in two virtual machines (file server and web server are together).
The application server workload is made by two tiers: a backed database server and of course the application server. They are hosted in two VMs.
The mail server workload is made by just a single tier: the IMAP mail server, which is hosted inside a single VM.
A sixth, additional workload called SPECpoll has been created and put into an idle virtual machines. It sends and acknowledges network pings against the idle server in 100% load phase to measure its responsiveness and to all VMs in the 0% load phase (active idle) during power-enabled runs.

SPECvirt_sc2010_Tile

The benchmark is also able to measure power consumption and power/performance relationship.

It’s not available for free: it costs $3,000 with discounts for qualified non-profit and educational institutions.

Customers may believe that the availability of this standard finally closes the endless discussions about the value of VMmark, originally introduced more than three years ago, the VMware’s restriction in using it to compare vSphere against other products, and the quality of independent alternatives like the Project Virtual Reality Check (VRC) or the Anandtech vApus Mark.

Unfortunately, it may not be the case yet. The SPEC virtualization subcommittee that developed it in fact includes the following members and contributors: AMD, Dell, Fujitsu Siemens Computers, HP, IBM, Intel, Microsoft, Parallels, Red Hat, Sun Microsystems, Trigence, Unisys, and VMware.
For some reasons anyway, the press announcement issued by the SPEC only includes a subset of the names: AMD, Dell, Fujitsu, HP, IBM, Intel, Oracle, Red Hat, Unisys and VMware.

Microsoft is not there. And Citrix is not even a member of the subcommittee. It’s unclear if the two support this benchmark or not.
If they don’t, then that customers will get only some benefits: now will be able to use SPECvirt_sc2010 to compare multiple hypervisors, but until all players will validate the benchmark, the results will always be questionable.

Update: IBM is the first and only OEM that already published some SPECvirt_sc2010 scores, and they are quite interesting because the benchmark has been used to measure the performance of KVM.

The System x x3650 M3 achieved an overall performance score of 1169 @ 72 VMs served by Red Hat Enterprise Linux (RHEL) 5.5 and KVM.

The x3650 M3 was configured with the Intel Xeon Processor X5680 (3.33GHz with 256KB L2 cache per
core and 12MB L3 cache per processor—2 chips/12 cores/6 cores per chip), 144GB of memory and ninety-six 73GB disk drives.



blog comments powered by Disqus


virtualization.info Newest articles
Release: VMware Mirage 5.2

December 11th, 2014

VMware has released version 5.2 of its centralized image management solution for Windows desktops: Mirage. Version 5.2 is the follow-up of version 5.1 which was released in September this year….

Release: VMware vRealize Log Insight 2.5

December 10th, 2014

After announcing vRealize Log Insight 2.5 in October this year, VMware today released the version to its customers. Log Insight is a log aggregation, management and analisys tool, and version…

Release: VMware vRealize Operations Manager 6.0

December 10th, 2014

VMware has released version 6.0 of its monitoring tool vRealize Operations Manager. vRealize Operations Manager provides performance, capacity and configuration management capabilities across physical, virtual and cloud infrastructures. Its…

VMware makes App Volumes generally available

December 10th, 2014

In August this year, VMware acquired CloudVolumes, a company providing an application containerization solution, similar to what Docker is currently providing. Actually buying the CloudVolumes solution was quite confusing, since…

Release: Microsoft Desktop Optimization Pack 2014 R2

December 8th, 2014

Microsoft has released its Desktop Optimization Pack (MDOP) 2014 R2, which is available for Microsoft Customers with Software Assurance. The MDOP 2014 R2 is the follow up of the…

Release: 5nine Manager 6.0 for Hyper-V

December 4th, 2014

5nine has released version 6.0 of its Microsoft hypervisor management solution 5nine Manager. 5nine Manager is capable of managing multiple versions of Microsoft Hyper-V hosts, and also provdes an…

Release: Veeam Management Pack v7 R2 for System Center

December 3rd, 2014

Veaam today released a new version of its Management Pack for Microsoft’s Operations Manager monitoring solution capable of monitoring VMware vSphere, Microsoft Hyper-V and Veeam Backup & Replication environments.
The…

Release: Dell Wyse vWorkSpace 8.5

December 3rd, 2014

Dell has released version 8.5 of its VDI Connection Broker Software Wyse vWorkspace. vWorkSpace was part of the product portfolio of Quest when it was acquired by Dell in September…

Tech: Microsoft Cloud Platform Integration Framework

December 3rd, 2014

Microsoft has published a series of articles on the subject: Cloud Platform Integration Framework. The Cloud Platform Integration Framework (CPIF) provides an enterprise or cloud service provider architect patterns…

Release: VMware Workstation 11 and VMware Player 7 Pro

December 2nd, 2014

On October 1st VMware announced the upcoming features for Workstation version 11 and Player Pro version 7. Today VMware released both version making them generally available to VMware customers. Starting…

OpenStack 2015 Board of Directors Elections

November 28th, 2014

The OpenStack Foundation regularly conducts Elections for Individual Directors of the Foundation’s Board, next elections for the 2015 Board of Directors are going to be held from Monday January…

Paper: Dell Wyse Datacenter for VMware Horizon View Reference Architecture v.6.6

November 28th, 2014

Dell has released a paper titled: "Dell Wyse Datacenter for VMware Horizon View Reference Architecture v.6.6". The paper which contains 110 pages contains a Reference Architecture for the design, configuration…

Paper: Creating a VMware Software-Defined Data Center

November 27th, 2014

VMware has released a paper titled:"Creating a VMware Software-Defined Data Center". The paper which contains 29 pages describes a reference architecture for a Software Defined Data Center (SDDC) using VMware…

Release: 5nine Cloud Security 5.0 for Hyper-V

November 26th, 2014

5nine has released version 5 of its Cloud Security for Hyper-V product. Cloud Security for Hyper-V is a solution which helps to ensure security and compliance for Hyper-V environments. It…

 
Monthly Archive