SPEC releases the first standard benchmark for hardware and OS virtualization – UPDATED

Posted by virtualization.info Staff   |   Thursday, July 15th, 2010   |  

Finally, the Standard Performance Evaluation Corporation (SPEC) released the first industry standard benchmark for hardware virtualization and OS virtualization: SPECvirt_sc2010
The standard body is working on this since November 2006.

The benchmark designed to simulate the activity of three typical workloads: a web server, a Java application server and a mail server.
To do so it leverages existing SPEC benchmarks, modified to measure performance in a virtualization environment: SPECweb2005, SPECjAppServer2004 and SPECmail2008.

SPECvirt_sc2010 adopts the same approach used by the VMware benchmark VMmark, the tiles, to measure scalability: the framework deploys additional tiles until overall throughput reaches a peak and all virtual machines continue to meet required quality of service (QoS) criteria.

The web server workload is made by three tiers: a file server, an infrastructure server and of course the web server. They are hosted in two virtual machines (file server and web server are together).
The application server workload is made by two tiers: a backed database server and of course the application server. They are hosted in two VMs.
The mail server workload is made by just a single tier: the IMAP mail server, which is hosted inside a single VM.
A sixth, additional workload called SPECpoll has been created and put into an idle virtual machines. It sends and acknowledges network pings against the idle server in 100% load phase to measure its responsiveness and to all VMs in the 0% load phase (active idle) during power-enabled runs.

SPECvirt_sc2010_Tile

The benchmark is also able to measure power consumption and power/performance relationship.

It’s not available for free: it costs $3,000 with discounts for qualified non-profit and educational institutions.

Customers may believe that the availability of this standard finally closes the endless discussions about the value of VMmark, originally introduced more than three years ago, the VMware’s restriction in using it to compare vSphere against other products, and the quality of independent alternatives like the Project Virtual Reality Check (VRC) or the Anandtech vApus Mark.

Unfortunately, it may not be the case yet. The SPEC virtualization subcommittee that developed it in fact includes the following members and contributors: AMD, Dell, Fujitsu Siemens Computers, HP, IBM, Intel, Microsoft, Parallels, Red Hat, Sun Microsystems, Trigence, Unisys, and VMware.
For some reasons anyway, the press announcement issued by the SPEC only includes a subset of the names: AMD, Dell, Fujitsu, HP, IBM, Intel, Oracle, Red Hat, Unisys and VMware.

Microsoft is not there. And Citrix is not even a member of the subcommittee. It’s unclear if the two support this benchmark or not.
If they don’t, then that customers will get only some benefits: now will be able to use SPECvirt_sc2010 to compare multiple hypervisors, but until all players will validate the benchmark, the results will always be questionable.

Update: IBM is the first and only OEM that already published some SPECvirt_sc2010 scores, and they are quite interesting because the benchmark has been used to measure the performance of KVM.

The System x x3650 M3 achieved an overall performance score of 1169 @ 72 VMs served by Red Hat Enterprise Linux (RHEL) 5.5 and KVM.

The x3650 M3 was configured with the Intel Xeon Processor X5680 (3.33GHz with 256KB L2 cache per
core and 12MB L3 cache per processor—2 chips/12 cores/6 cores per chip), 144GB of memory and ninety-six 73GB disk drives.



blog comments powered by Disqus


virtualization.info Newest articles
Amazon updates AWS plugin for Microsoft System Center Virtual Machine Manager

January 22nd, 2015

In October last year, Amazon released the AWS System Manager for Microsoft System Center Virtual Machine Manager (SCVMM). AWS System Manager is an add-in for SCVMM which allows customers to…

VMware announces vCloud Air Disaster Recovery

January 22nd, 2015

VMware has announced that later this quarter it will release an updated disaster recovery solution for its IaaS solution vCloud Air (formerly vCloud Hybrid Service). The solution which will be…

Paper: Cisco Unified Computing System with VMware Horizon 6 with View and Virtual SAN

January 20th, 2015

Cisco and VMware have released a paper titled: "Cisco Unified Computing System with VMware Horizon 6 with View and Virtual SAN". The paper which contains 59 pages describes a reference…

Release: Xen Project Hypervisor 4.5

January 19th, 2015

The Xen Project, the community which develops the Xen hypervisor under the GNU General Public License (GPLv2) last week announced the availability version 4.5 of the Xen Hypervisor. With major…

Microsoft and Persistent Systems release solution for Azure IaaS data center migration

January 15th, 2015

Microsoft, together with partner Persistent Systems has developed and released a tool which helps customers to move complete Infrastructure as a Service (Iaas) deployments from one Microsoft datacenter to…

Citrix announcements from its annual Summit partner conference

January 15th, 2015

Before and during its annual Summit partner conference, held on January 13 and 14, Citrix made several announcements. First of all Citrix announced the acquisition of Sanbolic, which virtualization.info already…

Citrix announces XenServer 6.5 – UPDATED

January 13th, 2015

Citrix today announced the next version of its virtualization platform XenServer which is a commercial product based on the open-source Xen Project hypervisor. XenServer 6.5 will be the follow up…

Citrix Acquires Sanbolic

January 12th, 2015

Citrix today announced that its has completed its acquisition of Sanbolic. Sanbolic which has a product called Sanbolic Scale-Out Platform which can be seen as an aggregation of both local…

Survey: State of the VDI and SBC union 2015

January 12th, 2015

Two years ago, Project Virtual Reality Check (VRC) started a survey asking participants to provide more insight in the non-technical issues related to Virtual Desktop Infrastructure (VDI) and Server Based…

Paper: VMware Mirage 5.0 Reviewer’s Guide

January 9th, 2015

VMware has released a reviewers guide for its End User Computing solution Horizon version 5. The paper titled: "VMware Mirage 5.0 Reviewer’s Guide" contains 209 pages and allows readers to…

Microsoft updates Azure with availability of G-series, Docker image support and a public preview of Azure Key Vault

January 9th, 2015

Microsoft yesterday announced several updates to its Microsoft Azure cloud platform. Updates include the availability of the new G-series type of VM’s, the availability of Ubuntu image with a…

Red Hat releases Red Hat Enterprise Virtualization for Power

January 7th, 2015

As we know IBM and Red Hat have a story of collaboration behind its back and they have been working lately together on various projects, the latest announcement they made…

Paper: Large scale Virtual Desktop Infrastructure deployment using Windows Server 2012 R2 with Storage Tiers and Data Deduplication

January 7th, 2015

Microsoft has released a paper titled:"Large scale Virtual Desktop Infrastructure deployment using Windows Server 2012 R2 with Storage Tiers and Data Deduplication". The paper which contains 49 pages explains…

Book: Microsoft System Center Introduction to Microsoft Automation Solutions

January 5th, 2015

Microsoft has released a free book titled: “Microsoft System Center Introduction to Microsoft Automation Solutions“. The book which contains 105 pages gives an introduction to the automation solutions provided…

 
Monthly Archive