SPEC releases the first standard benchmark for hardware and OS virtualization – UPDATED

Posted by virtualization.info Staff   |   Thursday, July 15th, 2010   |  

Finally, the Standard Performance Evaluation Corporation (SPEC) released the first industry standard benchmark for hardware virtualization and OS virtualization: SPECvirt_sc2010
The standard body is working on this since November 2006.

The benchmark designed to simulate the activity of three typical workloads: a web server, a Java application server and a mail server.
To do so it leverages existing SPEC benchmarks, modified to measure performance in a virtualization environment: SPECweb2005, SPECjAppServer2004 and SPECmail2008.

SPECvirt_sc2010 adopts the same approach used by the VMware benchmark VMmark, the tiles, to measure scalability: the framework deploys additional tiles until overall throughput reaches a peak and all virtual machines continue to meet required quality of service (QoS) criteria.

The web server workload is made by three tiers: a file server, an infrastructure server and of course the web server. They are hosted in two virtual machines (file server and web server are together).
The application server workload is made by two tiers: a backed database server and of course the application server. They are hosted in two VMs.
The mail server workload is made by just a single tier: the IMAP mail server, which is hosted inside a single VM.
A sixth, additional workload called SPECpoll has been created and put into an idle virtual machines. It sends and acknowledges network pings against the idle server in 100% load phase to measure its responsiveness and to all VMs in the 0% load phase (active idle) during power-enabled runs.

SPECvirt_sc2010_Tile

The benchmark is also able to measure power consumption and power/performance relationship.

It’s not available for free: it costs $3,000 with discounts for qualified non-profit and educational institutions.

Customers may believe that the availability of this standard finally closes the endless discussions about the value of VMmark, originally introduced more than three years ago, the VMware’s restriction in using it to compare vSphere against other products, and the quality of independent alternatives like the Project Virtual Reality Check (VRC) or the Anandtech vApus Mark.

Unfortunately, it may not be the case yet. The SPEC virtualization subcommittee that developed it in fact includes the following members and contributors: AMD, Dell, Fujitsu Siemens Computers, HP, IBM, Intel, Microsoft, Parallels, Red Hat, Sun Microsystems, Trigence, Unisys, and VMware.
For some reasons anyway, the press announcement issued by the SPEC only includes a subset of the names: AMD, Dell, Fujitsu, HP, IBM, Intel, Oracle, Red Hat, Unisys and VMware.

Microsoft is not there. And Citrix is not even a member of the subcommittee. It’s unclear if the two support this benchmark or not.
If they don’t, then that customers will get only some benefits: now will be able to use SPECvirt_sc2010 to compare multiple hypervisors, but until all players will validate the benchmark, the results will always be questionable.

Update: IBM is the first and only OEM that already published some SPECvirt_sc2010 scores, and they are quite interesting because the benchmark has been used to measure the performance of KVM.

The System x x3650 M3 achieved an overall performance score of 1169 @ 72 VMs served by Red Hat Enterprise Linux (RHEL) 5.5 and KVM.

The x3650 M3 was configured with the Intel Xeon Processor X5680 (3.33GHz with 256KB L2 cache per
core and 12MB L3 cache per processor—2 chips/12 cores/6 cores per chip), 144GB of memory and ninety-six 73GB disk drives.



blog comments powered by Disqus


virtualization.info Newest articles
EMC acquires Virtustream

May 26th, 2015

EMC has acquired Virtustream, a Infrastructure as a Service (IaaS) provider originally funded among the others from SAP for 1.2 billion US dollars. Virtustream also provides a Cloud Management Platform…

Release: Midokura Enterprise MidoNet (MEM)

May 25th, 2015

Japanese startup Midokura focused on network virtualization announced the availability of Midokura Enterprise MidoNet (MEM). MidoNet is a scalable network virtualization solution integrated with OpenStack Kilo networking project and…

Release: Red Hat CloudForms 3.2

May 20th, 2015

Red Hat yesterday announced the release of version 3.2. of its cloud management software, CloudForms. CloudForms allows the management of hypervisors from Red Hat, VMware and Microsoft as well as…

Gartner predicts that Worldwide Cloud IaaS spending will grow around 33 percent is 2015

May 20th, 2015

Besides releasing their annual Magic Quadrant for Cloud Infrastructure as a Service, research and consulting firm Gartner also made some predicitons about the growth of Cloud Infrastructure as a Service…

Gartner releases its Magic Quadrant for Cloud Infrastructure as a Service for 2015

May 20th, 2015

Gartner this week updated its Magic Quadrant for Cloud Infrastructure as a Service (IaaS) for the year 2015. The Magic Qudrant for 2014 was released in May last year (covered…

Red Hat announces the Cloud Suite for Applications

May 19th, 2015

Red Hat yesterday announced a new Suite which bundles its cloud portfolio to manage both Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) workloads. The suite, which…

VMware announces new SaaS offering for building, delivering and managing desktops and applications

May 18th, 2015

A day before the start of Citrix annual Synergy conference, rival VMware announced project Enzo. Project Enzo is the name of a new Software as a Service offering from VMware…

Announcements from Citrix annual Synergy Conference

May 18th, 2015

Last week, Citrix held its annual Synergy conference in Orlando. During the conference Citrix made several announcements which will be summarized in this blogpost.
Citrix Workspace Cloud, which is a…

Is Cisco planning to acquire Nutanix? – UPDATED

May 18th, 2015

Last Friday, Storage Newsletter published an article written by Jared Rinderer a senior research analyst at the Equity Capital Research Group stating that Cisco is planning to acquire Nutanix. While…

Release: Citrix XenServer 6.5 Service Pack 1

May 18th, 2015

During its annual Synergy conference last week, Citrix announced the availability of Service Pack 1 for version 6.5 of its virtualization platform XenServer. Version 6.5 of XenServer was released in…

Infographic: OpenStack adoption in Italy

May 11th, 2015

The IT roadmap throughout the digital transformation process is a journey, long and complex, and must not be underestimated. Like any other journey it starts with a first step which…

.infonews welcomes Alessandro Fontana as a team member

May 11th, 2015

12 years ago, in 2003, virtualization.info started to track virtualization market and, in 2010, cloudcomputing.info was launched as a sister publication with a focus on the cloud computing market.
In…

Microsoft announcements from its annual Ignite conference

May 10th, 2015

Last week, Microsoft held its annual Ignite conference in Chicago, Illinois. Ignite is the new name for a series of technical conferences targeted to IT Professionals, which replaces the…

Release: Update Rollup 6 for Microsoft System Center 2012 R2 and Windows Azure Pack – UPDATED

April 30th, 2015

Microsoft has released Update Rollup 6 for the following products within the System Center 2012 R2 suite: Service Manager (SCSM), Operations Manager (SCOM), Virtual Machine Manager (SCVMM) and Service…

 
Monthly Archive