SPEC releases the first standard benchmark for hardware and OS virtualization – UPDATED

Posted by virtualization.info Staff   |   Thursday, July 15th, 2010   |  

Finally, the Standard Performance Evaluation Corporation (SPEC) released the first industry standard benchmark for hardware virtualization and OS virtualization: SPECvirt_sc2010
The standard body is working on this since November 2006.

The benchmark designed to simulate the activity of three typical workloads: a web server, a Java application server and a mail server.
To do so it leverages existing SPEC benchmarks, modified to measure performance in a virtualization environment: SPECweb2005, SPECjAppServer2004 and SPECmail2008.

SPECvirt_sc2010 adopts the same approach used by the VMware benchmark VMmark, the tiles, to measure scalability: the framework deploys additional tiles until overall throughput reaches a peak and all virtual machines continue to meet required quality of service (QoS) criteria.

The web server workload is made by three tiers: a file server, an infrastructure server and of course the web server. They are hosted in two virtual machines (file server and web server are together).
The application server workload is made by two tiers: a backed database server and of course the application server. They are hosted in two VMs.
The mail server workload is made by just a single tier: the IMAP mail server, which is hosted inside a single VM.
A sixth, additional workload called SPECpoll has been created and put into an idle virtual machines. It sends and acknowledges network pings against the idle server in 100% load phase to measure its responsiveness and to all VMs in the 0% load phase (active idle) during power-enabled runs.

SPECvirt_sc2010_Tile

The benchmark is also able to measure power consumption and power/performance relationship.

It’s not available for free: it costs $3,000 with discounts for qualified non-profit and educational institutions.

Customers may believe that the availability of this standard finally closes the endless discussions about the value of VMmark, originally introduced more than three years ago, the VMware’s restriction in using it to compare vSphere against other products, and the quality of independent alternatives like the Project Virtual Reality Check (VRC) or the Anandtech vApus Mark.

Unfortunately, it may not be the case yet. The SPEC virtualization subcommittee that developed it in fact includes the following members and contributors: AMD, Dell, Fujitsu Siemens Computers, HP, IBM, Intel, Microsoft, Parallels, Red Hat, Sun Microsystems, Trigence, Unisys, and VMware.
For some reasons anyway, the press announcement issued by the SPEC only includes a subset of the names: AMD, Dell, Fujitsu, HP, IBM, Intel, Oracle, Red Hat, Unisys and VMware.

Microsoft is not there. And Citrix is not even a member of the subcommittee. It’s unclear if the two support this benchmark or not.
If they don’t, then that customers will get only some benefits: now will be able to use SPECvirt_sc2010 to compare multiple hypervisors, but until all players will validate the benchmark, the results will always be questionable.

Update: IBM is the first and only OEM that already published some SPECvirt_sc2010 scores, and they are quite interesting because the benchmark has been used to measure the performance of KVM.

The System x x3650 M3 achieved an overall performance score of 1169 @ 72 VMs served by Red Hat Enterprise Linux (RHEL) 5.5 and KVM.

The x3650 M3 was configured with the Intel Xeon Processor X5680 (3.33GHz with 256KB L2 cache per
core and 12MB L3 cache per processor—2 chips/12 cores/6 cores per chip), 144GB of memory and ninety-six 73GB disk drives.



blog comments powered by Disqus


virtualization.info Newest articles
Paper: VMware vCenter Server 6.0 Deployment Guide

March 3rd, 2015

VMware has released a paper titled: "VMware vCenter Server 6.0 Deployment Guide". The paper which contains 100 pages covers two new simplified deployment models introduced with version 6. Embedded which…

VMware hires new CIO: Bask Iyer, former Juniper Networks

March 2nd, 2015

On February 24 VMware disclosed that the former chief information officer of Juniper Networks, Bask Iyer, was elected as the new CIO at VMware. Iyer is taking Tony Scott position…

Parallels Acquires 2X Software

February 27th, 2015

On February 25, Parallels announced that it has acquired 2X Software. 2X Software is a Maltese company, with offices in the United States, Germany, UK, Australia and of course Malta,…

Release: Proxmox Virtual Environment 3.4

February 27th, 2015

Proxmox Server Solutions GmbH headquartered in Vienna, Austria announced to have released the version 3.4 of Proxmox Virtual Environment, an open source virtualization management solution that leverages KVM technologies. Proxmox joined…

Release: Red Hat Enterprise Linux OpenStack Platform 6

February 17th, 2015

Red Hat today announced the release of version 6 of its OpenStack platform management tool Red Hat Enterprise Linux OpenStack. Version 6 is the follow up of version 5 which…

Release: Red Hat Enterprise Virtualization 3.5

February 12th, 2015

Yesterday February 11, Red Hat has announced the global availability of version 3.5 of its virtualization platform: Red Hat Enterprise Virtualization (aka RHEV).
RHEV is based on the Kernel-based Virtual Machine (KVM) hypervisor and…

Release: Oracle VM 3.3.2

February 12th, 2015

Oracle has released version 3.3.2 of its enterprise virtualizatoin solution. The virtualization solution is available for both x86 and SPARC based processor architectures and uses the Xen hypervisor technology, supporting…

Release: Docker Engine 1.5

February 12th, 2015

Docker has released version 1.5 of its container virtualization solution. Docker provides so called container virtualization, allowing an application and its dependencies to run as an isolated process inside…

Release: Microsoft Assessment and Planning Toolkit 9.2

February 12th, 2015

Microsoft has released version 9.1 of its capactiy planning tool the Assessment and Planning Toolkit (MAP). This version is the follow-up of version 9.1 which was released in July…

NIMBOXX acquires VERDE assets from Virtual Bridges

February 10th, 2015

NIMBOXX a company which creates converged hardware appliances combining storage, server, networking and virtualization technology announced that it has acquired the Virtual Desktop Infrastructure (VDI) technology product VERDE from…

Citrix announces to cut 900 Jobs

February 6th, 2015

Last week Citrix announced its financial reports for Q4 and Fiscal year 2014, despite the positive results that the company had for full year 2014, the general opinion is that…

Paper: Reference Architecture – Lenovo Client Virtualization

February 5th, 2015

Lenovo has published a paper titled:"Reference Architecture: Lenovo Client Virtualization". The paper which contains 44 pages contains a reference architecture for Server Based Computing (SBC) and Virtual Desktop Infrastructure…

Book: Microsoft Azure Essentials: Fundamentals of Azure

February 4th, 2015

Microsoft has published a free ebook titled: “Microsoft Azure Essentials: Fundamentals of Azure“. The book, which contains 246 pages covers the fundamentals of Microsoft Azure’s cloud offering. It concentrates…

VMware acquires Immidio

February 3rd, 2015

VMware today announced that it has acquired Immidio. Immidio provides several User Environment Management related product: Immidio AppScriber and Immidio Flex+. With the acquisition of Immidio VMware wants to fill…

 
Monthly Archive