SPEC releases the first standard benchmark for hardware and OS virtualization – UPDATED

Posted by virtualization.info Staff   |   Thursday, July 15th, 2010   |  

Finally, the Standard Performance Evaluation Corporation (SPEC) released the first industry standard benchmark for hardware virtualization and OS virtualization: SPECvirt_sc2010
The standard body is working on this since November 2006.

The benchmark designed to simulate the activity of three typical workloads: a web server, a Java application server and a mail server.
To do so it leverages existing SPEC benchmarks, modified to measure performance in a virtualization environment: SPECweb2005, SPECjAppServer2004 and SPECmail2008.

SPECvirt_sc2010 adopts the same approach used by the VMware benchmark VMmark, the tiles, to measure scalability: the framework deploys additional tiles until overall throughput reaches a peak and all virtual machines continue to meet required quality of service (QoS) criteria.

The web server workload is made by three tiers: a file server, an infrastructure server and of course the web server. They are hosted in two virtual machines (file server and web server are together).
The application server workload is made by two tiers: a backed database server and of course the application server. They are hosted in two VMs.
The mail server workload is made by just a single tier: the IMAP mail server, which is hosted inside a single VM.
A sixth, additional workload called SPECpoll has been created and put into an idle virtual machines. It sends and acknowledges network pings against the idle server in 100% load phase to measure its responsiveness and to all VMs in the 0% load phase (active idle) during power-enabled runs.


The benchmark is also able to measure power consumption and power/performance relationship.

It’s not available for free: it costs $3,000 with discounts for qualified non-profit and educational institutions.

Customers may believe that the availability of this standard finally closes the endless discussions about the value of VMmark, originally introduced more than three years ago, the VMware’s restriction in using it to compare vSphere against other products, and the quality of independent alternatives like the Project Virtual Reality Check (VRC) or the Anandtech vApus Mark.

Unfortunately, it may not be the case yet. The SPEC virtualization subcommittee that developed it in fact includes the following members and contributors: AMD, Dell, Fujitsu Siemens Computers, HP, IBM, Intel, Microsoft, Parallels, Red Hat, Sun Microsystems, Trigence, Unisys, and VMware.
For some reasons anyway, the press announcement issued by the SPEC only includes a subset of the names: AMD, Dell, Fujitsu, HP, IBM, Intel, Oracle, Red Hat, Unisys and VMware.

Microsoft is not there. And Citrix is not even a member of the subcommittee. It’s unclear if the two support this benchmark or not.
If they don’t, then that customers will get only some benefits: now will be able to use SPECvirt_sc2010 to compare multiple hypervisors, but until all players will validate the benchmark, the results will always be questionable.

Update: IBM is the first and only OEM that already published some SPECvirt_sc2010 scores, and they are quite interesting because the benchmark has been used to measure the performance of KVM.

The System x x3650 M3 achieved an overall performance score of 1169 @ 72 VMs served by Red Hat Enterprise Linux (RHEL) 5.5 and KVM.

The x3650 M3 was configured with the Intel Xeon Processor X5680 (3.33GHz with 256KB L2 cache per
core and 12MB L3 cache per processor—2 chips/12 cores/6 cores per chip), 144GB of memory and ninety-six 73GB disk drives.

blog comments powered by Disqus

virtualization.info Newest articles
Microsoft loves Red Hat: The business implications

November 18th, 2015

As you may have heard, Microsoft recently announced its “historical” partnership with Red Hat, something that a number of analysts already claimed as a milestone for both companies but especially for…

Red Hat acquires Ansible

October 16th, 2015

Today Red Hat announced that it has signed a definitive agreement to acquire Ansible, provider of an IT automation tool popular especially within the developers’ community.
The acquisition is rumored…

Dell acquires EMC, owner of VMware and Pivotal

October 12th, 2015

Dell today announced that it signed a definitive agreement to acquire EMC Corporation for a total of $67 billion, making it the largest ever acquisition in the technology industry so…

Red Hat announces earnings for Q2 2015

October 2nd, 2015

Red Hat announced its financial results for the second quarter of fiscal year 2016.
Total revenue for the second quarter ended Augus 31, 2015 was $504 million, with an increase…

Citrix retires XenClient

September 21st, 2015

Citrix on its website announced that it started the End-of-Life process of Citrix XenClient Enterprise, the end of life date will be December 12, 2016.
XenClient consists of two technologies,…

Release: Altaro VM Backup V6

September 9th, 2015

Altaro Software, company specialized in backup for virtualized environments, specifically Microsoft Hyper-V, announced the release of Altaro VM Backup, previously known as Altaro Hyper-V Backup, rebranded to introduce VMware…

Release: CloudPhysics for vSphere

September 9th, 2015

CloudPhysics, provider of a SaaS performance and capacity analytics solution, announced to have released a new version of its SaaS offering allowing now VMware users to preempt and eliminate…

VMware announcements from its annual VMworld US conference

September 2nd, 2015

During its annual VMworld conference held between August 30 and September 3 in San Fransicso VMware made several announcements. During the keynote on Monday several data center related announcements and…

Release: VMware Workstation 12 Pro, Player 12, Fusion 8 and Fusion Pro 8

August 25th, 2015

VMware today released version 12 of its type-2 client hypervisor products for Microsoft Windows: Workstation Pro and Player and version 8 of its client hypervisor products for Mac OSX: Fusion….

Release: Veeam Management Pack v8 for System Center

August 25th, 2015

Veeam today released version 8.0 of its Management Pack for Microsoft System Center Operations Manager (SCOM or OpsMgr). Using this Management Pack customers running Operations Manager to monitor vSphere and…

Release: Oracle VM VirtualBox 5.0

August 24th, 2015

Oracle has released version 5.0 of its type-2 virtualization platform VM VirtualBox. Starting with a public beta in April this year and some more test builds and a release candidate…

Microsoft releases 3rd technical preview of Windows Server 2016 and System Center 2016

August 19th, 2015

Microsoft today announced the release of the 3rd technical preview of the next version of its Server Operating System (Windows Server 2016 TP3) and corresponding management software (System Center…

From VirtualizationMatrix comes WhatMatrix – UPDATED

August 19th, 2015

We quoted several times Virtualization Matrix, the website created to compare the capabilities of the various virtualization platforms on the market such VMware, Microsoft and Citrix.
This idea sparked…

Release: Linux Integration Services Version 4.0 for Hyper-V

August 18th, 2015

Microsoft has released version 4.0. of its Integration Services for Linux. Hyper-V supports both emulated (“legacy”) and Hyper-V-specific (“synthetic”) devices for Linux virtual machines. When a Linux virtual machine…

Monthly Archive