Paper: VMware vSphere Storage APIs – Array Integration (VAAI)

VMware has released a paper titled: VMware vSphere Storage APIs – Array Integration (VAAI), the paper which contains 19 pages details the current vSphere Storage APIs – Array Integration (VAAI) available today. When VAAI was introduced in the vSphere 4.1 timeframe there were three block based storage (Fibre Channel, iSCSI and FCoE) and with the introduction of vSpere 2 VAAI primitives for NAS storage and vSphere Thin Provisioning were introduced.

VMware vSphere Storage APIs – Array Integration (VAAI), also referred to as hardware acceleration or hardware offload APIs, are a set of APIs to enable communication between VMware vSphere ESXi hosts and storage devices. The APIs define a set of “storage primitives” that enable the ESXi host to offload certain storage operations to the array, which reduces resource overhead on the ESXi hosts and can significantly improve performance for storage-intensive operations such as storage cloning, zeroing, and so on. The goal of VAAI is to help storage vendors provide hardware assistance to speed up VMware I/O operations that are more efficiently accomplished in the storage hardware.


Without the use of VAAI, cloning or migration of virtual machines by the vSphere VMkernel Data Mover involves software data movement. The Data Mover issues I/O to read and write blocks to and from the source and destination datastores. With VAAI, the Data Mover can use the API primitives to offload operations to the array if possible. For example, if the desired operation were to copy a virtual machine disk (VMDK) file from one datastore to another inside the same array, the array would be directed to make the copy completely inside the array. Whenever a data movement operation is invoked and the corresponding hardware offload operation is enabled, the Data Mover will first attempt to use the hardware offload. If the hardware offload operation fails, the Data Mover reverts to the traditional software method of data movement.

In nearly all cases, hardware data movement will perform significantly better than software data movement. It will consume fewer CPU cycles and less bandwidth on the storage fabric. Improvements in performance can be observed by timing operations that use the VAAI primitives and using esxtop to track values such as CMDS/s, READS/s, WRITES/s, MBREAD/s, and MBWRTN/s of storage adapters during the operation.

The paper contains the following sections:

  • Introduction to VAAI
  • VAAI Block Primitives
  • VAAI NAS Primitives
  • VAAI Thin Provisioning Primitives
  • VAAI Implementation
  • Checking VAAI Support
  • Enabling and Disabling VAAI
  • Reverting to the SoftwareMover
  • Differences in VAAI between vSphere 4.x and 5.x
  • VAAI Caveats

Thanks to Eric Sloof for providing the news.