This is possibly the most exciting new storage feature in the vSphere 5.1 release. Space Efficient Sparse Virtual Disks (or SE Sparse Disks for short) were designed to alleviate two issues. Let’s describe these issues first of all.

Problem Statement #1 – Let’s take a Guest OS running on a linked clone (View desktop if you will), and this Guest OS issues a 4KB write. vmfsSparse disk (which is the format used by traditional linked clones) has a block allocation unit size of 512 bytes. In other words, this Guest OS is backed by 512 byte blocks. Depending on the applications deployed in the Guest OS, a worst case scenario is that these 512 byte blocks may not be contiguous on the VMDK, and thus may not be contiguous on the VMFS or NFS datastore. This could lead to multiple writes taking place on the back-end storage array for a single Guest OS write. Another side effect is that the partition created on Guest OS may also be misaligned (because of the very small allocation unit size), again causing multiple writes to take place on the array for a single Guest OS write. Finally, this 512 byte block allocation unit size may not match the block size preference of the storage array, leading to additional overhead in handling these smaller, partial writes.

Problem Statement #2 – The major space inefficiency issue of allocating as yet unused blocks in the Guest OS filesystem/database has basically been addressed by Thin Provisioning. However, another major space efficiency issues still exists – the issue of reclaiming Stale/Stranded data from within a Guest OS. While VMware has addressed this at the datastore level with the VAAI UNMAP primitive, it is still an issue from within the Guest OS. This is particularly problematic with VMware View Desktops deployed on linked clones. These desktops start off as very small in size, but over a period of time they will grow and may end up being as big as the base disk (again, worst case scenario). This then requires administrative intervention to reduce the size of the desktops.

Now that we understand the main issues, let’s see how the new SE Sparse Disk format helps to address them.

Addressing Issue #1 – By default the grain size/block allocation unit size for Virtual Machine disks on ESX is 4KB. The vmfsSparse format, used by snapshots and linked cloned have a grain size of 512 bytes or 1 sector. The vmfsSparse format get 16MB chunks at a time from VMFS, but then allocates it at 512 bytes at a time. This is the root cause of many of the performance/alignment complaints that we currently get with linked-clones/snapshots, and what we are addressing with SE Sparse Disks.

With the introduction of SE Sparse disks, the grain size/block allocation unit size is now tuneable and can be set based on the preferences of a particular storage array or application. Note however that this full tuning capability will not be exposed in vSphere 5.1.

Addressing Issue #2 – One of the major features of the new SE Sparse Disk is its ability to reclaim previously used space within the Guest OS. This stale data is data that was previously written to, but is currently in unaddressed blocks in a file system/database. Customers used to have to carry out some very manual processes to reclaim this stranded space in the past, using a combination of Guest OS tools and vSphere technologies (e.g. sdelete followed by Storage vMotion).

There are two steps involved in the space reclamation feature; the first step is the wipe operation which scans the Guest OS looking for stranded space and reorganizes the Virtual Machine Disk to frees up a contiguous area of free space.

The second step is the shrink operation which initiates either a SCSI UNMAP operation (block devices) or a RPC truncate (NFS) to delete the contiguous area of free space at the end of the VMDK, reducing its size, and then telling the storage array that it can now reclaim that area of free space.

The Wipe operation is initiated by an API call to the VMware Tools running in the Guest OS. This will allow the task to be scheduled out of hours so that there is no impact on the desktops. This initiates a scan of the filesystem looking for unused filesystem blocks.

When we know which blocks are free, we get the vSCSI layer to reorganise the SE Sparse Disk by moving blocks from the end of the SE Sparse disk to unallocated blocks at the beginning of the SE sparse disk. The SE Sparse disk metadata contains a bitmap where 1 bit represents a 4KB block and indicates if the block is allocated or unallocated.

When there is a contiguous range of free space at the end of the SE Sparse Disk, a SCSI UNMAP command is sent to reclaim those blocks, and truncate/shrink the SE sparse disk. Note that this is the same UNMAP primitive which we introduced in VAAI improvements in vSphere 5.0, so this will cause overhead on the storage arrays and could have a significant impact on performance for some storage arrays, just like dead space reclamation for VMFS-5 deployed on Thin Provisioned LUNs. This is why the recommendation is to run this reclaim feature out of hours or during a maintenance window.

During the shrink operation, allocated blocks at the end of the SE Sparse disk are moved to unallocated space at the beginning of the disk. This will leave a contiguous unallocated section at the end of the SE Sparse disk which can be truncated during the shrink operation.

Note that the Virtual Machines require HWv9 to handle the SCSI UNMAP command in the Guest OS – earlier versions will not know how to handle this command.

Use Case
There is a very specific use case for SE Sparse Disks in vSphere 5.1. The scope of SE Sparse Disks in vSphere 5.1 has been restricted to a VMware View use case when VMware View Composer uses “Linked Clones” for the roll-out of desktops.

VMware View desktops will also benefit from the new 4KB grain size, as it addresses the partial write and alignment issues experienced by some storage arrays when the 512 bytes grain size found in the vmfsSparse format is used by linked clones.

SE Sparse Disks also give far better space efficiency to desktops deployed on this virtual disk format since it has the ability to reclaim stranded space from within the Guest OS.

Get notification of these blogs postings and more VMware Storage information by following me on Twitter: @VMwareStorage