A short post today, but it highlights what I feel is an important enhancement to vSphere licensing. I’ve had lots of questions recently about why VAAI (Storage APIs for Array Integration) is not available in the standard edition of vSphere. This is especially true since I began posting about Virtual Volumes earlier this year, and it was clear that Virtual Volumes is available in the standard edition. One reason why this was confusing is that if a migration of a VVol could not be handled by the array using the VASA APIs, the migration would fall back to using VAAI offload primitives. But if you only had standard licensing for VVols, would you still be supported?
A few weeks, my good pal Cody Hosterman over at Pure Storage was experimenting with VAAI and discovered that he could successfully UNMAP blocks (reclaim) directly from a Guest OS in vSphere 6.0. VAAI are the vSphere APIs for Array Integration. Cody wrote about his findings here. Effectively, if you have deleted files within a Guest OS, and your VM is thinly provisioned, you can tell the array through this VAAI primitive that you are no longer using these blocks. This allows the array to reclaim them for other uses. I know a lot of you have been waiting for this functionality for some time. However Cody had a bunch of questions and reached out to me to see if I could provide some answers. After conversing with a number of engineers and product managers here at VMware, here are some of the answers to the questions that Cody asked.
The more astute of you who have already moved to vSphere 6.0, and like looking at CLI outputs, may have observed some new columns/fields in the PSA claimrules when you run the following command:
# esxcli storage core claimrule list --claimrule-class=VAAI
The new fields are as follows (slide right to view full output):
XCOPY Use Array XCOPY Use XCOPY Max Reported Values Multiple Segments Transfer Size --------------- ----------------- -------------- false false 0 false false 0 false false 0 false false 0 false false 0 false false 0 false false 0 false false 0 false false 0 false false 0 false false 0 false false 0 false false 0 false false 0
I’ve been hit up this week by a number of folks asking about “ATS Miscompare detected between test and set HB images” messages after upgrading to vSphere 5.5U2 and 6.0. The purpose of this post is to give you some background on why this might have started to happen.
First off, ATS is the Atomic Test and Set primitive which is one of the VAAI primitives. You can read all about VAAI primitives in the white paper. HB is short for heartbeat. This is how ownership of a file (e.g VMDK) is maintained on VMFS, i.e. lock. You can read more about heartbeats and locking in this blog post of mine from a few years back. In a nutshell, the heartbeat region of VMFS is used for on-disk locking, and every host that uses the VMFS volume has its own heartbeat region. This region is updated by the host on every heartbeat. The region that is updated is the time stamp, which tells others that this host is alive. When the host is down, this region is used to communicate lock state to other hosts.
In vSphere 5.5U2, we started using ATS for maintaining the heartbeat. Prior to this release, we only used ATS when the heartbeat state changed. For example, referring to the older blog, we would use ATS in the following cases:
- Acquire a heartbeat
- Clear a heartbeat
- Replay a heartbeat
- Reclaim a heartbeat
We did not use ATS for maintaining the ‘liveness’ of a heartbeat. This is the change that was introduced in 5.5U2 and which appears to have led to issues for certain storage arrays.
A short note to clarify something that has come up a number of times in recent weeks here at VMware. There have been a number of discussions about whether or not we support NFS over IPv6 on vSphere 5.x, and again, on whether or not we support the VAAI-NAS primitives in the same context.
VAAI is an API for offloading tasks to the storage array, but for offloading tasks to NAS arrays, storage vendors need to create their own plugins for the ESXi hosts to achieve this. You can learn more about VAAI-NAS by clicking here. So what about IPv6 support and NFS? And VAAI-NAS?
I was in a conversation with one of my pals over at Tintri last week (Fintan), and he observed some strange behaviour when provisioning VMs from a catalog in vCloud Director (vCD). When he disabled Fast Provisioning, he expected that provisioning further VMs from the catalog would still be offloaded via the VAAI-NAS plugin. All the ESXi hosts have the VAAI-NAS plugin from Tintri installed. However, it seems that the provisioning/cloning operation was not being offloaded to the array, and the ESXi hosts resources were being used for the operation instead. Deployments of VMs from the catalogs were taking minutes rather than seconds. What was going on?
Continuing on the series of vSphere 5.5 Storage Enhancements, we now come to a feature that is close to many people’s hearts. The vSphere Storage API for Array Integration (VAAI) UNMAP primitive reclaims dead or stranded space on a thinly provisioned VMFS volume, something that we could not do before this primitive came into existence. However, it has a long and somewhat checkered history. Let me share the timeline with you before I get into what improvements we made in vSphere 5.5.