In Virtual SAN 6.0, a new snapshot format was introduced called vsanSparse. This improves snapshot functionality by leveraging the new VirstoFS on-disk format used with VSAN 6.0. I had a question recently about what would happen if I migrated a VM with a traditional vmfsSparse/redo log type snapshot. The question was whether or not it would be converted to the new vsanSparse format. Similarly, what if a VM with a vsanSparse snapshot was migrated from VSAN to a traditional VMFS/NFS datastore? Would it also be converted between formats? I decided that the only way was to try it out.
I’ve been hit up this week by a number of folks asking about “ATS Miscompare detected between test and set HB images” messages after upgrading to vSphere 5.5U2 and 6.0. The purpose of this post is to give you some background on why this might have started to happen.
First off, ATS is the Atomic Test and Set primitive which is one of the VAAI primitives. You can read all about VAAI primitives in the white paper. HB is short for heartbeat. This is how ownership of a file (e.g VMDK) is maintained on VMFS, i.e. lock. You can read more about heartbeats and locking in this blog post of mine from a few years back. In a nutshell, the heartbeat region of VMFS is used for on-disk locking, and every host that uses the VMFS volume has its own heartbeat region. This region is updated by the host on every heartbeat. The region that is updated is the time stamp, which tells others that this host is alive. When the host is down, this region is used to communicate lock state to other hosts.
In vSphere 5.5U2, we started using ATS for maintaining the heartbeat. Prior to this release, we only used ATS when the heartbeat state changed. For example, referring to the older blog, we would use ATS in the following cases:
- Acquire a heartbeat
- Clear a heartbeat
- Replay a heartbeat
- Reclaim a heartbeat
We did not use ATS for maintaining the ‘liveness’ of a heartbeat. This is the change that was introduced in 5.5U2 and which appears to have led to issues for certain storage arrays.
There was a time when VMFS was the only datastore that could be used with ESXi. That has changed considerably, with the introduction of NFS (v3 and v4.1), Virtual Volumes and of course Virtual SAN. However VMFS continues to be used by a great many VMware customers and of course we look to enhance it with each release of vSphere. This post will cover changes and enhancements to VMFS in vSphere 6.0.
Welcome to the first in a series of posts related to new storage enhancements in vSphere 5.1. The first of these posts will concentrate on VMFS. There are two major enhancements to VMFS-5 in the vSphere 5.1 release.
VMFS File Sharing Limits Increase
Prior to vSphere 5.1, the maximum number of ESXi hosts which could share a read-only file on a VMFS filesystem was 8. This was a limiting factor for those products and features which used linked clones. Linked Clones are simply “read/write” snapshots of a “master or parent” desktop image. In particular, it was a limitation for vCloud Director deployments using linked clones for Fast Provisioning of vApps and VMware View VDI deployments using linked clones for desktops.
In vSphere 5.1, we are increasing this maximum number of hosts that can share a read-only file (or to state this another way, we are increasing the number of concurrent host locks) to 32. This will only apply to hosts running vSphere 5.1 and higher on VMFS-5. Now vCloud Director and VMware View deployments using linked clones can have 32 hosts sharing the same base disk image.
This makes VMFS-5 datastores as scalable as NFS for VDI deployments using VMware View and vCloud Director deployments when using linked clones.
It should be noted that versions of VMware View 5.0 (and earlier) limited the number of hosts which could use linked-clone based desktops to 8. This was true for both VMFS and NFS datastores. VMware View 5.1, released earlier in 2012, increased this host count to 32 for NFS on vSphere 5.0. With the next release of VMware View & vSphere 5.1, you can have 32 hosts sharing the same base disk with both NFS & VMFS datastores.
One final point – this is a driver only enhancement. There are no on-disk changes required on the VMFS-5 volume to benefit from this new feature. Therefore customers who are already on vSphere 5.0 and VMFS-5 need only move to vSphere 5.1. There is no upgrade or change required to their already existing VMFS-5 datastores.
VOMA – vSphere On-disk Metadata Analyzer
VOMA is a new customer facing metadata consistency checker tool, which is run from the CLI of ESXi 5.1 hosts. It checks both the Logical Volume Manager (LVM) and VMFS for issues. It works on both VMFS-3 & VMFS-5 datastores. It runs in a check-only (read-only) mode and will not change any of the metadata. There are a number of very important guidelines around using the tool. For instance, VMFS volumes must not have any running VMs if you want to run VOMA. VOMA will check for this and will report back if there are any local and/or remote running VMs. The VMFS volumes can be mounted or unmounted when you run VOMA, but you should not analyze the VMFS volume if it is in use by other hosts.
If you find yourself in the unfortunately position that you suspect that you may have data corruption on your VMFS volume, prepare to do a restore from backup, or look to engage with a 3rd party data recovery organization if you do not have backups. VMware support will be able to help in diagnosing the severity of any suspected corruption issues, but they are under no obligation to recover your data.
I’m sure you will agree that this is indeed a very nice tool to have at your disposal.
Get notification of these blogs postings and more VMware Storage information by following me on Twitter: @CormacJHogan