This is something that was recently brought to my attention, and I wasn’t aware of this difference in behavior between the various storage vendors who implement VAAI-NAS. VAAI-NAS implements a number of different offload primitives, but the one we are interested in here is the Fast File Clone primitive which is the ability to offload the creation of snapshots/clones to the NAS storage array. This mechanism is also referred to as Native Snapshots. However, some arrays cannot support a full chain of snapshots.
When a VM snapshot is taken with VAAI-NAS using the Fast File Clone primitive, the storage array (using native snapshot or clone technology) will create an object that will then be presented back to the ESX host and appear as a VM Snapshot. Given that this would be cloning at the file level, each clone would be independent per VM, and would not occur at the datastore level like we do with a normal volume snapshot.
Now, it appears that different vendors support different levels of snapshots. For example, if I create a Virtual Machine disk file, and snapshot it, that snapshot create operation will be sent to the storage array if VAAI-NAS primitives are in place. However, my next snapshot may or may not be offloaded, depending on the implementation by the storage vendor. If the snapshot operation exceeds the allowed chain depth, a message ‘The number of snapshot levels has exceeded supported limits‘ is displayed in the logs.
For example, EMC’s EMCNasPlugin 1.0-10 version for VNX only goes to a single level before reporting that it has exceeded the supported limits. [Update: It appears that a 2012 release of VNX firmware includes a flag for chained snapshot support. This must be set when creating the volume however; it cannot be set post volume creation. This allows the level to increase to 2 snapshots in a chain. See comment section for further details]. NetApp’s NAS plugin can go much deeper; internally, I’ve seen it go to a depth of 10 snapshots, but NetApp claim that it can go much deeper than that. In conversation with folks at HDS, it seems that they can accommodate much larger depths of snapshots on their HDS-HNAS product too. At the time of writing, I wasn’t able to get an exact figure from the EMC Isilon folks. However, considering we only support a snapshot chain of 32 for virtual machine snapshots, then you may consider that NetApp & HDS VAAI-NAS implementations both meet this. I’ll try to confirm the EMC Isilon figure at a later date. [Update: Isilon has unlimited snapshot support. See comments.]
What happens when the supported snapshot chain depth is reached? In that case, we simply revert back to using host based snapshot technology and create snapshots using the vmfsSparse (redo log) format.