CNS – not just for vSAN

After a very eventful VMworld, we received lots of questions about CNS, the Cloud Native Storage feature that was released with vSphere 6.7U3. Whilst most of the demonstrations and blog articles around CNS focused on vSAN, what may have been missed is that this feature also works with both VMFS and NFS datastores. For that reason, I decided to create some examples of how CNS can also bubble up information in vSphere about Kubernetes Persistent Volumes (PVs) created on both VMFS and NFS datastores. Let’s begin by creating some simple policies to tag my VMFS datastore and my NFS datastore.…

ATS Miscompare revisited in vSphere 6.5

Does anyone remember the ATS Miscompare issue? This blog post from 2 years ago might jog your memory. It is basically an issue that arose when we began using ATS, the VAAI Atomic Test and Set primitive, for maintaining the ‘liveness’ of a heartbeat in vSphere 5.5U2.  After making this change, a number of customers started to see “ATS Miscompare detected between test and set HB images” messages after upgrading to vSphere 5.5U2 or later. The HB reference in the message is shorthand for heartbeat. In previous releases, we did not use ATS for maintaining the ‘liveness’ of a heartbeat.…

Improved block allocation mechanism on VMFS-6

Along with other improvements to VMFS-6, there is also a new block allocation mechanism which aims to reduce lock contention between hosts sharing the same VMFS-6 filesystem. To understand how lock contention could arise, it is important to understand that resources on VMFS are grouped into resource clusters; this is the same for VMFS-6 and earlier versions on VMFS. Resources could be file descriptors, sub-block, file block, pointer blocks, and so on. Historically, we have always tried to allocate different resource clusters to different ESXi hosts, which meant that only VMs running on the same host shared resources within the…

A change to sub-blocks on VMFS-6

Something that I only just recently noticed is that we have made a change to the sub-blocks structure on VMFS-6, compared to VMFS-5. Sub-blocks are small allocations on a VMFS volume, and they are used to back small files. They were introduced as a space-saving measure to prevent using a full file block to back a very small file. To put this simply, when a file is created on VMFS, it is initially backed by a sub-block, and when the file grows above the size of a sub-block, it is switched to being backed by a file block (this has…

Storage for containers with VMware? You got it!

Last week during a visit to VMware headquarters in Palo Alto, I had the opportunity to catch up with our engineering team who are responsible for developing storage solutions for Docker and Kubernetes running on vSphere. I have written about our Docker volume driver for vSphere and Kubernetes on vSphere already, but it’s been a while since I caught up with the team, and obviously more and more enhancements are being added all the time. I thought it might be useful to share the improvements with you here. There also seems to be some concerns raised about the availability of…

What’s new in vSphere 6.5 Core Storage

Hello from VMworld EMEA in Barcelona. Well, we can finally talk about vSphere 6.5 today. In this post, I want to highlight a number of new and enhanced features that you will find in vSphere 6.5 related to core storage. I am not going to discuss Virtual SAN (VSAN), Virtual Volumes (VVols) or I/O Filter enhancements (VAIO) specifically in this post, although you will no doubt see some new features tie directly into the latter. Instead, I want to talk about those features that are specific to core storage.

Migrating a VM with snapshots to/from VSAN

In Virtual SAN 6.0, a new snapshot format was introduced called vsanSparse. This improves snapshot functionality by leveraging the new VirstoFS on-disk format used with VSAN 6.0. I had a question recently about what would happen if I migrated a VM with a traditional vmfsSparse/redo log type snapshot. The question was whether or not it would be converted to the new vsanSparse format. Similarly, what if a VM with a vsanSparse snapshot was migrated from VSAN to a traditional VMFS/NFS datastore? Would it also be converted between formats? I decided that the only way was to try it out.