New vSAN 6.7U1 Advanced Options

Most readers will be aware that vSAN version 6.7U1 was recently released. For those of you who wish to know more about the release, I wrote this blog article last month detailing the new features. In this post I want to cover an item which many of you may not be aware of. It is a new feature which makes the most common vSAN advanced options visible and configurable in the vSphere UI. There are 3 advanced options which we have surfaced up. The first is the VSAN.ClomRepairDelay timer which is the delay used before rebuilding ABSENT components. The second is VSAN.DomOwnerForceWarmCache which is used to determine whether reads should come from all replicas of a given object, or whether it should just read from replicas on a particular site/Fault Domain. Finally, there is an Advanced Setting called VSAN.SwapThickProvisionDisabled for making VM swap objects to thin. In the past, these changes had to be made on a per host basis, which was a lot of operational overhead, and had the added risk of making a mistake when editing one of the hosts. In 6.7U1, we can now make this change cluster wide with a single click. Let’s take a closer look at these advanced options and look at them in more detail.

Here is where to find the advanced settings in the UI. Navigate to Cluster > Configure > vSAN > Services > Advanced Options:

By clicking on the Edit option, you get the ability to change the advanced options:

I’ve highlighted the Site Read Locality in the screenshot above as that may be the one most people are unfamiliar with, but lets describe the 3 advanced options in more detail.

Object Repair Delay

For anyone who has used vSAN, you will be familiar with this setting. This determines how long vSAN will wait before rebuilding components that have entered into an ABSENT state, either due to a failure in the cluster, or some sort of maintenance task. By default this value is set to 60 minutes, but it can be changed if necessary. For example, if you are going through a proof of concept and wanted to test the rebuild behaviour, you could tune it down to a minimum of 15 minutes simply for test purposes. If you are doing a maintenance task that will take longer than 1 hour, you can also increase the timeout. However we do recommend leaving it at the default value of 60 minutes for most normal operations. This should be enough time for a host go through an upgrade cycle of entering maintenance mode, upgrading software, rebooting, and finally exiting maintenance mode. VMware Update Manager (VUM) uses the Ensure Accessibility maintenance mode option during upgrades, which means VM components are not moved from an ESXi host so long as the VM is still available (still has another copy of the data and has a quorum of votes). Those components then enter ABSENT state, and vSAN will wait 60 minutes before taking any action.

In the past, you had to make this changed in the advanced settings of every ESXi host. Now you can do it in a single place.

Site Read Locality

I’m guessing readers won’t be quite so familiar with this setting. With normal vSAN deployments, we read in a cyclical fashion from all replicas that make up a RAID-1 object. We do this for cache efficiency purpose so that a particular block of data is cached only once. However, with vSAN stretched cluster, we need to take a different approach as we do not want to be reading from the replica that is on the other site across the inter-site link (which typically has a smallish bandwidth capability). In this case, we want our VM to read locally from only the replica(s) that are in same fault domain as the VM. This avoids traversing the inter-site link. Note that this was always taken care of automatically, and the same is true here. The option is enabled for stretched clusters. Actually, you cannot modify this setting on standard vSAN deployments – it is only configurable for vSAN stretched clusters, and more importantly, vSAN 2-node deployments.

Now with 2-node deployments, the same site read locality as stretched cluster is implemented. In other words, a VM deployed on 2-node vSAN will only read from a single replica, the one which is in the same Fault Domain as the VM. Ideally we would want to revert to reading from both replicas and get the cache efficiency benefits. This is really the only time customers would need to modify this setting.

Thin Swap

For the longest time, VM swap objects were instantiated on vSAN as thick. Providing the ability to do thin swap objects provided considerable space savings, in the knowledge that there may be some overhead in growing the swap object should the need arise. Once again, in the past, customers had to make a per host advanced setting change to configure thin swap. Now it is available via a single click in the UI. In fact, Thin Swap is enabled by default. Thus you would only need to modify this setting if you wanted to revert to thick swap objects.