New vSAN 6.7U1 Advanced Options

Most readers will be aware that vSAN version 6.7U1 was recently released. For those of you who wish to know more about the release, I wrote this blog article last month detailing the new features. In this post I want to cover an item which many of you may not be aware of. It is a new feature which makes the most common vSAN advanced options visible and configurable in the vSphere UI. There are 3 advanced options which we have surfaced up. The first is the VSAN.ClomRepairDelay timer which is the delay used before rebuilding ABSENT components. The second is VSAN.DomOwnerForceWarmCache which is used to determine whether reads should come from all replicas of a given object, or whether it should just read from replicas on a particular site/Fault Domain. Finally, there is an Advanced Setting called VSAN.SwapThickProvisionDisabled for making VM swap objects to thin. In the past, these changes had to be made on a per host basis, which was a lot of operational overhead, and had the added risk of making a mistake when editing one of the hosts. In 6.7U1, we can now make this change cluster wide with a single click. Let’s take a closer look at these advanced options and look at them in more detail.

Here is where to find the advanced settings in the UI. Navigate to Cluster > Configure > vSAN > Services > Advanced Options:

By clicking on the Edit option, you get the ability to change the advanced options:

I’ve highlighted the Site Read Locality in the screenshot above as that may be the one most people are unfamiliar with, but lets describe the 3 advanced options in more detail.

Object Repair Delay

For anyone who has used vSAN, you will be familiar with this setting. This determines how long vSAN will wait before rebuilding components that have entered into an ABSENT state, either due to a failure in the cluster, or some sort of maintenance task. By default this value is set to 60 minutes, but it can be changed if necessary. For example, if you are going through a proof of concept and wanted to test the rebuild behaviour, you could tune it down to a minimum of 15 minutes simply for test purposes. If you are doing a maintenance task that will take longer than 1 hour, you can also increase the timeout. However we do recommend leaving it at the default value of 60 minutes for most normal operations. This should be enough time for a host go through an upgrade cycle of entering maintenance mode, upgrading software, rebooting, and finally exiting maintenance mode. VMware Update Manager (VUM) uses the Ensure Accessibility maintenance mode option during upgrades, which means VM components are not moved from an ESXi host so long as the VM is still available (still has another copy of the data and has a quorum of votes). Those components then enter ABSENT state, and vSAN will wait 60 minutes before taking any action.

In the past, you had to make this changed in the advanced settings of every ESXi host. Now you can do it in a single place.

Site Read Locality

I’m guessing readers won’t be quite so familiar with this setting. With normal vSAN deployments, we read in a cyclical fashion from all replicas that make up a RAID-1 object. We do this for cache efficiency purpose so that a particular block of data is cached only once. However, with vSAN stretched cluster, we need to take a different approach as we do not want to be reading from the replica that is on the other site across the inter-site link (which typically has a smallish bandwidth capability). In this case, we want our VM to read locally from only the replica(s) that are in same fault domain as the VM. This avoids traversing the inter-site link. Note that this was always taken care of automatically, and the same is true here. The option is enabled for stretched clusters. Actually, you cannot modify this setting on standard vSAN deployments – it is only configurable for vSAN stretched clusters, and more importantly, vSAN 2-node deployments.

Now with 2-node deployments, the same site read locality as stretched cluster is implemented. In other words, a VM deployed on 2-node vSAN will only read from a single replica, the one which is in the same Fault Domain as the VM. Ideally we would want to revert to reading from both replicas and get the cache efficiency benefits. This is really the only time customers would need to modify this setting.

Thin Swap

For the longest time, VM swap objects were instantiated on vSAN as thick. Providing the ability to do thin swap objects provided considerable space savings, in the knowledge that there may be some overhead in growing the swap object should the need arise. Once again, in the past, customers had to make a per host advanced setting change to configure thin swap. Now it is available via a single click in the UI. In fact, Thin Swap is enabled by default. Thus you would only need to modify this setting if you wanted to revert to thick swap objects.


14 Replies to “New vSAN 6.7U1 Advanced Options”

  1. In a vSAN 2 node direct connect deployment, there is no isolatin address to set and you get:

    – a red circle on the host (with all green circle in the vSAN Health)
    – a warning “This host ha no isolation address defined as required by vSphere HA”
    – a cluster configuration issue

    Is it acceptable?
    How to suppress this unresolvable warning?

    1. Hi Max, just to clarify, you have changed the defaultisolationaddress to false, but did not fill in the isolationaddress setting, is that correct? I think that is why you are getting the error.

      From : it simply states “When using vSAN 2 Node clusters in the same location, there is no need to have a separate das.isolationaddress for each of the hosts.”

      So to me, this reads like you must still configure a das.isolationaddress for the direct connected hosts to remove the error.

  2. Hi Cormac,
    this is exactly what I’ve done (default isalation address = false and no alternative address).
    So what is best?:
    – configure the normal isolation address on the vmk0 (not vSAN network), as a normal vSphere cluster
    – set default isolation address to false and use an IP in the vSAN network (which address? the are only to responding IP, the nodes vSAN IP)


  3. Hello Cormac, I opened also a support case but it seems no one can answer, which isolation address to use?

  4. Hi Cormac,

    According to the vSAN 6.7 Update 1 Release Notes, there should be a option about “large cluster support” in the vSphere Client, but I didn’t find that option even all my environment is vSphere 6.7U1.

    How to enable the option about “large cluster support” in vSphere Client?

    1. Should be in Cluster > Configure > vSAN > Services > Advanced Options.
      [Update – just checked – looks like that Advanced option never made it to the UI in 6.7U1]

  5. I have set the “/VSAN/goto11” to 1 in all the ESXi hosts in vSAN enabled Cluster, reboot all the hosts one by one then found that there is an error in vSAH Healthy Check and saying “Large scale cluster support” is Mismatch in vSphere Client.

    I am thinking there should be a setting in vCenter Server, Cluster or somewhere else to make the “Mismatch” to be “Match”.

    1. Interesting – sounds like the health check is expecting to find a global setting. I’d recommend speaking to GSS about it. They may know why this health check is failing in this way.

      1. Thanks. Agree with you, just realized that a VMware KB about large scale may need be updated accordingly. I will post the link later.

Comments are closed.