VSAN Part 23 – Why is my Storage Object Striped?
Those of you familiar with VSAN will know that one of the capabilities which can be placed in a VM Storage Policy is Number of Disk Stripes Per Object (stripe width for short). I covered this in an earlier post which looked at the various VSAN capabilities. Recently, a customer who had not specified a stripe width in the VM Storage Policy was perplexed to find that his storage objects had indeed been striped across a number of disks. He reached out to me if I could provide an explanation.
The customer in question was actually testing the behaviour of the default policy on his nested environment (running ESXi in a VM). The customer knew that the default policy (or None) would set Failures To Tolerate to 1 and stripe width to 1; in other words, there is no striping requirement requested. (Note that when Stripe Width is added to a VM Storage Policy, the value specified is a minimum number of stripes – it may be more). However, when he looked at the Physical Disk Placement view after the VM was deployed with a VM Storage Policy of None, he saw that there was indeed a stripe width of 2 used.
On another occasion, he deployed another virtual machine using the default policy, and was once again surprised to see that the storage object striped across 3 disks on this occasion.
So what was going on? I had two questions for the customer:
- How big are your nested ESX hard disks?
- How big are your VMDK storage objects?
The reason I asked these questions is because if the VMDK storage object is too big to fit onto the free space on a single hard disk, then it will automatically be striped across multiple hard disks.
I suspected this was what is happening here, and the customer confirmed it. When he Increased the size of the virtual SSD and HDD sizes in his nested environment, allowing the VMDK storage object to reside in the free space on a single disk, the striping issues went away, i.e. the storage object was no longer striped.
One final point on striping in VSAN. The Number of disk stripes per object setting in the VM Storage Policy means stripe across “at least” this number of magnetic disks per mirror”. VSAN may, when it sees fit, use additional stripes. And when it does decide to use more stripes (maybe because there is a lack of free space on some magnetic disks), those stripes may even be on the same spindle in some cases.
Thanks Cormac: I cannot understand the last sentence: those stripes may even be on the same spindle.
If it’s a stripe it must be on 2 or more spindles (aka magnetic disks). Or not ?
Yes, that is true but they will still be part of a RAID configuration. Remember that you have not requested any stripe width in the policy (if you do, VSAN will not choose stripes on the same magnetic disks). The reason why VSAN puts ‘stripes’ on the same disk is just to be able to use space on those disks when the free space is smaller than the requested storage object size. The reason I highlighted it is so that if you observe such behaviour, you will know it is by design.