VSAN 6.2 Part 4 – IOPS limit for object

limitsVSAN 6.2 has a new quality of service mechanism which we are referring to as “IOPS limit for object”. Through a policy setting, a customer can set an IOPS limit on a per object basis (typically VMDK) which will guarantee that the object will not be able to exceed this amount of IOPS. This is very useful if you have a virtual machine that might be consuming more than its fair share of resources. This policy setting will ensure that there are “guard rails” placed on this virtual machine so it doesn’t impact other VMs, or impact the overall performance of the VSAN datastore.

The screenshot below shows what the new “IOPS limit for object” capability looks like in the VM Storage Policy. Simply select “IOPS limit for object” for your policy, and then select an integer value for the IOPS limit. Any object (VMDK) that has this policy assigned will not be able to generate more than that number of IOPS.

IOPS Limit for objectNormalized to 32KB

The IO size for IOPS Limit is normalized to 32KB. This means that if you set the IOPS Limit to 10,000 and the typical I/O size from the VM was 64KB, then you could do only 5,000 IOPS. If your block size is 4KB/8KB/16KB or 32KB, you would be able to achieve the 10,000 IOPS limit. My understanding is that there is no way to change this normalized I/O size in this release. Note that this is a hard limit on the number of IOPS so even if there are plenty of resources available on the system to do more, this will prevent the VM/VMDK from doing so.

Considerations

One thing to consider is that not only is read and write I/O counted towards the limit, but also any snapshot I/O that occurs against the VM/VMDK is added to the IOPS limit.

If the I/O against a particular VM/VMDK rises about the IOPS Limit threshold, i.e. it is set to 10,000 IOPS and we receive the 10,001st I/O, then that I/O is delayed/throttled.

8 comments
  1. Didn’t we already pay for this functionality in vSphere with Storage I/O Control Limits with our expensive Enterprise+ licenses?

    • Unfortunately SIOC does not work with VSAN at this point in time Luke. As for licensing editions and which features go with which, I have absolutely no input into those decisions.

  2. I recently upgrade to vCenter, ESXi 6.0 U2 (and vSAN 6.2). However, I have not yet upgraded the on disk format. Currently, I do not see the “IOPS limit for object” rule option in my VM Storage Policies. I’ve been unable to find any references in the documentation that specify whether the on disk format upgrade is required, although maybe I missed it, or maybe it is assumed. Can you confirm?

  3. Hi Cormac,

    Are there any plans to add multi-tenancy capabilities to VSAN so you could break the VSAN datastore into logical datastores that each could have their own IOPS limits/guarantees, RBAC, IP addresses, etc.?

    Many thanks as always
    Mark

    • Not to my knowledge Mark. The idea with VSAN, as you know, is that we provide storage granularity on a per-VM basis. There are plans to do QoS on VSAN on a per-VM basis, but nothing that I can share publicly yet. Hopefully we will be able to talk more about these plans at VMworld 2016.

Comments are closed.