Storage I/O Control – Workload Injector Behaviour

You may remember an enhancement which we made to Storage I/O Control (SIOC) in the 5.1 vSphere release whereby SIOC can now automatically determine the characteristics and thus the latency threshold of a datastore. Prior to this change, SIOC used either a default value or had customers manually set it. Neither of these were ideal, so we introduced this automatic method. However, there was little detail on how often this latency threshold was calculated. In other words, did the calculation take place when SIOC was first enabled, or is there regular on-going calculations?

While trying to find the answer, I found this paragraph in this excellent Storage DRS white paper. It states: To assess the performance of each datastore, vSphere Storage DRS leverages vSphere Storage I/O Control, using the workload injector and a reference workload to determine the relative performance capability. To get a proper performance capability model, vSphere Storage I/O Control injects random read I/O to the disk, using various amounts of outstanding I/O to measure the latency. The duration of the complete cycle is 30 seconds; the procedure is triggered once a day per datastore. Although it’s a short-lived process, this workload generates some overhead on the array. vSphere Storage DRS is designed to enable storage performance for your virtual machines, not to interfere with them. Therefore this workload will not run when activity is detected on the devices backing the datastore.

However this is for Storage DRS, which leverages SIOC. I could not find a specific reference to how SIOC works in a stand-alone mode. I pinged some of our SIOC team at VMware, and I can now state that the behaviour for SIOC is much the same as above. Once SIOC is enabled, the frequency that the workload injector runs is about once a day, for a duration of about 30 seconds. If one adds additional disks to a datastore, thereby making the performance of the datastore better, this will be detected on the next run of the workload injector and the threshold will be adjusted appropriately.There is no extra step needed for this process to happen. For example you do not need to restart SIOC.

It should also be noted that the SIOC workload injector only runs when the datastore is in an idle period. If SIOC detects other workloads on the datastore while it is running the workload injector, it will throttle back so as not to cause any impact to the production workloads.

Finally, this behaviour is not to be confused with the real work of SIOC, the actual throttling of I/O. SIOC and workload injector are two separate processes. SIOC throttles I/O load if the current latency is higher than the threshold setting on the datastore. It runs every 4 seconds to determine if, and how much, I/O needs to be throttled for the subsequent 4-second interval.