Pluggable Storage Architecture (PSA) Deep-Dive – Part 2

As I highlighted in the PSA part 1 post, NMP, short for Native Multipath Plugin, is the default Multipath Plugin shipped with ESXi hosts. Once the PSA has associated the NMP with particular paths, it uses a number of sub-plugins to handle load balancing and path fail-over. In this post, I will look at the NMP in more detail. I will pay specific attention to the activity of the Storage Array Type Plugin (SATP) which is responsible for handling path fail-over for a given storage array and also the Path Selection Plugin (PSP), which determines which physical path is used to issue an I/O request (load balancing) to a storage device.

If a logical device and its paths are associated with the NMP by the PSA, VMware provides a default SATP for each supported array on the Hardware Compatibility List (HCL). Each SATP supports a specific type of storage array, and make the decision on when to fail-over to an alternate path. Typically, this is determined by the SCSI sense codes returned by the array. Each SATP also has a default PSP associated, although this can be changed if you believe your array would benefit from a different PSP to the default or if your storage array vendor provides you with an alternative PSP.

To look at the available SATPs on an ESXi host (and to see the default PSP associated with said SATP), you can use the following command:

• # esxcli storage nmp satp list

Note that only SATPs which are required are loaded. There is no point in loading all these plugins into memory if no storage device is going to use them. In this case, the default Active/Active plugin for SAN devices and the local plugin for direct attached devices are the only ones which are loaded. You may see different ones loaded depending on which storage array your ESXi host(s) is connected to. To examine the full list of PSPs, you can use this command:

• # esxcli storage nmp psp list

By default, we ship with three unique PSPs. The PSP handles local balancing operations and is responsible for choosing a physical path to issue an I/O request to a logical drive. There are a number of CLI commands to help you query the plugins used by particular logical devices and paths.

• # esxcli storage nmp path list
• # esxcli storage nmp device list

These commands will show you exactly which SATP and which PSP are currently being used by a particular device and path.

Now we know what the SATPs are, but how does NMP choose which SATP to associate to a path. Well, there is another set of rules used for that. To examine the complete set of rules for an SATP, use the command:

• # esxcli storage nmp satp rule list

This will list all the rules, and it is rather long. To narrow down the list and see the rules for a single SATP, use the -s option:

• # esxcli storage nmp satp rule list -s VMW_SATP_LOCAL

If a match is made based on transport as per the list above, then NMP will use the VMW_SATP_LOCAL for these devices. As you examine other SATPs, you will see different rules defined. You may observe an SATP referenced based on the array vendor, array models or other options.

By way of recap, what you’ll have observed is that the PSA will choose an NMP depending on a claim-rule. The NMP will then choose an SATP, again based on another set of predefined rules. An SATP then has a PSP associated with it by default. In my next post, I will examine the SATP & PSP in more detail.