Heads Up! vSphere 5.1 & EMC Storage

In this post, I want to call out two important matters related to the vSphere 5.1 release & EMC storage. The first is related to Round Robin Path Policy changes, and the second relates to a VMFS5 volume creation issue.

Round Robin Path Policy

Without delving too deeply into the Pluggable Storage Architecture found in ESXi hosts, VMware uses a Native Multipath Plugin for handling I/O between the host and a storage array. This has two important components – a Storage Array Type Plugin (SATP) for failover and Path Selection Policy (PSP) for load balancing. Each SATP has a default PSP associated with it. An esxcli storage nmp satp list will show you the relationship between SATP and its default PSP.

EMC have taken a very important step with the release of vSphere 5.1. My understanding is that a large portion of EMC storage is now going to use VMware’s Round Robin Path Selection Policy (PSP) by default.  Below is the output taken from a LUN from a VNX-5500 array presented to one of my ESXi 5.1 hosts. As you can clearly see, this is now using Round Robin without having to make any configuration changes.

naa.xxx
 Device Display Name: DGC Fibre Channel Disk (naa.xxx)
 Storage Array Type: VMW_SATP_ALUA_CX
 Storage Array Type Device Config: {navireg=on, ipfilter=on}
 {implicit_support=on; explicit_support=on; explicit_allow=on;
 alua_followover=on;{TPG_id=1,TPG_state=ANO}{TPG_id=2,TPG_state=AO}}
 Path Selection Policy: VMW_PSP_RR
 Path Selection Policy Device Config: {policy=rr,iops=1000,
  bytes=10485760,useANO=0;lastPathIndex=2: NumIOsPending=0,
  numBytesPending=0}
 Path Selection Policy Device Custom Config:
 Working Paths: vmhba2:C0:T3:L0, vmhba4:C0:T3:L0
 Is Local SAS Device: false
 Is Boot USB Device: false

In vSphere 5.1, the default PSPs for the Storage Array Type Plugins (SATPs) VMW_SATP_ALUA_CX and VMW_SATP_SYMM have changed from   VMW_PSP_FIXED  to  VMW_PSP_RR:

Using the command esxcli storage nmp satp list, we can see this change:

Name              Default PSP   Description
----------------  ------------  ---------------------------------
VMW_SATP_ALUA_CX  VMW_PSP_RR    Supports EMC CX that use the ALUA..
VMW_SATP_SYMM     VMW_PSP_RR    Placeholder (plugin not loaded)

I think that this is indeed a great move. I believe you’ll get optimal storage performance with the RR PSP. However if you use Microsoft Cluster Services (MSCS) you are probably aware that you cannot use the Round Robin path selection policy on the back-end storage. Without getting into too much detail, handling SCSI Reservations across multiple paths is the reason behind not supporting this. Therefore, if you use EMC storage, and you use virtualized MSCS environments, and you plan to upgrade to vSphere 5.1, keep this in mind. You will have to change the VMW_PSP_RR for those devices back to the original setting. There is plenty more information around MSCS supportability in vSphere environments in this KB article.

VMFS5 Volume Create on EMC VMAX/VMAXe & ATS

This second issue comes straight from the vSphere 5.1 Release Notes. If your ESXi host is connected to a VMAX/VMAXe array, you might not be able to create a VMFS5 datastore on a LUN presented from the array. If this is the case, the following error will appear: An error occurred during host configuration. The error is a result of the ATS (VAAI) portion of the Symmetrix Enginuity Microcode (VMAX 5875.x) preventing a new datastore on a previously unwritten LUN.

The workaround is to disable Hardware Accelerated Locking on the ESXi host, create the VMFS5 datastore and then re-enable Hardware Accelerated Locking on the host. Chad Sakac from EMC has further information about the issue and the solution on his blog post here.

Get notification of these blogs postings and more VMware Storage information by following me on Twitter: @CormacJHogan

8 comments
    • Yes – quoting directly from Chad’s blog, enginuity fix 62286 addresses this issue and is available via Enginuity Pack at 5875. Fix 62286 is also available in the GA release of 5875.267.201 and is in all GA versions of Enginuity 5876.

  1. Great Article, quick question regarding the multipathing, I’ve seen a best practise from EMC and NetApp that for Active Active controllers a fixed setting is the preferred setting. With EMC this is changed now to RR. But what with other vendors such as NetApp, GreenBytes etc.

    secondly, in the new webclient can you change the multipath setting?

    Thanks

    Harold

    • Hi Harold, to the best of my knowledge, this was an EMC only change. I don’t believe the others have changed their default PSP. And yes, you can change multipath settings through the new web client, but I think that if you need to change the default PSP associated with an SATP, you have to do this via the CLI.

  2. Hi Cormac, I’ve noticed the iops=1000 is still there while some storage vendors recommend iops=1 to evenly distribute IO’s on the SP’s. what’s your take on this?
    Thx,
    Didier

    • Hi Didier,
      I would follow the recommendations from the storage array vendor. One assumes that they will have done some testing and found a value which best suits their array.

  3. Just a heads up here – although you have seen the defaults change, it is always in your best bet to first verify with the vendor recommendations before accepting the defaults! In the case of the VNX array, the VMware HCL shows the PSP set to FIXED in many but one configuration!

    http://www.vmware.com/resources/compatibility/detail.php?deviceCategory=san&productid=19431&releaseid=201&deviceCategory=san&partner=30&keyword=VNX&isSVA=1&page=3&display_interval=10&sortColumn=Partner&sortOrder=Asc

Comments are closed.