This is an issue which has caught a number of customers out during the Virtual SAN beta, so will probably catch some folks out when the product goes live too. One of the requirements for Virtual SAN (VSAN) is to allow multicast traffic on the VSAN network between the ESXi host participating in the VSAN Cluster. However, as per our engineering lead on VSAN, multicast is only used for relatively infrequent metadata operations. For example, object creation, change in object status after a failure and publication of statistics such as a significant change of free disk space (the publication of statistics is throttled so that only significant changes will cause an update, so these are also very infrequent events).
So how does this lack of multicast support on the network manifest itself? Well, what you will see after enabling VSAN on the cluster is that the network status is shown in a misconfigured state (Misconfiguration detected), even though you can ping/vmkping all the VSAN interfaces on all the hosts:
How do you resolve it? Well, a number of our VSAN beta customers discussed some options on the community, and these were the recommendations:
- Option 1 – Disable IGMP Snooping. Now this will allow all multicast traffic through, but if the only traffic is VSAN, then this should be a negligible amount of traffic and should be safe to use.
- Option 2 - Configure IGMP snooping querier. If there is other multicast traffic and you are concerned that disabling IGMP snooping might open the network up to a flood of multicast traffic, then this is a preferred option. Cisco detail how to do this here.
Customers who ran into this situation stated that both methods worked for them.
Get notification of these blogs postings and more VMware Storage information by following me on Twitter: @CormacJHogan