NFS Best Practices – Part 4: Sizing Considerations

Welcome to the next installment of NFS Best Practices. In this fourth and final best practice section on NFS, I asked a number of our major NAS storage partners some sizing questions. The questions basically fell into 3 different categories:

1. Do you have a recommended NFS volume size/sweet-spot recommendation?
2. Do you have a volume block size recommendation that should be configured on the array?
3. Do you have a recommended number of VMs per NFS datastore?

In fact, the responses from the vendors were all pretty similar. Let’s take a look at what they had to say.

Maximum volume Size/Sweet Spot

Many of you will have observed the following statement in the VMware Configuration Maximums Guide – Contact your storage array vendor or NFS server vendor for information about the maximum NFS volume size. The purpose of this first question was to see if there was a volume size which worked well which the partners agreed on. In fact, all partners responded saying that there was no performance gain or degradation depending on the volume size – customers could build NFS volumes of any size, so long as it was below the array vendor’s supported maximum. This can be up in the hundreds of TB, but consensus appears to be that the majority of NFS datastores are in the 10s of TB in size. The datastores sizes vary quite a bit from customer to customer.

Sizing of volumes is typically proportional to the number of VMs you wish to deploy, plus snapshots/changed blocks for backup purposes. Another consideration is the fact that many arrays now have deduplication and compression features, which will also reduce capacity requirements.

Recommended Block Size

From the vendors that I spoke to, this parameter is not tuneable for the most part. Some vendors have it hard set to 4KB and others have it hard set to 8KB. Block sizes are typically a multiple of 4KB. These align nicely with the 4KB grain size used in VMware’s virtual disk format (VMDK). For those vendors who have it set to 8KB, the recommendation is to format the volumes in the Guest OS to a matching 8KB block size for optimal performance. The one vendor who did have a tuneable block size had different recommendations depending on the workload – 4KB for random workloads and 32KB for sequential workloads. Since having multiple virtual machines sharing an NFS datastore is going to result in random workloads, you probably would be using 4KB. In this area, it is best to speak to your storage array vendor to get vendor specific advice. The questions one should ask are:

1. What is the volume block size on the array?
2. Is it tuneable?
3. If so, what should I tune it to? (Be sure to explain that the datastore will be used by VMs, so the workload will be random for the most part)
4. Are there any considerations when formatting the volumes in the Guest OS?

RSIZE & WSIZE

I thought it useful just to add a note about the RSize and WSize parameters when discussing sizing. These parameters, defined at mount time, define the I/O chunk transfer sizes between the host and the target. These sizes are not tuneable. However, feedback from the partners suggest that the current size of 64KB meets their requirements. While some storage arrays can now support much larger Wsize & Rsize values (up to 1MB), there is still no way to change these settings in vSphere at the time of writing.

 Recommended number of VMs per NFS datastore

This was again met with a pretty consistent response from all vendors. The number of virtual machines which can run on a single datastore is directly proportional to the infrastructure and the workloads running in the VMs, i.e. hundreds of low I/O VMs versus a few very intensive I/O VMs. Network congestion was an important factor, which brings us back to some recomendations which were made in part 1 of this series which dealt with network best practices for NFS. Since VMware still only supports NFS v3 over TCP/IP, one recommendation was to use more datastores presented over different interfaces so that traffic could be balanced over multiple TCP sessions.

The other major factor is related to the backup and recovery SLA. If you have one single datastore with lots of VMs, how long are you willing to wait while this is restored in the event of a failure. This is starting to become the major consideration in the ‘how many VMs per datastore’ debate.

Another consideration which a customer highlighted is the snapshot technology used by the backup product – is it using array based snapshots or is it using virtual machine snapshots. Performance considerations are needed if using virtual machine snapshots to concurrently capture point in time copies of VMs. In many cases, array based snapshots have less impact on the datastores and are more scalable when it comes to backups. There may be some array based limitations to take into considerations too – for instance, the number of snapshot copies of a VM that a customer wishes to maintain versus the number of snapshot copies an array can support. This would vary from vendor to vendor so best to check this configuration maximum with your storage array vendor.

From a vSphere snapshot perspective, refer to this KB article 1015180 for further details around snapshots and their usage. From KB article 1025279, VMs can support up to 32 snapshots in a chain, but VMware recommends that you use only 2-3 snapshots in a chain, and to use no single snapshot for more than 24-72 hours.

Links to previous NFS Best Practices parts

• Part 1
• Part 2
• Part 3

Get notification of these blogs postings and more VMware Storage information by following me on Twitter: @CormacJHogan