vSAN Erasure Coding Failure Handling

I had a very interesting question recently about how vSAN handles a failure in an object that is running with an erasure coding configuration. In the case of vSAN this is either a RAID-5 or a RAID-6. On vSAN, a RAID-5 is implemented with 3 data segments and 1 parity segment (3+1), with parity striped across all four components. RAID-6 is implemented as 4 data segments and 2 parity segments (4+2), again with the parity striped across all of the six components. So what happens when we need to continue writing to one of these objects after a component/segment has…

A closer look at EBS-backed vSAN

At VMworld 2018, we announced an initiative to use EBS, Amazon Elastic Block Store, for vSAN storage. At present vSAN is configured using the current EC2 i3 configurations, which run ESXi on bare-metal. I have seen these referred to as i3p, but my understanding is that they correlate to the i3.metal instances as shown here. The Amazon EC2 i3 instances include Non-Volatile Memory Express (NVMe) SSD-based instance storage. These are configured with 10TB of storage per host, but there are some limitations. For one, if you wish to expand on capacity, you need to add another complete EC2 i3 instance.…

Why upgrade vSAN? Here is a list of features, release by release.

I was with one of our large EMEA customers last week, and we talked quite a bit about the evolution of vSAN. This customer is already one of our larger vSAN customers with very many clusters, and many, many nodes. They have also been a great vSAN reference for us, having first deployed the initial vSAN 5.5 release. One point that hit home was that they found it was quite difficult to determine the various features and enhancements that were introduced in each vSAN release. They mentioned that having a quick reference would be useful as they could use it…

Minio S3 object store deployed as a set of VMs on vSAN

Some time back, I looked at what it would take to run a container based Minio S3 object store on top of vSAN. This involved using our vSphere Docker Volume Server (aka Project Hatchway, and the details can be found here. However, I wanted to evaluate what it would take to scale out the Minio S3 object store on top of vSAN, paying particular attention to features like distribution and availability, and to examine the various data services that can be provided by both vSAN and Minio. I also wanted to take advantage of the new host-pinning feature in vSAN…

Which policy changes can trigger a rebuild on vSAN?

Some time ago, I wrote about which policy changes can trigger a rebuild of an object. This came up again recently, as it was something that Duncan and I covered in our VMworld 2017 session on top 10 vSAN considerations. In the original post (which is over 3 years old now), I highlighted items like increasing the stripe width, growing the read cache reservation (relevant only to hybrid vSAN) and changing FTT when the read cache reservation is non-zero (again only relevant to hybrid vSAN) which led to a rebuild of the object (or components within the object). The other…

A closer look at Minio S3 running on vSAN

While we are always looking at what other data services vSAN could provide natively, at the present moment, there is no native way to host S3 compatible storage on vSAN. After seeing the question about creating an S3 object store on vSAN raised a few times now, I looked into what it would take to have an S3 compatible store running on vSAN. A possible solution, namely Minio, was brought to my attention. While this is by no means an endorsement of Minio, I will admit that it was comparatively easy to get it deployed. Since the Minio Object Store…

Erasure Coding and Quorum on vSAN

I was looking at the layout of RAID-5 object configuration the other day, and while these objects were deployed on vSAN with 4 components, something caught my eye. It wasn’t the fact that there were 4 components, which is what one would expect since we implement RAID-5 as a 3+1, i.e. 3 data segments and 1 parity segment. No, what caught my eye was that one of the components had a different vote count. Now, RAID-5 and RAID-6 erasure coding configurations are not the same as RAID-1. With RAID-1, we deploy multiple copies of the data depending on how many…