Storage for containers with VMware? You got it!

Last week during a visit to VMware headquarters in Palo Alto, I had the opportunity to catch up with our engineering team who are responsible for developing storage solutions for Docker and Kubernetes running on vSphere. I have written about our Docker volume driver for vSphere and Kubernetes on vSphere already, but it’s been a while since I caught up with the team, and obviously more and more enhancements are being added all the time. I thought it might be useful to share the improvements with you here. There also seems to be some concerns raised about the availability of reliable, scalable, and persistent storage for containers. I’ll show you how VMware are solving this.

Let’s begin with the Docker volume driver for vSphere, which has since been renamed to vSphere Docker Volume Service. This service is integrated with the Docker Volume Plugin framework. Docker users are now able consume vSphere Storage (vSAN, VMFS, NFS) for stateful containers using Docker. VMs can run Docker and this service to create volumes as VMDKs on vSphere datastores. In a recent techtarget article that discusses what enterprises need before adopting containers, a lack of storage features, available to VMs, but unavailable to containers was quoted as a major blocker. Well, with the vSphere Docker Volume Service, this is no longer an obstacle. Since the docker volume is deployed as a VMDK on the datastore, vSphere treats it like any other VMDK. The most recent release of this service also includes some multi-tenancy improvements, which allows multiple users to create docker volumes on shared vSphere storage. Docker hosts (e.g. virtual machines) can be assigned privileges (e.g. create, read, write) on a per datastore basis, and resource consumption limits (such as maximum volume size) can also be specified. At present multi-tenancy is only available from within the same ESXi host, but plans are afoot to provide multi-tenancy across multiple ESXi hosts at the vCenter server level. There is an excellent set of demos and associated documentation on github.

From a Kubernetes (K8S) perspective, the previous time I wrote about it we were using the older “kube-up/kube-down” mechanism to deploy a K8S cluster running on vSphere. While it was still relatively quick to deploy, the mechanism could be problematic at the best of times. So much so that a new mechanism called “Kubernetes-Anywhere” was created, and the older “kube-up” mechanism has now been deprecated. Our team has included support for “Kubernetes-Anywhere” for simpler, more intuitive deployments of Kubernetes on vSphere. Included with this is the vSphere Cloud Provider for Kubernetes.  Kubernetes users, just like docker users with our docker volume service, are now able to deploy on vSphere and consume vSphere Storage (vSAN, VMFS, NFS) with the vSphere Cloud Provider. The provider supports Persistent Volumes and Storage Classes (e.g. thin, thick, eagerzeroed thick) on vSphere storage. Many of the storage features available to VMs are now available in Kubernetes when deployed on vSphere storage.

VMware has been doing storage for many years now.  Fiber Channel, iSCSI and NFS have been available on ESXi for a long time. Many more storage enhancements continue to be added. More recently we have introduced VVols (Virtual Volumes), which led to a great talk from HPE at VMworld last year on how the granularity of VVols and policy driven storage is ideal for container storage. Even our very own vSAN product provides an exceptional level of performance and availability for container storage, allowing users to associate a storage policy with a container for dynamic volume creation, rather than having to carve out volumes in advance. You can consider vSAN as HCI for Containers, which can be scaled up (with more storage) or scaled out (with more hosts). vSAN is a proven storage solution, having been GA for 3 years and with over 7,000 or so customers. A container volume created on vSAN is immediately highly available, and can tolerate multiple failures in the hyperconverged infrastructure (HCI) on which it is deployed. With vSAN, enterprise grade features such as policy based management, QOS, and data reliability are all available at container volume granularity. VMware has “virtualized” container volumes, allowing any vSphere Storage to be consumed using Docker or K8S.

On another note, Docker recently acquired Infinit, a storage startup. Containers may not require persistence, but their volume almost certainly do. In the techtarget article, one of the observations made by Mark Davis, ClusterHQ’s former CEO (creators of flocker), was as follows: “I don’t believe that anyone, and I don’t care how brilliant you are, can build an enterprise-grade reliable scalable distributed file system with half a dozen people in a couple of years,” Davis said. “That’s not a criticism of them, it’s just that’s how long it takes.” I personally don’t know how good or robust the Infinit technology is, but I’ve seen first hand how much hard work and effort goes into getting a reliable and stable storage product to market.  I read that Docker plan to open source the Infinit technology sometime in 2017. But the point I want to make in this post is that there is no need to wait. VMware, through the vSphere Docker Volume Server and the vSphere Cloud Provider for Kubernetes, enabled your containers to consume enterprise-grade, reliable, scalable vSphere storage right now!

Lastly, I have some questions for my readers. Are many of you using containers? Do you run them in VMs on vSphere? Do you use persistent storage? How do you consume this persistent storage? Is there anything we can add to the above functionality to make your life easier? We’d love to hear from you.