First, this has been designed specifically for docker version 1.12. So the first thing you will have to do is to make sure that your docker is at this latest version. For most distros, this is quite a simple thing to do. But since I predominantly use our Photon OS distro, which ships with docker version 1.11 currently, there are a few additional steps to consider. To update the version of docker on Photon OS, you can use the following steps:
Step 1. Make a backup of /etc/yum.repos.d/photon-dev.repo and then add the following stanza of text to the original:
[photon-dev] name=VMware Photon Linux Dev(x86_64) baseurl=https://dl.bintray.com/vmware/photon_dev_$basearch gpgkey=file:///etc/pki/rpm-gpg/VMWARE-RPM-GPG-KEY gpgcheck=1 enabled=1 skip_if_unavailable=True
Step 2. Run the following command to update docker:
tdnf install --refresh docker
Step 3. Make a backup of /usr/lib/systemd/system/docker-containerd.service and the edit the original to modify the “ExecStart” section. Replace it with the following:
ExecStart=/usr/bin/docker-containerd --listen
unix:///run/containerd.sock --runtime /usr/bin/docker-runc
--shim /usr/bin/docker-containerd-shim
Step 4. Run the following commands:
systemctl daemon-reload systemctl restart docker
Step 5. Verify that the docker version is at v1.12 and is functioning:
docker –v docker ps docker volume ls
OK. Now I can proceed with the installation of docker volume driver version 0.7.
The installation process is still the same as before. There are two components; the VIB and the RPM. You need to install the VIB on the ESXi host, and you need to install the Guest OS component as well. The binaries are available for download on github to get you started quickly. But you can also built it yourself. This time I pulled down the latest build from github to my Photon OS VM using “git clone”, and I built the components on my Photon OS. If you want to do the same on your Photon OS, here are the steps:
Step 1. Install git using the following command
tdnf install git
Step 2. Clone the docker volume driver repo
git clone https://github.com/vmware/docker-volume-vsphere.git
Step 3. Install the make utility
tdnf install make
Step 4. Make sure docker is running in Photon OS. The build steps uses docker.
systemctl start docker systemctl enable docker
Step 5. Build the code
cd docker-volume-vsphere/ make
Step 6. Install the RPM on Photon OS
cd build
rpm -ivh docker-volume-vsphere-0.7.8d42baa-1.x86_64.rpm
Step 7. restart docker
systemctl restart docker
Step 8. Copy the VIB to the ESXi host on which your Photon OS resides, and install it. The following command is run from the ESXi host, not the Photon OS, after the VIB was copied from the build folder on my Photon OS guest to the /scratch/downloads on the ESXi host:
esxcli software vib install -d \
/scratch/downloads/vmware-esx-vmdkops-0.7.8d42baa.zip \
--no-sig-check -f
Now we can look at some of the enhancements in this newer version. First, there are some new field in the volume view from the ESXi host. Here is how things used to look in the previous version of the driver:
There are some nice new enhancements to the “docker volume create” commands from within the Guest OS too. There is now a much nicer and more comprehensive help output if you make a typo/mistake. For example:
root@photon [ ~ ]# docker volume create -d vmdk \
-o bad_opt=bad_val
Error response from daemon: create
.....:
Valid options and defaults:
[('size', '100mb'),
('vsan-policy-name', '[VSAN default]'),
('diskformat', 'thin'),
('attach_as', 'independent_persistent')]
root@photon [ ~ ]#
There are a few things to highlight here.
- You can now select which filesystem to place on the volume, so long as the Guest OS supports it. This defaults to “ext4”.
- You can create Thin, LazyZeroedThick or EagerZeroedThick volumes. This defaults to thin.
- You can specify a VSAN policy, which I highlighted in a previous post here. If you do not specify a policy when creating volumes on VSAN, it uses the default policy.
- You can attach volumes as independent persistent, meaning that they can now be snapshot’ed for the purposes of backup. More on disk formats can be found here. The default is independent_persistent, meaning that no snapshot is created during backups, meaning that this volume is not backed up.
So lets create some new volumes for containers from my Photon OS Guest OS. I will create 3 volumes in total, with different specifications.
Volume 1 – Create a volume that is on the same datastore as my Photon OS VM, with a size of 10GB (which is a VSAN datastore by the way). Everything is default.
root@photon [ / ]# docker volume create --driver=vmdk \ --name=testvol1 -o size=10gb testvol1
Volume 2 – Create a volume on my NFS array with a size of 10GB, but make it read-only.
root@photon [ / ]# docker volume create --driver=vmdk \ --name=testvol2@isilion-nfs-01 -o size=10gb -o access=read-only testvol2@isilion-nfs-01
Volume 3 – Create a volume on my local VMFS datastore, but make it persistent rather than independent-persistent, so I can snapshot it for backup purposes.
root@photon [ / ]# docker volume create --driver=vmdk \ --name=testvol3@esxi-hp-08-local -o size=10gb \ -o attach-as=persistent testvol3@esxi-hp-08-local
Lets take a look at the docker volume listing:
root@photon [ / ]# docker volume ls DRIVER VOLUME NAME . . vmdk testvol1 vmdk testvol2@isilion-nfs-01 vmdk testvol3@esxi-hp-08-local . root@photon [ / ]#
Again, in this version of the docker volume driver, testvol1 is not shown with an “@datastore” as it is on the same datastore as the Guest OS. We can use the docker inspect command to show the attributes of a particular volume:
root@photon [ / ]# docker volume inspect testvol2@isilion-nfs-01 [ { "Name": "testvol2@isilion-nfs-01", "Driver": "vmdk", "Mountpoint": "/mnt/vmdk/testvol2@isilion-nfs-01", "Status": { "access": "read-only", "attach-as": "independent_persistent", "capacity": { "allocated": "164MB", "size": "10GB" }, "created": "Thu Oct 6 14:21:32 2016", "created by VM": "Photon-DVD4V", "datastore": "isilion-nfs-01", "diskformat": "thin", "fstype": "ext4", "status": "detached" }, "Labels": {}, "Scope": "global" } ]
Finally we can get an overview of all the volumes from an ESXi host perspective.
This is what happens if you attempt to create something on the read-only filesystem.
root@photon [ / ]# docker run -it --rm -v testvol2@isilion-nfs-01:/testvol2 busybox sh / # df Filesystem 1K-blocks Used Available Use% Mounted on overlay 8122788 2657456 5029676 35% / tmpfs 1026644 0 1026644 0% /dev tmpfs 1026644 0 1026644 0% /sys/fs/cgroup /dev/disk/by-path/pci-0000:0b:00.0-scsi-0:0:0:0 10190136 23028 9626436 0% /testvol2 /dev/root 8122788 2657456 5029676 35% /etc/resolv.conf /dev/root 8122788 2657456 5029676 35% /etc/hostname /dev/root 8122788 2657456 5029676 35% /etc/hosts shm 65536 0 65536 0% /dev/shm tmpfs 1026644 0 1026644 0% /proc/sched_debug / # cd /testvol2/ /testvol2 # ls lost+found /testvol2 # mkdir zzz mkdir: can't create directory 'zzz': Read-only file system /testvol2 #
So lots of really great improvements as you can see. If you want to build persistent storage around containers from VMs which are running on vSphere, the docker volume driver for vSphere is the perfect way to do it. You can get all the information you need from the github page – https://vmware.github.io/docker-volume-vsphere/ – and it is also the best place to get help/assistance, or indeed contribute to the project. The team would only be too happy to help.
If you are VMworld 2016 in Europe, check out the Storage for Cloud Native Applications session – STO7831 – with Mark Sterin. This takes place on Tuesday, Oct 18, 5:00 p.m. – 6:00 p.m. Mark will demonstrate a lot of the goodness I am talking about here, and some other cool storage/container activities taking place at VMware.