Revisiting persistent storage with vSphere Integrated Containers
I’ve been getting back into doing a bit of testing with vSphere Integrated Containers 1.1 (VIC for short) in my lab. One of the things that I am very interested in revisiting is how to do persistence of data with VIC and “Containers as VMs”. I did some work on this in the past, but a lot has changed since I last looked at it (which was VIC v0.4.0). In this post, we’ll download a nginx web server image and start it up. We’ll look at how you can make changes to the web server while it is running, but how these are lost when the container is stopped. We will then create a volume, move the nginx web server files to our volume, and then restart the container, specifying our volume as the source for the web server files.
Let’s begin with a straight forward nginx deployment. In this example, I am going to launch it in interactive mode, and drop into the bash shell so we can look around. If you want to know more about getting started with VIC v1.1 and deploying the Virtual Container Host (VCH) with the docker endpoint, check out this post here. At this point, my VCH is already deployed, so I’m just going to continue with deploying the nginx container:
E:\vic> docker -H 10.27.51.39:2376 --tls run -it -p 80:80 nginx /bin/bash Unable to find image 'nginx:latest' locally Pulling from library/nginx 36a46ebd5019: Pull complete a3ed95caeb02: Pull complete 57168433389f: Pull complete 332ec8285c50: Pull complete Digest: sha256:c15f1fb8fd55c60c72f940a76da76a5fccce2fefa0dd9b17967b9e40b0355316 Status: Downloaded newer image for library/nginx:latest root@92ff66b88fdf:/#
OK – now we are in the container. At this point the web server is not running. We can start the web server as follows:
root@92ff66b88fdf:/# service nginx start
Now if I point a browser to the IP address of the VCH (the mapping is container port 80 to VCH port 80), I will connect to the nginx landing page:
I will also start to see the following messages in the container, which also show the mapping of the nginx container network/port to the VCH network/port:192.168.0.1 - - [12/May/2017:11:11:40 +0000] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0" "-" 2017/05/12 11:11:40 [error] 230#230: *1 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 192.168.0.1, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "10.27.51.39" 192.168.0.1 - - [12/May/2017:11:11:40 +0000] "GET /favicon.ico HTTP/1.1" 404 169 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0" "-" 2017/05/12 11:11:40 [error] 230#230: *1 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 192.168.0.1, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "10.27.51.39" 192.168.0.1 - - [12/May/2017:11:11:40 +0000] "GET /favicon.ico HTTP/1.1" 404 169 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0" "-"
The next step is to make some changes to the nginx landing page. These can be found under /usr/share/nginx/html/.
root@92ff66b88fdf:/# service nginx stop root@92ff66b88fdf:/# cd /usr/share/nginx/html/ root@92ff66b88fdf:/usr/share/nginx/html# ls 50x.html index.html
Let’s make some changes to the index.html file. Instead of the “Welcome to nginx!”, let’s make it more personal. I’ll make the changes to a new file, and then overwrite the original index.html landing page.
root@92ff66b88fdf:/usr/share/nginx/html# sed -e 's/Welcome to nginx/Welcome to Cormacs nginx/' \ index.html >> new_index.html root@92ff66b88fdf:/usr/share/nginx/html# mv new_index.html index.html root@92ff66b88fdf:/usr/share/nginx/html# service nginx start
Now when I connect my browser to the VCH, I see the following (apologies for the poor grammar):
But here is the point, once I exit this container, my changes are lost. If I stop and start this application, I go back to the default landing page for nginx. So how can I used persistent volumes to make this change permanent?The answer is to create a docker volume, mount it to the nginx container and copy the landing files with the necessary changes to this mount point. On every subsequent nginx mount, we can point the nginx web server to use this new volume for its landing files. The first step is to create the volume, and then start the nginx web server, mount the volume to some other directory within the container (in this example /usr/share/nginx2) and finally copy the files over. Regarding the volume create command, the driver is called vSphere, and to specify a location and name, you need to include the –opt option. The volume store corvols also needs to be specified when the VCH is created (see the getting start post referenced earlier). The volume itself is called corvol1. All of these commands are run using a docker client, pointing to the VCH as shown here:
E:\vic> docker -H 10.27.51.39:2376 --tls volume create -d vsphere --opt VolumeStore=corvols \ --name corvol1 corvol1 E:\vic> docker -H 10.27.51.39:2376 --tls run -v corvol1:/usr/share/nginx2/ -it -p 80:80 \ nginx /bin/bash root@bdf1271cdf35:/# root@bdf1271cdf35:/# df Filesystem 1K-blocks Used Available Use% Mounted on devtmpfs 994524 0 994524 0% /dev tmpfs 1026528 0 1026528 0% /dev/shm tmpfs 1026528 0 1026528 0% /sys/fs/cgroup /dev/sda 7743120 149716 7177020 3% / tmpfs 65536 8316 57220 13% /.tether /dev/disk/by-label/759d5820c83641f7 999320 1284 929224 1% /usr/share/nginx2
At this point, we have started the nginx container, and mounted a new volume to /usr/share/nginx2. Now we can copy over the nginx landing files to our mounted volume, and make any changes needed to these files to personalize them. Once done, we can exit the container.
root@bdf1271cdf35:# cd /usr/share/nginx root@bdf1271cdf35:/usr/share/nginx# cp -r html/ ../nginx2 root@bdf1271cdf35:/usr/share/nginx# cd .. root@bdf1271cdf35:/usr/share# ls nginx2/html/ 50x.html index.html root@bdf1271cdf35:/usr/share# cd nginx2/html root@bdf1271cdf35:/usr/share/nginx2/html# sed -e 's/Welcome to nginx/Welcome to Cormacs nginx/' \ index.html >> new_index.html root@bdf1271cdf35:/usr/share/nginx2/html# mv new_index.html index.html root@bdf1271cdf35:/usr/share/nginx2/html# cd root@bdf1271cdf35:~# exit
Let’s start a new container with the nginx image once again. Without the volume, we simply end up with the default nginx landing page as before:
E:\vic> docker -H 10.27.51.39:2376 --tls run -d -p 80:80 nginx 6696d5aed9b92022cde8becd41170346573c581f493f246014dfe925144e8316
However if we launch using our new volume, which has persisted our customized landing pages, and mount that to where nginx expects to find it landing pages (/usr/share/nginx), we launch nginx with the modified landing page:
E:\vic> docker -H 10.27.51.39:2376 --tls run -v corvol1:/usr/share/nginx/ -d -p 80:80 nginx d8c49c1846f2566270fe0a78006adc58c5a9caa3dd3faf1b4bde3349cf6e9210
And now we have the new modified landing pages rather than the default ones from nginx. That’s basically it. Hopefully this example will give you a pretty good idea on how you can used persistent volumes with VIC v1.1.
Alright, this is pretty plain.
What about to make nginx to host something not so simple as a 1 html page?
How to copy a whole catalog to a volume and make it persistant?
Volume is a vmdk file int the datastore and cp command not supported at current VIC version.
Any idea?
Just so I understand correctly, would you like to be able to take an existing VMDK (for example, one that had a whole web server catalog), and convert that to a volume which can then be attached to a docker container?
If that is what you wish to do, I’ll make some queries? I’m not aware right now if this is possible, so please confirm if this is what you would like to do with the docker volume plugin.
Have not got notice about your reply.
What I ment is how to easily put a web server catalog or any other data/catalog to the persistant volume. I suppose reattach existing volume to the new VCH is possible.
I’m still not 100% sure what you want to do here. That is why I asked you to clarify in my previous update. Can you provide a detailed set of steps on the workflow you want to achieve?
On the reattaching part, volumes can be detached and attached to different “containers” for sure. For example, I could create a volume and attach it to an Ubuntu distro in container A, shutdown that container, start a new container B running a Debian distro and attach the same volume to that container. You wouldn’t be adding the volume(s) to the “VCH” as this is only providing the docker endpoint and resource pool. You would be adding them to the containers in that resource pool. I hope that makes sense.
What I’m looking for is a functional similar to “docker cp”
Ah – got it. OK, I am not aware of any way to do this in VIC, and I am not sure if this functionality is available in the VIC docker API endpoint. I am asking few folks who are closer to VIC to see what their views are on something like this, and if there is a plan to add it to the product.
You could also raise this under issues on the VIC GitHub site if you wish.
This is what I got from one of our VIC leads:
“There are a few ways to populate a volume.
One is to attach it to a temporary container and have that container pull down some data onto the mounted volume. We can do that today. The other is to push data into it using docker cp. Docker cp support will be in VIC 1.2.
We do now have NFS volume support, although that didn’t make it into 1.1. Obviously with shared RW storage, you can populate the volume from anywhere with write access at any time.”
So ‘docker cp’ is not in VIC right now, so at the moment you will have to work-around it in some way. Going forward, this functionality will be in VIC.