Simple test for Docker Swarm functionality with Photon OS
After highlighting how easy it is to run docker swarm in Photon OS, I had a follow on question on how easy it would be to test the functionality. Just to recap, the only additional step you need to get Docker Swarm running on Photon OS was to open port 2377 on the master node. After that, you simply initialize the master, and all the other nodes/VMs are added as swarm workers. You might be wondering if you need to do a bunch of other stuff in iptables for docker, but the answer is no (for this relatively simple test anyway). Once you enable docker on Photon OS, a whole bunch of docker chain rules are added automatically. So let’s look at the following example of a single master with two worker nodes (3 different Photon OS VMs). What I plan to do is create a web server service using Nginx, and scale it out from 1 to 2 to 3 replicas across my swarm cluster. Let’s kick it off.
Let’s start with a look at the iptables on one of my Photon OS VMs that will become a Docker Swarm worker. This is before I enable and start docker. You can see the difference for yourself.
root@photon-worker2 [ ~ ]# iptables --list Chain INPUT (policy DROP) target prot opt source destination ACCEPT all -- anywhere anywhere ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ACCEPT tcp -- anywhere anywhere tcp dpt:ssh Chain FORWARD (policy DROP) target prot opt source destination Chain OUTPUT (policy DROP) target prot opt source destination ACCEPT all -- anywhere anywhere root@photon-worker2 [ ~ ]# systemctl enable docker Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /lib/systemd/system/docker.service. root@photon-worker2 [ ~ ]# systemctl start docker root@photon-worker2 [ ~ ]#iptables --list Chain INPUT (policy DROP) target prot opt source destination ACCEPT all -- anywhere anywhere ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ACCEPT tcp -- anywhere anywhere tcp dpt:ssh Chain FORWARD (policy DROP) target prot opt source destination DOCKER-USER all -- anywhere anywhere DOCKER-ISOLATION all -- anywhere anywhere ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED DOCKER all -- anywhere anywhere ACCEPT all -- anywhere anywhere ACCEPT all -- anywhere anywhere Chain OUTPUT (policy DROP) target prot opt source destination ACCEPT all -- anywhere anywhere Chain DOCKER (1 references) target prot opt source destination Chain DOCKER-ISOLATION (1 references) target prot opt source destination RETURN all -- anywhere anywhere Chain DOCKER-USER (1 references) target prot opt source destination RETURN all -- anywhere anywhere root@photon-worker2 [ ~ ]#
OK – let’s flip over to the master node/VM, and open that port I mentioned and initialize the Docker Swarm.
root@photon-master [ ~ ]# docker info Containers: 0 Running: 0 Paused: 0 Stopped: 0 Images: 4 Server Version: 17.06.0-ce Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4 init version: 949e6fa Security Options: seccomp Profile: default Kernel Version: 4.9.47-2.ph2-esx Operating System: VMware Photon OS/Linux OSType: linux Architecture: x86_64 CPUs: 1 Total Memory: 1.958GiB Name: photon-master ID: IHWT:3LUB:TTMT:GZF2:YHI3:6RAE:AN7S:5SDY:COQA:2UNZ:4GAH:5AXW Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false No Proxy: 10.27.51.47 Registry: https://index.docker.io/v1/Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false root@photon-master [ ~ ]#iptables -A INPUT -p tcp --dport 2377 -j ACCEPT root@photon-master [ ~ ]#iptables --list Chain INPUT (policy DROP) target prot opt source destination ACCEPT all -- anywhere anywhere ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ACCEPT tcp -- anywhere anywhere tcp dpt:ssh ACCEPT tcp -- anywhere anywhere tcp dpt:2377 Chain FORWARD (policy DROP) target prot opt source destination DOCKER-USER all -- anywhere anywhere DOCKER-ISOLATION all -- anywhere anywhere ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED DOCKER all -- anywhere anywhere ACCEPT all -- anywhere anywhere ACCEPT all -- anywhere anywhere ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED DOCKER all -- anywhere anywhere ACCEPT all -- anywhere anywhere DROP all -- anywhere anywhere Chain OUTPUT (policy DROP) target prot opt source destination ACCEPT all -- anywhere anywhere Chain DOCKER (2 references) target prot opt source destination Chain DOCKER-ISOLATION (1 references) target prot opt source destination DROP all -- anywhere anywhere DROP all -- anywhere anywhere RETURN all -- anywhere anywhere Chain DOCKER-USER (1 references) target prot opt source destination RETURN all -- anywhere anywhere root@photon-master [ ~ ]# root@photon-master [ ~ ]# docker swarm init Swarm initialized: current node (jxk3918ei5sj9h0dso0j4jl34) is now a manager. To add a worker to this swarm, run the following command: docker swarm join --token SWMTKN-1-5r9kckkp9kcxmwcwyvrtczmk2e3bakp3izounhk19icc3gkxf5-cj26dynkq3z20wfgm9hyci6ed 10.27.51.47:2377 To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions. root@photon-master [ ~ ]#
Now you might like to save this permanently. This can be done by using the iptables-save command, and storing the output in ip4save, which is located in /etc/systemd/scripts. When iptables restarts, it will read the saved iptables rules in from this file.
Let’s now join both of our workers to the swarm cluster.
root@photon-worker[ ~ ]# docker swarm join --token SWMTKN-1-5r9kckkp9kcxmwcwyvrtczmk2e3bakp3izounhk19icc3gkxf5-cj26dynkq3z20wfgm9hyci6ed 10.27.51.47:2377 This node joined a swarm as a worker. root@photon-worker[ ~ ]# docker info Containers: 0 Running: 0 Paused: 0 Stopped: 0 Images: 17 Server Version: 17.06.0-ce Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog Swarm: active NodeID: u3rcmqv678hbmrz9vp5uycs1v Is Manager: false Node Address: 10.27.51.17 Manager Addresses: 10.27.51.47:2377 Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4 init version: 949e6fa Security Options: seccomp Profile: default Kernel Version: 4.9.60-1.ph2-esx Operating System: VMware Photon OS/Linux OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 7.792GiB Name: photon-worker ID: R7DL:MSZ4:MCAE:SKFS:2HN3:ZZOV:2TJC:T757:H5DM:DRWV:QC6P:YE2R Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false No Proxy: 10.27.51.47 Registry: https://index.docker.io/v1/Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false root@photon-worker [ ~ ]# root@photon-worker2 [ ~ ]# docker swarm join --token SWMTKN-1-5r9kckkp9kcxmwcwyvrtczmk2e3bakp3izounhk19icc3gkxf5-cj26dynkq3z20wfgm9hyci6ed 10.27.51.47:2377 This node joined a swarm as a worker. root@photon-worker2[ ~ ]# docker info Containers: 0 Running: 0 Paused: 0 Stopped: 0 Images: 0 Server Version: 17.06.0-ce Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog Swarm: active NodeID: pg8siha3bpwwxcjn395p2fshi Is Manager: false Node Address: 10.27.51.145 Manager Addresses: 10.27.51.47:2377 Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4 init version: 949e6fa Security Options: seccomp Profile: default Kernel Version: 4.9.47-2.ph2-esx Operating System: VMware Photon OS/Linux OSType: linux Architecture: x86_64 CPUs: 1 Total Memory: 1.958GiB Name: photon-worker2 ID: OP7U:3IGC:FA44:H42Q:ZM3J:7PA6:26FX:7OPK:S6SQ:34AI:DF7F:XR3Q Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false root@photon-worker2 [ ~ ]#
Looks like both nodes are active in the cluster. Let’s check it from the master. We should now see 3 nodes in the cluster.
root@photon-master [ ~ ]# docker info Containers: 0 Running: 0 Paused: 0 Stopped: 0 Images: 4 Server Version: 17.06.0-ce Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog Swarm: active NodeID: jxk3918ei5sj9h0dso0j4jl34 Is Manager: true ClusterID: w978k88h8fhk149i8w9lyvr1w Managers: 1 Nodes: 3 Orchestration: Task History Retention Limit: 5 Raft: Snapshot Interval: 10000 Number of Old Snapshots to Retain: 0 Heartbeat Tick: 1 Election Tick: 3 Dispatcher: Heartbeat Period: 5 seconds CA Configuration: Expiry Duration: 3 months Force Rotate: 0 Root Rotation In Progress: false Node Address: 10.27.51.47 Manager Addresses: 10.27.51.47:2377 Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4 init version: 949e6fa Security Options: seccomp Profile: default Kernel Version: 4.9.47-2.ph2-esx Operating System: VMware Photon OS/Linux OSType: linux Architecture: x86_64 CPUs: 1 Total Memory: 1.958GiB Name: photon-master ID: IHWT:3LUB:TTMT:GZF2:YHI3:6RAE:AN7S:5SDY:COQA:2UNZ:4GAH:5AXW Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): false No Proxy: 10.27.51.47 Registry: https://index.docker.io/v1/Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false root@photon-master [ ~ ]#
Everything looks good. Let’s now create our service to run in our Swarm. Like I said, I’m choosing Nginx as my web server, and I am mapping container port 80 to port 8080 on my VM/node. Initially, I will start with just a single replica, and I will give the service a name “web”:
root@photon-master [ ~ ]# docker service create --replicas 1 -p 8080:80 --name web nginx omv53xwfjzl1oicbwmylrto7y Since --detach=false was not specified, tasks will be created in the background. In a future release, --detach=false will become the default.
And lets see where the single replica of the container is running:
root@photon-master [ ~ ]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS omv53xwfjzl1 web replicated 1/1 nginx:latest *:8080->80/tcp root@photon-master[ ~ ]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES root@photon-master [ ~ ]# root@photon-worker2[ ~ ]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES root@photon-worker2 [ ~ ]# root@photon-worker[ ~ ]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 94c325a77e0c nginx:latest "nginx -g 'daemon ..." 18 seconds ago Up 18 seconds 80/tcp web.1.mtk6rsovie5mi5w0m46y00txs
Looks like it is running on one of the workers. Let’s see if we can access the welcome page from that worker.
root@photon-worker [ ~ ]# curl 127.0.0.1:8080 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html> root@photon-worker [ ~ ]#
OK – that was successful. Let’s scale it out with another replica.
root@photon-master[ ~ ]# docker service scale web=2 web scaled to 2 root@photon-master[ ~ ]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS omv53xwfjzl1 web replicated 2/2 nginx:latest *:8080->80/tcp
This one seems to have started on the master. Let’s see if we can access the web server from that node.
root@photon-master [ ~ ]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 806b6eee2e89 nginx:latest "nginx -g 'daemon ..." 6 seconds ago Up 5 seconds 80/tcp web.2.seg6tb6fayvv0q4sdgnvqmz9v root@photon-master [ ~ ]# curl 127.0.0.1:8080 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>
Looks good. Let’s scale the service out one more time, so we have 3 replicas. And as before, lets see where it ends up running, and if it can be accessed from the node.
root@photon-master[ ~ ]# docker service scale web=3 web scaled to 3 root@photon-master [ ~ ]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS omv53xwfjzl1 web replicated 3/3 nginx:latest *:8080->80/tcp root@photon-master [ ~ ]# root@photon-worker2[ ~ ]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 8d9df2a11660 nginx:latest "nginx -g 'daemon ..." 52 seconds ago Up 52 seconds 80/tcp web.3.ykq6lrfus4mgwv744a5uwq5pn root@photon-worker2 [ ~ ]# curl 127.0.0.1:8080 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>
So this time, it was started on the final worker node. Now we have 3 replicas of the service running.
Finally, lets examine the service in more detail from the master:
root@photon-master [ ~ ]# docker service ps web ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS mtk6rsovie5m web.1 nginx:latest photon-worker Running Running 5 minutes ago seg6tb6fayvv web.2 nginx:latest photon-master Running Running 3 minutes ago ykq6lrfus4mg web.3 nginx:latest photon-worker2 Running Running 2 minutes ago root@photon-master [ ~ ]#
Great. So hopefully that gives you some idea on how you can get started with a simply application to test Docker Swarm running on Photon OS.
