Site icon CormacHogan.com

Simple test for Docker Swarm functionality with Photon OS

Cormac

After highlighting how easy it is to run docker swarm in Photon OS, I had a follow on question on how easy it would be to test the functionality. Just to recap, the only additional step you need to get Docker Swarm running on Photon OS was to open port 2377 on the master node. After that, you simply initialize the master, and all the other nodes/VMs are added as swarm workers. You might be wondering if you need to do a bunch of other stuff in iptables for docker, but the answer is no (for this relatively simple test anyway). Once you enable docker on Photon OS, a whole bunch of docker chain rules are added automatically.  So let’s look at the following example of a single master with two worker nodes (3 different Photon OS VMs). What I plan to do is create a web server service using Nginx, and scale it out from 1 to 2 to 3 replicas across my swarm cluster. Let’s kick it off.

Let’s start with a look at the iptables on one of my Photon OS VMs that will become a Docker Swarm worker. This is before I enable and start docker. You can see the difference for yourself.
root@photon-worker2 [ ~ ]# iptables --list
Chain INPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh
Chain FORWARD (policy DROP)
target     prot opt source               destination
Chain OUTPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere

root@photon-worker2 [ ~ ]# systemctl enable docker
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /lib/systemd/system/docker.service.

root@photon-worker2 [ ~ ]# systemctl start docker

root@photon-worker2 [ ~ ]#iptables --list
Chain INPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh
Chain FORWARD (policy DROP)
target     prot opt source               destination
DOCKER-USER  all  --  anywhere             anywhere
DOCKER-ISOLATION  all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
Chain OUTPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
Chain DOCKER (1 references)
target     prot opt source               destination
Chain DOCKER-ISOLATION (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere
Chain DOCKER-USER (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere
root@photon-worker2 [ ~ ]#
OK – let’s flip over to the master node/VM, and open that port I mentioned and initialize the Docker Swarm.
root@photon-master [ ~ ]# docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 4
Server Version: 17.06.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a
runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4
init version: 949e6fa
Security Options:
seccomp
  Profile: default
Kernel Version: 4.9.47-2.ph2-esx
Operating System: VMware Photon OS/Linux
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.958GiB
Name: photon-master
ID: IHWT:3LUB:TTMT:GZF2:YHI3:6RAE:AN7S:5SDY:COQA:2UNZ:4GAH:5AXW
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
No Proxy: 10.27.51.47
Registry: https://index.docker.io/v1/Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

root@photon-master [ ~ ]#iptables -A INPUT -p tcp --dport 2377 -j ACCEPT
root@photon-master [ ~ ]#iptables --list
Chain INPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:2377
Chain FORWARD (policy DROP)
target     prot opt source               destination
DOCKER-USER  all  --  anywhere             anywhere
DOCKER-ISOLATION  all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
DOCKER     all  --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere
Chain OUTPUT (policy DROP)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
Chain DOCKER (2 references)
target     prot opt source               destination
Chain DOCKER-ISOLATION (1 references)
target     prot opt source               destination
DROP       all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere
RETURN     all  --  anywhere             anywhere
Chain DOCKER-USER (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere
root@photon-master [ ~ ]#

root@photon-master [ ~ ]# docker swarm init
Swarm initialized: current node (jxk3918ei5sj9h0dso0j4jl34) is now a manager.
To add a worker to this swarm, run the following command:
    docker swarm join --token SWMTKN-1-5r9kckkp9kcxmwcwyvrtczmk2e3bakp3izounhk19icc3gkxf5-cj26dynkq3z20wfgm9hyci6ed 10.27.51.47:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
root@photon-master [ ~ ]#
Now you might like to save this permanently. This can be done by using the iptables-save command, and storing the output in ip4save, which is located in /etc/systemd/scripts. When iptables restarts, it will read the saved iptables rules in from this file.
Let’s now join both of our workers to the swarm cluster.
root@photon-worker[ ~ ]# docker swarm join --token SWMTKN-1-5r9kckkp9kcxmwcwyvrtczmk2e3bakp3izounhk19icc3gkxf5-cj26dynkq3z20wfgm9hyci6ed 10.27.51.47:2377
This node joined a swarm as a worker.

root@photon-worker[ ~ ]# docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 17
Server Version: 17.06.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: active
NodeID: u3rcmqv678hbmrz9vp5uycs1v
Is Manager: false
Node Address: 10.27.51.17
Manager Addresses:
  10.27.51.47:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a
runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4
init version: 949e6fa
Security Options:
seccomp
  Profile: default
Kernel Version: 4.9.60-1.ph2-esx
Operating System: VMware Photon OS/Linux
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.792GiB
Name: photon-worker
ID: R7DL:MSZ4:MCAE:SKFS:2HN3:ZZOV:2TJC:T757:H5DM:DRWV:QC6P:YE2R
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
No Proxy: 10.27.51.47
Registry: https://index.docker.io/v1/Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
root@photon-worker [ ~ ]#


root@photon-worker2 [ ~ ]# docker swarm join --token SWMTKN-1-5r9kckkp9kcxmwcwyvrtczmk2e3bakp3izounhk19icc3gkxf5-cj26dynkq3z20wfgm9hyci6ed 10.27.51.47:2377
This node joined a swarm as a worker.

root@photon-worker2[ ~ ]# docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 17.06.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: active
NodeID: pg8siha3bpwwxcjn395p2fshi
Is Manager: false
Node Address: 10.27.51.145
Manager Addresses:
  10.27.51.47:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a
runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4
init version: 949e6fa
Security Options:
seccomp
  Profile: default
Kernel Version: 4.9.47-2.ph2-esx
Operating System: VMware Photon OS/Linux
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.958GiB
Name: photon-worker2
ID: OP7U:3IGC:FA44:H42Q:ZM3J:7PA6:26FX:7OPK:S6SQ:34AI:DF7F:XR3Q
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
root@photon-worker2 [ ~ ]#
Looks like both nodes are active in the cluster. Let’s check it from the master. We should now see 3 nodes in the cluster.
root@photon-master [ ~ ]# docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 4
Server Version: 17.06.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: active
NodeID: jxk3918ei5sj9h0dso0j4jl34
Is Manager: true
ClusterID: w978k88h8fhk149i8w9lyvr1w
Managers: 1
Nodes: 3
Orchestration:
  Task History Retention Limit: 5
Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 3
Dispatcher:
  Heartbeat Period: 5 seconds
CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
Root Rotation In Progress: false
Node Address: 10.27.51.47
Manager Addresses:
  10.27.51.47:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a
runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4
init version: 949e6fa
Security Options:
seccomp
  Profile: default
Kernel Version: 4.9.47-2.ph2-esx
Operating System: VMware Photon OS/Linux
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.958GiB
Name: photon-master
ID: IHWT:3LUB:TTMT:GZF2:YHI3:6RAE:AN7S:5SDY:COQA:2UNZ:4GAH:5AXW
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
No Proxy: 10.27.51.47
Registry: https://index.docker.io/v1/Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
root@photon-master [ ~ ]#
Everything looks good. Let’s now create our service to run in our Swarm. Like I said, I’m choosing Nginx as my web server, and I am mapping container port 80 to port 8080 on my VM/node. Initially, I will start with just a single replica, and I will give the service a name “web”:
root@photon-master [ ~ ]# docker service create --replicas 1 -p 8080:80 --name web nginx
omv53xwfjzl1oicbwmylrto7y
Since --detach=false was not specified, tasks will be created in the background.
In a future release, --detach=false will become the default.
 And lets see where the single replica of the container is running:
root@photon-master [ ~ ]# docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
omv53xwfjzl1        web                 replicated          1/1                 nginx:latest        *:8080->80/tcp
root@photon-master[ ~ ]# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
root@photon-master [ ~ ]#

root@photon-worker2[ ~ ]# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
root@photon-worker2 [ ~ ]#

root@photon-worker[ ~ ]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
94c325a77e0c        nginx:latest        "nginx -g 'daemon ..."   18 seconds ago      Up 18 seconds       80/tcp              web.1.mtk6rsovie5mi5w0m46y00txs
Looks like it is running on one of the workers. Let’s see if we can access the welcome page from that worker.
root@photon-worker [ ~ ]# curl 127.0.0.1:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
root@photon-worker [ ~ ]#
OK – that was successful. Let’s scale it out with another replica.
root@photon-master[ ~ ]# docker service scale web=2
web scaled to 2
root@photon-master[ ~ ]# docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
omv53xwfjzl1        web                 replicated          2/2                 nginx:latest        *:8080->80/tcp
This one seems to have started on the master. Let’s see if we can access the web server from that node.
root@photon-master [ ~ ]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
806b6eee2e89        nginx:latest        "nginx -g 'daemon ..."   6 seconds ago       Up 5 seconds        80/tcp              web.2.seg6tb6fayvv0q4sdgnvqmz9v
root@photon-master [ ~ ]# curl 127.0.0.1:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
Looks good. Let’s scale the service out one more time, so we have 3 replicas. And as before, lets see where it ends up running, and if it can be accessed from the node.
root@photon-master[ ~ ]# docker service scale web=3
web scaled to 3
root@photon-master [ ~ ]# docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
omv53xwfjzl1        web                 replicated          3/3                 nginx:latest        *:8080->80/tcp
root@photon-master [ ~ ]#

root@photon-worker2[ ~ ]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
8d9df2a11660        nginx:latest        "nginx -g 'daemon ..."   52 seconds ago      Up 52 seconds       80/tcp              web.3.ykq6lrfus4mgwv744a5uwq5pn
root@photon-worker2 [ ~ ]# curl 127.0.0.1:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
So this time, it was started on the final worker node. Now we have 3 replicas of the service running.
Finally, lets examine the service in more detail from the master: 
root@photon-master [ ~ ]# docker service ps web
 ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE           ERROR               PORTS
 mtk6rsovie5m        web.1               nginx:latest        photon-worker       Running             Running 5 minutes ago
 seg6tb6fayvv        web.2               nginx:latest        photon-master       Running             Running 3 minutes ago
 ykq6lrfus4mg        web.3               nginx:latest        photon-worker2      Running             Running 2 minutes ago
 root@photon-master [ ~ ]#

Great. So hopefully that gives you some idea on how you can get started with a simply application to test Docker Swarm running on Photon OS.

Exit mobile version