With the release of Photon Controller v0.9, there were a lot of nice new features. My colleague Sarge has a good bit of detail on the new features in his blog post. One of the interesting additions, in my opinion, is the introduction of support for kube-up and kube-down for deploying Kubernetes frameworks (I will have a blog post on this ready to go shortly). If you are a regular reader, you might remember that we had another, more photon controller-centric way of rolling out K8S on Photon Controller that I wrote about here. That was using the “cluster” concept in Photon Controller. This new functionality is natively built into K8S, so if you’ve deployed K8S in the past, then the user experience should now be pretty identical when deploying the framework on Photon Controller. Anyhow, as I said I will write more about this shortly. The purpose of this post is to tell you about the additional tooling that you might need if you wish to drive this deployment of K8S on Photon Controller via Photon OS. Of course, you can do this from other OS releases too since we support photon controller CLI on various distros, but I thought I would try to do all of this via VMware’s own Photon OS, and highlight what you need to achieve it.
Now remember, I’m not going to provide details on getting kube-up/kube-down running just yet. I will cover that shortly. This is just to get the tooling onto Photon OS to allow you to deploy it on Photon Controller.
If some of this seems a bit jargon-y, these links might help:
1. Give Photon OS extra resources
Since you will be pulling down source and building code, I added some extra resource to my Photon OS VM (originally deployed via OVA). I added a new 40GB VMDK, and gave the VM 4 vCPU and 8GB of memory. I then used fdisk to create a partition on the disk (/dev/sdb1), formatted with mkfs -t ext4 and mounted it on a new directory called /workspace by editing the /etc/fstab file.
2. Additional tooling on Photon OS for Photon Controller CLI
2.1 git: the first thing you will need installed on the Photon OS is the photon controller CLI. In order to build the CLI, you can clone the repo to the /workspace directory on the Photon OS VM.
# git clone https://github.com/vmware/photon-controller-cli.git
To clone it you need a tool called git. But this is not on Photon OS, so you will need to install it using tdnf – the Photon OS equivalent of yum (tiny, dandified yum to be exact). To install git, run the command:
# tdnf install git
If you ever wish to determine the correct package to install for a particular tool or binary, you can use the following command:
# tdnf whatprovides example
This will tell you the package that needs to be installed to provide the tool called example. Once installed, you need to setup some environment variables. These are GOPATH, GOROOT and PATH:
GOROOT=/usr/local/go GOPATH=/workspace/go PATH=$PATH:$GOPATH/bin
2.2 make: When the photon controller CLI source is installed, it comes with a Makefile. In order to compile the source, you need a make command. This is not present either. To install make, run:
# tdnf install make
2.3 go: You can now begin to trying making the photon controller CLI binaries. You usually begin by making tools. However, the next issue is that “go” or “golang”, the GO programming language, is not installed. This needs to be installed next. Now, when you use tdnf to install GO in the current Photon OS, it only deploys version 1.4. We need version 1.6 for photon controller CLI. So to get to version 1.6, there is a to step approach. First, install version 1.4 via tdnf:
# tdnf install go
And then, install the updated 1.6 package which you can download from golang.org. The package is downloaded as a tar.gz, and can be extracted in the GOPATH folder as follows:
# tar -C /workspace/go -xzvf go1.6.2.linux-amd64.tar.gz
2.4 tar: Unfortunately, there is no tar packaged installed either. So you need to install that next.
# tdnf install tar
Now you can complete the installation of golang v1.6. Now you may have to modify the PATH variable to ensure that golang v1.6 is found before the original golang v1.4. Running the command “go version” should reveal version 1.6:
# go version go version go1.6.2 linux/amd64
Now you should be able to complete the creation of the photon controller CLI tools:
# make tools # godep restore # make build
This will create the “photon” binary which can now be copied to somewhere in your PATH for ease of use.
3. Additional tooling on Photon OS for Kubernetes (K8S)
Now, you already have the tooling available to clone down the K8S repo, and you should have plenty of space in /workspace. Once K8S is downloaded, you should be able to run:
3.1 docker: However for this make to succeed, you will need to have docker running, or the make will fail. To start (and keep enabled) docker, run the following commands:
# systemctl restart docker # systemctl enable docker Created symlink from /etc/systemd/system/multi-user.target.wants/\ docker.service to /usr/lib/systemd/system/docker.service.
3.2 awk: The Makefile associated with K8S also has a dependency on “awk” which is not installed on Photon OS either. To install awk, a text parsing tool, run the following command. Notice the subtle difference in the pkg name, and how to find it with tdnf:
# tdnf whatprovides awk gawk-4.1.3-1.ph1.x86_64 : Contains programs for manipulating text files Repo : photon # tdnf install gawk
There is another dependency in the Makefile for K8S.
3.3 mkisofs: Now in order to be able to deploy K8S on Photon Controller, we need to provide an image which is used by Photon Controller to build VMs for the K8S control plane and workers. It relies on the binary “mkisofs” to do this, which is not installed on Photon OS. So once again, we need to install it. Now, I could not find this via “tdnf”, but that might have been me. Instead, I downloaded an RPM which contained “mkisofs”, and installed it manually on my Photon OS. the RPM can be find on any number of RPM repos, and this is how I installed it (note that curl is available on Photon OS to download files):
# rpm -ivh mkisofs-2.01-10.7.el5.x86_64.rpm warning: mkisofs-2.01-10.7.el5.x86_64.rpm: Header V3 DSA/SHA1 \ Signature, key ID 82fd17b2: NOKEY Preparing... ################################# [100%] Updating / installing... 1:mkisofs-9:2.01-10.7.el5 ################################# [100%] # find / -name mkisofs -print /usr/bin/mkisofs # which mkisofs /usr/bin/mkisofs #
Update: It looks like you can get mkisofs via tdnf:
# tdnf provides /usr/bin/mkisofs cdrkit-1.1.11-2.ph1.x86_64 : Utilities for writing cds. Repo : photon # tdnf install cdrkit Installing: cdrkit x86_64 1.1.11-2.ph1 3.48 M … # which mkisofs /usr/bin/mkisofs
3.4 diff: Now this is not essential for deploying the K8S framework but when I was modifying files, diff was extremely useful for comparing the changes with the original version. Therefore I would strongly recommend installing it. It is installed with the package diffutils:
# tdnf install diffutils
So there you have it. With the above in-place we are ready to roll-out a K8S framework using the kube-up and kube-down utilities – what we essentially have here is support in Kubernetes for Photon Platform v0.9. And if you like to drive this all from Photon OS, you will need the above added. A quick list of what we added:
- extra resources (cpu, memory, disk)
- go -> version 1.4, then 1.6
- docker running
- diff (nice to have)
I’ve highlighted this list of requirements to our CNA (Cloud Native Apps) team to see if we can streamline this process and get some of this tooling included. However I appreciate that they are trying to keep the Photon OS as minimal as possible, so this will be a balancing act.
I’ll follow up shortly to show you the steps that are needed to get K8S deployed with kube-up and kube-down.