x509 error logging into harbor registry via VIC VCH

In my last post, I showed some of the new command line functionality associated with deploying out a new Virtual Container Host (VCH) with vSphere Integrated Containers (VIC). I also highlighted how VIC now includes both Admiral for container orchestration via templates and the harbor registry is used for storing docker images. Harbor hosts docker images and Admiral hosts templates. An Admiral template describes how docker images hosted on Harbor gets instantiated (Kudos again to Massimo for this explanation). In my last post, I showed how I finally managed to deploy my VCH. Now the idea was that I should be able to login to my harbor registry from my Windows docker client via the docker API endpoint provided by my VCH. However, on attempting this, I got the following error:

C:\Users\chogan\Downloads\vic> docker -H 10.27.51.114:2376 --tls login 10.27.51.37
Username: admin
Password:

Error response from daemon: Head https://10.27.51.37:443/v2/: x509: certificate signed by unknown authority

It took a bit of time, and some help, but this is what we found to be the issue, and this is how I resolved it.

The root cause is that I was using a self-signed cert on Harbor and failed to let VCH trust that cert. So how do I make VCH trust the self-signed cert on Harbor? I found the answer in the Harbor registry documentation: we deploy a VCH and specify our [Harbor] CA cert via a –registry-ca parameter in vic-machine. Ah, so what I need to do is include this option in my vic-machine command and make sure the VCH trusts Harbor. Fine. So where do I get the Harbor CA cert? That is easy. Login to Harbor (aka VIC Registry), and under admin, there is an option to download the CA cert/Root cert:

This downloads a file called ca.crt. Once downloaded, you may now include the –registry-ca parameter in the vic-machine command used for building the VCH, and point it to the root certificate:

C:\Users\chogan\Downloads\vic> vic-machine-windows.exe create --name corVCH01 \
--compute-resource Cluster --target vcsa-06.rainpole.com \
--user administrator@vsphere.local --password XXX \
--thumbprint 4B:A0:D1:84:92:DD:BD:38:07:E3:38:01:4B:0C:F1:14:E7:5D:5B:00 \
--no-tlsverify --image-store isilion-nfs-01 --public-network "VM Network" \
--bridge-network BridgeDPG --bridge-network-range "192.168.100.0/16" \
--volume-store "isilion-nfs-01/VIC:corvols" --registry-ca="..\ca.crt"

May  4 2017 19:15:48.122+01:00 INFO  ### Installing VCH ####
May  4 2017 19:15:48.168+01:00 WARN  Using administrative user for VCH operation - use --ops-user to improve security (see -x for advanced help)
May  4 2017 19:15:48.175+01:00 INFO  Loaded server certificate corVCH01\server-cert.pem
May  4 2017 19:15:48.177+01:00 WARN  Configuring without TLS verify - certificate-based authentication disabled
May  4 2017 19:15:48.180+01:00 INFO  Loaded registry CA from ..\ca.crt
May  4 2017 19:15:48.413+01:00 INFO  Validating supplied configuration
May  4 2017 19:15:48.555+01:00 INFO  vDS configuration OK on "BridgeDPG"
May  4 2017 19:15:48.591+01:00 INFO  Firewall status: DISABLED on "/DC/host/Cluster/esxi-dell-i.rainpole.com"
May  4 2017 19:15:48.629+01:00 INFO  Firewall status: ENABLED on "/DC/host/Cluster/esxi-dell-j.rainpole.com"
May  4 2017 19:15:48.656+01:00 INFO  Firewall status: ENABLED on "/DC/host/Cluster/esxi-dell-k.rainpole.com"
May  4 2017 19:15:48.691+01:00 INFO  Firewall status: ENABLED on "/DC/host/Cluster/esxi-dell-l.rainpole.com"
May  4 2017 19:15:48.703+01:00 INFO  Firewall configuration OK on hosts:
May  4 2017 19:15:48.705+01:00 INFO     "/DC/host/Cluster/esxi-dell-i.rainpole.com"
May  4 2017 19:15:48.706+01:00 INFO     "/DC/host/Cluster/esxi-dell-j.rainpole.com"
May  4 2017 19:15:48.708+01:00 INFO     "/DC/host/Cluster/esxi-dell-k.rainpole.com"
May  4 2017 19:15:48.709+01:00 INFO     "/DC/host/Cluster/esxi-dell-l.rainpole.com"
May  4 2017 19:15:49.095+01:00 INFO  License check OK on hosts:
May  4 2017 19:15:49.096+01:00 INFO    "/DC/host/Cluster/esxi-dell-i.rainpole.com"
May  4 2017 19:15:49.099+01:00 INFO    "/DC/host/Cluster/esxi-dell-j.rainpole.com"
May  4 2017 19:15:49.100+01:00 INFO    "/DC/host/Cluster/esxi-dell-k.rainpole.com"
May  4 2017 19:15:49.102+01:00 INFO    "/DC/host/Cluster/esxi-dell-l.rainpole.com"
May  4 2017 19:15:49.118+01:00 INFO  DRS check OK on:
May  4 2017 19:15:49.121+01:00 INFO    "/DC/host/Cluster"
May  4 2017 19:15:49.160+01:00 INFO
May  4 2017 19:15:49.277+01:00 INFO  Creating virtual app "corVCH01"
May  4 2017 19:15:49.314+01:00 INFO  Creating directory [isilion-nfs-01] VIC
May  4 2017 19:15:49.326+01:00 INFO  Datastore path is [isilion-nfs-01] VIC
May  4 2017 19:15:49.329+01:00 INFO  Creating appliance on target
May  4 2017 19:15:49.342+01:00 INFO  Network role "management" is sharing NIC with "client"
May  4 2017 19:15:49.358+01:00 INFO  Network role "public" is sharing NIC with "client"
May  4 2017 19:15:50.221+01:00 INFO  Uploading images for container
May  4 2017 19:15:50.223+01:00 INFO     "bootstrap.iso"
May  4 2017 19:15:50.223+01:00 INFO     "appliance.iso"
May  4 2017 19:15:56.276+01:00 INFO  Waiting for IP information
May  4 2017 19:16:12.619+01:00 INFO  Waiting for major appliance components to launch
May  4 2017 19:16:12.675+01:00 INFO  Obtained IP address for client interface: "10.27.51.114"
May  4 2017 19:16:12.677+01:00 INFO  Checking VCH connectivity with vSphere target
May  4 2017 19:16:12.798+01:00 INFO  vSphere API Test: https://vcsa-06.rainpole.com vSphere API target responds as expected
May  4 2017 19:16:14.850+01:00 INFO  Initialization of appliance successful
May  4 2017 19:16:14.852+01:00 INFO
May  4 2017 19:16:14.855+01:00 INFO  VCH Admin Portal:
May  4 2017 19:16:14.858+01:00 INFO  https://10.27.51.114:2378
May  4 2017 19:16:14.860+01:00 INFO
May  4 2017 19:16:14.862+01:00 INFO  Published ports can be reached at:
May  4 2017 19:16:14.865+01:00 INFO  10.27.51.114
May  4 2017 19:16:14.866+01:00 INFO
May  4 2017 19:16:14.868+01:00 INFO  Docker environment variables:
May  4 2017 19:16:14.869+01:00 INFO  DOCKER_HOST=10.27.51.114:2376
May  4 2017 19:16:14.876+01:00 INFO
May  4 2017 19:16:14.878+01:00 INFO  Environment saved in corVCH01/corVCH01.env
May  4 2017 19:16:14.880+01:00 INFO
May  4 2017 19:16:14.881+01:00 INFO  Connect to docker:
May  4 2017 19:16:14.883+01:00 INFO  docker -H 10.27.51.114:2376 --tls info
May  4 2017 19:16:14.884+01:00 INFO  Installer completed successfully

This appears to have deployed successfully, and we now have the docker API endpoint. Let’s now see if this VCH now trust Harbor, and if we can log into the Harbor registry using that docker API endpoint:

C:\Users\chogan\Downloads\vic> docker -H 10.27.51.114:2376 --tls login 10.27.51.37 -u admin
Password:
Login Succeeded

C:\Users\chogan\Downloads\vic>

And to logout from the registry:

C:\Users\chogan\Downloads\vic>docker -H 10.27.51.114:2376 --tls logout 10.27.51.37
Removing login credentials for 10.27.51.37

Success! For steps on how to push and pull docker images to the Harbor registry, here is an earlier post on how to do just that.