Deploy DSM Consumption Operator in air-gap environment using Harbor

by Posted on

In this post, I will look at the steps involved in creating your own Harbor Registry, setting it up as a Supervisor Container Registry, and then pulling DSM Consumption Operator images for version 9.0.1 from the DSM appliance and pushing them up to the Harbor Registry. We will then deploy the Consumption Operator using the images in the Harbor registry, and modify the package.yaml and values.yaml to provide a true air-gapped environment for DSM 9.0.1. Other image registries can also be used.

Overview of Steps

If you operate in an air-gapped environment, you can use images and manifests bundled in the VMware Data Services Manager v9.0.1 appliance to configure VMware Data Services Manager in VMware Cloud Foundation Automation. Here are an overview of the steps involved:

  1. Verify that you have access to a client machine with the Carvel tooling and the Docker client installed. The primary Carvel tool that we will use is imgpkg. This is used to transfer images from the DSM appliance to a private registry such as Harbor.
  2. Ensure that you have access to a private container image registry that supports the OCI (Open Container Initiative) image format, such as, Harbor. In this post, we will use Harbor. Other image registries are supported.
  3. Configure the Private Registry in vCenter/Supervisor.
  4. Upload Consumption Operator Images to the Private Registry.
  5. Update the Package and Values YAML manifest to reflect the DSM environment.
  6. Add a New Consumption Operator Supervisor Service for VMware Data Services Manager using the updated YAML manifests.

Install Carvel

VMware Data Services Manager uses the Carvel packaging system.  A step to setting up air-gapped DSM with VCF Automation is to install Carvel. For more information about the Carvel imgpkg utility, see https://carvel.dev/imgpkg/docs/v0.42.x/install/.

Run the following commands to install the Carvel imgpkg utility:

wget -O- https://carvel.dev/install.sh > install.sh

sudo bash install.sh

Verify the Carvel installation has been successful by running the following command:

imgpkg version

Configure a Private Registry in vCenter/Supervisor

The next step is configure a private registry in  vCenter to pull images from the registry. As mentioned, I will use Harbor. There are plenty of guides available online to help you deploy this, such as here. Or if you want to do a quick install, you can use this. One thing to ensure is that TLS/ https is enabled on the registry. The  steps to configure appropriate certificates and key to enable TLS on Harbor are here. Once configured, the image registry should be added to your vCenter / Supervisor. To add the new registry / private repository as a container image registry in the supervisor ‘configure’ settings, follow these steps:

  • Log in to vCenter as an administrator.
  • From the vSphere Client home menu, select Supervisor Management.
  • Click the Supervisors tab, and select the Supervisor for which you want to configure DSM.
  • Click the Configure tab, and from the secondary left panel, select Container Registries.
  • Click Add Registry.
  • Enter a name for the registry and the registry host URL.
  • If the private registry is not trusted, enter the Certificate Authority (CA). This CA will have been created as part of step 1, TLS support.
  • If the private registry does not support anonymous access, enter the user name and password. Harbor default is admin/Harbor12345 if it has not been changed in the harbor.yaml configuration.
  • Click Add.

It should now appear as one of the Container Registries in the Supervisor. In this example, I also made it the default container image registry.

Download the Artefacts from the DSM Portal

Now, download the necessary artefacts from the VMware Data Services Manager portal. In a browser, navigate to the following URL:

https://<dsm-provider-ip>/consumption-operator/supervisor-service/

From this location,  three Carvel artifacts must be downloaded. We will do this in the next step.

    • dsm-consumption-operator-supervisor-thick.tar.gz
    • package.yaml
    • values.yaml

In a browser, now navigate to:

https://<dsm-provider-ip>/consumption-operator/docker-image/

Here, there are two base Docker images which need to be downloaded. We will also do this in the next step:

    • consumption-operator.tar.gz
    • kube-webhook-certgen.tar.gz

The next step is to now push the Supervisor package bundle along with the Consumption-Operator controller and Kube-Webhook-Certgen Docker images to the Harbor registry. We also need to decompress the Docker images consumption-operator.tar.gz and kube-webhook-certgen.tar.gz using gzip to produce a TAR file.

Run the following commands:

export dsm_host=<IP address of your DSM server>

export file_folder=./files
mkdir ./files 

curl -k https://${dsm_host}/consumption-operator/supervisor-service/dsm-consumption-operator-supervisor-thick.tar.gz \
-o ${file_folder}/dsm-consumption-operator-supervisor-thick.tar.gz 

curl -k https://${dsm_host}/consumption-operator/docker-image/consumption-operator.tar.gz \
-o ${file_folder}/consumption-operator.tar.gz 

curl -k https://${dsm_host}/consumption-operator/docker-image/kube-webhook-certgen.tar.gz \
-o ${file_folder}/kube-webhook-certgen.tar.gz  

gzip -d ${file_folder}/consumption-operator.tar.gz  

gzip -d ${file_folder}/kube-webhook-certgen.tar.gz

Caution!!! Do not change the image name. Do not use docker load  and  docker push/pull commands to handle Consumption Operator images. Only use imgpkg as described above.

At this point, all of the necessary components (Consumption Operator Images) are ready to be pushed to the private container image registry.

Create a New Image Registry Project

In this example, I am going to place the consumption operator images in their own project in Harbor. So make a new project in Harbor called “dsm” which matches the $new_folder variable used in the next step.

Upload the Consumption Operator images to Harbor

Use imgpkg to copy the images and bundles and push them to your own registry. Again, I am using a Harbor registry that has a login of admin and a password of “Harbor12345“. You will also need to include the path to the Certificate Authority (CA) that was used to create the Harbor certificate to enable TLS/https support.

export new_registry=harbor.rainpole.io
export new_folder=dsm
export new_repo=${new_registry}/${new_folder}
export file_folder=./files

imgpkg copy  --tar ${file_folder}/dsm-consumption-operator-supervisor-thick.tar.gz --to-repo ${new_repo}/dsm-consumption-operator-supervisor --cosign-signatures --registry-ca-cert-path  /home/cormac/harbor/certs-keys/ca.crt
imgpkg copy  --tar ${file_folder}/consumption-operator.tar --to-repo ${new_repo}/consumption-operator --cosign-signatures --registry-ca-cert-path  /home/cormac/harbor/certs-keys/ca.crt
imgpkg copy  --tar ${file_folder}/kube-webhook-certgen.tar --to-repo ${new_repo}/kube-webhook-certgen --cosign-signatures --registry-ca-cert-path  /home/cormac/harbor/certs-keys/ca.crt 

echo "======> Verify CO Images/Bundles Version"
curl -k https://${new_registry}/v2/_catalog -u admin:Harbor12345
curl -k https://${new_registry}/v2/${new_folder}/dsm-consumption-operator-supervisor/tags/list -u admin:Harbor12345
curl -k https://${new_registry}/v2/${new_folder}/consumption-operator/tags/list -u admin:Harbor12345
curl -k https://${new_registry}/v2/${new_folder}/kube-webhook-certgen/tags/list -u admin:Harbor12345

kapp-controller oddities

!!! Caution: If you already installed the Consumption Operator, and have now deleted it, the kapp-controller on the Supervisor may keep hold of the original image location and will not find the new Harbor registry. This be be obvious if you look at the Consumption Pod YAML and see that it continues to use whatever image registry was being used previously. To reset this behaviour, restart the kapp-controller on the Supervisor as follows.

root@4219ecb7e8d88b75a2022295717ec446 [ ~ ]# kubectl get deploy -A | grep kapp-controller
vmware-system-appplatform-operator-system   kapp-controller    1/1     1            1           7d2h


root@4219ecb7e8d88b75a2022295717ec446 [ ~ ]# kubectl rollout restart deployment/kapp-controller \
-n vmware-system-appplatform-operator-system
deployment.apps/kapp-controller restarted

Install the Consumption Operator

We can now proceed with the installation of the DSM Consumption Operator. We have already downloaded the package.yaml file from the DSM portal. Now we need to edit the package.yaml file by replacing the Broadcom registry with your image registry.

Here is a sample of my modified package.yaml. The only changed required was for the image line, which now points to the location of the image in my new Harbor container image registry.

apiVersion: data.packaging.carvel.dev/v1alpha1
kind: Package
metadata:
  name: consumption-operator.dsm.vsphere.vmware.com.9.0.1+vmware.0.24913742
spec:
  refName: consumption-operator.dsm.vsphere.vmware.com
  version: 9.0.1+vmware.0.24913742
  releasedAt: "2025-08-15T02:44:09Z"
  template:
    spec:
      fetch:
      - imgpkgBundle:
          image: harbor.rainpole.io/dsm/dsm-consumption-operator-supervisor:9.0.1.0.24913742
      template:
      - ytt:
          ignoreUnknownComments: true
          paths:
          - config/
      - kbld:
          paths:
          - .imgpkg/images.yml
          - '-'
      deploy:
      - kapp: {}
---
apiVersion: data.packaging.carvel.dev/v1alpha1
kind: PackageMetadata
metadata:
  name: consumption-operator.dsm.vsphere.vmware.com
spec:
  displayName: Data Services Manager Consumption Operator
  shortDescription: DSM from Harbor

Once the package.yaml is applied to create the Consumption Operator, it must now be configured with information about your own DSM environment. We can configure the Consumption Operator by providing a values.yaml file. This was also already downloaded from the DSM portal but now we update the contents with the specific configuration using your own registry in image name, your DSM endpoint, username, password, rootCA, and also setting the isSupervisor to true. The following is the values.yaml which I added to my Consumption Operator:

image:
  name: harbor.rainpole.io/dsm/consumption-operator
  tag: 9.0.1.0.24913742
dsm:
  authSecretName: dsm-auth-creds
  authSecretGeneration:
    create:
      endpoint: https://dsm.rainpole.io
      user: dsmadmin@rainpole.io
      password: VMware123!
      rootCA: |-
        -----BEGIN CERTIFICATE-----
        MIIDcDCCAligAwIBAgIGAZmFEfhXMA0GCSqGSIb3DQEBCwUAMFkxCzAJBgNVBAYT
        AlVTMQwwCgYDVQQKDANEU00xFTATBgNVBAsMDERTTSBQcm92aWRlcjElMCMGCSqG
        SIb3DQEJARYWZHNtX3N1cHBvcnRAdm13YXJlLmNvbTAgFw0yNTA5MjMwODA5MzFa
        GA8yMDc1MDkyNjA4MDkzMVowWTELMAkGA1UEBhMCVVMxDDAKBgNVBAoMA0RTTTEV
        MBMGA1UECwwMRFNNIFByb3ZpZGVyMSUwIwYJKoZIhvcNAQkBFhZkc21fc3VwcG9y
        dEB2bXdhcmUuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA0Rke
        Jet88CCKJOeeu0+ZTShSXTeURbr6M9wDpe+RuyXEaNRVXHmLOR8bYA5uVH1fo/p1
        HdtKAbpT5RaOzhIrB/zvq0HCvjcjXRP1QIS0LCWlPrflsSpzR5V5oLDdc+U+px20
        vFq5dvmnJCAg+5UYG0yjpOtOVITlQvsqcXjGrR16mKuLxGh3PSjGFq6XAIsjoUJ0
        w0EU7YqJCZZ9R5tVdRdGy9iR5g4OvQCuCRgT40BOqgzIrbV/WLNfnW1718woOGVR
        9L+K1efcjC8PFrTlZVamRVBmRL0useO4nTKBUcdVg8ILKjj0gRaXf6e/2mrbXetm
        t5JfNUiMU38pPsHEeQIDAQABozwwOjALBgNVHQ8EBAMCAQYwDAYDVR0TBAUwAwEB
        /zAdBgNVHQ4EFgQUXzPzk2/y/qLD49APujiW0UO4zZswDQYJKoZIhvcNAQELBQAD
        ggEBAMHLOftFxPSx/AGoUEKZMJKoD08N++u/ziIHnq+b5no5bPo87fxXX3dL52mU
        RaOk/ZXugIk3yGBo2mwRuY4DXtB9rjZyTZkYvl0H8csheCXNAh2v7eJlgrSoEId4
        zCrXQbUgAZDmUmYyNBXzZKLonEzIcSuybailVTvrEY/OTQJNqagcFOcHpF7MhEvB
        0+CfqwdI/bcsbOylgVCR5DxJIURefZ5hU8u7p4RpP7F5rXqpv+t7P3Sq6qfOfvAo
        RQxfNo08WQMWMq/4/m3JEjVbSwTyuA1EC3tQyh1SScfg/7ljI/63Ag+JhPeY8/DL
        YjeUC67LtaPKEk9RoiKzJjYxuD0=
        -----END CERTIFICATE-----
  allowedInfrastructurePolicies: []
  allowedBackupLocations: []
  applyToNamespaces:
    backupLocations: []
    infrastructurePolicies: []
    selector:
      matchAnnotations: {}
  adminNamespace: ""
  isSupervisor: true
  dataServicePolicyNamespaceLabels: {}
consumptionClusterName: "cormac-sv"

And if everything is working as expected, you should now see the Consumption Operator pod/deployment come successfully online and running. navigating to the svc-consumption-operator namespace in the vSphere Client, and select the Compute > Pods view:

DSM Consumption Operator images have been retrieved from the Harbor registry to provide a true air-gapped environment. You can verify this by looking at the YAML for the Pod via the vSphere Client. You should observe the images coming from your local registry as shown here:

At this point, you can begin to setup your VCF Automation environment, including configuring Data Service Policies and enable your tenants to start consuming the Databases as a Service (DBaaS).

Published by Cormac

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.