Data Services Manager Air-Gap deployment steps

Many customers have data centres which have restricted access to the outside world, commonly referred to as air-gapped. This was well understood in the early design stages of Data Services Manager (DSM), which led to the creation of an air-gap deployment methodology. Therefore customers who could not avail of the automatic download mechanism for database templates and updates could manually populate their provider repository with manually downloaded images.

However, customers who wish to deploy the current version of DSM (v2.0.3) today will also have to use the air-gap approach to add the database templates to their provider repository. This is because the automatic sync and download mechanism that used to exist via the Tanzu Network portal has now been deprecated. All former VMware customer portals are now consolidated onto a single Broadcom support portal. I have already had a customer reach out to me about this as they went through a deployment, and were met with a 401 Unauthorized: {} error when they tried to add a Tanzu Token which worked with previous deployments. For that reason, I decided to create a post which shows the steps to manually download the air-gap database images zip file, how to correctly extract them and and how to correctly upload them to the provider repo used by their DSM deployment. In tandem, I would also encourage you to please read the official documentation on the topic.

Step 1: Download the Air-gap Environment Repository

Whilst links to Tanzu Network now redirect customers to the Broadcom support portal, it may still be a little difficult to find the DSM files. To get the air-gap bundle, login to the Broadcom Support Portal and navigate to “My Downloads” on the left hand side. From here, you should see a full list of products that you can download. VMware Data Services Manager should be one of the items in that list. Click on it, and then click on the VMware Data Services Manager product once again to see the full list of downloadable files. Select version 2.0.3. One of the items in the list should be the Air-gap Environment Repository, which comes as a zip file. Here is a full link to the location.

Download this zip file, move it into its own empty folder and extract it. It should reveal 3 folders, dsm-control-plane, dsm-data-plane and manifests.

The next step is to upload these logs to an empty S3 bucket which you have designated as the Provider Repo Url in DSM.

Step 2: Upload air-gap files to Provider Repo Bucket

We will assume that the DSM appliance has been successfully deployed and the DSM Plugin to vCenter is operational. Through the vSphere Client, create a DSM Admin User permission and login to the DSM Appliance. In the dashboard, you will see the following settings not configured:

The configuration that we need to address for air-gap deployments is the Provider Repo. Click on the Configure link at the bottom left hand corner, and once in the Settings page, select Storage Settings. You should see something similar to the following:

At this point, one or more S3 buckets must be chosen for the Provider Repo, Provider Log Repo and Provider Backup Repo. Before adding a bucket, lets add the previously extracted Air-gap Environment Repository folders and files to the Provider Repo bucket. You can use any number of different tools to upload to the bucket. One such options using the aws command line tool is provided in the official documentation. When the air-gap folders have been uploaded, the Provider Repo bucket contents should have the three folders listed in the root folder of the Provider Repo bucket – dsm-control-plane, dsm-data-plane and manifests.

Step 3: Configure Provider Repo Url

With the Provider Repo Bucket now populated with the air-gap files and folders, we can proceed with the configuration of the External Storage. The first External Storage to configure is the Provider Repo Url. This is where we add the newly populated Provider Repo Bucket. Configure this bucket first. Here is an example of such a configuration taken from my DSM environment:

Once you have connected to the S3 bucket URL, accepted the thumbprint from your S3 object store and saved the configuration, you should observe a Release Processing operation initiate almost immediately once the Provider Report Url is configured:

And if you have correctly configured the Provider Repo bucket with the air-gap configuration, this Release Processing task should succeed very quickly.

You can now complete the configuration of the remaining External Storage items, such as the Provider Logs Repo Url, Provider Backup Repo Url and Database Backup Storage.

Step 4: Enable Data Services

We have one final step before we can begin to provision databases and that step is to enable the PostgreSQL and MySQL data services. I should point out that before a data service can be enabled, you must create at least one infrastructure policy. Once an infrastructure policy is in place, you can navigate to Version & Upgrade, and select the Data Services tab. From the actions (the three vertical dot in the column before the data service), select enable.

This should start the enabling process, and the status changes to Enabled [InProgress]:

This will create another folder on the Provider Repo bucket called ‘docker’ which acts as a docker registry and this folder has all of the images required to create a database. You can track the progress of the enable operation via the DSM command line and the /var/log/tdm/provider/containers/docker-registry.log file. The time to complete the Enable operation is entirely dependent on the performance of your network and S3 Object Storage, but after a short period of time, the data services should have an Enabled status.

And that completes the enablement of Data Services Manager using the air-gap approach. We plan to improve this process and make it less arduous in upcoming releases of DSM which I hope to be able to share soon.