Data Services Manager 9.0.1 Announced
It gives me great pleasure to announce the availability of VMware Data Services Manager (DSM) version 9.0.1. Over the coming weeks I will be deep-diving into many of these new features, but for now I want to provide you all with a brief overview of the capabilities and enhancements that you can find in this release.
Automated Active Directory integration for MS SQL Server
We continue to enhance our MS SQL Server integration. Although the data service is still in tech preview in DSM 9.0.1, a significant enhancement in this release is the ability to specify a privileged Active Directory user who can create Service Principal Names (SPNs) for database users. This means that the privileged user can now do the necessary tasks in Active Directory to allow Windows Authentication to work seamlessly on MS SQL Server instances and databases which have been provisioned via DSM. Coupled with this is the ability for DSM to use this privileged user to automatically update DNS entries. This is really useful for some MS SQL Server tooling, such as MS SQL Server Studio, which relies on DNS names (fully qualified domain names) to access the instance and databases. This privileged user is added to the Active Directory configuration in the SQL Server section of the UI. Then, when provisioning of instances begins, this AD configuration can be selected and the privileged user is used to do the automated tasks. I will definitely do a more detailed follow up on this feature very soon.
New SQL Server Configuration Options
After receiving a lot of feedback from the initial SQL Server integration offering in DSM 9.0, a bunch of customer-facing configuration features are now being offered at provisioning time in DSM 9.0.1 This includes the ability to set the collation (character representation and rules for sorting and comparing data), add certain configuration options to the mssql.conf file as well as set trace flags on the database. Auditing of user logins is also available in this version.
Aligning DSM RBAC to VCFA Multi-tenancy Controls
With the release of VMware Cloud Foundation (VCF) 9.0, and VCF Automation (VCFA), a number of new multi-tenancy objects were introduced. One such object was the concept of a Namespace. To put it very simply, a Namespace in VCFA is a way allocating certain resources and services to a set of users / tenants. As part of integrating DSM into VCFA multi-tenancy, we also made some enhancements to provide controls over which data services, which data service versions, which infrastructure policies, and which backup locations are available to the different tenants. To allow the VCFA Provider Administrators to control these aspects of data services from DSM, we introduced the concept of a Data Service Policy (DSP). The DSP is how we managed to give different tenants access to different data services and associated resources. In DSM 9.0.1, these RBAC (Role-Based Access Control) concepts are introduced into the standalone version of DSM – no VCFA required. This should help to get you started on your journey to multi-tenancy with VCFA. DSM Administrators will now be able to decide which sets of users (associated with a Namespace) have access to which aspects of a data service through a Data Service Policy. This has been requested for some time, so it great to be able to offer in this release. I will do a more detailed follow up on this DSM RBAC feature in a later post.
Provision Data Services with custom vSphere Tags
This is a feature that I know some of our customers have been asking for. Many of our customers use vSphere tags extensively, whether it is for billing/charging purposes, or simply as a way to automate certain activities such as choosing correct network placement. In DSM 9.0.1, vSphere Tags can now be associated with a database at provisioning time. These tags appear in the vSphere client. Note that the tag and its category must already exist in vSphere before it can be assigned to a database. After assigning the tag to the database in DSM, it appear in the summary view of the virtual machine used to back the data service. Here is an example of a vSphere tag being assigned to the data service during provisioning:
Here it is again on the VM running the database when viewed from the vSphere Client.
Enforce Client Certificate Authentication on Postgres
This is another feature we have added which came directly from user requests. The customer wanted to ensure that only certificate based authentication was allowed for clients – they did not want to manage passwords. Therefore, each client who requires access to the database is allocated a certificate, and will need to use this certificate to login to the database. Now when you create a Postgres database, you can add the following to the pg_hba.conf (HBA settings) to force certificate auth: cert clientcert=verify–full
When connecting to this Postgres database, the certificate, key and certificate authority must be in the connection string used by the client:
$ psql "host=pg-01.rainpole.com port=5432 dbname=pg-01 \
sslmode=verify-full \
sslcert=cormac.usr.cert.pem \
sslkey=pg-01.rainpole.com.no-pass.pem \
sslrootcert=ca.cert.pem"
psql (16.9 (Ubuntu 16.9-0ubuntu0.24.04.1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off)
Type "help" for help.
pg-01=# \conninfo
You are connected to database "pg-01" as user "cormac" on host "pg-01.rainpole.com" (address "xx.xx.xx.xx") at port "5432".
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off)
pg-01=#
If you try to connect without any of the components (key, cert, ca), you will be blocked:
$ psql "host=pg-01.rainpole.com port=5432 dbname=pg-01 \
sslmode=verify-full \
sslcert=cormac.usr.cert.pem \
sslkey=pg-01.rainpole.com.no-pass.pem"
psql: error: connection to server at "pg-01.rainpole.com" (xx.xx.xx.xx), port 5432 failed: \
root certificate file "/home/cormac/.postgresql/root.crt" does not exist
Either provide the file, use the system's trusted roots with sslrootcert=system, \
or change sslmode to disable server certificate verification.
Definitely a nice additional to secure your Postgres databases.
Postgres Disaster Recovery Improvements
Back in DSM 2.2, we introduced database replication for Postgres which allows our customers to replicate their databases to different sites. In fact, these secondary databases could be managed by a completely different DSM. In DSM 9.0.1, we have introduced a few additional operations in the DSM UI to make this process even more seamless. We now have the following steps to help customers handle a DR situation or even a controlled planned promotion of a secondary to a primary.
- Block database connections
- Wait secondary to sync up, which can be viewed via the UI
- Promote Secondary to Primary
- Demote (blocked) primary as read only and unblock its connections
All of the above steps are available with a single click. At this point, the admin can reconfigure any other secondaries to point to new primary. Simply edit its replication configuration and point it to the new primary’s replication user and slot.
Database Troubleshooting Toolkit
This is a feature that I am very excited about. A number of our customers discussed with us how they were troubleshooting databases when run on virtual machines and even bare-metal. They would like to take their methodology, which they have developed over many years, to DSM which, naturally, runs databases on containers. To that end, the DSM team created a new toolkit which can be readily accessed by the superuser of the DSM appliance. Once logged into the appliance as “root”, run the dsm-debug command and follow the instructions. The command will place you on the requested container of the requested database, and show you the tooling available for troubleshooting purposes. Here is an example of dsm-debug in action.
root@amslvcfdsm01 [ ~ ]# dsm-debug dsm-debug CLI Documentation dsm-debug is a command-line tool that enables DSM administrators to troubleshoot their Postgres data services. It allows the execution of a predefined set of commands (listed below) by attaching to a container and sharing the same namespace. This provides administrators with the necessary context to inspect and debug the running database environment. Running it starts a restricted session in the same network and process namespace as the data service node. Predefined Commands: basename, cat, cut, date, df, diff, dirname, du, echo, egrep, fgrep, free, grep, groups, head, hostname, htop, id, iftop, iostat, iotop-c, ls, lsblk, lscpu, lsmem, lsof, nload, pg_activity, pgrep, pidof, pmap, printenv, ps, pwd, readlink, sleep, sort, ss, stat, strace, tail, tcpdump, test, top, tr, uname, uniq, uptime, vmstat, wc, which, false, true Example Usage: # Connect to the first node of the data service dsm-debug --kind PostgresCluster --data-service-name my-pg --namespace services --node-index 0 # once connected to the node, for a list of available commands, type: ls /debug/bin Usage: dsm-debug [flags] dsm-debug [command] Available Commands: autocomplete Generate shell completion script completion Generate the autocompletion script for the specified shell help Help about any command Flags: --data-service-name string Data service name. Example: --data-service-name example-ds -h, --help help for dsm-debug --kind string Resource kind. Example: --kind example-kind (default "PostgresCluster") -n, --namespace string Namespace name. Example: --namespace example-ns (default "default") --node-index int The index of the node to connect to, taken from DSM UI / Topology / Database Nodes. Example: --node-index 0. Default is 0. For Postgres clusters, access the monitor node by specifying --node-index 2 for vSphere Cluster HA topology, or --node-index 1 for single-server topology. Use "dsm-debug [command] --help" for more information about a command. root@amslvcfdsm01 [ ~ ]# kg get postgresclusters -A NAMESPACE NAME STATUS STORAGE VERSION AGE marketing-ns marketing-pg-01 Ready 60Gi 17.5+vmware.v9.0.1.0 4d2h marketing-ns marketing-sec01 Ready 60Gi 17.5+vmware.v9.0.1.0 2d root@amslvcfdsm01 [ ~ ]# dsm-debug --kind PostgresCluster \ --data-service-name marketing-sec01 \ --namespace marketing-ns --node-index 0 ================================================================================ Welcome to the DSM PostgreSQL Debug Shell ================================================================================ For a list of available commands, type: ls /debug/bin You are in a restricted container in same network and process namespace as postgres node. Use with caution. ================================================================================ postgres@marketing-sec01-0:~$ ls /debug/bin basename date dirname egrep free head id iotop-c lscpu nload pidof ps sleep stat tcpdump tr uniq wc cat df du false grep hostname iftop ls lsmem pg_activity pmap pwd sort strace test true uptime which cut diff echo fgrep groups htop iostat lsblk lsof pgrep printenv readlink ss tail top uname vmstat postgres@marketing-sec01-0:~$
Whilst the features include a lot of generic command line tools, a very useful one for Postgres is pg_activity. I’m delighted to see that included. This is another useful feature that I am sure will be very much welcomed by our customers. You can find more information in this KB article.
MySQL Read Replica Scaling
As well as provisioning clustered MySQL databases, DSM 9.0.1 now introduces additional functionality which includes the ability to provision multiple MySQL read replica instances. These can then be used to scale the read throughput of the database engine, leading to better and improved performance overall. To enable this functionality, read replication must be enabled on the main database.
Next, pick up the replication string from the primary and use it to configure the read replicas.
This is yet another feature driven by customer demand.
Summary
Yet again, as I am sure you can appreciate, the team has pulled out all the stops to deliver a compelling set of new features and enhancements in this release. But we haven’t stopped yet. More and more enhancements are being worked on as we speak, and I look forward to bringing you news about these soon. And stay tuned as I dive deeper into some of the topics outlined in this post. As always, thanks for reading. You can download version 9.0.1 of Data Services Manager from this link to the Broadcom Download Site. Get all the latest information about this 9.0.1 release from the Release Notes.
