Benefits of DSM for a Virtual Infrastructure Admin
(i) Placement & Availability Controls
First and foremost, this product was designed with a VI Admin as top of mind. We wanted to help a VI Admin to control “data sprawl”, and maintain control over the placement of databases and data services on vSphere infrastructure. This is achieved via a DSM construct called an infrastructure policy. This defines which compute, storage, networking, and even which VM folder should be used for the placement of a particular database. This not only helps with the overall management and usage of the infrastructure, but also helps with licensing control, forecasting, billing, etc. This placement and control becomes even more granular with VCF 9.0 where DSM can leverage a vSphere Namespace based infrastructure policy to provision databases and data services. A vSphere Namespace is created in the Supervisor. As a vSphere administrator, limits can be set on CPU, memory, and storage in a vSphere Namespace.
Through the infrastructure policy, the VI Admin can also define the database availability. Whilst a DBA can choose whether a database is standalone (single node) or clustered (three nodes), the infrastructure policy will define the placement of the clustered database from a vSphere perspective. By default, DSM will place the three VMs of a clustered database on different ESX hosts in the same vSphere cluster. However the VI Admin can also create a cross-clustered infrastructure policy which, when selected by the DBA, will place the different nodes in the same database cluster on different vSphere clusters, providing an even higher level of availability to the database.
(ii) Visibility & Insight
While DSM has its own UI and API, we didn’t want a VI Admin to have to “context switch” between different UIs to watch over the databases and data services provisioned via DSM onto vSphere. Therefore, we bubbled up into the vSphere Client the ability to view database and data service details. At a glance, a VI Admin can see a list of provisioned databases, the database instance name, the engine type (Postgres, MySQL MS SQL) in the vSphere client, alongside database status, version, alert level, number of nodes, availability, infrastructure policy, storage policy, etc. VI Admins can drill into even more details, and view the matching VM name, placement details and even CPU, Memory and disk utilisation. While this is useful to a VI Admin as is, it is also extremely useful to allow VI Admins and DBAs to have 1:1 conversations about all aspects of the infrastructure consumed by a database, when there is a need to understand resource consumption, performance, scaling and other behaviours. An example of such a view is below.
(iii) vSphere & VMware Cloud Foundation Integration
DSM is integrated with core vSphere infrastructure products. In the previous paragraph, we mentioned how a VI Admin can see database details via the vSphere Client. However, there is also integration with Aria Automation 8.x for those customers who wish to have that level of integration. DSM ships with a custom resource for Aria Automation, which, when installed, create a Service Catalog item to support DBaaS. Going forward, along with the VCF 9.0 announcement, DSM is also integrated with VCF Automation. VI Admins can build Data Service Policies, and assign these policies to different organizations for a true multi-tenancy DBaaS experience.
Similarly, there is integration with Aria Operations, where DSM and its respective databases can send their metrics. There has been some work done on these integrations over the past couple of years, including the creation of a number of default dashboards for the various databases. Once again, going forward, we have made some significant improvements around monitoring of databases, shipping all metrics to VCF Operations in VCF 9.0. And indeed, metrics can also be shipped to a Prometheus endpoint in VCF 9.0 for customers who wish to use it.
Finally, in this section, I want to call out one more integration, and that is with Aria Operations for Logs. With a single step, DSM can ship all logs from both the DSM appliance and all its respective databases to a single endpoint. These logs can also be shipped to any SYSLOG endpoint.
All of the above integration points should be welcomed by VI Admins who are using the full VCF stack.
(iv) Support
Broadcom support covers the lifecycle management of data engines managed by DSM, including provisioning, OS and database software patching, scaling, and clustering. Support for the PostgreSQL or MySQL engine itself, including but not limited to bug fixes, performance issues, and security fixes, will be handled by the upstream community, with Broadcom providing best-effort assistance.
Additionally we work with the open source community to provide a complete fix downstream for any bugs discovered by our customers. Whilst we are not in a position to guarantee when any particular fix is going to be available in any of the open-source databases, we are in a position to create custom builds with fixes that are not yet available in the open source versions. This is a common support model for open source products and many DBaaS providers use the same support model. This should give our customers peace of mind when choosing DSM for their DBaaS.
Benefits of DSM for a DBA
Lets now focus on the DBA, and what benefits DSM brings to the table for that persona.
(i) Intuitive UI for Fleet Management
DSM provides DBAs with a single pane of glass to easily manage their database fleet. The default landing page for DSM Admins is shown below, along with all of the various management and configuration options.
(ii) Lifecycle Management
The DSM team provides regular updates to DSM. This includes updates to the DSM appliance, the guest OS used by the database nodes, the Kubernetes versions used by the database nodes, and indeed the databases themselves. The only thing the DBA needs to do is to download and stage the update. Both the appliance and the databases have maintenance windows. If an update is staged, and the window rolls around, normally Saturday night – Sunday morning, the update is automatically applied. No one needs to take care of the lifecycle management of the VM’s operating system or the K8s cluster – it is all taken care of by DSM. This aspect of DSM is often missed in the comparison with other DBaaS solutions.
One final notes on database lifecycle management. DSM now has the ability to upgrade both the major version and the minor version of a database. While some care and attention is required from a DBA when doing major version upgrades, such as making sure extensions are still compatible, it is still a significant feature to have available.
(iii) Automated Backups / Point-In-Time Restores
As with lifecycle management, backups of the database can also be automatically configured at deployment time. If enabled, DSM will automatically begin backing up databases once they are online. By default, for Postgres, a full backup is taken every week. Daily backups are taken daily, and the WAL (Write Ahead Logs) are shipped every 5 minutes /16 MB – whichever threshold is reached first. MySQL and MS SQL Server have their own default backup schedules. With this automated backup schedule available for all databases, Point-In-Time Restores are available for all databases.
(iv) Scaling In, Out, Up and Down
DSM allows DBAs to very simply scale database topologies out from a single node standalone to a three node cluster, and scale it back in again. It also allows DBAs to scale up by changing the VM Class used by nodes of a database. This increases the amount of CPU and Memory available to a node. Not only that, but if there are ‘stranded’ resources on a node, it is possible for the DBA to scale down to a lower VM Class so that the node consumes less CPU and Memory. This can be extremely useful for a month end or end of quarter processing, or a ‘Black Friday’ type event where more resources might be required by the database for a short length of time. Once the ‘spike’ is over, the resources can be freed up for other uses.
(v) Cloning
DSM enables DBAs to make clones of a database, should there be a need for such a task. This can be very useful for test-dev type environments where developers may wish to run some queries on ‘live’ data, but maybe not on the production database.
(vi) LDAPS integration
DSM supports secure LDAP for both DSM access and database access. This means that the task of assigning user access to a database becomes more secure and easier to manage. During database provisioning time, DBAs can choose if a database has Directory Service integration, and can then use a simple GRANT statement to give LDAP users access to the database. Of course, this is considering the traditional ticket based approach. DSM also has secure LDAP support for the DSM UI, enabling self-services for end users and developers. They can, if given permission to do so, login to the DSM UI as an LDAP user and self-service provision their own databases. Both the ticket-based and self-services approaches are supported by DSM.
(vii) Certificate Management
Certificates are necessary to ensure secure access to a database. DSM provides DBAs with two certificate options; either use DSM’s own self-signed certs, or DBAs provide their own custom certificates. These can be applied at either database creation time, or can be provided after the database is deployed. The DSM UI and API provide mechanisms to easily add the certificates. With security being top of mind for most IT teams, this is a must-have feature.
(viii) Auditing and Alerting
DSM provides numerous mechanisms to send alerts to interested parties, alongside some built-in alerting in the DSM UI. These mechanisms include sending notifications via email (SMTP), but there is also a webhook mechanism which can be used to integrate into third party applications such as ServiceNow or Slack. This means that the DBA team will never miss an event related to their databases.
For compliance purposes, DSM maintains an audit of events that a DBA can very quickly review. In DSM 9.0, audit logs are also forwarded to the configuring log receiving utility.
(ix) High Availability
While these may seem like something a VI Admin should be handling, they become very much a choice that the DBA can make when provisioning the database. These are options mostly controlled through infrastructure policies and storage policies. We saw in the VI Admin section that clustered databases could be provisioned on the same vSphere cluster or across different vSphere clusters. A DBA would choose the correct availability option using different infrastructure policies on a database by database perspective.
(x) Encryption
From an encryption perspective, it is possible to leverage the VM Crypt feature which would provide the option to encrypt some databases and not others. This would be defined in the storage policy which makes up the infrastructure policy, so when it comes to provisioning a database, the DBA should select the appropriate infrastructure policy to ensure encryption. Alternatively, you may wish to encrypt the whole of a datastore, e.g., vSAN. Then, any database which lands on this datastore is automatically encrypted. Both approaches are support by DSM.
(xi) Disaster Recovery
The last point to highlight is that a DBA can use replication to remotely protect a database. If there are multiple vSphere environments, a Postgres database could have a secondary node placed on a completely different vSphere environment using a completely different DSM appliance. Controls are available within the DSM UI for a DBA to promote a secondary node to a primary and vice-versa. In the event of a loss of a complete vSphere environment, a DBA can “failover” databases to another environment and run the business from there.
(xii) Host Based Access controls at provisioning time
Postgres’s pg_hba.conf defines which users from which networks can access which databases. Through the DSM UI, DBAs can pre-populate the host based access controls at database deployment time without need to access the database post-deployment and setting up these rules as a separate step. This also avoids having the database “insecurely” available for a period of time while these rules are put in place.
(xiii) Advanced Parameters at provisioning time
Similar to the previous item, DBAs can specify which advanced parameters to set of the database at deployment time, rather than having to do this as a post-deployment step. This speeds up database deployment times.
(xiv) Database Delete Protection
DSM users have the ability to recover a database after it has been deleted. The retention period of a deleted database is 30 days, but this can be reduced or extended if necessary.
Benefits of DSM for an end-user/developer
One of the primary objectives of DSM is to enable self-services or DBaaS. To that end, DSM needs to provide end-users or developers with a simple mechanism to request and access databases. Let’s look at those features next.
(i) Intuitive UI
DSM Users are provided with a very simple UI which allows them to easily provision, update and delete databases as needed. The do not see any configuration settings, nor do the see the databases of other users. For end users who wish to use a UI for deploying databases, this is a simple UI and is very intuitive to use.
(ii) Kubernetes and REST APIs
We understand that many developers do not wish to use a UI. Sometimes, they wish to provision a database as part of a much larger pipeline for testing purposes. To that end, DSM comes with both a K8s API and a REST API. This allows customers to automate the deployment of databases through DSM. The APIs are available here on the official DSM API site.
(iii) Provision DSM databases from remote K8s clusters
Many developers may already be using Kubernetes for their application development. These K8s clusters may be resource constrained, i.e., only using local storage. There may also be challenges for SREs with developers spinning up ‘uncontrolled’ databases and data services, leading to a combination of resource contention and compliance issues. Through DSM’s Consumption Operator, developers can spin up their own DSM databases on vSphere infrastructure and connect to them from their own Kubernetes clusters.
Summary
