Configuring webhook notifications in VMware Data Services Manager v1.5

In my most recent post, we looked at how to send email alerts via SMTP from VMware Data Services Manager (DSM) v1.5. In this post, we will look at how to send alert notifications using a different method, webhooks. For the purposes of this post, we will examine the steps involved in sending webhook notifications to a slack channel. However, VMware Data Services Manager v1.5 also supports the option to configure webhook notifications to ServiceNow as well as custom webhook configurations for other applications. We will begin this post by first creating a slack application, enabling the web hook features on the application and then finally selecting a slack channel to send alerts and notifications from VMware DSM. Then we will configure the webhook in DSM so that failures or status changes send a notification message to the configured slack channel.

Create a Slack Incoming Webhook URL

Creating a Slack Incoming Webhook URL is what allows us to post messages from DSM to a slack channel via a URL. Our webhook implementation for slack expects such a URL. There are 4 steps involved in this process which include (1) creating the slack app, (2) installing the slack app to a slack workspace, (3) enabling incoming webhooks on the app, and finally (4) creating an incoming webhook so that you get a URL for sending messages from DSM to a slack channel. Depending on your organization, on step (2) you may need to get administrator approval to publish the app to a particular workspace. Let’s assume that is possible. Therefore, the first step is to navigate to https://api.slack.com/apps and click on the “Create New App” button. You may need to login to your slack account if you have not already done so. At this point, you need to give the app a name and select the slack workspace. Note the message about needing admin approval – you may or may not see this.

After clicking on the “Create App” button, you should be dropped into the Basic Information section of the app. It is here that the “Incoming webhooks” are enabled. Under Settings > Basic Information, you can also change some of the features of the app, such as its appearance. You can even add an icon image for it. I added a nice DSM logo to my application. You can check this out for yourself. Back in Incoming Webhooks section, turn the “Activation Incoming Webhooks” button to On, and click on the “Add New Webhook to Workspace” at the bottom of the page, as per the screenshot below.

After clicking on the “Add New Webhook to Workspace”, you will now be asked to select a slack channel for the app to post notifications to. It will automatically use the slack workspace that you have logged into at the beginning of the process, but you will have to choose a particular user or channel in that workspace. I created my own slack channel for this purpose.If everything is approved and working correctly, the incoming webhooks view should now have the Webhook URL that we need to configure slack notifications in VMware Data Services Manager. You should also notice a new message in the slack channel, stating that you have added an integration to this channel, followed by the name of the slack application.

Copy the Webhook URL by clicking on the ‘Copy’ button. We can now proceed to step 2 and configure webhook notifications from VMware DSM.

Configuring Webhook Settings in VMware DSM

Login to DSM as the Provider Admin, navigate to Settings from the left hand menu and select webhook settings. Click on the link to create a new webhook. This should provide a UI view something similar to the following:

Populate the fields as follows. In the Name and Destination field, set the Endpoint Type to Slack. Provide a name for the webhook. Next, in the Destination URL field, place the URL that you copied from your slack application created in the previous step. Finally, in the HTTP Request Method, set this to POST. Click Next to proceed to the DB Status Change Payload.

This defines the DB Status Change payload, describing the contents of the notification message that will be sent via slack should the database status change. You can modify this payload as you see fit and include any of the parameters listed on the right hand side. To make sure everything is working as expected, click on the Test link at the bottom of the page to send a test “DB Status Change” notification.

The TEST should return something similar to the following.

Now we need to check our slack channel to see if the notification actually arrived.

Success! We have a slack message with the test payload. Let’s continue with the webhook configuration, but for the DB Operation Failure.

Once again, you can modify the Operation failure as required, using the parameters to the right. Click the TEST button again to check we can successfully send a DB Operation Failure payload to our slack channel via our slack app that has incoming webhooks enabled. This should also be display a successful message if all is working correctly.

And if we check our slack channel which was integrated with our slack application with webhooks:

This also appears to have been successful. Complete the creation of the webhook in DSM, and then we can do some actual status change and failure testing, like we did in the previous post of SMTP email alerts. Once completed, the configuration should look something like this.

Validation that slack webhooks are working

One subtle difference between SMTP and webhooks is that the webhook alerts are not automatically added to databases at provisioning time. You have to explicitly select them to include in the Webhook Alert / Email Alert Configuration section when creating a new database, as shown below. Note that DB Status Change and DB Operation Failure are distinct, and both need to have the webhook associated with them.

For existing databases, the webhooks for slack notifications can be associated by simply going to the database and adding them under the Alert Settings.

Now we can do the same tests as we did in the previous post on SMTP. The first is a test to cause an operation failure on a deployment of a new database. This is achieved by simply cancelling the task in the vSphere client UI.

After this operation is cancelled, the following slack message is sent to the slack channel. Note that this is a Database Operation Failure. The information fields that are in the slack message payload can be changed during the webhook configuration step, as previously mentioned.

Similarly, we can generate a ‘Lost Connection’ status change to an existing database by simply removing the management network from the DB VM.

Once more, we should observe another slack message appearing in our slack channel, but this time it is for a Database Status Change, where the status is reported as ‘Lost Connectivity’.

And that completes the webhook configuration testing. Hopefully you can appreciate that this a very powerful alerting feature of VMware Data Services Manager, along with SMTP alerting and other webhook configurations such as ServiceNow.

Thanks for reading this far. Of course, this week it is VMware Explore 2023. I will not be in Las Vegas personally, but if you are interested in learning more about VMware’s strategy around data management, especially databases, and you are attending VMware Explore 2023 in Las Vegas, check out this session from VMware Fellow, Christos Karamonlis. There are lots of interesting developments happening in this space at VMware, so be sure to check it out.