Send vRLI Alerts to Slack

Recently I had some issues with one of the ESXi hosts in my homelab. While investigating the issue I saw an event in vCenter and as I had recently installed and configured the integration with vRealize Log Insight I wanted to see if I could find the same event there.

I found the matching event in vRLI and also saw that the event had only happened on that specific host, and only a short time period before the host failed.

I decided to configure a query in vRLI for this issue to have it in a dashboard, and also set up an alert.

For other components in my homelab I have used Slack for notificiations. I checked the vRLI notification configuration and found that I could set up an outgoing web hook which fits well for Slack.

I tried to configure vRLI to send directly to my Slack web hook integration, but couldn't get it to work.

A quick Google search led me to this article from VMware explaining that you need to have a little component in the middle that translates the vRLI alert to Slack.

Luckily VMware have created this little component for us and it was really easy to set up. It also works with alerts from vRealize Operations Manager. I pretty much followed the article step by step, the only thing I changed was using PhotonOS instead of CentOS. PhotonOS has docker preinstalled so it was actually a bit easier.

Of course, if you already have a container platform in your environment you could just spin the container up there and save the extra VM.

Please note that the webhook-shim container comes with no official support from VMware. Use the Github repo for issues and feedback

Installation and setup

After doing a fresh install of PhotonOS I started the docker engine, pulled the webhook-shims container image from dockerhub and started the container. I started the container with the --restart=always parameter which ensures that the container will try to restart it self in case of any failures, OS reboot etc.

Let's check out some screen shots of the configuration:

Before configuring vRLI to send an alert to your webhook container be sure to have your webhook details for Slack ready. This can be found in the Slack administration pages for your workspace. It will look something like: https://hooks.slack.com/services/T11aaa/B222bbb/c3d4e5f6.

You'll need the pieces following https://hooks.slack.com/services for your configuration in vRLI. The webhook URI you configure in vRLI will be something like:

http://<ip-to-container>:<port>/endpoint/slack/T11aaa/B222bbb/c3d4e5f6

Configure alert in vRLI

After creating a query grabbing the event(s) you want to alert on, you create an alert from this. Give the alert a name, put in the webhook URI and adjust how the alert should be queried

When the alert gets a hit it sends a message to your Slack channel.

Slack alerts

I've created a couple of demo alerts for testing here, one that triggers if someone starts a Remote console to a VM from vCenter, and one when someone logs into an vCenter with the administrator@vsphere.local user.

Summary

I really like that I now can have my vRLI sending alerts to Slack, as I have several other components sending notification there as well.

Of course the best would be if we could skip the small webserver component translating vRLI notifications in to Slack friendly messages entirely, but for me it's not a big deal. I also got to play a little with PhotonOS as well!

The webhook shims container can also handle translations to other services than Slack, check out the Github repo for more information. One of them being vRO which can help you to auto remediate things.

I'll be sure to test this out with vRealize Operations Manager also!