[002.4] Monitoring and Alerting

Configuring alerts with HoneyBadger.

Subscribe now

Monitoring and Alerting [02.19.2018]

We are going to pick up right where we left off, at the HoneyBadger Dashboard Error.

honeybadger dashboard error

Now we're cooking with fire. Let’s dive into monitoring and alerting.

Uptime Monitoring

Before we move on to configuring alerts, let's take a look at setting up an uptime monitor.

Let's head over to HoneyBadger and go to our dashboard. Once there, click on the Uptime link in the top navigation.

uptime

Now, we should be on the Uptime Checks page for our app. From here, let's click Add an Uptime Check.

On this page, there are several settings we will need to configure for our check. Here's what I've set.

uptime check setup

As you can see, I've just chosen to name mine www.produciton.net, set the URL to point to https://www.produciton.net/healthcheck, and select the Check SSL certificate to check for invalid certificates. Everything else, we've left as the default.

If you've followed along in the previous videos, we used the aws-healthcheck gem to give us the /healthcheck endpoint that returns a 200 response.

This endpoint is also used by AWS to make sure our tasks are responding to add them to the target group for our load balancer.

Now, let's click Save Changes, which should redirect us back to our uptime checks dashboard. From here, we can see our new check in action:

uptime check dashboard

It might take a few seconds, but if everything is configured correctly, the check should be successful.

Great! Now we have an uptime check to make sure our application is running.

Alerting

Now that we're [logging our exceptions](Link to Previous episode) and monitoring our uptime, we've made some good progress toward getting an idea of how our app is running. But what happens when our app goes down or we push a new release and start having a spike in errors? Surely we don't want to have to keep HoneyBadger open or wait for emails or calls from our end users.

This is where alerting comes in. So let's set up some basic alerting to notify us of issues that arise.

Let's start by jumping back to our dashboard in HoneyBadger. Click on the Settings link in the right of the top navigation and Alerts & Integrations in the left nav. We should see a page similar to this:

alerts & integrations dashboard

First, you'll notice that we have a few alerting options in the Your Personal Alerts section. This is where we can configure email, SMS, and browser alerts.

If you click on Edit for email, you'll see that HoneyBadger has already set up some basic configuration for alerts by email. That's a good start, but I also prefer to get SMS alerts of uptime checks, and since I spend a majority of my time in Slack, I prefer to have notifications sent there as well.

Let's set up SMS alerts. Click on the Edit link for SMS alerts.

sms alert config

As you can see, there are several options for what type of alerts you can choose to receive notifications. You can go further and choose rate escalation, notification throttling, and error filtering. These options come in handy, especially when working with a larger set of services and multiple teams.

For my SMS alert, I only want to receive an SMS when our produciton.com app goes down and when it comes back up.

Once we've finished setting our changes, we'll click Save Changes, which should send us back to our Alerts & Integrations dashboard, but will also send a verification SMS to our phone.

All we need to do is reply with the code that was sent to our phone and our verification is complete.

That's it! Now we'll receive SMS alerts if our site goes down.

Let's move on to the last piece: setting up alerts for Slack. From the Alerts & Integrations dashboard, we can click on Slack in the Add a Project Integrations section.

We should be at a page similar to the configuration page for setting up our SMS notifications, but there are a few additional steps we'll need to take.

First, we need to click the Slack integration page link at the top.

slack integration page link

That should take us to the HoneyBadger Slack integration configuration. You might have to login to your Slack account first and you'll need the correct permissions to perform this for the Slack channel.

slack integration configuration

From here, you simply choose the channel where you want alerts to be posted, or create a new channel. We're using #errors and click Add HoneyBadger integration.

On the next page, we'll see some information about configuring HoneyBadger, but all we need to do is scroll down toward the bottom until we get to this section:

slack integration webhook

We just need to copy that integration URL and go back over to our HoneyBadger integration page and add that Webhook URL.

slack integration configuration 2

I'm going to leave the default setting for all of the notifications and scroll down and click the Test This Integration. If everything is configured correctly, we should be able to jump over to Slack and see that our test alert was successful.

slack test alert

Now, we can jump back over to HoneyBadger and click Save Changes and we're done configuring our Slack integration.

Summary

Today, We setup uptime monitoring and configuring how HoneyBadger alerts us via email, SMS, and the Slack integration, we we're easily aware of the current state of our app and any potential downtime.

We've only touched on some of the features offered by HoneyBadger, so feel free to head back over to their site to see a full list of all of the features they offer.

Also, it's worth spending time to cater your notifications to you, so they are meaningful and you’ll actually look at them when you receive them.

There's nothing worse than overlooking an alert of a serious problem because it got lost in a flood of other, less-critical alerts. When everything is a priority, nothing is.

Resources