Configure Alert Manager to Send Alerts from Prometheus

Video Lecture

Configure Alert Manager to Send Alerts from Prometheus Configure Alert Manager to Send Alerts from Prometheus Configure Alert Manager to Send Alerts from Prometheus

Description

We now configure the Alert Manager process to send emails when the alerting rules fire and resolve.

Edit Alert Manager Configuration

CD into the folder containing the Alert Manager config file called alertmanager.yml

cd /etc/prometheus

Backup the original configuration

cp alertmanager.yml alertmanager_orig.yml

Open alertmanager.yml and replace the full contents with this below.

sudo nano alertmanager.yml
route:
  receiver: smtp-local
receivers:
  - name: 'smtp-local'
    email_configs:
    - to: 'to@example.com'
      from: 'from@example.com'
      require_tls: false
      #auth_username: 'alertmanager'
      #auth_password: 'password'
      #auth_secret: 'secret'
      #auth_identity: 'identity'
      smarthost: localhost:25
      send_resolved: true

Check your configuration with the supplied amtool

amtool check-config alertmanager.yml

If all is ok, restart the Alert Manager service.

sudo service alertmanager restart
sudo service alertmanager status

To view an existing alertmanager configuration use

amtool config

You can also tail the Prometheus Alertmanager logs using the command

journalctl -u prometheus-alertmanager.service -f

In the email alerts, at the bottom, you get a link that points to the source metric that is firing/resolved. This url will not be configured by default. To configure it, we can change the prometheus default parameter --web.external-url I am using an Nginx reverse proxy, where I configured SSL and a domain, so I can set it to the external url to https://prometheus.sbcode.net.

sudo nano /etc/default/prometheus

Add --web.external-url to the ARGS= variable. eg,

ARGS="--web.external-url=https://example.com"

If you have existing settings in the ARGS parameter, then separate them using a space. eg,

ARGS="--web.enable-admin-api --web.external-url=https://example.com"