Create Email Alert Contact Point
Video Lecture
Description
I want to send alerts using Grafana, but I need to first create an alert contact point.
In this lecture I will create a new channel for email alerts.
Before starting, ensure that your version of Grafana is 8.3 or higher. We will be using the newer unified alerting user interface.
wget https://dl.grafana.com/oss/release/grafana_8.3.3_amd64.deb
sudo dpkg -i grafana_8.3.3_amd64.deb
Next, you will need an SMTP server. Many corporations will provide an SMTP service for their staff. This same service can be used to send emails from Grafana.
You will need an email account in your email provider, admin@your-domain.tld perhaps may be good. You will also need an accessible SMTP server address, e.g., mail.example.com and a port, usually 25 or 465.
In this video, I already have a domain that I purchased from Namecheap. For an extra $1 a month, I can get an email service add-on that I can use to send and receive emails. I will configure Grafana to use this email service.
If you have a corporate email service that you already connect your work mobile phone to, then the set-up process and settings will be very similar. You will be able to use Grafana to send emails using your work email address.
Note
If you don't have any email service that you can use to programmatically send emails from, then you can try installing PostFix on your Grafana server. In recent times though, this option has become much harder to implement since many cloud providers now block outbound SMTP for new users to minimize the possibility of sending spam. I have instructions below on setting up a send-only SMTP server for Grafana.
Next to configure Grafana SMTP settings
sudo nano /etc/grafana/grafana.ini
Find the SMTP specific settings, uncomment and edit.
Below are some sample settings. Your actual settings will depend on the domain name of your SMTP service.
[smtp]
enabled = true
host = mail.your-smtp-providers-domain.tld:465
user = admin@your-domain.tld
password = """your-password-wrapped-in-triple-quotes"""
from_address = admin@your-domain.tld
from_name = Grafana
ehlo_identity = your-smtp-providers-domain.tld
Restart Grafana and check its status.
sudo service grafana-server restart
sudo service grafana-server status
Go back into Grafana and into your new Email notification channel settings, and try to send a test email to yourself.
Set up Send-Only SMTP using PostFix
If you don't have an SMTP server available, you can install a local send only SMTP server on your Grafana server itself.
sudo apt install mailutils
At the prompt, I choose Internet Site
After the installation is finished, I need to configure the Postfix SMTP server.
sudo nano /etc/postfix/main.cf
I edit 2 values at the bottom being,
inet_interfaces = loopback-only
inet_protocols = ipv4
Then restart postfix
sudo service postfix restart
I can test the SMTP server sends emails using
echo "This is the body" | mail -s "This is the subject" -a "FROM:root@your-domain.tld" your@email-address
Sending an email from the command line must work before you continue. "Work", means that you got an email in your inbox.
Next to configure Grafana SMTP settings
sudo nano /etc/grafana/grafana.ini
Find the SMTP specific settings, uncomment and edit
[smtp]
enabled = true
host = localhost:25
skip_verify = true
from_address = admin@[your grafana domain name or ip]
from_name = Grafana
ehlo_identity = [your grafana domain name or ip]
Restart Grafana
sudo service grafana-server restart
Go back into Grafana, into your new Email notification channel settings and try to send a test email.
Check your SPAM folder.
Troubleshooting
If you have trouble sending and receiving emails through Grafana, you can check the Grafana log file.
cat /var/log/grafana/grafana.log | grep email
Note that passwords need to be wrapped in triple quotes if they have the characters # or ;
If you have set up a local SMTP server using PostFix, then also check my SMTP Troubleshooting page in my Zabbix course at https://sbcode.net/zabbix/setup-smtp/#troubleshooting
Grafana Cloud
If you use your own Grafana Cloud deployment, then an SMTP server will already be configured for your use of receiving alerts from Grafana.
Add your own email address as a recipient, and press the Test button.
While it is important to know the inner details of managing your own Grafana server, it can later be more strategic for your business to outsource the many aspects of it. Visit Grafana Cloud to start the process.
Benefits,
- Upgraded 28-day trial to Grafana Pro (vs. the standard 14-days)
- 3 users
- 10k metrics
- 50GB logs
- 50GB traces
- Automatic updates
- 30 notifications for OnCall
- 14-day retention
Sign up at Grafana Cloud