Log File Monitoring - Nginx Proxy HTTP Status Codes

Video Lecture

Log File Monitoring - Nginx Proxy HTTP Status Codes Log File Monitoring - Nginx Proxy HTTP Status Codes Log File Monitoring - Nginx Proxy HTTP Status Codes

Description

Monitoring Log Files - HTTP Status Codes of a Nginx Proxy

The file I monitor is located at /var/log/nginx/access.log

The default Zabbix user that the Zabbix agent user uses, does not have read access to many log files on the system.

You can usually add the zabbix user to a group to solve this problem.

The nginx access.log file can be read by the www-data or adm groups on ubuntu 18, so I add the zabbix user to the adm group.

To find out which groups a log file can be read by, for example, I typed,

1
$ ls -lh /var/log/nginx/

This tells me that the access.log file can be read by www-data and adm groups.

Then I check which groups the user zabbix is part of,

1
groups zabbix  

If it's not part of either group already, I then add it,

1
sudo usermod -a -G adm zabbix

and check again to confirm.

1
groups zabbix

After changing the zabbix users permissions, you will need to restart the zabbix agent.

1
sudo service zabbix-agent restart

I can read the most recent log file entries by typing

1
tail -f /var/log/nginx/access.log

You can also check this command works when using the zabbix user,

1
sudo -H -u zabbix bash -c 'tail -f /var/log/nginx/access.log'

I then created an item for the host, with settings

Key Value
Name HTTP Status Codes
Type Zabbix (active)
Key log[/var/log/nginx/access.log,"^(\S+) (\S+) (\S+) \[([\w:\/]+\s[+\-]\d{4})\] \"(\S+)\s?(\S+)?\s?(\S+)?\" (\d{3}|-) (\d+|-)\s?\"?([^\"]*)\"?\s?\"?([^\"]*)\"",,,skip,\8,]
Type of Information numeric (unsigned)
Update Interval 1m

The regex value that I copy into https://regex101.com is

1
^(\S+) (\S+) (\S+) \[([\w:\/]+\s[+\-]\d{4})\] \"(\S+)\s?(\S+)?\s?(\S+)?\" (\d{3}|-) (\d+|-)\s?\"?([^\"]*)\"?\s?\"?([^\"]*)\"

This regex can separate the values for both Nginx and Apache access logs.

The regex splits each row of the log into several groups.

The HTTP Status code is in the 8th group.

and since I am only interested in the status code, I can use the regex

1
^(\S+) (\S+) (\S+) \[([\w:\/]+\s[+\-]\d{4})\] \"(\S+)\s?(\S+)?\s?(\S+)?\" (\d{3}|-)

I can also create triggers to notify on

  • 101 Switching Protocols
  • 301 Moved Permanently
  • 302 Redirect
  • 304 not modified
  • 400 Bad Request
  • 401 Unauthorized
  • 403 Forbidden
  • 404 Not found
  • 405 Method Not Allowed
  • 500 Server Error

In this video I demonstrate creating triggers for HTTP 5XX errors and use count to detect 10 or more HTTP 404 Errors in 10 minutes.

How and whether you decide to trigger on HTTP status codes is up to you. The video just provides examples for you to follow.

Zabbix Agent Items

List of HTTP status codes