Install Second Prometheus Node Exporter

Video Lecture

Description

I will install a Prometheus Node Exporter on a different server and connect to it using the main Prometheus service.

On the other server install it,

sudo apt install prometheus-node-exporter

Check its status.

sudo service prometheus-node-exporter status

It has created a specific user called prometheus

ps -u prometheus

It is now exposing the metrics endpoint on http://[your domain or ip]:9100

We can create a scrape config on the Prometheus server that retrieves metrics from that URL.

But since my new node exporter is accessible from the internet, I will block port 9100.

iptables -A INPUT -p tcp --dport 9100 -j DROP
iptables -L

And add an Nginx location to an existing website config I already have on this server that already has an SSL certificate bound.

sudo nano /etc/nginx/sites-enabled/YOUR-DOMAIN-NAME.conf

...
    location /metrics {        
        allow  ###.###.###.###;
        deny all;
        proxy_pass           http://localhost:9100/metrics;
    }
...

If you don't want to use the Nginx reverse proxy method, you could instead restrict access to the port for a specific ip address using iptables. Note that the messages will not be encrypted as they travel across the internet unless you provide another solution.

iptables -A INPUT -p tcp -s ###.###.###.### --dport 9100 -j ACCEPT
iptables -A INPUT -p tcp --dport 9100 -j DROP
iptables -L

Next, Go back onto the main Prometheus server and edit the existing scrape config for node and add the new metrics endpoint for the other server.

sudo nano /etc/prometheus/prometheus.yml

...
scrape_configs:
...
  - job_name: node
    # If prometheus-node-exporter is installed, grab stats about the local
    # machine by default.
    static_configs:
      - targets: ['localhost:9100']
        labels:
          host: grafana
      - targets: ['sbcode.net']
        labels:
          host: sbcode

Prometheus Course

If you want to try a more detailed course on Prometheus, then you can visit my Prometheus tutorials.