Alerting Rules for Node Exporter and Prometheus Down
Video Lecture
Description
We will create alerting rules from when Prometheus detects a Node exporter is not up.
And we will create an alerting rule for when Prometheus is down.
When a node exporter is down, then Prometheus will return a 0 for the up metric. This is easy to write an alerting rule for.
But when Prometheus is down, then Grafana will get a timeout. There will be no response from Prometheus at all.
We will need to handle the symptom, rather than the metric value, as is returned when a Node exporter is down.