PromQL Example Queries
Video Lecture
Description
Now that we have a few scrape targets, it is time to delve into queries.
The query language used in Prometheus is called PromQL (Prometheus Query Language).
The data can either be viewed as a graph, as tabled data, or in external systems such as Grafana, Zabbix and others.
Example,
scrape_duration_seconds
This shows multiple time series results. 1 for each target,
we can filter for 1 target by including either the instance
, or job
labels
scrape_duration_seconds{instance="localhost:9100"}
Regular Expressions
We can also use regular expressions.
Go back to the console view and query for node_cpu_seconds_total
We should get about 8 time series results.
Let's filter for everything with mode
containing irq
node_cpu_seconds_total{mode=~".*irq"}
All regular expressions in Prometheus use the RE2 syntax.
Data Types
Scalar
A numeric floating point value. Examples of scalars include -1
, 12.34
and 12345678.9
Instant vector
A set of time series containing a single sample for each time series, all sharing the same timestamp
scrape_duration_seconds{instance="localhost:9100"}
Range Vector
A set of time series containing a range of data points over time for each time series.
Return a whole range of scrape_duration_seconds
(in this case 5 minutes) for the same vector, making it a range vector.
node_netstat_Tcp_InSegs{instance="localhost:9100"}[5m]
node_netstat_Tcp_InSegs{instance="localhost:9100"}[1m]
node_netstat_Tcp_InSegs{instance="localhost:9100"}[30s]
Press the graph, we get an error saying that expression type "range vector" for range query, must be Scalar or instant Vector
So when in the graph view, remove the [5m]
range option.
Functions
rate(scrape_duration_seconds{instance="localhost:9100"}[1m:20s])
Start with
node_netstat_Tcp_InSegs
create a range vector covering 1m
node_netstat_Tcp_InSegs[10m]
filter for the job prometheus
node_netstat_Tcp_InSegs{job="prometheus"}[10m]
calculates the per-second average rate of increase of the time series in the range vector
rate(node_netstat_Tcp_InSegs[10m])
sum the 2 values
sum(go_threads)
Sub Queries
Start with this instant vector
node_netstat_Tcp_InSegs{instance="localhost:9100"}
Convert it to a Range Vector and then convert it back to an instant vector using rate
rate(node_netstat_Tcp_InSegs{instance="localhost:9100"}[1m])
Wrap it in the ceiling function
ceil(rate(node_netstat_Tcp_InSegs{instance="localhost:9100"}[1m]))
Convert it to a range vector and get the per-second derivative of the time series
deriv(ceil(rate(node_netstat_Tcp_InSegs{instance="localhost:9100"}[1m]))[1m:])
Useful Links
Google Regular Expression (RE2) Syntax
Prometheus 2.31 and Ubuntu 22.04 Notes
There are no considerable differences to be aware of as shown and discussed in the video in case you decide to install Prometheus 2.31.2 on Ubuntu 22.04 LTS.