Skip to content


 Zabbix
 Grafana
 Prometheus
 Threejs and TypeScript
 SocketIO and TypeScript
 Blender 3D Topological Earth
 Sweet Home 3D
 Design Patterns in Python
   
 Course Coupon Codes
Design Patterns In Python (Book)
     
     

Install Promtail Binary and Start as a Service

Video Lecture

Install Promtail Binary and Start as a Service Install Promtail Binary and Start as a Service Install Promtail Binary and Start as a Service

Download and Install Promtail Binary

Now we will create the Promtail service that will act as the collector for Loki.

We can also get the Promtail binary from the same place as Loki.

To check the latest version of Promtail, visit the Loki releases page. https://github.com/grafana/loki/releases/

1
cd /usr/local/bin


1
sudo curl -fSL -o promtail.gz "https://github.com/grafana/loki/releases/download/v1.6.1/promtail-linux-amd64.zip"

1
sudo gunzip promtail.gz

And allow the execute permission on the Promtail binary 

1
sudo chmod a+x promtail

Create the Promtail config

Now we will create the Promtail config file.

1
sudo nano config-promtail.yml
And add this script, 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://127.0.0.1:3100/loki/api/v1/push

scrape_configs:
  - job_name: journal
    journal:
      max_age: 12h
      labels:
        job: systemd-journal
    relabel_configs:
      - source_labels: ['__journal__systemd_unit']
        target_label: 'unit'

Test Promtail Works,

You can now test Promtail by running 

1
sudo ./promtail -config.file ./config-promtail.yml

Open a browser and visit,

http://[Your Server Domain or IP]:9080

and

http://[Your Server Domain or IP]:9080/metrics

After having a good look around to verify it works, stop the Promtail server by pressing CTRL-C.

Configure Promtail as a Service

Now we will configure Promtail as a service so that we can keep it running in the background.

Create user specifically for the Promtail service

1
sudo useradd --system promtail

Create a file called promtail.service

1
sudo nano /etc/systemd/system/promtail.service
And add this script, 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
[Unit]
Description=Promtail service
After=network.target

[Service]
Type=simple
User=promtail
ExecStart=/usr/local/bin/promtail -config.file /usr/local/bin/config-promtail.yml

[Install]
WantedBy=multi-user.target

Now start and check the service is running.

1
2
sudo service promtail start
sudo service promtail status

We can now leave the new Promtail service running.

Now, since this example uses Promtail to read the systemd-journal, the promtail user won't yet have permissions to read it. So add the user promtail to the systemd-journal group

1
usermod -a -G systemd-journal promtail

If you ever need to stop the new Promtail service, then type

1
2
sudo service promtail stop
sudo service promtail status

Note it may take a minute to stop.

Warning

If you reboot your server, the Promtail Service may not restart automatically.

You can set the Promtail service to auto restart after reboot by entering,

sudo systemctl enable promtail.service

Configure Firewall

When your Promtail server is running, it will be accessible remotely. If you only want localhost to be able to connect, then type 

1
2
3
iptables -A INPUT -p tcp -s localhost --dport 9080 -j ACCEPT
iptables -A INPUT -p tcp --dport 9080 -j DROP
iptables -L

Be sure to backup your iptables rules if you installed iptables-persistent in the last section. 

1
2
iptables-save > /etc/iptables/rules.v4
iptables-save > /etc/iptables/rules.v6

After blocking port 9080 for external requests, you can verify that local request still work using

1
curl "127.0.0.1:9080/metrics"

Warning

iptables settings will be lost in case of system reboot. You will need to reapply them manually,

or

install iptables-persistent

1
sudo apt install iptables-persistent

This will save your settings into two files called,

/etc/iptables/rules.v4

/etc/iptables/rules.v6

Any changes you make to the iptables configuration won't be auto saved to these persistent files, so if you want to update these files with any changes, then use the commands,

iptables-save > /etc/iptables/rules.v4

iptables-save > /etc/iptables/rules.v6

Troubleshooting

If you see the error in the promtail status

msg="error creating promtail" error="open /tmp/positions.yaml: permission denied"

You shpuld check the owner of the file configured in the positions section of the config-promtail.yml matches the name of the user configured in the promtail.service script above.

My user is promtail, and the positions is set as /tmp/positions.yaml so I set the owner.

chown promtail:promtail /tmp/positions.yaml

If you set up Promtail service with to run as a specific user, and you are using Promtail to view systemd-journal and you don't see any data in Grafana, but you can see the job name, then possibly you need to add the user promtail to the systemd-journal group.

usermod -a -G systemd-journal promtail

You may need to restart the promtail service and checking again its status.

If you see the error "found character that cannot start any token" than that is likely to meen you have a tab somewhere in the yml indenting one of the tokens. Replace it with spaces.

IPTables Cheatsheet