Install Loki Binary and Start as a Service
Download and Install Loki Binary
To keep this as simple as possible, we will install the Loki binary as a service on our existing Grafana server.
To check the latest version of Grafana Loki, visit the Loki releases page. https://github.com/grafana/loki/releases/
curl -O -L "https://github.com/grafana/loki/releases/download/v2.4.1/loki-linux-amd64.zip"
And allow the execute permission on the Loki binary
chmod a+x "loki-linux-amd64"
Create the Loki config
Now create the Loki config file.
sudo nano config-loki.yml
And add this text.
auth_enabled: false server: http_listen_port: 3100 grpc_listen_port: 9096 common: path_prefix: /tmp/loki storage: filesystem: chunks_directory: /tmp/loki/chunks rules_directory: /tmp/loki/rules replication_factor: 1 ring: instance_addr: 127.0.0.1 kvstore: store: inmemory schema_config: configs: - from: 2020-10-24T00:00:00.000Z store: boltdb-shipper object_store: filesystem schema: v11 index: prefix: index_ period: 24h ruler: alertmanager_url: 'http://localhost:9093'
This default configuration was copied from https://raw.githubusercontent.com/grafana/loki/master/cmd/loki/loki-local-config.yaml when it was version 2.4.1. There may be changes to this config depending on any future updates to Loki.
Configure Loki to run as a service
Now we will configure Loki to run as a service so that it stays running in the background.
Create a user specifically for the Loki service
sudo useradd --system loki
Create a file called loki.service
sudo nano /etc/systemd/system/loki.service
Add the script and save
[Unit] Description=Loki service After=network.target [Service] Type=simple User=loki ExecStart=/usr/local/bin/loki-linux-amd64 -config.file /usr/local/bin/config-loki.yml [Install] WantedBy=multi-user.target
Now start and check the service is running.
sudo service loki start sudo service loki status
We can now leave the new Loki service running.
If you ever need to stop the new Loki service, then type
sudo service loki stop sudo service loki status
If you reboot your server, the Loki Service may not restart automatically.
You can set the Loki service to auto restart after reboot by entering,
sudo systemctl enable loki.service
Note it may take a minute to stop.
When your Loki server is running, it may be accessible remotely on port 3100. If you only want localhost to be able to connect, then type
iptables -A INPUT -p tcp -s localhost --dport 3100 -j ACCEPT iptables -A INPUT -p tcp --dport 3100 -j DROP iptables -L
After blocking port 3100 for external requests, you can verify that local request are still possible by using,
Also, Loki exposes port 9096 for gRPC communications. This port may also be accessible across the internet. To close it using
iptables, then use,
iptables -A INPUT -p tcp -s <your grafana servers domain name or ip address> --dport 9096 -j ACCEPT iptables -A INPUT -p tcp -s localhost --dport 9096 -j ACCEPT iptables -A INPUT -p tcp --dport 9096 -j DROP iptables -L
iptables settings will be lost in case of system reboot. You will need to reapply them manually,
sudo apt install iptables-persistent
This will save your settings into two files called,
Any changes you make to the iptables configuration won't be auto saved to these persistent files, so if you want to update these files with any changes, then use the commands,
iptables-save > /etc/iptables/rules.v4
iptables-save > /etc/iptables/rules.v6
If you any of these errors
Loki: Internal Server Error. 500. open /tmp/loki/index/index_2697: permission denied
"failed to flush user" "open /tmp/loki/chunks/...etc : permission denied"
Loki: Internal Server Error. 500. Internal Server Error
Data source connected, but no labels received. Verify that Loki and Promtail is configured properly.
You should check the owner of the folders configured in the storage_config section of the the config-loki.yml to match the name of the user configured in the loki.service script above.
My user is loki, and the folders begins with /tmp/loki so I recursively set the owner.
chown -R loki:loki /tmp/loki
You may need to restart the loki service and checking again its status.
If when connecting to Loki using the Grafana data source configuration, you see the error "Loki: Bad Gateway. 502. Bad Gateway", this will happen if the Loki service is not running, your have entered the wrong url, or ports are blocked by a firewall. The default Loki install uses both ports 3100 for HTTP and 9096 for gRPC. See my
iptables rules above for my explicit rules which I've setup on my Grafana server that is also hosting the Loki service.