Install Loki Binary and Start as a Service

Video Lecture

Download and Install Loki Binary

To keep this as simple as possible, we will install the Loki binary as a service on our existing Grafana server.

To check the latest version of Grafana Loki, visit the Loki releases page. https://github.com/grafana/loki/releases/

cd /usr/local/bin

sudo curl -fSL -o loki.gz "https://github.com/grafana/loki/releases/download/v1.6.1/loki-linux-amd64.zip"

sudo gunzip loki.gz

And allow the execute permission on the Loki binary

sudo chmod a+x loki

Create the Loki config

Now create the Loki config file.

sudo nano config-loki.yml

And add this text.

auth_enabled: false

server:
  http_listen_port: 3100

ingester:
  lifecycler:
    address: 127.0.0.1
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 5m
  chunk_retain_period: 30s
  max_transfer_retries: 0

schema_config:
  configs:
    - from: 2018-04-15
      store: boltdb
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 168h

storage_config:
  boltdb:
    directory: /tmp/loki/index

  filesystem:
    directory: /tmp/loki/chunks

limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h

chunk_store_config:
  max_look_back_period: 0s

table_manager:
  retention_deletes_enabled: false
  retention_period: 0s

This default configuration was copied from https://raw.githubusercontent.com/grafana/loki/master/cmd/loki/loki-local-config.yaml and may change depending on any updates to loki.

Test Loki Works,

You can now test Loki by running

sudo ./loki -config.file ./config-loki.yml

Open a browser and visit,

http://[Your Server Domain or IP]:3100/metrics

Now stop the Loki server by pressing CTRL-C. Note that it may take a minute for the process to stop.

Configure Loki as a Service

Now we will configure Loki as a service so that we can keep it running in the background.

Create user specifically for the Loki service

sudo useradd --system loki

Create a file called loki.service

sudo nano /etc/systemd/system/loki.service

Add the script and save

[Unit]
Description=Loki service
After=network.target

[Service]
Type=simple
User=loki
ExecStart=/usr/local/bin/loki -config.file /usr/local/bin/config-loki.yml

[Install]
WantedBy=multi-user.target

Now start and check the service is running.

sudo service loki start
sudo service loki status

We can now leave the new Loki service running.

If you ever need to stop the new Loki service, then type

sudo service loki stop
sudo service loki status

Warning

If you reboot your server, the Loki Service may not restart automatically.

You can set the Loki service to auto restart after reboot by entering,

sudo systemctl enable loki.service

Note it may take a minute to stop.

Configure Firewall

When your Loki server is running, it may be accessible remotely. If you only want localhost to be able to connect, then type

iptables -A INPUT -p tcp -s localhost --dport 3100 -j ACCEPT
iptables -A INPUT -p tcp --dport 3100 -j DROP
iptables -L

After blocking port 3100 for external requests, you can verify that local request still work using

curl "127.0.0.1:3100/metrics"

Warning

iptables settings will be lost in case of system reboot. You will need to reapply them manually,

or

install iptables-persistent

sudo apt install iptables-persistent

This will save your settings into two files called,

/etc/iptables/rules.v4

/etc/iptables/rules.v6

Any changes you make to the iptables configuration won't be auto saved to these persistent files, so if you want to update these files with any changes, then use the commands,

iptables-save > /etc/iptables/rules.v4

iptables-save > /etc/iptables/rules.v6

Troubleshooting

If you see the error

"failed to flush user" "open /tmp/loki/chunks/...etc : permission denied"

or

Loki: Internal Server Error. 500. Internal Server Error

or

Data source connected, but no labels received. Verify that Loki and Promtail is configured properly.

You should check the owner of the folders configured in the storage_config section of the the config-loki.yml to match the name of the user configured in the loki.service script above.

My user is loki, and the folders begins with /tmp/loki so I recursively set the owner.

chown -R loki:loki /tmp/loki

You may need to restart the loki service and checking again its status.