Skip to content


 Zabbix
 Grafana
 Prometheus
 Threejs and TypeScript
 SocketIO and TypeScript
 Blender Topological Earth
 Sweet Home 3D
 Design Patterns Python
 Design Patterns TypeScript
   
 Course Coupon Codes
Three.js and TypeScript
Kindle Edition
$9.99 $14.99 Paperback 
$29.99 $34.99




Design Patterns in TypeScript
Kindle Edition
$9.99 $14.99 Paperback
$19.99 $24.99




Design Patterns in Python
Kindle Edition
$9.99 $14.99 Paperback
$19.99 $24.99




Elasticsearch Filebeat

Video Lecture

Elasticsearch Filebeat Elasticsearch Filebeat Elasticsearch Filebeat

Description

I demonstrate how to setup a Filebeat service to read systemd logs.

Filebeat download instructions can be found at https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html#installation

I downloaded the debian package manager version.

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.16.1-amd64.deb
sudo dpkg -i filebeat-7.16.1-amd64.deb
cd /etc/filebeat
ls -lh

Enable a module for Filebeat to run.

Get a list.

filebeat modules list

I enable the system module

filebeat modules enable system

Now edit the filebeat configuration.

sudo nano /etc/filebeat/filebeat.yml

Update the address of your Elasticsearch server.

output.elasticsearch:
  hosts: ["<IP Address of your Elasticsearch Server>:9200"]

Restart and check status.

sudo service filebeat start
sudo service filebeat status

You can also disable a module

filebeat modules disable system

If you enable/disable a module, then restart Filebeat.

sudo service filebeat restart
sudo service filebeat status

Firewall

The Elasticsearch service may or may not have a firewall blocking this new filebeat from sending to it. If you used IPtables from the last lesson, then you can add another IPtables rule to allow the IP address of this new filebeat service to send.

So on my Elasticsearch server, I get the iptables rules line numbers.

iptables -L --line-numbers

I insert the new rule for my new Filebeat services IP before the DROP rule.

iptables -I INPUT 2 -p tcp -s x.x.x.x --dport 9200 -j ACCEPT

Persist changes.

iptables-save > /etc/iptables/rules.v4

Now we can set up a new data source in Grafana, or modify the existing and test it using the explore tab.

If you didn't use IPtables, but your cloud providers firewall options to mange your firewall, then you need to allow this servers IP address, that you just installed Filebeat onto, to send to your Elasticsearch servers IP address on port 9200. I demonstrate setting my firewall rules in the video.

I can verify that my Filebeat can send to my Elasticsearch server by making curl requests from the server running Filebeat.

curl "http://<IP Address of your Elasticsearch Server>:9200"

and to get the name of the new index created by this new Filebeat service,

curl "http://<IP Address of your Elasticsearch Server>:9200/_cat/indices"

IPTables Cheatsheet