Monitoring All the Things! on your Linux system with the Elastic Stack

Josh Rich

In this talk, we'll look at how you can easily ingest your Linux system logs and various OS metrics into Elasticsearch using Filebeat and Metricbeat modules. Modules are a new concept in the open-source Filebeat and Metricbeat tools made by Elastic. We can then visually examine both our systems performance and all events occurring on it over time with Kibana. This is a near complete open source monitoring solution for a Linux system.

Assuming the demo gods allow, We'll have a little bit of a play with our systems, by inducing CPU/memory load or spamming log lines to see it reacts in Kibana, and correlate the different sources of information together in a single Kibana dashboard, providing a relatively complete view of what is happening on the system.

Finally, anything missing we want to monitor or record we can do by writing our own Filebeat or Metricbeat module. So we will take a dive into the code to see how you can contribute your own Filebeat or Metricbeat module to these projects.

[KirillKovalevskiy]

Sound cool all of these API and tools... I spent two day trying to configure Topbeat when I realized that is not compatible with ElasticSearch 6.

[brenthcl]

Can you monitor a VM (a firewall VM like Opnsense) and have the logs in Kibana?

[sfincione]

Kayshing??? That has to be the single weirdest pronunciation of caching I have ever heard. =) And did he say "taint the demo gods" a few times? Temp maybe? I know... maybe it's the aussie accent... I'm a bit picky but what can I do? This is who I am.