Log Analysis with the ELK stack (Elasticsearch, Logstash, Kibana)

It’s your first day at the new job and your new manager swings by your desk to tell you about your first assignment. You’re going to be in charge of log management and log analysis. Your job is to consolidate the log output to a central location from sources all around the company, such as, web servers, mail servers, firewalls, database servers, etc. But as a starting point you’re going to be consolidating, managing, and analyzing Syslog events. Suddenly, that job offer at cousin Rickey’s Ready Lube doesn’t look so bad.

Instead of reaching for a grease gun, you reach for the ELK stack (Elasticsearch, Logstash, Kibana) The ELK stack makes searching and analyzing data easier than ever before. Using ELK you can gain insights in real-time from the log data from around the company.

In this presentation, we’ll explore how you can consolidate the syslogs into a central store and delve into each of member of the ELK stack. Then we’ll put it them together to view and analyze log data. Finally, we’ll look at how the ELK can be used to do forensic analysis. Yes, there will be a demo.

[JMichaelThurman]

Thanks for posting! I'm building an ELK and this has been one of the most useful over-views I've seen.

[WilsonMar]

[10:08] The list of alternatives to Splunk:
Graylog2
Nxlog
Octopussy
Logscape
ELSA
LOGanalyzer
Logwatcher
logHound
logReport
Logsurfer
PHP-Syslog-NG

[thelebbies]

Thanks for this overview sir. I work with Splunk now and I agree, both tools seem very, very similar. I will test ELK out and compare myself .

Appreciated-

[Melpheoser]

We are also seeing quite frequently SSH attacks from China (around 10 times a day)