Apache logs parser with Python for absolute beginners | Python project

How to write an Apache logs parser with Python and export evaluated data to a csv file.
Detailed log parsing using Python tutorial with examples for absolute beginners.

Logs parsing is a pretty regular way to identify offending IP addresses during a DDoS attack on your website. To reveal the attacker you have to collect data about quantity of requests from each IP-address.

Follow me @:

======================================
📎️ The SOURCE CODE is available via Buy Me a Coffee:
======================================

✴️✴️✴️ Web Scraping course ✴️✴️✴️
is available via Patreon here:

or its landing:

Sample log:

Covered Python topics:

1. Reading text files (logs), writing data to files
2. Searching IP addresses with Python regular expressions
3. Counter() class from the collections module
4. Writing data to a csv spreadsheet with Python

✴️✴️✴️ Also can be useful ✴️✴️✴️

➥➥➥ SUBSCRIBE FOR MORE VIDEOS ➥➥➥
Never miss a new video.

This Python beginner projects is a simple Python log parsing project.

Apache logs parser with Python for absolute beginners | Python project

#python #pythontutorial #redeyedcoderclub

[Tamim]

Watching your video made me realize that the Counter class creates a dict out of a list automatically. In my scrip I for looped 2 lists to create 2 new dicts with the keys and values respectively. Now that I know the Counter class creates a dict out of a list automatically, you saved my script runtime by 10 Thank you! My script now runs within 0.32 seconds, previously it was 10.32 seconds.

[slimpever]

awesome tutorial. just perfect because iam learning regular expressions atm with "automate the boring stuff"

[RedEyedCoderClub]

What video should I make next? Any suggestions? *Write me in comments!*
Follow me @:

Help the channel grow! Please Like the video, Comment, SHARE & Subscribe!

[RedEyedCoderClub]

Watch next - Web GUI for Apache logs parser with Python/Flask framework -

[mahendramungamuru]

You helped me alot exact output and clear tutorial thank you very much

[devops-boot-camp]

Thanks you so much, it's really helpful

[tumushiimebob]

Thanks for the video. How do you write a parser and regular expressions for detecting XSS and SQLI attacks in Apache log files?

[pipi_delina]

awesome tutorial... I am looking from some on email downloading attachments based on regex...

[bugsbane]

TL;DR: cat /var/log/httpd/access_log | awk '{print $2}' | sort | uniq -c | sort -nr

I think it would be better if the parser were more flexible. If pattern 1 exists, then determine if pattern 2 exists within N lines - and if not then emit an error. - something like that would be more useful...

In this context, if one of those IPs received a 503 proxy error for a given URI - did they then get a 200 success on the same URI sometime in the next 10 minutes?

[vaibhavwadate]

can i add this video, extract signature from email body .and save the signature in csv file .

[carloscab]

can't download the sample log file

[davidleal]

how would you sort the ips by higher frequency?

[evidence-vsbd]

actually your video is like 'music' to my ears as most of the american guys just make 'noise'

very cool, simple and to the point.. thanks bro