How To Add LOGGING to Your Python Projects for Beginners

As you scrape more and more sites and save more data its a good idea to learn the basics of logging for you Python files so you have a good understanding of whats going on, where its going on and any issues that come up. In this video I'll show you the simplest way to get some good logging for your web scrapers.

If you'd like to support me:

-------------------------------------
Disclaimer: These are affiliate links and as an Amazon Associate I earn from qualifying purchases
-------------------------------------

[adnanpramudio]

I didn't know logging like this would be really useful, thanks john!

[sheikhakbar]

What I like about your videos is that they start from the basic principle, and gradually evolve to the practical application, while keeping the video short!

[apexlegendsclips]

Thanks for this you actually created half the script I need, I just need to figure out how toopen each product in a window and scrap the title, description and images from each

[silkogelman]

Great intro to logging in Python. Thanks John!

[waleedahmed]

Very useful stuff you are teaching on your channel. Thanks a lot!
Also do you have any best books to read about advance stuff in web scraping?

[gisleberge]

Very useful. Thanks for putting it out.

[mayuriwagh]

Plz answer I hv one doubt where have u written that message...u have written only message how does that line got printed in output ?

[rtxmax]

What can we do when the file output reaches a certain size and then we can write a new file. I used to see this kind of logging in my previous organization for their application server.

[dp]

I used everything above def but my log file is always blank. It isn’t giving me details on each step like the video does. Has logging changed since 2 years ago? Seems like i would have to replace my print steps with logging() or logger() steps

[multigladiator]

logs are so important. imagine having an application server running and some customer reports errors/something not working. Now with logs you can see what happened very easily and support, but with print statements this is not possible .

[ameurchabane]

Hi John, thank for all jour efort.
I have question, how van i open many windows of chrome whene we use selenium like do login for many of the same script but such login in difrent window

[PalataoArmy]

Working perfectly with python 3.9.13, log file contains all the error, info, later then using the exact same code with other working env using python 3.10.8 the log file is empty but console shows a lot of things. So what did i missed?

[hariprasad-zqbp]

Make vedio on scrapping interactive charts as the data is inside svg tags

[miss_tech]

video suggestion : scraping Instagram user urls Who are liking a post 😁✌

[DittoRahmat]

If I want to use logging for scrapy, where do I put this code ? The spider file ?

[Suresh_Solution]

Which code editor is used for this tutorial

[apexlegendsclips]

I am a total noob but I have this cool script idea if anyone would like to collabe

[albievowles]

for beginners, yeah right, how the hell are we supposed to know what is where and how it knows what to do from a 6 minute video on an amazon website hacking thing, what the hell???

[gskluzacek]

Don’t use logging basic config, don’t even tell people that it exists. 👎🏼

[june]

This is a terrible beginner tutorial even if you know python!