Practical Python Project: Web Scraper Prototype (Semi-Livecoding)

A beginner-friendly, project-based approach to learning programming. In this video you'll watch me build a simple Humble Bundle web scraper prototype in Python, as if I was sitting next to you and explaining every line of code as I write it.

This video includes mistakes and wrong turns, because they're a great opportunity to learn what the basic programming and software development flow in Python *actually* looks like in real life.

2:04 - Basic Python Project Setup
4:30 - Basic virtualenv Usage
5:38 - Start programming our prototype!
6:42 - Making HTTP Requests with requests (HTTP client for Python)
12:48 - Parsing HTML with Python and Beautiful Soup (bs4)
18:20 - Inspecting HTML/DOM Elements in your browser
27:54 - Designing the Target Datastructure
32:10 - Python List Comprehensions
37:55 - Designing our most common Datastructure Operations
47:39 - Advanced Python List Comprehensions
49:57 - Splitting the page into product rows (Solution)
59:45 - Running the prototype

[tauri]

This is the first video Im watching after wasting THOUSANDS of hours, wanting to, trying, fumbling around, and giving up, on programming a complete program in Python. Its sad how right you are. Start at page one people! They are the most important pieces of information in the beginning, regardless of what you can use in the book from the later chapters. Start at the beginning!!! Mind blown! SO much easier now...

[JasonBunnell]

"I know you're not going to listen to me, but after you've wasted tons of time you will listen to me..." good stuff ;-)

[moisixhoxhi]

Thank you . As a beginner in Python this was very helpful for me not only understand webscraping better but also create an idea about the thought process involved in developing a programming project. I would like to see more projects like this in the future.

[dalekteta]

Your mistakes is _the best_ part of this kind of lessons. Don't be sorry for them, we will make mistakes while retracing your steps to learn and being able to see where these issues are waiting is instrumental. I'd only ask if you could articulate your thought process, e.g. why are you looking for missing bracket in one case and search for "python dictionary enumerate" in another.

[rd]

This is the first video I've watched of yours, but can say your teaching style is awesome! The ramble and tangents are great and teach way more than other tutorials I've watched. The live aspect is really nice as we get to hear your thought process as you go along. Hoping your channel has more videos like this one. Subscribed!

[sandeeprawat]

This is the content i have been madly searching over youtube.This is the ideal way to learn for everyone.
Thanks Mate Subscribed

[jayceeharb]

Great vid lesson!!! Be sure to keep the mistakes(and corrections) in future vids.... they are powerful teaching tools

[othaderekhernandezpena]

I'm about 20 minutes in to this video and this is already my favorite video on Python Programming I've encountered. You got yourself a new subscriber my friend. Keep up the great work. I really really appreciate this.

[tylerjames]

Remember, 4 hours of googling can save you just 5 minutes of reading the docs!

[vitormeireles]

I tried to follow the tutorial, but it became quite hard because the website used as an example it seems to been changed, so the JavaScrip part became a challenge in the 'class' identification part. Despite this, the video is awesome. Thank's for all!

[suzukimasashi]

I am a beginner studying python and scraping, but it is very easy to understand. Thank you!

[LukeOfWales]

Fantastic video for beginner Python developers! Thank you very much, please keep up the good work.

[wongright]

Love this BS4+Requests MVP.
I paid up for your Udemy course and from the dozens of others I've seen I reviewed that your material was definitely top percentile in applicability.
Now also following you Patreon.

[wz]

Thanks to the people like you I can learn faster and become a better developer. Thank you very much Sir!

[iSonikYT]

Dave was right, i was wrong, i am sorry.

Jk. Thanks to you i finally made me code work properly!

[cryptows]

Thanks! I hope you will make a second video 😁

[keijo]

Great video! I'd like to see more like this in the future.

[cdnuzzo]

loved this! keep making videos just like these! finally got around to watching this.

[MaximatumSupreme]

This was such a great exercise for learning python!

Thank you for making this

[nibblrrr]

Thank you, this was very helpful! Reading the docs is important, but I often can't tell from them what functions are actually important for the most common use cases. Seeing the process of how to go about building this helps with that!
(It's also reassuring to see that even veterans have brainfarts & little screwups :D )