Ultimate Guide To Web Scraping - Node.js & Python (Puppeteer & Beautiful Soup)

Check Out Our Sponsor: Bright Data @BrightData

Code:

Timestamps:
0:00 - Intro
2:01 - What Is Web Scraping?
2:58 - Legal & Ethical Considerations
3:42 - History Of Web Scraping
4:41 - Why Use Web Scraping?
5:53 - How Do Scrapers Work?
6:57 - Tools & Libraries
8:32 - Basic Steps
10:28 - Additional Steps
12:15 - What Are We Scraping?
33:44 - Python Web Scraper

[thewalrusdragon]

Just wanted to point out how much I appreciate that you take the time to make the audio loud but clear in your vids man. It’s an overlooked thing by many content creators, and it just shows how much you care about the quality of, not just the subject, but the overall experience of your content.

[gochaavsajanishvili]

Wow I've never been here so early, ty Brad for your tutoring, your videos made me land jobs many times throughout the years! Forever grateful to you!

[eddieantar]

Damn Brad.. Im one of your Day Oners... offered you that Yankees hat many years ago.. Its so awesome to see you have 2.26M subscribers mans. I think you had that tech guy shop.... I appreciate all your tutorials man but i also valued your off topic videos talking about your struggles in your past and coping with difficulties life throws at you. Thanks for everything you do for the aspiring developer.. Im sure thousands of your views have been gainfully employed by watching your tutorials. Keep it up!

[andybabenko]

I've been watching you since 2014 - great respect. Good diction and presentation of material. Lack of water and meaningless thoughts. I recommend it to all my students. Thank you

[rheavictor]

Thank you so much Brad, it was so clear on how and why you did the steps.

Always a valuable lesson!

[golden]

dude its crazy how i was looking for this exact video days ago, and here you upload it, much appreaciated.

[zainulkhan]

The only tutorial you need for a start. Great work sir.

[ZenSchafer]

this is very good way to teach online. Some ytbers jumps directly into coding part, but you did absolutely good job

[abdullahCoder-Tech]

I really appreciate your efforts in putting out this content

[kareemgates]

How does HasData handle scraping challenging sites like the ones shown using Node.js? Curious if it has good bot protection solutions.

[kushi]

That‘s exactly what I have been looking for! Thank you so much for sharing this, helped me a lot.

[pedroernesto]

If you could make an updated video specifically on Dynamic Websites.... That would be awesome!!!! Most of the valuable data nowadays seems to be on dynamic websites and it`s so hard to deal with them.

[sathishkumarthangavel]

Thanks for the informative video as usual

[unclelukedigitals]

You're a king Brad. Currently viewing ❤

[IainEmslie]

To setup a Python environment on Windows do this:
python -m venv env
sourcee env/Scripts/activate

[marianivanov]

i tried and it works a charm. excellent!. I used zendrows scrapper didnt open the site cause cf. When i try to bright data its work as well great.I will move my all project to bright thank you all

[bharatdubey]

You have revamped your HTML, CSS, Javascript and Next.js courses. When are you planning to revamp your react course?

[Kertsu]

Hey Brad, your channel really helped me when I was just starting to develop apps using Angular. There aren't many updated tutorials available for the latest versions of Angular. Could I request a tutorial on Angular PWA?

[maueucifeely]

This was super helpful! Do you think HasData makes scraping with Node.js easier?

[socialkruption]

@ 18:58 I keep getting the following ""TypeError: browser.newPage is not a function" using node 22.15.0