🛑 How to Scrape UNSCRAPABLE data! (super simple!) Node.js + API

__

⭐ New to code and none of this is making sense? Watch my '12hr+ YouTube Coding Bootcamp' in which you will learn HTML, CSS and JavaScript Fundamentals completely from scratch. It's on my channel and its 100% free.

You can also find me on:

[Hobbitstomper]

The best solution imo is NodeJS + Puppeteer + plugin.
It's free, doesn't rely on any 3rd party APIs and works 100% to avoid cloudflare blocking and other captchas. You can even log into any website, even if it uses OAuth for Google, Facebook, Amazon, Microsoft, Twitter, Apple etc.

[dystopian_]

See you in 3 days, Mother of Dragons.

[gaia]

Although there are many software solutions for automating and extracting data from a website, using NodeJS and its library ecosystem remains the most flexible option offering endless possibilities.

[hazemelbatawy]

great job I followed your steps and really it was fantastic, I am a data scientist and you impressed me. God bless you and if you need anything like Machine learning I am working on algorithms.

[silversolver]

Ania, fyi in case it might affect the algorithm: "unscrappable" should only have one 'P'—ie "unscrapable" :)
Also, not sure it's a common word.

[joseluisperez]

A little notecheck at 7:08, use the -D flag when installing nodemon, nodemon is just for development on this example

[shrikantjha]

Yes, I have also used cheeriojs with react native as an experiment and it worked well.

[irobot]

would love to see your computer setup, your desk, keyboard chair etc :)

[marekr.]

Pozdrawiam z Polski i życzę dalszych sukcesów w rozwoju kanału!

[philipbengtsson]

After viewing this video it would be interesting to see what we can do to prevent others from scraping our own website projects. 😅

[PySnek]

You're the best Ania! Thank you so much!

[BreakfastCupNoodles]

The problem with managed one is the cost. For custom one, you can pay for as low as $19/month for 100, 000 pages. It's also not hard to scale.

[drucifer]

Amazing content! I'd be curious how to scrape/store data in a database and use that for my own frontend.

[mgusa]

Killer look, light pink that's definitely you. Scrap 'UNSCRAPPABLE' data yeah I'm in, I'll be back, spoken in an Arnold hillbilly German accent. Love your stuff GO Ania.

[DevMadeEasy]

Great Content, as usual, thank you so much for sharing it with us, I know how hard is to build a project then edit it, post...
Thanks🙏

[NAHChannel]

Great video! I've had some better experience scraping using xpaths instead of classnames in sites which dynamically generate the classnames. But it seems to go down to the content being scraped. Scraping using CSS selectors seems to be faster also.

[js_models]

I wish there was an npm Ania command, because she is the total package. 😉

[ROVAKAN]

At the last video there was axios + express module, but i tried it on react result was CORS errors. Maybe this video is going to tell about that kind of errors and maybe about proxy set ups.

[briandsouza]

I think putting a premiere 24 hours would be better. This long wait feels annoying!

[Erwin_t]

WOW amazing tutorial! I love your style and your approach. I am starting web development. I want to learn Vanilla JS your way. What is the best practice to learn and retain the methodology of JS? Please help :)