This Simple String Blocks Your Web Scrapers

➡ JOIN MY MAILING LIST

➡ COMMUNITY

➡ PROXIES

➡ HOSTING (Digital Ocean)

If you are new, welcome. I'm John, a self taught Python developer working in the web and data space. I specialize in data extraction and automation. If you like programming and web content as much as I do, you can subscribe for weekly content.

⚠ DISCLAIMER
Some/all of the links above are affiliate links. By clicking on these links I receive a small commission should you chose to purchase any services or items.

This video was sponsored by ProxyScrape.

[brentsaner]

"TLS-spoofing"?

Brother that is just choosing preferred ciphers. Who knew, all this time I thought I was hardening it turns out I was TLS-spoofing

[TheJFMR]

Thanks for talking about fingerprint. An important topic in the captcha bypassing field that few people knows.

[DamianL-oe]

Are you sure? Playwright OpenCV + Agents.. Scraping is inevitable; and it always will be.

[Master_of_Chess_Shorts]

Thanks for this great presentation of fingerprinting.

[ZbigniewLoboda]

0:15

I believe you should begin by explaining how this hash string is obtained from the communication between the browser and the server.

[tomvice]

Selenium-base has an undetected web driver, not sure if it spoofs this fingerprint

You can run in it within docker to not use your host machine's main browser

[aarontalua]

Can you make a video where you are using curl-cffi asynchronously? Will look forward to that. Thank you!

[NachoDLF]

Thank you, didn't know about hrequests and I had trouble with TLS blocking in the past.

[kexec.]

7:30 I believe curl-cffi developers maintain forked versions of curl-impersonate which has more updated version like chrome 124.

[michaelmueller]

Just do a little vimdiff for comparing the answers ;-)

[DanDart]

Interested in using the Haskell version of this once I have to start blocking everybody

[ivaldirbatalha]

Awesome vid John!

Being wandering to ask, since you happear to be using helix, how you debug python code? Expecially bigger projects/scripts devided into multiple files.
Seing the pbd but i'm checking there ain't anything else around

[InspireSphere_asdasfd]

Can you make a video on scrapping sites that require logging in to view the information?

[hpdipto]

Hi John, what font you have used in this video?

[secureitguy]

You don't have this issue with python selenium; right?
I wonder why you are neglecting a discussion of that.

I find using that in conjunction with beautiful soup covers most of the bases...

[reedgraff]

Have you made a video surrounding scraping websites using tor? What are your thoughts on this?

[ImAkuserareta]

hey bro can you do a vid about using rotating proxies with async httpx? the docs sucks. thanks

[doronsever]

How did you get the TLS reports of all the different type of requests?

[im]

Can one specialize on web scraping? Do that only to earn a living?

[thomasslone]

I'm just a simple bot coder, pretending to be what society tells me to be