Will AI Kill Traditional Web Scraping? (GPT4V + Mistral Medium Project)

Will AI Kill Traditional Web Scraping? (GPT4V + Mistral Medium Project)

👊 Become a member and get access to GitHub:

Get a FREE 45+ ChatGPT Prompts PDF here:
📧 Join the newsletter:

🌐 My website:

GitHub to this project:

AI-Jasjon GitHub:

00:00 AI Vision Scraping Intro
02:15 AI Vision Scraping Test 1
08:36 AI Vision Scraping Test 2
11:31 My Conclusion

[lightweightabsorbent]

Hmm, it's interesting that you're not utilizing the Python version of Puppeteer, as it essentially encompasses all the options you've chosen. I've experimented with it, and it does function properly. However, a notable drawback I encountered is that when the information exceeds the page size, issues arise, and certain details are omitted. While this could be addressed with additional methods, it might involve unnecessary API calls. Overall, it's a promising concept that I appreciate, but there's room for improvement as it stands though, it's much easier to use BS4 and send it all to the 128k model alone and have it summarize pages using an enforced function_call and send them back to your main agent to respond.

[quickcinemarecap]

01:32 Using Puppeteer for website screenshots and additional functionalities
03:04 Using sub processes for web scraping with Node.js
04:48 Using GPT to collect and analyze tech news.
06:15 Web scraping allows quick access to information from various websites
07:44 Apple discontinues Apple Watch sales, tech giant faces challenges with AI and acquisition plans change
09:13 Using AI to track and report on live sports games.
10:48 Utilizing Puppeteer for web scraping instead of traditional methods.

[NeuroNinjaX]

Here's what I see... GPT4Vision will access the web pages I want and render a new page without any ADS or irrelevant stuff. Only the robots will see the ADS, so the adblocker detector won't ever work again since no ADS is actually being blocked (but never clicked).
What do you think about that?

[Jtitor]

This awesome! Thanks for sharing. I just forked the repo and have a great idea I am going to try with this. Thanks so much!

[InquilineKea]

can it access log-in restricted forums and robots.txt?

[micbab-vgmu]

Kris - great project - I am glad that you started usb VS code:)

[MattJonesYT]

Awesome content! How would you go about making the equivalent of a local perplexity ai that has links to its references and so forth?

[SunoCorner]

I would like to see a tutorial on how to use Autogen Assistant UI. Thanks, I really like your videos.

[MrSuntask]

Great video! Why didn't you use puppeteer in python?

[philq]

Great video! I can see a use for this.

[neon_Nomad]

These voices overs will be the majority of links with friends channels

[seadude]

Was everything factual or did some hallucinations occur?