Scrape any website with OpenAI Functions & LangChain

#ai #langchain #openai #webscraping #llm

[devlearnllm]

Additional details about scraping: only scrape for <span> tags on some sites (like WSJ or CNN) yields the best results. Others might be different.

[georgesanchez]

Refreshing to not see some bs clickbait video on LLM-uses. Just a clean, focused, and super differentiated walkthrough-video. Subscribed, and looking forward to more!

[emlincharly]

This video feels like a coworker showing me something cool. Really good video man!

[richmadrid]

This is exactly what I was looking for. A way to scrape websites like a human being, and done it via scripting.
Also, I like how you explain things clearly and how they work. I found this channel by accident, and decided to watch it. The next thing I knew, I'm a new subscriber!

[walkingwchris]

Well done for explaining the why so clearly . You had me in the first minute

[miltondavilaharjula]

Great video!! Thank you for sharing. I liked how you simplified the code and explanation. Your project really makes sense as webpages do change their structure and traditional approach may break due to those changes.

[alexanderroodt]

With AI assistance I can scrape hundreds of thousands of products/services a week and now have the facilities to talk to thousands of people at once. Learnt most of it from youtube from people such as yourself who are grossly underappreciated. Keep up the good work and thanks for sharing!

[devinschumacher]

You are now officially a real youtuber.

[tebblesfun]

Thank you so much! I didn't know how to implement this and I bumped into your video. Such a saver!

[techgeekguru]

Very cool stuff! Like the style of narration focusing on conveying information in a straightforward and matter-of-fact manner, without overemphasizing or exaggerating.

[emmanueladepoju]

First video and I like this channel already!🙂

[sandratoolan]

Good luck dude, just keep doing what you doing.

[diegosandoval]

🎯 Key Takeaways for quick navigation:

02:05 🌐 You can scrape websites using LangChain, OpenAI Functions, Playwright, and Beautiful Soup.
03:55 🧩 OpenAI Functions simplify web scraping by eliminating the need to manually declare HTML tags.
05:20 🛍️ You can use this approach to scrape e-commerce websites and extract specific information like item titles and prices.
15:41 🤖 LangChain simplifies interactions with OpenAI's GPT models for various applications, including information extraction.
23:32 ⚙️ Consider chunking large HTML content and building a FastAPI server to enhance this web scraping tool's capabilities.

Made with HARPA AI

[meinbherpieg]

Good video. Thanks for taking the time to explain the nuances in depth. You've got my sub ha

[lukeotwell]

That's a high quality vid right there.

[Wildhoneybush]

Great job, you are a real you-tuber and I can tell that you will become very popular. 😮🎉

[devlearnllm]

Just a slight correction: openai_api_key is a property in the llm object, in LangChain. It's not a global variable.

[StudioTatsu]

Note: 'kwargs' usually stand for keyword arguments. normally we call it "keyword args". Nice Vid. :)

[zgyqkhj]

Thanks a lot, you've earned a sub.

[MK-jnuu]

The beginning started mid sentence. Did I miss where you explained how ai will prevent us from rebuilding the scrape code when the website changes?