Automate web scraping with Python and AI (LangChain tutorial)

Large Language Models open powerful new options for web scraping. Where traditional scraping is fragile and context-dependent, now it's possible to create scrapers that are much more flexible and general-purpose.

In this video I show you an example of this using Python, the LangChain library and GPT models running on Azure. I also talk about the importance of optimizing token consumption when running these kinds of applications.

▬▬▬▬▬▬ T I M E S T A M P S ▬▬▬▬▬▬
0:00 - Intelligent web scraping
1:47 - Overview of the code
2:53 - Functions in more detail
8:19 - Running the script
8:50 - Monitoring Azure OpenAI