GPT-4 Vision API + Puppeteer = Easy Web Scraping

In today's video I do some experimentation with the new GPT-4 Vision API and try to scrape information from web pages using it.

00:00 Intro
01:04 Basic usage of GPT-4 Vision API
05:50 Test GPT-4 Vision with image from Unsplash
07:23 Taking a screenshot with Puppeteer
12:35 Test GPT-4 Vision with Wikipedia screenshot
18:14 Test GPT-4 Vision with Google weather info
19:29 Automating URL generation + screenshot taking
33:24 Handling timeouts and retries and making it conversational
44:30 Summarizing BBC news
45:33 Fixing slow loading pages
49:18 Asking for weather information
50:24 Tweaking system message
54:03 Asking for Tesla stock price
56:00 Outro

[Lewis]

Didn’t expect a coding video to be this entertaining. Love the frank display of your thought process.

[arclife]

Your tutorial helps with the excitement and anxiety as a fellow dev. I knew I could do this myself but keep procrastinating and eventually some tasks end up as a mental block in WFH mode. Just forcing myself to watch a fella do something like this really helps, thank you!

[dustinsoodak]

I love how much of the process of programming he includes in the demo

[Autoscraping]

A fabulous video that has been of great help in orienting our new collaborators. Your generosity is highly valued!

[PostMeridianLyf]

Its interesting that this is exactly what I was looking for. Llast night i spent a few hours asking copilot how to implement the same libraries. Thanks for the tutorial

[Salfie]

I just wanted to tell you that you are doing great and I really like your format.

[mooktakim]

This guy has superpowers. He can talk and code at the same time!

[fuba]

This was super cool! Don't mind the long format at all. Would love to see you evolve this concept in another video.

[thecount]

Use the retry library and set a low timeout; you can use a simple decorator. If the timeout needs to be high and this isn't very pleasant, consider running multiple requests concurrently and waiting only on the first result.

[gaming_for_sanity]

Legend has it, he’s still trying to find out what the weather is like in Alaska…

[reunac]

This is so cool and nerdy! Maybe the best site to follow and learn more and more on OpenAI API. Difficult but entertaining to follow.

[cutecute]

This is awesome. I love your videos. Please keep these videos going specially this one. I learned so much

[grant_vine]

So for cookies you just need to know what cookie is being set, in many cases it’s likely just a matter of causing the same effect in puppeteer, one way is to add to the cookie store directly (I’m sure puppeteer has a way to do this), and an alternative is specifying a “user directory” for puppeteer so you can actually agree to things like cookies, in many ways consent popups are easy to “locate” using standard html locators simply because it is often set to a priority load event and is often a div/container with a name/id containing the word consent or cookie etc, so regex can be used to find these reasonably easy. Use puppeteer to locate the “Ok” button and click it and then having that reusable user directory means you only check for any site if you have or haven’t accepted consent, if not click it if so just scrape it

[gmichael]

Really appreciate your information and style. Learning much!

[ScootLogix]

Great video dude. Im gonna rewatch later. I got a project this might help on.

[robbennett]

Seriously impressive. I'm a NodeJS API engineer and you're writing that JS code faster than me!

[digitalcivilulydighed]

I'd like to see a video from you about navigating websites with Puppeteer. Now that you ask, I'd like a tutorial on how it follows links, fills out data, crawls four or more links deep into a website, how to handle session cookies, automate and run loops, etc. :-)

[albertwang]

very interesting, thanks for sharing!

[yoyartube]

Great Video! Can these libraries handle auth like azure oauth flow in order to browse to the page?

[mohamedbasueny]

i was wondering how this is different from the web-search capapblilty of chatgpt-plus right now .
in other words, if i asked gpt to look for an answer on the web will it struggle to do so ?,
is this a hack way to use a better websearch via an api like method because it's not enabled yet in the openai dev tools .
any way i really like the video, can we use selenuim to do so also ?