How To Use ChatGPT To Fully Automate Web Scraping

Web scraping is the process of automatically extracting data from websites using scripts. ChatGPT is able to generate the web scraping script code for you.

The example of this tutorial is showing that ChatGPT is a great help when generating script for web scraping. In fact, we were able to just input our simple requirements into ChatGPT and as a result we received a Python script which is able to perform the task without the need of any adaptions. That is making web scraping a lot easier and gets you started in no time.

This is a CodingTheSmartWay tutorial.

---------------------
Musik:
Night Owl by Broke For Free is licensed under a Creative Commons Attribution License.

#chatgpt #webscraping #python #ai #openai #artificialintelligence #webdevelopment

[alkuzad]

For me, it did not show the potential of the tool. You set requirements and then instead of fixing it with ChatGPT you just removed it. Prompt has to be more descriptive and not just "parse the website" to get good results. Pip install command is incorrect but because you had it already pre-installed it works.

[mikecorbean]

Well, my CGPT buddy has been troubleshooting this w/ me now for about an hour.
I'm on my 4th version of code from my partner.
All 4 run successfully but the resulting csv file only contains the headers:
column A: "Movie"
column B: "Title"
Now she's suggesting we attack this one using the API - argh!

[JozefHLUBIKJr]

where do I put the code, pleas? I am not IT at all, BUT willing to learn this scrape skills as I considr it great stuff for us sales people. P.S. how would you suggest to scrape 1. all 5* hotels in New York and 2. car rentals in all Caribbean? Muchas gracias, joseph

[hamburger--fries]

This was super helpful - Like a gateway drug. I used it a little differently. I scrapped 2000 books from my companies site and then removed various html and other odd things that occured in a OCR scan years ago. Anyway thanks!!!!

[MyMindGeek]

I am feeling dizzy after this video. Still too complicated and unaccessible to the layman.

[ryandunner]

Hey!

I am looking to scrape data from a website, specifically news articles from a few sources I have. I want to scrape this data and then have it automatically plugged back into chat gpt for further prompting, is that possible?

[dr.samuelburbanks]

How do you get to the terminal. so many steps left out.

[ahmadmerei]

Thank you for sharing but still want more details on how I can learn this step by step? any reference, courses?

[yaakovk]

it's can work with linkedin to or google search ? for emails or more ?

[bigd]

Hello, can you please show me how to get just the actors from all the 250 movies without including the directors? Thanks

[TheCentoru]

Sorry mate, but the error fix at 8:15 makes this approach as NOT fully automated

[Peepstep]

How to about getting contacts from sites like jist dial india mart..make a video about getting contacts from websites use chat gpt

[comptegmail]

do you know how to scrape data frmo glassdoor

[dr.samuelburbanks]

You are skipping important steps like what is the app that you used to paste the code in, yes you said it but I didn't understand what you said.

[adied]

Hi thanks for the video,
How to visualise this data in wordpress?, please I'm trying to build a we scraper or url parser... Any ideas much appreciated thanks

[alphalobo]

To me this is ridiculous. You start off saying we will not need to know any code, and then we are up to our elbows in code!

[MarblelessMedia]

This is no longer working. I'm told it goes against internet policy.

[PranavPatel-oz]

What if BeautifulSoup and openai prompt to be used to analyse the data and fetch the information what is required.
any one has tried out.

[bifrostbeberast]

People asking stupid questions in the comments without realizing that they get an instant answer if they would ask ChatGPT, which is the very topic of the video. MAybe it's better those people don't know how to do the stuff they ask for. It's like giving a monkey a gun.

[shivamkk]

Basically Taught how to use chatGPT... That wasn't what I came here for