Build a Web Scraper API with Puppeteer

preview_player
Показать описание


Ethical scraping encouraged!

🧰 Resources

#colbyfayock #webscraping #puppeteer #nextjs #api #webdevelopment
  1. Colby Fayock
Рекомендации по теме
Build a Web 0:15:23

Build a Web Scraper API with Puppeteer

Always Check for 0:11:50

Always Check for the Hidden API when Web Scraping

Beginners Guide To 0:07:36

Beginners Guide To Web Scraping with Python - All You Need To Know

Build a Web 0:23:26

Build a Web Scraper (super simple!)

A Guide to 0:09:54

A Guide to Web Scraping with Node.js

Web Scraping 101: 0:13:07

Web Scraping 101: A Million Dollar Project Idea

Industrial-scale Web Scraping 0:06:17

Industrial-scale Web Scraping with AI & Proxy Networks

Let's Build a 0:15:38

Let's Build a WebScraper in 15 Minutes with NextJS!

Episode 6: Scraping 0:06:35

Episode 6: Scraping Linkedin Ads Library

How to scrape 0:05:30

How to scrape ANY website with Apify’s Web Scraper API

Python AI Web 0:45:36

Python AI Web Scraper Tutorial - Use AI To Scrape ANYTHING

Scrapy Course – 4:37:09

Scrapy Course – Python Web Scraping for Beginners

Web Scraping Full 4:01:41

Web Scraping Full Course 2024 | Build and Deploy eCommerce Price Tracker

The Biggest Mistake 0:10:21

The Biggest Mistake Beginners Make When Web Scraping

A Completely New 0:28:00

A Completely New Way To Build Make.com Scrapers: Hidden APIs

🛑 How to 0:18:52

🛑 How to Scrape UNSCRAPABLE data! (super simple!) Node.js + API

Web Scraping Secrets: 0:14:46

Web Scraping Secrets: From Beginner to Pro with Make.com

Turn Websites into 0:05:13

Turn Websites into Real Time API's with ScrapyRT

Web Scraping Using 0:03:08

Web Scraping Using ChatGPT #openai #chatgpt #webscraping | extract data from website

Web Scraping like 0:08:17

Web Scraping like a GOD with Javascript

This AI Agent 0:17:44

This AI Agent can Scrape ANY WEBSITE!!!

Web Scraping with 1:08:23

Web Scraping with Python - Beautiful Soup Crash Course

Web Scraping with 0:08:03

Web Scraping with ChatGPT is mind blowing 🤯

This is How 0:18:27

This is How I Scrape 99% of Sites

Комментарии
Автор

Oh man! Thank you so much for this video!! I was struggling all day to get puppeteer running with nextjs, hetzner hosting and coolify! But with help of your video it works now without problems!

ThomasWapps
Автор

Hi, I have used it in different way, but this video is very useful for me

THATIPAMULAVENKATESH-lpwb
Автор

At 2:21 you can instead create a file named “api/scraper/…” and it will create the folders for you.

Also, running puppeteer on a serverless env seems kinda inefficient. Fetching that big S3 file, starting up the browser, then closing it again, etc. I would probably expose it as a separate service.

Anyways, great video! Was a pleasure to watch!

bthero
Автор

Hi colby, thank you for this tutorial, this was super helpful as I was stuck with the executable path issue on production, I tried the hosting the tar file to s3 bucket and using the path and it worked, but now It throws this error ```/tmp/chromium: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory``` in netlify serverless functions. would you know any solution or point me to the right resource to get past this issue? many thanks!

techhguy
Автор

Does this implementation work within Netlify?

RagnarSwanson
Автор

Puppeteer guys if selenium gotcha mad for hours MAN this thing powerful it can pull things that I gave up on for a while ago

OpiumSmoke