How to read start urls in Scrapy from a csv file | Web Scraping with Python | Run as a script

preview_player
Показать описание
Demonstration of how to read a list of URLs from a CSV (in Python | Scrapy).

use each item in start_urls as the URL for each request made by the Scrapy start_request method.

===== Chapter times ======
0:00 Intro
1:00 Start coding
2:15 Crawler Process
3:56 Instantiate the spider
6:05 get the URLs to scrape ✅
13:02 successful spider
13:48 recap

Subscribe to the YouTube Channel
=================================

Follow on Twitter - to get notified of new videos
=================================================

Buy Dr Pi a coffee (or Tea)

🔥🔥🔥 Proxies 🔥🔥🔥
=================================================
If you need a good, easy to use proxy, I was recommended this one, and having used ScraperAPI for a while I can vouch for them. If you were going to sign up anyway, then maybe you would be kind enough to use the link and the coupon code below?

You can also do a full working trial first as well, (unlike some other companies). The trial doesn't ask for any payment details either so all good! 👍

◼️ Coupon Code: DRPI10
(You can also get started with 1000 free API calls. No credit card required.)

Thumbs up yeah? (cos Algos..)

#webscraping #scrapy #python
Рекомендации по теме
Комментарии
Автор

Guyzz!! He is a Superman!! ❤ believe me

lakchchayamdivyakhare
Автор

I watch a lot of Yatch videos and get the same chill vibe from this channel. lol

mykesmyth
Автор

Hello my friend, why do u code with sunglasses?! No i'm joking! Anyway what if I have to scrape a website that protect some information that I want to get with a pop up ? is like I've to go into the web page and press some button to make the content I want available... Should i use Selenium to set the pages or there's a way easier to get those information with scrapy?
Thanks,
Cheers

profga
Автор

I am getting an error as utf-8 codec can't decode byte 0x9d in position 54: invalid start byte

gowthampentapati
Автор

how to work with json file insteand of csv?

heenasolanki
join shbcf.ru