Python Scrapy Tutorial- 8 - Running our first spider ( web crawler )

In this video we will run our very first spider/crawler and finally scrape a website using Scrapy.

Next video - Extracting data using CSS Selectors

#python

[prawnydagrate]

If you are having errors like 'Unknown command: crawl, ' then do this:
cd <PROJECT_FOLDER>
scrapy genspider <SPIDER_NAME> <URL>
[EDIT FILE; REMEMBER TO REMOVE `allowed_domains`]
scrapy crawl <SPIDER_NAME>
This way, Scrapy creates the spider structure for you.

[pablohierrezuelomunoz]

best set of scrapy tutorials i've found so far. Thanks. I'll stay tuned

[yashpatel]

Your teaching skills are really good bro...never stop making videoes..you gonna be next corey schefer for india

[rishabhbhardwaj]

if you get this error
module 'scrapy' has no attribute 'spider'
then make sure s is capital in "Spider"

[rockyraunak]

Errors like * 'Unknown command: crawl, ' * then do this : first run this - cd quotetutorial . And after its execution write - scrapy crawl quotes (As said in video)

[leoguitargirl]

Wow, I created my first Python Bot with your Thank

[creativeworks]

awesome and clear video series. your videos are really helpful. thanks a lot sir.

[igormorozov]

Perfect. Your explanations are very clear. Thank you

[huxue]

my scrapy log doesn't show the DEBUG bit, just INFO, and then exits.

[chadvowels]

Hey, thanks for this! Really useful tutorial series.

[wordsgo]

Very very very helpful I love your videos !!!! You are the best Scrapy teacher ever !!!! :-)

[irishabhsahni]

Bhai maza aa raha hai. 😄
I learned Python few days back and man I'm getting all these without any fuss.

[jserenson]

Hi: Hopefully you are still around on youtube. Just curious do you have any resource files for your Python Web Scraping & Crawling using Scrapy tutorial series. If you had the source code files for all your tutorials that would be nice to have for reference. If you do how files how could I get them.

[Bihari_Chaman]

Unable to run this on my Macbook. Showing Error : ERROR: Spider must return request, item, or None, got 'str' i. What should I do? Print command is working on MacBook but not yield Why?

[mustanserbillah]

sir,
I, m getting the error "spider not found" although the path of the folder and virtual environment activation is all OK...

[Kispalkovich]

I get this error: ModuleNotFoundError: No module named 'quotetutorial'

[AnandSharma-ltwq]

Try:
from scrapy.spiders import Spider

class QuoteSpider(Spider):
.
.

[TranLamYoutube]

if you see this error:
sys.exit(execute())
because Twisted 23.8.0 unsupport.

Solve: you let install 22.10

>> pip install Twisted==22.10.0

[jonorl]

If you get a no module error, make sure you type scrapy with lower case, both on the import and on scrapy.Spider

[anjaligutunpalli]

sir it's showing like domain empty after running the program