Scrapy Basics - How to Get Started with Python's Web Scraping Framework

Scrapy is a Python framework for web scraping and in this video I will show you the basics of how to start:

* Create a scrapy project
* Use the scrapy shell to find elements
* How css selectors work with scrapy
* Create a simple spider to crawl a website for product information

-------------------------------------
-------------------------------------
Disclaimer: These are affiliate links and as an Amazon Associate I earn from qualifying purchases

[pythonantole]

Oh my! This channel deserves more subscribers. I scrape a lot of tables in my job but never knew i could use pandas (had never it heard of it) until i saw one of your videos on Pandas. I look forward to more videos on scrapy now that i have the motivation to move away from BS4 and try scrapy.

[sampatankar]

Really lucid, well-judged in terms of content, and excellent videography. Timely too, given what I happen to be doing this week! Thankyou!

[WildRover]

a useful start. Followed along and got this working myself (which doesn't often happen when following python tutorials on YT). |Looking forward to finding out now how to get the stuff from page two and then hopefully finding out how to follow links

[hardwaregenie]

Thanks John for your tutorial. Really liked how easy and approachable you made it.

[irfankalam]

Nice one as always! Hope you would continue this as a series.

[nadyamoscow]

The best scrapy basic tutorial I`ve seen. Thanks a lot!!

[userZAKCXKC]

Something you note at the 9:23 mark is that you can close the space with a dot (or period). To add a little bit more to that. Regardless of the number of spaces, you only need one period. So close the gap completely and put one dot. I struggled with this for a while, as I had a custom class with 5 spaces (no idea why the coder would do that) in the name and it just never occurred to me that I could just use one dot. None of the documentation in scrapy indicated that. I spent quite a while trying figure that out.

[julz]

Dude I am loving your videos!! Opening up the wonderful; world of web scraping with these excellent Python tools. Thank you for the content ;]

[sagar]

Man you're awesome! These videos are so informative and easy to understand, wish you all the success in this world

[sinamobasheri]

thanks and nice work John 👌🏻 i was waiting for this in long time 🙏🏻

[JohnMusicbr]

I'm a big fan of your work. Thanks, John.

[kavehmoradkhani]

It tells the educational content very well
You're Great.
Thanks John!

[celerystalk]

Great job again John! I've never used Scrapy but now I feel it may be something really useful and powerful. It'd be great if you could do a video comparing the different scraping approaches you've introduced and their scenarios. Thx.

[martpagente]

Thankyou so much for this John, I hope this will become series.

[mahdi]

Thank you very much your content is awesome

[daniel]

as usual...great content...keep on the good work!

[engineerbaaniya]

Thank John please upload all videos for scrapy

[stephenwilson]

Great intro to Scrapy! Everywhere I've looked people say Scrapy is hard to learn, but frankly this seems more straightforward to me than BS. Maybe that's not the case when things get more complex, but that's just my two cents - maybe you're just better at explaining it?

I'm trying to scrape products and prices from Newegg and running into a road bump - I can get the item name and such, but the price is nested in a <strong> tag inside a list and finally a div. Any tips on selecting that?

[edbull]

Thank You for this fantastic training. Now I understand where scrapy is all about :)

[kimodataworld]

thank you very much wiht your help i did my first web scraping