Scrapy for Beginners - A Complete How To Example Web Scraping Project

Scrapy for Beginners! This python tutorial is aimed at people new to Scrapy. We cover crawling with a basic spider an create a complete tutorial project, including exporting to a JSON file. We scrape products from a online shop and get names and prices. Learn how to use the Scrapy shell to parse the data, and get text and "href" attributes from the HTML, as well as scraping multiple pages. This is a full how to from start to finish for your first Scrapy spider project, all in Python 3.

DISCLAIMER This contains affiliate links. If you use these links to buy something we may earn a commission.

[grahamfeeley]

I struggle to understand all commands in Python, however John has opened the door to me with his videos on scraping, Thank you John

[eddiethinhvuong]

yours isn't the first scrapy video I watched, but definitely the best one out there. Thank you very much

[apk]

Best beginners scrapy tutorial to date.

Testing prior to building the spider.

[navturn]

This video is quite "old" but still perfectly relevant. I discovered you channel recently and love it. Thank you.

[SyedShah-osck]

This is first time I came across John's channel. What an amazing beginners tutorial on Scrapy..., it is clear, straightforward with an actual example project!! What I really like is John's non-salesman's method of providing all the relevant information and professionally nav through the content.

Thank you John. cheers mate and keep making quality content.

[omidasadi]

23 minutes teaching, without a second interrupt, just can say wonderful my friend..!

[asmuchican]

One of the best channel to learn web crawling. Good audio and video quality and easy to understand.

[GlennCarnes]

Thank-you, thank-you, thank-you. I was reading a book on Web-Scraping but was totally lost as they short-circuited some of the vital steps in the process. This was a clear as day, and now I feel confident in pursuing the next level.

[cornelius]

To anyone struggling with setting things up, for this to work in 2022 you'll need:

- Python 3.8
- pip 22.2.2
- Scrapy==2.6.2
- requests==2.6.0
- pyOpenSSL==22.0.0

Than it'll work. Thanks for the awesome tutorial, really helpful.

[k.k]

holy lol, this was exactly what I was looking for. Actually I was struggling with some paid online course using scrapy and I looked up your playlist but couldn't find any scraping via scrapy and now here it is.

[vitalchance]

Again, excellent video! There are so many idiotic tutorials online where the authors seemingly do not understand neither terminology nor the process flow of what they are teaching. In this great example even the recursive scraping was made easy and elegant and John actually pointed out that this is recursive scraping which, in its nutshell, is a foundation of any real life spider. Thank you!

[willian]

Man, what an amazing tutorial, honestly
I watched some other videos about Scrapy but none of them could make their lessons clear
I was having no progress at all, until I came across your video
Thanks a lot and congratulations for your work

[ferilukmansyah]

I just heard about scrapy framework, this tutorial is easy to understand, I am very grateful

[littlehonda]

I only finish the beginner guide for python and your tutorial is amazingly easy to understand.
looking forward to more demonstration tutorial! Many thanks!

[mitchdask]

That's exactly what i was searching for!A well explained example of scrapy - simply amazing!You made me understand how it works!Many

[dystopian_]

You are the only Scrapy specialist that I follow in YT... hoping that you will keep sharing knowledge.

[victormaia]

I had already tried to learn scrapy and failed many times to follow the results from other videos, but I finally got similar resultsfollowing your steps, I felt I learned a lot, even with my mistakes, just had to use custom_settings and it runned perfectly.

[gianfrancodagostino]

Man great tutorial. Pretty straightforward. The additional tips like the -o and -O are just gold. Thank you.

[shantanuraj]

This is one of the best videos I have seen so far. Thanks

[AmodeusR]

Awesome video, it helped me a lot to understand Scrapy and how to do somethings I wanted with a personal project.