Python Scrapy Tutorial - 11 - Web Scraping Quotes and Authors

In this video we will be scraping quotes and authors from our website using the concepts we have learned in the previous python web scraping videos.

Next video - Item containers in Scrapy

#python

[chadvowels]

I'd like to reiterate that these videos are gold for beginners.

[popcorntimes]

I am crawling in internet for this python web scraping and find your great and awesome channel

[rhn]

yesss, by showing the _'oopsie'_ part make it much more realistic and relatable!! Lovin' it hehe

[evertonnascimentodeoliveir]

Your tutorial is helping me like no other has. Thank you very much!

[jeroen]

Thank you so much for these videos. I have just started to learn scrapy yesterday and now I am already able to scrape websites for prices of product for example. Is it just amazing. Regards from the Netherlands

[JahanzaibNiazi]

Explained in the best way. I have been searching for best web scraping tutorials and it found out your videos.

[AliRaza-viqj]

I watched so many videos on youtube searching but this man has brilliant, thank you so much for making nice tutorial i want you to make more and more videos on web crawling, I am waiting for that and thank you so much

[mmh]

yo bro, love your videos! thank you very much for doing this. As a starter, Xpath looks like a lot of crazy syntax with no extra simplicity.

[subburaman]

thank you for the knowledge share ...your teaching skill is great ....good playlist

[kmonish]

Hi, I tried with another scraping site and I am getting the following output,
{'author': ['Michael Hutchence',
'Sam Shepard',
'Jerome Powell',
'Bryan Adams',
'Keith Ellison'],

'text': ['I manage to scrape together a private life, despite the press.',
"You can't make a living as a playwright. You can barely scrape by.",
"With customers' permission, fintech firms have increasingly turned "
"to data aggregators to 'screen scrape' information from financial "
'accounts. In such cases, data aggregators collect and store online '
"banking logins and passwords provided by the bank's customers and "
"use them to log directly into the customer's banking account."],

}


can you tell me where I am going wrong?

[lukaslescinskas]

Thank you! This was very useful to me!

[nancyimran]

When I place [0] at the end of the line
all_div_quotes = response.css('div.quotes')[0]
it gives me a lot of errors. So when I removed it, I got {'author':[ ], 'tags':[ ], 'title': [ ]}. There are no values in them. Does anyone know why?

[kaiumkhan]

Hello @buildwithpython, My response.css doesnot show any output on anaconda promt.what can I do? Please solve my problem.Thanks.

[high-oncode]

Thank You sir for providing such a great content

[shrinarayanyadav]

U are awesome bro, I am your biggest fan love you bro

[randomsymmetry]

how do we know when to use "."? For author we use .author instead of span.small.author? I don't understand this part.

[oskarbrynkus]

Is it possible to only scrape a quote if a specific word is present? For example if the word "Harry" is present in a quote then it would only specifically scrape the entire quote where "Harry" can be found.

[johnvucharlotte]

Thank you for these wonderful tutorials!
I have written code similar to your example, but mine is different in the DIV class. The issue I am having is that the FOR loop returns all the values for each key. I would like only one key value per line. Do you have any suggestions?

[humayunkabir]

i did the exact same code, but getting AttributeError: 'str' object has no attribute 'css'

[KapilSharma-coxq]

3:36 why didn't u use
all_div_quotes