Web Scraping Weather Data with Python

Here's a beginner level web scraping tutorial for you, scraping weather data from google. I am using requests-html & python - this is my preferred html parsing library as it gives a simple to use syntax and access to CSS selectors to make extraing elements very easy.

At the end of the video I talk a bit about why you should always use an API if there is one available for things like this, however scraping data this way for personal projects is just fine.

If you wish to support me, you can do so with any of these links:

-------------------------------------
Disclaimer: These are affiliate links and as an Amazon Associate I earn from qualifying purchases
-------------------------------------

[KhalilYasser]

Thank you very much. I had an issue with the result to be in English and I solved it by adding this to headers "Accept-Language": "en-US, en;q=0.9, ar;q=0.8"

[xilllllix]

i am becoming so good at python and web scraping thanks to all your videos! please don't stop making them. your channel is literally the only one on YT dedicated to python and web scraping!!!

[thelife]

The best and most underrated web scraping channel in YouTube

[vasudev]

Hello John! We are so lucky to have you in the python community. Keep going, John👍

[Old_SDC]

Works great! Thank you so much! I’m making an AI for my room and this is perfect. I’m using python 3.9.0 for anyone wanting to know if it still works as if that version

[xZzzit]

Great videos mate! I like how you keep it short and snappy, keep up the good work.

[loganfairbairn]

This was a fantastic intro into webscraping, thanks a ton!

[igordc]

Thank you John! Your video is clear and interesting to watch as always.

[decromax]

Nice, back to VS Code then!
My Raspberry pi weather station has had a spider crawling the met office twice a day on a CRON job for around 18 months now.
Use it on a dashboard for comparison with the station data.

[davidwisemantel]

Nice one John. Would be cool to see how you'd put those locations in a list and iterate in a for loop and save your results :)

[sunghin]

7:41 John says the selector is too vague, but actually ids are unique on the page, so no need to chain selectors there.

[bartoszjarczynski]

Wow, found everything I needed thanks, keep videos coming

[utkarshpandya]

Hi John.Great video as usual & looking forward to seeing you a solution that I have emailed you as a use-case.Thanks for python online learning.

[nrg-]

look instead of typing particular city name just put city=("current location") so it will search automatically instead of editing or input the city=" " value

[Jucrisr]

hi, It worked fine for the first time on colab. but when I ran the same code again, its showing temperature in Fahrenheit. Nothing is changed in the code. I am in london

[tiffaniejohnson]

Trying this in 2024 using python 3.12.5 and I’m getting the error lxml.html.clean module now separate project lxml_html_clean. Does this mean requests_html doesn’t work anymore?

[redaoutarid]

How to bypass request API limit of send.
I use VPN and proxy but I want to scrape 36k of pages. So I need 13k of IP's minimum.
Can you share with me a best solution for this case.

[juanignaciolopezlopez]

Hi John, I've been following for a long time. You do a great job. I've been trying to scrape alibaba in different ways. Could you make a video about alibaba?

[bextla]

Thanks for tutorial. Is it possible to get previous data? Not currently
May you share code link or in txt?

[randlyce]

is there a reason why you do the first one with span and then the second one with div class? can't you just use span for every single one? or maybe just use div for the first one as well?