How to scrape Understat for football data in Python with requests and BeautifulSoup

My favorite books for learning python and data visualization:

[alexbushnell]

The web scraping demo here is fantastic, very clear and easy to apply to other aspects of the website. Top man!

[braziliandre]

Thank you for taking the time to do this! Been wanting to learn it for a while but lacked the basic skills to start and run run by run. I'd be great if there was a way to just pick a team and start scraping their data from each game for a specific time period... Maybe there's already more work on this as well. Either way I appreciate it!

[inakigoya]

Hey man, excelent video!! I started a master in data science and i wanted to practice with something related with football. I will use this for my FPL team

[marianolambolla]

Great Video! Congrats! You could get the entire json converted directly to dataframe by doing:

import ast

[heina]

Just coming across and had to click that subscribe button. You're so informative I wish you were my prof 😂 awesome work man!

[henriquefriedrich]

Superb content man! Btw I have good memories of Barcelona, my team (Internacional) defeated them in 2006 with Adriano Gabiru's goal.

[richardogujawa-oldaccount]

Thanks McKay, learned a lot from this!

[andreascalleja]

Don't know if this has already been posted, but the nested for loops can be replaced with the following code:

for shot_event in data_home:
x.append(shot_event['X'])
y.append(shot_event['Y'])



And the same for the away team.
Much cleaner imo this way - No nested loops and no multiple ifs.

[chefjuan]

thanks man you saved few hours of my coding

[GuardianApe]

This had to be done, thanks for sharing your knowledge.

[surajshivshankar]

Excellent video. Keep up the good work!

[brandonflexer]

Great video! Have you found a way to iterate over the competitions to retrieve all match urls for each competition/season? Or given the structure of Understat we have to manually collect all of them?

[zoeksnarf]

Great tutorial, cheers McKay. Instant new sub!

[johnmoran]

My man! Unreal, helping me a ton rn!!

[sravanjs]

please do a video of scrap data and save to csv file for pizza, radr and other charts.
🙏

[Qwertythemouse]

As far as the transformation from json to pd.DataFrame is concerned that one also works :

# Combine 'h' and 'a' dictionaries into a single list
combined_data = data['h'] + data['a']

# Create a DataFrame from the combined data
df = pd.DataFrame(combined_data)

# Display the DataFrame
df

So, it does really create a full data frame from json, having that home/away parameter as a column. Then anyone could try his own cleaning wrangling or usage of understat data himself.

[SuperYash]

This is really helpful especially for someone starting with football analysis and getting stuck at the initial step of finding the right data. Is there a way to get pass or any event data in general from understat?

[joshcaldwell]

This is an awesome tutorial! Thanks so much!

[andrkevichandvetal]

Thank you very much, man! It is helpful for my graduation work in university

[BlueSkyGoldSun]

Nice. Where I can learn football analytics?
And is possible to land job in football analytics?