Python Part 15 - Scraping websites

By Andy Brown

This tutorial covers all aspects of extracting data from websites using Python: from the ethics and legality of web scraping, via the structure of a web page's document object model through to using the Python requests and BeautifulSoup modules to extract meaning from HTML.

Chapters
00:00 Topic list
02:03 Problems you may encounter
06:52 Our example HTML
08:20 The Document Object Model (DOM)
11:08 HTML tags and attributes
12:52 Element ids and class names (CSS)
18:37 Getting HTML from a website (requests module)
22:26 Getting HTML from a file
23:45 Getting started with BeautifulSoup
27:18 Chaining elements
30:43 Navigable strings
33:54 Navigating the DOM
37:52 Finding elements
46:11 Using CSS selectors

If you'd like to help fund Wise Owl's conversion of tea and biscuits into quality training videos you can join this channel to get access to perks:

[wayneedmondson]

More Python fun! Thanks Andy. Thumbs up!!

[Teaseweb]

<property object at error while scraping the website. Please help

[kennethlam]

The web site is down at 19:00. Any help?

[Teaseweb]

Traceback (most recent call last):
File "Desktop\python project\ve\fproject\WebScapingReal.py", line 7, in <module>
print(Response.text)
File "C:\Program Files\Python310\lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input, self.errors, encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 5379-5381: character maps to <undefined>