Python Web Scraping Tutorial Under 1 Hour - Using Selenium and Beautiful Soup

Note: a minor video ordering issue was noticed. I have fixed it in the timestamps description below.
0:49:39 Tracing Back XPATH Issues comes after 0:29:45 Interacting With HTML using Selenium, and not at the end.
Absolute Beginner Python Web Scraping Tutorial. 📚
This Tutorial is not only about Web scaping trivial harvesting, but also a step by step guide on many Challenges faced by Scrappers like:
- Passing HTML Between Beautiful Soup and Selenium
- XPATH is not found? Then let's trace back why.
- How to handle pages that does not change URL.

0:00:00 Reviews Harvesting Project
0:03:28 The Big Picture Explained On a Black board
0:08:57 Requesting a Web page
0:12:48 Extracting Nested HTML Elements With BS4
0:20:20 Extracting All Similar HTML Classes With BS4
0:24:31 XPATH Explained on Black Board
0:29:45 Interacting With HTML using Selenium
0:49:39 Tracing Back XPATH Issues
0:37:41 Passing HTML From Selenium To Beautiful Soup
0:42:30 Handling New HTML Elements

Check out Python Absolute Beginner Playlist 📃:

Get my Python All in-one course, including Python Basics, AI, Data analysis, UI, Web Development, Image/Video Processing, Games Development, Databases and tons of Hands on projects. 🎨🎁

[MrRamonZ]

Thank you very much for this video. Well done

[furqanamjad]

You also forgot to explain how you got these 4 lines of code :/

Look at time stamps 37:18... and then 38:03

partialPath=driver.find_element(By.XPATH, Xpath)
partialPath=driver.find_element(By.XPATH,
partialPath=driver.find_element(By.XPATH,
partialPath=driver.find_element(By.XPATH,

I'm not sure what elements you took the partial xpaths of

[usaneboltorez]

Hi. Great video. I faced an issue that reviews only from the 1st page are recorded to the list. How can i figure it out? Thanks!

[furqanamjad]

If i want to also get the star ratings for each corresponding review. How would you do that? Thanks!