Easiest way to scrape Infinite Scroll | Python + Selenium

Scraping websites which use "infinite scrolling" (meaning the website keeps loading items when you scroll down) can be extremely complicated. In this video I show you the easiest method which works on 99% of all websites!

⭐️ Copy the source code here ⭐

⭐️ The website we scrape ⭐

⭐️ Timestamps⭐
00:00 - Intro
00:51 - The website we want to scrape
01:55 - Install and import libraries
03:14 - Scrolling through the webpage and detecting the end
06:20 - Extracting the data and saving it in a txt file
08:37 - How to avoid scraping the website every time again?
10:26 - Extracting the fields from the webpage
15:34 - How to store extracted data to either an Excel or a CSV file?
20:47 - How to download the script for scraping websites with infinite scroll?

⭐️ Follow me ⭐

❗ Some of my videos contain affiliate links. When you click on these kind of links, I get a small percentage in case you make a purchase via the website. Be aware that this doesn't cost you anything - you never pay more than the standard retail price.

#webscraping #python #webscraper

[nitroneonicman]

Classic technique for sure. The real question is what are tricks to avoid running out of memory on pages like Twitter that have infinite content

[slurpasgasskagg]

Fantastic video, thanks!
Down to the point but still explaining, showing and going a bit further