Extract Data from Website using VBA and Selenium

Learn how to extract data from websites using `VBA` and `Selenium` after Internet Explorer support is discontinued. This guide provides a step-by-step solution to overcome data extraction challenges.
---

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: Extracting Data from URL VBA getting IE not suppoting

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
Extract Data from Website using VBA and Selenium: A Step-by-Step Guide

Many users rely on Excel VBA to scrape data from websites for analysis and reporting. However, recent changes in web technologies have led to the discontinuation of Internet Explorer support on some sites, leaving existing scripts broken. If you're facing an issue similar to this with a URL, you're not alone, and there’s a solution!

In this guide, we will guide you through the process of adjusting your VBA code to extract data efficiently using Selenium when the traditional MSXML2.XMLHTTP method fails.

The Problem: Using Old VBA Methods

You may have been using a VBA macro with the MSXML2.XMLHTTP method to retrieve data from a website, like this:

[[See Video to Reveal this Text or Code Snippet]]

When the website changed to no longer support Internet Explorer, the above macro ceased working, leading to errors. To resolve this issue, we’ll switch to using Selenium.

Solution: Extracting Data using VBA with Selenium

Selenium is a powerful tool for automating browsers. Using Selenium in conjunction with VBA allows you to interact with web pages flexibly, bypassing issues related to deprecated browser support.

Step 1: Setting Up Selenium

Install Selenium: First, you need to install the Selenium VBA wrapper, which can be found on GitHub or through reliable sources. This will allow you to drive a web browser using VBA.

Install ChromeDriver: Download ChromeDriver that matches your version of Google Chrome and make sure it’s in your PATH.

Step 2: Write the VBA Code

Here’s the updated VBA code to correctly extract data from the website:

[[See Video to Reveal this Text or Code Snippet]]

Breakdown of the Code:

Driver Initialization: The Selenium.ChromeDriver is used to initialize a new Chrome browser session.

Fetching the Page: The get method navigates to the specified URL.

Extract Data: The FindElementByCss method locates the elements containing the title and the votes on the page.

Display the Results: The data is printed to the immediate window in the VBA editor.

Important Notes

Wait Times: If elements take time to load, be patient. The timeout parameter can be adjusted as needed.

Maintain Your Browser: Ensure your version of Chrome matches the version of ChromeDriver you installed.

Conclusion

By adopting Selenium in your VBA projects, you can enhance your web scraping capabilities and avoid pitfalls associated with deprecating technologies like Internet Explorer. This transition not only ensures that your data retrieval remains intact but also prepares you for future advancements in web technologies.

Now that you have a working solution, you should be able to extract data from websites without being hindered by browser limitations. Happy coding!