Excel VBA Introduction Part 57.2 - Basic Web Scraping with Selenium and Google Chrome

By Andrew Gould

Chapters
00:00:00 Topic List
00:00:50 The Example We'll Create
00:03:43 Referencing the Selenium Library
00:04:30 Running Google Chrome
00:06:30 Navigating to a URL
00:07:11 Using a Base URL
00:09:08 Typing Text into an Input Box
00:12:37 Clicking a Button
00:14:49 Checking if an Element Exists
00:18:51 Quitting the Web Browser
00:20:35 Setting the Timeout Parameter
00:21:51 Checking if a Button Exists
00:23:40 Returning a Collection of Web Elements
00:27:44 Looping Through a Collection of Web Elements
00:29:27 Identifying an Element by its Text
00:31:35 Counting the Elements in a Collection
00:34:01 Finding an Element Inside an Element
00:36:39 Checking if an Element was Returned
00:38:46 Writing an Entire Table into Excel
00:41:37 Creating a New Subroutine
00:42:28 Looping Over Table Rows
00:45:42 Looping Through Cells in a Row
00:47:35 Extracting the Text of a Table Cell
00:50:00 Writing Table Cell Text into Excel
00:53:29 Returning a Hyperlink from a Cell
00:58:31 Assigning a Hyperlink to a Range
00:59:56 Formatting the Table
01:01:50 A Basic User Interface

In this video you'll learn how to perform basic web scraping tasks using Selenium Basic and Google Chrome. You'll see how to open Chrome, navigate to a web page and interact with items on the page including typing text into an input box and clicking a button. You'll learn how to check if a web element exists on the page to avoid run time errors and how to set a timeout value. You'll see how to reference a collection of elements and loop over them to identify the ones you're interested in. You'll also learn how to write a table into an Excel worksheet using two different techniques, as well as how to extract hyperlinks from a web table and assign them to cells in an Excel worksheet.

[walkedintoawall]

The hidden gem of YouTube ! Andrew deserves millions of subscribers! So glad I found you. Please keep on providing awesome tutorials and spreading amazing knowledge to the world !!!

[blakepeterson]

FYI - I was able to overcome the Chromedriver version error by saving the downloaded version, per the error message, to Basic\Chromedriver.exe. I tried various other solutions to no avail.
Thanks for a great video! I'm overdue updating my Internet Explorer web scrapers. This is a big help.

[tg]

I have watched 2 times, the first time is typing the code when the video is proceeding and 2nd time is reviewing and digesting the code. Thanks for the valuable video.

[AI-ecqb]

I don't want VBA to disappear 😭 Thank you Google for keeping VB alive

[excluzive.]

Andrew, your videos are just priceless. I watched a lot of them and It gave me much when I started to study VBA. I am so grateful to you! May the Force be with you!

[a_stray_dog]

WiseOwl! Just wanted to take a minute to show my appreciation for this video. I've been following it for the last few days as I build my own web-scrapper to automate a rather lengthy procedure I do daily as part of my workflow. Originally I was planning on creating a separate web-scrapper with Python that I would then have to format using macros, but thanks to you, I'm able to store the whole process neatly into a single workbook! Plus, I learned a whole lot of new stuff about VBA along the way. You're a fantastic and incredibly engaging teacher, god bless!

[italochiarabini]

Thank you!! With this video you saved my 10 years work building a VBA automated scraper system based on IE. Over 10k lines of code! I look forward Microsoft great thinkers get what they deserve, as they got with Windows 8.

[tuyoexcelypowerbi]

Honestly, I think you're the best instructor in YouTube

[AI-ecqb]

WOW! A Real Excel Máster!
I watch for 1 hour straight! Best Tutor!
Respect!

[gaganrastogi]

Your videos are the best. I love the way you teach 😊.
I never felt that your videos are too long because of your teaching style.

wise owl 🦉 Is One of my favorite channels on YouTube.

I can feel the hardwork, you put into every single video. Like adding chapter, or explaining code in simple words.

I really like your efforts from my heart.

[bencole]

I literally could not have asked for a more specifically relevant video needed than this one I have just watched and completed if I had actually paid for it. It was quite simply perfect. This is all thanks to (in my opinion) one of the very best tutors I could have ever wished for in Andrew Gould. You make me feel about VBA and coding as Yoda made Luke feel about being a Jedi. Do or do not. There is no try.

[dlwndrms]

You have relieved me of a huge burden. In particular, it was easy to follow the flow of your lecture because it was explained using one web site and it could be applied immediately because you showed representative cases. Thanks

[metalworker]

Excel-lent video with easy to follow process logic. Thanks!!

[ComercioExteriorUV]

Hi Andrew, your videos have helped me so much, they have taken my career to another level :)
Before I was being flooded by Excel reports, and now with Macros I do the same job with a couple of clicks,
And this allows me to have time to learn even more,
Thanks again,

[arvinthnair]

Your videos have been excellent. Although i have a finance background, your videos helped me automate my work for my team. I got tons of praise for it. The VBA for internet explorer used to work and since it's being decommissioned, i would need to amend the code for Chrome. I could not find any material elsewhere but the best place to find it is your channel. Thanks Andrew! & i made a donation as well

[mkhalil]

I must say this is the best web scrapping video for beginners. Thanks a lot

[bk]

Wow! Thank you Andrew! This is by far the best tutorial I've come across online on this topic!!

[robinsparkles]

the way you teach is AMAZING it’s so hard to find a good tutorial on vba thank u so much i hope u share more about vba :) i love u

[jacundill]

Absolutely fantastic, these videos are so easy to understand and follow, already thinking of the many ways to utilise the skills and knowledge learned in this particular video. Big thanks!

[FlyHigh]

Uff My poor little excel dedicated computer took a while to scrape all 130 results on "excel VBA"
Thank you so much! On to the Task!