Web Extraction in Microsoft Power Automate Desktop - Full Tutorial

In this complete use case, you'll learn how to do advanced web extraction in Microsoft Power Automate Desktop. By the end of this tutorial, you will have a solid understanding of using Power Automate Desktop to build a complete web scraping project.

VIDEOS FOR YOU:

POWER AUTOMATE DESKTOP GROUPS:
- Join my network with 6000+ Automation/RPA developers, where we solve Automation/RPA problems, network and help each other upgrade our careers. It's free!

FOLLOW ME:

00:00:00 Intro
Download the course materials. We will create a complete web extraction flow that reads an Excel sheet, web scrape data, and write it back to Excel. Everything is done with Microsoft Power Automate Desktop.

00:02:28 Initialization
We use Regions to structure our flow well. Use variables for things that can change, like the project path or the URL.

00:05:17 Browser Automation
Navigate to the data we want to extract with a Launch new Microsoft Edge action, a Set drop-down list value on web page action, and a Populate text field on web page action.

00:09:42 Web Extraction
Web scraping is straightforward in Power Automate Desktop. We can automatically extract web data by drag'n'drop and clicking.

00:15:53 Excel and DataTables
We work with DataTables in Power Automate Desktop since it's faster than doing it directly in Excel.

00:24:02 Dynamic File Naming
We use the current date and time for dynamic file naming.

00:26:58 Regular Expression on the Extraction
Regular Expression is a series of characters that defines a search pattern. Next, we'll use Regular Expressions, RegEx, to extract specific data, e.g., remove unwanted text from the extraction.

00:35:58 Extract URL from Search Results
With CSS selector work we extract the URL from the search results with Power Automate Desktop

00:44:48 Extract Multiple Topics
We want to do multiple web searches and extract each search result into a separate Excel sheet.

00:54:00 Extract Multiple Pages
Each search has its results on multiple pages. Therefore, our Power Automate Desktop flow must move to each page and extract the data.

01:05:59 Merge Two DataTables
We cant join two DataTables directly in Power Automate Desktop. So instead, we'll create a workaround with a loop to merge the DataTables.

#powerautomate #powerplatform #microsoft365

[andersjensenorg]

Thanks for all your likes and support 🙌🔥

[shaunrowley]

This was an excellent lesson took me 6 hours to complete with rewinding back to ensure that i had got it right but has enhanced my knowledge of RPA and PAD as it combined webscraping, Excel, variables/datatables, CSS Selectors and RegEx all into one project - thanks again 👍👏

[carlosaguilarc]

WOW! Thank you so much, Anders! I had the exact same problem extracting the URL behind a text from a webpage my company uses in the aquisitions department! I learned how to navigate the selectors route like you did with so many DIV and other elements.

[Pedrohd]

Mr Anders Jensen, excellent teachings. like this guaranteed. Thank you so much for the lesson.

[ALVINMAN]

Another excellent video for learning. Great quality. Thanks

[robertgreen]

Thank you. I've just come across this. I seem to remember that PAD used to have an 'Extract Href:' option when selecting a web element but seems to have disappeared. Thank you for showing how to use the selectors to do this.

[d.finance]

Really enjoying you channel Anders. Thank you

[PitakYT]

Thank you so much, Anders! This is great training video. 😆

[hostile_technics]

Love your tutorials. Thank you for the great content

[RavikumarKondala-mkij]

This is a very detailed training video. Thanks a lot for your efforts

[Mohsin.Siddique]

Tried this, thanks alot for such a detailed video ❤

[sbisca]

Great, thanks so much for the lesson. Like guaranteed !

[grzegorz]

another great, and super useful tutorial

[ryanlebear]

This is helping me get up to speed with PA, I think I'm ready to tackle pulling in products to my store from the wholesalers site now. I just need to figure out how to drill down into each product page and retrieve more data. Veyr helpful, I hope you do more on web scraping this is so useful. Thanks!

[PaweW-tdwt]

Great job as usual!
Thank you for sharing your invaluable knowledge with us.

Based on your instructions, I did some web scraping from another website, and it wasn't as easy as scraping from Amazon. This website provides different results for the same search term 2 out of 10 times, showing sponsored ads (for the searched product) and other content at the top of the search results that cannot be automatically detected.

I implemented a solution where I tried to read various elements that appear during the search. I'm not sure if it's the right approach, but I managed to get the results into an Excel file. Additionally, I added a short VBA code that converts "http ..." to clickable links.

[lss]

It's good to know how to get the next button by using css selector. Get data from one datatable to another is a good one as well. Thank you for the video.

[wmferreira]

Amazing solutions on paginations 👏👏👏tks

[nandansubramanya]

Thanks a lot for putting in time to do it. I learnt a lot of new things on PAD.
I was surprised that put did not use pager on "Extract Data From Webpage", then when I tried it, I could not get it to work... so I understand why you did it.
In 38:52, if you right click "The Smartest Way to Learn Python Regex: Learn the Best-Kept Productivity Secret of Code Masters" and click inspect, it will take you to the exact SPAN/<a>. Otherwise clicking the little box-arrow icon on the top left of the "Developer tools section, and clicking "The Smartest Way to Learn Python Regex: Learn the Best-Kept Productivity Secret of Code Masters" will take one to the HTML code. I find this the fastest way to get to it.

[craigduran]

Thank you Anders. This is an EXTRAODINARY video. It seems daunting but I am going to try to duplicate your training.

[robertgreen]

Thank you. Please, do you have a tutorial on how to search a list of websites for a particular word/phrase and return those who do have that term included?