Effortlessly Scrape Data from Websites using Power Automate and Power Apps

Are you looking to scrape data from a website using Power Automate? In this video, I'll show you how to do it step by step. You'll see how to retrieve an HTML table from a website and convert it to XML and JSON. I'll also demonstrate how easy it is to pass that array to Power Apps, save it as a collection and display it in a Gallery. This is a great solution for those who want to automate the process of collecting data from websites and use it in Power Apps. Whether you're looking to automate data collection for business or personal use, this video has got you covered. Don't miss out on this opportunity to learn how to scrape data from a website using Power Automate. Hit that like and subscribe button and let's get started!

00:00 Intro
00:50 Using HTTP Action in Power Automate
02:33 IndexOf to get Table Tag Position
03:53 Retrieving the HTML Table
05:25 Converting HTML to XML and JSON
06:30 Reporposing an array using Select
12:40 Responding to Power Apps with an Array
14:30 Connecting Flow to Power Apps
15:18 Displaying Flow Array in Power Apps
17:08 Outtro

[geralddahl]

Thank you. I very much appreciate the extra time and effort you expend in order to highlight code in the upper right corner of the video as that code is being inputted into the program’s interface.

[alexbenedicto]

That's several hours of reading I just saved myself just by watching this video!! Thanks for this! Outstanding, indeed!

Great demo as always Damien 👍 Appreciate all your efforts publishing it. Expressions FTW 🙌 Great example of nthIndexOf

[kerstinsolbu]

Thank you so much! Worked it out finally with another web site, couldn't find yours. 🤓

[SnyderGator]

Outstanding video! You have a knack for explaining these topics at the perfect cadence and with just the right level of detail. Well done!

[dougydoe]

Another great video and demonstration. I found it extremely useful as I picked up a lot of tips using the various expressions. Thank you for sharing Damien.

[jpettersson]

Great video, very helpful to someone like me who just started using Power Automate.

[tylerkolota]

Thanks Damien,
I had someone message me on the forums asking for help taking HTML data and inserting it to Excel. I’m not sure it is directly off a webpage, but the HTML parsing to JSON here should be very helpful nonetheless.

[vaughnbassett]

Awesome video ! You are a great teacher !

[Jorgelhus]

Absolutely great video. Absolutely everything I was looking for in a single video with a very straightforward and clear speech and explanation.
You just got a subscriber and a preacher for your channel on all my work colleagues. This video was THIS amazing to me.

[innoviv]

Thanks for this, it is excellent! The table I'm scraping has just one column and it includes list items. So, in my Select step I've been able to generate nice clean code but only when a table row contains a single list item. Where multiple items are present I get "null." Here is what I am using in the Select map:
I'm trying to wrap my head around some sort of "for each" within the "li" output. Any ideas would be great!

[bobosbigblock]

I'm wondering how would you approach scraping the data from div elements? E.g. if the data table is in the nested <div>'s structure - how you would do it in Power Automate flow?

[rsb]

Hi I work in a UK bank and have to track new items and regulations that have been added to the FCA website. Using .

Ideally I want to be notified about new items on the FCA website eg by teams or email then I want a summary of the new item to update a SharePoint or excel. Using ms office or power automate how do I do this. I don't have coding skills

[StudioGoldsmith]

Than you! Using your tutorial I wrote myself an alert system that checks if I could get a damned parking space from my municipality. There is no line, only "snag them up once they appear!"

[roman_manro]

Thank you for this video.
Can you tell, is it possible to do similar scrapping with power automate for desktop?
Which adjustments necessary?

[joelluis]

Hi . Do you know how to enable the ''Response task" using power Bi as a trigger? I want to insert an url as a response to open a file in the browser.

It seems that I can't use "response" with power bi ''Buttons'' as a trigger and I want to open an URL from Power app. This url may change depending on the user who execute the flow but I'm just having issues trying to figure out how to open URL with Power Automate and Powe bi .

[OverclockMedia]

Looks like this website doesn't work anymore. It could be that the data is coming from some dynamic source. I asked chatGPT why I can see the HTML code with all the table data fine on Chrome's Inspect Element but Power Automate can only retrieve the headers of the table and it said the following: The issue you're encountering is likely due to the way modern web applications dynamically load data. Many websites use JavaScript to fetch and render data on the client side after the initial HTML is loaded. This means that when you inspect the element in the browser, you're seeing the fully rendered HTML, but when Power Automate fetches the HTML, it only gets the static content without the dynamically loaded data.

[Eddyyc]

Thanks for the video, how about dynamic content? i followed your steps but the view source is empty and not showing really any data but in the web works

[cmeo]

Any way this would work when a table has links?
XML not happy with '='

Example
<thead>
<tr>
<th>
<a <i class="fa fa-sort fa-fw" aria-hidden="true"></i></a>
</th>

[deepeshbansal]

what can we use if we dont have a premium subscription?