How to do Data Scraping in UiPath - Full Tutorial

This video tutorial shows how to do data scraping (table scraping) in UiPath RPA. You'll learn how to extract data from a website to Excel, including selector work, and then do advanced LINQ filtering.

- Join my network with 2500+ RPA Developers, where we solve RPA problems, network, and help each other upgrade our careers. It's free!

FOLLOW ME:

0:00 Intro
Data Scraping in UiPath is very easy. This full tutorial with a use case shows how to do table scraping and apply the best practices to ensure the correct result. When the result is ready advanced LINQ filtering is applied. The case is that we want to scrape all results from several Stack Overflow pages to Excel, but the same technique goes for all web pages

1:50 Navigate and validate
We navigate to our web page, here Stack Overflow, and then we apply some settings to ensure we navigate the data we want to extract. Using anchors for UI elements helps us make robust selectors. For example, to ensure we are on the right page, we find an element that changes when we reach the correct page and then use the While loop to make a check.

13:10 Table Extraction in UiPath
Use the Table Extraction wizard to do the scraping. The intuition is that we need to help UiPath understand the pattern. So, for each data type of the table, click Add data and then Finish selection. The Next button is used to scrape multiple pages. The result is stored in a Data Table, which we can write on an Excel sheet.

18:13 Apply LINQ to the results
We use LINQ, a potent query language, to filter the scraped results.

#rpa #uipath #scraping

[wissembencheikh]

Really thank you man, your tutorial was coming in the perfect moment, am a student and i have a little project to be done before this weekend and your tutorial is exactly what i need,
Without forgetting of course your wonderful explanation which is could finally prevent me from following the annoying contents of the Indian youtubers.

[arazdavlund]

Thank you Anders great video and wonderful explanation

[entertainmentgalaxy]

amazing ❤❤❤❤. can you just help me, when i select multiple pages it automatically moves to next page until the last one but then it does not stop and start it over again and again.

[srinivaschannel]

Great Video . Thank you for sharing. I need help. When i am trying to get the "Questions" by enabling the selectors and disabling the fuzzy selector . I am trying to click on Ui explorer option and its giving me "its blocked by group policy". Can you please let me know if this step is necessary or any workaround? thanks

[strixx]

Hey Anders, I am using UI Path as a part of my masters thesis and you helped me so much so far! Keep up the amazing work.

I am trying to scrape only data that has not been scraped before and write it in the first empty row to the same table. Trying to do this based on on your example of comparing questions before and after by reading the latest timestamp from the excel and compare it with the ones on the website. Every row of data has a timestamp with date and time (dd.mm.yyyy hh:mm). But i cannot get it running. Any ideas? Maybe this is already included in one of your videos, much appreciated if you could provide me the link.

Anyways thanks for your effort!

[andianders]

Great video! But I have a question: " I want to scrap results of footballgames and I need to split the goals. I don´t want "1-0", I just need the first number ( in this case the "1"). thx

[AtulKumar-iygw]

Hi Anders, I have a question, Please tell us in simple way how you made your uipath resolution(View) like this. I checked no one tells us as you are using the view, it's look good ?

[Umura]

Great vid Anders, can you help me? I tying to read multiple csv files, but when i run, ui path sends a error message "Acess to the path (path name) denied"

[NileshHaval-my]

Hi Anders, why did we compare strQuestionsBeforeSerach and strQuestionsAfterFiltering ?, values of these two are string type, What is the purpose of of Do while loop ?, We need only UiPath tagged questions from corresponding pages, Right ?

[jagannathramanan]

Is it possible to scrap the data from programs and features to excel sheet? That way admin can confirm all the software version is current.

[crashnt]

Is this the new version? I don‘t have these activities

[sourabhcharwande]

The table extractor is not able to detect some UI elements what should I do in this case?

[Artech.Ranjit]

How to achieve if there are click to download action one after one? ex: rpa challenge page.

[tudkumar]

Hi Anders,

Case 1: I tried by scrapping URLs and description for the same scenario using table extraction. I am able to get 684 links in the datatable.
"Read node Orchestrator object from a stringified file and use its function again" link is missing in the datatable. Actual result is 685

Case 2: I tried the same using Table extraction, by adding votes, answers and views counts as additional data.
The datatable contains only 485 rows

Case 3: I tried case 1 using Data scrapping. I am able to get 635 rows in the datatable
Case 4: I tried case 2 using Data Scrapping, I am able to get only 100 rows in the datatable

I tried to find the root cause, But I couldnt. can you please help me