How to use INCREMENTAL REFRESH for Datasets (PRO) and Dataflows (PREMIUM) in Power BI

Get Demo Files here

In this video we cover how you can partially refresh your datasets and dataflows using incremental refresh in Power BI.

-

🚩 Support us to get perks such as demo files, early access and more!

MY FAVOURITE POWER BI BOOKS

MY GEAR

GET IN TOUCH

🌍 WHO IS SOLUTIONS ABROAD
Hi, my name is Fernan, the one-man band of Solutions Abroad. I work as a Data Scientist based in London, UK and have over 6 years of experience working in Business Intelligence. Solutions Abroad these days is dedicated to teaching the ins and outs of Power BI, through the power of YouTube.

If you want to learn more, I’ve conveniently created playlists to help you find the right topic for you.

🐎 First, watch this video about how a typical Power BI workflow looks like

🗺 Learn DAX to extend your Power BI calculations

📆 Stay up-to-date on all the features Power BI releases monthly

🧪 Get certified as a Microsoft Data Analyst Associate

💡 Learn Power BI Features to improve your dashboards
📊 WHAT IS POWER BI?
Power BI is a business analytics tool by Microsoft. It creates a working environment suited for normal users, meaning tasks such as data extraction, cleansing, analysis and storytelling; these are made easily-accessible to citizen-users, and extendable for technical users. At face value it provides visually-stunning, interactive dashboards, but it’s built on top of so much more.

🔻 Get started with Power BI Desktop for FREE

#PowerBI #DataAnalytics #BusinessIntelligence

[michaelarthurholloway]

You're using OrderDate both for the RangeStart / RangeEnd filtering on the table, and for the Detect Data Changes field. The documentation explicitly says you shouldn't do this, and that Detect Data Changes should be set on a different field: "This should not be the same column used to partition the data with the RangeStart and RangeEnd parameters." The docs don't clarify why it's a problem (e.g. what will go wrong), but I'm wondering if your example in the video might encounter issues because it's not following this guideline. Thoughts?

[AnshulMarele-gbaa]

Hi Dear,
I have power query table called "tbl_1".
I need to have another table called "tbl_2" sourcing data at "tbl_1" step no. 4.
There are other steps also after step#4. I don't want to duplicate table because it was decrease its efficiency. How can it be achieved without duplicating table?

[shanakaj]

I watched 2 videos in couldn't understand but your video is so precise!! THanks

[pthapa]

In order to understand incremental refresh, we need to be able to understand answers to these questions.
1. What is incremental refresh?
2. Why do we need it or what is the benefit of using this?
3. What are the license requirements to use it?
4. What are the draw backs of using it? What are its limitations and such?
5. Which fields are we considering for incremental refresh if there are more than one date field in the dataset?
6. What if dataset has date instead of date time? Will it create an issue?
And, there might be more unanswered questions to it which I have not mentioned above on my lists.
Thanks

[rizzrak]

this only works for 'date/time' selected, 'date' doesn't work. so you really have to follow the steps to get incremental refresh running.

[berknoyan]

thanks for the content fernan, i have one question which is very confusing to me. some vids on the web says you need directquery / queryfolding to use incremental refresh logically otherwise with import mode your incremental refresh is meaningless. however on incremental refresh page powerbi says "you need import mode to apply incremental refresh" i got confused. could you help me?

[norhashikinmohdyusoff]

my "parameter" is dimmed in filter rows. is it because im not premium license?

[etherlords]

I love this guy! Unlike that Yoooo one from guy in cube! He good too but this guy is to the point from the first second of the video!
Hugs and kisses!

[SanyuktaSuman-umpd]

Hi Fernan for this awesome Tutorial.
I need advise from you.
I have a dataset in BigQuery that changes every minute (new rows gets added every minute). The size of the dataset is around 13 GB. I need to find way to refresh my data every minute, but not the whole data set. As far as I understood, incremental refresh setting does not refresh the data every minute. Is there a way I can solve my issue?

[lucasamorim]

Hi, How to know if my incremental refresh is working in DataFlow?

With Power BI Desktop I can connect in SSMS with EndPoint XML, but how to do this with DataFlows?

Can you help me please ?

[hemamaleni]

I have a fact table with start_year (2020) and end_year(2023) parameters in it to limit the number of years worth of data. On top of this I have incremental refresh of last 13 months. I publish this file to web service and change the start_year parameter say from 2020 to 2019. Why does the parameter changes in web service doesn't take into effect? If I change from 2020 to 2021, it does make necessary changes but for the first time only after publishing the file. Any thought on this? My report should only hold only last 5 years of data with last 13 months of incremental refresh.

[mikedemon]

Hi,
My "Apply All" button is greyed out.. I can't click it... What could be the issue? Thanks...

[a-rock-]

I have a staging workspace where I set up a dataflow that imports data from an on-prem SQL Server database. On one table I set up an incremental refresh and it takes 30 seconds to update.

In a separate workspace dataflow, I linked to that same incrementally refreshed table. However, it takes 3 minutes to refresh.

The linked table has a Referenced table associated with it. Do I need set up an incremental refresh on the Referenced table, too?

[TechEduSphere]

Thanks, Fenan for this wonderful tutorial, quick question, will the Refresh button be clicked on Power BI service or Power BI desktop for the incremental refresh to work?

[amyperalt]

I have followed all the steps but when I check the next day it didn't refresh. and when we give incremental refresh to desktop will it not allow to refresh in services?

[thamizhansudip]

do i have to create dataflow to configure or enablle incremental refresh on power bi cloud service

[SajjalBajaj]

Can I use RangeStart and RangeEnd data type as Date as I have created date as date only? Also my dataset is in import mode and source is mysql db is it possible?

[paulpham]

Thanks for the video! How do we handle when in my SQL database the table has changed (new column, deleted column) and want to exposed those changes? Does this mean we have create a new power bi file and do a full refresh? Is there a way to avoid a full refresh?

[optimaxtb]

Very good video. The bit I am confused about is how it uses the 'order date' field to limit which rows it refreshes if that order date field only exists after the data has been transformed. For example my dates come in as a number (i.e. 9945684) which I have to convert into date format, so how does it decide if a row needs to be refreshed before it refreshes it? Does it look at the data it has gathered in a previous refresh and use that to exclude which rows are refreshed? Or does it apply the transformations to a single date column for all rows and use this to limit the refresh? Or am I missing something completely?

[readingexperience]

i load data from excel my new excel file contain only additional data that i want to append with existing on ... is there any way to do this