How to fit 200 Million rows in less than 1 GB in Power BI!

Quick question: what do you think, how much memory needs 200 million rows in #powerbi? 2GB, 1GB, 500MB? You're not even close!

Check how you can fit 200 million rows in only 80 MB, without sacrificing performance!

[drivetrainerYT]

Very thought-inducive, was listening in the background and had to check if subscribed already. Thank you

[SCoetzer]

Great, but those tables are tiny, only a couple of fields. The fundamental issue with POWERBI is its removing data from a SQL sever/DWH environment to store the data on its own environment and then process that data for visualization. While the SQL server (Synapse) sits with 60 clusters and numerous computing nodes, once it passed the data over to POWERBI, it takes a nap. Why ?

The architecture just duplicated data storage, data is processed twice. Why ?

My Spark pool is on constant holiday.
Synapse cluster is taking a vacation.

Where I need processing power is in visualization, processing, turning data into reports.

[DamienThorn]

This doesn't seem to work for more in depth visuals with a lot of unions in it . But it did work for simpler tables where all data is within the same table as long as i also turned of Totals column also.

[devinpikafire]

Very cool, this is extremely helpful. Well explained.
Thank you

[kumarprabhu]

what to do when I want to have high granular level details in my aggregated table...I want to create aggregated table with maximum possible drill down so that I can get all answers from agg table not go to fact table with if i add multiple fields my agg becomes heavy...how do we tackle that..please advice

[komanguy]

Thanks a lot for this tip. Very well explained

[jasonanderson]

Great tutorial !
I have a question, I notice that you have a date table in your model. Do you make it correspond to the date of your two other table ?

[mariamammar]

Can you have more than two hot/cold fact tables? Like a hot, cold, and coldest fact table? Working with ~300 million rows currently.

[Mega_world]

While connecting sharepoint excel to power bi getting error as the data in the preview has been truncated due to size limits. Pls assist i need to link the file to bi

[MoonShine-bscl]

Hi please suggest can i import 3-5 million records into power bi desktop from sql server... Is this possible for pro users. Thanks in advance

[rayemeric]

Great Stuff, thanks for the video! I have a question regarding this. Is it possible to make something like this with no aggreagation? I have a use-case where i want to have the last 2 months as "Hot" and the older data as "Cold", but because we show the data in a scatter plot with every measurement and limits up to the second, we cant aggreagate this. I tried appending the two queries (Import and Direct) but afterwards i saw its still import. Anyway, keep up the good work!

[pabeader]

Shouldn't there be a connection between the Hot and Hot Agg table? So that Power BI knows how to traverse from one to the other?

[pepper_lab]

How to handle like 2, 000 column😅😅😅😂😂😂??