Best tips when working with Power BI dataflows!

I have been creating quite many dataflows lately and in today's video, I am going to share my best tips on how to set them up and avoid common issues.

Chapters:
00:00 Intro
00:30 Is it worth to work with dataflows?
01:30 Managing gateways ...or not
02:30 Configure your own datalake
03:30 The Power Query Online experience
10:00 Computed entities
11:00 Save and refresh dataflows
14:00 Incremental refresh
14:30 Power BI dataflow connectors
Done!


SUBSCRIBE to learn more about Power and Excel BI!

Our PLAYLISTS:

☼☼☼☼☼☼☼☼☼☼

POWER BI COURSES:

Want to learn Power BI? How about you take one of our courses? Here you can find the available courses:

☼☼☼☼☼☼☼☼☼☼

ABOUT CURBAL:

▼▼▼▼▼▼▼▼▼▼

If you feel that any of the videos, downloads, blog posts that I have created have been useful to you and you want to help me keep on going, here you can do a small donation to support my work and keep the channel running:


Many thanks in advance!

▲▲▲▲▲▲▲▲▲▲


************





************


QUESTIONS? COMMENTS? SUGGESTIONS? You’ll find me here:
Twitter ► @curbalen, @ruthpozuelo

#CURBAL #SUBSCRIBE

[nevillewhite]

This is so spot on, Ruth! You nailed all my headaches and heartaches in one enjoyable video. Not only did you capture the technical issues, your face also captured the precise set of emotions I feel when faced with these challenges.

[seang]

I'm totally with you on the slowness. I did the same when I just work in desktop and copy to dataflow.

[vladsamoilov]

Ruth, thank you so much. I struggled with same issues and hope it will help others not to waste time and keep sanity. Should've watch this long ago. Your tips are golden.

[mcnater]

Dataflows had so much potential but are extremely frustrating to work with as you pointed out. I'm hoping they improve quickly as we'd like to leverage them more. I've run into everything you mention here over and over again. Great overview.

[stevenkramer]

Okay, I love dataflows - but all the issues you raise in the video are spot on! Especially the "Validating Queries" issue. The absolute worst. That gets a Golden Turkey for UX... but supposedly they are going to roll out a fix that allows you to save your work and let the validation run in the background.

[CyrilGagnaire]

That's where the privacy settings are !!! Thanks you so much for that video. I've encountered many of the issues you are referring to in the past.

[alexkim]

You have summed all my frustrations out so well. Earlier during my research phase, I was really convinced that Dataflows would make the entire process easier. I quickly reverted to Desktop. It's a really painful experience.

[shooben]

Thanks for sharing this! I'm experiencing the same issues. Very frustrating!

[stevenannegarn]

Happy dataflow user here. I wasn't very aware of all the workarounds you mentioned that I silently implemented myself. Anyway, room for improvement and for MS!

[martinbubenheimer]

Thank you for this clear call for improvement!

[MatissBI]

While I'm watching this my dataflow PQ keeps and keeps validating. :D

[JosefineNord]

Nice to see that I'm not the only one who is a bit frustrated about the PBI Dataflows.

In relation to: "01:30 Managing gateways ...or not"
Do you know if it is possible to connect to an Amazon Redshift Database, without a gateway?

[jyhyeongchan]

Thanks for great sharing. I felt the same pain. I find many value to use dataflow for me but I also find additional frustration like 1. Dataflow failure and stop running but it linked dataflow still remain running. 2.
Manually cancel dataflow refresh show status as cancelling (where I confuse whether cancel done and ready for me to run again refresh). 3. It is a waste where incremental refresh cannot work for power BI dataset with data source is dataflow.

[chrismartin]

Power Desktop = Development. Got it!

Thank you for saving me so much time :D

[DarkoVlajkovic]

I transferred almost everything on dataflows except for mapping because merge needs premium license. But it all works well for me. Yes in the beginning i had trouble of setting up gateways but after that it all worked smoothly. I couldnt have made those reports that i wanted in PBI if there werent for dataflows. Now, thank God, it all works as it is supposed to :)

[VivekDevalkar]

Thanks Ruth for this wonderful video and realistic feedback. I have recently started to dig deeper into dataflows and was looking forward to using them extensively. Are these issues/annoyances still around, especially the ones on slowness and error messages?

[paser]

Thanks for the insights. Conceptually, dataflows is awesome, but I can't wait until the bugs are worked out of it. Keep the videos coming.

[bcippitelli]

Hi Ruth! Thanks for the video. I have been suffering with many of the issues you mentioned here. Now K know I have to be aware of your tips and... Patience!!! Curbal mola!

[massimocolaceci]

Hi Ruth, thanks for the tips. I didn't know the datatype requirement and the automatic conversion of errors to null. Also, I agree that the validation queries process could be clearer. Beside that, the migration of ETL from PQ Desktop to dataflow is being useful for me, mostly in the development phase: I splitted the dataflow into two, a staging and transformation one, so I can pick the tables at the phase I need.

[reisamuelsson]

Yes! This video is 100% spot on! I am trying to move all my queries from PBI Desktop to Dataflow and its just sooo infuriating! Not only is it so slow to do the simplest of queries, there's so many messages without any explanation, and on top of that, you have to be so careful that you have the right license for each query! I have PBI Pro and I did a bunch of merging. in the end I get an error message saying nothing will be refreshed if I don't upgrade to premium....