Advancing Fabric - Lakehouse vs Warehouse

Microsoft Fabric has been released, there's a flood of information out in the world for you to start digesting! Fabric has different experiences, different workload types, but the underlying data is held as Delta tables.

With the T-SQL Synapse Warehouse workload, and the Spark-based Data Engineering workload, it's a confusing choice. Does it matter? Are there pros/cons to each side? Why are there two experiences??

In this video Simon walks through the differences and explains where he would use the two workloads!

If you're looking at starting a Microsoft Fabric journey, and need some real Lakehouse experts, talk to Advancing Analytics!

[effortLus]

Watching your videos is such a time saver it actually feels like I'm cheating within the industry

[dandical]

This was exactly the question I needed answering after the announcement.

[AndreasBergstedt]

Good take!
Just for reference, I've built dynamic T-SQL based metadata frameworks for longer than I would like to admit, going back to 2011. Just to highlight that data Engineering is not uniquely an Apache Spark paradigm.

[yoavlubo]

I love the simplicity approach!
Rear to find it these days, thank you!
It all comes down to your team/consumers skillsets, but if the have all the skillsets, it doesn’t matter at all

[johnfromireland]

Of course! Fabricator makes a LOT of sense!!

[BeArt]

One of the things that excited me in the preview was Dataflow Gen2! Are you going to touch that as well?

[jpcst]

Great and simple explanation! Thanks for sharing this!

[SurajSingh-ctdy]

Now do you choose Databricks or Microsoft Fabric to build a Lakehouse!?

[FakhrudinJivanjee]

Looking forward to it.. I would be more interested to see the cost impact (bang for buck) for organisations:
- Wanting to move from traditional ADF, Synapse, ADLS to using Fabric
- Wanting to make a choice between Fabric and Databricks, As much as they complement each other they can also be used exclusively.

And does storage taxonomy go out the window with whoever has access able to write to one lake in whatever way they feel (I could be incorrect about this assumption)

[oscarzapatajr]

Good analysis, but you didn't include data factory in the equation... You can use data flows (just power query online) to do some fairly complex data trasformations and automate when those data flows run to load data in a data wharehouse...

For someone with a background in data analysis and powerbi, this is a more natural choice, because we are already used to power query

[alexischicoine]

I enjoyed using data factory in the past it’s great for ingesting or moving data around and for orchestration. I wouldn’t use fabric for transformations at scale though.

[Phoenixspin]

"Agonizing indecision" - that's my life.

[PhilKoay]

Great work! But also should it be highlighted the scalability of the spark engine, and arguably that the Polaris is truly serverless. Is there much in the way of pros and cons around this? Cheers

[gauravchaturvedi]

Could you cover in your next video following topics related to Fabric - 1) The storage used by One Lake is deployed to customer's Azure Account or Microsoft's own Azure account? 2) Do we have the capability to define Data Life cycle Management policy for data on One Lake like we have ADLS G2? 3) Is the cost of One Lake storage same as cost of a blob or ADLS G2 storage?

[samuelbrown]

Good and clear explanation, thank you. Learn lots. 😎

[BajkeTheSerb]

Hi, are you sure about the "full T-SQL support" in the Warehouse? (Some features ie. NVARCHAR data type, temporary tables etc. are NOT supported.)

[preetipics]

great comparison, thanks. In my org, the power BI tenant is managed by europe and they dont know if they're going to be activating fabric for U.K to test. Will MS make us move to fabric at some point anyway?

[alexischicoine]

Is the fabric spark cheaper enough to offset the loss of performance you’d get from databricks optimizations and photon?

[antonyliokaizer]

I have two tables in a Lakehouse and I want them also in the Data Warehouse, should I use "Copy Data" in pipleline to clone them or any other way to create a "Shortcut" from Warehouse to Lakehouse??? Thanks.

[alisabenesova]

Did someone try to run same analysis-queries or so in Dbx and and Fabric? Is there any real comparison? I mean... If I am foinf to chose f16 is it equivavalent to what 16core 32G ram or???