Data Architecture 101: The Modern Data Warehouse

Want to build a reliable, modern data architecture without the mess?

In the modern data landscape, it's the tools & technologies that grab all of the headlines.

But a potentially more important decision you need to make is about your data strategy.

While there are many different approaches, today I want to cover one in particular known as "Modern Data Warehouse".

This is the most common approach I've seen and is ideal for most small-mid size companies looking to establish their architecture.

It's also important to remember that most companies aren't "big data" enterprises or require overly complex systems.

Avoid the urge to keep up with big tech companies if you don't feel it applies to you. Which is probably the case.

I'd take simplicity & clarity over complexity any day.

Thank you for watching!

Timestamps:
0:00 - Intro
0:34 - What is The Modern Data Warehouse?
1:46 - Comparing to Traditional Approaches
2:32 - What about the Data Lake?
3:43 - Example Tool Selections

Title & Tags:
Data Architecture 101: The Modern Data Warehouse
#kahandatasolutions #dataengineering #dataarchitecture

[KahanDataSolutions]

Want to build a reliable, modern data architecture without the mess?

[shakedm]

Just discovered your channel recently and I wanted to say it is a gold mine! Keep making this kind of content!

[aliahmaddata]

Great video! Your pace, presentation and visuals are really on point.

Keep up the good work :)

[Austin-dmbp]

Really appreciated seeing the different examples, as it helped to underline how the stages remain the same, regardless of the specific tools being used.

[shashankemani]

its really a great video for someone to understand the high-level architecture of modern data stack. It would be great if you can start a in-depth data modelling playlist as it plays a crucial role in designing data engineering pipelines. Thank you

[colter]

Best data modeling videos I've come across so far, great job!

[jayakrishna]

awesome this is really useful. Keep making these sample architecture videos.

[johnflanagan]

I just discovered your videos. They are excellent. Clear, concise and to the point. Great content! Thanks so much!

[rks.siddhartha]

These design and architecture videos are great to learn the concepts in bite sizes. Looking forward to more such videos.

[navoabey]

Simple and to the point explanation. I think it very important to understand the concepts as well not just tools, very useful for interviews also.

[thomasbrothaler]

How did I not found your channel much earlier. Your videos are extremely concise, well visualized and informative. I am a Data Scientist transitioning to Data Engineering (because in Gaming I am also always the healer/support 😉)

[AlexKashie]

Wow, great content broken down simply.... Thank you.

[jgianan]

awesome explanation and visuals! Keep it up!

[AdamWeisberg-yc]

I think it would be really cool to see how, once the data is landed in the data lake, you bring all the data together, since you wont necessarily have matching IDs from different sources to work with.

[tomastruchly]

Heh the concept you presented (collect data from various sources into Snowflake DWH) & transform it via dbt is exactly what we do for customer :) I worked in on-premise where we handled everything via scripts & Jenkins & must say this modern approach is in many aspects better :)

[SheranneTan-np]

Great content! I had a question - why would companies choose to use standalone ELT / ETL providers (e.g. Stitch, Matillion) over the native Amazon Glue / Azure data factory? Wouldn’t it be easier to use the cloud provides as it would be more integrated?

[pbxmy]

For products like Databricks that attempt to offer a full DE and Analytics package, can these concepts be applied similarly? Using Azure Data Lake, Databricks SQL for transforming, and Delta tables for analytics?

[patmclaughlin]

Do the “Data Models” (I assume these are synonymous with data marts) physically contain data? Or, are these like database views?

[michaelhunger]

I was curious coming from your "simple, small/mid-sided" data email. I expected something on efficient analytics databases like Do you plan to cover that in a future episode? Basically the other parts of the stack would stay the same just the processing goes from snowflake/synapse/bigquery to one of these more efficient, lower-cost tools.

[bananaboydan]

Amazing video man. As a senior CS student and aspiring data engineer, I get none of this in school! Love the channel man. Are you on instagram / twitter?