What Is The Modern Data Stack - Intro To Data Infrastructure Part 1

To put things simply, the modern data stack (MDS) is a set of tools that power data integration. In order, these tools include a fully managed data pipeline for “extract, load, and transform” (ELT) processes along with a cloud-based data lake or columnar warehouse for the data’s destination. On top of that, MDS requires a data transformation tool and a visualization or business intelligence (BI) platform so your company can make use of all the numbers.

The primary difference between a modern data stack and a legacy data stack lies in the host. The modern data stack is hosted in the cloud and, as a result, requires very little technical interference from its users. The modern data stack almost guarantees end-user accessibility, while the company at-large enjoys endless scalability that grows quickly without the expensive downtime associated with scaling the server room that supports a legacy data stack.

If you enjoyed this video, check out some of my other top videos.

What Skills Do Data Engineers Need?

Data Engineering Project Ideas

If you want to learn more about machine learning, check out DataCamps Machine Learning Course

If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.

Or check out my blog

Tags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio

_____________________________________________________________
_____________________________________________________________
About me:
I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.

*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.

[LukeBarousse]

If this intro doesn't convince you that Data Engineers are going to be one of the top most needed jobs for the foreseeable future... I don't know what will!

Great content, Ben!

[antonkostov]

Hello again, brother. I want to brag again. After successfully snagging a DE job inspired by you . Now I obtained the DP-203 Microsoft Certified DE. Thanks again, brother for helping me make the big step a year and a half ago.

[career-calling]

Happy to finally find a channel that talks about data 360 degrees. Thank you for your effort to bring this to the audience.

[matiaspirovanovarela]

Thanks a lot for the video, the series has a lot of potential. May I request if you could talk about cost effective tools? Maybe a sample stack for companies of different sizes or maturity (like the Analysts chart that you showed).

[rachelzhang]

finally!!!! been looking forward to this series to have a more concrete and contextual understanding of data infrastructure.

[GuyThompsonFWTX]

Great video! Can't wait to see where this series goes.

[joseangelmedinacornejo]

Great video, Ben! It is nice to hear this in simple words, so it helps me transmit such ideas to my team and the stakeholders, hopefully in a way the latter will understand.

[andi]

This is amazing sir. Thank you ahead for the series!

Definitely help all of us to keep up with the changing landscape.

[ekta_r]

Really looking forward to this series.
Would love if you would discuss an end to end data Infrastructure design and walk us through the thought process while selecting the tools for each of them with diff use cases.
Thank you again for guiding through your videos. They are a big help!☺️

[eth]

Perfect choice for a series! Looking forward to it

[reneeh]

Super userful chart in describing the datastack!!

[tethadam]

Another fantastic video. Thanks in advance for all your hard work!

[letechnicaljames]

Insightful video. Looking forward to the rest of this series!

[pushpanthkumar]

I Love this.. Ben please consider making videos how you are monitoring the data applications & Best practices to ensure Data Quality.

[Alez]

So useful video! I’m so excited to see what’s next on this series of videos! Thanks for your work

[elis]

Great info! Thanks for the steady stream of super useful videos!!

[kapuriaritik]

Amazing video! Excited for the next part!

[anathanholland]

Looking forward to this series! We're implementing a MDS in my org right now and I think it will greatly improve our efficiency and ability to make data-driven decisions.

[GiasoneP]

Great video. Looking forward to you finishing this series…and hopefully you finish the DE project series too 🙃

[obiradaniel]

Thank you very much, very insightful.