What Skills Do Data Engineers Need To Know

At a high level, data engineers help take data from point A to point B and remodel it into a format where analysts and data scientists can easily use it.

From a skills perspective, this means that data engineers specialize in ETLs (extract, transform, load), automation (usually with Python or other programming languages), data modeling/data warehousing, SQL and NoSQL data manipulation, and data visualization, to name a few.

The skills that are less known are skills like ETLs and data warehousing. Both of these are usually discussed more in master’s or certificate programs after getting your bachelor’s degree — although we suspect this will or has already changed.

More than likely you will need to learn these skills in some form of order.

Learn More about data engineering Googles DE certificate

Starting with python and SQL, then ETLs and data warehousing, cloud and data viz, etc.

0:00 Intro
2:08 Python And SQL
3:25 ETL and Data Warehousing
5:14 Cloud, DevOps and Dataviz
6:49 Specialize

The dashboard UI article i reference can be found here

If you need data consulting help, then reach out to our team here:

Also, if you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.

Check out my Medium here:

Watch our video on the reality of being a data engineer

[asd]

I am a data engineer, and this is the best video I’ve ever watched that describes exactly what I do in daily basis. Thank you for sharing.

[krispan]

'Seattle Data Guy' is going in my resume as a tech stack.

[TainuiaKid]

This is a great video, and perfectly timed for me. I know SQL and Power BI, and had been trying to figure the next skill to learn, Python (I already learnt a little 3 years ago), ETL/ELT, Data Factory, Data Warehouse design etc . Your video answered my question, brush up on Python. Thanks!

[ricardocruz]

Man your channel is exactly what was I looking for. I'm mexican and there isn't content about this topic, only data science and that stuff and I want information about this topic specifically. So many thanks.

[kphorce]

I like how your ever so subtlety promoting AWS databases with that box in the background lol. I see you lol

[brianonesto]

that was a great call to action at the end, " the question I have for you is why haven't you hit that like button" solid ending

[iFunktion]

Great expanation. I've just landed my first proper data job, i'm definitely out of my depth, but in a good way, this is super helpful sir thank you

[nicholasr]

Super helpful! Thanks Seattle Data Guy!

[Chowmeingeejah]

Thanks! would be really nice to have a short video explaining data warehouses/data lakehouses/cloud and where to look for best practices etc. I find it very hard to find information on these topics!

[jhwilson]

The issue with most Data Engineering postings is they put dozens and dozens of requirements no one person needs to have to do DE job. I have seen some DE jobs ask for PhDs and literally 24 software packages. Oh yes, they also only want to pay bare minimum.

[dume]

What references would you recommend for DW/ETL/ETL? Currently I'm going trough 'Star Schema: The Complete Reference' and Datacamp's Data Engineering with Python Track. What do you think?

[rfg]

Finally some content for us Data Engineers

[rt.]

git is basic programming skill, really unfortunate to put DevOps several layers higher there (or that it's a hierarchical in importance). Note that DevOps has 'dev' in it.

[abhijeetkorde]

Great video... thanks for explaining it nicely-it helps

[focusEngineered]

Man thank you so much. I'm trying to be a data-engineer. And I watched so many videos that really disappointed me. But your video ignited my mind one more time.

thanks so much you're amazing.

[jbird]

Thank you for this video and all your other videos. It's been really helpful. I am a software engineer with a pretty solid foundation in programming (java) and sql. Level 2 of your pyramid is where I'd like to get more experience, and luckily for me I have a real project at my company that would be perfect. What we have is a postgres database with some pretty large OLTP tables of around 50M - 90M rows. The goal is to build some data viz dashboards using this data, but other than that it's a totally blank canvas. Are there any specific tools you'd recommend to look into for this project? Thanks

[dearestperson]

Will be really helpful to have a session on - comparison and suggestions on best data engineer paths .. AWS vs GCP vs Azure !!

[amalnisham]

Thanks for this, I was trying to get into DE after a career gap. Didn't know where to start, so i learned sql basics and doing some exercises on it. Now next in line looks like data warehousing and ETL .
what do companies expect from a person trying become a DE ? do they look for all these topics from the person or they doing to stick with SQL and DW & ETL ?
how deep should my knowledge should be when applying for the interview.

Thanks again for this video. Looking forward to your more videos

[HHS_Analytics_Biaka]

Thanks so much. This was really informative. Please keep making more videos.

[papaamanor]

This is an excellent video, thank you so much.