Why NOT to become a Data Engineer

I have created multiple videos about data engineering, including a data engineering course for beginners. Why would I advise anyone against pursuing a career in data engineering? I like being as transparent as possible - while this job will be great for many people, it might be disappointing for the others. In this video I'm outlining three reasons why not to become a data engineer.

If you'd still like to learn data engineering, I recommend following the 4 simple steps below to land you the first job interview:

1. Learn Python
I recommend following the Python for Everybody specialization course on Coursera, which is one of the most popular courses there:

2. Learn SQL
SQL is still the lingua franca of data. I recommend going with Learn SQL Basics for Data Science course, because it contains some chapters which are very releavant to data engineering in partcular, e.g. distributed computing with Spark

3. Learn Bash scripting/Linux
I wouldn't take a full course on it, but at least read a good article.
if you do prefer to take a course/guided project, I think this one is short and good:

4. Learn how to develop on the cloud, e.g. on AWS
There are a few good courses around there, but I think the Coursera one is the most comprehensive

DISCLOSURE: The above contain affiliate links, meaning when you click the links and make a purchase, I receive a commission.
----------------------
Find me on Instagram:
@karo_sowinska

And if you'd like to get me a coffee, it'd surely warm my heart :)

[TheVivaciousNerd]

I'm actually currently trying to move away from data science and into data engineering, which is apparently a rare path to take. Personally, I don't really love the exploratory nature of data science (though I used to think that would be awesome)... I'm just the kind of person who likes well-defined tasks/problems that have a real solution - so much of data science is just seeing what's there, and sometimes, there's nothing useful at all (explaining to the business why you just did two weeks of work and have nothing to show for it sucks). I also really don't care about applications of the theory or how the data supports business goals, I simply have no interest in business at all. So engineering really appeals to me - all the tech without the boring business things! Also, less talking and meetings lol :P

[Musica]

Wow, many of those cons sounded like pros to me hahaha

[tonysamosa]

3 minutes into the video: “as a data engineer you are not at the heart of the business you’re behind the curtains”

That is music to my ears. I am an analyst and I don’t know what I got myself into. I HATE talking to clients and I hate talking business. I enjoy my job the most when I get to build automation tools for myself or others and feel that’s where I would keep my sanity; being ETL focused. I want to be a data engineer

[RemiGabillet]

You make such good points @Karolina.
It's so true that by being in a support function, data engineers are often invisible unless things break. But the counterpoint is that the work is less rushed, more long-term and so much more satisfying. This makes for an awesome work-life balance.

[wmgonzalbo]

As a data engineer/architect this is spot on!

Some analogies on reason #1 - data is your plumbing, while your data engineers are your plumblers. You dont see the plumbing but you are thankful that you have clean water in the house. It may be taken for granted at times but any leaks or burst pipes is a big problem.

Another analogy is in the body data is the blood. Its not a main organ but organs need it (clean blood) to function properly.

On #3 true same with data science or ML where they now can be a black box provided as a ready to use product. But in the future someone still needs to know when and how to use these tools.

Great video, more power!

[randomnessdiary]

Hey I just started my internship as an etl developer and while going through various articles online I found it comes under data engineering. And while research about that I found your profile and I'm so glad to have found you!

[alfredama]

Data Engineering is where is at guys :), we are the backbone of Machine Learning and AI

[karolinasowinska]

(also don't forget to switch on 4K for the best experience (I finally got a new camera!) haha)

[dorianzxc]

I can relate so much to this. I'm a data analyst being bribed so much to keep doing data engineering, I gave up on it and enjoying being a hot bun 🤑
btw part of my learning process was your ETL videos, so thank you! That's why I keep coming back to watch your channel!

[scienceaddicted]

You know, I have realized few weeks ago, that my current job is actually Data engineering :)). It has a very weird name, but I actually build data pipelines in clinical trials, and I love it. I'm happy with my job, and I got here without knowing how fancy it will be in 2021 :D

[nas]

The first reason sounds like an advantage to introverts.

[jannedrumsit]

I am an introvert but I am in consulting. Talking to clients drains me big time. After 31, I am now looking into starting all over again and shift to data engineering.

[Diegownz]

I just got rejected from my data engineering internship, so this video came at a good time 😃😭

[brendasofia]

As a data engineer student I have to say that your videos help a lot!!! There a lot of questions that you just easily answer. I hope one day I’ll be as talented as you.

[jameshizon]

You had me at work-life balance is great. I like to do many things outside of career, so I am sticking to the field (currently 95% finished with my DE Bootcamp). I also think that one can go from Data Engineer to ML Engineer later on (if they have data science background already, but the job market is too competitive).

[datadrivendecisions]

There are a few more examples where you can be closer to the business as a data engineer like myself! Companies where data is their main asset can be a huge one e.g. Opta who are the leading sports data provider, Nectar who collect data about customers to analyse their shopping habits and even Bloomberg/Reuters who have amassed so much financial data that they can sell it for tonnes of money!
I'm also hoping technologies like Denodo and Databricks will just change the role of a data engineer instead of getting rid of it.
Great video as always and the new 4K looks great :)

[IamDanish]

I am a Data Engineer now. Lets see how far I can take the load. LOL. Thanks for the wonderful video Karolina.

[gauravarora]

Hey, Thank you for the video.
At first i just wanted to stop the video when you said "these 3 reasons are just your thoughts".
But later on as you speak and explained the unbiased scenarios 'giving the listener a chance to decide on their own knowing the + and - of the field', i loved connecting to this fact.
and yes, i am also trying to switch to Data Engineer role from Business Analyst role and thats what pulled me to watch this video.
Again thank you for honest and helpful thoughts and contributing to the community.

[Jon-dkqu]

Love the quality of the camera, and thanks for the content!

[paulifea]

Thank you for adding your perspective and experience to this. I have been presented a choice to work either as a Data Engineer or a Cloud Engineer and I'm still deciding which to go for. I have barely started my career as a cloud engineer, but I've build quite a bit of knowledge over the past 2 years to prepare myself for a cloud engineer role, only to be presented with the choice of becoming a data engineer now. Both seems attractive to me because I like to grow as a technical professional, I like the hands-on and practical aspect of both roles.