How I would learn Data Engineering (if I could start over)

In this video, I’ll share my step-by-step process on how I would learn Data Engineering if I could start over. Data Engineering is a fast emerging field within the Tech industry; where more and more people from traditional data science/software backgrounds are pivoting towards. We’ll cover the fundamentals of Data Engineering, and talk about some advanced topics you’ll need to learn in order to maximize your chances of success. Hope you enjoy this one!

Timestamps ⏰
0:00 Intro
1:45 Fundamentals of Data Engineering
4:51 Core Data Skills
7:30 Advanced Data Skills
10:07 Mindset

Notes from video 📝

Who am I? 🙋🏻‍♂️
I'm Jay, I love making videos about travel, self-help and tech. I currently work in New York City as a data engineer, but I grew up in Malaysia and lived in the UK when I was 19. Back then, I had no idea what life was about, moving to so many places, navigating career in Tech. Today, I've learned a lot and wanna share my perspective through filmmaking.

Socials 📱

Sub Count: 152

[ekkyarmandi]

I'm a freelancer and also self-taught programmer who wrote a lot of Data Scraping scripts using Python for my clients. I have a loyal client who requires me to extract data for them every month. I repeated this manual process for about 3 months until I realized the scraped data was getting bigger in size and dimension. I thought it was not a data scraping job anymore, it was a data engineering job. After doing some research online, I figured out some useful tools I can use to manage all data collected by the data scraper. I start by storing all the data in an SQL database and automating the process using Apache Airflow. That was the first time I thought Data Engineering was the right direction for my career path as a developer. Thanks for your advice, Jay.

[vuyilemagwaza]

I recently signed up for the data engineering course from Datacamp and I have covered pretty much most SQL courses but I'd love to go back on a few of them again, I work as a data analyst but I feel a career in Engineering would be great.

[ministerbo]

Hands done, the best video I have seen on the subject. It's no nonsense, practical and easy to digest (of course after you put in the work). I am just starting out in my journey to become a Azure Data Engineer so I thank you Jay. Thanks for sharing and being so transparent. I'll keep you posted on my many adventures. For what it's worth, I'm 70 years young and still dedicated to life long learning.

[gamersant]

What a gem. I am currently pursuing AI/ML. Getting data scientist job as a fresher without experience is almost impossible so had to start with either as data analyst or data engineer. Since now everyone is becoming data analyst so their is demand and supply issue in the job market. Thinking of expanding my domain knowledge to data engineering as well but as you stated “got lost” with the available resources. This video helped me to make a roadmap to pursue data engineering.

[grandmundi]

I have been working with python so I think a few very important things 1) pyspark 2) pick AWS and/or Azure to learn their data engineering stack 3) very solid SQL skills.

[jesseantwi]

Hello bro, i completed my undergraduate in actuarial but failed miserable and i have been down, filled with regrets and lack motivation ever since but your has given something to believe and get me to regain my confidence . Today begin a new journey, my journey into teaching myself to become a data engineer hope to come back to this post after a uear to share my progress.

[cooltrucly]

i'm a junior data scientist transitioning to data engineering/MLOps. i'm also based in nyc! i'm recently discovering my interest in filmmaking too! looks like i stumbled upon your content at just the right time. just subscribed!!!

[tolaseadegbite]

I am transitioning from web development to data. Fortunately for me, I learnt development skills from Learnenough. I have gone through their Commandline course and now on Python and Python for Data Science. I am excited to be doing this. Thank you for this content.

[Cxba]

This is hands down the best high level introduction to data engineering I’ve come across. Thank you for making this amazing resource guide as well. I work as a BI analyst now and wanting to expand my skill set so I can truly become a data engineer. I appreciate your time and hard work!

[JuandeSouzaVargasCosta]

I'm starting my learning journey from here. It's a completely new world for me, both because I'm not a native English speaker and because I'm totally new to all this technology. I've made up my mind: I want to become a Data Engineer, and I'm going to work really hard to achieve that!

Thank you so much for the guidance; I’m going to put it into practice now.

[rabihbadr]

Great job with your career man, mixing technical and artistic paths, great balance.

[lunariasophie]

Thank you for such an informative video, I'm a junior data engineer and your channel is an invaluable resource!

[gustavojuantorena]

Best roadmap for Data Engineering on YouTube. New sub Here 😊. Greetings from Argentina 🇦🇷

[longdatadevlog]

I love the way you split the career to basic and advanced skills. I’ve been keep doing that to become the data engineer. Thanks for sharing!

[k.h.p.]

I appreciate this video. At 3:00 he mentions that "the objective is to learn about data concepts, not computer science." I'm literally a total novice (but getting deeper into this), and I always thought that computer science was partly focused on data concepts. Or, maybe he means that data concepts is more specifically focused, in contrast to computer science which is extremely broad.

[Scarletkillerxz]

Beginning my path here, great video with no filler!

[nagusameta]

I am a fresh graduate in Information Systems and my internship had me working with the team's data lead using Python, Apache Airflow, and PostgreSQL. I was glad and became more confident from the experience but once I was job hunting, I kept finding data engineering job postings demanding tools like Azure, AWS, Spark, Hadoop, etc. I was discouraged as I only had the three I mentioned. But I was a bit encouraged when you talked about tools as getting more specific on what the task demands, and that foundation through concepts can be better.

[dereklei]

This is super super SUPER practical and insightful and helpful. Looking forward to your future content and successes!

[DanielleBaylor]

About to dive into SQL this weekend. I'm just starting my journey into data analytics though. I don't think I have what it would take for an engineer 😅 but I still like watching videos like this cause I like looking at different things

[shubhamwaingade]

I wish I would have seen this sooner! Great and crisp content.