Why I QUIT Data Engineering

Welcome to my channel! In this video, I'll be sharing my incredible journey from a software engineer to a data engineer and then back to being a software engineer. Join me as I delve into the fascinating world of engineering and discuss the valuable insights I gained along the way.

Throughout my career, I've had the privilege of working in both software engineering and data engineering roles, allowing me to gain a deep understanding of the intricacies involved in each field. I'll be sharing my experiences, challenges, and the valuable lessons I learned during my transition from one role to another, and ultimately back to where I started as a software engineer.

As a software engineer, I thrived on building robust applications and solving complex problems. However, my curiosity led me to explore the realm of data engineering, where I discovered a whole new world of possibilities. The challenges I encountered in this domain pushed me to expand my skill set and delve into the world of big data, data pipelines, and data processing.

In this video, I'll provide insights into the key differences between software engineering and data engineering, including the tools, technologies, and methodologies that are unique to each discipline. By sharing my experiences, I hope to offer guidance to fellow engineers who may be considering a similar transition or seeking to broaden their skill set.

Join me on this journey as I discuss the exciting projects I worked on as a data engineer, the technical skills I acquired, and the impact it had on my career trajectory. Additionally, I'll explore the reasons that led me to transition back to software engineering and the valuable lessons I carried with me.

Whether you're a software engineer, data engineer, or simply interested in the field of engineering, this video will provide you with valuable insights and a unique perspective on the journey between these two disciplines. Make sure to subscribe to my channel to stay updated with more exciting content and hit that notification bell to never miss a video.

Thank you for joining me on this incredible journey from software engineer to data engineer and back again. Let's dive into the world of engineering together and unlock new possibilities!

#SoftwareEngineer #DataEngineer #EngineeringJourney

[AchuVlogs]

I've been in data engineering for about 4 years now and I can totally relate to what you mentioned in the video. I love building new applications, but 80% of the time feels like I'm just fixing things that keep breaking in the pipeline. I'm thinking of switching to a backend software engineer role since the skills are somewhat transferable.

[nyavecclasse]

The compensation for Data Engineering roles tends to be higher, but this may not be suitable for individuals who prefer more client-facing roles, such as frontmen. Some prioritize financial compensation over recognition. I couldn't care less about recognition; show me the money.

[Iqbd]

Thanks for the feedback. I was considering move from SWE to DS in my uni.
My objections for SWE were networking/security module, 2d/3d human interaction, advanced op system. where i preferred data related module, AI and programming.

[joyceshendre]

I moved from SDE to Data Engineering and was wondering if I made the right choice. I purchased many courses and i had no interest in learning, though I am forcing myself, I miss software engineering, I miss creating things, writing logic, and building 1000 APIs. Thanks for sharing! It's good to know I'm not alone.

[khacquyinh]

Firstly, we need to understand what data engineer is. Is it just for building pipelines, some batch or streaming jobs, or build some APIs, deploy models? These stuff just as half of data engineer, which is engineer/software engineer/backend engineer. The rest part is data, which requires DE to understand the data and have domain knowledge. These requirements could be much challenging, when DE need to know what is the meaning of data values in the database/table/column/etc in business, so that for them to build data warehouse, especially, datamarts. These things need a lot of tranformations and aggregations but assure the correct business logic and performance. Besides, DE could be a data governance, such as identifing and not exploding sensitive data.

DE often do not target to customers but more focus on supporting internal members, such as marketing, sales, DA, DS. But in some companies, DE aggregate data and build APIs to explode the aggregated data for BE/FE teams to build report features of apps serving customers, here, DE are recognized from customers, not just the internal side.

DE in fact could have many job titles, such as data pipeline engineer, data platform engineer, cloud engineer, Data analytics engineer, since big corps expect specific responsibilities for teams.

[aniket]

You captured it perfectly.. thanks a lot 😊
I have been a data engineer for 11 years and now trying to move to software engineering.

[scoc_siddhesh_waje]

I'm recently employed and worked for a month, I was hired as a data engineer but these people are making me work on very boring stuff that includes SAP Business Objects as a reporting tool and service now tickets, in my college i have made few projects in MERN stack I have also learnt data engineering tools like Snowflake, Databricks, Pyspark I like both the things Software Development as well as Data engineering in what field should I switch into based on the future scope, and remote jobs availablity 3:34 3:36 3:38

[erickfana]

I'm thinking about to explore Data Engineering and already studying about it. But, i'd want to stay as a backend player too!. Would you guys recommend me that?

[neutech]

I have been working with Data for past 4 years. And I am losing interest in it because, most of the time I am doing fixes to the existing data. I am trying to switch my career to software engineering. But seems like I have to learn a lot to do this switch.

[adamsaraya]

what do you think about data architecture and data analysis

[barneystinson]

You said you wanted to work with SQL, but don't data engineers use SQL mainly ? So what language were you working with?

[patrickchan]

well done, keep up with the good work.

[nicolasperalta]

Amazing video, clarify a lot of stuff

[РоманР-яп]

Hello everyone! I'm considering a transition from a Scala software engineer to a Scala data engineer. I've found that being a Scala dev is very challenging for me, but I'm not ready to give up on Scala just yet. Can anyone share insights on whether working as Scala Data engineer (Sql, Spark) might be a smoother path for me? Appreciate any advice!

[AtiaNasr]

Hi, I just graduated from college, and i am really confused about whether I should go for the backend or Data Engineering
I really love coding, working on very complex projects, Learning new things

Can you give me an advice where should I go

[mickyman]

Hi, I'm a fresher, just spent almost a year in my new job and hardly getting to do new development/coding, mostly it's some lambda functions for some automation or some other kind of functionality or the aws CDK code for Infrastructure as code, I'm feeling bored but I don't know what should I do now, I have tried react in depth also explored nodejs, and recently did springboot with java, i enjoyed node and Java wasn't bad, but since I don't have work experience in software development nor my skills will be as good as someone who's into software development for a year moreover people here in my country are asking system design dsa and tech related in depth questions, so need to preoare hard for that, I am thinking to learn sql in depth by doing it in depth (I don't even do sql a lot snd mostly it's adding few pyspark transformations), then I'll try to make my springboot better since my company already have people working on springboot although i don't know if I'll enjoy it, then will try to switch to a company with thah experience. can you give me some advice. I'm almost 24

[maximvassilev]

I would be glad if someone could give me some advice. I was backend Django developer for 8 months in a company (after finishing academy for 1 year). I left the place because the company was not the right one for me. But I actually was okay with the type of work.
Now I am looking for something new and I have opportunity as Junior data engineer making ETL's and pipelines.
I am afraid I will make a bad decision if I say yes to this offer. Working with data, cleaning it and sorting it sounds borining.
Should I even try it or keep looking for other stuff like another company with backend?

[VishalMeghwal]

Lower the background music, people would like to hear what you are saying more clearer rather than the music.

[deepanshurathore]

Now I'm dropping my decision to switch my job role from SDE to DE..😅😅

[loujon]

Was this video meant to be heard over music playing in the background? I’m so confused