Can Data Engineers Do Better Than SQL? - Should We Replace It?

I have seen plenty of arguments for and against SQL.

After all, it's been around for decades, shouldn't we replace it with something new and better?

Yet it remains.

So i wanted to read through an article that discusses arguments against SQL.

Article Source
If you enjoyed this video, check out some of my other top videos.

Top Courses To Become A Data Engineer In 2022

What Is The Modern Data Stack - Intro To Data Infrastructure Part 1

If you would like to learn more about data engineering, then check out Googles GCP certificate

If you'd like to read up on my updates about the data field, then you can sign up for our newsletter here.

Or check out my blog

And if you want to support the channel, then you can become a paid member of my newsletter

Tags: Data engineering projects, Data engineer project ideas, data project sources, data analytics project sources, data project portfolio

_____________________________________________________________
_____________________________________________________________
About me:
I have spent my career focused on all forms of data. I have focused on developing algorithms to detect fraud, reduce patient readmission and redesign insurance provider policy to help reduce the overall cost of healthcare. I have also helped develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget. I privately consult on data science and engineering problems both solo as well as with a company called Acheron Analytics. I have experience both working hands-on with technical problems as well as helping leadership teams develop strategies to maximize their data.

*I do participate in affiliate programs, if a link has an "*" by it, then I may receive a small portion of the proceeds at no extra cost to you.

[MartinAndersson]

I sometimes feel constrained that SQL is declarative. Sometimes you just want to loop over the data. That is possible in T-SQL but the syntax is horrible

[MilhouseBS]

Excellent video. Thanks! I'm working thru Chris Date's books and videos now.

[tomastruchly]

More complex SQL queries is for sure harder to read & debug. Still when you need to work with data with complexity there is nothing better out there. Python Pandas, Java Hibernate, NodeJs etc. are fine for quering & manipulating with data however are harder/impossible to use for such complex stuff (in the end you will write SQL scripts and call them within the code 😅)

[MBSnapper]

SQL has stood the test of time, and most data processing tech usually ends up with SQL compatibility to some degree (even if under the covers, it's something different). Alternatives definitely continue to enhance the data engineering toolkit with preferable options for some use cases, but I don't see a SQL killer on the horizon at the moment. I've been through several hype cycles, and it's still here, alive and kicking 😀

[jbejaran]

One of my favorite T-shirts:
1) NoSQL
2) N.O. (Not only) SQL
3) No, SQL!

[knkootbaoat]

at the end of day of day it's about problem solving. Idc about orthogonality, compactness as long as the problem is solved. the tricky part is when the problem definition changes Then orthogonality and compactness will become more relevant. but to me I try to keep the bigger picture in mind (in this case whatever problem I'm trying to solve). I will say in general consisteny good convenientions can never hurt. this a bit vauge but hopefully it made some sense

[bobcatmania]

While I get the issues this article raises, the fact is that this is not the first time the IT world is trying to get rid of SQL and they always get back to it

[maxedouthc]

Love dapper, keep the sql (sqeel). SQL is a way to talk to a db, it’s not c# or python. They have there place.

[DrJuanTaco]

SQL has its place, we dont need to mess with it.

[GambillDataEngineering]

SQL is definitely not going anywhere!

[MilhouseBS]

Microsoft also has Power Query language for ETL.

[tiaanstander]

I was expecting a suggestion of what to replace SQL with. But no, a handfull of strawmen and inconveniences is all we got.

[mioszbies]

I see no reason SQL not to be replaced with pandas for example.