Java VS Python

In one of my livestreams I was asked if Java is still my favourite. I love Java, but I've been working more and more with Python lately. In this video I show you what I like about both!

►Learn Data Engineering with my Data Engineering Academy:

►LEARN MORE ABOUT DATA ENGINEERING:
-Check out my free 100+ page data engineering cookbook on GitHub:

►PLEASE SUPPORT WHAT YOU LIKE:
Just use this link whenever you buy something from Amazon:

(As an Amazon Associate I earn from qualifying purchases from Amazon)
This is free of charge for you but super helpful for supporting this channel.

#datascience #dataengineer #plumbersofdatascience

[slr]

At a fundamental level Python can't take advantage of advanced concurrency features of modern CPUs. In Python memory accesses (loads and stores) are volatile
meaning they are written completely to (or loaded from) main memory, and instructions are not allowed to be re-ordered.

Modern CPUs (especially ARM based CPUs such as the M1) have dedicated silicone to re-order instructions and for reducing main memory access by using caches.
Because Python does not utilize these CPU resources it is slower and is a fundamental reason why Python is about 10X slower than Java
(and others such as C#, Go, C++, etc..).

The argument in favor of Python is development time, however I would argue that you need to find a balance between development time and CPU execution
time (how much time pure Python code is executed on the CPU). For example if you write a large CPU intensive application in pure Python (without calling out to
code written in other languages) and deploy it on a data center, you will need 10X more resources (building space, power, servers, more server maintenance time etc...),
and these costs mount over time.

The sweet spot is in the middle, you do your heavy lifting in a language such as Java (pyspark is a good example of this), and use Python to orchestrate the various components.

[shivasheeshyadav]

Completely agree with the point that how easy it is to parse json in python. I remember dealing with nested json in spark scala, and it wasn't a cakewalk for sure.

[ibexy]

I am an intermediate level programmer but I have to say Python is far easier to code than Java/C#. The amount of code I have to write in order to perform certain (simple) operations is far more in Java/C# compared to Python. Also, I find myself working more and more around data and based on my (little) past experience using Java/C#, I think Python should have been my first choice.

[gandapur]

I am really into Pyspark and perhaps dont like Spark over scala.Whats your thought on this, is this workable and sufficient, considering upcoming power of sparksql will normalize all of these...