Basics of Apache Spark | Spark Optimization | Adaptive Query Execution | Repartition | learntospark

In this video, we will discuss about one of the spark optimization technique available from spark 3.0. AQE - Adaptive Query Execution and in particular, we will see Auto repartition to reduce the shuffle partition number by Spark on its own.

DataSet for Demo:

Blog link to learn more on Spark:

Linkedin profile:

FB page:

[gowthami]

Most underrated channel..Thank u so much sir.

[desparadoking]

Thanks Azarudeen for the informative video 👏😊

[guptaashok]

How does this help in performance, could you show some difference in performance in some other video as well.

[ManishKumar-mjbt]

Hi, m trying with jupyter notebook it's running only kindly suggest for the same

[kanishkumar]

can i get list of all optimization techniques in spark?

[saikalyanCSE]

Really great channel brother. We should collaborate!

(PS: I am not any content creator) 😃