Spark performance optimization Part 2| How to do performance optimization in spark

Spark performance optimization is one of the most important activity while writing spark jobs. This video talks in detail about optimizations that can be done at configuration level , file formats, serializes etc.

[vinothraj]

I was worried where to start with optimization..Now i know..Thanks mam.

[rahulw]

It will be very helpful if you provide a spark-submit command including all the optimization factors explained in this video.
Thank you for such informative videos.

[washimrajaakash]

Hello didi, I have never came across any such video/blog where optimization factors are presented in such well organised manner. Thank you so much for that. It will be really a great help if you kindly make a video on optimising a spark job while running... Like one typical example : resolving more data getting into one executor or may be a comparative study in the spark job details page when a wide transformation is used instead of narrow etc... Anyway thank you so much for such simple & clear video on these topics .

[DataIsBusiness]

I dont understand why people wont subscribe to this type of channels... they do for movie prod channels all useless ones....Mam, Excellent videos.

[himanishkopalle]

Thank you. This is very informative.
I can now create a checklist and optimize my spark job accordingly

[sahilmittal]

Very nice vedio, summarizing the complete vedio is best

[ramkumarananthapalli]

Very useful mam. Thanks a lot for making this. Could you suggest which is the best documentation to refer to? In spark official documentation I could not find many concepts thorough enough.

[balajikasturi]

Hi Mem Thanks you so much for explanations it helped me lots i have one doubts dynamic alloction wiil help job size will be incresed suddenly in spark

[npl]

Could you please attach the ppt in the video description. These properties are really useful.

[kanikasaxena]

Hi, could you do a video on Spark Programming Interview Question