A Deeper Understanding of Spark Internals - Aaron Davidson (Databricks)

A Deeper Understanding of Spark Internals

Aaron Davidson (Databricks)

[李佳键-ir]

So far the best video i watched that explains Spark execution model in high level with a good understandable example. Thanks for sharing!

[ABHIS]

Awesome training. Short, simple easy to understand and full of content. Great work

[TheKavyajayan]

Can you add subs to this? The auto generated ones are not good enough

[RCMOULI]

Good video. Do you have any videos how Spark runs better or different compared to Hadoop and for which type of scenarios Spark is preferable than Hadoop.

[raksadi]

In my spark version 2.4.3 job after all my transformations, computations and joins I am writing my final dataframe to s3 in parquet format
But irrespective of my cores count my job is taking fixed amount for completing save action

For distinct cores count-8, 16, 24 my write action timing is fixed to 8 minutes
Due to this my solution is not becoming scalable
How should I make my solution scalable so that my overall job execution time becomes proportional to cores used

[hmartirosyan]

He's not super-active, and doesn't respond to emails ;-)

[ManuPresannakumar]

why the hell the text is written with a type writer/font?

[SonnyTheoTumburManurung]

A bit too fast at talking, but overall still understandable. Very good talk on the important concept.

[ravieze]

super fast.. very diffficult to understand

[micklejickles]

This guy at the beginning. Find a seat man. Ruined it for me.