Advanced Apache Spark Training - Sameer Farooqui (Databricks)

preview_player
Показать описание
Live Big Data Training from Spark Summit 2015 in New York City.

"Today I'll cover Spark core in depth and get you prepared to use Spark in your own prototypes. We'll start by learning about the big data ecosystem, then jump into RDDs (Resilient Distributed Datasets). Then we'll talk about integrating Spark with resource managers like YARN and Standalone mode. After a peek into some Spark Internals, we touch base upon Accumulators and Broadcast Variables. Finally, we end with Spark Streaming and a technical explanation of how the 100 TB sort competition was won in 2014." - Sameer

Slides:

Want to learn more about Spark?

Check out my new class, "Exploring Wikipedia with Apache Spark", recorded June 2016:

// About the Presenter //
Sameer Farooqui is a Technology Evangelist at Databricks where he helps promote the adoption of Apache Spark. As a founding member of the training team, he created and taught advanced Spark classes at private clients, meetups and conferences globally.

Follow Sameer on -
Рекомендации по теме
Комментарии
Автор

1:30 Agenda
5:14 History of Spark
27:40 RDD fundamentals
1:20:23 Spark Runtime architecture and resource managers
2:49:24 Memory and Persistence
3:15:30 Serialization
3:19:50 Staging
3:42:00 Shuffle
3:55:00 Broadcast and accumulators
4:31:25 PySpark
4:49:00 Next Gen Shuffle
5:32:00 Spark Streaming

MrTulufan
Автор

Probably the best Spark video on the Internet right now.

skipperkongen
Автор

Just want to share. I came across this video back in 2016 when spark was a buzz word mostly. Did not understand most of it back then and did not watch it. Now again watching it in 2022. It's true gem.

adrishpal
Автор

This is best tutorials I seen..I admire you Sameer for your patience while you answered all Q...

arunbm
Автор

Sameer thank you for putting a professional video that finally explains Spark at the pro level. Much appreciated.

christianlira
Автор

This is one of the best free videos ever available on the youtube community.

javaidmir
Автор

Excellent presentation of core spark, among the best I've ever watched, despite the older version it covers. Presenter's knowledge is very deep and he delivers it very clearly. Excellent job!!

singalong
Автор

Sameer, you have done us all a great service here, appreciate having this posted....very deep coverage of the core architecture, helpful from any number of aspects. Look forward to seeing more in the future as the platform evolves.

craigholley
Автор

Great work Sameer,
So far the best detailed Spark presentation I have seen online.
Appreciate a bunch.
Thank you,
Tushar Kale

TusharKale
Автор

Wow, fantastic presentation Sameer!  The topics you cover about Spark Core are awesomely explained.  Great work!

bpriorb
Автор

As a new spark learner I can't ask for more :) This is real developer talk and help in designing and modelling any initial spark projects. Thanks a ton Sameer!!!

surendratiwari
Автор

Ultimate video ever seen on Spark internals!

SandeshMendan
Автор

Thank You Sameer.I learned a lot about spark after watching your videos....Will be waiting for your next 5hrs hands on video in next Summit

jubinsoni
Автор

Thanks Sameer !! This is a best video on Spark Internals i came across.

comram
Автор

Best Spark tutorial I have ever come accross.... Thanks Sameer Farooqui....

jahartyagi
Автор

complex concepts explained nicely in diagrams, easy to grasp when Sameer explains :)

arunsundar
Автор

The best tutorials for spark, really.

yjwoo
Автор

Joining others, it's a must watch video

aleksandrivanov
Автор

the best presenter ever. Expert in spark as well.

arada
Автор

Hii.. this is one of the best presentation about spark. One question is, Spark evolved a lot from here. Are these concepts still relevant till today? Any changes or obsolete content of this video? Can any one tell me pls.

seenu