What is Apache Spark? Learn Apache Spark in 15 Minutes

preview_player
Показать описание
#apachespark #databricks #sparkteam #dataengineering #pyspark #architecture

In this video, I have covered the most important topic of Data Engineer which is "Apache Spark". Especially, I have talked about the complete end to end Architecture of Spark Spark covering all the individual components as below,

1. Driver Programme
2. Worker Node
3. Cluster Manager
4. Spark Context or Spark Session
5. DAG
6. RDD
7. Lazy Evaluation
8. Stages and
9. Tasks

Watch the complete video and get a complete understanding of Apache Spark.

– – – Book a Private One on One Meeting with me (1 Hour) – – –

– – – Express your encouragement by brewing up a cup of support for me – – –

– – – Other useful playlist: – – –

– – – Let’s Connect: – – –

Instagram: mrk_talkstech

– – – About me: – – –

Mr. K is a passionate teacher created this channel for only one goal "TO HELP PEOPLE LEARN ABOUT THE MODERN DATA PLATFORM SOLUTIONS USING CLOUD TECHNOLOGIES"

I will be creating playlist which covers the below topics (with DEMO)

1. Azure Beginner Tutorials
2. Azure Data Factory
3. Azure Synapse Analytics
4. Azure Databricks
5. Microsoft Power BI
6. Azure Data Lake Gen2
7. Azure DevOps
8. GitHub (and several other topics)

After creating some basic foundational videos, I will be creating some of the videos with the real time scenarios / use case specific to the three common Data Fields,

1. Data Engineer
2. Data Analyst
3. Data Scientist

Can't wait to help people with my videos.

– – – Support me: – – –

Рекомендации по теме
Комментарии
Автор

OMG, I even tried many UDEMY courses to understand this. None of the tutor explained this clearly .. iam loving it ... Sir please start full databricks course to help us. Please.. 🙏

MohanKrishna-yicc
Автор

simply great explanation about SPARK architecture, how its connected step by step it connects all the dots in Spark.

manikandan-fqsh
Автор

this is what i was looking for well explained. thank you

dprakash
Автор

this is so simple and clear explanation that
it made to share to my friends.

keep making video
your efforts putting great impact in our life.

shyammaths
Автор

Great explanation. Do you have a full pyspark tutorial?

dogzrgood
Автор

Excellent! Thank you for explaining this.

sharaniyaswaminathan
Автор

Simplest and excellent explanation Mr K.

rakeshverma
Автор

Wow! Just mind blowing brother💥💥!! Looking for more DE fundamentals videos ✨♥️👌

nanthagopalm
Автор

Great primer @Mr. K! Thanks. Quick question - How does the driver program create task partitions for the plan? For example, if there are duplicates across two worker nodes, wouldn't the count be misrepresented if it simply adds 4500 and 5500? Does this get auto-handled or do we have to control the partitioning logic?

seethaba
Автор

will this same topic be covered in the other channel (Mr.K Talks Tech Tamil)

kirankarthikeyan
Автор

Hi, what tools used to create this type of videos. Please help.

zakeerp