spark ecosystem | Lec - 4

In this video, I have talked about all the component of saprk. And how it stich to all of the component to run an spark application.

For more queries reach out to me on my below social media handle.

My Gear:-

My PC Components:-

[nilavnayan]

Manish bhai, watched all your last 4 videos (this series) in one sitting and loved it - waiting for more :)
Edit - Not just watched laet ke - but copy kalam ke saath :D

[Mitra-xbtz]

So, can we use HDFS cluster upon which Spark will act for data processing ?

[sairamguptha]

Hi Manish, you are doing an awesome job... Also pls try to do more vidoes on spark practicals like how to practice spark on data bricks account.

[kartikmod]

Hi, As per the video if the situation occurs when the cluster manager is down then the request will not go further, is this a possible case?

[gchanakya]

Bhaiya dsa python me hi karu ya fir muje java ya c++ me karna chahiya. Apne kya kiya.

[praveenkumarrai]

bro you said we will use dataframe and spark sql then what is the use of python ? I am really confused.

[alakanandade]

@manish_kumar_1 does hadoop create replication of data for intermediate data as well? And how are these deleted later on?

[navjotsingh-hljg]

manish bhai what are the roles of executor? executor means execute the data am i right ? or what is driver bro?

[saumyasingh]

Your videos are great...Can you pls make these little longer, capturing more concepts in one videos.

[explorewithaAB]

Since spark is open source, so who does pay for this hardware. Since I think that hardware is nothing but a physical machine so there has to be someone who is paying right?