3.7 Apache Spark Tutorial | Spark Broadcast Variables

As part of our spark Interview question Series, we want to help you prepare for your spark interviews. We will discuss various topics about spark like Lineage, reduceby vs group by, yarn client mode vs yarn cluster mode etc.

As part of this video we are covering What is Spark Broadcast variables

Please subscribe to our channel.
Here is link to other spark interview questions

Here is link to other Hadoop interview questions

[DataSavvy]

Apology For Quality of Audio in Video. I will create a new Video with Better Voice quality

[vijayanand]

How come even after unpersist, the map function works? Unpersist is like de-broadcast right in our scenario?

[vdrvideoschannel]

Thank you so much brother. Very useful video.

[paulfunigga]

I thought that db connections were not serializable? That's why you can't broadcast them to executors?

[dikshachourasiya]

Can we create Hive tables using Spark DF only, can't we create hive tables using spark RDD??

[pareshbapna]

Hi
can we use broadcast variables as DB connection string, asking because I suppose DB connection is being only used by driver and not by executors but not sure of this..

[gobieee]

Good Explanation, liked it... have couple of question, when you bundle (say jar file) the code, then dp (broadcast map) is also part of the bundle. isn't it ? then db map is available all the worker node where ever bundle is available..
if I dont want to do broadcast, then what would be the code change I need to make?
also, bit unclear about the concepts, if someone broadcasting something, then there should be some receiver isn't it? can you help how the worker node receiving it, how it is assigned to broadcasted variable in the worker node ?

[allwinjayson]

Please upload the recent updated spark interview questions and answers

[ravireddy]

Hi, how can i read the file(one am going to broadcast) from hdfs into spark and create a key, value pair ?? when you get using textFile, it will be RDD[String}...how can i convert this to Map[Key -> Value] ??

[vishalaaa]

make video on ci/cd and git integration - databricks, git, adf

[sreedivya]

Hi could you please explain the concept of salting on skewed dTa with a simple example

[rajatsaha]

is it possible to broadcast a dataframe??

[techtonicTushar]

Can you please make a video on the difference between mapPartitions and mapPartitionsWithIndex?
I am having some issue with this topic.

[watchmanling]

why your broadcast variable has get method?

[San-hszx]

Can you please make a video, how to solve group by out of memory issue ? Interviewer asked me how to solve out of memory issue without code changes ? Please explain with code changes and without code changes

[juniorfarmer]

sir can you post video about setMaster[local]

[ravurunaveenkumar]

hi sir can you explain accumulators with example

[nagarajua]

kindly do the video on another shared variable i.e accumulators

[vinaykumar-rdbv]

Hi Sir, Its nice video !!! can we have url (video) how the real time video how we can deploy the spark code .
step by step process

[deepakkini]

Make a video on how to convert an Rdd to a dataframe