Azure Databricks | How to Pass/Share Variable between Python/Scala/SQL

#dataengineering #databricks #apachespark
Azure Databricks | How to Pass/Share Variable between Python/Scala/SQL

In this Video, we will learn how to pass or how to share a variable across python/Scala/SQL in databricks notebook.

===================

More Videos on Apache Spark in Databricks:

===================

Blog link to learn more on Spark:

Linkedin profile:

FB page:

[prasadb]

You saved me❤. Is there any way to pass Scala dataframe to python?

[Abbulikesplanes]

Hello - Could you please make a video or share details about how to deploy the delta tables from one environment to another?

[ericzhang]

another question: when sharing variable with SQL, the key name has to be two parts like "myapp.var1"? I tried to name the key with only one part name like "my_new_key", but SQL couldn't find the corresponding value. Thanks!

[ericzhang]

it seems like if I change the variable in the source, the target variable won't change accordingly unless I rerun the spark.conf.set?

[dr]

Hi Azaruddin,

Can you make a video on how to convert exists clause inside where clause in SQL to pyspark

[nikhilkumar-yjzk]

how to pass timestamp between python, sql, scaa?