2. pyspark interview questions and answers | kpmg pyspark interview questions and answers

Azure Databricks #spark #pyspark #azuredatabricks #azure
In this video, I discussed kpmg pyspark interview questions & answers.

Create dataframe:
======================================================
-----------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------
============================================================

1. kpmg pyspark interview questions and answers | pyspark interview questions and answers :

Learn PySpark, an interface for Apache Spark in Python. PySpark is often used for large-scale data processing and machine learning.

Azure Databricks Tutorial Platlist:

Azure data factory tutorial playlist:

ADF interview question & answer:

[shivaniverma]

data=[('Goa', ' ', 'AP'), ('', 'AP', None), (None, ' ', 'Bglr')]
columns=["city1", "city2", "city3"]
df=spark.createDataFrame(data, columns)
display(df)

[satishgs]

from pyspark.sql.functions import coalesce, col
df.na.replace("", None).withColumn("output", coalesce(col("city1"), col("city2"), col("city3"))).show()

[Themotivationstationpower]

df1 = df.withColumn("City1", when(col("City1").isin("", "null"), \
.withColumn("City2", when(col("City2").isin("", "null"), \
.withColumn("City3", when(col("City3").isin("", "null"),

df1= df1.withColumn("NewColumn", coalesce(col("City1"), col("City2"), col("City3")))
df1.show()