33. fill() & fillna() functions in PySpark | Azure Databricks #pyspark #spark #azuredatabricks

In this video, I discussed about fill() & fillna() functions in pyspark which helps to replace nulls in dataframe.

Link for PySpark Playlist:

Link for PySpark Real Time Scenarios Playlist:

Link for Azure Synapse Analytics Playlist:

Link to Azure Synapse Real Time scenarios Playlist:

Link for Azure Data bricks Play list:

Link for Azure Functions Play list:

Link for Azure Basics Play list:

Link for Azure Data factory Play list:

Link for Azure Data Factory Real time Scenarios

Link for Azure Logic Apps playlist

#PySpark #Spark #databricks #azuresynapse #synapse #notebook #azuredatabricks #PySparkcode #dataframe #WafaStudies #maheer #azure #pivot #unpivot #dataframe

[starmscloud]

Good One Maheer !

May be you can replace different null values different columns using key-value pair . something like this .

inputData = [(1, 'Alex', 5000, None), (2, 'Liam', None, 'uk'), (None, 'Duckett', 7000, 'Eng'), (4, 'Hope', 7800, None)]

inputSchema=["Id", "Name", "Sal", "Loc"]

inputDF = spark.createDataFrame(inputData, inputSchema)

inputDF.na.fill({'Id':'-1000', 'Loc':'unknownLoc'}).show()

[sravanthibijivemula]

Could you please make more videos on azure databricks

[manu]

Thanks bhaiii. Would you please explain date functions in azure.

[vineethsetty]

Hello maheer can we handle nulls in this way .withcolumn("gender", expr("coalesce (gender, -1)")) here if gender is there value remains same if there us null then will be replaced with -1