08 Working with Strings, Dates and Null

preview_player
Показать описание
Video explains - How to use Case When in Spark ? How to manipulate String data in Spark DataFrames? How to cast dates in Spark ? How to extract date portions in Spark ? How to work with NULL data in Spark ?

Chapters
00:00 - Introduction
01:08 - How to use Case When in Spark?
04:30 - String Regex Replace
06:00 - How to convert string to date in Spark?
08:10 - How to add current date or timestamp in Spark ?
10:07 - How to drop NULL records in Spark ?
10:50 - How to transform NULL Columns in Spark ?
12:18 - Fix DataFrame
14:00 - Bonus Tip

The series provides a step-by-step guide to learning PySpark, a popular open-source distributed computing framework that is used for big data processing.

New video in every 3 days ❤️

#spark #pyspark #python #dataengineering
Рекомендации по теме
Комментарии
Автор

Wonderful.. I ever seen these kind of teaching.. thankyou bro!! Please add more videos.

vijayvavilapalli
Автор

great content, Please keep adding more videos, very helpful.

anonymous-zefg
Автор

need to understand one thing why yyyy and dd not in capital letter is there any reason for that

irannamented
Автор

You're a very awesome guy. Your explanation is straightforward to understand. I have a few clarifications. Why do we have to import the libraries for each function? Is there an option to import the main libraries and achieve the same? For example, for the date conversion, you import date_format and the_date. I believe we can use Import *

marimuthukalyanasundram
Автор

Can we use na.fill to fill missing values, instead of coalesce?

aryans
Автор

Bro, what is the purpose of using coalesce here??

pranavganesh