Master Databricks and Apache Spark Step by Step: Using Scala Dataframes & Datasets

In this video you learn how to use Scala Spark dataframes and datasets and how to get the most out of them. It's all demonstration and the link to the code is below.

Join my Patreon Community and Watch this Video without Ads!

Code Available at:

For training on how to create the tables, see:

[monkeymaster]

Thanks for reinforcing the advantages/marginal benefits of Scala over PySpark. It's something I've been trying to gain some clarity on. How about when you need to perform custom transformations on a column/columns? With PySpark, you'd use UDFs. Do those perform better with Scala?

[Andy-rwhn]

Just finished entire playlist. Thanks a lot for the effort of putting it together. Are there any major changes in Databricks since 2021 though?