So Long Hadoop - Running Spark On Kubernetes by Erik Schmiegelow

preview_player
Показать описание
"So Long Hadoop - Running Spark On Kubernetes" by Erik Schmiegelow at Functional Scala 2022.

For as long as we remember, running Spark in production was tied to Hadoop, a clunky, cumbersome framework. For easier integration and better performance, we can now free ourselves and run our Spark apps next to our Scala microservices.

This talk demonstrates the benefits of Spark on Kubernetes; how to customize it, and how it can make running Spark in production environments a more enjoyable experience.

Contents in the video:

00:14 Spark - Intro
12:23 In A Nutshell
14:21 Spark on Hadoop
18:32 Spark on K8S
23:19 In A Nutshell

#FunScala2022 #scala #zio #functionalprogramming #functionalscala #scalaconference #scaladeveloper
Рекомендации по теме
Комментарии
Автор

Excellent refresher on Spark with a k8s twist. Thank you!

maxreuv
Автор

I feel like most of the companies currently moved / moving away from direct config. of spark clusters to some more convenient to use: Databricks.

What's your opinion on that?

inny