Understanding Databricks & Apache Spark Performance Tuning: Lesson 02 - Spark Hardware

preview_player
Показать описание
Following up on Databricks Performance Tuning with the best place to start: allocating Spark clusters. If you don't allocate sufficient resources, nothing else will fix the problem. How many nodes? How large should the driver and workers be? Do you need GPUs or CPUs? Should you use Photon? These and many more questions will be covered in detail.

Support me on Patreon

Slides

Referenced Video by Daniel Tomes, Databricks

See the Full Playlist Here:
Рекомендации по теме
Комментарии
Автор

amazing explanation. are you planning to make more videos on performance tuning?

Ali-qdc
Автор

I think the main reason not to use photon, is that it costs extra^^

Noobsmove