Ray: Enterprise-Grade, Distributed Python

preview_player
Показать описание
1. Low-latency scheduling and execution of small-to-large ‘tasks’ that perform a wide variety of computation chores, with logical sequencing of dependent tasks.
2. Management of ‘arbitrary’, distributed state, with thread-safe updates and access from other Ray tasks across a cluster.
3. Near-linear scaling.
4. An intuitive API that hides complexity from the user.

Ray has been used for reinforcement learning, hyper parameter tuning, model serving, and other applications in clusters up to thousands of nodes. I’ll discuss examples that illustrate how Ray can be used with Spark to build robust, scalable data applications for enterprises, when to use Ray versus alternative choices, and how to adopt it in your projects.

About:
Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Connect with us:
Рекомендации по теме
Комментарии
Автор

How to initialize ray on Databricks? It's throwing error. Can you share any example notebook for ray usage on Databricks?

madhuful
Автор

Thanks for a nice introduction, really helped me as a Spark practitioner to understand the differennce

GeorgeVorobiov
visit shbcf.ru