Introduction to Distributed Computing with the Ray Framework

In this video, I give a brief introduction to distributed computing concepts and show how the Ray framework provides elegant abstractions for scaling data science in Python.

Stay in the loop! 👇

#python #ray #datascience

✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎⚡︎✌︎

0:00 - Introduction
2:07 - Introduction to the Ray framework
3:30 - Conceptual introduction to distributed systems
5:38 - Challenges of distributed systems
7:25 - Ray internals
11:33 - The larger Ray ecosystem
14:16 - Outro

[Nick-tvpu]

This is my second time going through your Ray videos, about 6mo after first watching them.

I just want to say not only is this some of the most thorough and digestible Ray content I've come across, its also some of the best technical tutorial/explanation content I've come across. You have a real knack for this stuff.

[MSalman]

Excellent explanation with simplicity and depth.

[SuperOnlyP]

After going through some basic tutorial from ray-core introduction, 7:25 is really helpful to have overview of it .thanks !

[curumo_curunir]

One of the clearest explaination about what is ray and why its useful. Kudos! Subscribed.

[juanpabloguerra]

This is great. Thank you for putting this together and I look forward to check out more Ray focused videos.

[thefullbridgerectifier]

Hi Jonathan - This is your 450th subscriber speaking. Just found this series and absolutely loving it! Please continue making more content on Ray (and maybe RLLib!)
Lets get you to 100k soon!

[cansucandan]

Thank you for the information you share! This video really helped to understand Ray :)

[argmax]

Awesome video ! Can yo cover more of the high level APIs Ray offers? And things like connecting to different - hybrid - clusters ?

[wyattholden]

Does a cluster have a max amount of workers it can hold?

[syedsohail]

Hey, when I start ray on my own system it gets started and I connect another laptop using ray start
The laptop gets connected to my computer but when I run heavy task only I see my computer is using 80% memory and 80% cpu while the laptop (worker) doesn’t use much cpu and memory it’s just getting connected but not using all the resources. I wonder why
Do you have any idea where am I going wrong?

Thanks

[SenthilKumar-kddm]

Is this applicable only to reduce time in maths computation ? why GPU is used then ? please explain

[tljstewart]

Hmm whats the difference between Ray and MPI? 🤔