filmov
tv
Design Principles of Distributed Systems with Dask and PySpark
Показать описание
On this week's Science Thursday, Holden Karau joins Matt Rocklin & Hugo Bowne-Anderson to discuss the design of Dask, how it compares to PySpark, and why these tradeoffs were chosen.
00:00 Going live!
01:35 Introducing Holden
03:15 What is Dask?
06:10 Princess of the Covariance Matrix
08:41 Holden's introduction to Dask
10:50 Why learn Dask in public?
12:50 Difficulties Holden faced when first learning Dask
16:37 Dask and supporting packages
19:50 Architectural differences between Spark and Dask and the impact
28:10 Local modes of Dask
32:30 How does Spark manage software environments for distributed clusters?
34:40 Dask adaptive scaling
44:45 Dask and memory
48:40 Resource management in Dask
55:55 Wrapping up
00:00 Going live!
01:35 Introducing Holden
03:15 What is Dask?
06:10 Princess of the Covariance Matrix
08:41 Holden's introduction to Dask
10:50 Why learn Dask in public?
12:50 Difficulties Holden faced when first learning Dask
16:37 Dask and supporting packages
19:50 Architectural differences between Spark and Dask and the impact
28:10 Local modes of Dask
32:30 How does Spark manage software environments for distributed clusters?
34:40 Dask adaptive scaling
44:45 Dask and memory
48:40 Resource management in Dask
55:55 Wrapping up
0:03:38
0:48:07
0:06:14
0:06:33
0:56:21
0:11:25
0:06:05
0:09:41
0:05:33
0:07:10
0:03:03
0:20:44
1:25:07
0:02:04
0:12:47
0:11:41
0:04:13
0:37:53
0:53:38
0:05:53
0:15:19
0:09:22
0:29:42
0:08:42