filmov
tv
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
Показать описание
In this video from 2018 Swiss HPC Conference, Torsten Hoefler from (ETH) Zürich presents: Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis.
"Deep Neural Networks (DNNs) are becoming an important tool in modern computing applications. Accelerating their training is a major challenge and techniques range from distributed algorithms to low-level circuit design. In this talk, we describe the problem from a theoretical perspective, followed by approaches for its parallelization.
Specifically, we present trends in DNN architectures and the resulting implications on parallelization strategies. We discuss the different types of concurrency in DNNs; synchronous and asynchronous stochastic gradient descent; distributed system architectures; communication schemes; and performance modeling. Based on these approaches, we extrapolate potential directions for parallelism in deep learning."
"Deep Neural Networks (DNNs) are becoming an important tool in modern computing applications. Accelerating their training is a major challenge and techniques range from distributed algorithms to low-level circuit design. In this talk, we describe the problem from a theoretical perspective, followed by approaches for its parallelization.
Specifically, we present trends in DNN architectures and the resulting implications on parallelization strategies. We discuss the different types of concurrency in DNNs; synchronous and asynchronous stochastic gradient descent; distributed system architectures; communication schemes; and performance modeling. Based on these approaches, we extrapolate potential directions for parallelism in deep learning."
0:44:22
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
1:00:31
High-Performance Communication Strategies in Parallel and Distributed Deep Learning
0:44:17
Parallel/Distributed Deep Learning and CDSW
0:24:19
A friendly introduction to distributed training (ML Tech Talks)
0:10:46
Parallel Training of Deep Networks with Local Updates
0:56:39
Alpa: Automating Inter- and Intra- Operator Parallelism for Distributed Deep Learning
0:17:39
OSDI '22 - Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning
0:19:47
An Uber Journey in Distributed Deep Learning
1:12:44
Demystifying Machine and Deep Learning for Developers : Build 2018
0:01:01
On the Acceleration of Deep Learning Model Parallelism With Staleness
0:29:42
Distributed Deep Neural Network Training using MPI on Python
0:42:07
A Resource Efficient Distributed Deep Learning Method without Sensitive Data Sharing | MIT
0:09:32
Model vs Data Parallelism in Machine Learning
0:14:25
OSDI '21 - P3: Distributed Deep Graph Learning at Scale
0:34:02
Distributed Deep Learning with Apache Spark and TensorFlow with Jim Dowling (Logical Clocks AB)
0:52:25
Tackling the Communication Bottlenecks of Distributed Deep Learning Training Workloads
0:45:18
Efficient Distributed Deep Learning Using MXNet
0:43:49
Weekly #94: Distributed deep learning
0:38:06
AutoML20: Demystifying NAS in Theory and Practice
![[SPCL_Bcast] Distributed Deep](https://i.ytimg.com/vi/Ixxhh8lftbI/hqdefault.jpg)
0:40:26
[SPCL_Bcast] Distributed Deep Learning with Second Order Information
0:27:38
Neil Gibbons - Demystifying Spark: A Deep Dive into Its Workings - SPS24
0:20:50
Generalized Pipeline Parallelism for DNN Training
![[Uber Seattle] Horovod:](https://i.ytimg.com/vi/fZ9P1v1jtFM/hqdefault.jpg)
0:22:29
[Uber Seattle] Horovod: Distributed Deep Learning on Spark
0:28:49
Distributed deep learning and why you may not need it - Jakub Sanojca, Mikuláš Zelinka
Комментарии