ModCon 2023 Breakout Session: MAX Engine Performance

In this session, Modular engineers Abdul Dakkak and Hengjie Wang discuss Modular AI Engine performance across models and hardware architectures. They dive deep into how AI Engine works and show its performance against Pytorch and TensorFlow and demonstrate how AI Engine scales to models of all sizes including LLMs.

- 00:00 Introduction and performance numbers
- 04:33 Runtime, compiler, and kernels working in unison
- 05:30 Runtime parralelism and memory management
- 05:50 Moving transforms out of inference to model initialization
- 06:04 Automatic fusion of graphs to a single op
- 06:30 Specialized kernels on dimensions
- 08:49 Simplification with Mojo
- 10:26 Generality across hardware
- 11:51 Cross platform development example
- 13:22 Kernel JIT
- 13:43 Develepor friendly
- 14:02 Autotuning, Custom Ops, Multi-model support
- 14:58 Stable diffusion example