Principles Of Good ML Systems Design

💻 Abstract:
Principles Of Good ML Systems Design. This talk covers what it means to operationalize ML models. It starts by analyzing the difference between ML in research vs. in production, ML systems vs. traditional software, as well as myths about ML production. It then goes over the principles of good ML systems design and introduces an iterative framework for ML systems design, from scoping the project, data management, model development, deployment, maintenance, to business analysis. It covers the differences between DataOps, ML Engineering, MLOps, and data science, and where each fits into the framework. It also discusses the main skills each stage requires, which can help companies in structuring their teams. The talk ends with a survey of the ML production ecosystem, the economics of open source, and open-core businesses.

🔊 Speaker bio:
Snorkel AI - Machine Learning Engineer & Open Source Lead
Chip Huyen works to bring the best practices to machine learning production. She’s built AI applications at Snorkel AI, Netflix, NVIDIA, and Primer. She graduated from Stanford, where she taught TensorFlow for Deep Learning Research. She’s also the author of four bestselling Vietnamese books.

Timestamps:

0:00 Intro
0:12 Getting to know Chip Huyen
2:47 Table of content
3:12 ML in research vs in productions
3:23 Objectives (Research)
3:53 Objectives (Production)
5:42 Computation of priority (Research)
5:57 Computation of priority (Production)
6:19 Difference between latency and throughput

ML Production Myths

9:52 Myth 1: Deploying is hard
11:33 Myth 2: You only deploy one or two ML models at a time
15:08 Myth 3: If we don't do anything, model performance remains the same
18:08 Myth 4: You won't need to update your models as much
20:22 Myth 5: Most ML engineers don't need to worry about scale
21:31 Myth 6: ML can magically transform your business overnight
23:24 Myth 7: Efficiency improves with maturity

Four Phases of Machining Adoptions

24:38 Phase 1: Before ML
26:15 Phase 2: Simplest ML models
28:25 Phase 3: Optimizing simple models
29:29 Phase 4: Don't be afraid of rules-based
30:00 Principles of good ML systems design

❓ Q&A section ❓

31:32 How do you track which versions of training features are used?
32:12 How do you check which version of training fishers are used?

34:30 Closing remarks

[tkirhgl]

These are amazing presentations but the slides are a bit blurry on all the videos on your channel, would be great if you could fix that in the future. Thank you.