Lecture 11B: Monitoring ML Models (Full Stack Deep Learning - Spring 2021)

Показать описание

New course announcement ✨

We're teaching an in-person LLM bootcamp in the SF Bay Area on November 14, 2023. Come join us if you want to see the most up-to-date materials building LLM-powered products and learn in a hands-on environment.

Hope to see some of you there!

--------------------------------------------------------------------------------------------- In this lecture, you will learn how to monitor models (aka, keeping them healthy in production).

00:00 - Introduction
01:10 - Model Performance Degrades Post-Deployment
04:00 - Data Drift
07:25 - What To Monitor?
10:42 - How To Measure When Things Change
24:07 - How To Tell If A Change Is Bad
26:19 - Tools For Monitoring
27:46 - Monitoring And Your Broader ML System
34:26 - Takeaways

Рекомендации по теме

Комментарии

These lectures are incredibly useful as intro to the field. Covers almost everything, practical and up-to-date, with just the right amount of theory and links for diving deeper.

hlinc

Lecture 11B: Monitoring ML Models (Full Stack Deep Learning - Spring 2021)

Lecture 11B: Monitoring ML Models (Full Stack Deep Learning - Spring 2021)

Monitoring ML Models

Lecture 11A: Deploying ML Models (Full Stack Deep Learning - Spring 2021)

How To Monitor Machine Learning Stacks

Arnaud Van Looveren - Monitoring machine learning models in production | PyData Global 2020

Automating Model Monitoring and Drift Detection

A simple way to validate and monitor the performance of your ML Applications.

#datalift: Monitoring ML models in production

Monitoring an ML Model in Production | Data Science Fundamentals

Monitoring ML Models in Production

Monitoring (6) - Testing & Deployment - Full Stack Deep Learning

MLOPS: Monitor, ML Models in Production - Hauke Brammer

Predictive Application Scaling with Prometheus and ML - Chris Dutra, Schireson

Monitoring your ML model with IntegratedML

Lecture 13: ML Teams (Full Stack Deep Learning - Spring 2021)

Monitoring ML Models & Data in Production

Model Monitoring Overview - Part 1

Machine Learning Performance Monitoring - Emeli Dral

Introducing Evidently, an open-source tool for ML model monitoring

Data and model monitoring using Python function decorators

Monitoring Half a Million ML Models, IoT Streaming Data, and Automated Quality Check on Delta Lake

Production Machine Learning Monitoring: Principles, Patterns and Techniques

How to Monitor Machine Learning Models (Evidently AI)

Monitoring ML Models at Production: A case study for Medical Imaging using Weights and Biases