ML Drift: Identifying Issues Before You Have a Problem

Over time, our AI predictions degrade. Full Stop.

Whether it’s concept drift, where the relationships of our data to what we’re trying to predict has changed, or data drift, where our production data no longer resembles the historical training data, identifying meaningful machine learning drift versus spurious or acceptable drift is tedious.

In this 15 minute overview you’ll learn about the different types of ML drift and how to monitor for the early warning signs. We’ll also cover strategies to intervene before “drift” impacts the bottom line.

0:00 Introduction
2:24 How We Experience ML Drift
5:40 Drift Examples for a Loan Application Moc
7:20 Triggers of ML Model Drift
8:34 Detecting Drift Issues
10:01 Data Drift Monitoring & Unsupervised Lea
12:20 Getting to the Root Cause
13:33 Drift Analytics

——————

[googol-boy-data]

hey congratulations for the presentation. It was very concise, I learned a lot and I really loved the references.

[hossein_haeri]

in 4:50, Is mathematically even possible to have label drift and not feature drift when drift is not real? (the same p(x) but different p(y)). Bayes rule tells us p(y|x) = p(x, y)/p(x). This means if p(y|x) and p(x) are fixed, then p(x, y) is fixed, hence, p(y) is fixed.
Long story short, the mathematics of this presentation is wrong! Virtual drift is when p(x) changes and real drift is when p(y|x) changes.