Expectation Maximization for the Gaussian Mixture Model | Full Derivation

Показать описание

Gaussian Mixture Models (GMMs) are extremely handy for clustering data. For example, think of clustering the grades of students after an exam into two clusters, those who passed and those who failed. For this we have to infer the parameters of the GMM (cluster-probabilities, means and standard deviations) from the latent. However, since the class node is latent we have to resort to an Expectation Maximization and the whole Maximum Likelihood Estimate will turn into an iterative procedure.

In this video we start at the derived general equations and fully derive all equations for the E-Step and the M-Step with NO EXCUSES - every derivative, manipulation and trick is presented in detail *.

The interesting observation is that although the EM implies we would need an expectation and maximization in every iteration, this is actually not the case. For the GMM, we can derive straight-forward update equations.

* If something is still unclear, please write a comment :)

-------

-------

Timestamps:
00:00 Introduction
01:10 Clustering
01:40 Infer Parameters w\ missing data
03:05 Joint of the GMM
04:45 E-Step: Un-Normalized Responsibilities
10:29 E-Step: Normalizing the Responsibilities
11:13 M-Step: The Q-Function
15:27 M-Step: Maximization formally
16:57 M-Step: Lagrange Multiplier
20:20 M-Step: Cluster Probabilities
30:50 M-Step: Means
35:00 M-Step: Standard Deviations
39:37 Summary
42:52 Important Remark
43:37 Outro

Рекомендации по теме

Комментарии

this is legitimately such a great explanation. thanks! <3

agrawal.akash

11:30 isn't it a lower bound of marginal log-likelihood instead?

vslaykovsky

There was an error on the hand-written M-Step in the beginning of the video. For the first 3 minutes I was able to overlay it. Please refer to this as the correct expression for the M-Step.

MachineLearningSimulation

How are you sure that the zeropoints of Q are maxima? Couldnt it be a saddle point or minima as well? Or did you just skip the part where you have to check the second derivatives?

patrickg.

Hi, what about EM algorithm for one bivariate Gaussian with missing values

bartosz

Is it possible to have the Gaussian distributions be latent and have the class be non-latent? Basically the continuous variable is latent now? What would this look like?

nickelandcopper

How about syntax in R if we want applied in survival mixture model?

sulasrisuddin

Just wondering. Could such EM approach work well in cases where X are high dimensional?

orjihvy

Expectation Maximization for the Gaussian Mixture Model | Full Derivation

Clustering (4): Gaussian Mixture Models and EM

Expectation Maximization for the Gaussian Mixture Model | Full Derivation

The EM Algorithm Clearly Explained (Expectation-Maximization Algorithm)

EM Algorithm : Data Science Concepts

Gaussian Mixture Models (GMM) Explained

Expectation Maximization | EM Algorithm Solved Example | Coin Flipping Problem | EM by Mahesh Huddar

Visualizing Expectation-Maximization for Gaussian Mixture Models

Visualization of EM-algorithm with a Gaussian Mixture Model

Deriving the EM Algorithm for the Multivariate Gaussian Mixture Model

visualization of training gaussian mixtures with expectation-maximization

The expectation maximization algorithm for the Gaussian mixture model

MLVU 8.4: Expectation-maximization from first principles

EM-algorithm for gaussian clustering: The intuition behind an important Machine Learning concept

Gaussian Mixture Model

Applied Machine Learning. Lecture 18. Part 3: Expectation Maximization in Gaussian Mixture Models

Cornell CS 5787: Applied Machine Learning. Lecture 18. Part 2: Expectation Maximization

Gaussian Mixture Models (GMMs): Expectation-maximization algorithm for GMMs.

EM Algorithm In Machine Learning | Expectation-Maximization | Machine Learning Tutorial | Edureka

Mixture of Gaussian Model implementation using EM algorithm

5.9 The Expectation Maximisation Algorithm

9.4 Gaussian Mixture Models And Expectation Maximization (UvA - Machine Learning 1 - 2020)

Expectation Maximization (EM) - 3.1 - GMM example - same mean, different variance

19-d LFD: Expectation Maximization (EM) algorithm for fitting a GMM to data.

#46 EM Algorithm - Expectation Maximisation - Steps, Usage, Advantages & Disadvantages|ML|