Deriving the VI by Mean Field Approach for the Posterior of the Normal with unknown mean & precision

Of course, for the Normal distribution fitted to data under unknown mean and unknown precision there is an analytical posterior - the Gauss-Gamma or Normal-Gamma distribution. But we can also use Variational Inference by the Mean Field Approach / Mean Field Approximation to derive a surrogate posterior.

It will turn out that this surrogate posterior is a product of two independent distributions; a Normal and a Gamma distribution. That is indeed an interesting observation and provides a "simple" example for using these techniques on Latent Variable Models.

-------

-------

Timestamps:
00:00 Introduction
00:53 The True Posterior
01:46 Distributions in the DGM
02:47 Recap: Variational Inference
03:19 Recap: Mean Field Approach
04:41 The joint distribution
08:58 The log joint distribution
11:29 Simplifications in the log joint
15:09 The final log joint
16:15 The big picture
16:32 Deriving q_μ - surrogate for μ
21:49 Deriving q_τ - surrogate for τ
27:32 Summary
29:08 Solving the Expectations
31:15 All equations derived here
33:36 Circular Dependency
34:38 Performance Note
35:49 Differences True vs. Surrogate Posterior
36:36 Surrogate MAP estimate
37:42 Outro

[orjihvy]

appreciate that you go through the steps in the algebra. for me, it really helps.

[haraldvoehringer]

Thanks for this great video! Would you mind to explain the step at 12:33 (sum of squared differences of the mean) ? Perhaps list the steps you left out. Thanks a lot.

[longfellowrose]

Thanks for this video. Do you have any recommended references or books on this subject for beginners? I want to further study the VI methods, but I don't know which material is easy to understand and follow. If you have some suggestions, please share with me lol~ THANK YOU~

[pravingaikwad]

how do we know there are only two parameters of surrogate distribution?