(Multivariate) Sum of two Quadratic Forms | Full Derivation with detailed explanation

Quadratic forms use matrix-vector multiplications to encode multidimensional polynomials of total degrees of two. For instance, if you have two variables, say x_0 and x_1, the polynomial is allowed to have terms of the form x_0^2, x_1^2, x_0*x_1, x_0, x_1 and 1. If you add two of these quadratic forms, you get another (makes sense because the polynomial total degree does not change by addition).

In this video, we are going to derive a closed-form solution to do these additions in a simpler way. For this, we first derive the univariate (=scalar) case and look at it with an example. Then we use our knowledge to derive the multivariate case. It will be a bit tricky at some spots. I will use the phrase "inverse" quite often. Can you count how often?

-------

-------

Timestamps:
00:00 Introduction
00:38 A Quadratic Form is still a scalar
00:58 Deriving the Univariate Case
02:43 Univariate Completing the Square
05:14 Univariate Simplifying
07:24 Finishing Univariate Derivation
09:32 Univariate Example
11:30 Deriving the Multivariate Case
15:55 Multivariate Completing the Square
19:09 Multivariate Simplifying
25:50 Woodbury Matrix Identity
26:54 Multivariate Simplifying (cont.)
30:55 Finishing Multivariate Derivation
35:04 Final Thoughts
35:36 Outro

[jacobwalker]

hey man, wanted to say i've been scouring the internet for mathematical explanations for concepts typically found in intro to statistical learning, 100 page machine learning book, mathematics for ml etc etc and your content has explained what 50 or so other youtube videos and articles couldn't . Great work, really well presented, exceptional teacher!

[christiansinger]

This playlist is really helpful for understanding the calculations in books about probabilistic machine learning!