CompX: Mathematics of PCA - Covariance matrices

Computational Thinking and Big Data is part of the Big Data MicroMasters program offered by The University of Adelaide and edX.

Learn the core concepts of computational thinking and how to collect, clean and consolidate large-scale datasets.

Enrol now!

[raghunandan]

Purpose of internet and youtube is served through content like this. Thank you so much

[vardhilmehta]

He's writing on glass and he has to write everything, every number backwards so that it appears normal on camera!
Hats off to you for that! 🙏

[nilsonsls]

I don't know why it's so difficult for some people to explain this subject. Thank you so much!

[thomaspiechulek]

i searched for you so long. now that i found you: i love you

[YichongMa]

Thank you so much! I've been trying to figure out the variance-covariance mtx and none of the resources made sense. You clearly explained in 5 min something I have been scratching my head for hours!

[abdulcustom]

I don't know what's more impressive, the good explanation or the fact that he wrote everything inverted

[TanishaPrasannaRA]

Wow man I understood this one so easily, I'm crying. I still can't understand why it was so hard for other youtubers to explain this. I've liked and subscribed, you deserve it, lots of love<3

[robindalehayden]

For COV(Y, Z) I worked it out as y1 - y mean = 1-2.5 = -1.5, z1 - z mean = 1 - 1.75 = -.25 ...etc. So I get ((1 - 2.5)*(1 - 1.75)+(2 - 2.5)*(1 - 1.75)+(3 - 2.5)*(2 - 1.75)+(4-2.5)*(3-1.75))/4. That works out to 0.875, so I'm not sure how you got 1.167. What am I doing wrong?

[sachinahankari]

The great person with simple and lucid explanation.. The greatest truths are the simplest

[TANMAYSANJAYGUPTABCE]

the value of the denominator in the given formula should be n-1... I was taking the value as n and worrying why my answers were incorrect. Kindly rectify your mistake

[sukumarroychowdhury]

The explanation is very good. However, the denominator for COV(X, Y) should be ( n-1). IIT, Kharagpur, India. In oil & gas applications, ideal independent variables are expected to have low covariance. Where a pair of independent variable have high co-variance, one of them must be dropped to get eigenvalues and eigenvectors to arrive at manageable PCA (Principal Component Analysis).

[JosekJak]

Thank you for the explanation, it was useful. However please note that you use n-1 not n to calculate the covariance betweeen vectors.

[RahulKumar-djbt]

Please correct the formula as n-1 in denominator Thanks

[chanpol]

I may doing not corrected addition for cov (y, z) to get different than 1.166?

[Nissearne]

Thanks again. I was try calculate but I get all your number when I use n=3. When I try use n=4 then I go the the cov(Y, Z) = 0.875. Should the covariance matrix contain the 1/n where n= the covariance matrix row size ?
Best Regards.
Olle Welin

[jubeidono]

Was the next video about combining this with Eigendecomposition for PCA ever published?

[ghazisanaullah]

is it 1/N or 1/N-1 in start of formula

[mukulsinghal]

In the case of a sample then the denominator value should be n-1 instead of n.

[nichtszusagen]

ForSamples we devide by n, for Samples we devide by n-1 in the given covariance Formula.Thanks for your nice explanation

[AI-xijk]

In Python numpy:
>>> import numpy as np
>>> m = np.array([[1, 1, 1], [1, 2, 1], [1, 3, 2], [1, 4, 3]])
>>> np.cov(m, rowvar=False, bias=False)
array([[0. , 0. , 0. ],
[0. ,
[0. ,
where:
bias : bool, optional

Default normalization (False) is by (N - 1), where N is the
number of observations given (unbiased estimate). If bias is True,
then normalization is by N.