17: Principal Components Analysis_ - Intro to Neural Computation

MIT 9.40 Introduction to Neural Computation, Spring 2018
Instructor: Michale Fee

Covers eigenvalues and eigenvectors, Gaussian distributions, computing covariance matrices, and principal components analysis (PCA).

License: Creative Commons BY-NC-SA

We encourage constructive comments and discussion on OCW’s YouTube and other social media channels. Personal attacks, hate speech, trolling, and inappropriate comments are not allowed and may be removed.

[SyedMohommadKumailAkbar]

Absolutely incredible teaching. started doing math again after a gap of 12 years and this just clicked.

[markdebono]

I have been watching videos on this topic for the last 2-3 months. However, it was ONLY after watching this lecture that I understood EVERYTHING! Not to discredit any of the other videos I watched, but this video is made at a very good pace and I did not need to pause a hundred times to give my mind time to process all the incoming information and cool down 😁 If I could give a thumbs up 10 times, I would do it!

[ksjksjgg]

Great thanks Prof. Fee and MIT for sharing this excellent lecture!!!

[prasanth_y_]

Wow, This is an amazing explanation by building the topic from basics, Even spectral clustering comes with a similar final analysis of the eigenvalues of the covariance matrix.

[ElectroCute]

This is nice, I tried to replicate the de-noise in Python using sine and cosine "signals" with some random noise (basically sine(timepoint) + np.random()) and realised that it does not work the way it is described here because the variance is always roughly the same in every dimension (i.e. every timepoint).

To be able to isolate the underlying trend using eigenvectors I had to skip the step Z = X - MU at 1:16:01, as that step causes the variance the be approximately the same in all dimensions. If we do not subtract the mean but define our "covariance" matrix as simply Z * ZT then our "variance" is actually higher when the underlying signal is higher and lower when the underlying signal is lower. That way I could isolate the signal.

Having said that, maybe I have done something completely wrong. This is MIT after all :)

[ganeshhampapura]

Superb clarity professor God bless you

[jonilyndabalos]

Thank you, MIT for sharing these awesome lecture series with the public. You make learning accessible to all, especially the underprivileged. Please keep these videos coming.

[CristianTraina]

Is really needed to zoom in and zoom out all the time? It doesn't let me focus on what I'm reading..

[djangoworldwide]

This cloud is basically the electron cloud, isnt it? amazing how everything just fits to that same radial gussain distribution

[lijisisi]

wonderful material! thank you so much Dr. Fee

[jorgegonzalez-higueras]

Thank you, Professor Fee, for a very clear explanation!

[et]

lol Lina always asks the exact questions I want to

[nshilla]

The application part of the lecture is at time 41.00

[djangoworldwide]

I dont believe anyone actually goes thru this process. nowadays its a simple line of code + biplot

[andrewfalcone]

Can someone confirm that lambda+ = a+b, lambda- = a-b is a mistake?
I couldn't simplify the radical and double checked with MATLAB's symbolic equation solver (but it doesn't always simplify correctly) and some numbers.

[jimmylok]

All those zoom in and zoom out make me dizzy...

[amiha]

40:24 Just tagging the PCA main part for my reference