How to implement PCA (Principal Component Analysis) from scratch with Python

In the 7th lesson of the Machine Learning from Scratch course, we will learn how to implement the PCA (Principal Component Analysis) algorithm.

Welcome to the Machine Learning from Scratch course by AssemblyAI.
Thanks to libraries like Scikit-learn we can use most ML algorithms with a couple of lines of code. But knowing how these algorithms work inside is very important. Implementing them hands-on is a great way to achieve this.

And mostly, they are easier than you’d think to implement.

In this course, we will learn how to implement these 10 algorithms.
We will quickly go through how the algorithms work and then implement them in Python using the help of NumPy.

▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬▬

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

#MachineLearning #DeepLearning

[luisxd]

Wow, amazing video of the course! I liked the theory part and how it is implemented with numpy 😄👍
It was all well explained, thanks! 😄👏💯😁

[business_central]

all the ones explained by the girl are very clearly explained and walked through, this guy seems he just wants to be done and he is not really explaining much at all.

[kachunpang]

HI, I am wondering the output of 'np.linalg.eig(cov)' in line 20. According to NumPy documentation the first output is the eigenvalues and the second should be set of eigenvectors stored inside a matrix. However, in line 20 the you swap the names between eigenvector and eigenvalues but still get a pleasant plot after PCA. Could someone explain this part to me? Thanks.

[ernestbonat]

You should implement PCA with NumPy only. In fact, you need to use NumPy everywhere possible. The NumPy is the faster Python numerical library today. We should not teach based on some student understanding definition. We should teach students with real Python production code for them to find a job only. Everyone needs to pass the job interviews.

[martinemond]

How would you go about reconstructing the original data from the X_projected based on PC1 and PC2, which kept only 2 dimensions from the original 4 dimensions?

[MinhNguyen-clpq]

Line 19 seems to have a bug, as return values should be swapped based on Numpy documentation

[michelebersani]

Good morning, this playlist is amazing and I was searching it for several weeks. I have a question about the interpretection of the eigenvectors. Why do the eigenvectors, of the covariance matrix, point in the direction of maximum variance?

[ASdASd-krft]

Nice video!, but i have one doubt, why you have more variance in the principal component 2 than principal component 1, is it cuz the scale?

[eugenmalatov]

Sorry, the theory part did not explain anything to me

[igordemetriusalencar]

Excellent video and beautiful OOP python programming, clean and easy to understand for a programmer, but OOP in data analysis is terribly ugly and not productive with a lot of not necessary abstraction with classes and methods. The functional paradigm is way way better for data analysis due to its easy (initial) concepts of data flow and functions that transform the data. This way anyone that learned "general system theory" could understand (managers, biologists, physicists, psychologists...) if you could do the same in a functional way would be amazing! (in Python, R, or Julia).

[chyldstudios]

You should implement PCA without using Numpy, just vanilla python (no external libraries). It's more pedagogically rigorous and leads to a deeper understanding.

[thejll]

Could you show how to do pca with gpu?

[MVR_]

> states 'from scratch'
> proceeds to import numpy