Least Squares Formula PROOF

Linear least squares is a method commonly used to fit curves to data. The equation used for least squares here is derived using the column space of A. The 'best' value of X is the one that minimizes the distance between the vector b and [A]X. Most math textbooks stop at the result: A^T A X = A b and use different algorithms to find the best X. The pseudo-inverse is commonly used to solve this final equation since it is based on Singular Value Decomposition (SVD) - but that's a talk for another video.

Chapters:
0:00 Intro
0:08 Previous Video Summary
0:36 Problem Definition
1:16 Column Space of A
2:42 Orthogonality
3:40 Calculations & Result
5:16 Confirmation
5:41 Pseudo-Inverse
6:07 Outro

Music:
Nuclear Lynx - Discovery
--

[martinzapata]

This proof uses the minimising vector theorem, but I guess it’s fine not to explain it since the presentation is very visual. Very clear, I like it!

[JUXIX]

Best video I've seen on this topic thus far

[MrRyanroberson]

Notice (AB)^-1 = B^-+ A^-1 and matrices are associative, therefore the final result is A^-1 b after simplification, so really this can be taken as a fancy way of inverting irregular shaped matrices

[SuperSeaguard]

I wish I had visualizations like this while learning calculus / differential equations.

We humans are visual learners. Math without visualization is very difficult to learn.

[chadx]

Very nice, now its stuck in my brain.

[ShalevWen]

Since A has only 2 columns, its rank is at most 2 and so is the rank of A^t*A, which means that if you have more than 2 data points there is no inverse

[aniruddhvasishta]

Awesome! When I saw that everything was on a plane I assumed you would find the orthogonal projection of b with the projection matrix that I vaguely recall from my intro linalg class. This solution seems more elegant tho.

[zeroheisenburg]

Cool Video. May I asked what did you use to draw animation for 3d planes and coordinates system?

[nehalkalita]

Which software do you use for the animations?

[dothuong-fe]

Help me explain sensitivity of variable A

[AlokKumar-kkpj]

Sir please don't put background music. Great content

[DavidKenny]

How much more complicated would it be if the pins were not in the shape of a circle, but maybe the shape of a horse? A picture in a silhouette.

[gigastein]

Sir can you make a whole playlist of mechanics, I really understand you really well. Please sir

[juanramonvazquez]

bro, change the title while not many people has seen it yet lol (typo)

[dariusgoh]

Doesn’t this proof only make sense for when A has 2 columns . Then the column space of A is a plane