Dot products and duality | Chapter 9, Essence of linear algebra

Why the formula for dot products matches their geometric intuition.
An equally valuable form of support is to simply share some of the videos.

Dot products are a nice geometric tool for understanding projection. But now that we know about linear transformations, we can get a deeper feel for what's going on with the dot product, and the connection between its numerical computation and its geometric interpretation.

Future series like this are funded by the community, through Patreon, where supporters get early access as the series is being produced.

Thanks to these viewers for their contributions to translations
Hebrew: Omer Tuchfeld
Vietnamese: @ngvutuan2811

------------------

3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted about new videos, subscribe, and click the bell to receive notifications (if you're into that).

Various social media stuffs:

[gorgolyt]

This series is basically perfect, congratulations.
It's strange that this subject is usually taught so badly. I attended a world class university and at no point did they properly discuss the intuition and motivation for the formal concepts.

[andrewdavies]

"Unlearn what you've just learned" - finally something i can do

[theunknownscientist]

I first watched this series 3 years ago. But each year, I rewatch it as I learn more and more maths. Each time, I realize that when I previously watched it, I missed something important and beautiful. It's amazing how much information there is in such a work, that can be easely be missed by the untrained eye, who can't appreciate the hidden connections of math.

[KamikazeBunnyGames]

Its a shame that mathematics has become such an infamous subject for the average person, otherwise this channel would be far more popular, which it really deserves.

[Brono]

I wish there was a 3blue1brown of physics also

[wpkzz]

I have a phD in physics. I was really good at geometry during my study years... and I have NEVER seen this. And not one of my (very good) professors talked about this in this way. I am REALLY shocked, in a good way. You are the best Math Theacher on line that there is. We need one of you for every language on Earth!.

[JordanPurcell]

"Sometimes you realize that it's easier to understand it (a vector) not as an arrow in space, but as the physical embodiment of a linear transformation - it's as if a vector is a conceptual shorthand for a linear transformation." Definitely one of the most beautiful ideas I have ever learned, thank you for articulating it so well.

[nabeelsherazi]

at this point, i've been pavlov conditioned to associate that piano sound with "you gon learn some shit"

[BharCode]

The much simpler real life application, I think, for finding a "dot product":
Imagine you and your friend are pulling a big rock, tied by 2 ropes. Your friend's rope is making an angle ø, with your rope, like V shape.

Now the *dot product* gives the actual amount of force your friend is adding to yours, given there is ø angle between you and your friend's direction of rope pulling.

If your friend stands in same line as you, pulls exactly in the same direction where you want to move the stone towards, (or both are pulling the same rope), angle ø=0, i.e Cos(0)=1, then thats the most efficient way of pulling the stone. Bcz all his force magnitude will absolutely add up to yours.

Otherwise, atleast, the lesser the angle ø between the forces, the higher the total magnitude will be.
But if your friends rope (force) direction is making some angle, ø with yours, then though he's putting some X amount of absolute Force, only a fraction of X (projection of your friend's on yours) will add up to yours.

If your friend makes an angle 90 degree with yours, i.e. he is pulling in perpendicular, then he's not adding up anything to your effort, cos(90)=0.

But if your friend is pulling the rope in opposite direction, cos(180)=-1==>Means your friend is working against you. Then the rock moves in the direction where the force is higher, but the amount of displacement would be much lesser, yourForce-friendForce.



So basically the Dot Product tells us, how much a vector is working FOR/AGAINST the other. It's called cosine similarities! This is also used in comparing 2 things (vectors) like how similar are they, do they add up to the entirety or negating each other..  

Man.. How I wish I had a board to write down, draw and explain..

[starship]

7:02 i feel stupid for just realizing that theres always "three blue and one brown" π in those talking animations

[imrsk]

I'm saving this playlist for my children.

[alvarol.martinez]

This series of videos is going to last. I think presents an almost purely geometrical explanation of linear algebra that can't be found anywhere else at the moment.

I believe this could be done with every math subject. As an undergraduate student of mathematics, I often find myself struggling to translate what the books say into a more intuitive mental image of what's going on. Of course, some subjects (e.g. linear algebra) are easier to visualize than others. But then again, mathematicians always try to build mental images of what they're working on, even if it is a very abstract subject. These mental images are crucial, and in my opinion they should be taught along with the (undeniably necessary) formal arguments. These videos show that animations are a very powerful tool to do this.

All of this could be said about your Multivariable Calculus Course as well.

Now, I realize it is a lot of work, but it would be great if you did the same thing with some other undergraduate subjects. Have you already considered this? If so, what subjects?

[wlzomiu]

After viewed almost 10 times, finally I interpret the essence of this episode .

[inaugurated]

For anyone who's lost like I was hours ago, I think I'm beginning to grasp it.

See the number line as a 1D vector space with basis vector u-hat, where u-hat has coordinates ux and uy. The basis vectors i-hat and j-hat define the 2D coordinate system. The coordinates ux and uy are projections of u-hat onto the x-axis and y-axis respectively.

Say you want to transform this 2D space into the 1D number line, such that L(i-hat) and L(j-hat) are projections of i-hat and j-hat onto the number line. You'll need a 1x2 matrix that will look like [L(i-hat) L(j-hat)]. As I wrote earlier, ux and uy are projections of u-hat onto the x-axis (i-hat) and y-axis (j-hat). Because of symmetry, the projections of i-hat and j-hat onto u-hat (the number line) will also be ux and uy. So, L(i-hat) = ux, L(j-hat) = uy! The matrix will be [ux uy].

Note that ANY vector, v, in the original 2D space is a linear combination of the basis vectors i-hat and j-hat. After the transformation, this still holds true: v = c1 i-hat + c2 j-hat and L(v) = c1 L(i-hat) + c2 L(j-hat). So, because L(i-hat) and L(j-hat) are projections of i-hat and j-hat, L(v) is a projection of v onto the number line!

Now, the vector equivalent of the transformation matrix [ux uy] is just u-hat. If you take the dot product between a vector v and u-hat, you do exactly the same as you do with the transformation; you project v onto u-hat!

So, to generalize: Taking the dot product of vectors v and w is equivalent to transforming the vector v by the matrix [wx wy]. But also equivalent to transforming the vector w by [vx vy].

[animatedsilicon]

To those who don't quite get the bit about the duality and how dot product could occur from that, after hours of thinking, I think I might have figured out a more direct explanation that could help you.

7:48 || 3b1b takes a copy of the number line and pastes it on the two-dimensional matrix grid, such that it is slanted and the "0" is at the origin. It is important to note that U-hat (the unit vector of the new number line) has the same length as I-hat and J-hat. We will see why in a moment.

7:50 || 3b1b explains why the new number line is a legal move, and that a function that converts 2-D vectors into a number on the new number line exists.

8:53 || 3b1b tells us that it is important to know where I-hat and J-hat land in the new number line. He explained why earlier, from 5:21 to 6:18. To add on to what 3b1b said, knowing where I-hat and J-hat lands in the new number line allows us to define ALL vectors that exist in the 2-D space just as they are in the form a * I-hat + b * J-hat. Suppose L(V) is the function to transform a given vector V onto the new number line, where V is a 2-D vector of the form a * I-hat + J-hat. Let L(I-hat) and L(J-hat) be where the I-hat and J-hat lands on the new number line respectively. Thus, L(V) = a * L(I-hat) + b * L(J-hat). a and b are given from vector V. If we can find what L(I-hat) and L(J-hat) are, then we know L(V) is possible.

8:59 || 3b1b explains to us that an arbitrary U-hat (with length equal to the lengths of I-hat and J-hat) with an x-component of Ux and y-component of Uy, has L(I-hat) and L(J-hat) as its x-component and y-component respectively. This proof can only work if U-hat has the same length as the lengths of I-hat and J-hat. Suppose otherwise, then the line of symmetry would no longer exist and the proof no longer exists.

10:07 || 3b1b shows us where the dot product comes in.

At this point, 3b1b has shown us the dot product between U-hat and another vector, say, V, is really just V projected onto the number line that U-hat sits on. We are meant to prove for two vectors, say V and W, and not one of them with an arbitrary unit vector. At 10:33, 3b1b tackles this by considering projections onto non-unit vectors, i.e, projecting V on W or vice versa.

He said that we should scale up the U-hat by a factor. In his example, he used 3. This took me a while to understand, but I think I finally got it. I don't think he explicitly stated this, but the new number line with U-hat, should lie on one of the vector you are trying to project on. Let's say that we want to project vector W on vector V, then the new number line should be on vector V. We need to use a unit vector, in this case U-hat, for this to work. This was explained in 8:59, I have written about it above as well.

Now, we have a projection of vector W on unit vector U-hat. However, we want the projection of vector W on vector V, and not its unit vector. This is where the scaling is necessary. Since U-hat is a unit vector of the number line that vector V sits on, that would mean that vector V is a multiple of U-hat. We know this is true because they are perfectly coincident, i.e, they share the same gradient and points. Now that we know that vector V is a multiple of U-hat, how much do we need to scale U-hat by to get vector V? The answer is just the length of vector V divided by the length of U-hat. It is like when a * b = c, and you know a and c, and b is the scaling factor, then you can divide a from both sides to get b = c / a. Recall that the length of U-hat is the same as the length of I-hat and J-hat. I-hat and J-hat both have a length of one, therefore, U-hat also has a length of one. Thus, we need to scale U-hat by the length of vector V divided by one, i.e length of vector V.

For the dot product, I am going to be using the asterisk (*). So A * B is the dot product between A and B.

Currently, we have (U-hat) * (vector W), or [Ux Uy] [A B] (A and B are meant to be on top of each other, A and B are arbitrary letters that I have picked to represent the x and y components of vector W respectively). Scaling up U-hat by length of vector V (denoted by |V|), |V| x (U-hat) * vector W. This becomes |V|(Ux x A + Uy x B) = |V| x Ux x A + |V| x Uy x B. Let C and D be the x and y components of vector V respectively. C = |V| x Ux and D = |V| x Uy. Finally, we have AC + BD, which is the outcome of vector V * vector W.

The new number line can be placed on vector V instead, and by the same arguments, we should have the same results.

To summarise, (1) get U-hat and place it on one of the vectors, say V. (2) Project other vector onto U-hat, so that it is projected onto the new number line. (3) Scale U-hat up to get the original vector V. If it helps anyone, I could have this down on paper and link it here.

[JesusPerez-rpit]

Sincerely, this series of videos has taught me more than the whole Linear Algebra subject I took many years ago when I was studying Computer Science. It addresses precisely the most important problem I found back them and that is the lack of geometrical interpretation. Now I'm a Postdoc in Computer Graphics and still find myself finally understanding some details here and there. Thank you very much. I would gladly pay tons for something like this applied to more advanced concepts.

[mancheaseskrelpher]

Holy moly, this was deep. I feel as if I don't have to memorize anything for my upcoming linear algebra test -- it's all intuition! Thanks so much!

[AryanSajith]

It took me a while to understand the point he was making(with regards to how projection occurs), but it makes sense in the context of previous chapters:

_(3-D Example below, but it generalizes pretty intuitively imo)_

1) *Linear Transformations and Matrix Multiplication→* Multiplying two matrices can be conceptualized as applying one linear transformation after another. I will make sense of the dot product in this context.

2) *Column Space→* The non-square vector on the right side of the dot product has column space one(rank = 1); thus, applying it as a linear transformation will shrink your original vector down into 1-dimensional space. → This is what Grant was getting at using û as the basis vector for 1-dimensional space.

3) *Basis→* The non-square matrix on the left can be conceptualized as the linearly-transformed î-intermediate, ĵ-intermediate, and k̂-intermediate coordinates(where the basis vectors land during the second linear transformation). The non-square matrix on the right can be conceptualized as just 1 basis vector(û) with 3 coordinates of landing.

4) *Dot Product→* Conceptualized as the projection of a 3-d vector onto 1-d space(the right matrix) where the product of transformed basis vectors(left matrix * right matrix) determines the extent to which each dimension contributes.

*What this means→ *
1) The dot product represents the extent to which 2-vectors combine in each dimension, represented by a singular value.
2) Column vectors can be conceptualized as a linear transformation from n-dimensional space onto 1-dimensional space.

[poipoi]

If anyone's having trouble understanding, this is how I've come to think of it after watching the video a few times:
1. To turn 1 unit (u) of the number line into a vector, you'll have to project it from the number line onto the xy-plane. That projection will be marked by u's coordinates (ux, uy), as shown at 9:30
2. To go back to the number line from the xy-plane, move ux units along the x-axis, and then move uy units parallel to the y-axis, and you'll get back to u on the number line.
3. To turn any vector on the xy-plane into a number on the number line, you'll also have to mark its location in the xy-plane by taking note of its coordinates, (x, y). Then if you scale those coordinates by u's coordinates, you'll be able to get to the number line by walking x*ux units along the x-axis, then y*uy units parallel to the y-axis. And x*ux + y*uy is basically just the dot product of [x, y] and u.

[TheAwakeningMission]

Man, even with your fancy well-animated and explained examples, I was having a very hard time to understand this concept and took some days when I finally got it. No, it's not your fault. I think it's simply because english is not my main language so my learning curve rely more on visual examples than anything (my english is not poor, but it's just not fluent enough to understand everything at the same pace of natural english speakers).
However, I took this difficulty as good thing, as I REALLY learned all the subject of your linear algebra series by wathing every video multiple times.
I just want to thank you for your effort to bring us this different method of teaching math which is actually much more stimulant than the old fashion way most of us are used to (copy and paste formulas). And your passion about how cool is this subject gives me even more inspiration to learn. Thanks from a brazilian guy.