The deeper meaning of matrix transpose

Transpose isn’t just swapping rows and columns - it’s more about changing perspective to get the same measurements. By understanding the general idea of transpose of a linear map, we can use it to visualise transpose much more directly. We will also heavily rely on the concept of covectors, and touch lightly on metric tensors in special/general relativity, and adjoints in quantum mechanics.

As far as I know, this way of visualisation of transpose is original. Most people use SVD (singular value decomposition) for such visualisation, but I think it is much less direct than this one, and also SVD is mostly used for numerical methods, so it feels somewhat unnatural to use a numerical method to explain linear transformations (though, of course, SVD is extremely useful). Please let me know if you know that other people have this specific visualisation.

The concept I am introducing here is usually called a “pullback” (and actually the original linear transformation would be called “pushforward”), but as said in the video, another way of thinking about transpose is the notion of “adjoint”.

Notes:
(1) I am calling covectors a “measuring device”, not only because the level set representation of covectors looks like a ruler when you take a strip of the plane, but also because of its connections with quantum mechanics. A “bra” in quantum mechanics is a covector, and can be thought of as a “measurement”, in the sense of “how likely will you measure that state” (sort of).

(2) I deliberately don’t use row vectors to describe covectors, not only because this only works in finite-dimensional spaces, but also because it is awkward for the ordering when we say a transpose matrix *acts* on the covector. We usually apply transformations on the *left*, but if you treat the covector as a row vector, you have to act the transpose matrix on the *right*.

(3) You can do the sort of “exercise” to verify this visualisation of transpose for all (non-singular) matrices, but I think the algebra is slightly too tedious. This is the reason why I spent a lot of time talking about the big picture of transpose - to make the explanation as natural as possible.

Further reading:

**GENERAL**

(a) Transpose of a linear map (Wikipedia)

(b) Vector space not isomorphic to its dual (for infinite-dimensional vector spaces):

**RELATIVITY**

**ADJOINT**

(d) Self-adjoint operators (Hermitian operators in QM, but also useful in Sturm-Liouville theory in ODEs):

Video chapters:
00:00 Introduction
00:56 Chapter 1: The big picture
04:29 Chapter 2: Visualizing covectors
09:32 Chapter 3: Visualizing transpose
16:18 Two other examples of transpose
19:51 Chapter 4: Subtleties (special relativity?)

Other than commenting on the video, you are very welcome to fill in a Google form linked below, which helps me make better videos by catering for your math levels:

If you want to know more interesting Mathematics, stay tuned for the next video!

SUBSCRIBE and see you in the next video!

If you are wondering how I made all these videos, even though it is stylistically similar to 3Blue1Brown, I don't use his animation engine Manim, but I will probably reveal how I did it in a potential subscriber milestone, so do subscribe!

Social media:

For my contact email, check my About page on a PC.

See you next time!

[mathemaniac]

Please consider sharing, liking, commenting on the video - this is probably one of my favorite videos on my channel.

By the way, if you are wondering why (A^T)^T = A, this will require the double duals, or co-covectors and the "canonical isomorphism". Essentially, the co-covectors are machines that measures covectors, and they can be thought of as vectors themselves. The explanation is not that visual, and so I am probably not making videos on the double duals.

[rgbill]

3b1b linear algebra series saved my life during college, but oh boy do i wish this have seen this video sooner

[samkirkiles]

This makes it a little more complicated then it actually is. Here it is simply

- Every vector space V has a dual space V*
- Elements of the dual space are called dual vectors or co-vectors. When you think of dual space, just think of this as the space of row vectors, while the original space is the space of column vectors.
- Every linear map from V -> W induces a linear map between the dual spaces W* -> V* called the transpose.
- Every finite dimensional vector space is isomorphic to its dual, so we can really view the transpose as going from W -> V

One more point:

Row vectors eat column vectors and return a scalar. In R^2, you can visualize row vectors as equally spaced stacks of planes with one through the origin, and the application of the row vector to the column vector is the number of planes that the column vector pierces. This is the intuition the video is trying to get at.

[orik]

This video is EXACTLY what I've been looking for! Please don't stop making these!

[inciaradible]

This was actually very refreshing and a great way to visualise what a transpose really is, even if there are limitations.

[dcterr]

I've always had a hard time visualizing matrices as vector transformations, but you do a very good job with that here. You also do a terrific job of explaining the transpose as an inverse vector transformation rather than as a new matrix. Great video!

[nangld]

In Haskell matrix transpose is called `zip`, because of how it unzips a list of M N-tuples into a N-tuple of N lists of scalars. Very useful in general programming, even if you never think about it being linear algebra.

[cobrametaliks]

Could you please make a video about tensors? I don't like the usual introduction to it, and I feel that you could give a more intuitive and tangible approach.
Thank you for your work, it is amazing. The 13:30 got me real good 😄

[SLopez]

I commend you for a simple and overlooked process and revealing the hidden intricacies as to why it happens to look like ‘just switching rows and columns’. There’s alway a reason we do something in math and when we are first introduced to new concepts it are often give the how before we are given the why.
Keep up the good work and please continue on this trend of explaining the hidden reasons why we some procedures that are surely taken for granted

[julianaharmatiuk]

it's interesting how I'm currently studying 4 topics and this video managed to get into all of them! Badass

[jacb]

Keep up the great work Mathemaniac as an early undergrad your content is at a perfect level and it’s so interesting different interpretations of linear algebra from those I’ve already seen.

[decreasing_entropy]

This was just a bit difficult to grasp at first, but then I got the feel for it. Linear Algebra never stops asking more and more questions, and no wonder why I like it so much. A video on metrics would really help, considering the fact that I have to tame Special Relativity next semester.

[hambonesmithsonian]

Oh boy you have no idea how much I’ve needed a video about this exact series of topics. Combining this with differential forms + knot theory and we about to *transcend.*

[koun.informatique]

I have been searching for an explanation about transposé for à decade, thank you so much 😁

[zhuolovesmath]

I’m glad that you didn’t get frustrated by the low view of videos! Keep going man I love your math videos

[cmilkau]

There is a difference between transpose and adjoint that should not be swept under the rug. It all boils down to the inner product. If the inner product is just multiply and sum, then yes, transpose is adjoint. If it's more complicated though, the adjoint follows suit.

[TheBrainn]

to me linearity and its duality between vector spaces is the most mesmerizing thing in all of math- especially because it is the motivation behind so many more complex entities like differential equations and laplace transforms which rely on those principles for their unshakeable power and accuracy for everything from interpolation to analysis

[pigeonapology]

My heart still lies with the view of the transpose that comes out of categorical quantum mechanics (somewhat akin to this one, given the emphasis on the notion of "measurement"), but this one is very nice (and nearly as intuitive) as well. Great video!

[methandtopology]

*NATURAL EXPLANATION OF COVECTORS AND TRANSPOSITION FOLLOWS*

The title caught my curiosity but, even though I tried to follow the arguments, they felt very prolonged and awkward, . I was ultimately left confused. I believe this is the second video I watch from this channel. The other one, on Galois theory, was a good outline but I found this to obfuscate a very simple concept.


*Less technical paragraph after this*
If I'm not mistaken, that concept would be the contravariant functor that maps vector spaces to their dual spaces and linear maps to their dual maps. If we denote by A the linear map V -> W, then the dual map A*: W* -> V* arises by taking covector a in W* (the dual of W) to covector b=W*a in V* with the definition
b(v)=a(Av)
for vector v in V. Put simply, you tell the image in V* of a covector a in W* to do on each vector v in V what a does on the image Av in W.
This is a very natural construction and by representing covectors as column vectors then writing out the equation for each component of b, it can be quickly derived that A*=A^T. But this is also the most confusing thing I found in this video: that both vectors and covectors are represented as column vectors. But apples are apples and bananas are bananas.

If we just represent covectors as row vectors then covector (a b) acting on vector (x y)^T is just the usual matrix multiplication (a b)(x y)^T=ax+by. Then transformations are represented from right multiplication, that is, bA*(which is actually =bA as shown later). This is clear because if A is an m x n matrix that maps n dim vectors, the m dimensional covectors should be transformed to n dimensional, and the only way to do that with an m x n matrix is by multiplying from the right.

Now look how simply transposition appears (using the notation from above):
bA=(A^Tb^T)^T.
That simply says if we have row vector representation of covectors and matrix multiplication from the right, if we insist on writing covectors as column vectors (like in this video) then matrix multiplication from the left of the transpose gives us the mapped covector in column vector form, and transposing that back to row vector gives the covector in its natural representation. So it is derivable from the definition of matrix multiplication and transposition.

[readingsteiner]

In my opinion, the explanations are a bit ambiguous, at least to me. I am not a native English speaker, I don't know how other people feel, but I think what confuses me the most is how you chose very long, compound sentences, which made it hard for me to follow. I also feel like I was not explained the terms like "co-vector", "measurement", ... and sometimes I have no idea what is being talking about. I really appreciate the hard work you put into this video though.