L6.2 Understanding Automatic Differentiation via Computation Graphs

As previously mentioned, PyTorch can compute gradients automatically for us. In order to do that, it tracks computations via a computation graph, and then when it is time to compute the gradient, it moves backward along the computation graph. Actually, computations graphs are also a helpful concept for learning how differentiation (computing partial derivatives and gradients) work, which is what we are doing in this video.

-------

This video is part of my Introduction of Deep Learning course.

-------

[manuelkarner]

thaaaank you, finaly I understand this perfectly (& can know repeat it for myself)
explaining backpropagation my lovely proffs always said "then this is just the chainrule" & skipped any explanation
for calculating (complicated) toy examples I knew the chainrule, but in the backprop context it was just to confusing
got a question: at 12:23 you said tehcnicaly canceling the delta terms is not allowed -> could you elaborate on the math/why or point me to some ressourece explaining this ?
Intuitively I always thought canceling delta´s is strange/unformal but I dont found out how this delta notation stuff fits into "normal" math notation :)

[karimelkabbaj]

Thank you very much for this simplified explanation, i've been struggling to understand it until i found this master piece.

17:27 In the formula on the top-left (as I understood) there is no sum, but stacking (or concatenating), then why should we add the results in different paths during the backward chain computation? Is it always work like this - just produce the sum in the chain when there is a concatenating????

[nak]

Love your textbooks and your videos. Thank you!

[Epistemophilos]

At 11:27 it gets confusing because you switch the terms around. Otherwise, very nice video.