Backpropagation explained | Part 5 - What puts the 'back' in backprop?

Let's see the math that explains how backpropagation works backwards through a neural network.

We've seen how to calculate the gradient of the loss function using backpropagation in the previous video. We haven't yet seen though where the backwards movement comes into play that we talked about when we discussed the intuition for backprop.

So now, we're going to build on the knowledge that we've already developed to understand what exactly puts the back in backpropagation. The explanation we'll give for this will be math-based, so we're first going to start out by exploring the motivation needed for us to understand the calculations we'll be working through.

We'll then jump right into the calculations, which, we'll see, are actually quite similar to ones we've worked through in the previous video.

After we've got the math down, we'll then bring everything together to achieve the mind-blowing realization for how these calculations are mathematically done in a backwards fashion.

🕒🦎 VIDEO SECTIONS 🦎🕒

00:43 Agenda
01:13 Calculations - Derivative of the loss with respect to activation outputs
13:06 Summary
13:40 Collective Intelligence and the DEEPLIZARD HIVEMIND

💥🦎 DEEPLIZARD COMMUNITY RESOURCES 🦎💥

👋 Hey, we're Chris and Mandy, the creators of deeplizard!

👉 Check out the website for more learning material:

💻 ENROLL TO GET DOWNLOAD ACCESS TO CODE FILES

🧠 Support collective intelligence, join the deeplizard hivemind:

🧠 Use code DEEPLIZARD at checkout to receive 15% off your first Neurohacker order
👉 Use your receipt from Neurohacker to get a discount on deeplizard courses

👀 CHECK OUT OUR VLOG:

❤️🦎 Special thanks to the following polymaths of the deeplizard hivemind:
Tammy
Mano Prime
Ling Li

🚀 Boost collective intelligence by sharing this video on social media!

👀 Follow deeplizard:

🎓 Deep Learning with deeplizard:

🎓 Other Courses:

🛒 Check out products deeplizard recommends on Amazon:

🎵 deeplizard uses music by Kevin MacLeod

❤️ Please use the knowledge gained from deeplizard content for good, not evil.

[deeplizard]

Backpropagation explained | Part 1 - The intuition

Backpropagation explained | Part 2 - The mathematical notation

Backpropagation explained | Part 3 - Mathematical observations

Backpropagation explained | Part 4 - Calculating the gradient

Backpropagation explained | Part 5 - What puts the “back” in backprop?

[envy]

I swear to God this level of explanation surpass ANY university teaching standard. Like after 6 years, new people like me still amaze with the level of details and breakdowns through out all this heavy based maths.

[rohitjagannath]

My God!!! Hey dear tutor... U are so very amazing with your explanation. I went through all your backprop math and was really amazed by your presentation. U broke down the complexity into such little chunks explained it so very well. The final touch with that image of a man with a blast about the concept of backpropogation was amazing!!! Thank you very much. I need to practice this a lot many times to get a grip over it

[AmrishKelkar]

This 5-video series is just mindblowing. As a teacher, I can imagine it being painful having to repeat mathematical observations repeatedly, especially when there are like a million sub and superscripts involved. But every time, you state clearly what each term means. So kudos on that dedication. You've clearly put in the effort to write down the script and then present. Thank you for the series.

[arslanzahid]

You know when a song is so good, and you dont even realize it that it's long cuz the song takes a grip of your attention. Well watching these videos is like that. Bravo. what an amazing explanation done here !!!

[mariodurndorfer]

Watched the series twice. I think i got it now. As already mentioned, i like your style. Clear steps and you take the right amount of time for each one. Great work! Thanks!

[paulinevandevorst]

THANK YOU! I'm writing a bachelor's thesis on deep learning for noise reduction in MRI images. This has helped me very much in understanding back propagation. The math I found in papers, seemed so difficult, that it was difficult to keep motivated. However, through this series of yours, I have discovered that the math really isn't that difficult, and that it is also really intuitive, once you grasp the notation. Good work! :)

[marcfelsenstein]

Hi there,
I've recently started working with NN at work and the 'Deep Learning Fundamentals' playlist is helping me get up to speed so much! Thanks for that.
Now, when speaking about the 5 Backpropagation videos - my derivatives are rusty.
But even that being the case, you managed to explain the concepts on a higher level while also getting down to the needy greedy calculations, so I was able to follow even if I didn't understand absolutely every step of the way.
Brilliantly explained.
Thanks again!

[xariskaragiannis]

Best explanation for backpropagation out there! All i have to say is THANK YOU <3

[kushagrachaturvedy]

These 5 videos on backpropogation are some of the best on YouTube.

[davewesj]

Of the many videos on the subject and web-pages, you are the only one to elucidate the Backprop error calculation
for precursor layers.
Thanks for your effort.

[nurali]

Just in short - brilliant! So much work to make such series of videos, thank you very much!

[yoyomemory]

BEst explanation of BP ever!!!! Thank you much!

[SebastianHudert]

Phew... really anything was already said in the other comments below but I just really wanted to also leave one: these videos are spot on the best educational content I have seen to this day. Period. And I have been working at a university myself and taught students ;-) Never seen anybody navigating mathematical equations in such a clear and understandable way.

[Hatesun-lzfi]

Outstanding, clear and concise explanation. The idea of splitting this topic in 5 videos was very helpful to me. Thank you!

[ismailelabbassi]

The greatest teacher of all time thank u so mu u're great keep going.

[wgc]

This was great. Part 1 set it up nicely. Part 2 you made it so clear what each ingredient was. That was dry but completely essential as you drew us through all the required calculations. You were able to patiently step through them for us as straightforwardly as they possibly could be. My hat is off to you and I look forward to viewing other of your videos to learn more.

[DEEPAKSV]

Got the exact same feeling at 12:56!
No words, to thank you for the amount of social service you are doing to the tech community.
I feel your work would have a major contribution to the AI development across the world, since a lot of young beginners like me are motivated into this field even more by your style of teaching.
The amount of effort you put up in keeping your content short, precise and yet interesting seems incredible. . .

[bishwasapkota]

This is by far the best one for Backprop!!!! Congrats and thanks!

[abhirajsingh]

By far the best explanation about this concept on internet. Thank you ! :)