SGD with Momentum Explained in Detail with Animations | Optimizers in Deep Learning Part 2

In this video, we will understand in detail what is Momentum Optimizer in Deep Learning. Momentum Optimizer in Deep Learning is a technique that reduces the time taken to train a model. The path of learning in mini-batch gradient descent is zig-zag, and not straight. Thus, some time gets wasted in moving in a zig-zag direction. Momentum Optimizer in Deep Learning smooths out the zig-zag path and makes it much straighter, thus reducing the time taken to train the model.

Momentum Optimizer uses Exponentially Weighted Moving Average, which averages out the vertical movement and the net movement is mostly in the horizontal direction. Thus zig-zag path becomes straighter.

Convex Vs Non-convex Cost Function:

============================
Do you want to learn from me?
============================

📱 Grow with us:

👍If you find this video helpful, consider giving it a thumbs up and subscribing for more educational videos on data science!

💭Share your thoughts, experiences, or questions in the comments below. I love hearing from you!

⌚Time Stamps⌚

00:00 - Intro
01:24 - Understanding Graphs
07:13 - Image Representation
10:41 - Convex vs Non-Convex Optimization
18:13 - Momentum Optimization
20:49 - The What?
24:23 - How to implement the concept Mathematically
28:41 - Effect of Beta
34:33 - Problems with Momentum Optimization
35:54 - Visualization
38:04 - Outro

[ashishmalhotra]

your ability to breakdown every concept in its simplest form is amazing! Thank you for making these videos public.

[himesh]

Although, I'm not watching it now because I'm learning ML. But this comment is to say you thank you to keep going. Bohot views aega kuch saalo mein. Take as much time you can (not just 100 days) taaki next competitor(jo bhi baad mein hindi me deep learning video banaye) ko koi dekhe bhi na and you maintain the hegemony.

[daudiomusic]

SGD with Momentum is one of the optimizers which is used to improve the performance of the neural network.

Let's take an example and understand the intuition behind the optimizer suppose we have a ball which is sliding from the start of the slope as it goes the speed of the bowl is increased over time. If we have one point A and we want to reach point B and we don't know in which direction to move but we ask for the 4 points which have already reached point B. If all 4 points are pointing you in the same direction then the confidence of the A is more and it goes in the direction pointed very fast. This is the main concept behind the SGD with Momentum.

[ashwnicoer]

No one can explain it better than you sir ! Incredible !

[sanjaisrao]

Thank you for all your efforts sir, You are amazing, Keep continuing the teaching, Please bless me for my future. Thank you

[jagritikumari]

Before a year ago, I heard the term contour plot, but I didn't understand it. My prof has tried to make me know but I can't. My classmates were like you are wasting our time. I felt so dumb at that time. I thought it was my problem that I couldn't understand it. But you are a real guru to me. You have made it very easy for everyone. You know actually where the student gets doubt. I felt so confident after watching your videos. Thank you so much sir for uploading.

[soyam]

16:20, i literally cried after watching your effort 🥺

[laxmiagarwal]

Very nice explanation
This 100 days of deep learning videos has developed my interest in deep learning
Thank you so much sir 😊

[kumarrohit]

It seems Nature finally want me to understand ML. I found this Channel. Great work with very good voice and very good teaching technique

[abhimishra_]

Watching now and it is golden. The content can never be old.

[NabidAlam]

Best playlist ever! I have never seen anyone teaching topics like you! Thank you so much!

[ThePri]

Your focus to build intuitions within your students is what sets you apart! Thanks for the videos.

[vikeshdas]

thank you sir to continue the Deep Learning series

[purubhatnagar]

awesome man, this shows how much dedication you have given to this

[sameeryadav]

Sir, Thankyou so much please do not stop teaching us. And one request kindly make video on imbalance class .(it seems our 100 days Ml playlist need some more videos )

[ayyamguari]

Last explanation diagram was just so good.

[siddhantsingh]

so fortunate that i find your channel : )

[sameerabanu]

Amazing lecture, thank you so much, your efforts are seen sir

[vaibhav]

Nitish sir, i request you to please complete interview question series.
Companies have started visiting campus for placement.

[LMessii]

This is an incredible explanation, the best one out there. It quite baffales me how can somebody explain something in such great details and with such lucidity. I am indebted to you sir ! Brilliance