Numerical Optimization Algorithms: Gradient Descent

In this video we discuss a general framework for numerical optimization algorithms. We will see that this involves choosing a direction and step size at each step of the algorithm. In this vide, we investigate how to choose a direction using the gradient descent method. Future videos discuss how to

Topics and timestamps:
0:00 – Introduction
2:30 – General framework for numerical optimization algorithms
18:41 – Gradient descent method
32:05 – Practical issues with gradient descent
36:53 – Summary

#Optimization

[ryanmeinhardt]

AE501 : Good introduction to the algorithms and the descent method, makes the process for calculation very clear and straightforward.

[GiovanniRiggi-jq]

AE 501 - Johnny Riggi: Good to see that we are still using the gradient function to break down complex problems. Proving that all of math builds on itself!

[ahmedashmaig]

AE501: As always, you were able to explain a difficult concept such as the gradient descent method in a very clear way. Thank you!

[aimeepak]

AE501: Your explanation of the numerical optimization process made it sound so simple! The gradient descent method makes a lot of intuitive sense in finding a local minima, especially with the visualization of descending a mountain in the direction of the local steepest downward. I can also see why selecting the appropriate step size is important for finding the optimization solution. Thanks, Professor!

[Chuan-YuTsai]

AE501: Good introduction of the gradient descent method.

[joshshort]

AE 501: Super helpful examples and very useful concept

[elijahleonen]

AE501: The graphical representation with numbers was a very great way of understanding gradient descent! Wonderful lecture!

[chayweaver.]

AE512: Very powerful use of an extremely simple concept - very cool

[Gholdoian]

AE 501: Wow very interesting how a simple concept such as the gradient can be used for optimization. Definitely opened my eyes to the various and complex optimization techniques that exist. Thank you!

[maggiesplantgirl]

AE501: While I had thought about inefficiency in looking for the min I hadn't even thought about the possibility of jumping over the local min completely, good thing to keep in mind for sure. -Maggie Shelton

[philipcasey]

AE501 - This playlist is really thorough and helpful; will be referring back to this even after the course has ended. Thank you!

[lukewideman]

AE501 You always do a great job in explaining these topics. Optimization is so cool.

[KennethWright-kh]

AE501: Thank you for introduction to the gradient descent method.

[edwardmau]

AE 501
Great video, broke the concept down and made it easy to understand.

[JulieWarchol]

AE 501: the hiking topography type map was a very helpful example to understand this concept!

[JustinYoung-jf]

AE501: Thank you for this great lecture, really helps understand the overall ideas!

[inderbhangal]

AE 501, this is quite neat and seems useful for many situations

[nghihoang]

Thank you very much for that big picture at 14:00, I would be lost without that
AE501

[wiloberlies]

Really great lecture Dr Lum, this topic has had some personal relevance for me lately, so as always, thank you!! Hope you and yours are well.

[ethanngo]

AE501: Your pseudo code for numerical optimization algorithms makes a great "cheatsheet". Thank you!