Newton's method | Backtracking Armijo Search | Theory and Python Code | Optimization Algorithms #2

Показать описание

In this one, I will show you what the (damped) newton algorithm is and how to use it with Armijo backtracking line search. We will approach both methods from intuitive and animated perspectives. Next, we talk about the line search we are going to use in this tutorial, which is the Armijo backtracking method. This is achieved by the Armijo condition, which sufficiently decreases our function ! Of course, looking at the Armijo condition equation as is might not reveal any insights, but geometrically looks beautiful, let me show you how.

⏲Outline⏲

00:00 Introduction
01:00 (Damped) Newton Method
03:30 Backtracking Armijo Line Search
06:27 Python Implementation
16:46 Animation Module
32:11 Animating Iterations
34:34 Outro

📚Related Courses:

🔴 Subscribe for more videos on CUDA programming
👍 Smash that like button, in case you find this tutorial useful.
👁‍🗨 Speak up and comment, I am all ears.

💰 If you are able to, donate to help the channel
BTC wallet - 3KnwXkMZB4v5iMWjhf1c9B9LMTKeUQ5viP
ETH wallet - 0x44F561fE3830321833dFC93FC1B29916005bC23f
DOGE wallet - DEvDM7Pgxg6PaStTtueuzNSfpw556vXSEW
API3 wallet - 0xe447602C3073b77550C65D2372386809ff19515b
DOT wallet - 15tz1fgucf8t1hAdKpUEVy8oSR8QorAkTkDhojhACD3A4ECr
ARPA wallet - 0xf54bEe325b3653Bd5931cEc13b23D58d1dee8Dfd
QNT wallet - 0xDbfe00E5cddb72158069DFaDE8Efe2A4d737BBAC
AAVE wallet - 0xD9Db74ac7feFA7c83479E585d999E356487667c1
AGLD wallet - 0xF203e39cB3EadDfaF3d11fba6dD8597B4B3972Be
AERGO wallet - 0xd847D9a2EE4a25Ff7836eDCd77E5005cc2E76060
AST wallet - 0x296321FB0FE1A4dE9F33c5e4734a13fe437E55Cd
DASH wallet - XtzYFYDPCNfGzJ1z3kG3eudCwdP9fj3fyE

#python #optimization #algorithm

Рекомендации по теме

Комментарии

You explain things so slow and step-by-step as it was being explained for chimps. Exactly what i needed. Thank you a lot.

eyupbayhan

Thanks so much for your feedback, really appreciated!

efe

God bless you boss. I have really struggled today for close to 24 hours in search for g2bbage online till I got your video.

turkintroadam

Many thanks for your feedback, appreciated.

anambabam

Whenever I get hung up about some hard to understand topic, I remember Ahmad Bazzi waiting you in Ahmad Bazzi; so be relax and enjoy learning something new with him; without any doubt.

maryagyemangkwartemaa

After all these years I finally understand the magic behind gradients! Thanks!

bilmiyom

Best Video of gradient descent I've ever found in the universe!! Thanks for saving my life

berksen

Amazing explanation! Greetings from Brazil!

fm

Very well explained and easy to follow. Thank you very much Sir.

muhammed-pjgi

I swear this is the most clear and FANTASTIC explanation I've ever found

hamzaandmaliksworld

One of the best lectures here on YouTube !

مسمكةالنور

Great video, especially when you coded it on Python.

jyotivishwakarma

Superb explanation sir. Love from India.

dizifragmantv

Sir, your lectures are awesome! Thank you very much.

yigiteslem

God level Explanation... 😍😍😍😍😍😍😍😍😍😍😍😍😍😍😍😍

johnnyzMusiq

Teaching is an Art and you explain this dry and difficult topic in a very comprehensive way! thanks for sharing your knowledge and skill.

POU-pubg

Very good video. I appreciate your hard work. Keep uploading more videos.

olamanrse

Simple and clear ... Yet need more detailing ...!!!!

calinity

each and every notation is explained serially, step by step with its meaning and relation to the problem

brawlradio

You are the best teacher ever Ahmad Bazzi... I'm going Andrew Ng ML course and dint understand what the hell was Gradient Descent.. so came to YouTube and found your video...BAM..Thank you so much for doing these videos.... Keep em coming...💜

cintyakireinaf

Newton's method | Backtracking Armijo Search | Theory and Python Code | Optimization Algorithms #2

Modified Newton method | Backtracking Armijo | Theory and Python Code | Optimization Techniques #5

Newton's method | Backtracking Armijo Search | Theory and Python Code | Optimization Algorithms...

Gradient descent, Newton's method

Numerical Optimization Algorithms: Step Size Via the Armijo Rule

Descent methods with line search: Newton method with line search

Modified Newton method | Wolfe Backtracking | Theory and Python Code | Optimization Algorithms #6

Newton's method | Wolfe Condition | Theory and Python Code | Optimization Algorithms #3

Newton method for optimization/ operation research/ how to minimize function from newton method

Global Newton's Method - It Always Converges

Lecture 5: Gradient Descent, Newton's Method, and Scaling Techniques for Signal Processing

NonlinearData04dBacktracking

Gradient-related sequences, Armijo points, and accelerated line search

Newton method for multidimensional minimization. Part 1

Kalkulus 4.4 - Newtons metode

Newton's method | Exact Line Search | Theory and Python Code | Optimization Algorithms #1

10-701 Machine Learning Fall 2014 - Lecture 11

OIP 2.2.2 Armijo-Schrittweitenregel

Descent methods and line search: validity of the Wolfe conditions

Unconstrained optimization: conjugate gradient, Gauss-Newton, Levenberg-Marquardt

Descent methods and line search: first Wolfe condition

Descent methods and line search: inexact line search

Optimization Techniques -W23- Lecture 9 (Conjugate Gradient, Quasi-Newton, Distributed Optimization)

Modified Newton method | Exact Line Search | Theory and Python Code | Optimization Algorithms #4

Unconstrained Optimization #3A: Descent Search Method - Steepest Cauchy with Armijo Line Search