Gauss Newton - Non Linear Least Squares

Become a member!
* 🎉 Special YouTube 60% Discount on Yearly Plan – valid for the 1st 100 subscribers; Voucher code: First100 🎉 *

Why become a member?
* All video content
* Extra material on complete-courses (notebooks)
* Access to code and notes
* Community Discussion
* No Ads
* Support the Creator ❤️

~~~~~ SUPPORT ~~~~~
~~~~~~~~~~~~~~~~~

If our function is non linear (w.r.t. the unknown coefficients) we can use non linear least squares to find them - but there will be no closed form solution. Numerical methods like the Newton-Raphson method could be used.

There are 2 ways to look at the Gauss-Newton method: either as Newton-Raphson method which uses an approximation of the Hessian (of the loss function), or as an iterative ordinary least squares which uses a 1st order (linear) Taylor approximation (of the non linear function itself).

[skymanaditya]

For someone who is wondering why the function stays linear even when we squared terms of observations is because that the observations can always be modified. E.g., an observation X*X can always be replaced X since it's just the observation but the same does not hold true for the parameters of the model. That's why if the function is linear w.r.t. to the betas, the least-squares optimization would be linear.

[mohamedgaal]

Thank you so much for this incredible tutorial!

[radimnovotny]

Thanks so much for this great video. It helped a lot

[elliekong]

Thank you so much for your clear explantion. It was incredible!

[galileo]

Great explanation! I got a quesion:
At 1:27 you mention that the function stays linear, even when adding the x-squared term, and that the linearity depends on the beta´s, not the x´s. This is mind blowing. I always thought the linearity depends on the x´s. Could you shortly elaborate why that is?

[raminbohlouli]

Awesome lecture. Thank you very much. One question tho. Why at 5:47, the final result has become the transpose of the Jacobian and not the Jacobian itself?

[noeliapalacios]

The code is not available, could anyone say me where find it, please?

[sircaioamaral]

What is Bk you use during the second derivative?

[rogerpoulopez]

Could you recommend me some books where I could learn from this?

[geophysicsadvancedseismice]

Dear can you share your email? I want to talk to you regarding the implementation of this method for another particular case.