Scalar Root-Finding - Pushforward/Jvp rule

In the spirit of differentiable programming, one could just call the forward-mode AD engine on the mathematical operations to the root-finding algorithm. However, this often introduces unnecessary overhead and for some algorithms it might not be possible. This could be either it contains certain non-differentiable functions or is itself a call to a foreign-language library (like a FORTRAN solver) that is interfacing with the AD framework. This video presents a classical remedy that involves an application of the implicit function theorem.

-------

-------

Timestamps:
00:00 What is scalar root finding?
00:54 Example Algorithms
01:26 Dimensionalities involved
01:54 Assumption that solver converges
02:22 Task or propagating tangent information
02:36 NOT by unrolling the iterations
03:23 Definition of the pushforward / Jvp
04:11 Implicit Function Theorem via total derivative
06:43 Assembling the tangent propagation
07:41 Final Pushforward operation
08:17 Obtain additional derivatives by forward-mode AD
08:39 Summary
09:05 Outro

[Fujitora-vedz]

Could you make a video on taking higher order derivatives