Linear System Solve - Pushforward/Jvp rule

Linear Solvers are essential to scientific computing. If they are part of a computational graph for which we want to use forward-mode Automatic Differentiation, we need to be able to propagate tangents. But how? Here are the notes:

-------

-------

Timestamps:
00:00 Linear System Solving as a function
00:36 Linear Systems are fundamental
01:25 Example algorithms
02:38 Dimensionalities involved
03:05 Task: Forward propagate tangent information
06:11 General Pushforward/Jacobian-vector product rule
07:02 Primal/Forward constraint in index notation
07:37 Pushfoward/Jvp in index notation
09:10 (1) Right-hand side contribution
12:54 (2) System Matrix contribution
16:51 Moving back to symbolic notation
17:46 Fusing two linear solves into one
19:26 Spelling out the axillary (tangent linear) linear system solve
19:51 Full Pushforward rule
21:11 Sharing workload with the primal pass
23:08 Summary
23:25 Outro