Simple reverse-mode Autodiff in Python

preview_player
Показать описание

-----

-------

-------

Timestamps:
00:00 Intro
00:18 Our simple (unary) function
00:35 Closed-Form symbolic derivative
01:16 Validate derivative by finite differences
01:55 What is automatic differentiation?
02:53 Backprop rule for sine function
04:27 Backprop rule for exponential function
06:06 Rule library as a dictionary
07:14 The heart: forward and backward pass
11:17 Trying the rough autodiff interface
13:15 Syntactic sugar to get a high-level interface
14:10 Compare autodiff with symbolic differentiation
14:59 Outro
  1. Machine Learning & Simulation
  2. griewank
  3. pytorch
  4. torch
  5. tf
  6. jax
Рекомендации по теме
Simple reverse-mode Autodiff 0:15:52

Simple reverse-mode Autodiff in Python

Simple reverse mode 0:07:07

Simple reverse mode autodiff in python

What is Automatic 0:14:25

What is Automatic Differentiation?

Simple reverse-mode Autodiff 0:12:51

Simple reverse-mode Autodiff in Julia - Computational Chain

Comparing Automatic Differentiation 0:00:38

Comparing Automatic Differentiation in JAX, TensorFlow and PyTorch #shorts

Intuition behind reverse 0:13:17

Intuition behind reverse mode algorithmic differentiation (AD)

4 Reverse Mode 0:05:52

4 Reverse Mode Automatic Differentiation

Finding The Slope 0:15:58

Finding The Slope Algorithm (Forward Mode Automatic Differentiation) - Computerphile

NN - 12 0:12:36

NN - 12 - Backprop vs. Forward-prop (aka Reverse vs. Forward mode)

PyTorch Tutorial : 0:04:29

PyTorch Tutorial : Backpropagation by auto-differentiation

Dual Numbers and 0:06:36

Dual Numbers and Automatic Differentiation in Python

automatic differentiation 0:04:09

automatic differentiation

Talk: Colin Carroll 0:19:18

Talk: Colin Carroll - Getting started with automatic differentiation

Tutorial on Automatic 0:06:01

Tutorial on Automatic Differentiation

Full Jacobian using 0:09:54

Full Jacobian using reverse-mode AD in JAX

NN - 11 0:19:10

NN - 11 - Automatic Differentiation

Keno Fischer: 'Optics 1:11:01

Keno Fischer: 'Optics in the wild: reverse mode automatic differentiation in Julia'

Automatic differentiation in 0:14:07

Automatic differentiation in scientific programming with jax

Automatic Differentiation 0:10:10

Automatic Differentiation

PyHEP 2020 Tutorial 0:49:24

PyHEP 2020 Tutorial on Automatic Differentiation

L6.2 Understanding Automatic 0:22:48

L6.2 Understanding Automatic Differentiation via Computation Graphs

The Simple Essence 0:21:45

The Simple Essence of Automatic Differentiation

Automatic Differentiation in 1:20:42

Automatic Differentiation in Python and PyTorch (Serverless Machine Learning)

Lecture 4 - 1:03:35

Lecture 4 - Automatic Differentiation

Комментарии
Автор

Very clear explanation! Thanks and hope you'll get more views

sfdv
Автор

Oh boy this seems a bit above my comprehension level

micah
Автор

Thank you for the clear explanation! I wonder where does the term "cotangent" come from? A google search shows it comes from differential geometry, do I need to learn differential geometry to understand it ...?

houkensjtu
Автор

can you please tell your recommend resources to learn maths? or how did you learned math?

harikrishnanb
Автор

Thanks for the great content as always. One question and one comment. How would you handle it if it is a DAG instead of a chain? Any reference (book/paper) that you can share? I noted that for symbolic differentiation, you pay the price of redundant calculation (quadratic in the length of chains) but with constant memory. On the other hand, the auto-diff caches the intermediate values and has linear calculation but also linear memory.

zhenlanwang