Calculus - Math for Machine Learning

In this video, W&B's Deep Learning Educator Charles Frye covers the core ideas from calculus that you need in order to do machine learning.

In particular, we'll see a different way of thinking about calculus -- based on linear approximations -- that makes thinking about vector- and matrix-valued derivatives easier. Then, we'll talk about the gradient descent algorithm, which is ubiquitous in machine learning, and how it arises naturally from thinking this way about calculus, and briefly touch on how calculus gets automated away.

0:00 Introduction and overview
2:01 Vector calculus involves approximation with linear maps
3:48 The Fréchet derivative definition for single-variable calculus
12:50 Little-o notation makes calculus easier
16:50 The Fréchet derivative makes vector calculus easier
25:43 Gradient descent: tiny changes using calculus
34:38 Automating calculus
40:09 Additional resources

[Gowthamsrinivasan]

This makes so much more sense. For ex. For x^2, derivative is 2x and even if u understand the limit definition, why it is 2x doesnt make intuitive sense but now, it is essentially that the approximation to nearby point on the curve x^2 is based on thr straight line 2x.

[teogiannilias]

Give this man a nobel. Half of the master's students in AI struggle in understanding when seeing too many indices. It's better to demonstrate indices right on the code rather than in notations

[Ian]

Beautiful application of semi-abstract math to a whole class of problems. I've never seen such an elegant presentation of gradient descent - usually it gets lost in the clutter of "multivariable calculus". The whole section on the Frechet derivative was also excellent. Great long-form style that's getting harder to find on YT these days.

[tomthanhswe]

Thank you for the explanation. The graph where you show how to approximate scalar changes with calculus and little-o really helps.

[vinciardovangoughci]

I've been waiting for this material since I met you Charles!!! So excited.

[quantummath]

Loved it dude. From a fellow researcher in ML and Econ.

[venkatthimmappa]

Excellent, this is what I wanted for a long time.

[angelanfish]

Just found your channel and thank you so much for contents, they are super helpful!

[empowercode]

Hey, nice to meet you! I just found your channel and subscribed, love what you're doing!

I like how clear and detailed your explanations are as well as the depth of knowledge you have surrounding the topic! Since I run a tech education channel as well, I love to see fellow Content Creators sharing, educating, and inspiring a large global audience. I wish you the best of luck on your YouTube Journey, can't wait to see you succeed! Your content really stands out and you've put so much thought into your videos!

Cheers, happy holidays, and keep up the great work ;)

[tam]

I'm going to repeat myself, this is extremely cool!

[ShaunJW]

How do you use calculus for ML? I mean are we talking derivatives or the entire calculus including integrals etc

[Gowthamsrinivasan]

What the actual f, this derivative definition needs to be as standard as the limit ones.

[mathpotty]

hey, @charles_irl it would be great if you make a course like this but only for getting programmers ready for mathematics.
call that Into Mathematics for Programmers????