Neural Differential Equations

This won the best paper award at NeurIPS (the biggest AI conference of the year) out of over 4800 other research papers! Neural Ordinary Differential Equations is the official name of the paper and in it the authors introduce a new type of neural network. This new network doesn't have any layers! Its framed as a differential equation, which allows us to use differential equation solvers on it to approximate the underlying function of time series data. Its very cool and will ultimately allow us to learn from irregular time series datasets more efficiently, which applies to many different industries. I'll cover all the prerequisites in this video and point to helpful resources down below. Enjoy!

Code for this video:

Please Subscribe! And Like. And comment. Thats what keeps me going.

Want more education? Connect with me here:

More learning resources:

Join us at the School of AI:

Join us in the Wizards Slack channel:

Please support me on Patreon:

Signup for my newsletter for exciting updates in the field of AI:

#NeuralDifferentialEquations #SchoolOfAI #SirajRaval

[flyingzipper]

1 - Basic neural network theory |
8:30
2 - "Residual" neural network theory
| 12:40
3 - Ordinary Differential Equations (ODEs)
| 17:00
4 - ODE Networks
| 22:20
5 - Euler's Method to Optimize an ODENet
| 27:45
6 - Adjoint Method for ODENet Optimization
| 29:15
7 - ODENet's Applied to time series data
| 30:50
8 - Future Applications of ODENets | 33:41

[sashas]

The input-times weight-add a bias-activate song is brilliant and should be used in elementary schools

[OnlyGoodJawn]

Siraj dropped the most fire freestyle of 2019 in this video.

[thoyo]

I love how you're always excited about what you're talking about. It's infectious.

[Bbb]

This made me fall in love with AI and ML again. Thank you so much. I was going through a slump, but when watching this I couldnt stop smiling throughout the entire video

[amanasci]

Only channel on YouTube that motivates me to study Maths..

[pranavsreedhar]

thank you siraj for putting the effort to enclose a much larger, broader audience. Everyone benefits from this.

[arnau]

I'm only half way through the video and I can already tell this is my favorite one of 2019, and possibly my favorite research paper ever! Thanks, Siraj!

[motog]

I am feeling more happy and proud now for learning Mathematics as my favourite subject.
Another interesting reason to explore the AI more and more
Thanks, Sirj :)

[carlsilverberg]

You've gotten way better than the last time I checked you out. That was 4 years ago, lol, so I guess thats just normal. But great man! Loved it! Absolutely amazing content.

[irisgu]

Thank you! I watched many videos on ODE with ResNet and yours is the best!!!

[vuppumadhuri]

The code which was shown in this video at the end of the video, doesn't show the ODE definition block. I mean, where the ODE was specified, except for the solver. Without defining ODE, how's it possible to solve dx/dt or d2x/dt2?

[vman]

I regularly watch Siraj’s videos and this is one of the best I’ve seen... got my adrenaline pumping when I saw that list of topics to be covered at 8:30!

[loaywael]

Awesome Video, Hopping to cover more about new research papers in that simple way, I really enjoyed even I'm not mathematician.

[yasinilulea]

This is awesome, you're killing it mate!

[mlguy]

This could be interesting for me as someone that spent many years during his PhD looking at nonlinear ODEs. Now as a ML guy this would be great to relate back to my original work. There is a caveat that I was not clear on, there is a difference between stability conditions for ODEs which was not clear in the paper how they treat this.

[theaichannel]

Really interesting research, AI is moving so fast right now. There is so many doors going to be opened. Modelling more complicated functions but still keeping the memory tied in. Amazing stuff, your videos are first class!

[trycryptos]

Siraj... please tell me that you have travelled back in time to help us catchup with the future. I am just flabbergasted by the volume & intensity you handle.!
I have no words to comment just a dropped jaw in pure awe!!!😘

[saratbhargavachinni]

About a week back, I started working as Teaching Assistant to Under grad Differential Equations course, I wondered when I was reading the text, I had learnt all these theory myself I was in fresh men year but very rarely used these differential equations after the course and I wondered if I can use these in Machine learning (my area of interest). I am really excited after watching your video.

[carosare]

Ohh come on!
I needed this for my differential equations proyect last semester:/
such an interesting topic!