Watching Neural Networks Learn

A video about neural networks, function approximation, machine learning, and mathematical building blocks. Dennis Nedry did nothing wrong. This is a submission for #SoME3

My Links

Links and Content:

Timestamps
(0:00) Functions Describe the World
(3:15) Neural Architecture
(5:35) Higher Dimensions
(11:55) Taylor Series
(15:20) Fourier Series
(21:25) The Real World
(24:32) An Open Challenge

[EmergentGarden]

Some notes:
- A lot of you have pointed out that (tanh(x)+1)/2 == sigmoid(2x). I didn't realize this, so the improvement I was seeing may have been a fluke, I'll have to test it more thoroughly. It is definitely true that UNnormalized tanh outperforms sigmoid.
- There are apparently lots of applications of the fourier series in real-world neural nets, many have mentioned NERF and Transformers.

[MH-pqoo]

Having a PhD on Neural Networks, I can vouch that this video is a gem and needs more views. Great work.

[youngentrepreneurs]

When a neural network video feels like watching an Oscar-winning documentary

[debuggers_process]

I've actually done something quite similar – I had the network learn a representation of a 3D scene using a signed distance function. In this context, I found that using a Leaky ReLU gives the models a pseudo-polygonal appearance, while tanh creates smoother models but is somewhat less effective in terms of learning efficiency. Interestingly, the Mish function seems to strike a balance between these two approaches, producing smooth models while maintaining nearly the same learning efficiency as the Leaky ReLU.

[greenstonegecko]

This is BY FAR the most understandable AI ... that I have ever seen. This is amazing!!

Cannot overstate how beautifully this is executed

[godfreytshehla]

I am currently studying PhD in Applied Mathematics and my research focuses on Mathematical Finance and Machine Learning.
This is the best video that explains what artificial neural networks are. This is well executed!
Thank you for this.

[hasalinahstevenson]

The tone, the background soothing music, the images, you made something so complicated so easy to digest. Great job. I know you are brilliant!

[kingKai]

I've been interested in this field for years but 30 minutes of this explained to me what I couldn't fully understand for years now. 🎉 THANK YOU!

[WinstonWalker-fcty]

This is amazing! I’ve been learning the fundamentals over the last few weeks and this is the best video I’ve seen so far. I’m not a math expert by any means, but I actually understood almost everything you said! Thank you so much.

[jordanzamora]

Great Video! This video actually made me cry seeing sorta more viscerally how functions are stitched into EVERYTHING, makes you think that maybe we are a lot like the mandlebrot, the universe recursively calculating itself. Thank you for this video!

[pavansaish]

Best ever video on NN with higher level viz. This gave me a vibe of watching Interstellar movie when comparing NN with higher-level math. Also, Kudos to the video editor😄

[AB-wfek]

This is an amazing explanation. I'm actually a visual artist and have been deep into image generation for the past year. At this point I have a good basic knowledge and strong intuitive understanding of machine learning and training (I'm familiar with things like Fourier transforms, gradient descent, and overfitting), but this really validated and clarified a lot of those concepts. Many thanks for taking the time to create such an elegant video.

[Beerbatter]

Wow, this is exceptional. As a semi-retired mechanical engineer studying on my own to better understand neural networks and AI, this is incredibly interesting and educational. Bravo on your excellent presentation on difficult topics. I really enjoy getting the nitty-gritty math behind it all. Subscribed. Thanks and cheers.

[zaktoid]

Math student here
The link you made between taylor series and neural network is amazing, it gave me very good insight about both of them !!!
Thank you !

[aaronlowe]

This video was absolutely amazing. I had some hypotheses about the Fourier Transform being the key to understanding patterns in multi-dimentsional data, but this video beautifully tied all those hypotheses together for me. Absolute hats off. Thank you and hope to see more of this kind of content.

[benedwards]

By far the best SoME3 video I’ve seen so far. Great intuitive explanation and stunning visuals.

[wrxtt]

Really incredible video! It is really interesting to see why we use different networks- thank you for making this!

[que_]

I cannot begin to tell how brilliant this video is, and how insightful-- far, far better than the innumerable contents here. You must not, however, claim that you find Maths difficult-- as the person who truly finds it 'difficult' would not have explained two critical mathematical concepts with this comprehensive clarity. The pacing of your words, the contents, the realism, the sequence of topics, and the effort to describe the concepts visually makes it every bit worth the time the viewers put in, and it only speaks of your immense caliber. First visit, and worth every bit!

[muhannadobeidat]

This video is amazing. The ideas, the animation, the examples, even the voice and narration style. Excellent in every detail.

[ignessrilians]

Wow these videos are INSANELY well made and well explained.
You're awesome!