How to Create a Neural Network (and Train it to Identify Doodles)

Exploring how neural networks learn by programming one from scratch in C#, and then attempting to teach it to recognize various doodles and images.

The data is from:

Chapters
0:00 Introduction
2:39 The decision boundary
3:49 Weights
5:42 Biases
6:45 Hidden layers
7:45 Programming the network
9:57 Activation functions
12:42 Cost
15:07 Gradient descent example
18:22 The cost landscape
19:55 Programming gradient descent
21:10 It's learning! (slowly)
23:21 Calculus example
27:34 The chain rule
29:50 Some partial derivatives
33:14 Backpropagation
39:25 Digit recognition
43:56 Drawing our own digits
47:37 Fashion
48:25 Doodles
52:00 The final challenge

Music:
Cosmic Waves - Michael FK
Amber - The Stolen Orchestra
Beyond the Horizon - Sounds Like Sander
Air - Assaf Ayalon
Purest Form - Sounds Like Sander
Hear Wide Open - Sounds Like Sander
Universal Wonder - Moments
Roman P - Moments
All In Good Time - Shimmer
It Will Come Back - The Stolen Orchestra
Frontier - Shimmer
New Moon - Cloud Wave
Sunflower - Cody Martin
Inner Peace - Moments
Enchanted - Cody Martin
Just Around The Corner - Shimmer

[SebastianLague]

Hey everyone, at 20:17 it should be -= costGradientW (the minus sign is missing). I somehow managed to delete it while formatting the code for the recording! Thanks for letting me know in the comments.

[niceguysayshi]

As someone who has some experience with machine learning i can say this has to be the most intuitive explanation i have ever seen

[rienkthegamer]

I’ve never had someone explain calculus so intuitively.
The quality of this content is absolutely incredible.

[wlockuz]

I am a software engineer and I've always wanted to learn machine learning, being able to code is not a problem for me but being terrible at maths and statistics makes it hard to get accustomed to all the terms and concepts.

I've tried multiple courses from different platforms and instructors and all of them try to teach you "what" to do instead of "why" to do it. I personally find learning more intuitive when I know why am I doing something instead of blindly following steps.

This video is exactly the type of introduction to ML that I've always wished for, The explanations are highly intuitive and most importantly visual, there are no assumptions and no brushing over concepts, Nothings being done just for the sake of it, Everything is explained in simple language. I admit I'll still have to watch this video a couple of more times to make full sense of it because its jam-packed with so much information.

You'll probably miss this comment in the sea of other comments (although I hope note) but I genuinely want to thank you, This video has relit my once dead interest in ML, I would love to see more videos from you on this topic or least get some recommendation on where I can learn more.

[CYOND]

The production quality is off the charts. As a software developer I can't even imagine the amount of hard work, research, technical knowledge, expertise, patience and determination this must have required. Hats off to you :)

[TAPa]

"I'm bad at naming things"

There are only 2 hard problems in computer science: cache invalidation, naming variables, and off-by-one errors

[saar]

You're probably not gonna see this message, but I want you to know you give me inspiration and motivation to learn, do and achieve more as i believe you do for many others

[Suburp]

I wish somebody had explained calculus like this in school. Intuitive, descriptive, visual, simple, elegant. This content is marvellous.

[blitzarsun]

This is the most intuitive explanation of machine learning I've watched. I hope you return back to it soon!

[mangoalias]

everything else aside, can we all appreciate the incredible visuals all of Sebastian's videos have? the animated graphs, the visualizations, the explanations, its so pleasing :)

[TheSome]

25:18 OMG, and at this point I've completely realized the true nature of the derivative — why it becomes a slope function, why x^2 turns into 2x and so on.
It was one of those mind-blowing moments of insight, which most of us have experienced at least once in our life.
Thank you, Sebastian!

[tartarugabradipo]

Sebastian: "The End"
Neural Network: "Oh, that's for sure a tractor"

[nocturnx]

I began loving this video the second I realized you had explained derivatives without actually mentioning them. I love practical approaches!

[cvntdav]

I love every single video that you make, you make such amazing stuff! I wish you could bring back the "How computers work" series which made me discover my passion ★

[Kibito]

It’s such a bittersweet moment watching your videos as they’ve recently come out because I know such great content with such level of detail takes so long to produce and it’s going to be a long and sad time until your next video comes out. I just love your videos man, everything you touch becomes gold, you make so many topics that are so boringly taught at uni seem exciting!

[gpazsilva]

It's awesome how, after even having classes in college about neural networks, I finally understood how they work *in practice*. I studied the theory and saw a lot of "for dummies" explanations about NNs, but people usually abstract the actual code from their explanations and this used to frustrate me a lot. Thank you so much for the explanations, Sebastian; your content is gold.

[hulmaji]

I cannot believe how lucky I am (how ALL of us are) that you sir exist. This quality, effort and precision put into these videos... Just wow... Thank you!

[avwie]

Having worked extensively with neural networks some 10 years ago, I must say this is hands-down the best explanation I have seen for people who are new to it. Excellent visualisations and explanations. It is so great for someone to start working on the absolute basics (simple perceptron) and working up, instead of directly going to PyTorch of TensorFlow.

[RabbleRousy]

As someone who watched a lot of these videos when writing my Bachelor's thesis on NNUEs (a specific kind of neural network for chess), I can safely say this is the best introduction to neural networks I've seen. Absolutely love all the visualization, how you start from the ground up but still include the calculus etc. Fun fact: my thesis was somewhat inspired by your Chess Engine video as well. I love your content, becoming a patreon now!

[YMandarin]

I love the visual explanations and the hand setting of the weights, its a really intuitive explanation for the networks