Neural Networks Explained from Scratch using Python

When I started learning Neural Networks from scratch a few years ago, I did not think about just looking at some Python code or similar. I found it quite hard to understand all the concepts behind Neural Networks (e.g. Bias, Backpropagation, ...). Now I know that it all looks quite more complicated when you see it written mathematically compared to looking at the code. In this video, I try to provide you an intuitive understanding through Python code and detailed animations. Hope it helps you :)

Code:

Find me on:

Citation:

Additional Notes:
1. You might’ve seen that we haven’t used the variable e at all.
This is for two reasons. First, normally we would’ve used it to calculate ‘delta_o’,
but due to some tricks, it is not needed here. Second, it is sometimes helpful to print the average error during training to see if it decreases.

2. To see how it performs on images not seen during training, you could only use just the first 50000 images for training and then analyze the results on the remaining 10000 samples. I haven’t done it in this video for simplicity. The accuracy, however, shouldn’t change that much.

To create a 1-tuple in python we need to write x=(1,). If we would just write x=(1), it gets converted to the integer 1 in Python.

Numpy introduces the shape attribute for arrays. Because the shape of a matrix has to be represented by a tuple like (2, 5) or (2, 4, 7), it is a good idea to represent a vector as a 1-tuple instead of an integer for consistency. So it is (X,).

If we want to use this vector in a matrix multiplication with a matrix, it doesn't work because you can't matrix multiply a vector with a matrix in numpy. So we need to add this 'invisible' second dimension of size 1. The line basically adds a (1,) vector to the shape of the (X,) vector which results in a matrix of size (X, 1). That's also why it doesn't work with (2,) because that would require more values. For example (5,) and (5, 1) both contain 5 values while (5, 2) would contain 10 values.

I should've shown the shapes in the shape information box as (X,) instead of just X. I think that also made it more confusing.

Credits:
17.08 - End
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Music: Ansia Orchestra - Hack The Planet
Music provided by: MFY - No Copyright
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

The animations are created with a python library called manim. Manim was first created by Grant Sanderson also known as 3blue1brown (YouTube) and is now actively developed by the manim community. Special thanks to everyone involved in developing the library!

Chapters:
00:00 Basics
02:55 Bias
04:00 Dataset
05:25 One-Hot Label Encoding
06:57 Training Loops
08:15 Forward Propagation
10:22 Cost/Error Calculation
12:00 Backpropagation
15:30 Running the Neural Network
16:55 Where to find What
17:17 Outro

[BotAcademyYT]

Please share this video if you know somebody whom it might help. Thanks :)

edit: Some people correctly identified the 3Blue1Brown style of the video. That is because I am using the python library manim (created by 3Blue1Brown) for the animations. Link and more information in the description. Huge thanks for all the likes and comments so far. You guys are awesome!

[hepengye]

As an ML beginner, I know how much effort and time is needed for such visualization of a program. I would like to give you a huge thumb! Thank you for the video.

[blzahz]

I can't say anything that hasn't been said already: This video is golden. The visualization, explaining, everything is just so well done. Phenomenal work.
I'm basically commenting just for the algo bump this video rightfully deserves.

[ejkitchen]

FANTASTIC video. Doing Stanford's Coursera Deep Learning Specialization and they should be using your video to teach week 4. Much clearer and far better visualized. Clearly, you put great effort into this. And kudos to using 3Blue1Brown's manim lib. Excellent idea. I am going to put your video link in the course chat room.

[magitobaetanto]

You've just explained very clearly in a single video what others try to vaguely explain in series of dozens videos. Thank you. Fantastic job! Looking forward to more great videos from you.

[craftydoeseverything]

I know I'm watching this 2 years after it was released but I really can't stress enough how helpful this is. I've seen heaps of videos explaining the math and heaps of videos explaining the code but this video really helped me to link the two together and demystify what is actually happening in both.

[Transcnder]

This is so interesting. I always wondered how a neural net works... but it's also good to remind ourselves that we're not as clever as we thought. I feel humbled knowing that there's some fierce minds out there working on these complicated problems.

[ElNachoMacho]

This is the kind of video that I was looking for to get beyond the basics of ML and start gaining a better and deeper understanding. Thank you for putting the effort into making this great video.

[mici]

Saw your post on Reddit. Thank you very much for the work you put in your videos. New subscriber.

[mrmotion]

Love this so much. So organised and was really helpful. So glad you put the effort into the animation. Keep up the great work!

[pisoiorfan]

That's it! Comprehensive training code loop for a 1 hidden layer NN in just 20 lines. Thank you sir!

[angelo]

Amazing video! The explanation was very clear and I understood everything. Really hope you're gonna be posting more videos on neural networks.

[hridumdhital]

As someone beginning machine learning, this video was so useful to really getting a deep understanding on how neural networks work!

[michaelbarry]

Amazing video. Especially the matrix effect on the code in the first second. Love it.

[bdhaliwal]

Fantastic job with your explanation and and especially the animations. All of this really helped to connect the dots

[eldattackkrossa]

oh hell yeah :) just got yourself a new subscriber, support your small channels folks

[chrisogonas]

Superbly illustrated! Thanks for sharing.

[photorealm]

Excellent video and accompanying code. I just keep staring at the code, its art. And the naming convention with the legend is insightful, the comments tell the story like a first class narrator. Thank you for sharing this.

[vxqr]

Subscribed. We need more channels like this!

[Lambertusjan]

Thanks for a very clear explanation. I was doing the same from scratch in python, but got stuck at dimensioning the weight matrices correctly, especially in this case with the 784 neuron input. Now i can check if this helps me to complete my own three layer implementation. 😅