Neural Network from Scratch | Mathematics & Python Code

In this video we'll see how to create our own Machine Learning library, like Keras, from scratch in Python. The goal is to be able to create various neural network architectures in a lego-fashion way. We'll see how we should architecture the code so that we can create one class per layer. We will go through the mathematics of every layer that we implement, namely the Dense or Fully Connected layer, and the Activation layer.

Chapters:
00:00 Intro
01:09 The plan
01:56 ML Reminder
02:51 Implementation Design
06:40 Base Layer Code
07:55 Dense Layer Forward
10:42 Dense Layer Backward Plan
11:23 Dense Layer Weights Gradient
14:59 Dense Layer Bias Gradient
16:28 Dense Layer Input Gradient
18:22 Dense Layer Code
19:43 Activation Layer Forward
20:46 Activation Layer Input Gradient
22:30 Hyperbolic Tangent
23:24 Mean Squared Error
26:05 XOR Intro
27:04 Linear Separability
27:45 XOR Code
30:32 XOR Decision Boundary

====

Corrections:
17:46 Bottom row of W^t should be w1i, w2i, ..., wji
18:58 dE/dX should be computed before updating weights and biases

====

[GX]

In the backward function of the dense class you're returning a matrix which uses the weight parameter of the class after updating it, surely you'd calculate this dE/dX value before updating the weights, and thus dY/dX?

[ldx]

This video, instead of the plethora of other videos on "hOw tO bUiLd A NeUrAl NeTwOrK fRoM sCraTcH", is the literal best. It deserves 84 M views, not 84 k views. It is straight to the point, no 10 minutes explanation of pretty curves with zero math, no 20 minutes introduction on how DL can change the world

I truly mean it, it is a refreshing video.

[robinferizi]

I like how he said he wouldn’t explain how a neural network works, then proceeds to explain it

[orilio]

I love the 3b1b style of animation and also the consistency with his notation, this allows people to learn the matter with multiple explanations while not losing track of the core ideas. Awesome work man

[faida.]

This is basically ASMR for programmers

[ardumaniak]

The best tutorial on neural networks I've ever seen! Thanks, you have my subscription!

[rogeliogarcia]

Thanks for making such great quality videos. I'm working on my Ph.D., and I'm writing a lot of math regarding neural networks. Your nomenclature makes a lot of sense and has served me a lot. I'd love to read some of your publications if you have any.

[aflakmada]

Very clean and pedagogical explanation. Thanks a lot!

[mohammadrezabanakermani]

It is the best one I've seen among the explanation videos available on YouTube!
Well done!

[generosonunezarias]

This might be the most intuitive explanation of the backpropagation algorithm on the Internet. Amazing!

[neuralworknet]

Best tutorial video about neural networks i've ever watched. You are doing such a great job 👏

[samirdaniels]

This was the best mathematical explanation on YouTube. By far.

[rubenfalvert]

Probably the best explaination of neural network of Youtube ! The voice and the musique backside is realy soothing !

[SleepeJobs]

This video really saved me. From matrix representation to chain rule and visualisation, everything is clear now.

[wagsman]

Not only was the math presentation very clear, but the Python class abstraction was elegant.

[rumyhumy]

Man, I love you. How many times i tried too do the multilayer nn on my own, but always faced thousand of problems. But this video explained everything. Thank you

[darshangowda]

This could be 3Blue1Brown for programmers! You got yourself a subscriber! Great video!

[nkryxsu]

Thank you very much for your videos explaining how to build ANN and CNN from scratch in Python: your explanations of the detailed calculations for forward and backward propagation and for the calculations in the kernel layers of the CNN are very clear, and seeing how you have managed to implrment them in only a few lines of code is very helpful in 1. understanding the calculations and processes, 2. demistifying the what is a black box in tensorflow / keras.

[bernardcrnkovic]

Absolutely astonishing quality sir. Literally on the 3b1b level. I hope this will help me pass the uni course. SUB!

[rishikeshkanabar]

This is such an elegant and dynamic solution. Subbed!