Derivative of Sigmoid and Softmax Explained Visually

The derivative of the sigmoid function can be understood intuitively by looking at how the denominator of the function transforms the numerator. The derivative of the softmax function, which can be thought of as an extension of the sigmoid function to multiple classes, works in a very similar way, and in this video, I explain that relationship. The sigmoid and softmax are commonly used in neural networks, so having a more intuitive understanding of their derivatives will help us better understand how the gradients propagate through our neural networks during backpropagation.

My previous video, "Why We Use the Sigmoid Function in Neural Networks for Binary Classification":

My previous video, "Softmax Function Explained In Depth with 3D Visuals":

Desmos graph for sigmoid derivative:

Desmos graph for softmax derivative:

Join our Discord community:

Connect with me:

🎵 Kazukii - Return

[papaaemeritus]

This channel is going to explode soon. Top shelf content!

[shvprkatta]

I love mathematics...when I see such a great contribution from a pro like you...I feel ecstatic!...thank you so much!!

[nigelstafford]

what an incredible way of demonstrating mathematics... my mind is blown

[raufbhat]

Amazing video! never thought I could interpret functions like this. All the best

[chitrakbiswas]

mann..I just love your videos...just awesome explanation

[jackkensik]

Your videos are great, I really like your explanation but I think I could understand it more if you talked about the reasoning of going from one step to the next. Thanks!

[rembautimes]

This lecture reminds of my first high school calculus class 30 years ago. Back then polynomials played nice with derivatives and exponentials were full of little tricks. I always wondered why should exponentials always return a positive range. With today’s NN we have the answer

[danielleivy]

Currently learning ML, and have taken multivariable calculus but sometimes the instructor pulls things out of nowhere with minimal background explanation and wants you to research it on your own. Your videos are like gold - thanks!
I've stopped feeling like I was in a "prerequisites black hole".

[pkmcdonald]

Awesome! As someone with limited background with machine learning I find these videos fascinating! Great job Elliot! I wonder if you could partner with different courses in academia to help people approaching these subjects. Great visual walk through and intuitive explanatiln

[forthrightgambitia]

Where does the negative sigmoid come from? If you treat dP_b/da for example as a differential equation I don't see where the sigmoid comes from?

EDIT: I think I see, the shift s is negative. The other values will increase s So if we split it out to e^a*e^(-s) then you have the form of a negative sigmoid. The values increase s linearly as, isolating for the change in that value the logarithm and exponential cancel out.

[学堂上]

consult, Exp(x)-->1, Exp(x)+1 -->Sigmoid is implemented here. If x-->sigmoid is implemented, how to visualize the settings?
English is not good, please forgive me if I am impolite

[Rami_Elkady]

That is a very short video, you should have covered 200 hours of pre calculus, 200 hours of calculus and then explain this in another 200 hours ...

Why do people think it is smart to explain in 20 minutes what could have easily been explained in 2 minutes - it so frustrating. Explain this in 2 minutes, do this with 20 more videos and spend the difference with your girl friend ...