Neural Networks From Scratch - Lec 13 - Swish Activation Function

Building Neural Networks from scratch in python.
This is the thirteenth video of the course - "Neural Networks From Scratch". This video covers the Swish activation function and its importance. We look at the derivative of swish and discuss the advantages and disadvantages of using Swish activation function. We looked at the performance comparisons of swish against all existing activation functions and finally we saw the python implementation

Course Playlist:

Please like and subscribe to the channel for more videos. This will help me in assessing your interests and creating more content. Thank you!

Chapter:
0:00 Introduction
0:28 Neural Architecture Search
2:02 Dsicovered Activations by NAS
3:45 Swish Activation Function
4:55 General form of Swish
5:25 Derivative of Swish
6:54 Properties of Swish
7:17 Comparison with other Activations
8:22 Python Implementation

#swishactivationfunction, #swishactivationfunctioninneuralnetwork, #reluactivationfunction, #activationfunctioninneuralnetwork, #vanishinggradient, #NeuralArchitectureSearch, #selfgatedactivationfunction

[abdullahsheriff_]

So nice that you introduced the NAS concept. Thank you :)
Keep gently introducing us to new concepts.

[tarunreddy]

Clear and crisp explanation with explainable plots and statistics. You have a bright future, bro.