filmov
tv
Weight Initialization explained | A way to reduce the vanishing gradient problem
Показать описание
Let's talk about how the weights in an artificial neural network are initialized, how this initialization affects the training process, and what YOU can do about it!
To kick off our discussion on weight initialization, we're first going to discuss how these weights are initialized, and how these initialized values might negatively affect the training process. We'll see that these randomly initialized weights actually contribute to the vanishing and exploding gradient problem we covered in the last video.
With this in mind, we'll then explore what we can do to influence how this initialization occurs. We'll see how Xavier initialization (also called Glorot initialization) can help combat this problem. Then, we'll see how we can specify how the weights for a given model are initialized in code using the kernel_initializer parameter for a given layer in Keras.
Reference to original paper by Xavier Glorot and Yoshua Bengio:
🕒🦎 VIDEO SECTIONS 🦎🕒
00:30 Help deeplizard add video timestamps - See example in the description
09:42 Collective Intelligence and the DEEPLIZARD HIVEMIND
💥🦎 DEEPLIZARD COMMUNITY RESOURCES 🦎💥
👋 Hey, we're Chris and Mandy, the creators of deeplizard!
👉 Check out the website for more learning material:
💻 ENROLL TO GET DOWNLOAD ACCESS TO CODE FILES
🧠 Support collective intelligence, join the deeplizard hivemind:
🧠 Use code DEEPLIZARD at checkout to receive 15% off your first Neurohacker order
👉 Use your receipt from Neurohacker to get a discount on deeplizard courses
👀 CHECK OUT OUR VLOG:
❤️🦎 Special thanks to the following polymaths of the deeplizard hivemind:
Tammy
Mano Prime
Ling Li
🚀 Boost collective intelligence by sharing this video on social media!
👀 Follow deeplizard:
🎓 Deep Learning with deeplizard:
🎓 Other Courses:
🛒 Check out products deeplizard recommends on Amazon:
🎵 deeplizard uses music by Kevin MacLeod
❤️ Please use the knowledge gained from deeplizard content for good, not evil.
To kick off our discussion on weight initialization, we're first going to discuss how these weights are initialized, and how these initialized values might negatively affect the training process. We'll see that these randomly initialized weights actually contribute to the vanishing and exploding gradient problem we covered in the last video.
With this in mind, we'll then explore what we can do to influence how this initialization occurs. We'll see how Xavier initialization (also called Glorot initialization) can help combat this problem. Then, we'll see how we can specify how the weights for a given model are initialized in code using the kernel_initializer parameter for a given layer in Keras.
Reference to original paper by Xavier Glorot and Yoshua Bengio:
🕒🦎 VIDEO SECTIONS 🦎🕒
00:30 Help deeplizard add video timestamps - See example in the description
09:42 Collective Intelligence and the DEEPLIZARD HIVEMIND
💥🦎 DEEPLIZARD COMMUNITY RESOURCES 🦎💥
👋 Hey, we're Chris and Mandy, the creators of deeplizard!
👉 Check out the website for more learning material:
💻 ENROLL TO GET DOWNLOAD ACCESS TO CODE FILES
🧠 Support collective intelligence, join the deeplizard hivemind:
🧠 Use code DEEPLIZARD at checkout to receive 15% off your first Neurohacker order
👉 Use your receipt from Neurohacker to get a discount on deeplizard courses
👀 CHECK OUT OUR VLOG:
❤️🦎 Special thanks to the following polymaths of the deeplizard hivemind:
Tammy
Mano Prime
Ling Li
🚀 Boost collective intelligence by sharing this video on social media!
👀 Follow deeplizard:
🎓 Deep Learning with deeplizard:
🎓 Other Courses:
🛒 Check out products deeplizard recommends on Amazon:
🎵 deeplizard uses music by Kevin MacLeod
❤️ Please use the knowledge gained from deeplizard content for good, not evil.
0:04:10
Weight Initialization for Deep Feedforward Neural Networks
0:10:12
Weight Initialization explained | A way to reduce the vanishing gradient problem
0:06:12
Weight Initialization in a Deep Network (C2W1L11)
0:12:22
L11.6 Xavier Glorot and Kaiming He Initialization
0:12:56
Tutorial 11- Various Weight Initialization Techniques in Neural Network
0:06:01
L11.5 Weight Initialization -- Why Do We Care?
0:06:03
Weight Initialization techniques In Neural Network|How to initialize weight in a deep neural network
0:21:07
Xavier/Glorat And He Weight Initialization in Deep Learning
1:42:51
Day 2 - Workshop on 'From Scratch to Deployment: Hands-On Model Building with Keras and TensorF...
0:05:22
Introduction to Weight Initialization
0:02:32
Weight Initialization and Regularization Techniques for NNs
0:07:37
L11.7 Weight Initialization in PyTorch -- Code Example
0:18:16
Tutorial 8 - Weights Initialization Techniques in Neural Networks
0:24:46
Tutorial 17- Create Artificial Neural Network using Weight Initialization Tricks
0:07:58
Random Initialization (C1W3L11)
0:22:01
Tutorial 98 - Deep Learning terminology explained - Kernel (weights) initialization and padding
0:02:14
Initialization
0:49:24
Weight Initialization Techniques | What not to do? | Deep Learning
0:12:55
L8/2 Stabilize Training - Weight Initialization
0:17:56
NN - 19 - Weight Initialization 2 - What to do? Xavier Glorot & Kaiming He inits
0:03:48
Deep Learning for Python Developers : Weight Initialization for Deep Networks | packtpub.com
0:09:44
NN - 18 - Weight Initialization 1 - What not to do?
0:04:47
Pytorch Quick Tip: Weight Initialization
0:13:10
Weight initialization
Комментарии