Batch Norm in PyTorch - Add Normalization to Conv Net Layers

In this episode, we're going to see how we can add batch normalization to a convolutional neural network.

🕒🦎 VIDEO SECTIONS 🦎🕒

00:30 What is Batch Norm?
04:04 Creating Two CNNs Using nn.Sequential
09:42 Preparing the Training Set
10:45 Injecting Networks Into Our Testing Framework
14:55 Running the Tests - BatchNorm vs. NoBatchNorm
16:30 Dealing with Error Caused by TensorBoard
19:49 Collective Intelligence and the DEEPLIZARD HIVEMIND

💥🦎 DEEPLIZARD COMMUNITY RESOURCES 🦎💥

👋 Hey, we're Chris and Mandy, the creators of deeplizard!

👉 Check out the website for more learning material:

💻 ENROLL TO GET DOWNLOAD ACCESS TO CODE FILES

🧠 Support collective intelligence, join the deeplizard hivemind:

🧠 Use code DEEPLIZARD at checkout to receive 15% off your first Neurohacker order
👉 Use your receipt from Neurohacker to get a discount on deeplizard courses

👀 CHECK OUT OUR VLOG:

❤️🦎 Special thanks to the following polymaths of the deeplizard hivemind:
Tammy
Mano Prime
Ling Li

🚀 Boost collective intelligence by sharing this video on social media!

👀 Follow deeplizard:

🎓 Deep Learning with deeplizard:

🎓 Other Courses:

🛒 Check out products deeplizard recommends on Amazon:

🎵 deeplizard uses music by Kevin MacLeod

❤️ Please use the knowledge gained from deeplizard content for good, not evil.

[bieberssaman]

This series is great. We have to appreciate deep lizard hard work for the fine and smooth tutorial.

[sanaullahaq]

One by one reached to end of the series. This series was a great journey. Thanks, deep lizard all of your great efforts.

[towardsgeneralintelligence]

You guys are the best. Soon, this "Deep Lizard" channel will be the most suggested brand to learn Deep Learning. Just Beautiful.😍💖👏

[min-youngchoi]

This series was great journey. Thanks deeplizard all of your masterpieces.

[tomergonen]

I'm now going to backpropagate through this series and optimize all weights to "like" - that was simply amazing. Humanity thanks you so much!

[rushirajparmar]

This is the best Pytorch playlist in the youtube! Loved it❤️❤️ Keep it up guys!

[chriskorfmann]

I will be forever thankful for the amazing information you've shared throughout this series, and more so for how you presented it. You have a great teaching style! I enjoyed the vids.

[Ip_man]

This is one of the best tutorial series I've seen so far, Thank you so much :)

[ameerhussain]

Awesome!!!!
Testing multiple networks is my take away.
Would you be able to do a tutorial on how to do kfold cross validation when using PyTorch model?

[deepjyotiroy]

This channel is a gem! Thank you! Can you make a video on ResNet?

[Felix]

Chris, thanks again!
As for the Tensorboard issue, I found the following:

- The maximum pathlength has been defined by 260 characters. Since Windows 10, one can enable this to be longer. NTFS now allows for up to around 32, 000 characters.
- I guess this has something to do with increased memory availability overall.
- You can enable this through your Registry Editor
- After enabling it, Tensorboard's issue with writing the files is resolved, even with the longer comments.

[Bristolian]

Amazing amazing job! Thank you so much.

[JohnWick-fuvt]

when can we expect to see the next videos?? please upload them as soon as you can. and i have watched all the videos from fundamentals, then with keras and now this pytorch series also..you guys did a tremendous job teaching CNN. Thanks a lot <3

[fraoney]

I hope you make a video on pre-trained VGG16 transfer learning and fine tuning

[PergalingaSistema]

Hi Chris, you are the best, this series is a treasure! I am much more confident about my knowledge. Thank you!

I have only one practical question: could you elaborate more on methods for visualizing the structure of dataset? I mean if I use size, shape, type or indexes it is quite okay.
But from the ground up - having only the train_set - I can hardly imagine where and how labels are stored and why train_set.data differs from train_set[0] etc etc. Is the best way just looking into the constructor code?

[FarooqComputerVision]

Dear Chris, It is an excellent course. I have a question: "why didn't you cover the testing part of the trained model"?

[pranavagarwal]

Thanks for the awesome contents. I need some help with a pytorch implementation. Do you have any advice on how to implement "SAME" padding in pytorch as is done in tensorflow and keras?

[gravitas-]

Possible bug: are the network/weights reset between each run? It looks like the network is created once then reused between each run?

[phongphamxuan]

I have a question: When we are normalize previous Relu and when we are normalize after Relu. Thanks !!!

[shaheermehmood]

Hey, that was a great tutorial, can you tell me how should I add batch norm in the normal code, not in the sequential ones. do I need to add it in the constructor than in methods, it would be a great help if you could let me know. Cheers !!!