ResNet (actually) explained in under 10 minutes

Want an intuitive and detailed explanation of Residual Networks? Look no further! This video is an animated guide of the paper 'Deep Residual Learning for Image Recognition' created using Manim.

Sources / credits

[nialperry]

Cracking video, Rupert. Well animated and explained. I am already satisfied with my understanding of ResNets after this.

[sarthakpatwari]

Mark my words, if he become consitent, this channel will become one of the next big thing in AI

[Cypher]

Thanks. Been out of touch with AI for far too long so this summary is very helpful.

[poopenfarten]

legit one of the best explanations i found

[prammar]

everyone is praising the video, maybe it's just me but i really didn't understand what the residual connection hopes to achieve? and how does it do that? didn't make it clear.

[agenticmark]

lol, I have fought that exact trendline so many times in ML :D Great humor. Great video work.

[louisdante]

7:53 Why is there a need to preserve the time complexity per layer?

[Omsip]

I pushed it to exactly 1k likes, cause it deserves it ... and many more

[logon]

You say that the identity function is added elementwise at the end of the block. So say I have an identity [1, 2] and the result of the block is [3, 4]. So would the output of the layer be [4, 6]? So its not a concatenation of the identity function which would be [1, 2, 3, 4], correct? You basically ensure the identity function is the same dimensionality as the output of the block then add them element-wise.

[TheBlendedTech]

Thank you, this was well put together and very useful.

[sergioorozco]

Is the right hand side of the addition supposed to have height and width dimension of 32x32 at 7:08? I think there is a small typo in the visual.

[Antagon]

Idk why, but simply adding bicubicly upscaled image to output of CNN with pixel shuffling layer achieves much better results than having any amount residual blocks. Also it's much faster.

[devanshsharma]

love the animation! Thanks for the clean and clear explanation!

[egesener]

Everyone say ResNet solves vanishing/gradient problem but dont we already use ReLu function istead of sigmoid to solve it ? Also part 4.1 of article say plain counterpart with batch normalization doesn't causes vanishing problem but still causes more error rate when layers are increased 18 to 34. Can you explain it ?

[ShahidulAbir]

Amazing explanation. Thank you for the video

[firefistace]

What is the residual in the image classification task?

[wege]

6:38 this is the part that really made me understand, thank you

[januarchristie]

Hello, I apologize for my question, but I still don't quite understand why learning residuals can improve model predictions better?
Thank you

[the_random_noob]

Lifesaver! Also, for classification, it's inevitable that the dimensions go down and channels go up across the network. But the 1 x 1 convolution on the input features to 'match the dimensions' kinda loses the original purpose i.e to retain/boost the original signal.. In a sense it's another conv operation that is no longer similar to the input (I mean it could be similar but certainly as not as the input features themselves). It's just the original idea was to have the same input features so that we could zero out the weights if no transformation is needed.

Atleast they're not as different from how the input features as transformed across the usual conv block(conv, pooling, batch norm and activation). Let me know if I am missing anything

[djauschan]

Amazing explanation of this concept.
Thank you very much