Entropy | Cross Entropy | KL Divergence | Quick Explained

After a long time, finally, here's a topic which inherited in a lot of things, especially when it comes to generative modeling. In this video, we'll see about Entropy, Cross-Entropy, and most important KL Divergence which is used frequently in Generative Adversarial Networks.
Let's see what it is, how it originated, and what it does.
I hope you'll like it. If not, please leave your feedback in the comments.
And as always,
Thanks for watching ❤️

Timestamps:
0:00 Entropy
2:13 Cross Entropy
3:23 KL Divergence
4:35 KL Divergence in code
5:25 Things to remember

[pashamorozov]

this video is just a treasure. the simplest explanation. thanks!!

[durandekamga]

Nice explaination
Thank you for this clear presentation

[ucembaikbangerparah]

im really thankful with your video, its so simple and make me understand more clearly about kl divergence <3 i hope i found your video earlier

[utkuaslan]

finally a very concise explanation on youtube.thank you very much.

[johnny]

Really Insightful Explanation that solves things struggled me the most!

[lincolnsimms]

Thanks. Really well put together. Excellent explanation.

[PoojaKumawat-zi]

What will happen if instead of one-hot encoding hard-labels we have soft labels distribution?

[michoello]

Thank you so much for amazing explanations!
What software do you use to create animations in your videos?

[dareeul]

Your video is great, but i see some mistake in the cross entropy calculation ln(0.2)= -1.6 not 1.6, also ln(0.15) and ln(0.05) . I think you just forgot minus sign. 😅