Intuitively Understanding the KL Divergence

This video discusses the Kullback Leibler divergence and explains how it's a natural measure of distance between distributions. The video goes through a simple proof, which shows how with some basic maths, we can get under the KL divergence and intuitively understand what it's about.

[Vroomerify]

I just want to say. This is--by far--the best explanation of KL divergence I've found on the internet. Thanks so much!

[unsaturated]

The most intelligent people are the one's who are able to explain the hardest concepts in the most intutive way possible. Thanks.

[niofer]

This was actually one of the most helpful videos. Thank you

[liliz]

KL divergence confused me for so long, and I understood it just by watching your video for one time, thank you very much!

[AashraiRavooru]

A question here why will the number of heads and number of tails be the same for both the distributions at 3:04. If the probabilities for both the coins are different then the number of occurrences of heads and tails can also be different

[karstenhannes]

This type of explanation is perfect! First boiling the problem down to the most intuitive understanding and from there deduce the general formula. Thanks so much!

[RedwanKarimSony_napstar_]

Best explanation of the KL divergence in YouTube for sure....
Thanks...

[nericarcasci]

You are unbelievably good at teaching man. You explained it better than they did in my course.

[jimmygan]

holy smoke, you are legit GOAT. so concise yet clear and intuitive explanation.

[SunilKumarSamji]

Excellent video. Can someone help me understand why is it called Divergence in the first place? Why are we taking 1/N power to normalise it to sample space, I did not understand the logic behind this.

[wynandwinterbach]

I'm just rewatching this video to freshen up my deep learning fundamentals. Super clear video, thank you so much!

[balasubramanyamevani]

@3:26 I don't understand how are we normalizing by raising it to the power of 1/N. Could you please explain that?

[baskaisimkalmamisti]

I didn't expect that good explanation from a randomly suggested youtube video

[haresage]

Great explanation! One technical remark I have is that (from my understanding) KL divergence is not technically a measure of distance, since it's not symmetric ( Dlk(P||Q) != Dlk(Q||P) ).

[adityakulkarni]

Perfectly explained in 5 minutes. Wow.

[sharingpurpose]

Bro, this intuition was not normal, u r just genius!!

[matakos]

Thanks so much for this, needed to understand what KL Divergence is for a paper I'm reading and you just saved me so much time!

[marcegger]

Great video! Loved the intuition behind the KL distribution. For some thinking about applications, this is used in the loss function of Variational Auto Encoders, a class of deep networks, and is used to find low dimensionality features of high dimensionality input data as an encoder. (e.g. use this to deconstruct images into "features")

[germangarcia]

One of the most useful explanations ever. Thanks!!

[alkanair]

Thank you so much for this content. By far the explanation of KL Divergence seen so far