KL Divergence - CLEARLY EXPLAINED!

This tutorial explains what KL Divergence is as well as show its derivation using log-likelihood ratio.

Friend link to "What is Entropy?" article:

#kldivergence
#variationalinference
#bayesianstatistics

[orjihvy]

These videos should be highly recommended by YouTube algorithm

[Vikram-wxhg]

Fantastic tutorial! What I find great is that you anticipate the questions arising in the student's mind and then address them with very satisfying explanations!

[hannes]

Your explanations and visualizations are very good! Also you teaching style has the perfect tempo. Thank you very much for this great explanation

[kaiponel]

The best explanation I've heard about KL-Divergence. Keep up the great work.

[paedrufernando]

I cannot express the gratitude I have for your explanation. What a beautiful soul you are .wow

[homakashefiamiri]

It was fantastic. The most informative video of KL divergence

[aclapes]

Amazing value, Kapil. I like several of the things you do when you teach: refreshing necessary concepts (expectation), the precision of your language and notation, equivalent expressions, and so on. The pace is also great. Thank you very much!

[TheProblembaer]

These videos are pure gold. Thank you so much. You can explain incredible well.

[shantanudixit]

was searching for some tutorials on approximate inference and the pre-reqs for it and stumbled upon this, literally my mind got blown with the way you explained the concepts here.

[matiascaceres]

this is the most simple and clear explanation of KL divergence, thank you

[pauledam]

He is good! Very good to say, for example, we want the average difference, but when talking about rv we talk about expected value ... . And many other very careful explanations.

[puck]

Thank you for the video. I am preparing a paper for my math stats class and after many videos yours gave me the best total explanation with terminology I am familiar with so far.

[nikhilsrajan]

Hi, I had one doubt, in 5:18, why do we multiply p_theta(xi) to log(p_theta(xi)/q_phi(xi)) and not multiply q_phi(xi) to log(p_theta(xi)/q_phi(xi))?
In 7:30 you show the variation with q_phi(xi). It seems like the probability distribution that is multiplied with the function of the random variable log(P1(x)/P2(x)) is the probability distribution that appears in the numerator. Is there a reason to not multiple the probability distribution in the denominator?

[mdekramnazar]

i loved this small session on KL Divergence. Thank you sir for this beautiful lecture.

[bluepeace]

Thank you very much! Your explanations are really clear and neat. Thanks to your video, now I understand KL divergence much much better than I did before.

[sergiobromberg]

This is amaaaazing! What a nicely paced and deep explanation!

[gauravkumarshah]

You have a talent for teaching. Good explanation.

[raghav]

Great content! Definitely need more views. Please keep uploading videos.

[spandanbasu]

I fell in love with the explanation. Thanks a lot Kapil.

[TJ-zssv]

Thank youu for great explanation. @9:27 I can't get how come reverse KL divergence has mode seeking behaviour and forward has mean seeking. I understood that P(x) is multimodal gaussian distribution, but what is Q(X) as we needed both distribution for finding K-L divergence.