The Kernel Trick - THE MATH YOU SHOULD KNOW!

Some parametric methods, like polynomial regression and Support Vector Machines stand out as being very versatile. This is due to a concept called "Kernelization".

In this video, we are going to kernelize linear regression. And show how they can be incorporated in other Algorithms to solve complex problems.

If you like this video, hit that like button. If you’re new here, hit that SUBSCRIBE button and ring that bell for notifications!

FOLLOW ME

REFERENCES

[CodeEmporium]

Going to make a video on SVM and how it uses this kernel trick. So if you want to understand the math behind one of the most common Machine learning Algorithms, *subscribe* to keep an eye out for it ;)

[pavan]

Few of my friends wanted to get into ML at some point, but when they realized ML is just maths they went back to web dev. I love math and your videos make me love ML even more. Keep up!

[achillesarmstrong]

very good explanation. also in depth. Most other videos are just explaining without formula, which are too simple

[krishnasumanthmannala]

I think at 2:50 the suffix to y and x should be i.

Thank you for the great explanation.

[johnfinn]

At about 2:56 you need to explain to customers the logic here: w* is not the solution because alpha depends on w. Also, at about 4:16 you cancel K, although K is singular. At lest you need some discussions of the range and null space of the kernel.

[abhinav]

Amazing concept. it helped me a lot to learn the algorithm from the grass root level.

[harry]

Damn Dude!!, you really deserve a lot more views and subscriptions.Keep doing the great work.

[chaosido]

omg I watched plethora of videos and read so many articles trying to explain me what kernel method actually gains, and I finally understand not only conceptually but also down to the technical level

[niteshkans]

There are grand errors in the equations that you solved. But, yes the explanation is on point.

[jamesfulton]

I think there might be a mistake in your equation for alpha_n at ~3:08. The summation shouldn't be there

[msjber]

At 3:37, isn't the Kernel matrix K of size (m, m) rather than (m, n) since we do a dot product of every observation (from X1 to Xm) with every other one, so doing a square matrix (as you mentioned yourself just before), this gives a matrix K of size m * m, and not m * n unless I missed stgh.
So the last element of the first row for instance should be Phi(X1)t * Phi(Xm), and not Phi(X1)t * Phi(Xn).
Correct ?

[JI]

I understand data scientists might want to shy away from Hilbert spaces, but this stuff is so much clearer if you just use the Finite Representer Theorem to reformulate Ridge regression as a simple regression problem involving the kernel matrix K. :) Just my opinion.

[RichardBrautigan]

Great Video. Thank you. However, there is a mistake at 6:30, y_pred = w'*phi(x) (w was phi'*(K+lambda)^-1*y ). Hence, w' = y'*(K+lambda)^-1*phi and y_pred = y'(K+lambda)^-1*phi*phi(x). But you wrote phi'*phi(x) and it's a inner product! It is not Kernel. phi is a nxm matrix, m can be infinite (phi' * phi > mxm covariance matrix and phi*phi' > n*n Kernel matrix) We know Kernel matrix. It cannot be infinetexinfinete dimensions. It should be nxn matrix. There is also a problem with the notation you wrote at 3:32. If phi = [phi(x1)' ; phi(x2)'; ... ; phi(xn)' ]nxm then, phi(x)*phi(x)' can be Kernel matrix as phi(x)' = [phi(x1) phi(x2) ... phi(xn)] and phi(x)*phi(x)' = [phi(x1)'phi(x1) phi(x1)'phi(x2) ... phi(x1)'phi(xn); ; phi(xn)'phi(x1) phi(xn)phi(x2) ... phi(xn)phi(xn)]nxn. But you wrote m*n matrix at 3:32. This cannot be possible. if m is not equal to n, then it cannot be symmetric.

[mikel]

How to get the vector 'k' in the last slide?

[sathyakumarn]

Good speed in video. But I should say perhaps it is a bit too under-detailed. Maybe, one should be able to get through if he went through all your videos. I would request for more details in derivations in future videos.

[rajkundaliya]

Damn! Damn! Damn! Couldn't have been better. Thanks a lot! A lot! As simple as it can get!

[rishabtomar]

It would be a great help to understand this better if can you please make a video that takes a dataset as a m samples and n features and how would we calculate this K matrix and use this for transforming the features.

[meloyang]

Thank you a lot! It really soole my problems about kernal tricks as I felt extremely puzzled in our professor's lecture.

[rembautimes]

It is indeed an awesome video but viewers should have some background knowledge so that is easy to follow. What is nice is that ties in so many concepts in a single 7 min video. A good warm up video for those who have to go out and develop some code. I have resisted watching Code Emporium for a long time, now I'm a subscriber.

[ABC]

This video is awesome, thanks for the lecture!