Support Vector Machine Fundamentals - Practical Machine Learning Tutorial with Python p.23

In this tutorial, we cover some more of the fundamentals of the Support Vector Machine.

[Anas-guqg]

If you're having a hard time following the math on this video, MIT OCW lecture about SVM is very helpful and does a great job explaining the mathematical concepts and formulas. Things should make sense after you watch this.
However, before delving into that, I advise you to learn the basics about Lagrange multiplier in order to understand the reason why and how it was put
to use in the calculations.
I put the two links to the videos below for you.
Links:

[jadetan]

im a uni student doing machine learning for my final year project and your videos have really helped me ALOT. More than my teachers could've taught me as this is an independent project. Thank you

[austinhaider]

There's a knock on the door... it's Lagrange and he brought his multipliers 😂

[prakharmishra]

I believe SVM needs a better mathematical explanation to be understood properly. If any of you guys need further help with SVM or want to revisit the math, I would suggest you check out Professor Patrick Winston's SVM lecture on the MIT Open Courseware YouTube channel. Its pretty helpful.

[JacobSmithodc]

If you're not lost or confused . . . you're a robot.

[AladdinPersson]

I find your explanation pretty good, but I don't understand how you can jump to Lagrange multipliers and not even mention that most of you probably won't understand this. I mean you had a video when you explained what norm of a vector and the dot product is, giving the idea that you can follow these videos without knowledge of calculus / linear algebra

[kingking-bvvh]

I took 1 hour to figure out a 15 minutes tutorial, finally

[parias]

I actually got pretty happy when you said I'm a robot lol. I've a bachelor in physics btw. Usually I've hard times with the code

[CoolDude]

You assume there is an equal number of of positive instances to negative instances around 4:00.


This works out in the Lagrangian of the problem as we can derive that there is equal total "cost" or importance placed on each class. That is to say the bias is selected so that sum y_i * alpha_i = 0. w is selected so that alpha_i * (( x_i * w + b ) y_i - 1 ) = 0) (*).


We minimise w so that we get the smallest possible scaling of the axis ( x_i * w + b ) while maintaining a fixed separation between the axes of 1. It is equivalent to ask for a large separation but keep ||w|| fixed. Which is basically what (*) says.

[leshamokhov]

If you're confused go check out MIT OCW lecture about SVM. It's the same math formulas covered in MIT lecture, but it is more clear, I can say.

[TheOlderSoldier]

Can you think of what model would be best for predicting the next answer in a coin toss that is generated by a basic PRNG? I get that this wouldn't be possible with a true coin toss as each toss has no bearing on the previous, but PRNGs might show a pattern over time. I have data from 3000 sequential coin tosses.

[mini_frank]

if you guys find the math part confusing, here's a link that helped me, just remember dot product is the projection of the first vector onto the unit length second vector, norm(a)*norm(b)*cos(theta):

[meetmerchant]

he had me at "we, the scientist"

[aravamuthanl]

@sentdex, Great video really helped me at the exact time

[thesilenthero]

Everything after 6:40 flew over my head, moved too fast. These principles deserve their own video.

[SanketPatole]

Isn't the projection of vector u on vector w = u.w/||w|| ?
Why did you consider only the dot product?

If we consider ||w|| also in the equation, Value of width comes to be exactly 2 (which is actually true since we have assumed length of support vector from decision boundary is exactly one).
How can we find extremes of this width (2) ?

Am I interpreting something wrong?
Please help.


Also why did you take sum of constraint function instead of just the constraint function in the Lagrangian?

[vdosaahab]

though I was following step by step but seems i missed somewhere and now its all over. Can I skip this and come back later to this portion

[sanjivgautam]

b is never maximized. Correct me if I am wrong, but we use Langrange so that our objective function (which is to maximize the margin (m), which is same as minimizing w), with our constraint. So W and b just play the role of being parameters for the hyperplane. So our job is to find W and b, keeping the constraints. So when we train them, we have W and b, and for any unknown, if W.Unknown + b >=0, we say it is positive and for <=0, we say it is negative. I am not sure why you said b is something we need to maximize, but I see it and W being just the parameters for hyperplane.

[juleshenri]

A little too slow sometimes so it gets really boring, and in contrast sometimes too quick and without enough mathematical explanations. But still helpful :)

[nevermore]

5:38 Can anyone explain why would you want to minimize 1/2||W||^2? And how did he derive that? He just kind of mentioned it like it was trivial...