How to implement Logistic Regression from scratch with Python

In the third lesson of the Machine Learning from Scratch course, we will learn how to implement the Logistic Regression algorithm. It is quite similar to the Linear Regression implementation, just with an extra twist at the end.

Welcome to the Machine Learning from Scratch course by AssemblyAI.
Thanks to libraries like Scikit-learn we can use most ML algorithms with a couple of lines of code. But knowing how these algorithms work inside is very important. Implementing them hands-on is a great way to achieve this.

And mostly, they are easier than you’d think to implement.

In this course, we will learn how to implement these 10 algorithms.
We will quickly go through how the algorithms work and then implement them in Python using the help of NumPy.

▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬▬

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

#MachineLearning #DeepLearning

[josiahtettey]

Best concise video on logistic regression I have seen so far

[sreehari.s]

went through when I first started video editing, now it's taking a whole new switch and learning soft will only boost my courage for the

[ricardoprietoalvarez]

Great video, but my only dubt comes when J'() is calculated as the derivate of MSE and not as the derivate of the Cross Entropy, which is the loss function that we are using

[sarvariabhinav]

def sigmoid(x):
x = np.clip(x, -500, 500)
return (1/(1+np.exp(-x)))

To avoid overflow runtime error as the return statement can reach large values

[prodipsarker]

Studying CSE in GUB from Bangladesh, Love the way you teach the explanation & everything ; )

[OmarAmil]

need more algorithms, you are the best

[salonigandhi]

how is the derivative of loss function w.r.t weights same for cross entropy loss and MSE loss ?

[ronakverma]

Great work from @AssemblyAI 👍✨thank you from India.

7:51 Why you didn't multiplied by 2 the derivatives?

[akhan]

superb video! I am saying that because coding from scratch is important for me.

[ABCEE]

you are the best ever.. please keep this great work up ..

[DanielRamBeats]

Thanks for sharing this, I am doing something similar in JavaScript. The part about calculating the gradients for backpropagation is very helpful!

[rizzbod]

Wow, what a great video, very helpful

[jaredwilliam]

This was a great video, will there be one in the future that covers how to do this for multiple classes?

[karimshow]

We should maximize likelihood or minimize minus likelihood, I think the cost function is missing a minus, Am i right ?

[ABCEE]

would you please tell me what environment of python you use? and whether it is fast enough

[MOTIVAO]

This is amazing, thank you for this video.

[LouisDuran]

I have seen other videos where people use a ReLU function instead of sigmoid. Would this logistic regression algorithm be an appropriate place to use ReLU instead of Sigmoid? If not, why not?

[ryanliu]

The same problem of missing the 2 multiplication in calculating dw, db in the .fit() method:
dw = (1/n_samples) * np.dot(X.T, (y_pred-y)) * 2
db = (1/n_samples) * np.sum(y_pred-y) * 2
It does not affect too much, but we follow the slide to not be confusing

[mayankkathane]

why you have not used summation for dw for calculating error