Naive Bayes in Python - Machine Learning From Scratch 05 - Python Tutorial

Get my Free NumPy Handbook:

In this Machine Learning from Scratch Tutorial, we are going to implement the Naive Bayes algorithm, using only built-in Python modules and numpy. We will also learn about the concept and the math behind this popular ML algorithm.

~~~~~~~~~~~~~~ GREAT PLUGINS FOR YOUR CODE EDITOR ~~~~~~~~~~~~~~

📓 Notebooks available on Patreon:

If you enjoyed this video, please subscribe to the channel!

The code can be found here:

Further readings:

You can find me here:

#Python #MachineLearning

----------------------------------------------------------------------------------------------------------
* This is a sponsored link. By clicking on it you will not have any additional costs, instead you will support me and my project. Thank you so much for the support! 🙏

[patloeber]

There is a slight fix in the fit method that must be applied if class labels do not start at 0:
for idx, c in enumerate(self._classes)
instead of
for c in self._classes

[kougamishinya]

I love the way you explain what each line is doing and relate it back to the formulae, that's super helpful thank you!

[mattgoodman]

Thank you for this. I had no clue how to conceptually grasp Naive Bayes, but after watching your video I understand it very well

[heidycespedes]

Awesome explanation! It helped me to understand the concept and work on my project. Thanks a lot!

[dinarakhaydarova]

exactly what i needed! thank you bunchesss

[tkaczoro]

Looks like for the same reason you removed P(X) from formula for y, you can also remove the prior term P(y). You will get the same result in calculation of accuracy.

[FreshPL]

Great video, thanks! Just one thing - how we can avoid the zero-frequency problem in this implementation?

[andreaq.y]

very good tutorial !!! hope you will update more about algorithm implementations

[matthewcallinankeenan]

@PythonEngineer I'm using this on a large dataset with 8 columns and ~16000 rows. Its saying 'IndexError: index 10000 is out of bounds for axis 0 with size 210" Do you know how I can fix this?

[riber]

just... thank you !! for your help! ^^

[ГарикКубич]

Thank you so much friend, very helpfull

[vanshikajain]

In the second function predict, under the for loop, there is misplaced x which can be replaced by c in class conditional otherwise you get an exception of ValueError.

[akshaygoel]

Amazing implementation!

Small question/point - for the PDF shouldn't the numerator var have a square term? i.e. (2 * var**2)?

[posadzd]

Good video, learnt a lot, please can you implement Bayes-classifier based on parzen window density estimation?

[changsinlee]

A great tutorial and implementation. Just one correction on the implementation.

_pdf is implemented differently than the formula. It should be:
numerator = np.exp(- (x-mean)**2 / (2 * var**2))
denominator = np.sqrt(2 * np.pi * var**2)


The implemented code is missing the squared part.
numerator = np.exp(- (x-mean)**2 / (2 * var))
denominator = np.sqrt(2 * np.pi * var)

[godwingeorgethekkanath]

Great tutorial😍
It was useful for me.

[robertrey]

Hey man that was a great tutorial! I would just like to ask however, is there a way to know when you should use the Naive Bayes classifier?

[samii]

So i'm trying to run the algorithm for a dataset which have features for y_train first half 0 and second half 1.
The problem is that when im trying to get the predict for the first half of y_train im getting error of dividing with 0.
Is there anyway using laplace in the code help me???

[jossyrayonieram]

Hi. What do you mean by "classes" here. You mention classes "0" and "1", but still not sure what you meant or why they are called "classes".

[BlueSkyGoldSun]

Any book you recommend to learn ml in native python?