Part of course:
Naive Bayes (and text classification)
- Deriving the algorithm
- Scikit-learn implementation
Naive Bayes is a widely used classification algorithm. It is a supervised learning algorithm based on Bayes’ Theorem. The word naive comes from the assumption of independence among features. That is, if our input vector is (x1, x2,...,xn), then xi's are conditionally independent given y.
Let's start with Bayes' theorem (for naive bayes, x is the input and y is the output):
When we have more than one feature, we can rewrite Bayes' theorem as:
Since we are making the assumption that xi's are conditionally independent given y, we can rewrite the above as
but we also know that P(x1, x2, .., xn) is a constant given the input, i.e.
There are several variants of naive bayes which use different distributions for P(xi|y) such as gaussian distribution (gaussian naive bayes), multinomial distribution (multinomial naive bayes) and bernoulli distribution (bernoulli naive bayes).
# we will use iris datasetfrom sklearn.datasets import load_irisfrom sklearn.naive_bayes import GaussianNBimport numpy as np# load the datasetdata = load_iris()model = GaussianNB()model.fit(data.data, data.target)# evalauteprint(model.score(data.data, data.target))# output = 0.96# predictmodel.predict([4.2, 3, 0.9, 2.1])# 0 = setosa,1 = versicolor, and 2 = virginica
Naive bayes is one of the simplest yet effective algorithms for