What is naive about Naive Bayes?
5 Answers
60
votes
There's actually a very good example on Wikipedia:
In simple terms, a naive Bayes classifier assumes that the presence (or absence) of a particular feature of a class is unrelated to the presence (or absence) of any other feature, given the class variable. For example, a fruit may be considered to be an apple if it is red, round, and about 4" in diameter. Even if these features depend on each other or upon the existence of the other features, a naive Bayes classifier considers all of these properties to independently contribute to the probability that this fruit is an apple.
Basically, it's "naive" because it makes assumptions that may or may not turn out to be correct.
13
votes
If your data is composed of a feature vector X = {x1, x2, ... x10} and your class labels y = {y1, y2, .. y5}, a Bayes classifier identifies the correct class label as the one that maximizes the following formula:
P(y|X) = P(X|y) * P(y) = P(x1,x2,...,x10|y) * P(y)
For now, it is still not naive. However, it is hard to calculate P(x1,x2,...,x10|y), so we assume the features to be independent, this is what we call the Naive assumption, hence, we end up with the following formula instead:
P(y|X) = P(x1|y) * P(x2|y) * ... * P(x10|y) * P(y)
6
votes
It's called naive because it makes the assumption that all attributes are independent of each other. This assumption is why it's called naive as in lots of real world situations this does not fit. Despite this the classifier works extremely well in lots of real world situations and has comparable performance to neutral networks and SVM's in certain cases (though not all).
1
votes
For classification when we find the joint distribution the problem is that it just mirrors the training data and is also very hard to compute. So we need something which generalizes more usefully.
The naive model generalizes strongly that each attribute is distributed independently of any other attributes.
It really helps in not caring about the dependency among the attributes to great extent.
