Smoothing in python NLTK

4
votes

I am using Naive Bayes classifier in python for text classification. Is there any smoothing methods to avoid zero probability for unseen words in python NLTK? Thanks in advance!

1
pythonnltksmoothing

1 Answers

2
votes

I'd suggest to replace all the words with low (specially 1) frequency to <unseen>, then train the classifier in this data. For classifying you should query the model for <unseen> in the case of a word that is not in the training data.