Naive base classifier of nltk giving unhashable type error

Question

Following is the code that I wrote using nltk and Python.

import nltk
import random
from nltk.corpus import movie_reviews
#from sklearn.naive_bayes import GaussianNB
documents = [(list(movie_reviews.words(fileid)), category)
    for category in movie_reviews.categories()
    for fileid in movie_reviews.fileids(category)]

random.shuffle(documents)

#print(documents[1:3]) 

all_words= []
for w in movie_reviews.words():
    all_words.append(w.lower())

all_words = nltk.FreqDist(all_words)
#print(all_words.most_common(15))
#print(all_words["great"])
word_features = list(all_words.keys())[:3000]

def find_features(document):
    words = set(document)
    features = {}
    for w in word_features:
        features[w] = {w in words}

    return features

#print((find_features(movie_reviews.words('neg/cv000_29416.txt'))))

featuresets = [(find_features(rev), category) for (rev, category) in documents]

training_set  = featuresets[:1900]
testing_set = featuresets[1900:]

classifier = nltk.NaiveBayesClassifier.train(training_set)
print("Naive Bayes Algo Accuracy percent:", (nltk.classify.accuracy(classifier, testing_set))*100)
classifier.show_most_informative_features(15)

# clf = GaussianNB()
# clf.fit(training_set)

I am getting this error

traceback (most recent call last): File "naive_bayes_application.py", line 37, in classifier = nltk.NaiveBayesClassifier.train(training_set) File "C:\Users\jshub\Anaconda3\lib\site-packages\nltk\classify\naivebayes.py", line 198, in train feature_freqdist[label, fname][fval] += 1 TypeError: unhashable type: 'set'

Please help.

The errors seems related to the type of training_set... Is it correct to pass a set to nltk.NaiveBayesClassifier.train() ? — araknoid
I am not sure. But in the above code, isn't the training_set a list? — blastoise
@shubhamjain, each element inside the training_set is a set. Change that to a list - [[find_features(rev), category] for (rev, category) in documents] — Clock Slave
its still showing the same error. I don't get it though. Don't we have to specifically use the word "set" to make something a set? — blastoise
Got it. made the following changes to find_features def find_features(document): words = set(document) features = {} for w in word_features: features[w] = (w in words) — blastoise

Akoffice Akoffice · Accepted Answer · 2018-03-16T18:06:24

Just in def find_features while constructing the features dictionary pass value in normal brackets.

example:

for w in word_features:
    features[w] = (w in words)

Naive base classifier of nltk giving unhashable type error

1 Answers