Save naive bayes classifier nltk

Question

I am having trouble pickling a naive bayes classifier trained via nltk. Here is the code I am using to save the classifier:

pickledfile=open('my_classifier.pickle','wb')
pickle.dump(classifier,pickledfile)
pickledfile.close()

This seems to work ok. However, when I try and load the pickled file using the following code:

f1=open('my_classifier.pickle')
classifier=pickle.load(f1)
f1.close()

I get an EOF error. I got this code straight from this question and it doesnt work for whatever reason for me: Save Naive Bayes Trained Classifier in NLTK. Let me know if you know what is going on with this.

I had some trouble with this, too. Dumping and loading with the cPickle module worked for me. If it doesn't work either, try setting the protocol to 1: pickle.dump(classifier, pickledfile, 1) — Suzana
Yes, setting then protocol parameter fixes similar problems with MaxEnt models. — winwaed

Khma Khma · Accepted Answer · 2014-12-09T06:52:00

I don't have the environment setup to test out your code, but I have the feeling it's not right in the part where you save/load the pickle.

Referring to the Storing Taggers section of the NLTK book, I would change your code and do it like this:

def save_classifier(classifier):
   f = open('my_classifier.pickle', 'wb')
   pickle.dump(classifier, f, -1)
   f.close()

def load_classifier():
   f = open('my_classifier.pickle', 'rb')
   classifier = pickle.load(f)
   f.close()
   return classifier

Hope it helps.

From https://stackoverflow.com/posts/17635668/

Save naive bayes classifier nltk

1 Answers