Saving FastText custom model binary with Gensim

Question

I am trying to save a custom FastText model trained with gensim. I want to save the binary files to have the possibility of training again the model, if it may.

The code to save the binary file is the next one:

from gensim.models.fasttext import save_facebook_model

save_facebook_model(model,'own_fasttext_model.bin')

But I am obtaining the next error in that same line:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-192-c9c2c41985af> in <module>
      2 from gensim.models.fasttext import save_facebook_model
      3 
----> 4 save_facebook_model(model,'own_fasttext_model.bin')

/opt/conda/lib/python3.7/site-packages/gensim/models/fasttext.py in save_facebook_model(model, path, encoding, lr_update_rate, word_ngrams)
   1334     """
   1335     fb_fasttext_parameters = {"lr_update_rate": lr_update_rate, "word_ngrams": word_ngrams}
-> 1336     gensim.models._fasttext_bin.save(model, path, fb_fasttext_parameters, encoding)

/opt/conda/lib/python3.7/site-packages/gensim/models/_fasttext_bin.py in save(model, fout, fb_fasttext_parameters, encoding)
    666     if isinstance(fout, str):
    667         with open(fout, "wb") as fout_stream:
--> 668             _save_to_stream(model, fout_stream, fb_fasttext_parameters, encoding)
    669     else:
    670         _save_to_stream(model, fout, fb_fasttext_parameters, encoding)

/opt/conda/lib/python3.7/site-packages/gensim/models/_fasttext_bin.py in _save_to_stream(model, fout, fb_fasttext_parameters, encoding)
    629 
    630     # Save words and ngrams vectors
--> 631     _input_save(fout, model)
    632     fout.write(struct.pack('@?', False))  # Save 'quot_', which is False for unsupervised models
    633 

/opt/conda/lib/python3.7/site-packages/gensim/models/_fasttext_bin.py in _input_save(fout, model)
    573 
    574     assert vocab_dim == ngrams_dim
--> 575     assert vocab_n == len(model.wv.vocab)
    576     assert ngrams_n == model.wv.bucket
    577 

AssertionError:

Any clue on what could be happening?

Thanks in advance.

gojomo gojomo · Accepted Answer · 2020-06-10T19:11:00

The .save_facebook_model() method is a brand-new feature. Such an AssertionError typically indicate ssome bug: the assertion has caught a contradiction, in the current state, compared to what was expected or required for the code to safely proceed.

If you can put together a small, self-contained set of steps that reliably create the error, you can report it to the project's bug tracker at:

https://github.com/RaRe-Technologies/gensim/issues

(In particular be sure to review & note in any report: how was that model object created/trained? Was it modified in any non-standard ways?)

Saving FastText custom model binary with Gensim

2 Answers