Pre-trained FastText hyperparameters

0
votes

I'm using the pre-trained model:

import fasttext.util 
fasttext.util.download_model('en', if_exists='ignore') # English 
ft = fasttext.load_model('cc.en.300.bin')

Where can I find an exhaustive list of the values of the hyperparameters used to train the model? https://fasttext.cc/docs/en/options.html list the default values, that differ from the used one: for example, the dimension of the word vectors is 300 and not 100 (citing https://fasttext.cc/docs/en/crawl-vectors.html that doesn't list them all).

2
nlpfasttext

2 Answers

1
votes

From looking at the _FastText Python model class in Facebook's source...

https://github.com/facebookresearch/fastText/blob/a20c0d27cd0ee88a25ea0433b7f03038cd728459/python/fasttext_module/fasttext/FastText.py#L99

...it looks like, at least when creating a model, all the hyperparameters are added as attributes on the object.

Have you checked if that's the case on your loaded model? For example, does ft.dim report 300, and other parameters like ft.minCount report anything interesting?

Update: As that didn't seem to work, it also looks like the _FastText model wraps an internal instance of a native (not-in-Python) FastText model in its .f attribute. (See a few lines up from the source code I pointed to earlier.)

And that native-instance is set up by the module specified by fasttext_pybind.cc. That code looks like it specified a bunch of read-write class variable, associated with the metaparameters - see for example starting at:

https://github.com/facebookresearch/fastText/blob/a20c0d27cd0ee88a25ea0433b7f03038cd728459/python/fasttext_module/fasttext/pybind/fasttext_pybind.cc#L88

So: does ft.f.minCount or ft.f.dim return anything useful from a post-loaded model ft?

1
votes

Citing NVS Abhilash from https://github.com/facebookresearch/fastText/issues/887#issuecomment-649018188 the right code to write is:

args_obj = ft.f.getArgs()
for hparam in dir(args_obj):
    if not hparam.startswith('__'):
        print(f"{hparam} -> {getattr(args_obj, hparam)}")

This will print all the hyperparameters of the trained model!