Spacy on GAE/standard/second/Python exceeds memory of largest instance

Question

I've been using GAE for awhile without any issues. Only recent change is that I added Spacy along with a model I trained.

When I run locally with the dev_appserver, the app consumes about 153 MB. After deployment, I get memory exceeded errors. Even with the F4_1G instance, I exceed the memory:

Exceeded hard memory limit of 1228 MB with 1280 MB after servicing 0 requests total. Consider setting a larger instance class in app.yaml.

The deployment works if I import Spacy and don't load my model (the instance has about 200MB) so Spacy itself isn't the problem, but when I load my model with spacy.load() the memory then exceeds the limit. Note that this happens before I even use my Spacy model so just loading the model causes the problem.

My Spacy model is a tagger and parser that takes up 27 MB on disk. I can't understand why the memory requirements would be so much larger on app engine than on my Mac.

Looks like others have been able to run Spacy on app engine. Any idea what I could be doing wrong?

If this is happening before using your model, when is it happening? When you're loading the model? When you're importing Spacy? — Dustin Ingram
@DustinIngram, great question. I updated my question to state that importing spacy is fine, but loading the model causes the large memory usage. — gaefan

gaefan gaefan · Accepted Answer · 2019-03-19T13:09:19

I was able to find a solution. I was loading my model into a module-level variable so when the module was imported, the model would be loaded.

When you deploy a second-gen GAE app, a bunch of worker threads get deployed (8 in my case). I don't understand the details of the worker threads, but I suspect that several of the worker threads import the module and that all of the worker threads contribute to memory usage.

I changed my code so that the model gets loaded on first use instead of at module import. With this change, the memory usage is 428MB.

Here is an example of what not to do:

import spacy

nlp = spacy.load('my_model')

def process_text(text):
    return nlp(text)

Instead do this:

import spacy

nlp = None

def process_text(text):
    global nlp
    if nlp is None:
        nlp = spacy.load('my_model')
    return nlp(text)

Spacy on GAE/standard/second/Python exceeds memory of largest instance

1 Answers