Google App Engine "Exceeded soft memory limit on "

Question

We have a long-time running service on Google App Engine(GAE). However, after a while, we began to get the error in the image. No changes to the code. I don't understand why we're having the problem we didn't have before.

2020-01-05 08:31:32.704 UTC-8 Exceeded soft memory limit of 2048 MB with 2068 MB after servicing 0 requests total. Consider setting a larger instance class in app.yaml.

2020-01-05 08:31:32.705 UTC-8 This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application.

2020-01-05 08:31:32.705 UTC-8 The warmup request failed. Please check your warmup handler implementation and make sure it's working correctly.

Edit: System Info;

We are running python. I have cronjobs.

This means that there's a memory leak happening. Can you tell me a bit more about the systems you are using? Are you running Java/Python? If you are using ndb, loading quite a lot of entities in a single query fetch will cause the memory limit to exceed. Or you might be loading quite some chunks of data from a third party system. (Keeping all the data loaded in memory). — yaswanth
We are running python. Actually, I checked to see if there was too much data loading. I deleted the cache of the data taken from the database. I even set up another new service in a different structure. The same error always occurs. I can't find the source of this error that doesn't normally come. Could it be due to a new operation of Google? @yaswanth — sumeyyeozkaynak
@sumeyye as yaswanth said, it's a memory issue, you probably have a memory leak somewhere, not related to google. Carefully review your application with a profiler, check memory usage and try to understand what you can do to fix it or if you need an instance with bigger memory — Pievis
If you are using the "inbound_service: warmup" try remove it — Doug

Nibrass H Nibrass H · Accepted Answer · 2020-01-06T11:41:26

Do you know, how much memory it consumes when you run it locally?

With Stackdriver Profiler which is in beta, you can find what's happening with your memory usage.

When there is the Exceeded soft private memory limit error, you can follow two alternatives:

1) You can upgrade your instance to an another with more memory

2) You can reduce the chunks of data you process in each request, process the XML file in smaller pieces and keep the smaller instance doing the work.

Here is a similar post which can help.

Google App Engine "Exceeded soft memory limit on "

3 Answers