Using requests in Google App Engine local development server

Question

I have an SDK for my web service that is distributed as a Python library via PyPI. My library uses requests for communicating with the backend using typical REST-like requests.

I would like my library to be compatible with applications that are hosted on Google App Engine (GAE). According to the GAE documentation on HTTP requests:

To use requests, you'll need to install both requests and requests-toolbelt using the vendoring instructions.

Once installed, use the requests_toolbelt.adapters.appengine module to configure requests to use URLFetch:

So I follow the example given there and have this in my library's main module:

  if os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine/'):
        import requests_toolbelt.adapters.appengine
        requests_toolbelt.adapters.appengine.monkeypatch()

This seems to do the trick when a client application using my library is actually running on an App Engine instance.

However, when the client application is run locally using the development web server (dev_appserver.py), os.getenv('SERVER_SOFTWARE') returns "Development/2.0" and so the monkeypatch is not executed. I subsequently get these errors when trying to issue requests:

ConnectionError: ('Connection aborted.', error(13, 'Permission denied'))

How can I detect that the host application for my library is running either in Google App Engine itself, or inside the development web server? Checking for "Development/2.0" doesn't seem like it would be discriminating enough.

Or, is there a better general pattern to follow when shipping a shared Python library that needs support for "typical" network requests?

Myk Willis Myk Willis · Accepted Answer · 2017-12-29T18:38:23

Digging through the Google Cloud SDK, it seems that Google's own method for determining if we are running in either the production or development (dev_appserver.py) Google App Engine environment is indeed to look for either of those values for SERVER_SOFTWARE. From apitools/base/py/util.py

def DetectGae():
    """Determine whether or not we're running on GAE.

    This is based on:
      https://developers.google.com/appengine/docs/python/#The_Environment

    Returns:
      True iff we're running on GAE.
    """
    server_software = os.environ.get('SERVER_SOFTWARE', '')
    return (server_software.startswith('Development/') or
            server_software.startswith('Google App Engine/'))

Using requests in Google App Engine local development server

1 Answers