python - How can I add a background thread to flask?

Question

I'm busy writing a small game server to try out flask. The game exposes an API via REST to users. It's easy for users to perform actions and query data, however I'd like to service the "game world" outside the app.run() loop to update game entities, etc. Given that Flask is so cleanly implemented, I'd like to see if there's a Flask way to do this.

You mean something like Flask-Admin? Or if you are using an ORM (SQL-Alchemy), then you can just create a new db session to query the database even if the application is running. — reptilicus
It looks like there's a hackish way to do it, but I don't think this is technically supported. I also found this answer, which talks about using flask-celery for this. — girasquid
If you actually need to do a lot of computation, you might want to use the subprocess module, and simply spawn new processes to do that additional computation. — Maus
@girasquid Agreed, celery or some other task queue system is ideal for this sort of thing - you generally have less control over threads or sub-processes (since the parent process may be reaped by the server without notice). — Sean Vieira
That is a plan, however the sub process will be manipulating data structures, that you'd like to access and set via the exposed flask api. Will I not run into problems? — Marinus

caio caio · Accepted Answer · 2014-04-06T21:27:07

Your additional threads must be initiated from the same app that is called by the WSGI server.

The example below creates a background thread that executes every 5 seconds and manipulates data structures that are also available to Flask routed functions.

import threading
import atexit
from flask import Flask

POOL_TIME = 5 #Seconds

# variables that are accessible from anywhere
commonDataStruct = {}
# lock to control access to variable
dataLock = threading.Lock()
# thread handler
yourThread = threading.Thread()

def create_app():
    app = Flask(__name__)

    def interrupt():
        global yourThread
        yourThread.cancel()

    def doStuff():
        global commonDataStruct
        global yourThread
        with dataLock:
        # Do your stuff with commonDataStruct Here

        # Set the next thread to happen
        yourThread = threading.Timer(POOL_TIME, doStuff, ())
        yourThread.start()   

    def doStuffStart():
        # Do initialisation stuff here
        global yourThread
        # Create your thread
        yourThread = threading.Timer(POOL_TIME, doStuff, ())
        yourThread.start()

    # Initiate
    doStuffStart()
    # When you kill Flask (SIGTERM), clear the trigger for the next thread
    atexit.register(interrupt)
    return app

app = create_app()

Call it from Gunicorn with something like this:

gunicorn -b 0.0.0.0:5000 --log-config log.conf --pid=app.pid myfile:app

python - How can I add a background thread to flask?

3 Answers