python - Stop processing Flask route if request aborted

36
votes

I have a flask REST endpoint that does some cpu-intensive image processing and takes a few seconds to return. Often, this endpoint gets called, then aborted by the client. In these situations I would like to cancel processing. How can I do this in flask?

In node.js, I would do something like:

req.on('close', function(){
  //some handler
});

I was expecting flask to have something similar, or a synchronous method (request.isClosed()) that I could check at certain points during my processing and return if it's closed, but I can't find one.

I thought about sending something to test that the connection is still open, and catching the exception if it fails, but it seems Flask buffers all outputs so the exception isn't thrown until the processing completes and tries to return the result:

An established connection was aborted by the software in your host machine

How can I cancel my processing half way through if the client aborts their request?

3
pythonhttprestflaskwsgi
What do you mean "aborted by the client"? Does the client attempt to close the connection? Or do they hit another endpoint? I don't think there is a way to do this using Flask, although you could certainly code your endpoints to allow for them to stop processing (using a database to keep track of these tasks and have the long-running task check that database for the stop flag, for example). - Mark Hildreth
They close the connection. I regenerate an image every time they change some options (checkboxes): if they check a new box, a request gets fired, if they immediately check a second box before the first request completes, the initial request gets cancelled (jqXHR.abort()) and a new one is made with the new options. I thought about firing an extra request from the client to write some "wascancelled" files / DB records (as you suggest), but that seems very ugly. PHP has connection_aborted(), nodejs has req.on('close'), I expected flask to have something similar. - Crashthatch
You may find this e-mail thread interesting. It discusses how a WSGI application could detect when a connection is closed. I believe it's similar to how connection_aborted() works in PHP: you cannot simply call it and determine if the connection was closed. Instead, you need to try to send data back first (which you might be able to pull off using streaming). - Mark Hildreth
I do not know Flask very well, so I can't help you determine when a client has closed the connection. However, if you can determine this, have you thought about doing the image processing in a separate process using multiprocessing. When a connection is closed you can call process.terminate() to end the process. - Michael David Watson

3 Answers

19
votes

There is a potentially... hacky solution to your problem. Flask has the ability to stream content back to the user via a generator. The hacky part would be streaming blank data as a check to see if the connection is still open and then when your content is finished the generator could produce the actual image. Your generator could check to see if processing is done and return None or "" or whatever if it's not finished. 

from flask import Response

@app.route('/image')
def generate_large_image():
    def generate():
        while True:
            if not processing_finished():
                yield ""
            else:
                yield get_image()
    return Response(generate(), mimetype='image/jpeg')

I don't know what exception you'll get if the client closes the connection but I'm willing to bet its error: [Errno 32] Broken pipe

1
votes

I was just attempting to do this same thing in a project and I found that with my stack of uWSGI and nginx that when a streaming response was interrupted on the client's end that the following errors occurred

SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request
uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 404] during GET
IOError: write error

and I could just use a regular old try and except like below

    try:
        for chunk in iter(process.stdout.readline, ''):
            yield chunk
        process.wait()
    except:
        app.logger.debug('client disconnected, killing process')
        process.terminate()
        process.wait()

This gave me:

  1. Instant streaming of data using Flask's generator functionality
  2. No zombie processes on cancelled connection
0
votes

As far as I know you can't know if a connection was closed by the client during the execution because the server is not testing if the connection is open during the execution. I know that you can create your custom request_handler in your Flask application for detecting if after the request is processed the connection was "dropped".

For example:

from flask import Flask
from time import sleep
from werkzeug.serving import WSGIRequestHandler


app = Flask(__name__)


class CustomRequestHandler(WSGIRequestHandler):

    def connection_dropped(self, error, environ=None):
        print 'dropped, but it is called at the end of the execution :('


@app.route("/")
def hello():
    for i in xrange(3):
        print i
        sleep(1)
    return "Hello World!"

if __name__ == "__main__":
    app.run(debug=True, request_handler=CustomRequestHandler)

Maybe you want to investigate a bit more and as your custom request_handler is created when a request comes you can create a thread in the __init__ that checks the status of the connection every second and when it detects that the connection is closed ( check this thread ) then stop the image processing. But I think this is a bit complicated :(.