Flask request.stream.read() is stopping when uploading file using SocketIO

Question

I am running a Flask production server using flask-socketio and eventlet and when trying to submit a form which contains a file to upload, Flask fails to read the entire request. This occurs when the file reaches above a few kb (around 50kb or more). The file I am trying to upload is a 60kb .txt file with a word on each line. Things work as expected with smaller file sizes of 1-2kb.

def get_file(request):
    if 'uploadFile' not in request.files:
        return redirect(request.url)
    return = request.file['uploadFile']

Having done some tests I was able to determine that the code does not get passed checking request.files. As an example, I would try to print(request.files) and the code would not move on from there, it would just hang.

I understand that Flask's built-in methods may not be the most effecient so I found a library streaming-form-data that can assist with loading large files and so implemented this as a replacement

def get_file(request):
    parser = StreamingFormDataParser(headers=request.headers)
    parser.register('file', FileTarget('/temp/file.txt')

    while True:
        chunk = request.stream.read(8192)
        if not chunk:
            break
        parser.data_received(chunk) # add read bytes to file
        socketio.sleep(1)

The file would still not finish uploading however I was able to find that it stops when trying to do chunk = request.stream.read(8192). This wouldn't happen straight away, it would usually stop around 4-5th iteration. Tested with and without socketio.sleep() as I understand this can be needed with tasks that require more time.

I did some more testing using the Flask Dev Server and found that if I use app.run(threaded=True) it would complete as expected and the code would continue. However I haven't been able to get it to work using socketio.run() and eventlet, example of my main.py

import eventlet
eventlet.monkey_patch()

from app import app, socketio

if __name__ == "__main__":
    # app.run(threaded=True) # This will work and finish loading the file
    socketio.run(app) # does not work

During the while loop to read the stream I do not receive any errors, it merely does not continue to to get the next chunk = request.stream.read(8192)

Meshi Meshi · Accepted Answer · 2020-09-01T15:43:20

Found the solution I needed for this and turns out it really wasn't down to Flask. In my POST I just needed to include chunking: true and it seems to be working fine from there.

E.g

$.ajax({
    type: 'POST',
    chunking: true,
    data: form_data,
    contentType: false,
});

As a test, I reverted the changes I made above (though these changes are probably for the better anyway) and confirmed it worked.

Flask request.stream.read() is stopping when uploading file using SocketIO

1 Answers