How to keep Werkzeug from crashing after NGINX closes pipe?

0
votes

I have a Werkzeug server running behind NGINX. When a client disconnects while waiting for the Werkzeug server to respond, NGINX closes the pipe to Werkzeug. When the python program writes the response to Werkzeug, the following exception occurs and Werkzeug crashes:

Traceback (most recent call last):
File "server.py", line 81, in
app.run(host=args.host, port=args.port, debug=False)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 843, in run
run_simple(host, port, self, **options)
File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 694, in run_simple
inner()
File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 659, in inner
srv.serve_forever()
File "/usr/local/lib/python2.7/dist-packages/werkzeug/serving.py", line 499, in serve_forever
HTTPServer.serve_forever(self)
File "/usr/lib/python2.7/SocketServer.py", line 238, in serve_forever
self._handle_request_noblock()
File "/usr/lib/python2.7/SocketServer.py", line 297, in _handle_request_noblock
self.handle_error(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 295, in _handle_request_noblock
self.process_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 321, in process_request
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.7/SocketServer.py", line 651, in init
self.finish()
File "/usr/lib/python2.7/SocketServer.py", line 710, in finish
self.wfile.close()
File "/usr/lib/python2.7/socket.py", line 279, in close
self.flush()
File "/usr/lib/python2.7/socket.py", line 303, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
socket.error: [Errno 32] Broken pipe

Is there some configuration option I'm missing to keep it from crashing? Normally all exceptions are caught and a 500 error returned, with the server remaining alive.

1
pythonflaskwerkzeug

1 Answers

0
votes

As far as I can guess from the debug trace and without reading the source code, you are probably closing the socket prematurely.

Your server process has received a SIGPIPE writing to a socket. This usually happens when you write to a socket fully closed on the other (client) side. This might be happening when a client program doesn't wait till all the data from the server is received and simply closes a socket (using close function).

In a C program you would normally try setting to ignore SIGPIPE signal or setting a dummy signal handler for it. In this case a simple error will be returned when writing to a closed socket. In your case a python seems to throw an exception that can be handled as a premature disconnect of the client.

How to prevent errno 32 broken pipe?