Write to the client returns EWOULDBLOCK when server is slow

2
votes

I am trying to write some data in the socket after specific time interval. I have two threads, one thread maintaining the TCP connection, and another thread generating data.

The data generating therad generates data, and writes it to a shared memory. The server threadreads data from the shared memory, and sends it to the client.

But, when the data generating thread becomes slow, when a lot of calculations involved, the server thread gets a EWOULDBLOCK error when trying to write to the client. But, surprisingly, from the client side, there is no such error.

If I am not wrong, EWOULDBLOCK error is returned when server is faster than client and the socket buffer is not completely read before it gets written again. But, here the case is totally opposite.

Can it be because the server therad is kept on sleep till the data generation thread completes( data thread has a higher priority).

Can somebody explain what might be happening here?

1
c++socketstcp

1 Answers

14
votes

EWOULDBLOCK is returned when you are using a non-blocking socket and there is not enough space in the kernel's outgoing-data-buffer for that socket to hold any of the data you want to send. That could happen if the client is reading too slowly, but it could also happen (even with a fast client) if the server tries to send a large amount of data at once; for example, if your calculation thread spend a long time calculating data, and then at the end of the calculation suddenly passed over 300,000 bytes of data at once, your server thread might do something like this:

  1. See that some data is available in the shared-memory area, time to start sending it!
  2. Call send() with len=300000. send() absorbs as much of that data as it can fit into the kernel's outgoing-socket-data-buffer (e.g. 131,072 bytes) and returns 131072 to indicate what it has done.
  3. Now your server thread sees that it still has 168928 bytes of data left to send, so it calls send() again with len=168928. At this point, the kernel's outgoing-socket-data-buffer is still completely full (because there has been no chance to send any packets yet), so send() returns EWOULDBLOCK.

In general, as long as you are using non-blocking sockets, EWOULDBLOCK is something your code will need to handle. The way I usually handle it is:

  1. Wait inside select() (or poll() or etc) until the socket returns ready-for-write
  2. When the select()/poll() indicates that the socket is ready-for-write, call send() on the socket until you've sent all the available data, OR until send() returned EWOULDBLOCK (whichever comes first).
  3. If you got EWOULDBLOCK in step 2, goto 1.

That way your send-thread will always feed the outgoing data to the kernel as fast as possible, but no faster -- i.e. you don't waste any CPU do busy-looping, but you also don't waste time by inserting any unnecessary delays into the process.