When a non-blocking send() only transfers partial data, can we assume it would return EWOULDBLOCK the next call?

19
votes

Two cases are well-documented in the man pages for non-blocking sockets:

  • If send() returns the same length as the transfer buffer, the entire transfer finished successfully, and the socket may or may not be in a state of returning EAGAIN/EWOULDBLOCK the next call with >0 bytes to transfer.
  • If send() returns -1 and errno is EAGAIN/EWOULDBLOCK, none of the transfer finished, and the program needs to wait until the socket is ready for more data (EPOLLOUT in the epoll case).

What's not documented for nonblocking sockets is:

  • If send() returns a positive value smaller than the buffer size.

Is it safe to assume that the send() would return EAGAIN/EWOULDBLOCK on even one more byte of data? Or should a non-blocking program try to send() one more time to get a conclusive EAGAIN/EWOULDBLOCK? I'm worried about putting an EPOLLOUT watcher on the socket if it's not actually in a "would block" state to respond to it coming out of.

Obviously, the latter strategy (trying again to get something conclusive) has well-defined behavior, but it's more verbose and puts a hit on performance.

2
clinuxsocketsnonblocking
@Damon your edit completely changed the meaning of the question. - user207421
@EJP: The OP is obviously aware of EWOULDBLOCK (or how non-blocking sockets generally work, for the most part), so it's in my opinion a safe bet that the wording "would block" which seemed to confuse you is merely a bad wording, but not what's intended. - Damon
@Damon That isn't obvious to me at all. Clearly that is exactly what has confused the OP. Not me. That was the whole entire and complete point. By removing that from the question, you removed its entire meaning. Don't do that. If you want to answer the question, by all means do so. But don't just change it to suit yourself. - user207421
Damon is correct. I've updated the question to be more precise. I'm aware that nonblocking sockets never actually block, just return that they would. - David Timothy Strauss
You cannot assume anything. The nic card driver is probably asynchronous, your computer is probably asynchronous, etc... The send buffer could be drained by another core while your send was in process etc... - JimR

2 Answers

36
votes

A call to send has three possible outcomes:

  1. There is at least one byte available in the send buffer →send succeeds and returns the number of bytes accepted (possibly fewer than you asked for).
  2. The send buffer is completely full at the time you call send.
    →if the socket is blocking, send blocks
    →if the socket is non-blocking, send fails with EWOULDBLOCK/EAGAIN
  3. An error occurred (e.g. user pulled network cable, connection reset by peer) →send fails with another error

If the number of bytes accepted by send is smaller than the amount you asked for, then this consequently means that the send buffer is now completely full. However, this is purely circumstantial and non-authorative in respect of any future calls to send.
The information returned by send is merely a "snapshot" of the current state at the time you called send. By the time send has returned or by the time you call send again, this information may already be outdated. The network card might put a datagram on the wire while your program is inside send, or a nanosecond later, or at any other time -- there is no way of knowing. You'll know when the next call succeeds (or when it doesn't).

In other words, this does not imply that the next call to send will return EWOULDBLOCK/EAGAIN (or would block if the socket wasn't non-blocking). Trying until what you called "getting a conclusive EWOULDBLOCK" is the correct thing to do.

7
votes

If send() returns the same length as the transfer buffer, the entire transfer finished successfully, and the socket may or may not be in a blocking state.

No. The socket remains in the mode it was in: in this case, non-blocking mode, assumed below throughout.

If send() returns -1 and errno is EAGAIN/EWOULDBLOCK, none of the transfer finished, and the program needs to wait until the socket is isn't blocking anymore.

Until the send buffer isn't full any more. The socket remains in non-blocking mode.

If send() returns a positive value smaller than the buffer size.

There was only that much room in the socket send buffer.

Is it safe to assume that the send() would block on even one more byte of data?

It isn't 'safe' to 'assume [it] would block' at all. It won't. It's in non-blocking mode. EWOULDBLOCK means it would have blocked in blocking mode.

Or should a non-blocking program try to send() one more time to get a conclusive EAGAIN/EWOULDBLOCK?

That's up to you. The API works whichever you decide.

I'm worried about putting an EPOLLOUT watcher on the socket if it's not actually blocking on that.

It isn't 'blocking on that'. It isn't blocking on anything. It's in non-blocking mode. The send buffer got filled at that instant. It might be completely empty a moment later.

I don't see what you're worried about. If you have pending data and the last write didn't send it all, select for writability, and write when you get it. If such a write sends everything, don't select for writability next time.

Sockets are usually writable, unless their send buffer is full, so don't select for writability all the time, as you just get a spin loop.