How to check the capacity of a TCP send buffer to ensure data delivery

Question

I would like to add delivery confirmation to my TCP interface. A non-blocking write could populate the send buffer, but the data might never arrive if the connection fails - meanwhile write has reported that the data was received (by the local socket).

If I added extra acks to the stack, I could verify each piece of data was received. I think ftp does this. But I can't borrow anyone's code and would rather not implement ack/resend, particularly since TCP does most of this already.

I think it can be accomplished if I can verify that the socket send buffer is empty after each write(). If there's no more data, it should have all been delivered. I don't mind the latency of emptying the buffer.

Edit: I realize TCP/socket implementations differ by system, but if there's a Berkeley or Linux solution, it's probably available to me.

Edit: To address some suggestions, I would like to implement three levels of TCP interface, where I don't know how to accomplish the *'d one.

• Normal TCP with no assurance of delivery, for applications that like to live dangerously.
• *TCP that verifies receipt by a remote host, even if the packet was never delivered to the remote app. This reduces most of my connection risk (mobile client) and can be implemented completely on the client side. I think this can be implemented if I can verify the socket outgoing buffer is empty.
• Application layer ACK and resend implemented on top of TCP or UDP so the endpoints are verified.

Answer 1

Actually acking stuff at application level is the only way. Here are a few issues:

• When you send data, even if it leaves your kernel it's not done until your TCP receives an ACK about it. The API doesn't provide you with this information.

• The fact that your application received an ACK can mean:

  • The remote kernel (not the application) received the data. There's no telling if the remote process called recv. Maybe it crashes before getting a chance to recv.
  • Some proxy (MANY cheap devices do this) is talking to you and is using a separate connection with the true receiver. You received the ACK but you don't know if the remote system has it. And if the remote system has it, read the bullet above.

In conclusion:

• You don't have access to TCP ACKs
• Even if you would have acess to them, they would be useless

So you need an application-level ACK, but of course without retransmissions. The ACK should merely tell you "I received and processed your data".

EDIT

I see you persevere in your idea. Go look for SIOCOUTQ as described in a few Linux manual pages. Of course it's Linux-onlye, but see if it does what you need.

Answer 2

TCP ensures only OS-level delivery. It means, for example, that if your receiver application crashes unexpectedly in the middle of transmission, your sender can't be sure that all data is received -- consider when remote TCP stack sends ack about data received, but your application don't get a chance to process this data.

That's why you may need application-level acknowledgement scheme. For example, you may employ the same technique as TCP stack does -- use ack numbers, only this time on application level. If your remote app sends you ack number X, you may be sure that X data items are indeed received and processed by app, not by OS TCP stack.