Packet Size ,Window Size and Socket Buffer In TCP

Question

After studying the "window size" concept, what I understood is that it keeps packet before sending over wire and till acknowledgement come for earliest packet . Once this gets filled up, subsequent packet will be dropped. Somewhere I also have read that TCP is a streaming protocol, and packet is what related to IP protocol at Network layer .

What I assumed till was that I have declared a Buffer (inside code) which I fill with some data and send this Buffer using socket. I declared a buffer of 10000 bytes and send it repeatedly using socket over 10 Gbps link .

I have following assumptions and questions. Please verify and help

1. If I want to send a packet of 64,256,512 etc. bytes, declared buffer inside code of that much space and send over socket. Each execution of send() command will send one packet of that much size .

2. So if I want to study the packet size variation effect on throughput, what do I have to do? Do I need to vary buffer size in code?

3. What are the socket buffer which we set using SO_SNDBUF and SO_RECVBUF? Google says it's buffer space for socket. Is it same as TCP window size or something different? Which parameter is more suitable to vary or to increase throughput?

Also there are three parameter in socket buffer: Min, Default and Max. Which one should I vary to my experiment and to get more relevance?

Answer 1

TCP does not directly expose a way to control the way packets are sent since it is a stream protocol. But you can make the TCP stack send packets by disabling the Nagle algorithm. That way all data that you send will be sent out immediately instead of being buffered. Data will be split into packets of MTU size which is like ~1400 bytes. Depends on the link.

To answer (2): Disable nagling and invoke send with buffers of < 1400 bytes. Use Wireshark to make sure you got what you wanted.

The buffer settings have nothing to do with any of this. I know of no valid reason to touch them.

In general this question is probably moot since you seem to want to send a lot of data. Just leave Nagling enabled and send big buffers (such as 64KB).

Answer 2

If I want to send a packet of 64,256,512 etc. bytes , Declared buffer inside code of that much space and send over socket .Each execution of send() command will send one packet of that much size.

Only if you disable the Nagle algorithm and the size is less than the path MTU. You mustn't rely on this.

So if I want to Study the Packet size variation effect on throughput, What I have to do , vary buffer space in Code?

No. Vary SO_RCVBUF at the receiver. This is the single biggest determinant of throughput, as it determines the maximum receive window.

what are the socket buffer which we set using SO_SNDBUF and SO_RCVBUF

Send buffer size at the sender, and receive buffer size at the receiver. In the kernel.

It's Same as TCP Window size

See above.

or else different ? Which parameter is more suitable to vary to increase throughput ?

See above.

Also there are three parameter in Socket Buffer min Default and Max . Which one should I vary for My experiment to get more relevance

None of them. These are the system-wide parameters. Just play with SO_SNDBUF and SO_RCVBUF for the specific sockets in your application.

Answer 3

I do some experience on Windows 10:

code from https://docs.python.org/3/library/socketserver.html#asynchronous-mixins,

RawCap for loopback capture,

WireShark for watching result.

The primary client code is:

def client(ip, port, message):
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.setsockopt(socket.SOL_SOCKET,socket.SO_RCVBUF, 100000)
    sock.connect((ip, port))
    sock.sendall(bytes(message, 'ascii'))
    response = str(sock.recv(1024), 'ascii')
    print("Received: {}".format(response))

Here is the result(the server port is 11111):

you can see, the tcp recive window size is the same as SO_RCVBUF, may it is platform indepent, you can verify it on other platform.

on https://msdn.microsoft.com/en-us/library/windows/hardware/ff570832(v=vs.85).aspx

The SO_RCVBUF socket option determines the size of a socket's receive buffer that is used by the underlying transport.

verified this.

Also, when I set SO_SNDBUF = 100000, it have no affects on the tcp transmission between client and server, as server just can discard data if client send much data one time.

So, if you want to change SO_RCVBUF to max Throughput, you can refer http://packetbomb.com/understanding-throughput-and-tcp-windows/, the os may offer func to detect ideal send backlog (ISB).