I have a server with multiple clients. The simulated network is in heavy congestion. What I found is that the server reset some TCP connections after received the ACK segment of three-way handshake. But it doesn't happen when the network is in good condition.
What I found is that the ACK of three-way handshake is received about 3.5s later than the SYN-ACK. Is that because the three-way handshake SYN-ACK time-out? If SYN-ACK time out, why not resend SYN-ACK.
Thank you for any suggestions.
This looks like related to SYN cookies.
When a Linux host receives too much SYN traffic, it activates the SYN cookies mechanism.
When SYN cookies is enabled, a server answers to SYN by issuing a SYN-ACK segment with specific data encoded in the TCP sequence field. In that field it encodes the timestamp, the MSS and a cryptographic hash of the two endpoints (local and remote IPs and ports) plus the timestamp.
This is done so that the server does not have to store anything about the connection at this point, it simply send the answer and forget about it.
Then, when the client answer with its ACK, the server checks the hash in the ack field (the ack of the client is the sequence of the server). If it is correct, it creates the connection with the data stored in the field.
SYN cookies explain why the server does not resend SYN-ACK packets when they timeout.
But, why the reset after receiving the ACK?
Maybe clients (or server) are behind a NAT that modifies ports and the NAT also gets congested, so that it cannot link the final ACK to the previous SYN, and assigns a new source port. When the server receives it, it resets the connection (it does not matter if SYN cookies are enabled or not).
Or maybe the server process is not accepting connections at the same speed they are arriving, the kernel queue has filled and newer ones are discarded that way.
