Windows TCP Handshake Problem

Question

I am trying to create a TCP connection from an embedded controller to a Windows Vista server. I am writing the Windows server part of the application.

When the controller attempts to connect, it can take many attempts to establish the connection. I have used Wireshark to debug the problem and it appears that the Windows TCP stack is not following the correct handshake protocol.

Wireshark dump:

"No","Time","Source","Destination","Protocol","Info"

Try1:

"39","9.025322","10.0.0.252","10.0.0.92","TCP","49153 > xinuexpansion4 [SYN] Seq=0 Win=127 Len=0"
"40","9.025377","10.0.0.92","10.0.0.252","TCP","xinuexpansion4 > 49153 [ACK] Seq=1 Ack=1 Win=2048 Len=0"
"47","10.031750","10.0.0.252","10.0.0.92","TCP","49153 > xinuexpansion4 [RST] Seq=0 Win=127 Len=0"

Try 2:

"55","12.193941","10.0.0.252","10.0.0.92","TCP","49154 > xinuexpansion4 [SYN] Seq=0 Win=127 Len=0"
"56","12.194045","10.0.0.92","10.0.0.252","TCP","xinuexpansion4 > 49154 [ACK] Seq=1 Ack=1 Win=2048 Len=0"
"57","13.200431","10.0.0.252","10.0.0.92","TCP","49154 > xinuexpansion4 [RST] Seq=0 Win=127 Len=0"

Try 3:

"67","18.529871","10.0.0.252","10.0.0.92","TCP","49156 > xinuexpansion4 [SYN] Seq=0 Win=127 Len=0"
"68","18.529957","10.0.0.92","10.0.0.252","TCP","xinuexpansion4 > 49156 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460"
"69","18.536318","10.0.0.252","10.0.0.92","TCP","49156 > xinuexpansion4 [ACK] Seq=1 Ack=1 Win=127 Len=0"

10.0.0.252 is the controller initiating the connection, 10.0.0.92 is the Windows PC.

As I understand it, the correct sequence is SYN, SYN+ACK, SYN. What I get most of the time is SYN, ACK, RST (i.e. Windows is responding with ACK rather than SYN+ACK). In the above dump it shows 3 connection attempts, the 3rd one works.

Is there anything I can do to 'fix' Windows so that it responds correctly ?

EDIT - 2 packet captures

• shows the embedded controller taking 4 attempts to connect.
• shows the windows client connecting and disconnecting twice without problem.

Answer 1

I've looked at your pcap files, and the only differences I can see are:

(1) The Windows client is sending "random-looking" initial sequence numbers in their SYN packets, whereas the embedded client is sending initial sequence numbers like 1, 2, 3 in their SYN packets. As long as the server has absolutely no TCP connection state for the (source IP, source port, dest IP, dest port) 4-tuple, that shouldn't make any difference that I know of, but I wanted to mention it in case it helps you or someone else think of an idea.

(2) The Windows client is sending SYN packets with TCP options, whereas the embedded client is sending SYN packets with no TCP options. Again, as far as I know that should not cause the behavior you are seeing, and again, perhaps it will ring a bell for someone else.

If you want to see more details of packet headers and their decoding, I recommend tshark, which comes as part of the wireshark package. You can get lots of detail using a command line like this:

tshark -n -V -x -r Embeded-4-attempts.pcap > Embeded-4-attempts.txt

Regarding the "it takes 1 more attempt to connect" observation you make above, I will point out that it takes 1 more time for the client to reach a new initial sequence number that the Windows Vista server has not seen yet, because I'd bet that every time you reboot the client, it is starting out with sending a SYN packet with sequence number 1 again, followed by 2 on the next connection attempt, then 3, etc. The Windows Vista server may be waiting to see a new initial sequence number before it responds properly to the connection.

Hmm. Now that I think about it, perhaps the problem is the Vista server is not responding correctly to the RST packets from the client? If so, the server thinks all of these client connections are still active, while the client has no state associated with them. The server responds with ACKs to connection attempts that it still has state for, rather than SYN-ACKs, because it still has state for them. The client does not have state for them, and behaves as if they are new connections. Rebooting the client makes it go back to trying the same TCP source port and initial sequence numbers as it did last time it booted, because its simple TCP stack does not randomize these values like the Windows client probably is.

Anyway, food for thought. If you have access to source code for the simple TCP client, see if it can be made to use random source ports and initial sequence numbers when it makes new connection requests.

Answer 2

while the handshake is failing, this does follow the protocol. RST is allowed to reset the connection as part of the protocol. The question is, why is the reset ocurring? Is there a system between the two machines, whcih is sending the reset? If you run botht he server and the client on the same system, do you still get the reset (this would suggest a bug in your code)? if you run the server on a different OS, in the same network jack as the Windows server, do you see the RST?

Answer 3

Have you tried connecting to your Vista server from a client running a full OS like Windows or Linux, using say telnet? On Linux at least you can specify a TCP port number to connect to on the command line, and see whether that can make a connection to your Vista server or not.

One possibility to check: Is the Vista server running some kind of firewall that prevents the connection from occurring?