C++ int via socket send(): differnent value in TCP package on Linux / MacOS

Question

I have a Client/Server application written in C++. The client sends a structure (proxy_header) containing three int 32. This triggers the server to send information back. Problem is that the server doesn't reply if the client runs on OS X (El Capitan). Tcpdump reveals different values of the structure sent.

The C++ code is as follows:

Initialising the message to the server:

struct { int m:32; int x:32; int y:32; } proxy_header;
proxy_header.m = 0x02212102;
proxy_header.x = 1;
proxy_header.y = 1;

... Sending:

    {   int r;
if ((r = send(t->fd, (void*)&proxy_header, sizeof(proxy_header), 0)) != sizeof(proxy_header)) {
    error("Failed to send proxy header (%d != %d)", r, sizeof(proxy_header));
    goto error;
    }

... Waiting for answer - recv():

if ((r = recv(t->fd, (void*)&(proxy_answer), sizeof(proxy_answer), MSG_WAITALL)) != sizeof(proxy_answer)) {
  printf("Failed to receive ...")
    }
else {
  printf("Successfully received...");
}

The program compiles and works well on Linux. On MacOS I get "Failed to receive...".

tcpdumping the packages, I discovered the following:

On Ubuntu, I can see a SYN/ACK, then PSH from Client to Server which responds ACK, then PSH from Server to client which responds ACK.

The first push (from client) contains this:

char peer0_0[] = { /* Packet 4 */
0x02, 0x21, 0x21, 0x02, 0x01, 0x00, 0x00, 0x00, 
0x01, 0x00, 0x00, 0x00 };

On MacOS, the same happens up to the first PSH from the client which is ACK by the server. The server then just sends a FIN.

This is what the push contains:

char peer0_0[] = {
0x02, 0x21, 0x21, 0x02, 0x00, 0x00, 0x00, 0x01, 
0x00, 0x00, 0x00, 0x01 };

Apparently it is different... Why?

What is recv returning when it fails? If it returns -1, use perror() to see the reason for the error. — Barmar
I've used errno, apparently it doesn't fail but just does not receive anything (0 bytes). — Marcel Boldt
That's EOF, which means that the client closed the connection before sending anything. — Barmar
Yes, the socket shuts down orderly. On TCPDUMP I can also see the FIN package. The server just does not send data before finishing... which brought me to the question if the client sent the right init message. — Marcel Boldt
It might be helpful if you posted the tcpdump. Also, do you see any difference if you do the dump on both the client and server? That would indicate packets being lost. — Barmar

shrike shrike · Accepted Answer · 2016-04-30T21:49:31

It looks like a problem of endianness (reversed data bytes).

Considering what you get when sending from the MAC OS client, data format seams to be "big-endian", while the server likely uses "little-endian" format. (Is the computer running MAC OS a PowerPC ?)
https://en.wikipedia.org/wiki/Endianness

One way to fix this is to use, on both sides (client & server), data buffers independant of the platform endianness. You may use, for example, boost endian_buffers:
http://www.boost.org/doc/libs/1_58_0/libs/endian/doc/buffers.html

Your data structure may look like this:

struct {
    little_int32_buf_t m;
    little_int32_buf_t x;
    little_int32_buf_t y;
} proxy_header;

C++ int via socket send(): differnent value in TCP package on Linux / MacOS

1 Answers