Data loss in Delphi using TIdTCPServer/TIdTCPClient components

Question

I'm writing a client-server application using TIdTCPClient/TIdTcpServer indy components in Delphi.

Data transfering usually works fine, but often I read wrong data from server; I get previous requests' answers, not current ones.

During debug, both apps are working locally, so there are no way for data to be lost during transfer.

Timeouts are 1000-3000 msec, this is far enough to avoid sending second requests before receiving answer on the first one.

I use simple data format: first 4 bytes is data packet length, the rest is binary data of that length.

The server-side code is (simplified for sending strings only; I also use binary buffers the same way, but this code is simpler to understand and check):

Var
  lng: LongInt;
  ib: TIdBytes;
begin
  // Prepare data to send:
  lng:=length(s);// s is an AnsiString to be sent
  SetLength(ib,lng+4);
  Move(lng,ib[0],4);
  Move(s[1],ib[4],length(s));
  // Send:
  AContext.Connection.IOHandler.WriteDirect(ib);
end;

The client-side code for sending request is the same (calling TIdTcpClient.IOHandler.WriteDirect() in last line). The client-side code for reading server answer is:

Var 
  ib: TIdBytes;
  size,done,lng: LongInt;
begin
  Result:=false;
  //  answer length:
  try
    SetLength(ib,0);
    tcp.IOHandler.ReadBytes(ib,4,false);
    Move(ib[0],size,4);
    if length(ib)<0 then Exit;// wrong data
  except
    on E: Exception do;// code skipped
  end;
  //  read answer body:
  done:=0;
  b.Clear;// b is my buffer, TStream descendant
  while done<size do
    begin
    lng:=Min(size-done,MaxBlockSize);
    // read:
    SetLength(ib,0);// to be sure
    tcp.IOHandler.ReadBytes(ib,lng,false);
    if length(ib)=0 then Exit;// error reading
    // append my buffer:
    b.Wr(ib[0],length(ib));
    // progress:
    Inc(done,length(ib));
    end;
end;

The data exchange order is:

1. Client sends request to server,

2. Server reads the request and sends answer back to client,

3. Client reads the answer.

Wrong data appears on step 3.

Maybe I'm doing something generally wrong?

I've tried to ReadBytes() right before sending request to server to clear the incoming buffer, but that doesn't helps too, as many other things I've tried...

Now I'm just out of ideas :(

Answer 1

Your I/O logic is much more complicated than it needs to be, especially on the client side. You are manually doing things that Indy can do for you automatically.

On the client side, since you are saving the data into a TStream, you can have Indy read the data into the TStream directly:

begin
  ...
  b.Clear;// b is my buffer, TStream descendant
  // ReadStream() can read a '<length><bytes>' formatted
  // message.  When its ASize parameter is -1 and its
  // AReadUntilDisconnect parameter is False, it reads
  // the first 4 or 8 bytes (depending on the LargeStream
  // property) and interprets them as the byte count,
  // in network byte order...
  tcp.IOHandler.RecvBufferSize := MaxBlockSize;
  tcp.IOHandler.LargeStream := False; // read 4-byte length
  // read answer:
  try
    tcp.IOHandler.ReadStream(b, -1, false);
  except
    on E: Exception do begin
      // the socket is now in an indeterminate state.
      // You do not know where the reading left off.
      // The only sensible thing to do is disconnect
      // and reconnect...
      tcp.Disconnect;
      ...
    end;
  end;
  ...
end;

On the server side, there are two different ways you can send a message that would be compatible with the above code:

var
  lng: LongInt;
  ib: TIdBytes;
begin
  // Prepare data to send:
  // s is an AnsiString to be sent
  lng := Length(s);
  SetLength(ib, lng);
  Move(PAnsiChar(s)^, PByte(ib)^, lng);
  // Send:
  AContext.Connection.IOHandler.Write(lng); // send 4-byte length, in network byte order
  AContext.Connection.IOHandler.Write(ib); // send bytes
end;

Or:

var
  strm: TIdMemoryBufferStream;
begin
  // Prepare data to send:
  // s is an AnsiString to be sent
  strm := TIdMemoryBufferStream.Create(PAnsiChar(s), Length(s));
  try
    // Send:
    // Write(TStream) can send a '<length><bytes>' formatted
    // message.  When its ASize parameter is 0, it sends the
    // entire stream, and when its AWriteByteCount parameter
    // is True, it first sends the byte count as 4 or 8 bytes
    // (depending on the LargeStream property), in network
    // byte order...
    AContext.Connection.IOHandler.LargeStream := False; // send 4-byte lengtb
    AContext.Connection.IOHandler.Write(strm, 0, True);
  finally
    strm.Free;
  end;
end;

As you can see, this code is sending the same type of messages you were originally sending, what changed is the code that manages the messages. Also, it is forcing the message byte count to be sent in network byte order whereas you were sending it in host byte order instead. Multi-byte integers should always be sent in network byte order when possible, for consistency and multi-platform compatibility.