DataSnap XE2 and TStream method parameters

Question

I am working on DataSnap project in Delphi XE2 using TCP/IP protocol that needs to pass a stream of binary data to the server as a method parameter. The problem I am running into is that there seems to be a size limit of about 32 KB on the stream contents. Beyond this limit the stream received at the server is empty. If I pass additional method parameters they arrive intact so it seems to be an issue at the parameter level.

Here is how the DataSnap service class is declared:

  TDataSnapTestClient = class(TDSAdminClient)
  private
    FSendDataCommand: TDBXCommand;
  public
    constructor Create(ADBXConnection: TDBXConnection); overload;
    constructor Create(ADBXConnection: TDBXConnection; AInstanceOwner: Boolean); overload;
    destructor Destroy; override;
    procedure SendData(Data: TStream);
  end;

The approach I am using should work, at least according to the article by Jim Tierney. That said, there apparently have been changes since Delphi 2009 that have broken Jim Tierney's sample code.

DataSnap Server Method Stream Parameters

Any ideas on how to resolve this issue would be greatly appreciated.

I traced through the client code in the debugger and found the issue is in TDBXRowBuffer.WriteBytes. Looks like this is a known problem: qc.embarcadero.com/wc/qcmain.aspx?d=90995 — David Taylor
Do you see the problem only when you push data to the DS server, or when you pull data as well? I commonly pull several megs in a single byte stream from a DS server using TStream. I used Ethereal to watch the transferred packets, and it transfers the binary data in 32k chunks until all chunks are received. — James L.
FYI - Embarcadero QC 90995 has now been closed with the resolution "As Designed". The official and correct way to handle DataSnap stream parameters is to check the stream Size property. If the value is -1 then the stream needs to be read until the Read method returns a result of zero. Thanks to James L. for pointing out the solution to this problem. — David Taylor

James L. James L. · Accepted Answer · 2012-08-23T19:55:39

DataSnap transfers the data in 32k chunks. The receiving end has no way of knowing how many bytes will be received until after all chunks have been reassembled. Once all the data has been received, DataSnap doesn't set the size of the TStream that received the data, so you can't use it until you move it to another stream that knows how many bytes are in the stream.

I know that pulling 32k+ from a DataSnap server is not the same as pushing 32k+ to a DataSnap server, but this may work for you as well. Try running the TStream through this code after the DataSnap server finishes receiving the data:

procedure CopyStreamToMemoryStream(const ASource: TStream; var ADest: TMemoryStream; const Rewind: Boolean = True);
const
  LBufSize = $F000;
var
  LBuffer: PByte;
  LReadCount: Integer;
begin
  GetMem(LBuffer, LBufSize);
  ADest.Clear;
  try
    repeat
      LReadCount := ASource.Read(LBuffer^, LBufSize);
      if LReadCount > 0 then
        ADest.WriteBuffer(LBuffer^, LReadCount);
    until LReadCount < LBufSize;
  finally
    FreeMem(LBuffer, LBufSize);
  end;
  if Rewind then
    ADest.Seek(0, TSeekOrigin.soBeginning);
end;

I can't remember where I found this code (years ago), so I can't give credit where credit is due, but it has been working for me reliably for years now.

DataSnap XE2 and TStream method parameters

2 Answers