Why Delphi TMemoryStream Read TBytes and Array Of Bytes is different?

Question

I am writing a procedure to do Z85Encoding of a bytes sequences using Delphi 2010. So I wrote a function:

function Z85Encode(input, output: TStream): integer

and expect to pass in a Stream. Next I wrote an overloaded function

function Z85Encode(b: TBytes): string;

which will write the byte sequence into a TBytesStream (same to TMemoryStream), call the encode function, then read the encoding data to the Result string.

Problem was I found the TStream.Read behave very different from the documentation and I cannot understand it. Thou I can use other way to finish the function, but I don't quite understand why? Which may me wonder how Delphi implement the TBytes type.

To illustrate my question, I wrote the following test procedure:

procedure Test;
var
  iStream: TMemoryStream;
  n: integer;
  a: TBytes;
  b: TBytes
  c: array [0..4] of Byte;
begin
  b := TEncoding.UTF8.GetBytes('abcdefghij');  // byte from 97 to 106
  iStream := TMemoryStream.Create;
  try
    iStream.Write(b, Length(b));
    iStream.Seek(0, soBeginning);
    n := iStream.Read(a, 5); // expect only 5 bytes read, but the whole array is back.
    ShowMessage(IntToStr(n));  // n is 5
    ShowMessage(IntToStr(Length(a)));  // length is 10!!
    iStream.Seek(0, soBeginning);
    n := iStream.Read(c, 5);   // c contains random number, not 97, 98, ...
    ShowMessage(IntToStr(n));  // n is 5
  finally
    iStream.Free;
  end;
end;

First question, in according to the document, Read should only read count of byte. But if I pass in a TBytes varaible, it read the entire byte array but the return read count is 5. I also wonder why I did not need to allocate memory to the variable a first. It seems the Stream.Read will allocate memory for the variable, which is no expected.

Second question, when TStream.Read(c, 5) where c is an array [0..4] of byte. The value in the c array is some random value, not 97, 98, 99, 100, and 101.

With further testing, if I change the TStream.Write to

for n := 0 to High(b) do begin
  iStream.Write(b[n], 1);
end;

Then Read(a, 5) will not get any result, Length(a) will get access violation, yet Read(c, 5) will get the correct result.

It seems both Read and Write take an open type parameter and behave differently in according to the parameter type. Behave differently can be understandable if eventually it is to read or write the byte value of the variable, but it seems it do more then just determine the byte content from the variable. Then the Read and Write are expected to use the same variable type and was not my expect.

At least in Delphi2010 the iStream.Write(b, Length(b)); would cause troubles. Instead use iStream.Write(b[0], Length(b)); . The reason is the if you write b the data in the (hidden) dynamic array record is written instead of the data. — mrabat
@mrabat Note that in XE3 and later, there are overloads of Write() that take a TBytes as input. — Remy Lebeau
@mrabat not really. A lot of the new stream overloads are actually confusing and initially very badly implemented with pointless heap allocations. — David Heffernan

David Heffernan David Heffernan · Accepted Answer · 2018-04-17T07:08:00

You aren't writing the content of the array. You are writing the address of the array. A dynamic array is a pointer. That pointer's value is what you are writing.

To write the array you must do

iStream.Write(b[0], Length(b));

Or

iStream.Write(Pointer(b)^, Length(b));

I prefer the latter because it works for length 0 arrays even when range checking is enabled.

Do likewise to read from the array. Note that it is your responsibility to allocate the read buffer first. You didn't do that.

SetLength(a, 5);
iStream.Read(Pointer(a)^, Length(a));

Finally, it is probably preferable to use WriteBuffer and ReadBuffer since they wrap up error checking that the requested number of bytes are actually transferred.

Why Delphi TMemoryStream Read TBytes and Array Of Bytes is different?

1 Answers