Tailor byte array size to NetworkStream response with protocol length included

Question

I have a custom binary protocol response I'm receiving from a TCP server in the following format:

Response Structure

Name Length Description
Header 2 bytes Header is a fixed value Hex 0x0978.
Status 1 byte A value of 0 is Success. A value other than 0 indicates an error. A full description of each possible error is described below.
Length 4 bytes Unsigned integer of total request length including all bytes in the request (server returns little endian UInt32)
Data Variable, 0 to 1,048,576 bytes Data sent from client to server to be encoded or decoded depending on the operation being requested.
Checksum 1 byte The checksum of bytes in the request from Header to Data (i.e. excluding checksum byte).

The problem I have is that the data is of variable size, so I don't know what size to make the byte array that the response is read into from the stream. How can I achieve this?

EDIT: I want the first 7 bytes to be also included with the data in the final byte array.

You can always loop until you've read the first 7 bytes, then read the rest. The joys of a stream-based protocol. Note that regardless of your actual array size, the stream can give you any number of bytes, as long as it will fit in the buffer, so your actual logic must handle piecemeal reading of a response regardless. — Jeroen Mostert
First read 7 bytes. Now you know the length, create array of this size and read the rest. — Evk
dont I have to read the first 7 bytes into an array though? and I also want the first 7 bytes to be included in the final array — Chris V.
You can use an array of 1048576 bytes (plus 8), but that's rather wasteful. Use a smaller, more reasonable size; if the data exceeds the buffer, allocate a new buffer and copy the data. Reading the first 7 bytes is no problem, as all Read overloads take an offset and a length. You can read the rest of the data right after the header, within the same array. Note that you'll have a much better time performance-wise if your code, too, can handle data with an array, an offset and a length (or an ArraySegment), rather than only working on "exact" arrays, which will involve a lot of copying. — Jeroen Mostert
how can you allocate a new buffer if you've already provided the buffer array via params? Would you use a stream.read for the first 7 bytes, giving the Read a offset of 0 and a size of 7, then create the new array [buffer] of the data size + 7, copy the original buffer to the new one, then do another Read passing the new buffer, and offset of 8 and a size of streamlength - 7? — Chris V.

Igors Sutovs Igors Sutovs · Accepted Answer · 2017-11-16T22:50:57

One possible solution:

class Program
{
    private static byte[] data = new byte[8]
    {
        // header
        0,
        0,

        // status
        1,

        // message size
        8,
        0,
        0,
        0,

        // data
        1
    };

    static byte[] Read(Stream stream)
    {
        const int headerLength = 7;
        const int sizePosition = 3;

        var buffer =  new byte[headerLength];
        stream.Read(buffer, 0, headerLength);

        // for BitConverter to work
        // the order of bytes in the array must 
        // reflect the endianness of the computer system's architecture
        var size = BitConverter.ToUInt32(buffer, sizePosition);

        var result = new byte[size];
        Array.Copy(buffer, result, headerLength);
        stream.Read(result, headerLength, (int)size - headerLength);

        return result;
    }

    static void Main(string[] args)
    {
        var stream = new MemoryStream(data);
        byte[] bytes = Read(stream);

        foreach (var b in bytes)
        {
            Console.WriteLine(b);
        }
    }
}

Tailor byte array size to NetworkStream response with protocol length included

1 Answers