How do I read exactly n bytes from a stream?

14
votes

This is a little more tricky than I first imagined. I'm trying to read n bytes from a stream.

The MSDN claims that Read does not have to return n bytes, it just must return at least 1 and up to n bytes, with 0 bytes being the special case of reaching the end of the stream.

Typically, I'm using something like

var buf = new byte[size];
var count = stream.Read (buf, 0, size);

if (count != size) {
    buf = buf.Take (count).ToArray ();
}

yield return buf;

I'm hoping for exactly size bytes but by spec FileStream would be allowed to return a large number of 1-byte chunks as well. This must be avoided.

One way to solve this would be to have 2 buffers, one for reading and one for collecting the chunks until we got the requested number of bytes. That's a little cumbersome though.

I also had a look at BinaryReader but its spec also does not clearly state that n bytes will be returned for sure.

To clarify: Of course, upon the end of the stream the returned number of bytes may be less than size - that's not a problem. I'm only talking about not receiving n bytes even though they are available in the stream.

3
c#.netstream
the BinaryReader.ReadBytes(int) returns the number of bytes requested; if the stream ends earlier, it will return what it read until that point (so less than requested). – Andrei Bozantan
@bosonix That would be convenient. Do you have a source for this information? – mafu
this is specified in the MSDN page msdn.microsoft.com/en-us/library/… and I also looked at the disassembled code. – Andrei Bozantan
@bosonix I see. It is stated rather unambigious there, and if the code matches, this seems to be the best solution. I am confused why I did not notice that method (apparently it was available at the time I asked this question), and even Marc Gravell did not suggest it. – mafu

3 Answers

16
votes

A slightly more readable version:

int offset = 0;
while (offset < count)
{
    int read = stream.Read(buffer, offset, count - offset);
    if (read == 0)
        throw new System.IO.EndOfStreamException();
    offset += read;
}

Or written as an extension method for the Stream class:

public static class StreamUtils
{
    public static byte[] ReadExactly(this System.IO.Stream stream, int count)
    {
        byte[] buffer = new byte[count];
        int offset = 0;
        while (offset < count)
        {
            int read = stream.Read(buffer, offset, count - offset);
            if (read == 0)
                throw new System.IO.EndOfStreamException();
            offset += read;
        }
        System.Diagnostics.Debug.Assert(offset == count);
        return buffer;
    }
}
14
votes

Simply; you loop;

int read, offset = 0;
while(leftToRead > 0 && (read = stream.Read(buf, offset, leftToRead)) > 0) {
    leftToRead -= read;
    offset += read;
}
if(leftToRead > 0) throw new EndOfStreamException(); // not enough!

After this, buf should have been populated with exactly the right amount of data from the stream, or will have thrown an EOF.

1
votes

Getting everything together from answers here I came up with the following solution. It relies on a source stream length. Works on .NET core 3.1

/// <summary>
/// Copy stream based on source stream length
/// </summary>
/// <param name="source"></param>
/// <param name="destination"></param>
/// <param name="bufferSize">
/// A value that is the largest multiple of 4096 and is still smaller than the LOH threshold (85K).
/// So the buffer is likely to be collected at Gen0, and it offers a significant improvement in Copy performance.
/// </param>
/// <returns></returns>
private async Task CopyStream(Stream source, Stream destination, int bufferSize = 81920)
{
    var buffer = new byte[bufferSize];
    var offset = 0;
    while (offset < source.Length)
    {
        var leftToRead = source.Length - offset;
        var lengthToRead = leftToRead - buffer.Length < 0 ? (int)(leftToRead) : buffer.Length;
        var read = await source.ReadAsync(buffer, 0, lengthToRead).ConfigureAwait(false);
        if (read == 0)
            break;
        await destination.WriteAsync(buffer, 0, lengthToRead).ConfigureAwait(false);
        offset += read;
    }
    destination.Seek(0, SeekOrigin.Begin);
}