FileStream: is buffer automatically flushed when full?

Question

In a WCF service I have a FileStream that is supposed to create a huge file.

A procedure within the WCF service regularly receives a chunk of bytes that must be written to the FileStream. The FileStream uses the default buffer size.

My procedure receives bytes in chunks with the same size as the size of the FileStream's buffer, except maybe for the last chunk. The procedure writes the complete chunk to the FileStream. Therefore after most writes the FileStream's buffer is expected to be full.

Question: Is a full buffer automatically flushed or should I Flush whenever I think that the buffer is full?

If full buffers are automatically flushed then my Flush would do more harm than good, because it would Flush immediately after the buffer is already flushed.

Code is as follows:

async Task PersistData(byte[] receivedData)
{
    // write all bytes in one write to the FileStream:
    await stream.WriteAsync(receivedData, 0, receivedData.Length);

    // This will have filled the buffer to the brim.
    // Should I Flush or did the FileStream already Flush?
    await stream.FlushAsync();
}

Consider this thought experiment - you decide to do the flush each time but you do it before each write rather than afterwards. You'd expect that to have approximately the same overall behaviour. And that's what will be happening when you receive the next chunk and if the buffer happens to still be full. So I'd expect adding the flush to have no overall effect. — Damien_The_Unbeliever

Rawitas Krungkaew Rawitas Krungkaew · Accepted Answer · 2018-01-24T08:47:10

WriteAsync methods always call flush for you according to the implementation

private FileStreamAsyncResult BeginWriteAsync(byte[] array, int offset, int numBytes, AsyncCallback userCallback, Object stateObject)
        {
            Contract.Assert(_isAsync);

            if (!CanWrite) __Error.WriteNotSupported();

            Contract.Assert((_readPos == 0 && _readLen == 0 && _writePos >= 0) || (_writePos == 0 && _readPos <= _readLen), "We're either reading or writing, but not both.");

            if (_isPipe)
            {
                // Pipes are ----ed up, at least when you have 2 different pipes
                // that you want to use simultaneously.  When redirecting stdout
                // & stderr with the Process class, it's easy to deadlock your
                // parent & child processes when doing writes 4K at a time.  The
                // OS appears to use a 4K buffer internally.  If you write to a
                // pipe that is full, you will block until someone read from 
                // that pipe.  If you try reading from an empty pipe and 
                // FileStream's BeginRead blocks waiting for data to fill it's 
                // internal buffer, you will be blocked.  In a case where a child
                // process writes to stdout & stderr while a parent process tries
                // reading from both, you can easily get into a deadlock here.
                // To avoid this deadlock, don't buffer when doing async IO on
                // pipes.   
                Contract.Assert(_readPos == 0 && _readLen == 0, "FileStream must not have buffered data here!  Pipes should be unidirectional.");

                if (_writePos > 0)
                    FlushWrite(false);

                return BeginWriteCore(array, offset, numBytes, userCallback, stateObject);
            }

            // Handle buffering.
            if (_writePos == 0)
            {
                if (_readPos < _readLen) FlushRead();
                _readPos = 0;
                _readLen = 0;
            }

            int n = _bufferSize - _writePos;
            if (numBytes <= n)
            {
                if (_writePos == 0) _buffer = new byte[_bufferSize];
                Buffer.InternalBlockCopy(array, offset, _buffer, _writePos, numBytes);
                _writePos += numBytes;

                // Return a synchronous FileStreamAsyncResult
                return FileStreamAsyncResult.CreateBufferedReadResult(numBytes, userCallback, stateObject, true);
            }

            if (_writePos > 0)
                FlushWrite(false);

            return BeginWriteCore(array, offset, numBytes, userCallback, stateObject);
        }
#endif // FEATURE_ASYNC_IO

FileStream: is buffer automatically flushed when full?

1 Answers