Failed to allocate a managed memory buffer of 268435456 bytes. The amount of available memory may be low

Question

I am trying to save some big data to database through a WCF service call. I am not able to invoke the service. Its throwing an error:

Failed to allocate a managed memory buffer of 268435456 bytes. The amount of available memory may be low

public async Task<int> UploadExportPackage(DTO.Upload.UploadPackage package)
{
    int result = await serviceProxy.UploadResultsAsync(package);
    return result;
}

Server side config:

    <!-- 
      Place limits on the amount of data that can be received in one go.
      This helps to protect the server from attack and abuse.
    -->
    <binding name="primaryBinding"
             maxReceivedMessageSize="6000000"
             maxBufferSize="6000000"
             maxBufferPoolSize="12000000">

      <readerQuotas maxArrayLength="6000000"
                    maxStringContentLength="6000000" />

      <!--
          Set the security mode. Apply at:
            - Transport layer
            - Message layer
      -->
      <security mode="TransportWithMessageCredential">
        <message clientCredentialType="UserName" />
      </security>
    </binding>
  </basicHttpsBinding>
</bindings>

Stack trace:

Exception no: 0

Next Exception: Message:

Failed to allocate a managed memory buffer of 268435456 bytes. The amount of available memory may be low.

Stack trace: Server stack trace: at System.Runtime.Fx.AllocateByteArray(Int32 size) at System.Runtime.InternalBufferManager.PooledBufferManager.TakeBuffer(Int32 bufferSize) at System.ServiceModel.Channels.BufferManagerOutputStream.Init(Int32 initialSize, Int32 maxSizeQuota, Int32 effectiveMaxSize, BufferManager bufferManager) at System.ServiceModel.Channels.BufferedMessageWriter.WriteMessage(Message message, BufferManager bufferManager, Int32 initialOffset, Int32 maxSizeQuota) at System.ServiceModel.Channels.TextMessageEncoderFactory.TextMessageEncoder.WriteMessage(Message message, Int32 maxMessageSize, BufferManager bufferManager, Int32 messageOffset) at System.ServiceModel.Channels.HttpOutput.SerializeBufferedMessage(Message message, Boolean shouldRecycleBuffer) at System.ServiceModel.Channels.HttpOutput.SendAsyncResult.SendWithoutChannelBindingToken() at System.ServiceModel.Channels.HttpOutput.SendAsyncResult.Send() at System.ServiceModel.Channels.HttpOutput.SendAsyncResult..ctor(HttpOutput httpOutput, HttpResponseMessage httpResponseMessage, Boolean suppressEntityBody, TimeSpan timeout, AsyncCallback callback, Object state) at System.ServiceModel.Channels.HttpOutput.BeginSendCore(HttpResponseMessage httpResponseMessage, TimeSpan timeout, AsyncCallback callback, Object state) at System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.SendWebRequest() at System.ServiceModel.Channels.HttpChannelFactory1.HttpRequestChannel.HttpChannelAsyncRequest.BeginSendRequest(Message message, TimeSpan timeout) at System.ServiceModel.Channels.RequestChannel.BeginRequest(Message message, TimeSpan timeout, AsyncCallback callback, Object state) at System.ServiceModel.Channels.SecurityChannelFactory1.RequestChannelSendAsyncResult.BeginSendCore(IRequestChannel channel, Message message, TimeSpan timeout, AsyncCallback callback, Object state) at System.ServiceModel.Security.ApplySecurityAndSendAsyncResult1.OnSecureOutgoingMessageComplete(Message message) at System.ServiceModel.Security.ApplySecurityAndSendAsyncResult1.Begin(Message message, SecurityProtocolCorrelationState correlationState) at System.ServiceModel.Channels.SecurityChannelFactory1.SecurityRequestChannel.BeginRequest(Message message, TimeSpan timeout, AsyncCallback callback, Object state) at System.ServiceModel.Dispatcher.RequestChannelBinder.BeginRequest(Message message, TimeSpan timeout, AsyncCallback callback, Object state) at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.StartSend(Boolean completedSynchronously) at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.FinishEnsureOpen(IAsyncResult result, Boolean completedSynchronously) at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.StartEnsureOpen(Boolean completedSynchronously) at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.FinishEnsureInteractiveInit(IAsyncResult result, Boolean completedSynchronously) at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.StartEnsureInteractiveInit() at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.Begin() at System.ServiceModel.Channels.ServiceChannel.BeginCall(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, TimeSpan timeout, AsyncCallback callback, Object asyncState) at System.ServiceModel.Channels.ServiceChannel.BeginCall(ServiceChannel channel, ProxyOperationRuntime operation, Object[] ins, AsyncCallback callback, Object asyncState) at System.Threading.Tasks.TaskFactory1.FromAsyncImpl[TArg1,TArg2,TArg3](Func6 beginMethod, Func2 endFunction, Action1 endAction, TArg1 arg1, TArg2 arg2, TArg3 arg3, Object state, TaskCreationOptions creationOptions) at System.ServiceModel.Channels.ServiceChannelProxy.TaskCreator.CreateGenericTask[T](ServiceChannel channel, ProxyOperationRuntime operation, Object[] inputParameters) at System.ServiceModel.Channels.ServiceChannelProxy.TaskCreator.CreateGenericTask(ServiceChannel channel, ProxyOperationRuntime operation, Object[] inputParameters) at System.ServiceModel.Channels.ServiceChannelProxy.TaskCreator.CreateTask(ServiceChannel channel, IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.InvokeTaskService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]: at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) at RossendaleClient8.DAL.RR.IRossendaleService.UploadResultsAsync(UploadPackage package) at RossendaleClient8.DAL.RR.RossendaleServiceClient.UploadResultsAsync(UploadPackage package) at RossendaleClient8.DAL.RossendaleServiceClient.d__13.MoveNext()

Next Exception: Message:

Exception of type 'System.OutOfMemoryException' was thrown.

Stack trace: at System.Runtime.Fx.AllocateByteArray(Int32 size)

You may need to partition your data into smaller chunks and do multiple sends to the service. — Kevin
@Kevin is there no way to send the whole lot? It's only about 200mb — user1800674
Maybe add memory to the server. Don't think this will really fix the problem though. Just pushes out how long until it happens. — Kevin
The problem is that the system has to allocate big chunks of memory repeatedly, and is running the server low on available memory before the GC can collect and free up all of that memory. — Kevin

Vladimir Vladimir · Accepted Answer · 2016-08-04T14:16:58

If your binding allows, try using TransferMode.Streamed. You can also have a look at BinaryMessageEncoding. From MSDN:

Buffered transfers hold the entire message in a memory buffer until the transfer is complete.

Streamed transfers only buffer the message headers and expose the message body as a stream, from which smaller portions can be read at a time.

The default TransferMode is Buffered, so the whole message is loaded to memory. If you turn on Streamed TransferMode, you can manage the size of memory usage by editing value of MaxBufferSize.

BinaryMessageEncoding, in its turn, allows you to compress data, you send over network. Here is a sample App.Config contents

 <customBinding>
    <binding name="primaryBinding" openTimeout="00:01:00"  closeTimeout="00:01:00" 
              sendTimeout="00:30:00" receiveTimeout="00:30:00">
      <binaryMessageEncoding compressionFormat="GZip">
        <readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647"
                      maxArrayLength="2147483647" maxBytesPerRead="2147483647"
                      maxNameTableCharCount="2147483647" />
      </binaryMessageEncoding>
      <httpsTransport transferMode="Streamed" maxReceivedMessageSize="6000000" 
                      maxBufferSize="6000000" maxBufferPoolSize="12000000" />          
    </binding>      
  </customBinding>

Same configuration in code

 private Binding GetBinding()
    {
        var config = new BindingElementCollection();            
        config.Add(new BinaryMessageEncodingBindingElement()
        {
            CompressionFormat = CompressionFormat.GZip,
        });
        config.Add(new HttpsTransportBindingElement()
        {
            MaxBufferPoolSize = 12000000,
            MaxBufferSize = 6000000,
            MaxReceivedMessageSize = 6000000,
            TransferMode = TransferMode.Streamed,
        });

        var resultBinding = new CustomBinding(config)
        {
            OpenTimeout = TimeSpan.FromMinutes(1),
            CloseTimeout = TimeSpan.FromMinutes(1),
            ReceiveTimeout = TimeSpan.FromMinutes(1),
            SendTimeout = TimeSpan.FromMinutes(1),
        };

        return resultBinding;
    }

Failed to allocate a managed memory buffer of 268435456 bytes. The amount of available memory may be low

5 Answers