BizTalk Custom Pipeline Component System.OutOfMemoryException

Question

I'm currently developing a BizTalk Custom Send Pipeline that accepts an xml file and converts it to Excel. Unfortunately, after deploying the pipeline, I'm receiving a System.OutOfMemoryException. I've included the code for Execute Method of IComponent interface. All suggestions are welcome.

public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(IPipelineContext pContext, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
    {

        MemoryStream outMemStream = new MemoryStream();
        try
        {

            if (inmsg.BodyPart.Data != null)
            {
                // Read the source message coming from the messaging engine and convert it to memory stream 
                byte[] buffer = new byte[16 * 1024];
                using (MemoryStream ms = new MemoryStream())
                {
                    int read;
                    while ((read = inmsg.BodyPart.Data.Read(buffer, 0, buffer.Length)) > 0)
                    {
                        ms.Write(buffer, 0, read);
                    }
                    buffer = ms.ToArray();
                }

                if (buffer != null)
                {
                    var binaryWriter = new BinaryWriter(outMemStream);
                    binaryWriter.Write(buffer);
                }

                OpenXMLOffice oOffice = new OpenXMLOffice();
                outMemStream.Position = 0;
                oOffice.XMLToExcel(outMemStream, TemporaryFileLocation);
                inmsg.BodyPart.Data.Position = 0;
                inmsg.BodyPart.Data = outMemStream;
                pContext.ResourceTracker.AddResource(outMemStream);
            }

            return inmsg;
        }
        catch (Exception ex)
        {
            throw new ApplicationException(String.Format("Error converting XML to Excel:{0} - Stack Trace: {1}", ex.Message, ex.StackTrace));
        }
    }

Here is the most recent error received:

Log Name:      Application
Source:        BizTalk Server
Date:          2/14/2012 9:29:00 AM
Event ID:      5754
Task Category: BizTalk Server
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      IASDev-PC
Description:
A message sent to adapter "FILE" on send port "ExcelSendPort" with URI "C:\SeleneFTPFile\Excel\%MessageID%.xml" is suspended. 
 Error details: There was a failure executing the send pipeline: "IAS.SeleneFTPFile.ExcelEncodePipeline, IAS.SeleneFTPFile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=2add433e7764165f" Source: "Excel File Encoder" Send Port: "ExcelSendPort" URI: "C:\SeleneFTPFile\Excel\%MessageID%.xml" Reason: Error converting XML to Excel:Exception of type 'System.OutOfMemoryException' was thrown. - Stack Trace:    at System.IO.MemoryStream.set_Capacity(Int32 value)
   at System.IO.MemoryStream.EnsureCapacity(Int32 value)
   at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at IAS.SeleneFTPFile.Components.ExcelPipeline.EncodeExcel.Execute(IPipelineContext pContext, IBaseMessage inmsg)  
 MessageId:  {ED37CDD1-EF0C-46E7-9519-061AF3D4F8A4}
 InstanceID: {B0E448B3-3DAD-4E52-8F87-07C5D5AA5224}

One thing possibly - BinaryWriter is also IDisposable - stackoverflow.com/questions/1079434/… — StuartLC
@nonnb so should I add the BinaryWriter to the PipelineContext's ResourceTracker? — dustyhoppe
you've scoped the binaryWriter very tightly (good) but by explicitly disposing you may free the binarywriter earlier than GC gets around to collecting it. I doubt that this is the main cause of the OutofMemoryException though. — StuartLC

SliverNinja - MSFT SliverNinja - MSFT · Accepted Answer · 2012-02-15T15:06:56

You could try pre-allocating the MemoryStream buffer (as suggested here). The error message states that it can't allocate enough memory for the line when performing the buffered write (ms.Write(buffer, 0, read);).

using (MemoryStream ms = new MemoryStream(buffer.Length))

The other issue you have is that your buffer could overflow - producing this exact error when the MemoryStream is written to (ms.Write(buffer, 0, read)).

byte[] buffer = new byte[2 * 1024 * 1024]; // try increasing to 2MB buffer

BizTalk Custom Pipeline Component System.OutOfMemoryException

2 Answers