MSXML and .NET - Memory Usage

Question

I have been troubleshooting a production web application (.NET 4) that was written by a vendor.

We have been having some speed issues with this site for the past few weeks.

Originally, the focus has been on the database side. Now that I feel comfortable ruling that part of the equation out... I spent a good bit of time today debugging the web application to see where the actual speed issue was coming from.

What I have found is that in this specific part of the web application it opens a new internet explorer window to display a large amount of XML data that comes back from the database.

After looking over the code and using the debugger to try to figure out what was happening... here is what I found:

When the code on this specific page is done populating everything, the iexplore process has a private working set of 692,044K. This seems to be the result of numerous in process activex controls for msxml3.dll.

After some research, I discovered this microsoft article: http://support.microsoft.com/?scid=kb%3Ben-us%3B815112&x=15&y=14

It states: "Microsoft does not support the use of MSXML (Microsoft’s COM-based XML parser) in .NET applications. MSXML uses threading models and garbage-collection mechanisms that are not compatible with the .NET Framework. Using MSXML in .NET applications through COM interoperability can result in unexpected problems that are difficult to debug. Microsoft does not recommend or support directly instantiating and using MSXML objects in .NET code, nor does Microsoft recommend or support marshalling MSXML interface pointers across the interop boundary."

This is one part of the code that is creating the activex object:

                var oTmp = new ActiveXObject("MSXML2.Domdocument");
                oTmp.loadXML('<plan_data>' + returnValue + '</plan_data>');

Process explorer picks up several MSXML3.dll loaded into the iexplore process as the memory increases. Even after closing this specific window it does not seem to clean up after itself properly because it still uses around 220,000 K of memory.(probably due to the garbage-collection mechanisms listed above).

So, my question is... Can anyone here who is familiar with this offer some advice on if this seems like a poorly designed section of the web application?

I would like to be able to take this information to the developer and have them take a look at it but I was hoping someone could give me some professional advice first.

Thank you.

Why did you put the focus on the database side before you measured the bottleneck? ;-) The first rule of optimization is "measure first". — Eric J.
one question: Why would you need to "display a large amount of XML data that comes back from the database"?? Users aren't going to read in any sensible way a huge amount of XML.... Where does this XML end up? — Mitch Wheat
Well, that is why the developer (not me) is using a large number of activex objects to display the data. It is coming back in smaller sets over an apparently large amount of single threaded MSXML containers. — user1441145
Horse crap! There's TONS of stuff out there that's been happily using MSXML for close to 20 years now! You've found a bottleneck ... now you need to figure out how to deal with it. To blindly say "MSXML is Bad" is either disingenuous ... or stupid. The actual problem is deeper than just "MSXML vs no MSXML". IMHO... — paulsm4
My concern would not be with MSXML itself but with the implementation within the .NET code, which seems to go against the microsoft recommendation listed above. — user1441145

Eric J. Eric J. · Accepted Answer · 2012-06-07T02:17:22

Any time you process a large XML structure into memory and then act on it, it's a fairly memory intensive operation. If you really need to hold the whole document in memory, look at replacing your current code with XDocument from the System.Xml.Linq namespace.

If you only need to stream through the XML document once, performing operations as you read each node, look at Streaming Transform of Large XML Documents.

EDIT

@Yahia makes a good point that I missed... this code seems to be executed client side. Is that the case? If so, why not just let IE (your ActiveX will only run in IE, some other browsers with plugins ;-) display the resulting XML. If the XML is not too large, it does a great job of it (though above a few MB it takes a very long time to do the initial processing).

MSXML and .NET - Memory Usage

2 Answers