Why does my COM component throw OutOfMemoryException, but runs fine on VB6?

Question

I have a COM component written in VB6 that I'm using from a Visual Basic .NET application through COM interop.

There is a method that throws an OutOfMemoryException when called from VB.NET. However, when I run the exact same call from a Visual Basic 6 application, everything works fine.

I cannot post code here because it is too long and hard to follow (I'm tasked with fixing it) and I can't pinpoint the problem, because when I try to debug it from VB6, the error doesn't show up.

What could be causing this distinct behavior?

This was working fine until I fixed a performance problem using a Dictionary object (a COM one from the Microsoft Scripting Runtime). Unless the Dictionary is growing irrationally or leaking, I don't see how it could be causing this, since it never grows past 100-200 elements and only one is created before the exception is thrown.

I tried the same fix with a Collection object and an awkward Exists() function and the same problem happens. If I bail out early by returning Nothing, it works (i.e., it throws NullReferenceException, as would be expected).

Is there any meaningful stacktrace? and what about memory? what size is the process ? — Simon Mourier
@Simon: stacktrace ends at the COM/.NET boundary. The process is about 100 megs when the exception flies across the stack. — R. Martinho Fernandes
The Scripting.Dictionary COM object is marked as STA (Apartment), as stated below by weloytty, have you checked the threading? And BTW, are you accessing this dictionary using multiple threads? — Simon Mourier
Try compiling for 32-bit target instead of anyCPU or 64-bit, if you are running on a 64-bit machine. — xpda
@Simon: no, I'm accessing it only once, from a single thread. I will have to check the apartment tomorrow, but I believe it is STA, because this is an WinForms application, and WinForms runs on STA. — R. Martinho Fernandes

Hans Passant Hans Passant · Accepted Answer · 2010-12-07T20:47:03

Memory management in VB.NET is very different. VB6 uses reference counting to release COM objects, VB.NET uses the garbage collector. You can get into a situation, especially when you are unit-testing the component, where the GC doesn't run frequently enough to release the COM objects. Those objects allocate unmanaged memory, that doesn't put pressure on the garbage collector to kick off a collection.

Start diagnosing this with Perfmon.exe, Performance Monitor. Right-click the graph, Add Counters and select .NET CLR Memory, # Gen 0 collections and pick your process. Verify that you see the counter incrementing at a reasonable rate, at least one a second.

Fixing such a problem is fairly unpleasant, make sure that this is not an artificial problem that is induced by isolating the component. Or a plain bug in your code, keeping a reference to the COM object. Calling GC.Collect() followed by GC.WaitForPendingFinalizers() will release COM objects. So does calling Marshal.ReleaseComObject(). Using GC.AddMemoryPressure() is a possible workaround as well, but you do have to have a reasonable idea how much unmanaged memory the COM object requires.

Why does my COM component throw OutOfMemoryException, but runs fine on VB6?

2 Answers