win7 .net code (calling an interop vb6 dll) works in debugger, fails stand-alone

Question

Environment: Win7 64 bit installed with VS2005 and VB6. All appls previously worked fine on XP, I am porting to Win7 64.

A: a Win7 VB.NET app, the main application program. B: an interop com dll coded in VB6. B references yet another COM dll coded in VB6.

A's call to B works ok when either 1) running A in VS2005 debugging mode or 2) running B in VB debugging mode with A as start program.

But when running A stand-alone, i.e. out of any IDE, A fails on a call to B.

At first, a memory access violation was generated when A calls B. Then I made A call a different method on B, this time, B pops up a dialog box that missed certain GUI controls, but the call didn't crash the whole app.

Per George's question: I started VS2005 and VB6 as admin, so debug executions were under local admin. Stand-alone run was under admin as well, since I started a powershell window as admin and started the application from there. Thank you George.

Yes, Target CPU is set to X86 (not "Any CPU"), thx tcarvin.

Update: I made a little progress - several COM interfaces have code like this:

Public Sub Proc (ByRef conn As ADODB.Connection)
Set mbrCN = conn

where mbrCN is private member of a class module, I changed the object assignment to duplicating a new instance

Set mbrCN = New ADODB.Connection
With mbrCN
    .ConnectionString = conn.ConnectionString
    .CursorLocation = adUseClient
    .Open
End With

Now some exception messages are gone, but I am still faced with partly shown form issue. Specifically, A calls B.method, which in turn displays a form that contains a 3rd party grid control (VSFlexGrid8.ocx). Everything seems to work when I either debug-run A in VS2005 or debug-run B in VB6, but when I run A stand-alone, the form pops up, but the VSFlexGrid on it is missing, trying to find out more about VSFlexGrid under interop......

Solved: I found a more up-to-date build of VSFlexGrid8.ocx, to be exact VSFlex8 Build Number 8.0.20132.297 Build Date: September 13, 2013, file name vsflex8_8.0.20132.297.zip, that solved the issue. sorry I didn't keep the download URL.

I started VS2005 and VB6 as admin, so debug executions were under local admin. Stand-alone run was under admin as well, since I started a powershell window as admin and started the application from there. Thank you George. — huirong
Did you set your .NET application to x86? VB6 COM objects are 32 bit, so you should avoid the use of the default "Any" that Visual Studio gives you. — tcarvin

Pásztor Zoltán Pásztor Zoltán · Accepted Answer · 2013-10-01T09:41:58

we've experienced the same error in an almost identical setup (VsFlex7 grid instead of VsFlex8).

The key to resolving it was the "Data Execution Prefention (DEP)" feature. Try to set it to 'OptIn' mode and put your process on the exception list.

HTH

Zoltán

win7 .net code (calling an interop vb6 dll) works in debugger, fails stand-alone

2 Answers