How to pass an uninitialized array from VB6 to .NET via COM?

Question

I want to replace a component written in VB6 with a new component written in VB.NET. The new component has to work with other VB6 applications until these are replaced as well. I'm having trouble with a function that takes string arrays as parameters.

I'm using a VB6 class that contains this function to replicate the problem:

Public Function MyMethod(arr() As String, arr2() As String, Result$) As Integer     
    Result = Join(arr, ", ")
    MyMethod = 0        
End Function

I can successfully call it from an test program (VB6) like this, which displays "Hello, World":

    Dim obj As Object
    Dim arr() As String
    Dim arr2() As String
    Dim result As String

    Set obj = CreateObject("MyHelloWorld.MyClass")
    'Set obj = CreateObject("HelloWorldCOMNet.MyNetClass")

    arr = Split("Hello World", " ")
    'ReDim arr2(0)
    result = ""

    If Not obj.MyMethod(arr, arr2, result) Then
        MsgBox result
    End If

I can't modify the actual VB6 application, but I want to replace the VB6 ActiveX component with class written in .NET. The new class looks like this:

<ComClass("cd74ab4a-76ca-4c84-9f49-147e6f6ac01f", "b3314f71-cb8d-48ea-bfe6-9d1995aa4f58", "40c30052-0cc8-4ef6-b1e8-92e4ddbcb515")> _
Public Class MyNetClass

    Public Sub New()
        MyBase.New()
    End Sub

    Public Function MyMethod(ByRef arr() As String, ByRef arr2() As String, ByRef result As String) As Short
        result = String.Join(", ", arr) + " from .NET"
        Return 0
    End Function

End Class

I test it by replacing Set obj = CreateObject("MyHelloWorld.MyClass") with Set obj = CreateObject("HelloWorldCOMNet.MyNetClass") in the test application.

I used a breakpoint in the constructor of the .NET class to verify that it has been executed.

When the test application calls MyMethod, a FatalExecutionEngineException occurs.

I narrowed the error down to the second parameter arr2. When I initialize the variable using ReDim arr2(0), the .NET code works. So my guess is that it's related to the fact that this variable is uninitialized. Unfortunately I can't modify the actual VB6 application.

I compared the method signatures using OleWoo and they appear to be the same (except for the id).

How can I modify the signature of the .NET function to accept uninitialized string arrays?

Edit: Output from OleWoo:

[id(0x00000001)]
short MyMethod(
   [in, out] SAFEARRAY(BSTR)* arr,
   [in, out] SAFEARRAY(BSTR)* arr2,
   [in, out] BSTR* result
);

Based on stackoverflow.com/q/35190653/11683 and stackoverflow.com/q/39351476/11683, I would try declaring it as Array and/or applying the marshaling attributes. — GSerg
I'm hoping for a solution similar to this. Unfortunately, I can't get it to work. With ByRef arr2 As Array I get "Invalid procedure call or argument (Error 5)" and <MarshalAs(UnmanagedType.SafeArray, SafearraySubType:=VarEnum.VT_BSTR)> ByRef arr2 As Array causes the same exception as before. I expected the last one to work. Also ByRef or ByVal makes no difference. I updated the question with the output from OleWoo. — BenZinra
While you are looking for a solution, you can write a dll in VB6 with the expected signature that receives the arrays, redims as needed and calls the actual implementation in .NET. But I'm surprised the marshaller is unable to marshal an empty array - at least as Nothing. — GSerg
I recreated your case in VBA(closest I have to VB6) and got the same error. If I add a reference to the .Net assembly and declareobj as MyNetClass, then the code works. So issue appears to be in the the late-binding marshaling. — TnTinMn

BenZinra BenZinra · Accepted Answer · 2017-09-24T01:55:59

I believe that this is a bug in the .NET Framework.

I attached a debugger to the VB6 application. This is the stack trace at the moment of the crash:

This is the disassembled code at that location:

That is the function DispatchInfo::IsVariantByrefStaticArray. It dereferences the result of (*V_ARRAYREF(pOle)) without checking if it's NULL. The member fFeatures is at offset 2 resulting in an access violation at address 0x00000002.

I believe this is the problem I stumbled upon. I think I will have to find another way to solve my original problem.

How to pass an uninitialized array from VB6 to .NET via COM?

1 Answers