Access Violation while exporting an unmanaged function pointer

Question

I have been trying for the past 4 hours to solve a very mysterious problem.

I am writing some plugin for Notepad++. To achieve syntax highlighting one has to export such a function:

//this function is exported via exports.def file
LexerFactoryFunction SCI_METHOD GetLexerFactory(unsigned int index)
{
    return (index == 0) ? RTextLexer::LexerFactory : nullptr;
}

where,

LexerFactoryFunction is typedef ILexer *(*LexerFactoryFunction)();
#define SCI_METHOD __stdcall

I have managed to get this thing working perfectly with C++, however another part of the plugin is written in C#, so I tried to merge the two using Fody Costura NuGet package ( so that the CLI .dll is embedded into the main .dll ), however with no success.

What I've tried :

public ref class RTextLexerCliWrapper
{
public:
    delegate ILexer * GetLexerFactoryDelegate();

IntPtr GetLexerFactory()
{
    return System::Runtime::InteropServices::Marshal::GetFunctionPointerForDelegate(_lexerFactoryPtr);
}

RTextLexerCliWrapper();
private:
    GetLexerFactoryDelegate ^ _lexerFactoryPtr;
    GCHandle gch;

    ~RTextLexerCliWrapper();
};

RTextLexerCliWrapper::RTextLexerCliWrapper()
{
    _lexerFactoryPtr = gcnew GetLexerFactoryDelegate(&RTextLexer::LexerFactory);
    gch = GCHandle::Alloc(_lexerFactoryPtr);
}


RTextLexerCliWrapper::~RTextLexerCliWrapper()
{
    gch.Free();
}

This CLI wrapper, is referenced in my main .dll like this :

static RTextLexerCliWrapper _lexerWrapper = new RTextLexerCliWrapper();

[DllExport(CallingConvention = CallingConvention.Cdecl)]
static IntPtr GetLexerFactory(uint index)
{            
     return (index == 0) ? _lexerWrapper.GetLexerFactory() : IntPtr.Zero;
}

So what happens is, my .net function gets indeed called and the cli wrapper function is also called, and a function pointer is indeed returned. However any attempts to call that function pointer results in an access violation. Which means that either the type of the pointer is wrong or something else which I am currently missing. I have tried countless variations of the .net exported function with void *, StdCall etc. All result in the same problem.

Is there any other way to return a function pointer of a C++ class? Or well am I doing something completely wrong?

Thanks in advance!

This is a complete zoo, you put the tiger in the monkey house. — Hans Passant

FailedDev FailedDev · Accepted Answer · 2015-04-18T13:49:33

So I have finally managed to found the solution to my problem.

First step was exporting the functions with the correct calling convention:

    static RTextLexerCliWrapper _lexerWrapper = new RTextLexerCliWrapper();

    [DllExport(CallingConvention = CallingConvention.StdCall)]
    static IntPtr GetLexerFactory(uint index)
    {            
        return (index == 0) ? _lexerWrapper.GetLexerFactory() : IntPtr.Zero;
    }

The convention in this case had to be StdCall. Otherwise the stack pointer is invalidated, hence the exceptions.

Now in order to return a function pointer of a C++ instance things were a bit more tricky.

I am statically storing an instance of the CLI wrapper class so that it doesn't get GCed. ( _lexerWrapper ).

This instance has a function called GetLexerFactory which return a function pointer of the C++ instance ( which is then used by some other .dll to get actual instances of some object ).

The CLI Wrapper class looks like this:

    public ref class RTextLexerCliWrapper
    {
    public:

        delegate ILexer * GetLexerFactoryDelegate();

        IntPtr GetLexerFactory()
        {
            return System::Runtime::InteropServices::Marshal::GetFunctionPointerForDelegate(_lexerFactoryPtr);
        }

        RTextLexerCliWrapper();

    private:
        GetLexerFactoryDelegate ^ _lexerFactoryPtr;
        GCHandle gch;

        ~RTextLexerCliWrapper();
    };

Where ILexer * is the type of the object we shall be returning later on.

    RTextLexerCliWrapper::RTextLexerCliWrapper()
    {
        _lexerFactoryPtr = gcnew GetLexerFactoryDelegate(&RTextLexer::LexerFactory);
        gch = GCHandle::Alloc(_lexerFactoryPtr);
    }


    RTextLexerCliWrapper::~RTextLexerCliWrapper()
    {
        gch.Free();
    }

So, what we have managed here is, exporting through .NET a function pointer which is able to return a pure C++ object.

Access Violation while exporting an unmanaged function pointer

1 Answers