DllMain not called after using LoadLibrary

Question

I'm a C++ beginner (3-4 month) and I'm trying to do lean about windows hooking. I had a bug with a DLL that I'm trying to inject and after a while I realized that my DllMain is not being called! I looked at almost every thread on StackOverflow and can't figure out my problem. I found that out by intializing a variable to 5, changing it to 1 in DllMain and I output the variable in a function. The variable never change. Here is the code:

int i = 5;

BOOL APIENTRY DllMain( HANDLE hModule, DWORD  ul_reason_for_call, LPVOID lpReserved  )
{
    i=1;
    switch (ul_reason_for_call)
    {
    case DLL_PROCESS_ATTACH:
        hDll  = (HINSTANCE) hModule;

        break;

    case DLL_THREAD_ATTACH:  break;
    case DLL_THREAD_DETACH:  break;
    case DLL_PROCESS_DETACH:  break;

    }

    return TRUE;
}

bool InstallHook(){
    cout << "INSTALLING HOOK... " << endl;
    cout << i << endl;
    hHook = SetWindowsHookEx(WH_CBT, (HOOKPROC) CBTProc, hDll, 0);
    return hHook != NULL;
}

And here is my loading the DLL...

typedef bool (*InstallHook)();
typedef void (*UninstallHook)();
InstallHook ih;
UninstallHook uh;
LRESULT CALLBACK WndProc(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam)
{
    switch(msg)
    {
        case WM_CLOSE:
            DestroyWindow(hwnd);
        break;
        case WM_DESTROY:
            uh();
            PostQuitMessage(0);
        break;
        default:
            return DefWindowProc(hwnd, msg, wParam, lParam);
    }
    return 0;
}

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,
    LPSTR lpCmdLine, int nCmdShow)
{
    // Bunch of code to initialize a simple window until this: 
    HINSTANCE hDll = LoadLibrary("e:\\projects\\DLL\\ToInject.dll");
    ih = (InstallHook)GetProcAddress(hDll, "InstallHook");
    uh = (UninstallHook)GetProcAddress(hDll, "UninstallHook");
    if (!ih()){
        cout << "SUCCESS" << endl;
    }else{
        cout << "FAILED" << endl;
    }
    // other stuff to create a window
    return Msg.wParam;
}

The output:

INSTALLING HOOK... 
5 // We can see here that the DLL never changed the value of i to 1.
SUCCESS
UNINSTALL HOOK...

FYI, the first argument to DllMain should be HINSTANCE, not HANDLE, but as they're both typedef void * I don't think that could be causing your problem. Is DllMain being exported? — Harry Johnston

Laurence Laurence · Accepted Answer · 2013-01-11T07:30:50

Follow these steps to make a working example (This is for visual studio):

Create a new console application;
Check Dll;
Place this in dllmain.cpp:

dllmain.cpp

// dllmain.cpp : Defines the entry point for the DLL application.

#include "stdafx.h"
#include <string>

std::string test = "not Loaded"; 

BOOL APIENTRY DllMain( HMODULE hModule,
                       DWORD  ul_reason_for_call,
                       LPVOID lpReserved
                     )
{
    test = "loaded"; //You also change on this location the value of a variable
    switch (ul_reason_for_call)
    {
    case DLL_PROCESS_ATTACH:
        MessageBoxA(NULL,"test","test",NULL);
        break;
    case DLL_THREAD_ATTACH:
    case DLL_THREAD_DETACH:
    case DLL_PROCESS_DETACH:
        break;
    }
    return TRUE;
}

extern "C" __declspec (dllexport) bool example()
{
    MessageBoxA(NULL,test.c_str(),"test",NULL);
    return true;
}

Compile the code, make sure it's on release mode;
Create a new console application;
Check empty project;
Create a new file with the name main.cpp;
Copy this in the file:

Main.cpp

#include <windows.h>

typedef bool (*testFunction)();

testFunction dllFunction;

int main()
{

    HINSTANCE hDll = LoadLibraryA("example.dll");
    if(hDll)
    {
        dllFunction = (testFunction)GetProcAddress(hDll, "example");
        dllFunction();
    }

    return 0;
}

Build the project, make sure it's on release mode;
Copy your DLL that you made in the release folder from your console application;
Run the program.

It workes here. I get a MessageBox with test and 1 time with loaded. You can compare this code with yours. I hope you find the problem.

DllMain not called after using LoadLibrary

3 Answers