AddDllDirectory interferes with subsequent LoadLibraryExW

Question

In our application we have a special structure for the dll search path. Due to an update of a third party library, the resolving of dlls is now broken. Attempts to fix the search path with AddDllDirectory resulted in unexpected problems.

MyApp.exe is a C++/MFC-application and uses Managed#1.dll, which in turn calls a native dll. Furthermore, it needs MyNativeDll.dll which is installed in a different folder:

Folder structure:
Program Files\MyApp\MyApp.exe
                   \Managed#1.dll                  
                   \x86\Native#1.dll
Program Files\MyDll\MyNativeDll.dll
                   \Managed#2.dll
                   \x86\Native#2.dll

The original problem we had, was that Native#2.dll was not found (this dll was added in an update of a third party library in Managed#2.dll). So I added the following to the DllMain of MyNativeDll:

MyNativeDll.dll::DllMain
{
    [...]
    SetDefaultDllDirectories(LOAD_LIBRARY_SEARCH_DEFAULT_DIRS);
    AddDllDirectory("[...]Program Files\MyDll\x86");
}

After this, Native#2.dll is loaded as expected.

But: Now when MyApp.exe loads Managed#1.dll, this call fails, as Managed#1.dll fails to load Native#1.dll. The problematic call now is (the following is in some 3rd party library):

Managed#1.dll::Foo()
{
    LoadLibraryExW("Native#1.dll", NULL, 0);  //<-- This searches in the wrong path
}

According to the documentation, AddDllDirectory should only add some directory to the search path.

How can AddDllDirectory cause a latter call to LoadLibraryExW to fail?

I'm not sure why Native#1.dll would ever have been found, given that it's in a sub-directory. Normal DLL search paths includes the program folder, but not sub-directories. — Jonathan Potter
"How can AddDllDirectory cause a latter call to LoadLibraryExW to fail?" - By forcing the system to load a module from a location it otherwise wouldn't. If a dependency is inaccessible from that location added to the search path, the request to load the module fails. — IInspectable
@JonathanPotter: Thanks for the hint. I had a look into the source code of the 3rd party dll. There is some elaborate code that determines the "x86"-subpath, and (correctly) loads the library from there. I suppose, the actual problem appears shortly after, when this library tries to load its own dependencies (Native#1.dll actually is 20 dlls) , and does not search relative to its own path anymore (see my answer below). Thank you. — hofingerandi

hofingerandi hofingerandi · Accepted Answer · 2020-02-26T11:18:56

It turned out, that the problematic call was not AddDllDirectory(), but the previous SetDefaultDllDirectories().

The documentation lists a number of flags (LOAD_LIBRARY_SEARCH_APPLICATION_DIR, LOAD_LIBRARY_SEARCH_SYSTEM32, LOAD_LIBRARY_SEARCH_USER_DIRS) and states that LOAD_LIBRARY_SEARCH_DEFAULT_DIRS is a combination of those three.

Anyways, there is another option LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR, which is not mentioned in this list. So the call to

SetDefaultDllDirectories(LOAD_LIBRARY_SEARCH_DEFAULT_DIRS);

effectively removed LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR from the search path. Replacing the above with

SetDefaultDllDirectories(LOAD_LIBRARY_SEARCH_DEFAULT_DIRS | LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR);

fixed the problem.

AddDllDirectory interferes with subsequent LoadLibraryExW

1 Answers