What to do when FreeLibrary API call fails?

11
votes

Question

I have a third-party DLL that throws an unhandled exception when attempting to unload it from my native C application. This results in the call to FreeLibrary failing, and the module remaining loaded in my process.

Are there any options to forceably unload the library?

What do you do when the FreeLibrary calls?

Additional Background

When using load-time dynamic linking this is annoying enough, but ultimately the application gets torn down by the OS. The problem comes when using run-time dynamic linking. I load up this DLL, use it, and then in some instances I need to unload it from my process's virtual address space and then continue running. When I call FreeLibrary on the third-party library, it does some cleanup work (i.e. in DllMain when DLL_PROCESS_DETACH is called). While it's doing it's cleanup, it causes an exception to be thrown which it doesn't handle, and bubbles up as an unhandled exception to FreeLibrary. This results in the call failing and the module remaining loaded.

I've put a ticket in with the vendor so hopefully I can get a fix which will allow this specific library to unload successfully. In case I don't however, and for the general case of this issue, I'm curious as to what the options are.

3
cwindowswinapidllnative
Does FreeLibrary() catch the exception, or does it propagate back to your code? If it's the latter case, I wouldn't be surprised if the next call to LoadLibrary() deadlocks because FreeLibrary() didn't get a chance to release the loader lock. - bk1e

3 Answers

9
votes

If you are after only unloading dll from the memory you can use

UnmapViewOfFile

providing bases address of your loaded dll as an argument.

Example:

HINSTANCE hInst = LoadLibrary( "path_to_dll" );

if( !FreeLibrary( hInst ) )
{
   fprintf( stderr, "Couldn't unload library. Error Code: %02X\n. Attempting to unmap...", GetLastError() );
   if( !UnmapViewOfFile( hInst ) )
   { 
     fprintf( stderr, "Couldn't unmap the file! Error Code: %02X\n", GetLastError( ) );
   }
}

Or if it's a library that you didn't explicitly load (e.g. a library dependency that was loaded by a library that you loaded) and you don't have the handle, then use GetModuleHandle:

HINSTANCE hInst = GetModuleHandle( "dllname_you_didn't_load" );
if( hInst != NULL )
{
   if( !UnmapViewOfFile( hInst ) )
   { 
     fprintf( stderr, "Couldn't unmap the file! Error Code: %02X\n", GetLastError( ) );
   }
}
1
votes

I think there is only one possible solution - interop with dll through dedicated thread. So in time you need to unload dll you just exit (or may be kill) that thread and all associated with it resources will be freed. In this case you are not guaranteed about memory leaks, but I suggest this solution as temporary, till 3d party fixes bugs in dll

1
votes

This is probably not the issue you're experience, but in case it is:

When using linker support for run-time dynamic linking, don't forget to use the /Delay:Unload argument.