Wrapping a 32-bit COM shell extension within a "thin" 64-bit DLL wrapper

5
votes

The premise:

I have an old, 32-bit COM shell extension (written in C++). After years of incompatibility with newer, 64-bit systems, we are now updating it to work in a 64-bit Windows environment.

The trick:

The 32-bit COM DLL contains dependencies on third-party libraries, for which there are no 64-bit builds available. As such, we cannot simply 'recompile in 64-bit.' Given this restriction, we have decided the easiest approach will be to create a 'thin' 64-bit DLL; essentially an empty DLL defining only the required interfaces, and redirecting all calls to the underlying 32-bit COM DLL.

I believe this communication between a 64-bit COM DLL and a 32-bit COM DLL is possible, through the use of COM surrogates. Unfortunately, this appears to be a very specific/niche topic, and I have been unable to find many resources about how to call 32-bit COM APIs from a 64-bit COM DLL.

The questions:

  • Is this 'thin COM wrapper' a reasonable approach, for enabling 64-bit functionality of a 32-bit COM DLL?
  • Is there any reason this approach would not work specifically for Windows Shell Extensions?
  • Would the 64-bit interface definitions use identical GUIDs as their 32-bit counterparts?
  • Is this a strategy which has been documented in the past?
  • Are there any good resources for calling a 32-bit COM API from within a 64-bit COM DLL?

Clarifications:

This question has a number of aspects which make it unique, setting it apart from the question "Using 32-bit shell extensions in Windows 7 64-bit."

I am not asking if a 32-bit shell extension can be loaded in a 64-bit explorer process, as the referenced question does. Rather, I am asking if it is possible to create a thin 64-bit implementation of the DLL, which acts as a bridge between the 64-bit explorer process and the 32-bit implementation.

To be clear, this is not possible.

enter image description here

But maybe this is?

enter image description here

2
c++windowscom32bit-64bitshell-extensions
No you cant. Whilst COM surrogates as a concept can be used between say a x64 client and say a 32-bit in process COM server hosted in a 32 COM surrogate, the mix of 32 and 64 bit won't matter but that won't work for the Windows Shell which requires all extensions match the "bit-ness" of the Windows Explorer process which is derived from the OS. So if the OS is 64 bit so too must the extensions. – stackoverflow.com/questions/13747836/… – MickyD
In this specific proposal, the bit-level of the shell extension DLL will be 64-bit. Within the 64-bit DLL, however, the specific implementation will be redirected via COM surrogate to a 32-bit DLL. – BTownTKD
The original 32bit shell extension must be hosted in something, a surrogate yes, but it would need to mimic arguably all the relevant shell extension hosting functionality of 32bit Windows Explorer I suspect, not a trivial task. Like ActiveX, there is much two-way communications between the component and the "site" - in this case Explorer. Particularly if the extension is a shell namespace extension. Depending on the type of extension, one cannot merely load an COM extension without the proper ecosystem in which to host it. – MickyD
Consider rewording your diagram above. In the diagram, the green box can not act as a COM surrogate. A COM surrogate by definition is a Windows process but your green box must be an in-process COM server (i.e. a .DLL) in order to be loaded into Windows Explorer 64bit. – MickyD
@BTownTKD, you can do this for as long as you don't have to deal with data that COM can't marshal. And your thin 64-bit DLL might need to handle requests back from the 32-bit surrogate. For instance, you must turn creation of Explorer objects in the 32-bit DLL (with CoCreateInstance or similar) into requests back to the 64-bit DLL (if you're using IClassFactory::CreateInstance, this basically only requires you to get the factory from the 64-bit DLL instead of with CoGetClassObject). – acelent

2 Answers

1
votes

Yes, it is doable with DCOM and your worst problems will be setting up permissions for the DCOM "component" and making data marshaling work.

Is this 'thin COM wrapper' a reasonable approach, for enabling 64-bit functionality of a 32-bit COM DLL?

Yes, we've done this once when we had to provide an in-proc COM server with both 32-bit and 64-bit implementations. We basically had such thin server which would be implemented in both 32 and 64 bits (the consumer would load the appropriate one) and redirect calls to a 32-bit outproc server hosted in DCOM.

Is there any reason this approach would not work specifically for Windows Shell Extensions?

You may be unable to proper setup permissions for the outproc server. Most likely this will work. Cannot imagine any other reason.

Would the 64-bit interface definitions use identical GUIDs as their 32-bit counterparts?

Yes. A COM interface is a contract. You can have as many implementations of a contract as you wish.

Are there any good resources for calling a 32-bit COM API from within a 64-bit COM DLL?

Nothing special required. Just call CoCreateInstance() with CLSCTX_ALL and that's it.

Your primary concern will be marshaling. You have to marshal data between client and server which will now be in different processes. Your best bet is using Automation marshaling (aka typelib marshaling). If it happens that the original interface is not Automation compatible then try introducing a new interface which is Automation compatible and your wrapper will then server as an adapter. It may so happen that you will be unable to design a new Automation compatible interface for your task but try this because this is your best bet.

Whenever anything fails to work - something doesn't marshal, some dependent files are not found - use Process Monitor to see what's going on that fails.

1
votes

We have 64-bit application that successfully makes use of 32 bit only COM applications (such as VB6 ones). It seems complicated, but essentially you just have to add a few registry entries for the COM component you are trying to use and then you can use it. You can see the details here:

http://www.gfi.com/blog/32bit-object-64bit-environment/ http://www.codeproject.com/Tips/267554/Using-bit-COM-Object-from-bit-Application

So yes, it will work, but of course there is extra complexity here, and simple solutions are usually the best.

Alternatively, you can always wrap the 32-bit functionality as a service or something of that sort and access it that way. Or as an executable and use a text file or something to communicate.