How to implement a C++ ATL/COM EXE Server via a plugin DLL

2
votes

I have an application that is implemented in c++. The application has a plugin API where you can compile dll's and put them into a certain location, and they will be auto-loaded into the host application when it starts up, customizing functionality.

Is there a way to compile a dll using ATL in such a way that the plugin dll will "transform" the application into a COM exe server.

The following ATL attribute code would work if it were an inproc server, but it's not an inproc:

[module(dll, ...)];

For an exe server I need:

[module(exe, ...)];

However, the compiler gives me errors on that because I'm actually compiling a dll, so I'm forced to use the following:

[module(unspecified, ...)];

This compiles fine, and MSDN for the "unspecified" option says:

Disables injection of ATL code related to the module attribute: the injection of ATL Module class, global instance _AtlModule and entry point functions. ... attributes in the project.

So, first, is it even possible to make an exe into a COM server via a plugin dll? Second, if so, what would be the right "ATL Module class, global instance _AtlModule and entry functions" to add to the project to get it to work right.

  • The application is a standard windows application, so its primary entry point is a WinMain() function.
  • The plugin dll would be delay-loaded, so it's not instantaneously loaded with the exe.

I'm a bit of a COM amateur, and the COM/ATL documentation is so deep and so vast and so wide, I don't even know where to start.

2
c++winapidllcomatl
So you have two applications, A (with a plugin API), and B (which does fun stuff). You want to write a plugin DLL for A that invokes B. Is that correct? If so, what you want is to make B an exe COM server, and then write a separate plugin DLL for A that CoCreateInstance B (as an out-of-process server), and then invokes B. – Eric Brown
No, it's the other way around. I have application A (with a plugin API), and B. I want to write a plugin DLL for A that makes A into a COM exe server, so that B can call CoCreateInstance A as an out-of-process server. – Anthony Johnson

2 Answers

2
votes

You can sort of do what you're asking (add COM support for an exe via 3rd party support), but it's going to be difficult, and a lot of the tooling will not help you.

I'm working off the 2nd Edition of ATL Internals, which explains what ATL is doing for you under the cover.

All COM servers must support registration, must expose the class object, and manage its lifetime.

We're going to play a little fast & loose on the registration part, as the standard method invokes the executable with some commandline options, which clearly won't work for you. You could have your plugin DLL auto-register the CLSIDs in the user hive (see this question for details) - this would avoid issues with elevation. Registration is a bit tricky, as you need to set the registration keys for an exe server, but do so from a dll.

Your plugin will call CoRegisterClassObject to expose the class object when the plugin is loaded, and call CoRevokeClassObject when the plugin is unloaded or the app terminates.

Note that you need to call CoRegisterClassObject "soon" after the exe starts.

Lifetime management for EXE servers is a bit fast & loose - you may (or may not) care to have your host app unload when the last reference is released.

Once you've implemented this, you can use ATL's CComObject without trouble, and I believe you can use the ATL Object Map for CoRegisterClassObject.

1
votes

The attributes are expanded into C++ code (you can change a setting in project settings and have this generated code saved into files for review, by the way). [module(exe, ... gets you CAtlExeModuleT as the additional base class to inherit from, then [module(dll, ... gets you CAtlDllModuleT. However the difference is not only the base class and this explains why you cannot switch so easily.

Here is the problem: attributed C++ code once looked cool but was deprecated afterwards. It still builds to build legacy projects but one should stay well clear from it.

If the project is relatively small, the best would be to get rid of attributes. This is quite doable since you can have a quick start with expanded non-attributed code generated by preprocessor. On the other hand it might be not as easy as it sounds since attributes helped to do COM without thinking too much of IDL files, how they are built into project etc. If you can afford stripping attributes, you want to do it.

Then second step is to understand the difference between ATL DLL server and ATL EXE server. The easiest is to generate two empty projects using wizard and compare the code. You will need to mirror the same difference into your project converting it from DLL to EXE. Understanding the differences, it might so happens that the easiest would be using new ATL EXE project and importing relevant source code files from old attributed project.

If you prefer to stay with attributes, perhaps you want anyway to start from empty projects, however I suppose you need old Visual Studio because this option was taken away/hidden quite some time ago. To create a new attributed project you will need, I suppose, VS 2005 or 2008.