Prevent COM from activating the server executable?

1
votes

TL;DR - How do I prevent -Embedding startup of a CLSCTX_LOCAL_SERVER server? Preferably so that the clients get a decent error code immediately.

Background

We have a native C++ interactive Desktop application that interacts with a COM object in another native C++ interactive Desktop application.

Basically, COM is used as an interprocess communication mechanism.

Now, when the "server" application is started interactively after the user has brought it into the correct state, it will prepare the COM interface: CoRegisterClassObject etc etc.

When the client application is used and then does it's CoCreateInstance for the coclass, it will communicate with the already running other Desktop application, which is what is intended.

However, when the "server" application is not running, starting the client will launch the server application interactively, which is NOT what we want, because it needs quite some processing and setting up before the client can meaningfully communicate with it.

Question

So, what would make more sense is for the client to just error out in case the server is not running, instead of having the COM infrastructure start an interactive application that can't service the request meaningfully anyways.

We've toyed around with the following ideas:

  • Unregister the server each time it is closed.
    • It seems this won't work, as we need administrative rights to register/unregister the server.
  • Use the CLSCTX_DISABLE_AAA flag on the client side.
    • This seems to work, if the client specifies this and the server ain't running, it will get 0x80070005 ERROR_ACCESS_DENIED, but we're rather unsure whether this is the correct approach.
  • Do an early check in the server application to detect the -Embedding switch and immediately exit the application on this.
    • The client application will run into a timeout (~ 2 min), which is not very user friendly.
  • from comments Do not write the info to the registry - CoRegisterClassObject should be enough.
    • Currently, I have:
      • HKCR\AppID (...)
      • HKCR\CLSID\{...} plus subkeys ProgID, VersionIndependentProgID, LocalServer32, Typelib
    • Clients currently resolve the CLSID from the VersionIndependentProgID
    • So I'm left wondering which parts of the registry are really optional.

Is there any "standard" way to prevent COM local server executable activation for a given registered COM class?

2
windowswinapivisual-c++com
COM registration doesn't need admin rights, but you'll have to register everything under HKCU instead of HKLM (which seems to make sense in your context). Otherwise you don't have to use CoCreateInstance to get a reference, you can use the Running Object Table, from the client check your server has registered something in it and call IRunningObjectTable::GetObject – Simon Mourier
When detecting the -Embedding arg, you may try to issue immediately the CoRegisterClassObject call and THEN exit. The client will get a CO_E_SERVER_EXEC_FAILURE without timeout... Yes, that's ugly. – manuell
For inproc com objects you don't need to have them in the registry - you can register their class factories at runtime. I'm wondering if this can be applied to out-of-process COM objects by registering the proxy objects... – Chris Becke
CoRegisterClassObject is just to publish your object to COM (as an oop server), it does nothing with respect to the ROT. Once you've called CoRegisterClassObject, you must call IRunningObjectTable::Register from the server and IRunningObjectTable::GetObject from the client. – Simon Mourier
simply not register your object clsid in registry. if server yet not call CoRegisterClassObject client get error REGDB_E_CLASSNOTREG, otherwice call CoRegisterClassObject is enouth – RbMm

2 Answers

1
votes

My idea is this ...

Just set a global flag in your EXE when you detect the server is started with -Embedding. I would probably create a special class factory only when you startup with the -Embedding flag. This class factory would return a fail code when IClassFactory::CreateInstance() is called. You would not register the standard class factory as running with CoRegisterClassObject(), but you'd register only your alternative factory that always returns the fail code.

Yes, there would still be a slight delay on starting up the EXE, but when the CreateInstance() is called, it would immediately return the fail code and so the caller would not have a long timeout...maybe 1-5 seconds only.

-1
votes

Write up of what I learned from comments so far:

You do not have to use CoCreateInstance, but you can use GetActiveObject, which

Retrieves a pointer to a running object that has been registered with OLE.

so there you have a way to get an object without activating it, but relying on it already being registered. (But this is not done by CoRegosterClassObject, instead you need to ...?)

Similar to the GetActiveObject approach, you can use the machinery around IRunningObjectTable - which may be what's used by GetActiveObject under the covers anyway. i git kind of lost there.

Another piece of info is that the registry is claimed to be kinda "optional" in all this: (paraphrasing)

CoRegisterClassObject is just to publish your object to COM (as an oop server), ...

simply not register your object CLSID in the registry. If the server has not yet called CoRegisterClassObject, client gets error REGDB_E_CLASSNOTREG, otherwise call CoRegisterClassObject is enough. ...

you not need add it to Running Object Table - the single call CoRegisterClassObject is enough - the client can create instance after this. No need for any registration in the registry - ...

... you of course need, for any remote interface, an entry in registry Interface\{..}\ProxyStubClsid32 but not need this for CLSID.

CoRegisterClassObject does not need the registry at all ... But you need still marshal your interface(s). For this you need ther Interface\{..}\ProxyStubClsid32 key.

The TypeLib you need if you do this type of marshaling - set {00020424-0000-0000-C000-000000000046} here. The CLSID and AppID you do not need.

CLSID and APPID you need for starting your app, if it is not started (or load dll).

If you already are running and call CoRegisterClassObject - this is enough for client call connect to you. But without CLSID, client can not exec your app (simply unknown what).

If you do not do custom marshaling - need for every interface have info in registry - which dll ProxyStubClsid32 do this marshaling. This can be or custom dll or if standard {00020424-0000-0000-C000-000000000046}, then you need a typelib for use by oleaut32.

In summary, it would appear the way to prevent having an app activated by Windows is to

  • NOT write the info needed for activation into the registry in the first place.
    • but only the info required for marshaling.
  • and/or NOT calling CoCreateInstance and retrieving the object via another route.