Shell Namespace Extension in Delphi Using the VCL

3
votes

Shell namespace extensions are quite complex. We have been building a shell namespace extension for the past 10 years; the latest incarnation of which is the Archive Folders feature in MagicRAR (www.magicrar.com).

Unfortunately, there are still occasional crashes with our shell namespace extension, despite extremely careful coding, ensuring threads properly access shared memory, and such. The Explorer host process crashes during or outside of the use of our shell namespace extension.

We have used a variety of tools such as AQTime Pro to troubleshoot our shell namespace extension code. There are no memory overwrites or other similar access problems reported. This leaves only one culprit: the VCL is not thread safe!

Indeed, we are using the VCL as part of our shell namespace extension; the file list in Explorer is actually a hosted control and in the case of our own shell namespace extension, it is actually a VCL window. We are now even wondering (after multiple generations of development) whether this is an allowed scenario in the first place...

The main application object doesn't even exist in our shell namespace extension DLL. Using TThread.Synchronize deadlocks Explorer, since a main VCL thread has not been created anywhere. Do we need to manually create a main VCL thread (how?) - possibly inside another DLL - and re-route all our UI creation/updates/destruction through that DLL?

Remember, Explorer may be showing any number of windows containing our VCL window. Explorer may also be running as multiple independent processes, or as a single process, based on the configuration of the target system.

We have based our shell namespace extension on John Lam's starting point (as most Delphi shell namespace developers would know). Of course, as you can see in the final product, there have been heavy modifications to this starting point. John Lam never even once discusses the issue of the VCL being thread-unsafe in his slides and example projects.

We have also attempted to use multiple versions of ShellPlus components over the past decade. They have done some excellent work, but unfortunately, in our experience, even very elementary efforts based on their code have offered significantly worse results than our own code.

ShellPlus actually also offers the capability to use Explorer's own pre-defined host window, instead of creating a custom VCL window; while this might sidestep any VCL threading issues, in our experience even this has not been a viable solution - because the ShellPlus shell namespace extensions have always been less stable than our home brewed code, whether VCL windowed or not.

So first and foremost; this question is a theoretical one - can the VCL be used in a shell namespace extension that uses a VCL window inside Explorer as a process host?

If so, how is the issue of VCL thread safelessness to be handled in this scenario?

2
delphicomvcl
Have you actually checked whether or not your code that touches the GUI runs in more than one thread per process? Win32 has its own threading rules. Specifically windows have thread affinity. I'd expect that anytime explorer wants you to work with a window handle, you'd always get called in the same thread. In other words, I question whether or not thread affinity is in fact your problem. – David Heffernan
That's nice if Explorer is careful about thread affinity, but what happens when multiple Explorer windows are showing with multiple copies of our VCL window in them? The VCL is not thread safe, so there will still be the problem of multiple threads doing GUI work inside the same process - instead of a single thread handling all of the GUI work. Indeed Explorer creates multiple threads that call the shell namespace extension. – user2118012
OK, at that point you are hosed. – David Heffernan
So does this mean Delphi at large is a no-go for building shell namespace extensions, at least as far as the GUI is concerned? – user2118012
Not at all. If you can do it in C++, you can do it in Delphi. But the VCL sounds like it's not an option. – David Heffernan

2 Answers

4
votes

The VCL is indeed not thread safe. Running outside a VCL application doesn't offer relief from that rule. Lacking a main VCL thread only makes it worse; VCL controls expect to be running in the main thread's context, and there is no such thread, nor can there be, in general. The controls you're using access various global variables without any synchronization protection, and there's nothing you can do about that short of patching all the units that any VCL control uses.

Use regular Windows API techniques, not VCL functions, for creating and manipulating windows inside Explorer.

You can solve your synchronization hangs by making sure something checks the queue when told, but without a "main" thread, it's not clear what you'd be synchronizing with anyway, so Synchronize just might not be the right tool for the job.

0
votes

We have finally been able to solve this issue. The fixed shell namespace extensions will be featured in the upcoming MagicRAR 9.0. The problem was indeed with the VCL - our code was perfectly fine otherwise. For anyone else trying to build shell namespace extensions with Delphi:

Yes, you can build shell namespace extensions with Delphi. BUT - if you are using the VCL, you will experience countless random crashes and instability. AND - if you are are not using the VCL, there's little point to using Delphi. The good news is you won't need to patch the VCL sources. Just use a dedicated VCL thread.