Cannot add Custom Action in Visual Studio 2010 installer project

Question

I converted a working project from Visual Studio 2008 to VS2010. The c++ solution builds a single executable and 3 DLLs, and a windows installer project.

After converting to VS2010, I received a build error "Unable to build custom action named 'Primary Output from xxxx (Active)' from project output group 'Primary Output' because the project output group does not have a key file."

I removed and re-added the output groups, then attempted to add a custom action. The executable output was not listed as a candidate for a custom action. I then noticed when I selected the executable "Primary Output" properties, that the "KeyOutput" property was set to (None) and was grayed out.

When revisiting the VS2008 version, the KeyOutput property was not grayed out. I found the error message on MSDN, with a useless fix:

"Remove the custom action and replace it with a custom action pointing to a project output group that has a key file."

Obviously the tech writer didn't know how to fix it either. Keep in mind the original build worked and installed correctly. I believe the root issue is why the KeyOutput property is disabled - but why? (When viewing "Outputs" it correctly shows the exe or dll in each project)

Answer 1

I had a similar problem and I figured out why the setup project could not find the key output files.

My project Output Directory was defined as $(SolutionDir)\bin\$(Platform). SolutionDir already has a trailing backslash so the \bin was placing a double backslash in the path. The project would build fine but the setup project couldn't resolve the key outputs. Setting the output to $(SolutionDir)bin\$(Platform)\ resolved the issue.

Answer 2

After further investigation, I found this is a bug in Visual Studio 2010.

If you specify an output name for a project that differs from the name of the project itself, it fails to do the right thing.

To reproduce this, change

Project properties->Linker->General->OutputFile from $(OutDir)$(TargetName)$(TargetExt) to be something like $(OutDir)foobar.exe

The project then changes the KeyOutput value to empty, and is not available for custom actions in installations. No workaround other than living with the default project naming convention.

Answer 3

Very late answer but hopefully it will save someone the 2 hours I spent banging my head on the wall with VS 2010 ( what an awful piece of software it is - slow to load and takes up 300 megs of working memory per devenv , and now this bug!!!!!????) here goes - Roger Dunn was correct is saying that it is the Project properties->Linker->General->OutputFile that causes the problem - if you have something like $(OutDir)foobar.exe the custom action barfs , if you change it back to $(OutDir)$(TargetName)$(TargetExt) it works again!

The work around is to change the macro $(TargetName) to foobar and then user $(OutDir)$(TargetName)$(TargetExt) if you want custom name for your exe.

$(TargetName) can be changed at - Project properties->Configuration Properties->general->TargetName.

But what a waste of time this VS 2010.