WiX Toolset - why is CustomActionData collection empty for an immediate CustomAction after InstallFinalize?

Question

I am using WiX v3.14 to build a .Net Core installer. I have a CustomAction - UpdateJsonAppSettings - written in C#, which is intended to update the appsettings.json file which is part of the installation (with a database connection string built from fields entered by the user performing the install). When I schedule the CustomAction as immediate to run After="InstallFinalize", the CustomActionData collection which has been scheduled with Before="UpdateJsonAppSettings", the session.CustomActionData collection is empty.

<CustomAction Id="SetCustomActionData"
              Return="check"
              Property="UpdateJsonAppSettings"
              Value="connectionString=[CONNECTION_STRING_FORMATTED];filepath=[PATH_TO_APPSETTINGS_JSON]" />

<CustomAction Id="UpdateJsonAppSettings"
              BinaryKey="CustomActions"
              DllEntry="UpdateJsonAppSettings" 
              Execute="immediate" 
              Return="ignore" />

<InstallExecuteSequence>
  <Custom Action="ConnectionString" Before="SetCustomActionData" />
  <Custom Action="SetCustomActionData" Before="UpdateJsonAppSettings" />
  <Custom Action="UpdateJsonAppSettings" After="InstallFinalize">NOT Installed AND NOT PATCH</Custom>
</InstallExecuteSequence>

The Session.Log:

Session.Log:
MSI (s) (C8:8C) [11:15:44:363]: Doing action: SetCustomActionData
Action 11:15:44: SetCustomActionData. 
Action start 11:15:44: SetCustomActionData.
MSI (s) (C8:8C) [11:15:44:379]: PROPERTY CHANGE: Adding UpdateJsonAppSettings property. Its value is 'connectionString=Data Source=localhost\SQLEXPRESS;;Initial Catalog=DB;;User Id=dbuser;;Password=dbuserpassword;;MultipleActiveResultSets=true;;App=EntityFramework;filepath=[#appSettings]'.
Action ended 11:15:44: SetCustomActionData. Return value 1.

[SNIP]

Action start 11:15:44: UpdateJsonAppSettings.
MSI (s) (C8:B8) [11:15:44:382]: Invoking remote custom action. DLL: C:\windows\Installer\MSI95BF.tmp, Entrypoint: UpdateJsonAppSettings
SFXCA: Extracting custom action to temporary directory: C:\TEMP\MSI95BF.tmp-\
SFXCA: Binding to CLR version v4.0.30319
Calling custom action CustomActions!CustomActions.CustomAction.UpdateJsonAppSettings
Session.CustomActionData.Count(): 0
Exception thrown by custom action:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.

When I modify CustomAction UpdateJsonAppSettings to Execute="deferred" and schedule it After="InstallFiles", CustomActionData is correctly set and is available tothe CustomAction but the installation fails with a File not found exception. Scheduled Before="InstallFinalize" fails with the same exception.

Session.Log:
Calling custom action CustomActions!CustomActions.CustomAction.UpdateJsonAppSettings
Session.CustomActionData.Count(): 2
key: connectionString, value: Data Source=localhost\SQLEXPRESS;Initial Catalog=DB;User Id=dbuser;Password=dbuserpassword;MultipleActiveResultSets=true;App=EntityFramework
key: filepath, value: C:\inetpub\wwwroot\ServiceApi\appsettings.json
UpdateJsonAppSettings() returned: NotExecuted; _result: File not found:C:\inetpub\wwwroot\ServiceApi\appsettings.json; filepath: C:\inetpub\wwwroot\ServiceApi\appsettings.json; connectionString: Data Source=localhost\SQLEXPRESS;Initial Catalog=DB;User Id=dbuser;Password=dbuserpassword;MultipleActiveResultSets=true;App=EntityFramework

This looks like a Catch-22 situation. Any help gratefully received.

PS - for some reason my original post ended up on META-StackExchange?

Answer 1

A big thanks for the suggestion Stein - it eventually allowed me to find a solution, though that was by no means easy. I am not sure how to put this explanation up as well as to set your comment to be the correct answer.

What I found was that setting the CustomDataAction element that defined the path to appsettings.json in a CustomAction, whilst being displayed correctly when sent to the session.Log, was NOT what was actually being set (see WARNING in-line).

    [CustomAction]
    public static ActionResult SetCustomActionData(Session session)
    {
        CustomActionData _data = new CustomActionData();
        // ..escape single ';'
        string _connectionString = session["CONNECTION_STRING"];
        _connectionString.Replace(";", ";;");
        _data["connectionString"] = _connectionString;
        // ..correctly output in install log
        session.Log(string.Format("SetCustomActionData() setting _connectionString: {0}", _connectionString));
        // Property set to [#appSettings] in Product.wxs
        string _filePath = session["PATH_TO_APPSETTINGS_JSON"];
        _data["filepath"] = _filePath;
        // ..correctly output in install log
        session.Log(string.Format("SetCustomActionData() setting _filepath: {0}", _filePath));
        // ..set the CustomActionData programmatically
        session["UpdateJsonAppSettings"] = _data.ToString();

        return ActionResult.Success;
    }

    Install.log:
    [SNIP]
    SetCustomActionData() setting _connectionString: 'Data Source=localhost\SQLEXPRESS;;Initial Catalog=...'
    SetCustomActionData() setting _filepath: 'C:\inetpub\wwwroot\UServiceApi\appsettings.json'

    [CustomAction]
    public static ActionResult UpdateJsonAppSettings(Session session)
    {
        // ..as per Stein's suggestion
        MessageBox.Show("Attach run32dll.dll now");
        // ..correctly output to log (i.e. 2)
        session.Log(string.Format("Session.CustomActionData.Count(): {0}", session.CustomActionData.Count));
        // ..correctly output two key/value pairs to log
        foreach(string _key in session.CustomActionData.Keys)
            session.Log(string.Format("key: {0}, value: {1}", _key, session.CustomActionData[_key]));

        string _connectionString = session.CustomActionData["connectionString"];
        string _pathToAppSettings = session.CustomActionData["filepath"];
        // WARNING: _pathToAppSettings has reverted to the literal "[#appSettings]" - which of course triggers File not found Exception.
        ActionResult _retVal = UpdateJsonConnectionString(_connectionString, _pathToAppSettings, out string _result);
        // ..log failure
        if (_retVal != ActionResult.Success)
            session.Log(string.Format("UpdateJsonAppSettings() returned: {0}; _result: {1}; filepath: {2}; connectionString: {3}", 
                                      _retVal, _result, _pathToAppSettings, _connectionString));

        return _retVal;
    }

    Install.log:
    [SNIP]
    key: connectionString, value: Data Source=localhost\SQLEXPRESS;Initial Catalog=...
    key: filepath, value: C:\inetpub\wwwroot\UServiceApi\appsettings.json
    [SNIP]
    UpdateJsonAppSettings() returned: NotExecuted; _result: File not found:C:\inetpub\wwwroot\ServiceApi\appsettings.json...

I tried a number of different combinations of Properties and CustomAction implementations, but eventually found that the only way to correctly set CustomActionData to contain the 'real' filepath element was to set it in a Type 51 (I think) CustomAction in Product.wxs. No other combination that I tried worked.

Working code snippet from Product.wxs:

<Property Id="PATH_TO_APPSETTINGS_JSON" Value="[#appSettings]" />

<CustomAction Id="SetCustomActionData" 
              Return="check" 
              Property="UpdateJsonAppSettings" 
              Value="connectionString=[CONNECTION_STRING_FORMATTED];filepath=[#appSettings]" />// NOTE: cannot use filepath=[PATH_TO_APPSETTINGS_JSON] here

Conclusion: a possible WiX bug - the log was not telling the truth (it selectively 'translates' CustomActionData elements whilst the compiled installer code actually uses different values).