2
votes

We have just started using Service Fabric and the only pain point so far has been ETW with WAD, which always seems to log out with missing data (message, eventmessage.)

Our experience so far has that it always works in visual studio (sometimes you have to add the provider name) and that it rarely works when deployed to a cluster in Azure. When it does work in Azure - versioning & updating a function on the event source or adding another will then log out with empty data points.

This is the section we have in our ARM script for ETW/WAD which we are deploying using the Azure CLI.

"name": "[concat('VMDiagnosticsVmExt','_vmNodeType0Name')]",
"properties": {
    "type": "IaaSDiagnostics",
    "autoUpgradeMinorVersion": true,
    "protectedSettings": {
        "storageAccountName": "[parameters('applicationDiagnosticsStorageAccountName')]",
        "storageAccountKey": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('applicationDiagnosticsStorageAccountName')),'2015-05-01-preview').key1]",
        "storageAccountEndPoint": "https://core.windows.net/"
    },
    "publisher": "Microsoft.Azure.Diagnostics",
    "settings": {
        "WadCfg": {
            "DiagnosticMonitorConfiguration": {
                "overallQuotaInMB": "50000",
                "EtwProviders": {
                    "EtwEventSourceProviderConfiguration": [
                        {
                            "provider": "Microsoft-ServiceFabric-Actors",
                            "scheduledTransferKeywordFilter": "1",
                            "scheduledTransferPeriod": "PT5M",
                            "DefaultEvents": {
                                "eventDestination": "ServiceFabricReliableActorEventTable"
                            }
                        },
                        {
                            "provider": "Microsoft-ServiceFabric-Services",
                            "scheduledTransferPeriod": "PT5M",
                            "DefaultEvents": {
                                "eventDestination": "ServiceFabricReliableServiceEventTable"
                            }
                        },
                        {
                            "provider": "Company-Project-API",
                            "scheduledTransferPeriod": "PT1M",
                            "DefaultEvents": {
                                "eventDestination": "ApiEventTable"
                            }
                        }
                    ],
                    "EtwManifestProviderConfiguration": [
                        {
                            "provider": "cbd93bc2-71e5-4566-b3a7-595d8eeca6e8",
                            "scheduledTransferLogLevelFilter": "Information",
                            "scheduledTransferKeywordFilter": "4611686018427387904",
                            "scheduledTransferPeriod": "PT5M",
                            "DefaultEvents": {
                                "eventDestination": "ServiceFabricSystemEventTable"
                            }
                        }
                    ]
                }
            }
        },
        "StorageAccount": "[parameters('applicationDiagnosticsStorageAccountName')]"
    },
    "typeHandlerVersion": "1.5"
}

This is our EventSource, though we have tried a ton of variations.

using Microsoft.Diagnostics.Tracing;

namespace Project.API

[EventSource(Name = "Company-Project-API")]
public sealed class ApiEventSource : EventSource
{
    public static ApiEventSource Current = new ApiEventSource();

    [Event(1, Level = EventLevel.Informational, Message = "{0}", Version = 1)]
    public void Log(string message)
    {
        this.WriteEvent(1, message);
    }
}

This is what we get in WAD every time.

Logs in storage explorer

Running .NET 4.5.2 / .net core.

Please assist.

EDIT - Okay, we've had some success moving up to .NET 4.6 - it looks as if the message payload is being logged out. All we're missing now is the eventmessage field. It also seems that the "message" field is always null now in VS.

EDIT2 - It seems that the message field is missing when using EventSourceSettings.EtwSelfDescribingEventFormat as a constructor argument to your event source like below. This seems to be the case in VS & in WAD.

public sealed class ApiEventSource : EventSource 
{
    public ApiEventSource() : base(EventSourceSettings.EtwSelfDescribingEventFormat) {}
}

At the moment I can either choose between no eventmessage (Self Describing) or not being able to version the methods (even with incrementing the attribute, empty lines are still dumped into WAD when using manifest style.

1
Can you share an example of a message that does not get picked up by WAD? Any special characters inside?Karol Z
@KarolZ In our case it was all messages - some were simply "Performing X with Y" where Y would be the value of a simple type. Some would be fully qualified urls which I read would be better html encoded. We've switched over to 4.6 and are now using the self describing event source which seems a lot better - now we're just missing the eventmessage and the activityId - we tried using the Start/Stop as mentioned here blogs.msdn.microsoft.com/vancem/2015/09/14/… but nothing seems to be auto generated.Nosmadas
The task activity IDs require proper enabling and decoding on the consumer side. VS diagnostics data viewer currently does not do this, but we have a release of the Fabric tools in the works that will fix it. I will inquire about support for this in WAD and post in the comments hereKarol Z
Update: WAD currently does not support this. I asked the WAD team to consider it for one of their upcoming releases. No ETA at this timeKarol Z
@NickDarvey Switching to 4.6 and using the self describing event format solved those two things but you lose any data that would be in the eventmessage field.Nosmadas

1 Answers

1
votes

Self-describing ETW is not currently designed to support the EventAttribute.Message property. Message is a manifest based concept (that is they are not logged in the event, but rather placed in the manifest. Since Self-describing ETW does not have a manifest, the Message property effectively gets ignored.

One could imagine extending the Meta-data associated with Self-describing ETW events so it could hold the Message string, but that does not currently exist and would require a change to ETW.

Simply embedding the message in the payload is the expected way to avoid the issue.