How to exclude Exception and Error logging from sampling in Application Insights in .NET Core 2.1?

Question

I'm running a web api with these specs:

ASP.NET Core 2.1
Microsoft.ApplicationInsights.AspNetCore 2.13.1
Hosted in Azure App Service.

In Startup.cs ConfigureServices I have added:

services.AddApplicationInsightsTelemetry();

_loggerFactory.AddAzureWebAppDiagnostics();

I've setup custom exception handler in the Startup.cs:

if (env.IsDevelopment())
{
    app.UseDeveloperExceptionPage();
}
else
{
    app.UseCustomExceptionHandler(telemetryClient, _loggerFactory);
}

Inside this CustomExceptionHandler I try to log the Exception like:

var logger = loggerFactory.CreateLogger("Unhandled Exception");
logger.LogError(ex, errorId);

var telemetryProperties = new Dictionary<string, string>();
telemetryProperties.Add("errorId", errorId);
telemetryProperties.Add("traceIdentifier", context.TraceIdentifier);

telemetryClient.TrackException(ex, properties: telemetryProperties);

With this configuration in place, not all Exceptions or Error Logs reached the Log Analytics buckets. So I have found this configuration for Application Insights:

        var builder = aiTelemetryConfiguration.DefaultTelemetrySink.TelemetryProcessorChainBuilder;
        builder.UseAdaptiveSampling(excludedTypes: "Trace;Exception");
        builder.Build();

Here I exclude Trace and Exception from the Adaptive Sampling.

Currently this configuration is on production. It's handling +/- 50k requests a minute. But the exception bucket stays empty.

I noticed these messages between the Traces:

AI (Internal): [Microsoft-ApplicationInsights-Core] [msg=Log Error];[msg=Exception while initializing Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.AspNetCoreEnvironmentTelemetryInitializer, exception message - System.ArgumentException: The key already existed in the dictionary. at System.Collections.Concurrent.ConcurrentDictionary`2.System.Collections.Generic.IDictionary.Add(TKey key, TValue value) at Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.AspNetCoreEnvironmentTelemetryInitializer.Initialize(ITelemetry telemetry) at Microsoft.ApplicationInsights.TelemetryClient.Initialize(ITelemetry telemetry)]

and

AI: A Metric Extractor detected a telemetry item with SamplingPercentage < 100. Metrics Extractors should be used before Sampling Processors or any other Telemetry Processors that might filter out Telemetry Items. Otherwise, extracted metrics may be incorrect.

To be clear, I'm looking at these locations:

Azure Application Insights -> Search -> Trace|Custom Event|Exception
Log Analytics with this query:
```
 exceptions
 | order by timestamp desc
```

Is this at least the correct way to disable sampling?

Many thanks in advance.

The errors do get logged in the Blob logging: 2020-03-26 14:04:45.718 +00:00 [Error] Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware: An unhandled exception has occurred while executing the request. System.DivideByZeroException: Attempted to divide by zero. But this error is not shown in AI. — JonHendrix
Exceptions are also visible in the Application Insights Live Metrics dashboard — JonHendrix

JonHendrix JonHendrix · Accepted Answer · 2020-03-31T12:54:05

I'm now adding this TelemetryInitializer:

public class ExceptionTelemetrySamplingFilter : ITelemetryInitializer
{
    public void Initialize(ITelemetry telemetry)
    {
        if (telemetry is ExceptionTelemetry)
        {
            ((ISupportSampling)telemetry).SamplingPercentage = 100;
        }
    }
}

And registered it:

services.AddSingleton<ITelemetryInitializer, ExceptionTelemetrySamplingFilter>();

It's from the official documentation about sampling. And I removed the added configurations from startup.

I'll keep you posted on improvements.

How to exclude Exception and Error logging from sampling in Application Insights in .NET Core 2.1?

1 Answers