Telemetry sampling without affecting the errors/failures

3
votes

I want to log a percentage of the success calls in app insights. I came across this post https://docs.microsoft.com/en-us/azure/azure-monitor/app/sampling and I think Fixed-rate sampling is appropriate here. But does this affect all logging equally? Will some errors/failures no longer be logged?

I am looking for a solution that logs a percentage of the success calls, but keeps all failed requests/errors.

2
.netazureazure-application-insightstelemetry

2 Answers

1
votes

I don't think this is supported out of the box, but you can write your own ITelemetryProcessor.

See: https://docs.microsoft.com/en-us/azure/azure-monitor/app/api-filtering-sampling#filtering-itelemetryprocessor

Application Insights in .NET uses a chain of telemetry processors that you can use to filter telemetry, so you can write your own that checks the resultCode (I think that's what Application Insights calls the HTTP status code, but you'll have to double check) of a request telemetry object, and approves it if it's 500 (or 5xx) but only has a 10% chance of sending it if it's 2xx or 3xx. You can override the OKToSend() method to perform the above check on the ITelemetry input, and return true / false accordingly.

Maybe something like (I wrote this in the browser, it won't necessarily work flawlessly as-is):

// Approves 500 errors and 10% of other telemetry objects
private bool OKtoSend (ITelemetry telemetry)
{
    if (telemetry.ResponseCode == 500) {
        return true;
    } else {
        Random rnd = new Random();
        int filter = rnd.Next(1, 11);
        return filter == 1;
    }
}
1
votes

To exclude failed events from being subject to sampling, (while doing sampling for everything else) write a TelemetryInitializer with this logic.

public class PreventSamplingForFailedTelemetryInitializer: ITelemetryInitializer
{
  public void Initialize(ITelemetry telemetry)
  {
        if(failed)
        {
            // Set to 100, so that actual SamplingProcessors ignore this from sampling considerations.
            ((ISupportSampling)telemetry).SamplingPercentage = 100;
        }
   }
}

(Make sure to add this TelemetryInitializer to the TelemetryConfiguration)

Failed or not can be determined from RequestTelemetry and DependencyTelemetry from their `Success` field.

(the last one in FAQ sections has hints to answer your question https://docs.microsoft.com/en-us/azure/azure-monitor/app/sampling#frequently-asked-questions)