Will Application Insights Sampling sample away exceptions?

Question

The application insights sampling documentation (https://docs.microsoft.com/en-us/azure/application-insights/app-insights-sampling) states the following:

"For example, if for a failed request your app sends additional telemetry items (such as exception and traces logged from this request), sampling will not split this request and other telemetry. It either keeps or drops them all together."

It's hard to imagine but does this mean that some exceptions or better requests that caused an exception, might just get ignored and not logged if the sampling algorythm needs to reduce the amount of logging being done? I find it hard to place this in the correct context, just getting started with application insights and trying to figure out all the possibilities. I understand this is very valuable to filter out a percentage of telemetry for performance indicators etc, but for actual webapi requests for example..?

I've read this question but my doubt still remains: Azure App Insights Sampling (ItemCount)

ZakiMa ZakiMa · Accepted Answer · 2018-06-07T03:31:29

With default configuration Application Insights will decide to either sample or not at the beginning of a request and then correspondingly either will collect everything or drop everything. This approach provides some guarantees such as if you're looking at some transaction then you see it entirely, including some downstream components.

You're right that with this approach it is possible miss some errors. Our experience tells that even for high scale applications (5000 RPS / instance) such approach still captures errors if they keep occurring.

There are two things you can to mitigate this:

Collect all exceptions (still will not collect failed requests). For this you can modify the following adaptive sampling section from this:

<Add Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryProcessor, Microsoft.AI.ServerTelemetryChannel">
    <MaxTelemetryItemsPerSecond>5</MaxTelemetryItemsPerSecond>
    <ExcludedTypes>Event</ExcludedTypes>
</Add>
<Add Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryProcessor, Microsoft.AI.ServerTelemetryChannel">
    <MaxTelemetryItemsPerSecond>5</MaxTelemetryItemsPerSecond>
    <IncludedTypes>Event</IncludedTypes>
</Add>

To:

    <Add Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryProcessor, Microsoft.AI.ServerTelemetryChannel">
        <MaxTelemetryItemsPerSecond>5</MaxTelemetryItemsPerSecond>
        <ExcludedTypes>Event,Exception</ExcludedTypes>
    </Add>
  <Add Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryProcessor, Microsoft.AI.ServerTelemetryChannel">
    <MaxTelemetryItemsPerSecond>100</MaxTelemetryItemsPerSecond>
    <IncludedTypes>Exception</IncludedTypes>
  </Add>
  <Add Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryProcessor, Microsoft.AI.ServerTelemetryChannel">
        <MaxTelemetryItemsPerSecond>5</MaxTelemetryItemsPerSecond>
        <IncludedTypes>Event</IncludedTypes>
    </Add>

Note, Application Insights will not necessarily collect failed requests.

It is possible to write own sampling processor (TelemetryProcessor). For instance, collect failed requests, dependencies, exceptions and for the rest apply own sampling logic.

Will Application Insights Sampling sample away exceptions?

1 Answers