Does the Azure WebJobs SDK support pushing TextWriter logs into App Insights?

Question

With the Azure WebJobs SDK, the process of adding logging to your functions is relatively straightforward: Add a TextWriter param to your triggered function, and write to it. That's it.

The SDK will will then associate and display these logs with their execution instances in the WebJobs Dashboard, which provides a relatively data-rich, yet frictionless view into the operationalization of your webjobs.

While this data is replicated into a user-accessible Azure Storage Blob Container, more custom code would be required to periodically push these logs to App Insights, which is undesirable.

Looking for ideas or solutions for how to push all logs pushed via the injected TextWriter to be pushed to AppInsights (or OMS, for that matter), complete with the webjobs execution/trigger instance metadata, thereby allowing a unified consumption experience for various log analytics.

Based on this Feature being tracked in the WebJobs SDK, I'm assuming that for now this is not possible? A long while back I looked into trying to inject my own TextWriter instance, but I would've had to fork the WebJobs SDK and use my customized assembly that changed a lot of architecture.

Answer 1

You can write a custom TraceWriter that sends log to AppInsights:

using System.Collections.Generic;
using System.Diagnostics;
using Microsoft.ApplicationInsights;
using Microsoft.Azure.WebJobs.Host;

public class AppInsightsTraceWriter : TraceWriter
{
    private readonly TelemetryClient _telemetryClient;

    public AppInsightsTraceWriter(TraceLevel level, TelemetryClient telemetryClient)
        : base(level)
    {
        _telemetryClient = telemetryClient;
    }

    public override void Trace(TraceEvent traceEvent)
    {
        var eventTelemetry = new EventTelemetry() {Name = "WebjobTraceEvent"};
        eventTelemetry.Properties.Add(traceEvent.Level.ToString(), traceEvent.ToString());
        _telemetryClient.TrackEvent(eventTelemetry);
    }
}

In this example, I inject the TelemetryClient class because you should only have one instance of the TelemetryClient class in your application.

So now you just need to configure the Jobhost to use your custom writer :

// Initialize the webjob configuration.
var config = new JobHostConfiguration();

// Only one instance of the telemetry client is needed
var telemetryClient = new TelemetryClient() {InstrumentationKey = "MyInstrumentationKey"};

// Add the app insights tracer for webjob logs/traces.
config.Tracing.Tracers.Add(new AppInsightsTraceWriter(TraceLevel.Info, telemetryClient));

// Detect when the webjob shut down
var cancellationToken = new WebJobsShutdownWatcher().Token;
cancellationToken.Register(() =>
{
    // Before shut down, flush the app insights client.
    telemetryClient.Flush();
});

new JobHost(config).RunAndBlock();

So if you have a function like that:

public static void ProcessQueueMessage([QueueTrigger("myqueue")] string logMessage, TextWriter log)
{
    log.WriteLine(logMessage);
}

Every time you use log.WriteLine, an event will be sent to App Insights.

Note: if this sample also logs from the JobHost are sent to AppInsights.

Answer 2

This is super old (not sure why SO decided to put it in the sidebar for me after this long) but for anyone else that stumbles upon this, app insights is now the recommended way to monitor webjob executions.

Check out the documentation here which steps through the process of connecting app insights to webjobs.

This link walks you through configuring the logging portion of a new webjobs project. Check through the earlier sections to make sure that you've got all the prerequisites. https://docs.microsoft.com/en-us/azure/app-service/webjobs-sdk-get-started#add-application-insights-logging

static async Task Main()
{
    var builder = new HostBuilder();
    builder.UseEnvironment(EnvironmentName.Development);
    builder.ConfigureWebJobs(b =>
            {
                b.AddAzureStorageCoreServices();
                b.AddAzureStorage();
            });
    builder.ConfigureLogging((context, b) =>
            {
                b.AddConsole();

                // If the key exists in settings, use it to enable Application Insights.
                string instrumentationKey = context.Configuration["APPINSIGHTS_INSTRUMENTATIONKEY"];
                if (!string.IsNullOrEmpty(instrumentationKey))
                {
                    b.AddApplicationInsightsWebJobs(o => o.InstrumentationKey = instrumentationKey);
                }
            });
    var host = builder.Build();
    using (host)
    {
        await host.RunAsync();
    }
}

Answer 3

I will share the detailed steps to use Application Insights in Azure Web Job, please refer to it.

1. Create an new Azure Application Insights in Azure portal
2. Create Azure Web Job project in Visual Studio and install Microsoft.ApplicationInsights

3. Set the instrumentation key and send telemetry

   public static void ProcessQueueMessage([QueueTrigger("queuename")] string message, TextWriter log)
   {
   
       TelemetryClient tc = new TelemetryClient();
   
       tc.InstrumentationKey = "key copied from Azure portal";
       tc.TrackTrace(message);
   
       tc.Flush();
   
       //log.WriteLine(message);
   }
   

This documentation explained how to monitor usage and performance in Windows Desktop apps, you could refer to it to know how to use Azure Application Insights in non-web application. Besides, ApplicationInsights.Helpers.WebJobs could be also helpful.