• Stars
    star
    219
  • Rank 180,227 (Top 4 %)
  • Language
    C#
  • License
    Apache License 2.0
  • Created over 9 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A Serilog sink that writes events to Microsoft Azure Application Insights

Serilog.Sinks.ApplicationInsights NuGet Version

A sink for Serilog that writes events to Microsoft Application Insights. This sink comes with several defaults that send Serilog LogEvent messages to Application Insights as either EventTelemetry or TraceTelemetry.

Configuring

The simplest way to configure Serilog to send data to a Application Insights dashboard via instrumentation key is to use current active telemetry configuration which is already initialised in most application types like ASP.NET Core, Azure Functions etc.:

var log = new LoggerConfiguration()
    .WriteTo.ApplicationInsights(TelemetryConfiguration.Active, TelemetryConverter.Traces)
    .CreateLogger();

.. or as EventTelemetry:

var log = new LoggerConfiguration()
    .WriteTo.ApplicationInsights(TelemetryConfiguration.Active, TelemetryConverter.Events)
    .CreateLogger();

You can also pass an instrumentation key and this sink will create a new TelemetryConfiguration based on it, however it's actively discouraged compared to using already initialised telemetry configuration, as your telemetry won't be properly correlated.

Note: Whether you choose Events or Traces, if the LogEvent contains any exceptions it will always be sent as ExceptionTelemetry.

TelemetryConfiguration.Active is deprecated in the App Insights SDK for .NET Core, what do I do?

The singleton TelemetryConfiguration.Active has been deprecated in the Application Insights SDK on .NET Core in favor of dependency injection pattern .

Therefore, now we need to pass in the TelemetryConfiguration instance that was added either by services.AddApplicationInsightsTelemetryWorkerService() (if you're developing a non-http application) or services.AddApplicationInsightsTelemetry() (if you're developing an ASP.Net Core application) during Startup in ConfigureServices.

Log.Logger = new LoggerConfiguration()
    .WriteTo.ApplicationInsights(
        serviceProvider.GetRequiredService<TelemetryConfiguration>(),
	TelemetryConverter.Traces)
    .CreateLogger();

However, you probably want to setup your Logger as close to the entry point of your application as possible, so that any startup errors can be caught and properly logged. The problem is that now we're in a chicken-and-egg situation: we want to setup the logger early, but we need the TelemetryConfiguration which still haven't been added to our DI container.

Luckily from version 4.0.x of the Serilog.Extensions.Hosting we have the possibility to configure a bootstrap logger to capture early errors, and then change it using DI dependant services once they are configured.

// dotnet add package serilog.extensions.hosting -v 4.0.0-*

public static class Program
{
    public static void Main(string[] args)
    {
        Log.Logger = new LoggerConfiguration()
            .WriteTo.Console()
            .CreateBootstrapLogger();

        try
        {
            CreateHostBuilder(args).Build().Run();
        }
        catch (Exception ex)
        {
            Log.Fatal(ex, "An unhandled exception occurred during bootstrapping");
        }
        finally
        {
            Log.CloseAndFlush();
        }
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .UseSerilog((context, services, loggerConfiguration) => loggerConfiguration
                .WriteTo.ApplicationInsights(
		    services.GetRequiredService<TelemetryConfiguration>(),
		    TelemetryConverter.Traces))
            .ConfigureWebHostDefaults(webBuilder => { webBuilder.UseStartup<Startup>(); });
}

Configuring with ReadFrom.Configuration()

Configuring in code, as shown above, is recommended because the existing TelemetryClient can be injected.

The following configuration shows how to create an ApplicationInsights sink with ReadFrom.Configuration(configuration).

{
  "Serilog": {
    "Using": [
      "Serilog.Sinks.ApplicationInsights"
    ],
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft": "Warning",
        "System": "Warning"
      }
    },
    "WriteTo": [
      {
        "Name": "ApplicationInsights",
        "Args": {
          "connectionString": "[your connection string here]",
          "telemetryConverter":
	    "Serilog.Sinks.ApplicationInsights.TelemetryConverters.TraceTelemetryConverter, Serilog.Sinks.ApplicationInsights"
        }
      }
    ],
    "Enrich": [ "FromLogContext" ],
    "Properties": {
      "Application": "Sample"
    }
  }
}

The telemetryConverter has to be specified with the full type name and the assembly name.

A connectionString can be omitted if it's supplied in the APPLICATIONINSIGHTS_CONNECTION_STRING environment variable.

What does the sink submit?

By default, trace telemetry submits:

  • rendered message in trace's standard message property.
  • severity in trace's standard severityLevel property.
  • timestamp in trace's standard timestamp property.
  • messageTemplate in customDimensions.
  • custom log properties as customDimensions.

Event telemetry submits:

  • message template as event name.
  • renderedMessage in customDimensions.
  • timestamp in event's standard timestamp property.
  • custom log properties as customDimensions.

Exception telemetry submits:

  • exception as standard AI exception.
  • severity in trace's standard severityLevel property.
  • timestamp in trace's standard timestamp property.
  • custom log properties as customDimensions.

Note that log context properties are also included in customDimensions when Serilog is configured with .Enrich.FromLogContext().

How custom properties are logged?

By default custom properties are converted to compact JSON, for instance:

var position = new { Latitude = 25, Longitude = 134 };
var elapsedMs = 34;
var numbers = new int[] { 1, 2, 3, 4 };

Logger.Information("Processed {@Position} in {Elapsed:000} ms., str {str}, numbers: {numbers}",
    position, elapsedMs, "test", numbers);

will produce the following properties in customDimensions:

Property Value
Elapsed 34
Position {"Latitude":25,"Longitude":134}
numbers [1,2,3,4]

This is a breaking change from v2 which was producing these properties:

Property Value
Elapsed 34
Position.Latitude 25
Position.Longitude 134
numbers.0 1
numbers.1 2
numbers.2 3
numbers.3 4

You can revert the old behavior by overriding standard telemetry formatter, for instance:

private class DottedOutTraceTelemetryConverter : TraceTelemetryConverter
{
    public override IValueFormatter ValueFormatter => new ApplicationInsightsDottedValueFormatter();
}

Customizing

Additionally, you can also customize whether to send the LogEvents at all, if so which type(s) of Telemetry to send and also what to send (all or no LogEvent properties at all) by passing your own ITelemetryConverter instead of TelemetryConverter.Traces or TelemetryConverter.Events by either implementing your own ITelemetryConverter or deriving from TraceTelemetryConverter or EventTelemetryConverter and overriding specific bits.

Log.Logger = new LoggerConfiguration()
    .WriteTo.ApplicationInsights(configuration, new CustomConverter())
    .CreateLogger();
// ...

private class CustomConverter : TraceTelemetryConverter
{
    public override IEnumerable<ITelemetry> Convert(LogEvent logEvent, IFormatProvider formatProvider)
    {
        // first create a default TraceTelemetry using the sink's default logic
        // .. but without the log level, and (rendered) message (template) included in the Properties
        foreach (ITelemetry telemetry in base.Convert(logEvent, formatProvider))
        {
            // then go ahead and post-process the telemetry's context to contain the user id as desired
            if (logEvent.Properties.ContainsKey("UserId"))
            {
                telemetry.Context.User.Id = logEvent.Properties["UserId"].ToString();
            }
            // post-process the telemetry's context to contain the operation id
            if (logEvent.Properties.ContainsKey("operation_Id"))
            {
                telemetry.Context.Operation.Id = logEvent.Properties["operation_Id"].ToString();
            }
            // post-process the telemetry's context to contain the operation parent id
            if (logEvent.Properties.ContainsKey("operation_parentId"))
            {
                telemetry.Context.Operation.ParentId = logEvent.Properties["operation_parentId"].ToString();
            }
            // typecast to ISupportProperties so you can manipulate the properties as desired
            ISupportProperties propTelematry = (ISupportProperties)telemetry;

            // find redundant properties
            var removeProps = new[] { "UserId", "operation_parentId", "operation_Id" };
            removeProps = removeProps.Where(prop => propTelematry.Properties.ContainsKey(prop)).ToArray();

            foreach (var prop in removeProps)
            {
                // remove redundant properties
                propTelematry.Properties.Remove(prop);
            }

            yield return telemetry;
        }
    }
}

If you want to skip sending a particular LogEvent, just return null from your own converter method.

Customising included properties

The easiest way to customise included properties is to subclass one of the ITelemetryConverter implementations. For instance, let's include renderedMessage in event telemetry:

private class IncludeRenderedMessageConverter : EventTelemetryConverter
{
    public override void ForwardPropertiesToTelemetryProperties(LogEvent logEvent, 
        ISupportProperties telemetryProperties, IFormatProvider formatProvider)
    {
        base.ForwardPropertiesToTelemetryProperties(logEvent, telemetryProperties, formatProvider,
            includeLogLevel: false,
            includeRenderedMessage: true,
            includeMessageTemplate: false);
    }
}

How, When and Why to Flush Messages Manually

Or: Where did my Messages go?

As explained by the Application Insights documentation , the default behaviour of the AI client is to buffer messages and send them to AI in batches whenever the client seems fit. However, this may lead to lost messages when your application terminates while there are still unsent messages in said buffer.

You can control when AI shall flush its messages, for example when your application closes:

  1. Create a custom TelemetryClient and hold on to it in a field or property:
// private TelemetryClient _telemetryClient;

// ...
_telemetryClient = new TelemetryClient()
            {
                InstrumentationKey = "<My AI Instrumentation Key>"
            };
  1. Use that custom TelemetryClient to initialize the Sink:
var log = new LoggerConfiguration()
    .WriteTo
	.ApplicationInsights(_telemetryClient, TelemetryConverter.Events)
    .CreateLogger();
  1. Call .Flush() on the TelemetryClient whenever you deem necessary, i.e. Application Shutdown:
_telemetryClient.Flush();

// The AI documentation mentions that calling `Flush()` *can* be asynchronous and non-blocking so
// depending on the underlying Channel to AI you might want to wait some time
// specific to your application and its connectivity constraints for the flush to finish.

await Task.Delay(1000);

// or 

System.Threading.Thread.Sleep(1000);

Including Operation Id

Application Insight's operation id is pushed out if you set operationId LogEvent property. If it's present, AI's operation id will be overridden by the value from this property.

This can be set like so:

public class OperationIdEnricher : ILogEventEnricher
{
    public void Enrich(LogEvent logEvent, ILogEventPropertyFactory propertyFactory)
    {
        if (logEvent.Properties.TryGetValue("RequestId", out var requestId))
        {
            logEvent.AddPropertyIfAbsent(new LogEventProperty("operationId", requestId));
        }
    }
}

Including Version

Application Insight supports component version and is pushed out if you set version log event property. If it's present, AI's operation version will include the value from this property.

Using with Azure Functions

Azure functions has out of the box integration with Application Insights, which automatically logs functions execution start, end, and any exception. Please refer to the original documenation on how to enable it.

This sink can enrich AI messages, preserving operation_Id and other context information which is already provided by functions runtime. The easiest way to configure Serilog in this case is to use the injected TelemetryClient which should be automatically configured by the environment through the APPLICATIONINSIGHTS_CONNECTION_STRING appsetting. You can, for instance, initialise logging in the static constructor:

[assembly: FunctionsStartup(typeof(MyFunctions.Startup))]
namespace MyFunctions
{
    public class Startup : FunctionsStartup
    {
        public override void Configure(IFunctionsHostBuilder builder)
        {
            builder.Services.AddSingleton<ILoggerProvider>((sp) => 
            {
                Log.Logger = new LoggerConfiguration()
                    .Enrich.FromLogContext()
                    .WriteTo.ApplicationInsights(sp.GetRequiredService<TelemetryClient>(), TelemetryConverter.Traces)
                    .CreateLogger();
                return new SerilogLoggerProvider(Log.Logger, true);
            });
        }
    }
}

Copyright © 2022 Serilog Contributors - Provided under the Apache License, Version 2.0.

See also: Serilog Documentation

More Repositories

1

serilog-sinks-elasticsearch

A Serilog sink that writes events to Elasticsearch
C#
434
star
2

serilog-ui

Simple Serilog log viewer UI for several sinks.
C#
207
star
3

serilog-sinks-grafana-loki

A Serilog sink sending log events to Grafana Loki
C#
188
star
4

Serilog.Enrichers.Sensitive

A Serilog LogEvent enricher that masks sensitive data
C#
112
star
5

serilog-sinks-richtextbox

A Serilog sink that writes log events to a WPF RichTextBox control with colors and theme support
C#
97
star
6

serilog-enrichers-clientinfo

Enrich logs with client IP, correlation id and HTTP request headers.
C#
90
star
7

serilog-sinks-graylog

Serilog sink for graylog
C#
81
star
8

serilog-sinks-notepad

A Serilog sink that writes log events to Notepad as text or JSON
C#
61
star
9

Serilog.Sinks.Postgresql.Alternative

Serilog.Sinks.Postgresql.Alternative is a library to save logging information from https://github.com/serilog/serilog to https://www.postgresql.org/.
C#
59
star
10

SerilogSinksInMemory

In-memory sink for Serilog to use for testing
C#
53
star
11

serilog-sinks-splunk

A Serilog sink that writes to Splunk
C#
45
star
12

serilog-sinks-slack

A simple (yet customizable) Slack logging sink for Serilog
C#
41
star
13

serilog-sinks-azuretablestorage

A Serilog sink that writes events to Azure Table Storage
C#
40
star
14

serilog-sinks-sentry

A Sentry sink for Serilog
C#
36
star
15

Serilog.Sinks.Telegram.Alternative

Serilog.Sinks.Telegram.Alternative is a library to save logging information from Serilog to Telegram.
C#
35
star
16

Serilog.Sinks.MicrosoftTeams.Alternative

Serilog.Sinks.MicrosoftTeams.Alternative is a library to save logging information from Serilog to Microsoft Teams.
C#
30
star
17

serilog-sinks-slackclient

Slack Sink for Serilog
C#
27
star
18

serilog-enrichers-globallogcontext

A Serilog Enricher for adding properties to all log events in your app
C#
25
star
19

serilog-diagnostics-tracelistener

A System.Diagnostics.TraceListener that writes trace data into Serilog
C#
25
star
20

Serilog.Sinks.AmazonS3

Serilog.Sinks.AmazonS3 is a library to save logging information from Serilog to Amazon S3. The idea there was to upload log files to Amazon S3 to later evaluate them with Amazon EMR services.
C#
21
star
21

serilog-sinks-teams

A Serilog event sink that writes to Microsoft Teams
C#
18
star
22

Serilog-Sinks-Discord

Serilog discord sink
C#
17
star
23

Serilog.Sinks.Network

A serilog network sink. Designed with logstash and the Elastic stack in mind
C#
16
star
24

home

This is the hub for all the projects that are part of the Serilog Contrib Organization
16
star
25

Serilog.Sinks.Logz.Io

C#
15
star
26

serilog-sinks-exceldnalogdisplay

A Serilog sink that writes events to Excel-DNA LogDisplay
C#
13
star
27

serilog-sinks-azureeventhub

A Serilog sink that writes events to Azure EventHubs
C#
13
star
28

serilog-sinks-rollbar

A Serilog sink which writes events to Rollbar
C#
10
star
29

Serilog.Logfmt

Simple Serilog formatter to output Logfmt
C#
9
star
30

serilog-enrichers-exceldna

A Serilog Enricher with properties from Excel-DNA add-ins
C#
6
star
31

serilog-enrichers-memory

An enricher for Serilog which outputs memory usage
C#
6
star
32

serilog-sinks-apachekafka

A sink for Serilog that writes events to Kafka
C#
6
star
33

serilog-sinks-msbuild

An MSBuild sink for Serilog.
C#
5
star
34

serilog-settings-xml

Xml file based configuration for Serilog (https://serilog.net)
C#
5
star
35

serilog-sinks-OnePageSink

Sink that displays real-time messages from Serilog in a separate web page. Uses SignalR as a transport. Stores nothing, uses no memory. If you refresh your page you will lose your messages
C#
4
star
36

serilog-formatting-log4net

Format Serilog events in log4net or log4j compatible XML format
C#
4
star
37

Serilog.Sinks.DelegatingText

A Serilog sink to emit formatted log events to a delegate.
C#
3
star
38

brand

Guide and reference to designers, writers, and developers to create consistent, on-brand content for Serilog
2
star
39

serilog-sinks-youtrack

A Serilog sink that writes events to YouTrack
C#
2
star
40

Serilog.Enrichers.ActivityTags

C#
1
star
41

Serilog.Sinks.Logcat

Android logcat sink extension for the Serilog logger
C#
1
star
42

Serilog.Enrichers.ApplicationName

PowerShell
1
star
43

serilog-sinks-amqp-batching

C#
1
star
44

serilog-settings-xml2

A Serilog settings provider that reads from XML sources with zero dependencies and full configuration support
C#
1
star