Sitemap

Step-by-Step Guide to Logs in C#: Best Practices and Tips

19 min readNov 17, 2024

--

Press enter or click to view image in full size

As you may guess from the title, we will talk about logs today.

The article is a big chunk to swallow 😅. Not to exhaust you with a long preambular, here’s a quick overview of the main concepts we’ll discuss, ensuring that both junior and senior developers will find valuable insights and something new to learn.

We will begin with a basic explanation of the core logging concepts:

  • Back to the roots
  • Benefits of Logging
  • Log Levels
  • Logging Points
  • Trace Identifier/Correlation Id
  • Log Distribution
  • Unstructured Logs Vs Structured Logs
  • Audit Logs

Next, we’ll explore how logging is implemented in C# and see some practical examples:

  • Fundamentals of Logging
  • Log Levels
  • Begin Scope
  • Structured Log
  • Logger in Unit Tests
  • Logging in ASP.NET Core

We’ll also take a brief look at Serilog and explore why it’s so widely used:

  • Serilog
  • Formatting
  • Minimum Level
  • Enrichers
  • Sinks
  • Logging in ASP.NET Core

Then, you will see how logs can be analyzed locally using different tools:

  • Monitoring tools
  • WatchDog.NET
  • Seq
  • ELK Stack (Elasticsearch, Logstash, Kibana)

Finally, we’ll discuss performance issues related to logs:

  • Performance Issues

Without any further ado, let’s begin

Back to the Roots

Before, we start let’s make sure you are in the loop.

Logging is the process of recording informational or error messages from an application to a log file or external system.

Press enter or click to view image in full size

Here is the simplest example of the logging:

public int Sum(int a, int b)
{
var sum = a + b;

// log to the file
File.AppendAllText("log.txt", $"Sum of {a} and {b} is {sum}");

return sum;
}

Benefits of Logging

It is always easy to debug the program locally. However, debugging is not an option when your application is live and handles millions of requests. This is why you need logs:

  • debugging and troubleshooting: logs provide a historical record of what happened in your program. Developers can trace through the log entries to identify the sequence of events leading up to a problem
  • error analysis: when errors occur, logs can capture detailed information about the error, including the stack trace, variables, and conditions at the time of the error. This information is invaluable for developers when addressing and fixing bugs
  • monitoring: you can track metrics, such as response times, resource usage, and error rates. Monitoring logs over time helps identify performance bottlenecks or areas that need optimization
  • auditing: it is important to keep records of done actions for security and compliance
  • user activity: you can monitor and analyze user interactions to deliver features they need

As you can see there are multiple benefits of having logs.

Log Levels

In this small example, you can see that we just added a message to the file.

public int Sum(int a, int b)
{
var sum = a + b;

// log to the file
File.AppendAllText("log.txt", $"Sum of {a} and {b} is {sum}");

return sum;
}

However, in the real world, you would have tons of messages. Some are vital, while others are just routine. To quickly identify the importance of the record, each log is assigned a category.

Log levels are categories used to classify the severity and importance of log messages. They help developers and system administrators filter and prioritize logs.

Common log levels include:

  • trace: detailed information, typically for diagnosing specific issues
  • debug: information useful for debugging, less granular than Trace
  • information: general information about the application’s normal behavior
  • warning: indicators of potential issues or unexpected events that don’t disrupt the operation
  • error: logs errors that affect functionality but don’t crash the application
  • critical: logs severe issues that cause application failure

Category can be just prepended to the message:

public int Sum(int a, int b)
{
var sum = a + b;

// log entry with Debug category
File.AppendAllText("log.txt", $"[DEBUG] Sum of {a} and {b} is {sum}");

return sum;
}

If you struggle to decide which level to use, you can follow the next decision tree:

Press enter or click to view image in full size

Logging Points

There are no strict rules about when you should write logs. The general recommendation is to write logs in your code at key moments, including:

  • start and end of methods: to trace the flow of execution
  • flow changes: when the execution flow of an algorithm changes, usually an if statement
  • exception handling: to capture error details and stack traces
  • performance metrics: to log timing for performance-critical operations
  • data processing events: when processing data, log inputs and outputs for validation
public int? Divide(int a, int b)
{
File.AppendAllText("log.txt", $"[DEBUG] Divide is called with a={a} b={b}");

if (b == 0)
{
File.AppendAllText("log.txt", $"[WARN] An attempt to divide by zero. No result.");
return null;
}

var res = a / b;

File.AppendAllText("log.txt", $"[DEBUG] Return result={res}");
return res;
}

Just see when it helps you to troubleshoot and understand application behavior.

I usually add logs in my business logic, in every new if statement, and before return.

Trace Identifier/Correlation Id

There is still a problem with our logs. In case we open our log file, you will see its content:

[DEBUG] Divide is called with a=2 b=1
[DEBUG] Divide is called with a=4 b=2
[DEBUG] Return result=2
[DEBUG] Return result=2

While we have log levels to identify important messages, we still cannot distinguish records related to each other. Therefore, we need to add an identifier that will bind all logs in the same execution flow.

A Trace Id (aka Correlation Id) is a unique identifier assigned to a specific request in an application. It helps track the flow of that request through various components and services, making it easier to correlate logs

public int? Divide(int a, int b, string traceId)
{
File.AppendAllText("log.txt", $"[DEBUG] [{traceId}] Divide is called with a={a} b={b}");

if (b == 0)
{
File.AppendAllText("log.txt", $"[WARN] [{traceId}] An attempt to divide by zero. No result.");
return null;
}

var res = a / b;

File.AppendAllText("log.txt", $"[DEBUG] [{traceId}] Return result={res}");
return res;
}

Now it is evident which logs are generated by the first call and which are produced by the second one.

[DEBUG] [42] Divide is called with a=2 b=1
[DEBUG] [19] Divide is called with a=4 b=2
[DEBUG] [19] Return result=2
[DEBUG] [42] Return result=2

Log Distribution

It is often useful to write logs to different sources. Locally developers can have simplified logs in a console, support teams might need logs on an accessible dashboard, while the production environment can write detailed information in a secure storage.

Press enter or click to view image in full size

Anyway, distributing logs to multiple sources can retain the information, ensuring data is not lost. So, let’s update our example:

public int? Divide(int a, int b, string traceId)
{
Log("DEBUG", traceId, $"Divide is called with a={a} b={b}");

if (b == 0)
{
Log("WARN", traceId, "An attempt to divide by zero. No result.");
return null;
}

var res = a / b;

Log("DEBUG", traceId, $"Return result={res}");
return res;
}

public void Log(string logLevel, string traceId, string message)
{
var logEntry = $"[{logLevel}] [{traceId}] {message}";

File.AppendAllText("log.txt", logEntry);
Console.WriteLine(logEntry);
DataBase.Logs.Add(logEntry)
}

Unstructured Logs Vs Structured Logs

The logs we are writing right now are called unstructured.

Unstructured logs are logs that do not follow a predefined format or structure. They are often human readable plain text

[DEBUG] [19] Divide is called with a=4 b=2
[DEBUG] [19] Return result=2

While those are suitable for simple logging scenarios, developers find a better way to store logs.

Structured logs are logs that are formatted in a predefined way, like JSON, and have a common structure

{
"Level": "Debug",
"MessageTemplate": "Divide is called with a={property:a} b={property:b}",
"TraceId": "19",
"Properties": {
"a": "4",
"b": "2"
}
},
{
"Level": "Debug",
"MessageTemplate": "Return result={property:res}",
"TraceId": "19",
"Properties": {
"res": "4"
}
}

As practice shows, developers don’t read millions of raw logs directly. It is much easier to feed them into a tool that will build beautiful diagrams and visual charts that are much easier to analyze.

As you may guess, those tools are more comfortable to work with a consistent format.

Press enter or click to view image in full size

Audit Logs

We are currently discussing technical logs that help us debug the system. However, some systems require logs to track user activities for security and compliance purposes.

An audit logs are logs that records users activities, who performed the action, the date and time of the operation, changes in the DB, and how that affected the system.

Often, those logs are part of business logic and their implementation mostly depends on system requirements.

Sometimes, it can be something simple as new columns in the database:

Press enter or click to view image in full size

The other time, you may want to have a detailed explanation of the executed action. You may send events to a different server, to process and store them, and so on.

Press enter or click to view image in full size

We won’t focus on audited logs in this article, just know they exist.

Fundamentals of Logging

As you can see while being a simple concept, logs require a lot of work. Therefore, people don't write their own solutions but use existing tools.

Microsoft provides logging API as a NuGet package Microsoft.Extensions.Logging.

So, let’s create a new console project and see how it works.

First, you need to install those NuGets:

PM> Install-Package Microsoft.Extensions.Logging
PM> Install-Package Microsoft.Extensions.Logging.Console

It will add essential classes and interfaces.

The ILogger interface has logging methods:

interface ILogger
{
bool IsEnabled(LogLevel logLevel);
IDisposable BeginScope<TState>(TState state);

void Log(LogLevel logLevel, string message);
void LogDebug(string message);
void LogWarning(string message);
. . .
}

You cannot just create a logger instance. For that, you can use ILoggerProvider. As the name implies, it provides a logger 😅.

There are different providers like ConsoleLoggingProvider, ApplicationInsightsLoggingProvider, and so on.

interface ILoggerProvider : IDisposable
{
ILogger CreateLogger(string categoryName);
}

However, the traditional approach is to use ILoggerFactory. It also can create an ILogger, but additionally, it allows combining multiple ILoggerProvider’s and creating a logger that will write into multiple sources.

interface ILoggerFactory : IDisposable
{
ILogger CreateLogger(string categoryName);
void AddProvider(ILoggerProvider provider);
}

Now, let’s see it all together:

ILoggerFactory factory = LoggerFactory.Create(config =>
{
config.AddConsole(); // 👈 Add ConsoleLoggerProvider
});

// 👇 create ILogger with a factory
ILogger logger = factory.CreateLogger("Category.Medium");

// log messages
logger.Log(LogLevel.Warning, "message");
logger.LogWarning("message");
logger.LogWarning(new EventId(23), "message"); // 👈 EventId is rarely used

This will output the next text in the console:

warn: Category.Medium[0]
message
warn: Category.Medium[0]
message
warn: Category.Medium[23]
message

Notice, that it shows the log level (warn), logger’s category (Category.Medium), event id ([0]), and the message itself (message).

Plain text is not the only thing you can log. The logger methods have overloads that take an exception parameter:

try
{
throw new Exception("Exception");
}
catch (Exception ex)
{
_logger.LogWarning(ex, "An error occurred");
}

Log Levels

In case you want to disable some log messages, without removing them, you can tune the minimum log level:

ILoggerFactory factory = LoggerFactory.Create(config =>
{
config.AddConsole()
config.SetMinimumLevel(LogLevel.Warning); // 👈
});

ILogger logger = factory.CreateLogger("Category.Medium");

// log messages
logger.LogTrace("trace"); // ❌
logger.LogDebug("debug"); // ❌
logger.LogInformation("info"); // ❌
logger.LogWarning("warn"); // ✅
logger.LogError("error"); // ✅
logger.LogCritical("fatal"); // ✅

Notice, that subsequent log levels with higher severity will still be enabled.

Press enter or click to view image in full size

Thereby, the code above will produce the next output:

warn: Category.Medium[0]
warn
fail: Category.Medium[0]
error
crit: Category.Medium[0]
fatal

You probably, noticed, that when creating a logger, we have to specify the category name. The category name is just an arbitrary string associated with a logger:

interface ILoggerProvider : IDisposable
{
ILogger CreateLogger(string categoryName);
}

This is done so we can more granularly disable logs. For example, let’s disable all logs for a specific category:

ILoggerFactory factory = LoggerFactory.Create(config =>
{
config.AddConsole();
config.AddFilter("Category.Medium", LogLevel.None); // 👈
});

ILogger logger1 = factory.CreateLogger("Category"); // ✅
ILogger logger2 = factory.CreateLogger("Category.Medium"); // ❌

logger1.LogWarning("logger 1");
logger2.LogWarning("logger 2");

Hence, the result:

warn: Category[0]
logger 1

When we disable the “parent” category, it also disables all the children:

ILoggerFactory factory = LoggerFactory.Create(config =>
{
config.AddConsole();
config.AddFilter("Category", LogLevel.None); // 👈
});

ILogger logger1 = factory.CreateLogger("Category"); // ❌
ILogger logger2 = factory.CreateLogger("Category.Medium"); // ❌

logger1.LogWarning("logger 1");
logger2.LogWarning("logger 2");

In fact, you can write any custom logic in the filters:

ILoggerFactory factory = LoggerFactory.Create(config =>
{
config.AddConsole();
config.AddFilter((providerName, category, logLevel) =>
{
return
providerName.Contains("ConsoleLoggerProvider") &&
category.Contains("Category") &&
logLevel >= LogLevel.Information;
}
});

As mentioned before, when an ILogger is created, a category is specified. That category is an arbitrary string, but you can also use a generic parameter:

ILogger logger1 = factory.CreateLogger("Category.Medium");
ILogger<MyClass> logger2 = factory.CreateLogger<MyClass>(); // 👈

In that case, the category is the fully qualified class name:

warn: Category.Medium[0]
logger 1
warn: MyNamespace.MyClass[0] // 👈
logger 2

Begin Scope

It is often useful to add the same parameters to multiple log messages.

var traceId = Guid.NewGuid();

logger.LogInformation($"TraceId: {traceId}. Log 1");
logger.LogInformation($"TraceId: {traceId}. Log 2");

Not to duplicate the code, you can use .BeginScope() method.

ILoggerFactory factory = LoggerFactory.Create(config =>
{
config.AddConsole(config =>
{
// it is disabled by default for ConsoleLoggerProvider
config.IncludeScopes = true; // 👈
});
});

ILogger logger = factory.CreateLogger("Category.Medium");

// log messages
logger.LogInformation("Log 1"); // outside the scope

using (logger.BeginScope($"TraceId: {Guid.NewGuid()}"))
{
logger.LogInformation("Log 2");
logger.LogInformation("Log 3");
}

You can see that TraceId is added to all log messages within the scope:

info: Category.Medium[0]
Log 1
info: Category.Medium[0]
=> TraceId: 9adae293-c5c1-4694-b7dc-948abb223777
Log 2
info: Category.Medium[0]
=> TraceId: 9adae293-c5c1-4694-b7dc-948abb223777
Log 3

Structured Log

The logs we are currently writing in the console are unstructured. This makes sense since logs from the console are primarily designed for developers.

Get iamprovidence’s stories in your inbox

Join Medium for free to get updates from this writer.

However, you can also write structured logs in the console:

ILoggerFactory factory = LoggerFactory.Create(config =>
{
config.AddJsonConsole(c =>
{
c.JsonWriterOptions = new JsonWriterOptions
{
Indented = true,
};
});
});

ILogger logger = factory.CreateLogger<Program>();

logger.LogWarning("This! Is! Structured Log!!!");

The result will be as follows:

{
"EventId": 0,
"LogLevel": "Warning",
"Category": "Program",
"Message": "This! Is! Structured Log!!!",
"State": {
"Message": "This! Is! Structured Log!!!",
"{OriginalFormat}": "This! Is! Structured Log!!!"
}
}

⚠️ There is one important point about parameters.

In case you are using string interpolation, values won’t be properly inserted in the logs. You should use the message template and pass additional arguments in the parameter list.

logger.LogWarning($"Value = {27}");     // ❌
logger.LogWarning("Value = {Val}", 27); // ✅

Compare the two output logs from the code above:

`{
"EventId": 0,
"LogLevel": "Warning",
"Category": "Program",
"Message": "Value = 27",
"State": {
"Message": "Value = 27",
"{OriginalFormat}": "Value = 27" // 👈
}
}
{
"EventId": 0,
"LogLevel": "Warning",
"Category": "Program",
"Message": "Value = 27",
"State": {
"Message": "Value = 27",
"Val": 27, // 👈
"{OriginalFormat}": "Value = {Val}" // 👈
}
}

Other providers like ApplicationInsightsLoggingProvider write structured logs by default. Therefore, it is important to always pass the parameters separately, and not use string interpolation.

Logger in Unit Tests

Since your services inject ILogger<T> you would often face a need to mock it in unit tests.

It can be achieved with regular mocking:

var mock = new Mock<ILogger<Program>>();

Or by using NullLogger, NullLoggerProvider, NullLoggerFactory, logging abstractions that do nothing:

var logger = new NullLogger<Program>();
// or get shared instance
var logger = NullLogger<Program>.Instance;

Logging in ASP.NET Core

When you create a new ASP application, you don’t need to configure logging, since it is already done.

You can just inject ILogger<T> and use it.

var builder = WebApplication.CreateBuilder(args);

var app = builder.Build();

app.MapGet("/", (ILogger<Program> logger) =>
{
logger.LogWarning("message");
});

app.Run();

Logging configurations are present in appsettings.json files. You don’t have to set a minimum log level or configure filters for categories, in the code. This can be done through the configuration:

In the appsettings.json you can see

  • "Microsoft.AspNetCore" log-level category is specified and applied to all categories that start with "Microsoft.AspNetCore.*". For example, middlewares, internal classes, and so on
  • The "Default" value is applied to all categories that aren't otherwise specified
  • You can also specify a value for your own category ("MyNamespace.Program"), the same way we did with filters
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning",
"MyNamespace.Program": "None" // 👈
}
}
}

You can also specify log levels for each provider:

{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning",
},
// 👇
"Console": {
"IncludeScopes": true,
"LogLevel": {
"MyNamespace.Program": "Warning",
"Default": "Information"
}
},
"ApplicationInsights": {
"LogLevel": {
"Default": "Information"
}
}
}
}

You can still configure the logging providers if you want:

var builder = WebApplication.CreateBuilder(args);

builder
.Logging
.ClearProviders()
.AddJsonConsole();

var app = builder.Build();
. . .

However, the most important thing, is that you don’t have to include the trace id in each log:


app.MapGet("/", (ILogger<Program> logger, HttpContext httpContext) =>
{
logger.LogError("{TraceId}", httpContext.TraceIdentifier);
});

ASP implicitly adds request-id to each log entry, so you can easily track all logs for a specific request.

Serilog

If you tried logger from Microsoft you can see it is quite limited. Therefore usually developers go for third-party solutions.

One of the most trending libraries is Serilog. It offers structured logging, various sinks (e.g., console, files, databases, cloud, etc), and ease of use.

Let’s see it in practice. Install next packages:

// this adds basic classes for Serilog
$ dotnet add package Serilog

// this enables direct log output to the console
$ dotnet add package Serilog.Sinks.Console

// this allows to integrate Serilog with Microsoft.Extensions.Logging
$ dotnet add package Serilog.Extensions.Logging

Natively Serilog can be used this way:

// configure Serilog's logger
var logger = new LoggerConfiguration()
.WriteTo.Console(new JsonFormatter())
.CreateLogger();

// log messages
logger.Warning("Message");

However, we can integrate Serilog with Microsoft.Extensions.Logging without rewriting our codebase.

ILoggerFactory factory = LoggerFactory.Create(config =>
{
// configure Serilog's logger
var logger = new LoggerConfiguration()
.WriteTo.Console(new JsonFormatter())
.CreateLogger();

config.AddSerilog(logger); // 👈 register Serilog provider
});

// the rest of application stays the same 👇
var logger = factory.CreateLogger<Program>();

// log messages
logger.LogWarning("Message");

Formatting

Usually, log message templates support placeholder formatting:

// Logged on January 06, 2022
logger.LogInformation("Logged on {Date:MMMM dd, yyyy}", DateTimeOffset.UtcNow);

However, Serilog adds this one important feature.

If you have an object you want to be present in the log:

class User
{
public string Name { get; set; }
}

Serilog will call its .ToString() method:

// Person details = User
logger.LogWarning("Person details = {User}", new User { Name = "John" });

But, when you add the @ operator in front of the variable, the logger will serialize the object with JsonSerializer.Serialize():

// Person details = {"Name": "John", "$type": "User"}
logger.LogWarning("Person details = {@User}", new User { Name = "John" });

Minimum Level

As before, you can set a minimum log level:

var logger = new LoggerConfiguration()
.MinimumLevel.Debug()
.WriteTo.Console()
// .WriteTo.Console(restrictedToMinimumLevel: LogEventLevel.Information)
.CreateLogger();

Or write complex filter conditions:

var logger = new LoggerConfiguration()
.WriteTo.Console(new JsonFormatter())
.Filter.ByExcluding(logEvent => logEvent.Level > LogEventLevel.Warning)
.CreateLogger();

Enrichers

Additionally, you can add properties not only with .BeginScope(), but also with enrichers:

var logger = new LoggerConfiguration()
.Enrich.When(
(logEvent) => logEvent.Level > LogEventLevel.Information,
(conf) => conf.WithProperty("TraceId", Guid.NewGuid()))
.WriteTo.Console(new JsonFormatter())
.CreateLogger();

Sinks

As was mentioned before, the log can be written in multiple destinations. In Serilog those targets are sinks. Serilog comes with a number of built-in sinks. Some popular sinks are console, file, emails, database, elasticsearch, etc.

Each sink comes as a separate NuGet:

$ dotnet add package Serilog.Sinks.Console
$ dotnet add package Serilog.Sinks.File

You can combine multiple sinks at the same time:

var logger = new LoggerConfiguration()
.WriteTo.Console()
.WriteTo.File("log-.txt", rollingInterval: RollingInterval.Day)
.WriteTo.Seq("http://localhost:5341")
.WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri("http://localhost:9200")))
.CreateLogger();

You can also create your own custom sinks:

var logger = new LoggerConfiguration()
.WriteTo.Sink(new MySink())
.CreateLogger();

Here is the interface for a custom sink. Notice, that its method returns void and not Task.

interface ILogEventSink
{
void Emit(LogEvent logEvent);
}

Your sink implementation should be fast and should not impact your app’s performance. If you need to write logs to a database (or another high-latency storage), it is better to temporarily store logs in memory and have a background process that will flush those messages from time to time.

Logging in ASP.NET Core

As expected, you can use Serilog with ASP.

$ dotnet add package Serilog.AspNetCore
$ dotnet add package Serilog.Sinks.Console

The beauty here is that you don’t have to write configuration statically in the code. You can read them from appsettings.json:

var builder = WebApplication.CreateBuilder(args);

// configure Serilog
var loggerConfig = new LoggerConfiguration()
.ReadFrom.Configuration(builder.Configuration); // 👈

// replace the default logging with Serilog
builder.Host.UseSerilog(loggerConfig.CreateLogger());

var app = builder.Build();

// log HTTP request information like method, path, timing, status code, etc
// app.UseSerilogRequestLogging();

app.MapGet("/", (ILogger<Program> logger) =>
{
logger.LogWarning("message");
});

app.Run();

Where you can set log level, sinks, their configurations, etc:

{
"Serilog": {
"MinimumLevel": {
"Default": "Information",
"Override": {
"Microsoft": "Information"
}
},
"WriteTo": [
{
"Name": "Console"
},
{
"Name": "File",
"Args": {
"path": "logs.txt"
}
},
{
"Name": "Seq",
"Args": {
"serverUrl": "http://localhost:5341"
}
}
]
}
}

Monitoring tools

Even though we are developers, used to consoles, we still like reading our logs with a beautiful UI 🙃.

Sure, you can connect, your local application to AppInsights in Azure to see all logs and statistics, however, there are other lightweight solutions.

Let’s review some of the available options.

WatchDog.NET

The first, simple tool to view log messages I want to show you is WatchDog.Net.

Install it via .NET CLI:

$ dotnet add package WatchDog.NET

Add some configuration to the project:

using WatchDog;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddWatchDogServices(); // 👈

var app = builder.Build();
app.UseWatchDog(opt => // 👈
{
opt.WatchPageUsername = "admin";
opt.WatchPagePassword = "admin";
});

app.MapGet("/", () =>
{
WatchLogger.Log("message", level: "Debug");
WatchLogger.LogWarning("message");
});

app.Run();

And voila. Just open https://localhost:[your-port]/watchdog and you will have a beautiful UI to search, filter, and analyze your logs.

Press enter or click to view image in full size

Two things to notice:

  • WatchDog.Net writes logs to an in-memory database, but you can also connect external databases (MSSQL, MySQl, Postgres, MongoDB)
  • it does not support ILogger. You need to wrap it with a custom logger provider, sink, etc

Seq

Another powerful log aggregation tool is Seq. It is designed to manage structured log data, making it easier to search, filter, and analyze.

First, you need to download and install it on your PC, or via Docker.

Later, Seq can be easily configured with Serilog.

using Serilog;

var builder = WebApplication.CreateBuilder(args);

builder.Host.UseSerilog((hostContext, loggerConfiguration) =>
{
loggerConfiguration
.WriteTo.Seq("http://localhost:5341"); // 👈
});

var app = builder.Build();

app.MapGet("/", (ILogger<Program> logger) =>
{
logger.LogWarning("message");
});

app.Run();

The dashboard and logs are available by this URL: http://localhost:5341.

Press enter or click to view image in full size

Seq provides a centralized location for logs with structured analysis tools, making it easier to track and manage log data across multiple applications.

ELK Stack (Elasticsearch, Logstash, Kibana)

The ELK Stack is a powerful log management and analytics solution, composed of three main tools: Elasticsearch, Logstash, and Kibana.

Elasticsearch stores logs, Logstash ingests them, and Kibana provides visualization.

As before, you need to download and install it on your PC, or via Docker.

Configure it with Serilog.

using Serilog;

var builder = WebApplication.CreateBuilder(args);

builder.Host.UseSerilog((hostContext, loggerConfiguration) =>
{
// 👇
loggerConfiguration
.WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri("http://localhost:9200/")) // elasticsearch URL
{
IndexFormat = $"{hostContext.HostingEnvironment.ApplicationName}~logs~{hostContext.HostingEnvironment.EnvironmentName}~{DateTimeOffset.UtcNow:yyyy-MM}",
AutoRegisterTemplate = true,
NumberOfShards = 2,
NumberOfReplicas = 1,
})
});

var app = builder.Build();

app.MapGet("/", (ILogger<Program> logger) =>
{
logger.LogWarning("message");
});

app.Run();

Boom! Kibana is available at http://localhost:5601:

Press enter or click to view image in full size

The ELK Stack can be used in local development, however, its versatile and robust solution is better suited for production environments.

Performance Issues

There is one weird stuff about logging I want to discuss with you.

Let’s say we have our minimum log level at Warning:

builder.Logging.SetMinimumLevel(LogLevel.Warning);

In the code, we log information. Respectively, it should not be logged.

app.MapGet("/", (ILogger<Program> logger) =>
{
logger.LogInformation("Value = {Val}", 23);
});

Still, the method is called. If you look at the method signature, you can see that all parameters are passed as object list:

public static class LoggerExtensions
{
. . .
public static void LogInformation(
this ILogger logger,
string? message,
params object?[] args); // 👈
. . .
}

This means, that even when the log level is disabled, you still waste memory on allocating an array on the heap and boxing the value.

In case, you seek a highly performant logger, you need to wrap every log operation with .IsEnabled(logLevel) check:

app.MapGet("/", (ILogger<Program> logger) =>
{
if (logger.IsEnabled(LogLevel.Information))
{
logger.LogInformation("Value = {Val}", 23);
}
});

By the way, if you are using Serilog directly, you don’t have to do it:

app.MapGet("/", (Serilog.ILogger serilogger) =>
{
serilogger.Information("Value = {Val}", 23);
});

This is because Serilog has a couple of generic method overloads that will prevent unnecessary allocation and boxing:

interface ILogger
{
. . .
void Information(string messageTemplate);
void Information<T>(string messageTemplate, T propertyValue);
void Information<T0, T1>(string messageTemplate, T0 propertyValue0, T1 propertyValue1);
void Information<T0, T1, T2>(string messageTemplate, T0 propertyValue0, T1 propertyValue1, T2 propertyValue2);
void Information(string messageTemplate, params object?[]? propertyValues);
. . .
}

I honestly think this is just stupid 🙃.

The only valid reason why you would wrap your log with .IsEnabled(), is when you actually have a performance-heavy operation, like in the code below:

if (logger.IsEnabled(LogLevel.Debug))
{
_logger.LogDebug("Value {Val}", await DataBase.GetValue());
}

Summary

I think you are tired at this point, so let’s just quickly sum up the important stuff 😅.

Logging is a fundamental block in any application. It is crucial for tracking and diagnosing the application behavior. You should not write your own logger but use an established battle-proved logging framework (like Microsoft.Extensions.Logging, Serilog, etc).

It is highly recommended to use structured logging. It allows logs to be output in a consistent, queryable format (such as JSON), making it easier to analyze and monitor.

Do not use string interpolation. Give preference to string template. This way it is properly structured and can be better parsed by log management tools:

logger.LogWarning($"Value = {27}");     // ❌
logger.LogWarning("Value = {Val}", 27); // ✅

The @ operator in the message template tells Serilog to serialize the object passed in, rather than convert it using ToString().

// Person details = {"Name": "John", "$type": "User"}
logger.LogWarning("Person details = {@User}", new User { Name = "John" });

If you see something like this in the code, it means your team is obsessed with performance gain 🥴:

if (logger.IsEnabled(LogLevel.Information))
{
logger.LogInformation("Value = {Val}", 23);
}

Even though it was not highlighted much, it is important to use TraceId/CorrelationId in logs. It helps track a request's flow across different systems, making it easier to trace and troubleshoot distributed applications, like microservices.

Press enter or click to view image in full size

Just kidding 😅.You made it my friend to the very end, be proud of yourself 😎. Let’s do our usual, and call it a day.

💬 Let me know in the comments what are other log-related things worth mentioning

👏 Clap for this article if you enjoyed

👇 If you don’t like this one, there are other articles you may want to check out

☕️ You can also buy me a coffee with a link below

✅ Follow not to miss out more

😉 But most importantly, stay awesome

--

--

Written by iamprovidence

👨🏼‍💻 Full Stack Dev writing about software architecture, patterns and other programming stuff https://www.buymeacoffee.com/iamprovidence