Develop Reference

How to write logs to a specific sink depending on the developement environment in Serilog?

I'm using Serilog with Azure Functions v4 and .NET 7. I am trying to write logs to both the local console and Application Insights. I already spent hours trying different solutions and the situation at the time of writing seems to be that this technology stack (Azure Functions + Application Insights) is not mature or too recent, and not yet very well documented.
Setting up appsettings.{env}.json files by environment and trying to add a different Serilog config by environment does not work with Azure Functions. Depending on the solution I tried, either Microsoft logger is used or there are no logs.
The code below is working for the console but I get duplicate logs on Application Insights. And if I remove Console sink, it works well in Application Insights but I get no more local logs:
.UseSerilog((hostBuilderContext, serviceProvider, loggerConfiguration) =>
loggerConfiguration
.MinimumLevel.Debug()
.WriteTo.Console()
.WriteTo.ApplicationInsights(
serviceProvider.GetRequiredService<TelemetryConfiguration>(),
TelemetryConverter.Traces)
)
I also tried other ways of setting up the logger, for example in .ConfigureServices(..., injecting it differently in the function code, but no luck. Many samples talk about a Startup method that I don't have. If it helps I can post the whole Program.cs and the function code here.
So I now added .Enrich.WithEnvironmentName() to get the environment name from an environment variable, and the question is: how can I filter the logs using ByIncludingOnly in the code above (not config file) to include logs depending on the environment name (Local or Development)? Same question for ByExcluding - I guess answer will be similar.

This is the way to do it. I ended up not using the Environment enricher because it is at the time of writing not supporting the Azure environment variable for Azure Functions (AZURE_FUNCTIONS_ENVIRONMENT). See this issue:
https://github.com/serilog/serilog-enrichers-environment/issues/49
This is how I got this working:
First option enriching the log event:
// Careful: Serilog is not able to enrich with the value of this variable and will default to 'Production'
var value = Environment.GetEnvironmentVariable("AZURE_FUNCTIONS_ENVIRONMENT") ?? "Development";
var predicate = $"EnvironmentName = 'Local'";
Log.Logger = new LoggerConfiguration()
.Enrich.WithProperty("EnvironmentName", value)
.MinimumLevel.Debug()
.WriteTo.Logger(lc => lc
.Filter.ByIncludingOnly(predicate)
.WriteTo.Console(outputTemplate: "{Properties} {Message}"))
.WriteTo.Logger(lc => lc
.WriteTo.File(path: Directory.GetCurrentDirectory() + #"\test.txt"))
.CreateLogger();
Second option without enriching the log event:
var value = Environment.GetEnvironmentVariable("AZURE_FUNCTIONS_ENVIRONMENT") ?? "Development";
var predicate = $"{value} = 'Local'";
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Debug()
.WriteTo.Logger(lc => lc
.Filter.ByIncludingOnly(predicate)
.WriteTo.Console(outputTemplate: "{Properties} {Message}"))
.WriteTo.Logger(lc => lc
.WriteTo.File(path: Directory.GetCurrentDirectory() + #"\test.txt"))
.CreateLogger();

Logging to Azure Log Analytics custom fields using Serilog

I'm new to azure and also a rookie in .net, so this may be a noob question :)
Anyway, I have successfully managed to log to Azure Log Analytics using this code snippet:
var loggerConfig = new LoggerConfiguration().
Enrich.WithExceptionDetails().
Enrich.WithApplicationInformation(serviceName).
MinimumLevel.Debug().
MinimumLevel.Override("Microsoft", LogEventLevel.Information).
Enrich.FromLogContext().
Destructure.ByTransforming<ExpandoObject>(JsonConvert.SerializeObject).
//Enrich.WithProperty("ErrorMsg_CF","test").
WriteTo.AzureAnalytics(workspaceId: "MyWorkSpaceID",
authenticationId: "MyAuthID",
logName: "MyCustomLog_CL",
restrictedToMinimumLevel: LogEventLevel.Debug)
And:
Logging.Logger.Initialize(serviceName: "Logging Web Test v1.1");
Logging.Logger.GetLogger().Log(LogLevel.Debug, "{ErrorMsg_CF}:{TraceID_CF}:{UserName_CF}", errorMsg,traceID,userName);
I have, in my Log Analytics custom log table, created 3 custom fields: "ErrorMsg_CF", "TraceID_CF" and "UserName_CF", and I want to either be able to directly log to these custom fields, or somehow split my errormsg into these three custom fields.
When I import a file, I see that the message is stored in the "RawData" field, and with that field I am able to use the custom field generator and create custom fields, and also let Log Analytics split the message into the fields for me. But, when I do it through my application, the message is stored in the "LogMEssage_s" field, and seems like it's not possible to create custom fields from that field.
So, anyone know how I can log to my custom fields from my application?

The default formatting configuration is a line by line event logging. ALN can be ingested using JSON so that will be the best way to format log entry in that way.
.WriteTo.DESTINATION(new CompactJsonFormatter(), OTHER OPTIONS HERE)
Serilog formatting

serilog multiple instance per functionality

While I have a general purpose logger for errors, exceptions etc that use rolling file sink to log stuff to my-{date}.log.
However, I need another instance for audit to audit-{date}.log and another instance to write perf info to perf-{date}.log.
How do I create multiple instances of serilog with different configuration or sink?

You can do this like following:
var loggerConfiguration = new LoggerConfiguration()
.MinimumLevel.Verbose()
.Enrich.FromLogContext();
var fileBasePath = "<base log path>";
loggerConfiguration
.WriteTo.Console()
.WriteTo.RollingFile(fileBasePath + "log-info-{Date}.txt")
.WriteTo.Logger(fileLogger => fileLogger
.MinimumLevel.Error()
.WriteTo.RollingFile(fileBasePath + "log-error-{Date}.txt"))
.WriteTo.Logger(fileLogger => fileLogger
.Filter.ByIncludingOnly(x =>
x.Level == LogEventLevel.Information &&
x.Properties.ContainsKey("<Audit Key>"))
.WriteTo.RollingFile(fileBasePath + "log-audit-{Date}.txt"));
Log.Logger = loggerConfiguration.CreateLogger();
Note: Audit Key is a key which used in audit log

I assume you would need different interface name to write audit and perf data (to be injected to your main business code). If that's the case just implement your IAudit, IPerfData with a new Serilog instance which you can totally customize where/how you save the log.

Using SqlBotDataStore for bot state falls back instead to state.botframework.com

I'm using the Nuget Microsoft.Bot.Builder.Azure package extensions to store my bot's state. BotBuilder 3.14.0, Bot.Builder.Azure 3.2.5.
I've been successfully using TableBotDataStore up until now, using Azure table storage. On starting a conversation, I can immediately inspect the table and see relevant rows created in it.
I tried using SqlBotDataStore, having run the script to create the table in my (Azure) SQL Server DB. I registered it as per the sample code, but on redeployment I find that, although no errors are thrown, no entries are created in the SqlBotDataEntities table, and instead state.botframework.com is being called as part of the request dependencies.
var sqlStore = new SqlBotDataStore(ConfigurationManager.
ConnectionStrings["SqlBotStorage"].ConnectionString);
builder.Register(c => sqlStore)
.Keyed<IBotDataStore<BotData>>(AzureModule.Key_DataStore)
.AsSelf()
.SingleInstance();
has replaced the (working)
var azureTableStore = new TableBotDataStore(
ConfigurationManager.ConnectionStrings["TableStorageCS"].ConnectionString,
sBotStorageTableName);
builder.Register(c => azureTableStore)
.Keyed<IBotDataStore<BotData>>(AzureModule.Key_DataStore)
.AsSelf()
.SingleInstance();
builder.Register(c => new CachingBotDataStore(azureTableStore,
CachingBotDataStoreConsistencyPolicy
.ETagBasedConsistency))
.As<IBotDataStore<BotData>>()
.AsSelf()
.InstancePerLifetimeScope();
I know the connection string is kosher, as I tried a test in a dummy action method:
IBotDataStore<BotData> sqlds = new SqlBotDataStore(
ConfigurationManager.ConnectionStrings["SqlBotStorage"].ConnectionString);
var key = new Address("botidhere", "effbee", "uid1", "c1", "x.com");
var bd = new BotData(data: new Dialogs.SimpleResponseDialog("message here"));
await sqlds.SaveAsync(key, BotStoreType.BotUserData, bd, default(CancellationToken));
and that adds a row just fine.
What else am I missing? The relevant connection string is stored in Azure application settings, as a connection string of type 'SQLAzure':
Server=tcp:xxxx.database.windows.net;Initial Catalog=xxxx;Persist Security Info=False;User ID=xxxx;Password=xxxx;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;

I appear* to have fixed it - I was missing the AzureModule registration line:
builder.RegisterModule(new AzureModule(System.Reflection.Assembly.GetExecutingAssembly()));
Here's the thing though - this wasn't needed for using TableBotDataStore. Only SqlBotDataStore.
Another thing: adding the line fixed the issue immediately in an Azure deployment slot with a debug build, system.debug=true. However, deploying to the main webapp (non-slot) with a release build, system.debug=false caused the app to be unable to start. '503 Service Unavailable'. No logs. Reverting immediately to table storage and all was well again.
I'm currently try..catching the above line, but I can't see an exception thrown, yet it has started working now. One to keep an eye on!
*with caveats above...

Logging important info in WebAPI 2 using Serilog

Just started working with Serilog, Its quite nice but I am a bit confused.
In case of errors in my Controller's actions, I'd like to log as much useful information as possible from the request (headers, parameters, etc).
What would be the best way to go about it?

You could look at using the Enrichment feature
As per the documentation from the link, you would construct your logger similar to
var log = new LoggerConfiguration()
.Enrich.WithThreadId()
.WriteTo.Console()
.CreateLogger();
"All events written through log will carry a property ThreadId with the id of the managed thread that wrote them. (By convention, any .WithXyz() methods on Enrich create properties named Xyz.)"
An example on Github is SerilogWeb.Classic. It provides a number of enrichers to capture certain information from the Request, e.g.
var log = new LoggerConfiguration()
.WriteTo.ColoredConsole()
.Enrich.With<HttpRequestIdEnricher>()
.Enrich.With<UserNameEnricher>()
.CreateLogger();
You could follow a similar apporach, creating an enricher that captures the information you always want to output in your logs, then initialising your logger using the enricher.