.NET HttpLogging of request to custom storage - c#

We work in .NET 6 and want to log all incoming requests. We plan to allow this primarily in DEV and TEST environments but potentially and temporarily also in PROD.
The new HttpLogger sounds like a perfect match. But it will output to the standard ILogger which in our case is Elastic. From GDPR reasons we want to avoid that where potentially sensitive data will be exposed in an uncontrolled way.
Does anyone have another solution:
Redirect messages from HttpLogger to other storage
Create custom middleware to be able to sniff requests and output to custom storage
I found this interesting post on reading the body twice but was not able to make it work: https://codetalk.in/posts/2022/01/04/read-request-body-multiple-times-in-asp-dot-net-core

You can do that just by configuration, for example in production app settings, disable HTTP logging for Elastic and enable it for file logger, something like this (really depends on your loggers and configurations):
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware": "Information"
},
"Elastic": {
"IncludeScopes": true,
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware": "None"
}
},
"File":
{
"IncludeScopes": true,
"LogLevel": {
"Default": "None",
"Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware": "Information"
}
}
}
}
To be more restrictive, configure HttpLoggingOptions for production to not log some fields for example.

Related

How to control log levels in C#

For a C# library we're building, we've implemented a logging facility using Microsoft.Extensions.Logging. The library is using a nullable ILogger, which is injected by the user of the library, like this:
var client = new Client(app.Logger)
The app is an AspNetCore application, which has a configured logger, and is injected into the Client. However, I'd like to control the log levels using the standard logging configuration as in appsettings.json:
"Logging": {
"LogLevel": {
"Default": "Debug",
"Microsoft.AspNetCore": "Information",
"System.Net.Http.HttpClient": "Information"
}
}
I'd like to add an entry here, to control the logging, like this:
"Logging": {
"LogLevel": {
"Default": "Debug",
"Microsoft.AspNetCore": "Information",
"System.Net.Http.HttpClient": "Information",
"MyLibrary": "Trace" // <--- like this
}
}
The challenge is that the namespace of the logger is not MyLibrary, but the name of the current logger printed is rp-dotnet-sample[0], instead of something like MyLibrary.
My question now is: how do I get to a situation in which I can develop this library and control the log levels external to the library?

Azure Functions Not able to hit HttpTrigger

I have a simple Azure function with HttpTrigger like so:
[FunctionName("Heartbeat")]
public async Task<IActionResult> Heartbeat(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "heartbeat")] HttpRequest req,
ILogger log)
{
log.Log(LogLevel.Information, "Received heartbeat request");
var status = await _healthCheck.CheckHealthAsync();
return new OkObjectResult(Enum.GetName(typeof(HealthStatus), status.Status));
}
local.settings.json:
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=false",
"FUNCTIONS_WORKER_RUNTIME": "dotnet",
"APPINSIGHTS_INSTRUMENTATIONKEY": "*****"
},
"ConnectionStrings": {
"ServiceDb": "Server=.;Initial Catalog=Acquire;Integrated Security=True;"
},
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft": "Warning",
"Microsoft.Hosting.Lifetime": "Information"
}
},
When i run the function locally I'm able to hit a breakpoint inside Heartbeat() and get 200OK. However, when it's deployed to Azure, It does not work and I get 404 Not Found.
Local:
Azure:
What is the problem? why can't I execute an HttpTrigger?
Also, is there a way to view local.settings.json configururation in Azure? I though these settings would somehow populate from local.settings.json in Azure (like connection strings)-yet they're blank. Is there a way to show these settings in Azure?
is there a way to view local.settings.json configururation in Azure?
There is no local.settings.json in Azure, hence the name "local".
You should add settings with the same names to your App Settings.
There should be an option for azure to give you the function url. Should look something like this
I found the solution. My issue was that Runtime version was set to: ~1 instead of ~3. Another change I had to make for IOptions to bind is edit project, and modify these options:
<Content Include="local.settings.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
</Content>
To make logging work:
https://nuggets.hammond-turner.org.uk/2019/10/tuesday-quickie-ilogger-in-azure.html
I added this and now I see all logs fine:
{
"version": "2.0",
"logging": {
"logLevel": {
"ServiceLogger": "Information"
}
}
}
but all the answers from other users got me on the right path.

How do you read from appsettings.json in the same style as it's predecessor, webforms?

In WebForms we would have a web.config file that we could do ConfigurationManager.AppSettings["SomeKey"]; to retrieve the value from the key-value pair.
I've just started a project in .NET 5.0 and there doesn't seem a simple way to do something that seems to trivial?
I've looked online and have been unsuccessful in following tutorials on how to access these keys in appsettings.json from a .cshtml file using # notation.
appsettings.json:
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft": "Warning",
"Microsoft.Hosting.Lifetime": "Information"
}
},
"AllowedHosts": "*",
"MyKey": "wwwwwwwwwww"
}
Index.cshtml:
<h1>
Title - #ConfigurationManager.AppSettings["MyKey"];
</h1>
The above illustrates what I am trying to achieve, is there a simple way to do this rather than creating classes etc as I don't seem to be able to follow their examples.
To access configuration settings in a view in a .NET project, you should be able to use # annotation. This is done by injecting the configuration into the page:
#page
#model Test5Model
#using Microsoft.Extensions.Configuration
#inject IConfiguration Configuration
Configuration value for 'MyKey': #Configuration["MyKey"]
Take this appsettings.json for example:
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft": "Warning",
"Microsoft.Hosting.Lifetime": "Information"
}
},
"AllowedHosts": "*",
"Test": "Hello World",
"ConnectionStrings": {
"SomeContext": "SomeConnectionStringProperties"
},
"SectionOne": {
"SectionOneTestVal": "SomeTestValue"
}
}
In order to access the Test key, it would simply be #Configuration["Test"] while to access the SectionOneTestVal "key" in the SectionOne section, you would do something like Configuration.GetSection("SectionOne")["SectionOneTestVal"]:
Thus adding this to a view:
<p>#Configuration["Test"]</p>
<p>#Configuration.GetSection("SectionOne")["SectionOneTestVal"]</p>
...would yield:
For more information and examples, also check out dependency injection into views.

Trace, Debug and Information messages not reaching App Insights from WorkerService

I've spent the entire last day trying to find something to explain why I'm not seeing any Trace, Debug or Information messages in app insights when I run my WorkerService in Azure. The WorkerService is being hosted in a Linux (Ubuntu 18.4) VM in our enterprise space and in all other ways works fine.
The Critical, Error and Warning messages are getting through so the connection is correctly established but something appears to be filtering the messages at the process end.
The relevant logging part of the appsettings.json is as follows:
"Logging": {
"LogLevel": {
"Default": "Trace",
"System": "Trace",
"Microsoft": "Trace"
}
},
"ApplicationInsights": {
"LogLevel": {
"Default": "Trace",
"System": "Trace",
"Microsoft": "Trace"
}
},
...which should allow everything through I would expect. My test code is...
log.LogTrace("WorkerService: Trace");
log.LogDebug("WorkerService: Debug");
log.LogInformation("WorkerService: Information");
log.LogWarning("WorkerService: Warning");
log.LogError("WorkerService: Error");
log.LogCritical("WorkerService: Critical");
...but I have noticed when I run the code locally that the output window shows the following...
Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker: Information: Executing action method MyWorkerService.Features.HealthCheck.HealthCheckController.GetAdvancedHealthCheck (MyWorkerService) - Validation state: Valid
MyWorkerService.Features.HealthCheck.GetAdvancedHealthCheck.Handler: Trace: Trace
MyWorkerService.Features.HealthCheck.GetAdvancedHealthCheck.Handler: Debug: Debug
MyWorkerService.Features.HealthCheck.GetAdvancedHealthCheck.Handler: Information: Information
MyWorkerService.Features.HealthCheck.GetAdvancedHealthCheck.Handler: Warning: Warning
Application Insights Telemetry: {"name":"AppTraces","time":"2020-12-15T08:04:49.6657815Z","iKey":"8d4d3ae1-xxxx-xxxx-xxxx-029e472df754","tags":{"ai.application.ver":"1.0.0.0" ...<snip>... "TraceId":"520378xxxxxxxxxxxxxxxxxxxxe5a1b5"}}}}
MyWorkerService.Features.HealthCheck.GetAdvancedHealthCheck.Handler: Error: Error
Application Insights Telemetry: {"name":"AppTraces","time":"2020-12-15T08:04:49.6688389Z","iKey":"8d4d3ae1-xxxx-xxxx-xxxx-029e472df754","tags":{"ai.application.ver":"1.0.0.0" ...<snip>... "TraceId":"520378xxxxxxxxxxxxxxxxxxxxe5a1b5"}}}}
MyWorkerService.Features.HealthCheck.GetAdvancedHealthCheck.Handler: Critical: Critical
Application Insights Telemetry: {"name":"AppTraces","time":"2020-12-15T08:04:49.6854384Z","iKey":"8d4d3ae1-xxxx-xxxx-xxxx-029e472df754","tags":{"ai.application.ver":"1.0.0.0" ...<snip>... "TraceId":"520378xxxxxxxxxxxxxxxxxxxxe5a1b5"}}}}
...which also suggests that the Trace, Debug and Information messages are not being sent to Azure.
I'm using the "Microsoft.ApplicationInsights.WorkerService" Version="2.16.0" nuget package but cannot see what else I need to configure to change the filter which is limiting the transmitted data.
Any help or suggestions gratefully accepted.
For appsettings.json, please make sure you have set the property copy to output directory as copy if newer(in visual studio, right click appsettings.json -> select properties).
And then put the ApplicationInsights section into Logging section. Like below:
{
"Logging": {
"LogLevel": {
"Default": "Trace",
"Microsoft": "Trace",
"Microsoft.Hosting.Lifetime": "Trace"
},
"ApplicationInsights": {
"LogLevel": {
"Default": "Trace",
"System": "Trace",
"Microsoft": "Trace"
}
}
}
}
Here is the test result at my side:

Azure Function stops working when I use dependencies, works locally

I'm trying to setup an Azure Function, Linux based in consumption mode, Queue triggered.
It works perfectly locally in debug (said every programmer ever), but when deploying nothing happens. I can't find any logs.
I started over, uploaded an empty function - it worked, but as soon as I add my own libraries, it stopped working.
I then tried to remove my libraries and re-upload the 'hello world' function but it still doesn't work.
This is the host.json:
{
"logging": {
"fileLoggingMode": "always",
"logLevel": {
"default": "Information",
"Host.Results": "Information",
"Function": "Information",
"Host.Aggregator": "Information"
},
"applicationInsights": {
"samplingExcludedTypes": "Request",
"samplingSettings": {
"isEnabled": true
}
},
"console": {
"isEnabled": "true"
}
},
"Values": {
"AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=****;AccountKey=*****;BlobEndpoint=https://***.blob.core.windows.net/;TableEndpoint=https://*****.table.core.windows.net/;QueueEndpoint=https://****.queue.core.windows.net/;FileEndpoint=https://****.file.core.windows.net/",
"FUNCTIONS_WORKER_RUNTIME": "dotnet"
},
"version": "2.0"
}
The function code (without my libraries), works on first upload only.
[FunctionName("EmailQueueWorker")]
//public static async Task Run(
public static async Task Run(
[QueueTrigger(queueName: "email", Connection = "AzureWebJobsStorage")] string queueItem,
ILogger log
)
{
log.LogWarning("Start run()");
}
What am I doing wrong (or where can I find logs? the Application Insights is empty)? Thanks
I ran into the same problem a week or two ago; I'd bet good money the problem is your connection to the queue. For comparison, this is my full and complete host.json for my (working) queue trigger function:
{
"version": "2.0",
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true
}
},
"fileLoggingMode": "always",
"logLevel": {
"default": "Information",
"Host.Results": "Error",
"Function": "Trace",
"Host.Aggregator": "Trace"
}
}
}
Connection String
Our logLevel section is a bit different, and you'll note there aren't any connection strings there. I'm still farely new to Azure but from what I've learned, that isn't where they go.
In Visual Studio 2019, right click on the Project, then Publish. Under Actions, click Manage Azure App Service settings. There, you can add any needed connection string settings. If you need to specify a storage account, the setting name should be the name of the storage account plus "_STORAGE". For example, if your storage account was named MyVault then the name of the setting would be MyVault_STORAGE.
In VS Code, it's a bit different to get to. You have to look under Azure, Functions, and then be sure you select your Azure subscription (not the local copy!) and drill down into the function, Application Settings, where you can add/edit.
In the Azure portal, you can manage app settings this way.
Logs
In Azure portal, start by going to Function App. Click on your primary function name. Now, under the new menu for that function, under Functions sub-menu, click on Functions. Now you'll see a list of all the different functions that comprise your queue trigger function. Among them should be EmailQueueWorker - click on it. Now, you should see the execution count, and you can click on Monitor in the left hand menu, then Logs in the middle area. You can Start/Stop/Clear as needed.
For whatever reason, I find that I see the actual log data a lot faster when I use either Visual Studio 2019 or VS Code to stream it rather than the web console. There seems to be a bit of a delay at times with the web console.

Categories

Resources