I'm trying to setup a CorrelationID across threads to establish a link between calls to my server, and the corresponding calls that I make to external web services. The correlation ID is a GUID which I'm saving in Logical Context structures of NLog (logical contexts work fine across threads, supposedly).
The idea is to have a GUID that is shared between any request to my server, and the corresponding requests that I issue to various web services due to this request. I tried using both MDLC and NDLC.
The problem is that the value is getting stored correctly only for the first request, and it's saving blank values for all subsequent ones, even though a GUID is correctly generated for each new request to my server.
I tried logging either to a database or in a file. The problem seems to solve itself if I add a breakpoint within the code, or if I add a System.Threading.Sleep anywhere around the logging method. What's also weird is that I can add the Sleep either before or after the method that sets the value in the logical context, and it still works either way. Removing the Sleep/breakpoint would cause it to break again.
I am using NLog v4.5.2.
Logging module:
using System;
using System.Web;
using NLog;
namespace Shift.Stardust.Engine.Modules
{
public class LoggingHttpModule : IHttpModule
{
public void Init(HttpApplication context)
{
context.BeginRequest += HandleBeginRequest;
}
public void Dispose()
{
}
private void HandleBeginRequest(object sender, EventArgs e)
{
System.Threading.Thread.Sleep(500);
var guid = Guid.NewGuid().ToString();
NestedDiagnosticsLogicalContext.Push(guid);
}
}
}
Placing a breakpoint anywhere in HandleBeginRequest produces correct output. Similarly for adding System.Threading.Thread.Sleep(500). Naturally, I wouldn't want to add such a line in my code just to resolve this issue.
NLog config:
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" internalLogFile="c:\temp\nlog-internal.txt" internalLogLevel="Trace">
<variable name="logDirectory" value="${basedir}/logs"/>
<targets>
<target name="asyncdatabase"
xsi:type="AsyncWrapper"
queueLimit="5000"
overflowAction="Block">
<target xsi:type="Database"
connectionStringName="ConnectionStringHere"
keepConnection="true">
<commandText>[db].[P_Log_Insert] #CreateDate, #ApplicationName, #MachineName, #LoggerName, #LogLevel, #Message, #Exception, NULL, #EngineSessionId, #CorrelationId</commandText>
<parameter name="#CreateDate" layout="${date}"/>
<parameter name="#ApplicationName" layout="${appsetting:name=Shift.Stardust.ApplicationName}"/>
<parameter name="#MachineName" layout="${machinename}"/>
<parameter name="#LoggerName" layout="${logger}"/>
<parameter name="#LogLevel" layout="${level}"/>
<parameter name="#Message" layout="${message}"/>
<parameter name="#Exception" layout="${exception:format=tostring}"/>
<parameter name="#EngineSessionId" layout="${aspnet-sessionid}"/>
<parameter name="#CorrelationId" layout="${ndlc}"/>
</target>
</target>
</targets>
<rules>
<logger name="Http.*" minlevel="Info" writeTo="asyncdatabase" final="true" />
</rules>
</nlog>
I expect to have a different CorrelationID for each incoming request, but this is only true for the first one. All of the subsequent ones have an empty string as value.
I think it's better to write to the HTTP context for this case.
e.g.
HttpContext.Current.Items["myvariable"] = 123;
and usage:
${aspnet-item:variable=myvariable} - produces "123"
See docs
You need the package NLog.Web (ASP.NET non-core) for that.
Note: ASP.NET Core uses should use NLog.Web.AspNetCore instead of NLog.Web
Related
I use nlog dll to write to database - oracle with entity frameWork in the line :
logger.Log(logLevel, "try");
I get in the logs of NLog the following error:
Warn DatabaseTarget: Parameter: 'TIME_STAMP' - Failed to assign DbType=OracleDbType.Date
Error DatabaseTarget(Name=WS_TRACE): Error when writing to database. Exception: System.Data.OracleClient.OracleException (0x80131938): ORA-01843: Invalid month
the code is:
SetPropGDC(LogEntity);
NLog.LogLevel logLevel = SetLogLevel(Level.Debug);
logger.Log(logLevel, "try");
ClearGDC();
private void SetPropGDC(LogEntity LogEntity)
{
GlobalDiagnosticsContext.Set(processId, LogEntity.PROCESS_ID.ToString());
GlobalDiagnosticsContext.Set("TIME_STAMP", DateTime.Now);
GlobalDiagnosticsContext.Set(customerId, LogEntity.CUSTOMER_ID.ToString());
}
<targets>
<target name="TRACEDatabase" type="DataBase" keepConnection="false"
dbProvider="Oracle.ManagedDataAccess.Client" connectionString="${gdc:connectionString}"
commandText="insert into TLOG_SITE_GENERAL_TRACE( PROCESS_ID,TIME_STAMP,CUSTOMER_ID)
values(:PROCESS_ID,:TIME_STAMP,:CUSTOMER_ID)">
<parameter name="PROCESS_ID" layout="${gdc:PROCESS_ID}" />
<parameter name="TIME_STAMP" layout="${gdc:TIME_STAMP}" dbType="OracleDbType.Date" />
<parameter name="CUSTOMER_ID" layout="${gdc:CUSTOMER_ID}" />
</target>
</targets>
I tried in the Web.config to change the line:
<parameter name="TIME_STAMP" layout="${gdc:TIME_STAMP}" dbType="OracleDbType.Date" />
to:
<parameter name="TIME_STAMP" layout="${longDate}" dbType="OracleDbType.Date" />
and I got the same error
First of all, it's highly recommend to use the built in layout renderers.
For example, ${longdate}, but also ${processid}. Those are optimized and easier to use.
You can find all of them here
Be aware is that using global context (GDC), could be dangerous in multi threaded programs. Also in single threaded you could "leak" some context.
But I guess the real issue here is that your NLog is out to date. DbType support is added in NLog 4.6.
So first update NLog (using Nuget is highly recommend). I use here NLog 4.6.3+, but would recommend to update to NLog 4.7
Then for the code I would recommend to do this:
logger.WithProperty("CustomerId", LogEntity.CUSTOMER_ID).Log(loglevel, "Message");
and configure like this:
<targets>
<target name="TRACEDatabase" type="DataBase" keepConnection="false"
dbProvider="Oracle.ManagedDataAccess.Client" connectionString="${gdc:connectionString}"
commandText="insert into TLOG_SITE_GENERAL_TRACE( PROCESS_ID,TIME_STAMP,CUSTOMER_ID)
values(:PROCESS_ID,:TIME_STAMP,:CUSTOMER_ID)">
<parameter name="PROCESS_ID" layout="${processid}" dbType="Int32" />
<parameter name="TIME_STAMP" layout="${longdate}" dbType="Date" />
<parameter name="CUSTOMER_ID" layout="${event-properties:CUSTOMER_ID}" dbType="Int32" />
</target>
</targets>
See also ${event-properties}
There is no need for using a specific OracleDbType, so therefor I choose to just use "Date" (which is DbType.Date, see DbType
I found the solution!
I changed the dbType parameter from:
<parameter name="TIME_STAMP" layout="${gdc:TIME_STAMP}" dbType="OracleDbType.Date" />
To:
<parameter name="TIME_STAMP" layout="${gdc:TIME_STAMP}" dbType="DateTime" />
and it works!
I have setup NLog on my project that will save the logging to my audit database (separate from the default database). However, it is failing to write to the database. I have a console logger target as well and that is logging as expected. No errors are shown or given. It merely fails to write to the database.
I have tried various methods provided from google, but none seemed to have worked. I have also tried using NLogBuilder to configure the config for that specific controller but still, it doesn't write to the database
nlog.config file:
<configSections>
<section name="nlog" type="NLog.Config.ConfigSectionHandler, NLog" />
</configSections>
<nlog internalLogLevel="Trace">
<targets>
<target name="ConsoleLogger" type="Console"
layout="${longdate}|${level:uppercase=true}|${logger}|${message}"/>
<target name="DatabaseLog" type="Database">
<commandtext>
INSERT INTO Logs
(LogDate, LogLevel, Message, Exception)
VALUES
(#log_date, #log_level, #message, #exception)
</commandtext>
<parameter name="#log_date"
layout="${log_date}"
dbType="DateTime"/>
<parameter name="#thread"
layout="${thread}"
dbType="String"
size="255"/>
<parameter name="#log_level"
layout="${log_level}"
dbType="String"
size="20" />
<parameter name="#logger"
layout="${logger}"
dbType="String"
size="250" />
<parameter name="#message"
layout="${message}"
dbType="String"
size="4000" />
<parameter name="#exception"
layout="${exception}"
dbType="String"
size="4000" />
<dbProvider>MySql.Data.MySqlClient.MySqlConnection, MySql.Data</dbProvider>
<connectionString>User Id=username;Password=password;Host=localhost;Database=audit_database;TreatTinyAsBoolean=false</connectionString>
</target>
</targets>
<rules>
<logger name="*" minlevel="Debug" maxlevel="Fatal" writeTo="ConsoleLogger,DatabaseLog" />
</rules>
</nlog>
Controller method:
public static Logger _logger = LogManager.GetCurrentClassLogger(typeof(ActionerController));
_logger.Info("text");
Main.cs
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureLogging(logging =>
{
logging.ClearProviders();
logging.AddNLog();
}).ConfigureAppConfiguration((hostingContext, config) =>
{
config
.AddJsonFile($"environment-mount/appsettings.json", optional: true)
.AddEnvironmentVariables();
}).ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});
For MySQL, make sure you have installed the MySql.Data package.
There are also some mistakes in your config and code:
${log_date}, ${log_level} and ${thread} don't exist. I guess you mean ${date}, ${level} and ${threadid}. See all options here.
LogManager.GetCurrentClassLogger with an argument expects a logger type - so inherits from NLog.Logger. I think you are looking for LogManager.GetCurrentClassLogger() or LogManager.GetLogger(typeof(ActionerController).FullName). See API docs
Of course there could be other errors, e.g. a wrong query, mistyped colum names or wrong column types. NLog could of course tell you the problem:
Or enable exceptions. In your config <nlog throwExceptions=true >
Or enable the internal log: <nlog internalLogFile="c:\log.txt" internalLogLevel="Warn">. Read more here
Is it possible to use a variable to specify a specific target?
I want to switch between database and filelogging depending on the environment i'm running my application on.
This however does not seem to work:
<logger name="MyLogger" minlevel="Warn" writeTo="${var:myTargetName}" />
In my application startup I have (error occurs on first line):
var logger = NLog.Web.NLogBuilder.ConfigureNLog("nlog.config").GetCurrentClassLogger();
var config = LoadConfiguration();
NLog.LogManager.Configuration.Variables["myTargetName"] = config.GetSection("LogTargets:TargetName").Value;
NLog.LogManager.KeepVariablesOnReload = true;
NLog.LogManager.Configuration.Reload();
When startin the application the following exception is thrown:
"Target ${var:myTargetName} not found."
I guess the variables are not available when parsing the config file.
How can I set the variables so NLOG uses them while parsing the config-file?
Is it possible to use a variable to set an attribute value?
Or is this not supported by NLOG?
Note: I do have another variable in the NLOG config which does work
<target xsi:type="Database" name="databaseTarget">
<connectionString>${var:connectionString}</connectionString>
...........
</target>
I guess the loggers are checked once on startup and the actual target is evaluated when a logevent occurs.
Update: got it working without variable in config file.
I had to remove the logger from the config and create it through code
var myCustomTarget = NLog.LogManager.Configuration.FindTargetByName(config.GetSection("LogTargets:TargetName").Value);
NLog.LogManager.Configuration.AddRuleForAllLevels(myCustomTarget , "MyLogger", true);
One way to do it is to have a different Web.config file based on the environment and there you will change the connection string. That is the method I am using.
Also NLOG is initialized once the application starts thus you cannot change what's written in the NLOG.config
Setting variables via NLog.LogManager.Configuration.Variables does not work for all attributes in nlog.config file. I don't know why, but that's a known problem and that is unfortunate how it works. But there is a simple workaround, here is an example how I solved this problem for the attribute connectionString of a target.
<target xsi:type="Database" name="tDatabase"
dbProvider="Microsoft.Data.Sqlite.SqliteConnection, Microsoft.Data.Sqlite"
connectionString="Data Source =.\${environment:DATABASE_FILE_NAME};"
commandText="INSERT INTO ...">
<parameter name="#MachineName" layout="${machinename}" />
...
</target>
You can than simple set the environment variable in your code like this
Environment.SetEnvironmentVariable("DATABASE_FILE_NAME", "foo.db");
So you just use ${environment:DATABASE_FILE_NAME} and
Environment.SetEnvironmentVariable("DATABASE_FILE_NAME", "foo.db");
instead of ${var:DATABASE_FILE_NAME} and
NLog.LogManager.Configuration.Variables["DATABASE_FILE_NAME"] = "foo.db";
NLog 4.6.7 makes it easier to update the logging-rules at runtime like this:
<nlog>
<variable name="fileMinLevel" value="Off" />
<variable name="databaseMinLevel" value="Off" />
<rules>
<logger minLevel="${var:fileMinLevel}" writeTo="fileTarget" />
<logger minLevel="${var:databaseMinLevel}" writeTo="databaseTarget" />
</rules>
</nlog>
Then you can do this:
if (IsDevelopment())
LogManager.Configuration.Variables["fileMinLevel"] = "Debug";
else
LogManager.Configuration.Variables["databaseMinLevel"] = "Debug";
LogManager.ReconfigExistingLoggers();
See also: https://github.com/NLog/NLog/wiki/Filtering-log-messages#semi-dynamic-routing-rules
See also: https://github.com/NLog/NLog/wiki/Environment-specific-NLog-Logging-Configuration
I'm writing an ASP .NET MVC3 application and as application is expected to be secure, I need a good enterprise application logging architecture.
So I was seeking over existing loggin frameworks and picked the NLog. So at them moment I'm stuck with making DB schema for logs.
Does anyone have good experience in this area? It's anticipate to log a set of actions such as user interactions with system objects, background works, user membership actions, payment transactions and so on.
I've been using NLog for a while now and i'm very happy with it. What I most like about NLog is that you can configure different loglevels to be written to different files and/or databases. It's a very powerfull logging library.
For logging to the database you can use something like below in your config. This is similar to what I use at the company i'm working.
<target xsi:type="Database"
name="TestDatabaseLogging"
connectionString="Data Source=127.0.0.1;Initial Catalog=NLog_Test;User ID=MyLogin;Password=MyPassword"
dbDatabase="NLog_Test">
<commandText>
insert into INNO_LOG ([createDate], [Origin], [LogLevel], [Message], [Exception], [StackTrace]) values (#createDate, #origin, #logLevel, #message, #exception, #stackTrace)
</commandText>
<parameter name="#createDate" layout="${date}"/>
<parameter name="#origin" layout="${callsite}"/>
<parameter name="#logLevel" layout="${level}"/>
<parameter name="#message" layout="${message}"/>
<parameter name="#exception" layout="${exception:format=Message,StackTrace}"/>
<parameter name="#stackTrace" layout="${stacktrace}"/>
</target>
You can use the rules section to log different levels to different files, see example below;
<rules>
<logger name="*" minlevel="Fatal" writeTo="mail" />
<logger name="*" minlevel="Error" writeTo="TestDatabaseLogging" />
<logger name="*" minlevel="Debug" writeTo="file-debug" />
<logger name="*" minlevel="Info" writeTo="file" />
<!--Log to specific files for specific classes.-->
<logger name="_Default" minlevel="Trace" writeTo="file-default" />
<logger name="TestClass" minlevel="Trace" writeTo="file-testclass" />
</rules>
EDIT: Added possible table layout for logging information.
Id | int
CreateDate | datetime
LogLevel | nvarchar
Message | nvarchar(max)
Exception | nvarchar(max)
StackTrace | nvarchar(max)
SourceUrl | nvarchar(255)
UserId | uniqueidentifier
OrderId | int
The layout above is just an rough idea. It totally depends on what you want to log in this table. Though you have to try if it's possible to add additional paramaters other than the ones used by NLog by default.
Like Rob has answered .. stick with NLog :)
If you need a good viewer, don't forget to use Sentinal as a free nLog viewer :)
Don't forget, you can programatically turn on logging sections, while in production .. to view specific areas of the website. And then see them in real-time using Sentinal.
another option is to take advanctage of something like LoggR.net. This streams data (ie. errors or logging information in this case) in real time. I think it's using the awesome SignalR framework to use WebSockets or Long Polling (2 real-time technologies) :)
I am having a bit of a problem using a stored procedure instead of a SQL INSERT statement when using NLog in a C# web application. The connection string "Logger" is correctly configured in Web.config and works properly when replacing the commandText with a SQL statement. I would appreciate a hint in the right direction. In this example the stored procedure is under the "Logs" schema and it is called "LogError".
<targets>
<target xsi:type="Database"
name="dberrorlog"
connectionStringName="Logger"
keepConnection="true"
commandText="[Logs].[LogError]" >
<parameter name="#ProgName" layout="MyAppName"/>
<parameter name="#CompName" layout="${machinename}"/>
<parameter name="#LogLevel" layout="${level}"/>
<parameter name="#UserName" layout="${identity}"/>
<parameter name="#Error" layout="${exception:format=Message}"/>
<parameter name="#SourceObj" layout="${exception:format=Method}"/>
<parameter name="#StackTrace" layout="${exception:format=StackTrace}"/>
</target>
</targets>
<rules>
<logger name="*" minlevel="Error" writeTo="dberrorlog" />
</rules>
From this NLog forum post, try using the text to execute the stored procedure:
commandtext="exec AddActivityLog
#ApplicationName,
#ApplicationTime,
#Severity,
#Logger,
#SaxoID,
#EventID,
#Message,
#URL,
#URLReferrer,
#RemoteAddress,
#Callsite,
#CurrentUICulture,
#ThreadIdentity,
#WindowsIdentity,
#MachineName,
#ProcessID,
#ThreadID,
#ThreadName,
#Stacktrace,
#Exception,
#Cookie,
#FormVariables,
#QueryString,
#HTTPUserAgent"
Side note: Claus Rathje's answer wouldn't render in my browser, so I had to look a the page source to see the configuration he posted.
Note that while #JeffOgata's solution works, it's probably not the way you want to go about solving the problem.
You can do something like this instead:
commandText="AddActivityLog"
commandType="StoredProcedure"
That way you don't have to worry about properly formatting an EXEC query.