Serilog : log different data to different places - c#

I want to log my whole data in a file and log only the errors in my oracle database. I'm using the below code but it's not working.
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Verbose().WriteTo.File("logs/myLog.txt", rollingInterval: RollingInterval.Minute)
.MinimumLevel.Error().WriteTo.Oracle(cfg =>
cfg.WithSettings(logConnectionString, tableSpaceAndTableName: "MY_TABLE")
.UseBurstBatch(true, 1000, true, 1)
.CreateSink())
.CreateLogger();
In this case, i log only error data both in file and oracle database. How must i configure the above code, to store the whole data in a file and the error data in database?

Almost all sinks have a restrictedToMinimumLevel parameter. There's no Serilog.Sinks.Oracle project in the Serilog Github repo so I assume you use Serilog.Sinks.Oracle. The Oracle() method accepts a restrictedToMinimumLevel parameter as well.
You can change your code to :
Log.Logger = new LoggerConfiguration()
.WriteTo.File("logs/myLog.txt",
rollingInterval: RollingInterval.Minute,
restrictedToMinimumLevel:LogEventLevel.Verbose)
.WriteTo.Oracle(cfg =>
cfg.WithSettings(logConnectionString, tableSpaceAndTableName: "MY_TABLE")
.UseBurstBatch(true, 1000, true, 1)
.CreateSink(),
restrictedToMinimumLevel:LogEventLevel.Error )
.CreateLogger();
Configuring database logging typically takes more code than this, so it's a good idea to extract the code to a different method. You'll probably want to use additional columns to extract common attributes like categories, activity IDs etc so you don't have to parse the JSON payload to find specific events:
ILogEventSink ConfigureOracle(BatchLoggerConfiguration cfg)
{
const string column = "ADDITIONALDATACOLUMN";
var columnOptions = new ColumnOptions
{
AdditionalDataColumns = new List<DataColumn>
{
new DataColumn(column , typeof(string))
}
};
return cfg.WithSettings(logConnectionString, tableSpaceAndTableName: "MY_TABLE")
.UseBurstBatch(true, 1000, true, 1)
.CreateSink();
}
...
Log.Logger = new LoggerConfiguration()
.WriteTo.File("logs/myLog.txt",
rollingInterval: RollingInterval.Minute,
restrictedToMinimumLevel:LogEventLevel.Verbose)
.WriteTo.Oracle(ConfigureOracle,LogEventLevel.Error )
.CreateLogger();
Organizing database logging code
I also log to a database. Putting everything into a single expression may be fashionable but the resulting code can quickly become too hard to read and maintain. Fluent APIs aren't always a good idea.
In this case, one almost always needs to specify extra columns to hold common properties like categories and activity IDs, perhaps even customer IDs. This means specifying extra columns. No matter the database product, querying the raw JSON data is more expensive than querying materialized and indexed columns.
It will take some experimentation until one get a table you can actually use for troubleshooting. The database logging configuration should be extracted into a separate a separate method, if not a separate file. Otherwise Startup.cs or, in .NET 6, Program.cs, will become an unreadable mess.

Related

Maintaining a (single) stable filename using Serilog File sink

I am logging the data like below :
.WriteTo.File("log.txt", rollOnFileSizeLimit: true,retainedFileCountLimit: 1,
fileSizeLimitBytes: 10)
This creates a file set like this:
log.txt => log_001.txt => log_002.txt
Since I set retainedFileCountLimit = 1 once the log.text was deleted then log_001.txt
Is there any way to maintain only the log.log file instant of keeping int values in an increased format ?
There is a forked sink, Serilog.Sinks.PersistentFile that does this.
But it's a very debatable ask which is why it's not supported by the standard Serilog.Sinks.File for the foreseeable future.

Serilog Dispose/Close log file-stream

Recently I focused in Serilog to point out a templated path based on the current Date of every LogEvent.
After figuring how to implement this, I finally resolve the path on the fly by using the Date field into LogEvent by using Serilog.Sinks.Map, such as shown below:
return new LoggerConfiguration().WriteTo
.Map(
// Log key
(LogEvent le) => le.Timestamp.Date,
// Log Action
(DateTime date, LoggerSinkConfiguration lc) =>
{
string path = GetFilesPath(date, logName);
lc.File(path);
}
);
public string GetFilePath(DateTime date, string logName) =>
Path.Combine("./Logs", $"{date:yyyy-MM-dd}", $"{logName}.log");
With this, I achieved my goal: writing logs with in a sub folder based on the Date.
The issue is, since Serilog does not know that the pointing path changed, it does not close or dispose the file stream as expected. So, my application leaves files opened day to day, ad infinitum.
It'd be great if someone has faced this approach, to manually close the stream, or if Serilog API exposes somehow automatically close those streams.
Btw, I am using
Serilog 2.9.0
Serilog.Sinks.File 4.1.0
Serilog.Sinks.Map 1.0.1
Edit 05/06/2020 for those reading this afterwards.
Keying every single log event by the Timestamp is a bad idea. By doing so, we are in fact adding an entry per log event (supposing that no events are being emitted at the same time, for simplicity).
Even if we specify the sinkMapCountLimit to 0, which in theory won't keep any event in our map, if that event is configured to write to file (specially with the RollingFile sink), those sinks won't be disposed nor erased from memory.
So, the chunk of code above is leaking memory (and pretty fast).
The Map.Sink documentation warns about this, indeed.
...but isn't suitable when the set of possible key values is open-ended.
Serilog.Sinks.Map accepts a parameter sinkMapCountLimit to control this:
return new LoggerConfiguration().WriteTo
.Map(
// Log key
(LogEvent le) => le.Timestamp.Date,
// Log Action
(DateTime date, LoggerSinkConfiguration lc) =>
{
string path = GetFilesPath(date, logName);
lc.File(path);
},
sinkMapCountLimit: 5
);

c# logging framework - logfiles with explicit names

I´m searching for a logging framework for c# where I can set the logfile name easy from code.
My program starts several background workers which all should log into their own log files.
They use the same classes for their work and sometimes in different threads.
e.g.
backgroundworker1:
var aboImport = new CSVImport(file1); <-- logs into backgroundWorker1.log
aboImport.Execute();
ArchiveFile(file1);
backgroundworker2:
var addressImport = new CSVImport(file2); <-- logs into backgroundWorker2.log
addressImport.Execute();
ArchiveFile(file2);
e.g.
backgroundWorker1.log
backgroundWorker2.log
backgroundWorker3.log
...
heartbeat.log <-- I´can see my workers or main thread is running an working
I´m using using log4net at the moment, but this does not seem to fit my needs.
I found (Create multiple Logfiles with dynamic Names with log4net) where they create dynamic appenders in code.
But I do not understand whow this works, because the log4net definitions are normally tied to the namespaces.
Can other logging frameworks do this more elegant? Which logging framework do you recommend for this?
Stay away from Log4Net. It's not an actively developed project. Last release was almost 1 yr ago (12/2015). NLog is much better supported.
With that being said, all the logging frameworks let you do everything template driven with a bunch of different "macros"... you might base the filename on threadId or some other dynamic value... or just pass it in through a key/value pair. You have to look at all the "macros" available and pick the one that works in your scenario.
With nlog I could easy create loggers with different file targets.
private static void Main(string[] args)
{
var config = LogManager.Configuration;
AddLogger(config, "logger1", #"C:\Temp\Logfile\Log1.txt");
AddLogger(config, "logger2", #"C:\Temp\Logfile\Log2.txt");
LogManager.Configuration = config;
var logger1 = LogManager.GetLogger("logger1");
logger1.Debug("Test1");
var logger2 = LogManager.GetLogger("logger2");
logger2.Debug("Test2");
}
private static void AddLogger(LoggingConfiguration config, string loggerName, string fileName)
{
var fileTarget = new FileTarget();
config.AddTarget(loggerName, fileTarget);
fileTarget.FileName = fileName;
var rule = new LoggingRule(loggerName, LogLevel.Debug, fileTarget);
config.LoggingRules.Add(rule);
}

Serilog - How do I render formatted message to string manually?

I have written a logging framework that uses Log4Net, Nlog and Serilog interchangeably. Every call to the logger, fires an event before the log entry is written. This optionally pushes entries via SignalR to connected web clients.
Before the serilog addition, I used string.Format to get the formatted text. Now with that great destructuring has come great responsibility. string.Format obviously doesn't like {#0} or {data} in the string.
// log the event before engaging with the logger
LogEventBus.Handle(LogLevels.Info, DateTime.Now, msg, args);
if (DiagnosticLevel < level)
return;
_logger.Info(msg, args);
Is there any way to get the serilog generated output, directly as string?
I started writing a memory sink, but that moves away from my centralised event based logging, and completely breaks away from the other libraries I have implemented.
Any suggestions?
You can convert Serilog's message format to a standard .NET format string ({0} etc) like this:
var parser = new MessageTemplateParser();
var template = parser.Parse(templateMessage);
var format = new StringBuilder();
var index = 0;
foreach (var tok in template.Tokens)
{
if (tok is TextToken)
format.Append(tok);
else
format.Append("{" + index++ + "}");
}
var netStyle = format.ToString();
Once you have a standard format string you can pass this through or use string.Format() with it and args.
It's not going to be super-efficient - hooking deeper into the Serilog pipleine (ILogEventEnricher) should be better. As another commenter suggested, it may be better just to embrace a single logging framework here.
Do your logging in two steps.
Write log message to a TextWriter and read the value from the
TextWriter
(https://github.com/serilog/serilog/wiki/Provided-Sinks#textwriter)
Write that already formatted value into the real logger
Whilst this might work, I worry about your architecture here. It all sounds like yuo are creating huge dependencies on Serilog, whilst you are also using several other logging frmeworks. Choose one logging framework OR use really generic features. C# has introduced string interpolation, not as fancy as Serilogs serialization etc, but works. I'd go back to KISS.

Change Location and Name of Log file at runtime, based on GUID passed in QueryString

Right now whenever I run application It creates a file (file name specified in Web.Config Logging Section).
What I want is to change the path and name of the file (based on GUID passing as query sting) at run time.
I did some research, but examples available are not working for me. Will appreciate if someone can provide a code sample.
As you've discovered, your requirements are not a natural fit for the configuration based approach of Enterprise Library. That's not to say that it can't be done.
One way to accomplish what you want is to use programmatic configuration of the logging block.
One approach would be to create an IDictionary to hold a mapping of IDs to LogWriters. When an ID comes in check to see if a LogWriter already exists in the IDictionary. If it does then use it and if it doesn't then create a new LogWriter.
Here is some sample code that uses the Logging Fluent Interface to configure a UnityContainer and then resolves a LogWriter and saves it to a Dictionary:
int id = 123123;
Dictionary<int, LogWriter> loggers = new Dictionary<int, LogWriter>();
ConfigurationSourceBuilder builder = new ConfigurationSourceBuilder();
builder.ConfigureLogging()
.WithOptions
.DoNotRevertImpersonation()
.SpecialSources.LoggingErrorsAndWarningsCategory.SendTo.FlatFile("Flat File Listener").ToFile(#"trace.log")
.LogToCategoryNamed("General")
.WithOptions.SetAsDefaultCategory()
.SendTo.FlatFile("AppSpecificFlatFile" + id)
.ToFile("logging" + id + ".log")
;
DictionaryConfigurationSource configSource = new DictionaryConfigurationSource();
builder.UpdateConfigurationWithReplace(configSource);
coreExtension = new EnterpriseLibraryCoreExtension(configSource);
IUnityContainer container = new UnityContainer();
container.AddExtension(coreExtension);
var logger = container.Resolve<LogWriter>();
loggers[id] = logger;
This is just a sample and is not thread safe or abstracted but should hopefully help.
If you just need to change the trace log file at runtime, you can simply remove the current tracelistener using
System.Diagnostics.Trace.Listener.Remove("Listener1");
Then add a new Listener with the new log file name as,
System.Diagnostics.Trace.Listener.Add(new System.Diagnostics.TextWriterTraceListener("NewLogFileName.txt","Listener2");
Now your logs will be recorded to NewLogFileName.txt

Categories

Resources