Recently I focused in Serilog to point out a templated path based on the current Date of every LogEvent.
After figuring how to implement this, I finally resolve the path on the fly by using the Date field into LogEvent by using Serilog.Sinks.Map, such as shown below:
return new LoggerConfiguration().WriteTo
.Map(
// Log key
(LogEvent le) => le.Timestamp.Date,
// Log Action
(DateTime date, LoggerSinkConfiguration lc) =>
{
string path = GetFilesPath(date, logName);
lc.File(path);
}
);
public string GetFilePath(DateTime date, string logName) =>
Path.Combine("./Logs", $"{date:yyyy-MM-dd}", $"{logName}.log");
With this, I achieved my goal: writing logs with in a sub folder based on the Date.
The issue is, since Serilog does not know that the pointing path changed, it does not close or dispose the file stream as expected. So, my application leaves files opened day to day, ad infinitum.
It'd be great if someone has faced this approach, to manually close the stream, or if Serilog API exposes somehow automatically close those streams.
Btw, I am using
Serilog 2.9.0
Serilog.Sinks.File 4.1.0
Serilog.Sinks.Map 1.0.1
Edit 05/06/2020 for those reading this afterwards.
Keying every single log event by the Timestamp is a bad idea. By doing so, we are in fact adding an entry per log event (supposing that no events are being emitted at the same time, for simplicity).
Even if we specify the sinkMapCountLimit to 0, which in theory won't keep any event in our map, if that event is configured to write to file (specially with the RollingFile sink), those sinks won't be disposed nor erased from memory.
So, the chunk of code above is leaking memory (and pretty fast).
The Map.Sink documentation warns about this, indeed.
...but isn't suitable when the set of possible key values is open-ended.
Serilog.Sinks.Map accepts a parameter sinkMapCountLimit to control this:
return new LoggerConfiguration().WriteTo
.Map(
// Log key
(LogEvent le) => le.Timestamp.Date,
// Log Action
(DateTime date, LoggerSinkConfiguration lc) =>
{
string path = GetFilesPath(date, logName);
lc.File(path);
},
sinkMapCountLimit: 5
);
Related
I'm using the OnEntryWritten event in order to get events from the event log when they are fired.
The problem I started to see today is that some messages come with parameters unset.
For example:
The Windows Filtering Platform has permitted a bind to a local port. Application Information: Process ID:9852 Application Name:\device\harddiskvolume7\program files (x86)\google\chrome\application\chrome.exe Network Information: Source Address::: Source Port:51714 Protocol:17 Filter Information: Filter Run-Time ID:0 Layer Name:%%14608 Layer Run-Time ID:38
You can see the %%14608 parameter. This comes with a value if I see the same log in the Event Viewer.
I'm running a windows service as LocalSystem, so I don't know if this is a permission issue or this technology is not useful at all.
I have tried the rest offered on C# and they also don't meet my requirements.
UPDATE: this is the code I'm using to read the events.
1) First I subscribe to the corresponding Event log:
private void Subscribe()
{
_eventLog.EnableRaisingEvents = true;
var callbackFunction = new EntryWrittenEventHandler(OnEntryWritten);
_eventLog.EntryWritten += callbackFunction;
// Save a reference for callbackFunction
_eventHandler = callbackFunction;
}
2) Then on the callback method, I read data from the message:
public void OnEntryWritten(Object source, EntryWrittenEventArgs entryArgs)
{
// When overwrite policy is enabled, this will trigger for all elements when it starts writing new ones
try
{
var entry = entryArgs.Entry;
var timeWritten = entry.TimeWritten;
// This comes with %% values depending on the log
string message = entry.Message;
}
catch(Exception ex)
{
...
}
}
3) The event log variable is simply initialized as:
var eventLog = EventLog.GetEventLogs().FirstOrDefault(el => el.Log.Equals(logName, StringComparison.OrdinalIgnoreCase));
I need some help on this, honestly I don't know what else to try.
UPDATE
I'm adding some images here so everybody can understand the situation a little bit better. To be honest, it looks like there's no solution but to implement a dictionary and replace manually the required values, which appear to be always constants.
This is what I see on the Event Viewer for a given Event ID:
This is what I see on my program when I read that entry:
You can clearly see that the following values:
"Machine key." (Key type)
"Read persisted key from file." (Operation)
Are coming unmapped in the ReplacementStrings and the Message properties as: %%2499 and %%2458
This is the message value I get on the program:
"Key file operation.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-1-5-18\r\n\tAccount Name:\t\tMyAccount$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3e7\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t6644\r\n\tProcess Creation Time:\t2019-04-03T12:17:24.587994400Z\r\n\r\nCryptographic Parameters:\r\n\tProvider Name:\tMicrosoft Software Key Storage Provider\r\n\tAlgorithm Name:\tUNKNOWN\r\n\tKey Name:\t816339d2-c476-4f1e-bc40-954f0aa0f851\r\n\tKey Type:\t%%2499\r\n\r\nKey File Operation Information:\r\n\tFile Path:\tC:\ProgramData\Microsoft\Crypto\Keys\6d55a45fd69327293e9ed3e7930f4565_5663a8bb-2d1d-4c0d-90c1-624beddabe9c\r\n\tOperation:\t%%2458\r\n\tReturn Code:\t0x0"
What can be done here? There also nothing in entry.Data that might help me out to obtain both values.
No, I believe you're mistaken, sorry that this answer is too late. I found a similar event that was raised by chrome and evaluated if there's anything missed by the event handler. There wasn't anything missed. The message I got in my console output exactly matched what I saw in my Event Viewer.
A better solution would be to
Use entry.Data to get the complete data instead of entry.Message
entry.Data will return a byte[] which you can convert to a string. Here's the link to all properties that an entry will have.
I have written a logging framework that uses Log4Net, Nlog and Serilog interchangeably. Every call to the logger, fires an event before the log entry is written. This optionally pushes entries via SignalR to connected web clients.
Before the serilog addition, I used string.Format to get the formatted text. Now with that great destructuring has come great responsibility. string.Format obviously doesn't like {#0} or {data} in the string.
// log the event before engaging with the logger
LogEventBus.Handle(LogLevels.Info, DateTime.Now, msg, args);
if (DiagnosticLevel < level)
return;
_logger.Info(msg, args);
Is there any way to get the serilog generated output, directly as string?
I started writing a memory sink, but that moves away from my centralised event based logging, and completely breaks away from the other libraries I have implemented.
Any suggestions?
You can convert Serilog's message format to a standard .NET format string ({0} etc) like this:
var parser = new MessageTemplateParser();
var template = parser.Parse(templateMessage);
var format = new StringBuilder();
var index = 0;
foreach (var tok in template.Tokens)
{
if (tok is TextToken)
format.Append(tok);
else
format.Append("{" + index++ + "}");
}
var netStyle = format.ToString();
Once you have a standard format string you can pass this through or use string.Format() with it and args.
It's not going to be super-efficient - hooking deeper into the Serilog pipleine (ILogEventEnricher) should be better. As another commenter suggested, it may be better just to embrace a single logging framework here.
Do your logging in two steps.
Write log message to a TextWriter and read the value from the
TextWriter
(https://github.com/serilog/serilog/wiki/Provided-Sinks#textwriter)
Write that already formatted value into the real logger
Whilst this might work, I worry about your architecture here. It all sounds like yuo are creating huge dependencies on Serilog, whilst you are also using several other logging frmeworks. Choose one logging framework OR use really generic features. C# has introduced string interpolation, not as fancy as Serilogs serialization etc, but works. I'd go back to KISS.
I am trying to use WebConfigurationManager.AppSettings[string] to read/write values to be stored in a configuration file on the server. I thought that it read/wrote it to the Web.config file, or maybe app.config. However, I ran the following test code - which the first time through (as desired) it throws an exception and but writes 'NOTSet' to that configuration entry - this allows me go easily go edit the file and change it to the correct value.
After running it a 2nd time, I can see the value returned is "NOTSet" - exactly as I would like. The code is working exactly as planned.
Except - Where is it written? I did a 'baregrep NOTSet .' - which recursively searched the ENTIRE project source directory and ONLY found the line in code that set the var - which means it was not written to anything in the entire project tree - not web.config, app.config or any other file.
I sighed, and said it must be in the registry - I searched the whole registry for anything with that value, then tried the key value - nothing.
Yet, running the program IS pulling the value that was set on the last run! Where did my data go? I want to be able to edit the value- and more importantly, have a FILE that I can copy to the production web server with all the GUIDs set correctly.
public enum Guids
{
IncidentSourceEnumPortal,
WorkItemClass,
WorkItemManagementPack
}
public class GuidConsts
{
static readonly Dictionary<Guids, Guid> GuidList = new Dictionary<Guids, Guid>();
public static Guid Guids(Guids guidId)
{
if (!GuidList.ContainsKey(guidId))
{
string id = WebConfigurationManager.AppSettings[guidId.ToString()];
Guid newGuid;
if ( (id == null) || Guid.TryParse(id, out newGuid))
{
WebConfigurationManager.AppSettings[guidId.ToString()] = "NOTSet";
throw new Exception(String.Format("Invalid guid - not found in Config: {0}", guidId));
}
GuidList.Add(guidId, newGuid);
return newGuid;
}
return GuidList[guidId];
}
}
To save the web.config file you need to call Configuration.Save() to persist changes. They are not automatically saved when you change a value:
https://msdn.microsoft.com/en-us/library/ms134088%28v=vs.110%29.aspx
Writing to web.config on the fly is not really advisable anyway, for one thing it will cause the app pool to be restarted potentially resulting in loss of session state across your application and probably other application-wide things you don't want to happen.
I have source "Source401" used for log "Log401". I need to use this source for "Log402" log and delete the log "Log401". (If we can rename “Log401” as “Log402” that is also fine. But all this need to be done programmatically)
With the code below, I am getting the following exception. What is the best way to achieve it?
Source Source401 already exists on the local computer.
Note: When I delete the old log, it is working fine. But the events are not getting created.
UPDATE
From MSDN
The operating system stores event logs as files. When you use EventLogInstaller or CreateEventSource to create a new event log, the associated file is stored in the %SystemRoot%\System32\Config directory on the specified computer. The file name is set by appending the first 8 characters of the Log property with the ".evt" file name extension.
The source must be unique on the local computer; a new source name cannot match an existing source name or an existing event log name. Each source can write to only one event log at a time; however, your application can use multiple sources to write to multiple event logs.
CODE
string source = "Source401";
string logName = "Log402";
string oldLogName = "Log401";
string eventName = "Sample Event";
string machineName = ".";
if (!EventLog.Exists(logName, machineName))
{
////Delete old log
//if (EventLog.Exists(oldLogName, machineName))
//{
// EventLog.Delete(oldLogName, machineName);
//}
//Create Source for the Log
EventLog.CreateEventSource(source, logName, machineName);
//Create Event
EventLog eventLog = new EventLog(logName, machineName, source);
eventLog.WriteEntry(eventName);
try
{
eventLog.WriteEntry(eventName, EventLogEntryType.Warning, 234, (short)3);
}
catch (Exception exception)
{
int x = 0;
}
The exception is telling you exactly what the problem is. The event source named "Source401" already exists. You're deleting the old event log, "Log401", but you're not deleting the event source.
As the documentation says:
The operating system stores event logs as files. When you use EventLogInstaller or CreateEventSource to create a new event log, the associated file is stored in the %SystemRoot%\System32\Config directory on the specified computer. The file name is set by appending the first 8 characters of the Log property with the ".evt" file name extension.
The source must be unique on the local computer; a new source name cannot match an existing source name or an existing event log name. Each source can write to only one event log at a time;
Also, this little nugget:
If a source has already been mapped to a log and you remap it to a new log, you must restart the computer for the changes to take effect.
In addition, you might want to consider this, also from the documentation:
Create the new event source during the installation of your application. This allows time for the operating system to refresh its list of registered event sources and their configuration. If the operating system has not refreshed its list of event sources, and you attempt to write an event with the new source, the write operation will fail
Finally, the CreateEventSource method you're calling is marked obsolete, and has been since .NET 2.0. There's usually a good reason for methods to be marked obsolete. You should be calling CreateEventSource(EventSourceCreationData).
I think you need to re-think the way you're using event logs. Your application shouldn't be creating and deleting logs that way. It's not how they're intended to be used.
I am trying to find a way to get back my previous variable's value so that I can resume my application to it's previous running state before it crashed when I MANUALLY relaunch it. I am currently using a 'config' file that is saved in the project folder. Is there a better way to do this?
Some small parts of my code that I want to save.
if (EventID == WIA.EventID.wiaEventItemCreated)
{
if (d != null)
{
foreach (Property p in d.Properties)
{
if (p.Name.Equals("Pictures Taken"))
Console.WriteLine("Taken");
}
wiaImageFile = (WIA.ImageFile)(d.Items[d.Items.Count].Transfer(FormatID.wiaFormatJPEG));
wiaImageFile.SaveFile(Properties.Settings.Default.FolderNameRaw + "\\" + imageCount + ".jpg");
imageCount++;//I want to save this count so that I can continue the sequence even after the application crashes
Pluck.Default.PhotoExistsInDirectory = true;
FacebookControls fbc = new FacebookControls();
if(Properties.Settings.Default.UseFB == true)
fbc.UploadPhotos();
}
}
A config file is a good answer in general. Your other options are usually the registry or the database, but I would argue that a config file is a safer option.
The thing about persisting this information is that it may cause an error again, and if so you'll want to be able to discard it easily. A file (of course stored in user settings space) is perhaps the way to do that. If need be you can instruct the user to delete the file. It's a more complicated fix for a user to access the registry or the database.
Also, you should wrap up your state in an appropriate object, and build initialization logic that initializes the state object and has mechanism for not loading for the config file.
I use config files. I also have a global exception handler that catches any exceptions and offers the chance to save any files (including those that the user is working on) before the app closes.
I would also agree with C Ross that you may persist the data that caused the app to fail. Another option that will not get you right back is to persist the settings at regular intervals using a timer or background process. I use this with several backups a bit like the system restore feature in windows.
You can handle UnhandledException, Application_ThreadException and Application.ApplicationExit Event, and try saving your data there:
http://www.switchonthecode.com/tutorials/csharp-tutorial-dealing-with-unhandled-exceptions
As #C. Ross said, user config file is a good choice.
Of course, first you'll have to preserve your application's state in some object during runtime.