In our C# app, we write files to Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments). Our log4net logfile should go there too, so we've defined application.conf as follows:
<appender name="LogFile" type="log4net.Appender.RollingFileAppender">
<appendToFile value="true"/>
<file value="%USERPROFILE%\My Documents\MyApp\log.txt"/>
...snip...
</appender>
This works, until we run in on a PC which has a non-English Windows. Because then, SpecialFolder.MyDocuments points to the folder Mijn Documenten, while the log still goes to My Documents. Confusion ensues, because now our files are in two places.
I want to write my log to the "real" My Documents folder. How do I do this?
I tried to find an environment variable like %USERPROFILE%, but there doesn't seem to exist one for My Documents.
There's a registry key that defines the true location of My Documents but it's not accessible from application.conf.
I tried to override the File parameter of my appender programmatically, like this:
public static void ConfigureLogger()
{
XmlConfigurator.Configure();
Hierarchy hierarchy = (Hierarchy)log4net.LogManager.GetRepository();
foreach (var appender in hierarchy.Root.Appenders)
{
if (appender is FileAppender)
{
var fileAppender = appender as FileAppender;
var logDirectory = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments), "MyApp");
var pathElements = fileAppender.File.Split(Path.DirectorySeparatorChar);
var logFileName = pathElements.Last();
var fullLogFilePath = Path.Combine(logDirectory, logFileName);
fileAppender.File = fullLogFilePath;
}
}
}
This doesn't work either: when I inspect the internals of my logger, the File property happily reports Mijn Documenten, but in the mean time the logs still go to My Documents.
I'm running out of ideas!
Change the line
<file value="%USERPROFILE%\My Documents\MyApp\log.txt"/>
to
<file type="log4net.Util.PatternString" value="%envFolderPath{MyDocuments}\MyApp\log.txt" />
The accepted answer is out of date.
You should now use:
<file value="${UserProfile}\Documents\log-messages.log" />
(This should work even on Windows 7, where "Documents" is referred to by the alias "My Documents".)
${UserProfile} will map to C:\Users[UserName], even if you don't see this variable explicitly defined in your environment variable list.
There seem to be at least 2 approaches. The simplest is kind of a hack:
Specify a custom environment variable to indicate the root path in you log4net config:
<file value="%MYAPP_USER_ROOTFOLDER%\MyApp\log.txt"/>
At startup, before initializing logging, set this environment variable value:
Environment.SetEnvironmentVariable("MYAPP_USER_ROOTFOLDER", Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments));
The more complicated but recommended approach is explained here: http://marc.info/?l=log4net-user&m=110142086820117&w=2 and here http://ziqbalbh.com/articles/log4net-another-way-to-change-log-file-location-on-runtime/
Related
I'm looking at using log4net as my logging framework of choice for a new project starting shortly. One issue that I've run into during prototyping that I can't find a definitive answer for is how you can clean or mask message content in a configurable and tidy way.
Hypothetically let's say I want several cleaners to be put in action but I also want to follow the single responsibility principle. Some cleaner examples:
Cardnumber/PAN cleaner
Password cleaner
Private data cleaner
I know that you should never be logging this sort of information in plain text and the code executing the logs will never knowingly be doing this. I want to have a last level of protection however in case data becomes malformed and sensitive data somehow slips into somewhere it shouldn't; logs being the worst case scenario.
Option 1:
I've found this StackOverflow article which details a possible solution however it involves the use of reflection. This is not desirable for performance but it also seems hacky to manipulate internal storage mechanisms.
Editing-log4net-messages-before-they-reach-the-appenders
Option 2:
The suggested answer on the same question suggests the use of a PatternLayoutConverter. This is fine for a single cleaner operation but you are unable to use multiple operations such as the below:
public class CardNumberCleanerLayoutConverter : PatternLayoutConverter
{
protected override void Convert(TextWriter writer, LoggingEvent loggingEvent)
{
string message = loggingEvent.RenderedMessage;
// TODO: Replace with real card number detection and masking.
writer.Write(message.Replace("9", "*"));
}
}
<layout type="log4net.Layout.PatternLayout">
<converter>
<name value="cleanedMessage" />
<type value="Log4NetPrototype.CardNumberCleanerLayoutConverter, Log4NetPrototype" />
</converter>
<converter>
<name value="cleanedMessage" />
<type value="Log4NetPrototype.PasswordCleanerLayoutConverter, Log4NetPrototype" />
</converter>
<conversionPattern value="%cleanedMessage" />
</layout>
In the case of a naming collision as demonstrated above, the converter loaded last will be the one which is actioned. Using the above example, this means that passwords will be cleaned but not card numbers.
Option 3:
A third option which I've tried is the use of chained ForwarderAppender instances but this quickly complicates the configuration and I wouldn't consider it an ideal solution. Because the LoggingEvent class has an immutable RenderedMessage property we are unable to change it without creating a new instance of the LoggingEvent class and passing it through as demonstrated below:
public class CardNumberCleanerForwarder : ForwardingAppender
{
protected override void Append(LoggingEvent loggingEvent)
{
// TODO: Replace this with real card number detection and masking.
string newMessage = loggingEvent.RenderedMessage.Replace("9", "*");
// What context data are we losing by doing this?
LoggingEventData eventData = new LoggingEventData()
{
Domain = loggingEvent.Domain,
Identity = loggingEvent.Identity,
Level = loggingEvent.Level,
LocationInfo = loggingEvent.LocationInformation,
LoggerName = loggingEvent.LoggerName,
ExceptionString = loggingEvent.GetExceptionString(),
TimeStamp = loggingEvent.TimeStamp,
Message = newMessage,
Properties = loggingEvent.Properties,
ThreadName = loggingEvent.ThreadName,
UserName = loggingEvent.UserName
};
base.Append(new LoggingEvent(eventData));
}
}
public class PasswordCleanerForwarder : ForwardingAppender
{
protected override void Append(LoggingEvent loggingEvent)
{
// TODO: Replace this with real password detection and masking.
string newMessage = loggingEvent.RenderedMessage.Replace("4", "*");
// What context data are we losing by doing this?
LoggingEventData eventData = new LoggingEventData()
{
Domain = loggingEvent.Domain,
Identity = loggingEvent.Identity,
Level = loggingEvent.Level,
LocationInfo = loggingEvent.LocationInformation,
LoggerName = loggingEvent.LoggerName,
ExceptionString = loggingEvent.GetExceptionString(),
TimeStamp = loggingEvent.TimeStamp,
Message = newMessage,
Properties = loggingEvent.Properties,
ThreadName = loggingEvent.ThreadName,
UserName = loggingEvent.UserName
};
base.Append(new LoggingEvent(eventData));
}
}
Matching configuration (very hard to follow):
<log4net>
<appender name="LocatedAsyncForwardingAppender" type="Log4NetPrototype.LocatedAsyncForwardingAppender, Log4NetPrototype">
<appender-ref ref="CardNumberCleanerForwarder" />
</appender>
<appender name="CardNumberCleanerForwarder" type="Log4NetPrototype.CardNumberCleanerForwarder, Log4NetPrototype">
<appender-ref ref="PasswordCleanerForwarder" />
</appender>
<appender name="PasswordCleanerForwarder" type="Log4NetPrototype.PasswordCleanerForwarder, Log4NetPrototype">
<appender-ref ref="LogFileAppender" />
</appender>
<appender name="LogFileAppender" type="Log4NetPrototype.LogFileAppender, Log4NetPrototype">
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%m" />
</layout>
</appender>
<root>
<level value="DEBUG" />
<appender-ref ref="LocatedAsyncForwardingAppender" />
</root>
</log4net>
Does anyone have another suggestion for how this could be implemented where theoretically n number of cleaners could be configured at the cost of performance?
In your question you are already saying that you should go to the cause and not logging any sensitive data. This can be enforced by a fourth option of using code reviews, and look at the data being logged. Your logging statements should never log any sensitive data, because the cause a security risk. Trusting on any code with filters the sensitive data will probably fail if you make changes to your project. Your QA process has to be really good to catch this kind of mistakes (I've never seen a tester going through all logs). So I would go for option 4 which is making sure you do not log this kind of information in the first place.
I'm reworking an existing logging system to use NLog Instead. I've manually added a reference to NLog in my configuration file:
<targets>
<target xsi:type="File" name="fileLogger" fileName="${basedir}\TRACE\${date:format=yyyy-MM-dd}.log" layout="TimeStamp:[${date}]|${message}" />
<target xsi:type="Memory" name="MemLogger" layout="TimeStamp:[${date}]|${message}|${Type}" />
</targets>
<rules>
<logger name="ApplicationLogger" levels="Info,Warn,Error" writeTo="fileLogger" />
<logger name="ApplicationLogger" levels="Info,Warn,Error" writeTo="fileLogger" />
</rules>
What I want to do is in code pull out the MemLogger Logs and access the different parts (Type, Message, Timestamp). How would I accomplish this? I've seen how to create a new log from scratch, but I don't want to create a new memorytarget, I want to access the existing one in my config, pull the log data out of it, and then clear the memory (so that I don't have a memory leak).
How to I access the MemLogger MemoryTarget in C#?
Unfortunately you cannot access different parts of log. Logs stored as rendered strings, with all layout renderers already replaced with their values. All you can do is manually parse each log string. E.g.
var target =(MemoryTarget)LogManager.Configuration.FindTargetByName("MemLogger");
foreach (string log in target.Logs)
{
var parts = log.Split('|');
var date = parts[0].Replace("TimeStamp:[", "").TrimEnd(']');
var message = parts[1];
var type = parts[2];
//...
}
Something like this:
var target = LogManager.Configuration.FindTargetByName("MemLogger");
Optionally, you can cast received target to MemoryTarget type.
I've just had to do this myself:
Natively:
IList<string> logs = LogManager.Configuration.FindTargetByName<MemoryTarget>("MyTarget").Logs;
or using a Cast:
IList<string> logs = ((MemoryTarget)LogManager.Configuration.FindTargetByName("MyTarget")).Logs;
I have a program which uses SLF for logging. This program runs 24/7 and I would like to upload the log files to a remote server every night for later review.
My question is, how do I release the file lock for the log without closing the program?
I was hoping to suspend logging, upload the logs, either delete the log file or erase the contents, and then resume logging.
ILogger logger = LoggerService.GetLogger(typeof(TaskScheduler).FullName);
// Other initialization here
foreach (var task in managedTasks.OrderBy(t => t.Priority))
{
if (task.NextRunTime <= DateTime.Now)
{
dataManager.CurrentStatus = AppStatus.Running;
if (task.Name == "Log Sender")
{
logger = null;
}
// Run the task
if (task.Name == "Log Sender")
{
logger = LoggerService.GetLogger(typeof(TaskScheduler).FullName);
}
dataManager.CurrentStatus = AppStatus.Idle;
}
}
Currently, when I do this, I'm still getting an IOException because the file is still locked by the Task Scheduler.
EDIT: I'm using SLF over log4net if that helps.
I would consider this as a design issue rather than technical. Log events for each day in an individual file and upon next day you can do anything you want with yesterday's log file. You can narrow log period according to your need (i.g. every 12 hours).
After posting this question and getting the recommendations to reconsider my design. I started to look into creating a rolling log file for each date. I happened across this, which solved my problem completely:
<appender name="InfoRollingLogFileAppender" type="log4net.Appender.RollingFileAppender,log4net">
<param name="File" value="log.txt" />
<param name="StaticLogFileName" value="true"/>
<maximumFileSize value="1024KB" />
<appendToFile value="true" />
<lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%logger: %date{dd MMM yyyy HH:mm:ss} [%thread] %-5level - %message %newline" />
</layout>
</appender>
I configured the log4net appender in my app.config to follow the lockingModel to include a "MinimalLock" this enabled me to pull the contents of the logs, upload them to the server and then erase the local logs without having to create separate log files for a specified time/date span.
Again, thanks to everyone for your help, I wouldn't have stumbled across this handy change without your suggestions.
I want to change the location where my application looks for the app.config file.
I know that I can use ConfigurationManager.OpenExeConfiguration() to access an arbitrary config file - however, when the .Net Framework reads the config file (for ConnectionStrings or EventSources, for instance), it will look at the default location. I want to actually change the location, globally for the entire .Net Framework (for my application, of course).
I also know that I can use AppDomainSetup to change the location of the app.config for a new AppDomain. However, that doesn't apply to the primary AppDomain of the application.
I also know that I can override function Main() and create a new AppDomain as above and run my application in that new AppDomain. However, that has other side-effects - for instance, Assembly.GetEntryAssembly() will return a null reference.
Given how everything else works in .Net, I would expect there to be some way to configure the startup environment of my application - via a Application Manifest, or some such - but I have been unable to find even a glimmer of hope in that direction.
Any pointer would be helpful.
David Mullin
I used the approach with starting another AppDomain from Main(), specifying the "new" location of the configuration file.
No issues with GetEntryAssembly(); it only returns null, when being called from unmanaged code - or at least it doesn't for me, as I use ExecuteAssembly() to create/run the second AppDomain, much like this:
int Main(string[] args)
{
string currentExecutable = Assembly.GetExecutingAssembly().Location;
bool inChild = false;
List<string> xargs = new List<string>();
foreach (string arg in xargs)
{
if (arg.Equals("-child"))
{
inChild = true;
}
/* Parse other command line arguments */
else
{
xargs.Add(arg);
}
}
if (!inChild)
{
AppDomainSetup info = new AppDomainSetup();
info.ConfigurationFile = /* Path to desired App.Config File */;
Evidence evidence = AppDomain.CurrentDomain.Evidence;
AppDomain domain = AppDomain.CreateDomain(friendlyName, evidence, info);
xargs.Add("-child"); // Prevent recursion
return domain.ExecuteAssembly(currentExecutable, evidence, xargs.ToArray());
}
// Execute actual Main-Code, we are in the child domain with the custom app.config
return 0;
}
Note that we are effectively rerunning the EXE, just as a AppDomain and with a different config. Also note that you need to have some "magic" option that prevents this from going on endlessly.
I crafted this out from a bigger (real) chunk of code, so it might not work as is, but should illustrate the concept.
I am not sure why you want to change the location of your config file - perhaps there can be different approach for solving your actual problem. I had a requirement where I wanted to share configuration file across related applications - I had chosen to use own xml file as it had given me extra benefit of having complete control over the schema.
In your case, it's possible to externalize sections of your config file to a separate file using configSource property. See here under "Using External Configuration Files" to check how it has been done for connection strings section. Perhaps, this may help you.
var configPath = YOUR_PATH;
if (!Directory.Exists(ProductFolder))
{
Directory.CreateDirectory(ProductFolder);
}
if (!File.Exists(configPath))
{
File.WriteAllText(configPath, Resources.App);
}
var map = new ExeConfigurationFileMap
{
ExeConfigFilename = configPath,
LocalUserConfigFilename = configPath,
RoamingUserConfigFilename = configPath
};
Configuration config = ConfigurationManager.OpenMappedExeConfiguration(map, ConfigurationUserLevel.None);
Then use config member as you want.
Another approach is to leave the config file with the executable file and move the relevant changeable sections to external xml files which can be in whatever location you choose.
If you are using your config file in a readonly capacity, then you can add the relevant chunks to an XML file in a different location using XML Inlcude. This won't work if you are trying to write values back directly to app.config using the Configuration.Save method.
app.config:
<?xml version="1.0"?>
<configuration xmlns:xi="http://www.w3.org/2001/XInclude">
<appSettings>
<xi:include href="AppSettings.xml"/>
</appSettings>
<connectionStrings>
<xi:include href="ConnectionStrings.xml"/>
</connectionStrings>
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7"/></startup>
</configuration>
ConnectionStrings.xml:
<?xml version="1.0"?>
<add name="Example1ConnectionString"
connectionString="Data Source=(local)\SQLExpress;Initial Catalog=Example1DB;Persist Security Info=True;User ID=sa;Password=password"
providerName="System.Data.SqlClient" />
<add name="Example2ConnectionString"
connectionString="Data Source=(local)\SQLExpress;Initial Catalog=Example2DB;Persist Security Info=True;User ID=sa;Password=password"
providerName="System.Data.SqlClient" />
AppSettings.xml:
<?xml version="1.0"?>
<add key="Setting1" value="Value1"/>
<add key="Setting2" value="Value2"/>
A file URI looks like this:
file:///C:/whatever.txt
You can even define failover files in case the one you are trying to reference is missing. This pattern is from https://www.xml.com/pub/a/2002/07/31/xinclude.html:
<xi:include href="http://www.whitehouse.gov/malapropisms.xml">
<xi:fallback>
<para>
This administration is doing everything we can to end the stalemate in
an efficient way. We're making the right decisions to bring the solution
to an end.
</para>
</xi:fallback>
I currently use log4net with a RollingFileAppender.
As each Log call is made I'd like to store this in memory. At the end of my console application run I'd like to (if an app.config setting is true) take only the Warns and Fatals and send all these messages in an Email. I notice MemoryAppender but not quite sure how to use it. Also see SMTPAppender but not sure it is the right tool, else I'll use MemoryAppender and somehow filter out only events of Levels Warn/Fatal and then email using the SmtpClient class.
How to achieve this?
Thanks
Update
My last part of log4net config now looks like.
<appender name="MemoryAppender" type="log4net.Appender.MemoryAppender" >
<onlyFixPartialEventData value="true" />
<threshold value="WARN" />
</appender>
<root>
<level value="DEBUG" />
<appender-ref ref="Console" />
<appender-ref ref="RollingFile" />
<appender-ref ref="MemoryAppender" />
</root>
In code I do:
private static MemoryAppender MemoryAppender
{
get
{
if (memoryAppender == null)
{
Hierarchy h = LogManager.GetRepository() as Hierarchy;
memoryAppender = h.Root.GetAppender("MemoryAppender") as MemoryAppender;
}
return memoryAppender;
}
}
Then when I want the events I call:
MemoryAppender.GetEvents();
I've tried MemoryAppender.GetEvents()[0].RenderedMessage but that is not the correct output, how do I get the message string as it was written to the File/Console logs with the correct pattern and time etc and build myself a StringBuilder? I'll then put this in the body of my Email and send it using the SmtpClient. RenderMessage is just giving me the string that was provided to the Log.Warn() call not what was written to the log. Is this due to not setting a layout pattern on the MemoryAppender?
Thanks
MemoryAppender will only "append" to memory and is thus mostly useful only for development and testing purposes. And there is currently no appender that will only append on application shutdown.
The SMTPAppender is something in between, since it inherits the BufferingAppenderSkeleton. These appenders have a BufferSize property which controls how many messages are kept in memory before they are flushed.
Which messages to pass to the appenders are controlled with the level settings either on the root element or on individual logger elements. In your case use a level of WARN which will let through WARN, ERROR and FATAL. If you don't want the ERROR messages you will have to put a level filter on your appender.
Update: MemoryAppender is not using any layout to "render" message objects. What you get from MemoryAppender is just the raw message objects as they are produced by log4net. You will have to convert those to meaningfull text yourself.
Alternatively, if you require both layout functionality and in-memory appending you could look into subclassing AppenderSkeleton. That way you get the basic Layout support. When implementing the Append method you can do what MemoryAppender does, that is just appending to an internal list of messages.
Update 2: to implement the MemoryAppender alternative I suggest taking the MemoryAppender as a starting point. MemoryAppender is a subclass of AppenderSkeleton and have thus access to the RenderLoggingEvent method. So, we subclass MemoryAppender and add a method that renders the current batch of logging events:
public class RenderingMemoryAppender : MemoryAppender
{
public IEnumerable<string> GetRenderedEvents()
{
foreach(var loggingEvent in GetEvents())
{
yield return RenderLoggingEvent(loggingEvent);
}
}
}
You can use SMTPAppender and look at how the flush functionality works. Log4net keeps all messages in memory until flush is called (if it's setup this way), so the email will be sent when you flush it.
Another thing you can do is create a separate appender (Rolling or FileAppender) with filters WARN and FATAL, then attach this appender to the same logger, and at the end of your run email this file if it's non-empty (and you can choose to send it as an attachment or right in the body of the email). Let me know if you want more details, this is almost the same I'm doing know.
Good luck!
Ricardo.