Develop Reference

Docker missing log4net ConsoleAppender logs

I've got a simple C# program with the following log4net configuration:
<log4net>
<appender name="stdout" type="log4net.Appender.ConsoleAppender">
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%%-5level - %message%newline" />
</layout>
</appender>
<root>
<level value="DEBUG" />
<appender-ref ref="stdout" />
</root>
</log4net>
And the following code (abbreviated for clarity):
public static void Main(String[] args)
{
var log = LogManager.GetLogger("Logger");
log.Info("Hello From Logger");
Console.WriteLine("Hello From Console");
}
I'm compiling and running this using .NET 4.5 mono on an Ubuntu 16.04 system. When I run the program using the console the output is as expected:
$ mono Program.exe
INFO - Hello From Logger
Hello From Console
However, when I run this same program in a Docker container in detached mode (i.e, "docker run -d") and then check the logs, I only get the log from the Console.WriteLine:
$ docker logs <container_id>
Hello From Console
While trying to track this down, I tried running the program using the daemonize tool (more info here) and it illustrated the same behavior, leading me to believe it was an issue with log4net instead of docker.
I also wrote a custom appender for log4net that only does Console.Write (I believe that's the same functionality as ConsoleAppender, but I wanted to eliminate any other possible variables):
public class CustomAppender: AppenderSkeleton
{
protected override void Append(LoggingEvent loggingEvent)
{
Console.Write(RenderLoggingEvent(loggingEvent));
}
}
But no luck.
Can anyone with a deep knowledge of log4net point me to where I could be doing something wrong, or explain what log4net could be doing to stdout such that other programs that should be capturing that stream don't?
UPDATE: The plot thickens! I wrote a quick and dirty executable that uses the Process class to execute my example program. It then grabs the stdout from the example program and writes it with Console.WriteLine, like so:
Process proc = new Process();
proc.StartInfo.FileName = "/path/to/example.exe";
proc.StartInfo.UseShellExecute = false;
proc.StartInto.RedirectStandardOutput = true;
proc.Start();
Thread.Sleep(1000);
Console.WriteLine(proc.StandardOutput.ReadToEnd());
This works fine when I run it directly from the console, but has the same problem when I run it from docker / daemonize. So now I'm really confused.

It's not much of an answer, but in case someone else runs into this problem, I was able to get logging to work by using the NLog library here.

Modify app.config <system.diagnostics> section at runtime

I need to modify the <configuration><system.diagnostics> section of an app.config at runtime so that I can:
add a CustomTraceListener under the <sharedListeners> element, which requires special initializeData that can only be ascertained at runtime.
add the CustomTraceListener shared listener to an existing source under the <source><listeners> element.
persist the CustomTraceListener to other assemblies which load their trace source and listener configurations from the config file.
The relevant sections in app.config looks something like this presently:
<system.diagnostics>
<sources>
<source name="mysource" switchName="..." switchType="...">
<listeners>
<add name="console" />
<add name="customtracelistener" /> /// need to add new listener here
</listeners>
</source>
</sources>
<sharedListeners>
<add name="console" type="..." initializeData="..." />
<add name="customtracelistener" type="..." initializeData="..."> /// need to add custom trace listener here
<filter type="..." initializeData="Warning"/> /// with a filter
</add>
</sharedListeners>
<switches>
<add name="..." value="..." />
</switches>
</system.diagnostics>
Using ConfigurationManager I can easily do:
Configuration config = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None);
ConfigurationSection diagnostics = config.GetSection("system.diagnostics");
And when I do this, diagnostics is a System.Diagnostics.SystemDiagnosticsSection type. Interestingly I can't cast diagnostics to a SystemDiagnosticsSection type because I can't find it within any namespace. Regardless, ConfigurationSection doesn't seem to have any methods that I can use to write data into the section.
I also can't cast it to a NameValueConfigurationCollection because diagnostics base type is ConfigurationSection. I heard about this technique but it seems I can't use it.
Do I have to revert to using plain-old XML to accomplish this? I really don't like reinventing the wheel.

You can locate the path to the app.exe.config file through the ConfigurationManager, then load the config file as an XDocument.
string configPath = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None).FilePath;
XDocument config = XDocument.Load(configPath);
XElement diagnostics = config.Descendants().FirstOrDefault(e => e.Name == "system.diagnostics");
if (diagnostics == default(XElement))
{
/// <system.diagnostics> element was not found in config
}
else
{
/// make changes within <system.diagnostics> here...
}
config.Save(configPath);
Trace.Refresh(); /// reload the trace configuration
Once the required changes are made, save the XDocument back to disk, and call Trace.Refresh() to reload the trace configuration.
See MSDN regarding the Trace.Refresh method here.

For experience i would warn you to make app.config changes from application if the app is deployed with a good install procedure under protected directories, eg. Program files in MS OS with UAC activated.
To update config files sometimes you need some admin privileges.
The Bad thing is than all run correctly under visual studio / debug or some test procedure, after deploy, in production,you may have some issue...

If you make direct changes to the <configuration><system.diagnostics>
section of the app.config file at run time, the app needs to be restarted or Trace.Refresh() must be called to have the changes take effect.
Another option is to programmatically add TraceListeners at application start-up, e.g.
Trace.Listeners.Add(new TextWriterTraceListener("output.log", "myListener"));
see https://msdn.microsoft.com/en-us/library/sk36c28t(v=vs.110).aspx.
To add a filter with the initializeData value as in your question you can use the TraceListener.Filter property
To share settings across applications you can use the configSource property on the <system.diagnostics> element and put that element in a separate config file. The downside of this is that the file needs to be in the same folder as the app.config. So a change to one file would either need to be copied and pasted to other locations or shared in some other way.
An alternative would be to save a custom config file containing the trace listener information at a location all apps can access and then in each app load the file at start-up and configure the trace listeners as above.
Update
To share logging throughout your application you could create a class that implements the singleton pattern to return your TraceSource instance or to wrap your logging activities. This way you don't have to pass round the same instance
public class Logger
{
private static Logger _logger;
private TraceSource _ts;
private Logger()
{
// Configure TraceSource here as required, e.g.
_ts = new TraceSource("StackOverflow", SourceLevels.All);
_ts.Listeners.Add(new TextWriterTraceListener(#"c:\temp\tracefile.log"));
}
public static Logger Get()
{
if (_logger == null)
{
_logger = new Logger();
}
return _logger;
}
public void TraceInfo(string message)
{
_ts.TraceInformation(message);
_ts.Flush();
}
}
// To use
Logger.Get().TraceInfo("This is a trace message");
You could extend this to then encapsulate the actual messages you want to log so that the code doing the logging doesn't know the specifics and you have just one place where your events are defined, e.g.
public void TraceApplicationStarting()
{
_ts.TraceEvent(TraceEventType.Verbose, 1, "Application Starting");
}

Using 'Environment' values in log4net configuration

My application can be started both as a Windows Service and in console environment. For each case, I need some log4net appenders to be active (or not, respectively) as well as some which are active in both cases. (Service: RollingFileAppender, DebugAppender (if enabled), EventLogAppender | Console: ColoredConsoleAppender, DebugAppender (if enabled))
The only way to achieve something similar is using the PropertyFilter together with ThreadContext.Properties like so:
<filter type="log4net.Filter.PropertyFilter">
<key value="ApplicationMode" />
<stringToMatch value="Service" />
</filter>
if(!Environment.UserInteractive)
ThreadContext.Properties["ApplicationMode"] = "Service";
However, since the property is declared on the thread context, it only works on the current thread. If the thread changes, the configuration is being reset and I have to declare it again.
Does log4net support a way to declare a PropertyFilter in configuration to setup the desired environment automatically? Like this:
<filter type="log4net.Filter.PropertyFilter">
<key value="{Environment.UserInteractive}" />
<stringToMatch value="false" />
</filter>
Or... is there a better approach? Since I didn't found a solution yet.. is this an uncommon practice?

See the last part of my answer to this question:
Capture username with log4net
To summarize, you can implement an object that contains the logic that you have above and put that object in GlobalContext.Properties. When log4net retrieves the value (your object) from the Properties, it will call ToString to get the actual value. Put your logic inside ToString.
Maybe something like this:
public class ApplicationModeProvider
{
public override string ToString()
{
return Environment.UserInteractive ? "Console" : "Service";
}
}
Put it in your dictionary at startup:
GlobalContext.Properties["ApplicationMode"] = new ApplicationModeProvider();
In effect, this is sort of like adding
if(!Environment.UserInteractive)
ThreadContext.Properties["ApplicationMode"] = "Service";
before ever logging statement.
I'm not sure, but I think that then you can configure your filter as you describe in your post.

dynamic log4net appender name?

Let's say i have 3 smtp appenders in same log4net file whose names are:
<appender name = "emailDevelopment".. />
<appender name = "emailBeta".. />
<appender name = "emailProduction".. />
Let's say i have 3 different servers(Dev, Beta, Production). Depending upon the server, i want to fire the log. In case of Development server, it would fire log from "emailDevelopment". I have a system variable in each server named "ApplicationEnvironment" whose value is Development, Beta, Production based on the server names. Now is there anyway i can setup root in log4net so that it fires email depending upon the server name.
<root>
<priority value="ALL" />
<appender-ref ref="email<environment name from whose appender should be used>" />
</root>

This doesn't directly answer your question, but another approach is to simply have multiple log4net configuration files and call XmlConfigurator.Configure() on the right one. For example, you might have Logging.Development.Config, Logging.Beta.Config and so on.
Somewhere in code, you determine the "environment" and configure using the file you want.
I've even gone so far as to have multiple config files and pull different parts of them out into a single XML representing the "true" config, and then calling the Configure() method on that. For example, Logging.Appenders.Config which has all the appenders, and takes all of them and combines it with one of your environment-specific config files above; the environment-specific ones simply reference what they need, and the rest are effectively inactive/unreferenced for that environment.

Even after having written the only XSD file for log4net configuration I'm still not aware of an easy way to achieve this.
You might be able to do something like:
log4net.GlobalContext.Properties["host"] = new ClassThatToStringsHost();
class ClassThatToStringsHost
{ public override string ToString() { return "whatever"; } }
Now you can reference this value from the Log format with: "%property{host}"
To perform the filtering you will need to use a filter configuration in the adapter(s):
<appender name="file" type="log4net.Appender.RollingFileAppender">
<filter type="log4net.Filter.PropertyFilter">
<Key value="host" />
<StringToMatch value="whatever" />
</filter>
<!-- Anything not accepted by the above should be excluded -->
<filter type="log4net.Filter.DenyAllFilter" />
</appender>
There may even be a built-in property you could leverage and this should work. See also this post: http://geekswithblogs.net/rgupta/archive/2009/03/03/dynamic-log-filenames-with-log4net.aspx
For me, myself, and I... I would approach it another way all together. I would derive my own SMTP appender from the default and in the ActivateOptions() method I'd configure the values according to the environment. This would allow you to use one SMTP appender with consistent rules and yet provide three public properties for each of the email addresses you want to send from. It's not hard, give it a try!

How can I change the file location programmatically?

I am totally new to Log4net. I have managed to get something going by adding a config file and simple logging. I have hardcoded the value to be "C:\temp\log.txt" but this is not good enough.
The logs must go to the special folders
path = Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData);
and this path changes depending whether you are using Windows Server 2008 or Windows XP or Vista etc...
How can I just change the location of the file in log4net programmatically?
This is what I have done:
<configSections>
<section name="log4net"
type="log4net.Config.Log4NetConfigurationSectionHandler,Log4net"/>
</configSections>
<log4net>
<root>
<level value="DEBUG" />
<appender-ref ref="LogFileAppender" />
</root>
<appender name="LogFileAppender" type="log4net.Appender.RollingFileAppender">
<param name="File" value="C:\temp\log.txt" />
<param name="AppendToFile" value="true" />
<rollingStyle value="Size" />
<maxSizeRollBackups value="10" />
<maximumFileSize value="10MB" />
<staticLogFileName value="true" />
<layout type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%-5p%d{yyyy-MM-dd hh:mm:ss} – %m%n" />
</layout>
</appender>
</log4net>
class Program
{
protected static readonly ILog log = LogManager.GetLogger(typeof(Program));
static void Main(string[] args)
{
log4net.Config.XmlConfigurator.Configure();
log.Warn("Log something");
path = Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData);
// How can I change where I log stuff?
}
}
Just need to figure out how I can change to log stuff to where I want to.
Any suggestions?
Thanks a lot

log4net can handle this for you. Any appender property of type string can be formatted, in this case, using the log4net.Util.PatternString option handler. PatternString even supports the SpecialFolder enum which enables the following elegant config:
<appender name="LogFileAppender" type="log4net.Appender.RollingFileAppender" >
<file type="log4net.Util.PatternString"
value="%envFolderPath{CommonApplicationData}\\test.txt" />
...
</appender>
Here's a unit test that proofs the pudding:
[Test]
public void Load()
{
XmlConfigurator.Configure();
var fileAppender = LogManager.GetRepository()
.GetAppenders().First(appender => appender is RollingFileAppender);
var expectedFile =
Path.Combine(
Environment.GetFolderPath(
Environment.SpecialFolder.CommonApplicationData),
"test.txt");
Assert.That(fileAppender,
Is.Not.Null & Has.Property("File").EqualTo(expectedFile));
}
The following test verifies that log4net actually writes to disk (which basically makes this an "integration" test, not a unit test, but we'll leave it at that for now):
[Test]
public void Log4net_WritesToDisk()
{
var expectedFile =
Path.Combine(
Environment.GetFolderPath(
Environment.SpecialFolder.CommonApplicationData),
"test.txt");
if (File.Exists(expectedFile))
File.Delete(expectedFile);
XmlConfigurator.Configure();
var log = LogManager.GetLogger(typeof (ConfigTest));
log.Info("Message from test");
LogManager.Shutdown();
Assert.That(File.ReadAllText(expectedFile),
Text.Contains("Message from test"));
}
NB: I strongly suggest using the compact property syntax demonstrated in the above sample. Removing all those "<property name=" makes your config that much more readable.

I found a mutation of this code in the interwebs:
XmlConfigurator.Configure();
log4net.Repository.Hierarchy.Hierarchy h =
(log4net.Repository.Hierarchy.Hierarchy) LogManager.GetRepository();
foreach (IAppender a in h.Root.Appenders)
{
if (a is FileAppender)
{
FileAppender fa = (FileAppender)a;
// Programmatically set this to the desired location here
string logFileLocation = #"C:\MySpecialFolder\MyFile.log";
// Uncomment the lines below if you want to retain the base file name
// and change the folder name...
//FileInfo fileInfo = new FileInfo(fa.File);
//logFileLocation = string.Format(#"C:\MySpecialFolder\{0}", fileInfo.Name);
fa.File = logFileLocation;
fa.ActivateOptions();
break;
}
}
This works for me. Our application needs to put the log file in a folder that contains the version number of the app based on the AssemblyInfo.cs file.
You should be able to set the logFileLocation programmatically (e.g. you can use Server.MapPath() if this is a web application) to suit your needs.

Looks like Peter's answer doesn't work for Log4net v1.2.10.0.
An alternative method is described here.
Basically the method is to implement a custom pattern converter for the log4net config file.
First add this class to your project:
public class SpecialFolderPatternConverter : log4net.Util.PatternConverter
{
override protected void Convert(System.IO.TextWriter writer, object state)
{
Environment.SpecialFolder specialFolder = (Environment.SpecialFolder)Enum.Parse(typeof(Environment.SpecialFolder), base.Option, true);
writer.Write(Environment.GetFolderPath(specialFolder));
}
}
Then set up the File parameter of your FileAppender as follows:
<file type="log4net.Util.PatternString">
<converter>
<name value="folder" />
<type value="MyAppName.SpecialFolderPatternConverter,MyAppName" />
</converter>
<conversionPattern value="%folder{CommonApplicationData}\\SomeOtherFolder\\log.txt" />
</file>
Basically the %folder tells it to look at the converter called folder which points it to the SpecialFolderPatternConverter class. It then calls Convert on that class, passing in the CommonApplicationData (or whatever) enum value.

How about a simple:
XmlConfigurator.LogFullFilename = #"c:\ProgramData\MyApp\Myapp.log";
Why is it so complex to do a really simple thing?

To also change the error log's path (based on JackAce's answer):
private static void SetLogPath(string path, string errorPath)
{
XmlConfigurator.Configure();
log4net.Repository.Hierarchy.Hierarchy h =
(log4net.Repository.Hierarchy.Hierarchy)LogManager.GetRepository();
foreach (var a in h.Root.Appenders)
{
if (a is log4net.Appender.FileAppender)
{
if (a.Name.Equals("LogFileAppender"))
{
log4net.Appender.FileAppender fa = (log4net.Appender.FileAppender)a;
string logFileLocation = path;
fa.File = logFileLocation;
fa.ActivateOptions();
}
else if (a.Name.Equals("ErrorFileAppender"))
{
log4net.Appender.FileAppender fa = (log4net.Appender.FileAppender)a;
string logFileLocation = errorPath;
fa.File = logFileLocation;
fa.ActivateOptions();
}
}
}
}

This worked for me:
<log4net>
<appender name="RollingFileAppender" type="log4net.Appender.RollingFileAppender">
..
<file value="${APPDATA}\MyApp\MyApp Client\logs\Log.txt"/>
..
</log4net>
If needing to write to special folders I found help here (2nd and 3rd example).
Edit:
To answer OP.. This works for 'all users' area:
...
<file value="${ALLUSERSPROFILE}\MyApp\MyApp Client\logs\Log.txt"/>
...
Which is normally "C:\ProgramData" in newer versions of Windows.
See these too:
How to specify common application data folder for log4net? == https://stackoverflow.com/a/1889591/503621 and comments
&
https://superuser.com/q/405097/47628
https://stackoverflow.com/a/5550502/503621

JackAce's answer, just more concise using Linq:
C#
XmlConfigurator.Configure();
var appender = (LogManager.GetRepository() as Hierarchy).Root.Appenders
.OfType<FileAppender>()
.First();
appender.File = logPath;
appender.ActivateOptions();
VB.NET
XmlConfigurator.Configure()
Dim appender = CType(LogManager.GetRepository(), Hierarchy).Root.Appenders _
.OfType(FileAppender)() _
.First()
appender.File = logPath
appender.ActivateOptions()

Great use case for LINQs OfType<T> filter:
/// <summary>
/// Applies a transformation to the filenames of all FileAppenders.
/// </summary>
public static void ChangeLogFile(Func<string,string> transformPath)
{
// iterate over all FileAppenders
foreach (var fileAppender in LogManager.GetRepository().GetAppenders().OfType<FileAppender>())
{
// apply transformation to the filename
fileAppender.File = transformPath(fileAppender.File);
// notify the logging subsystem of the configuration change
fileAppender.ActivateOptions();
}
}
If the filename in the app.config is log.txt this will change the log output to logs/some_name_log.txt:
ChangeLogFile(path => Path.Combine("logs", $"some_name_{Path.GetFileName(path)}"));
To Answer the OPs original problem that would be:
ChangeLogFile(path => Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData), path));

As an alternative to doing this programatically, you can use environment variables and customizable patterns in the configuration file. See this response to a similar question.
Look at "PatternString for pattern based configuration" in the Log4Net V1.2.10 release notes.
Also if you are thinking of writing to a directory such as Enviroment.SpecialFolder.CommonApplicationData you need to consider:
Will all instances of your application for all users have write access to the log file? E.g. I don't believe non-administrators will be able to write to Enviroment.SpecialFolder.CommonApplicationData.
Contention if multiple instances of your application (for the same or different users) are attempting to the same file. You can use the "minimal locking model" (see http://logging.apache.org/log4net/release/config-examples.html to allow multiple processes to write to the same log file, but there probably will be a performance impact. Or you could give each process a different log file, e.g. by including the process id in the filename using a customizable pattern.

In the current version of Log4Net (2.0.8.0) you could simply use <file value="${ProgramData}\myFolder\LogFiles\" /> for C:\ProgramData\.. and ${LocalAppData} for C:\Users\user\AppData\Local\

If you have to deploy to unknown systems and you want to use the simple solution by Philipp M even with different special folders you can retrieve the special folder path you want and set a custom env variable before loading the log4net configuration.
string localData = Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData);
Environment.SetEnvironmentVariable("MY_FOLDER_DATA", localData);
XmlConfigurator.Configure( ...
... just to be sure the env variable exists and has the value you want.

Those who are looking to set the file path to Project root dicrectory Path can refer below code. I have implemented this in .NET 5 API Project
Changes for Log4net.config file. Remember type="log4net.Util.PatternString" this line is very important
<appender name="RollingFile" type="log4net.Appender.RollingFileAppender">
<file type="log4net.Util.PatternString" value="%property{LoggerFilePath}\Logs\server.log" />
<appendToFile value="true" />
<maximumFileSize value="100KB" />
<maxSizeRollBackups value="2" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date %5level %logger.%method [%line] - MESSAGE: %message%newline %exception" />
</layout>
Then, in Program.cs file
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureLogging((context,loggerBuilder) =>
{
log4net.GlobalContext.Properties["LoggerFilePath"] = context.HostingEnvironment.ContentRootPath;
loggerBuilder.ClearProviders();
loggerBuilder.AddLog4Net();
})
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});
Thanks!

//Make sure you include System.Linq
//Though is an answer to old question,
//but more concise, just two lines of code.
((FileAppender)LogManager.GetRepository().GetAppenders().Select(a => a).First()).File = #"C:\log.txt";
((FileAppender)LogManager.GetRepository().GetAppenders().Select(a => a).First()).ActivateOptions();