Develop Reference

How to log exception as structured JSON with NLog

This is the code:
try
{
throw new AssertionException("This is exceptional");
}
catch (AssertionException ex)
{
// Rider nags me to write Error logs with exceptions this way
logger.Error(ex, "Houston, we have a problem!");
}
And the config is this:
<targets async="false">
<target name="NoPiiLog" xsi:type="Memory">
<layout type="SuppressPiiJsonLayout" includeAllProperties="true" maxRecursionLimit="2"
excludeProperties="UserId,EmailId" excludeEmptyProperties="true">
<attribute name="time" layout="${longdate}" />
<attribute name="level" layout="${level:uppercase=true}" />
<attribute name="message" layout="${message:raw=true}" />
<attribute name="exception">
<layout type="JSonLayout" includeAllProperties="true" maxRecursionLimit="2"
excludeProperties="UserId,EmailId" excludeEmptyProperties="true" />
</attribute>
<attribute name="Properties">
<layout type='JsonLayout' includeAllProperties="true" maxRecursionLimit="2"
excludeEmptyProperties="true"/>
</attribute>
</layout>
</target>
SuppressPiiJsonLayout is a class I wrote and registered that takes the final JSON and replaces all the PII key values with "*****". What I can't do is to log the exceptionunless I include "{exception}" in the message string and the exception object in the params, like this:
logger.Error(ex, "Houston, we have a problem! {exception}", ex);
This is because the exception does not get into the Params or the Properties of the LogEvent, it is in an Exception member on the LogEvent. The docs don't cover structured logging of exceptions. https://github.com/NLog/NLog/wiki/How-to-Log-Exceptions
So that you don't think my suppressor is screwing things up, here is that code:
[Layout("SuppressPiiJsonLayout")]
[ThreadAgnostic]
[ThreadSafe]
public class SuppressPiiJsonLayout : NLog.Layouts.JsonLayout
{
/// <summary>
/// This digs thru the JObject and replaces the values with "*****". It recurses thru all the JObjects it
/// finds along the way, replacing the values in ExcludeProperties
/// </summary>
/// <param name="logEvent">The logging event.</param>
private void CleanKeys(JObject obj)
{
foreach (var entry in obj)
{
if(this.ExcludeProperties.Contains(entry.Key))
{
obj[entry.Key] = "*****";
}
else
{
if (obj[entry.Key] is JObject)
{
this.CleanKeys((JObject)obj[entry.Key]);
}
}
}
}
/// <summary>
/// This intercepts the rendering of the formatted message after it is turned into JSON and then cleans
/// all of the "ExcludeProperties" from the JSON about to be outputted, recursing thru all the objects
/// it finds. Unlike the ExcludeProperties of the JSONLayout which is practically useless. It might not be
/// the most efficient thing, but considering how immutable anonymous objects are, it is definitely the
/// only way I found to implement it.
/// </summary>
/// <param name="logEvent">The logging event.</param>
/// <param name="target"><see cref="StringBuilder"/> for the result</param>
protected override void RenderFormattedMessage(LogEventInfo logEvent, StringBuilder target)
{
var intercept = new StringBuilder();
base.RenderFormattedMessage(logEvent, intercept);
var j = JObject.Parse(intercept.ToString());
this.CleanKeys(j); // Remove all the ExcludeProperties it finds recursively
target.Append(j.ToString(Formatting.None)); // And then pass the modified version to the target for writing
}
}
I guess I could alter this code to make ex.Exception a named parameter, but it seems like NLOG should already be handling this so I'm kind of stumped.

Yes the LogEventInfo.Exception is not included in LogEventInfo.Properties, so it will not be included when using includeAllProperties="true":
Instead of doing this:
<attribute name="exception">
<layout type="JSonLayout" includeAllProperties="true" maxRecursionLimit="2"
excludeProperties="UserId,EmailId" excludeEmptyProperties="true" />
</attribute>
Then you can do this (encode="false" is to avoid encoding the json-output from format=#):
<attribute name="exception" layout="${exception:format=#}" encode="false" />
See also: https://github.com/NLog/NLog/wiki/How-to-use-structured-logging#output-captured-properties

Unable to get NLog to send fields as numbers/ints/longs from code

Our tack is c# .net 4.7 logging through NLog library to a "Graylog" GELF input which of course is storing them in an elasticsearch cluster.
The application utilizes logging heavily, but EVERYTHING is sent through as strings - no matter if it was something other than a string in code.
I want to be able send my custom logger properties as their true data types so that I can aggregate on my statistical number fields.
I DO NOT want to constrain my developers and do this with scripted fields, or pre-mapping an index in elasticsearch - in fact, graylog manages my index for me. Nor do I want to utilize graylog pipeline processor - my poor graylog does enough work. I want them to be able to send stat data as their real data types and have them mapped as "long" or "number".
A code snippet from my stuff:
public void InfoExtended2(String Message, Dictionary<string, int> extrafields, [CallerMemberName] string callerMethodName = "", [CallerFilePath] string callerFilePath = "", [CallerLineNumber] int sourceLineNumber = 0)
{
var fileName = callerFilePath.Substring(callerFilePath.LastIndexOf(#"\") + 1).Replace(".cs", "");
var caller = $"{fileName}.{callerMethodName}()Line:{sourceLineNumber}";
var logger = LogManager.GetLogger(caller);
if (!(extrafields.Count == 0))
{
foreach (var field in extrafields)
{
string mykey = field.Key.ToString();
extrafields.TryGetValue(field.Key, out int myvalue);
//be careful whats in object!
if (!logger.Properties.ContainsKey(mykey)) { logger.Properties.Add(mykey, "{#myvalue}"); } else { logger.Properties[mykey] = "{#myvalue}"; }
}
}
logger.Info()
.Message(Message)
.Property("ComeOnNow3", 87)
.Write();
// Create the LogEventInfo object
//LogEventInfo logEvent = new LogEventInfo();
// Now add the event characteristics
//logEvent.Properties["EventCode"] = 2222;
//logEvent.Level = NLog.LogLevel.Info;
//logEvent.Properties["EventCode]."]
//logEvent.Message = "My Message";
////logEvent.Exception = ex;
//logger.Log(logEvent);
//logger.Info("{#Message}", new Dictionary<string, int> { { "key1", 1 }, { "key2", 2 } }); // dict. Result: Test "key1"=1, "key2"=2
}
I've tried a few different ways there with no luck.
If I make a POST GELF call myself with JSON fields representing ints instead of strings, it works. My field is brand new to the index and when I go check the mapping it comes back as a long. I want my code to do that.
Test GELF Call:
{
"version": "1.1",
"host": "example.org",
"short_message": "A short message that helps you identify what is going on",
"full_message": "Backtrace here\n\nmore stuff",
"timestamp": 1385053862.3072,
"level": 1,
"_user_id": 9001,
"_some_info": "foo",
"_some_env_var": "bar"
}
Resulting mapping for user id:
"user_id" : {
"type" : "long"
What our nlog target currently looks like:
<targets>
<target name="gelftcp"
type="gelftcp"
facility="Custom Facility"
remoteaddress="192.168.95.15"
remoteport="12201"
layout="${longdate}${newline}type: ${level:uppercase=true}${newline}class: ${logger}${newline}stacktrace: ${stacktrace}${newline}error message: ${message}${newline}exception: ${exception:format=tostring,data:maxinnerexceptionlevel=10}">
<parameter name="logged_at" layout="${longdate}" />
<parameter name="type" layout="${level:uppercase=true}" />
   <parameter name="class" layout="${logger}" />
<parameter name="CustomInt" layout="${event-properties:item=CustomInt}" as="number" />
   <parameter name="stacktrace" layout="${stacktrace}" />
   <parameter name="error_message" layout="${message}" />
<parameter name="ComeOnNow3" layout="${event-properties:item=ComeOnNow3} " />
   <parameter name="exception" layout="${exception:format=tostring,data:maxinnerexceptionlevel=10}" />
<attribute name="eventProperties" encode="false" >
<layout type='JsonLayout' includeAllProperties="true" maxRecursionLimit="2"/>
</attribute>
<variable name="ComeOnNow2" value ="${event-context:ComeOnNow2}" />
</target>
Take it easy on me, I'm not the original author of this code. Just someone tasked with picking up slack & deploying in its current state.
UPDATE:
Tried the first suggestion. Even added the field I'm testing:
<field name="ComeOnNow3" layout="${threadid}" type="System.Int32" />
Attempted this log entry:
logger.Info()
.Message(Message)
.Property("ComeOnNow3", 87)
.Write();
My mapping still comes back as "keyword".
SOLVED:
The trick was to switch to Http instead of Tcp.

Instead of using "gelftcp". Then you can try out GelfLayout instead:
https://www.nuget.org/packages/NLog.GelfLayout
You can combine it with the TCP network-target:
<nlog>
<extensions>
<add assembly="NLog.Layouts.GelfLayout" />
</extensions>
<targets async="true">
<target type="Network" name="gelftcp" address="tcp://192.168.95.15:12201" newLine="true" lineEnding="Null">
<layout type="GelfLayout" facility="MyFacility">
<field name="threadid" layout="${threadid}" type="System.Int32" />
</layout>
</target>
</targets>
<rules>
<logger name="*" minlevel="Debug" writeTo="gelftcp" />
</rules>
</nlog>
It also has some automatic conversion logic for simple NLog LogEventInfo.Properties (Decimal + Double + Integer + Boolean)
More examples can be found here: https://github.com/farzadpanahi/NLog.GelfLayout

C#/XNA XML serialization

First time using serialization in XNA so forgive me if I'm over looking soemthing obvious.
Basically, in my game, I have two objects, Square (storing data about each square in the game) and one called Enemy (storing information and behaviours about the enemey).
I want to serialize this data so I can save and load it in at a later date.
I've put them both into a single object called Data and serialised that.
However the xml data reads like so:
<?xml version="1.0" encoding="utf-8"?>
<Data xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<squares>
<Square />
<Square />
<Square />
<Square />
<Square />
<Square />
<Square />
<Square />
<Square />
etc
My code:
Data.cs
public class Data
{
public Square[] squares;
public Enemy[] enemies;
public Data()
{
}
}
SaveFucntion:
Data data = new Data();
data.squares = squares;
data.enemies = enemies;
StreamWriter file2 = new StreamWriter(Content.RootDirectory + "\\xml.txt");
XmlSerializer x = new XmlSerializer(typeof(Data));
x.Serialize(file2, data);
What am I doing wrong? or what would be the best way to achieve this?

How can I programmatically update log4net configuration file?

I currently have the log4net config in the applications' app.config file, as such:
...
<section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net"/>
</configSections>
<log4net>
<appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">
<file value="logs\Service.log"/>
<appendToFile value="true"/>
<rollingStyle value="Date"/>
<datePattern value="yyyyMMdd"/>
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger - %message%newline"/>
</layout>
</appender>
<appender name="ConsoleAppender" type="log4net.Appender.ConsoleAppender">
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level %logger - %message%newline"/>
</layout>
</appender>
<logger name="Data.WebService">
<level value="ALL"/>
<appender-ref ref="ConsoleAppender"/>
<appender-ref ref="RollingLogFileAppender"/>
</logger>
<logger name="Data.Host.HostService">
<level value="ALL"/>
<appender-ref ref="ConsoleAppender"/>
<appender-ref ref="RollingLogFileAppender"/>
</logger>
</log4net>
I know I can read this in via log4net.Config.XmlConfigurator.Configure();, however, I would like to be able to update it via some sort of call as well. I'll be accepting configuration from a web service and, once I've set the new config (currently only log level, but I'm not precluding other things being configurable down the road), I need to update what is in the config file.
Having all of the configs in one file is very convenient, however, I'm open to locating the config in another file if that makes it simpler.

Since there is no official method to do so, I wound up writing a method that uses xpath to locate the element(s) to change and then update accordingly. Works well enough for what I need to do and is more elegant than a brute-force "readinthefiletoastringthenreplacethetextthensavethestringtothefile" approach.
public enum Log4NetConfigItem
{
LOGLEVEL
}
public const string LOG4NET_CONFIGFILE = "log4net.config";
public void UpdateConfiguration(Log4NetConfigItem which, object value)
{
// Load the config file.
XmlDocument doc = new XmlDocument();
doc.Load(LOG4NET_CONFIGFILE);
// Create an XPath navigator for the document.
XPathNavigator nav = doc.CreateNavigator();
try
{
XPathExpression expr;
// Compile the correct XPath expression for the element we want to configure.
switch (which)
{
default:
case Log4NetConfigItem.LOGLEVEL:
// Compile a standard XPath expression
expr = nav.Compile("/configuration/log4net/logger/level");
break;
}
// Locate the node(s) defined by the XPath expression.
XPathNodeIterator iterator = nav.Select(expr);
// Iterate on the node set
while (iterator.MoveNext())
{
XPathNavigator nav2 = iterator.Current.Clone();
// Update the element as required.
switch (which)
{
default:
case Log4NetConfigItem.LOGLEVEL:
// Update the 'value' attribute for the log level.
SetAttribute(nav2, String.Empty, "value", nav.NamespaceURI, value.ToString());
break;
}
}
// Save the modified config file.
doc.Save(LOG4NET_CONFIGFILE);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
void SetAttribute(System.Xml.XPath.XPathNavigator navigator, String prefix, String localName, String namespaceURI, String value)
{
if (navigator.CanEdit == true)
{
// Check if given localName exist
if (navigator.MoveToAttribute(localName, namespaceURI))
{
// Exist, so set current attribute with new value.
navigator.SetValue(value);
// Move navigator back to beginning of node
navigator.MoveToParent();
}
else
{
// Does not exist, create the new attribute
navigator.CreateAttribute(prefix, localName, namespaceURI, value);
}
}
}
Note: The SetAttribute code I got from here.

This is basically this open issue for log4net - unfortunately it is not resolved yet.

Set log name in appender of log4net

I have class MyLogger, where I use log4net. How can I modify my appender to save log in specific logname (I want to set it by parameter logName).
public void AddEntry(string source, string logName, string contextInfo, string message, EventLogEntryType eventType)
{
log4net.ILog Log = log4net.LogManager.GetLogger(source);
Log.Error(String.Format("Context Info: {0}{1}{2}{3}", contextInfo, Environment.NewLine, Environment.NewLine, message));
}
<log4net>
<root>
<priority value="ALL" />
<appender-ref ref="EventLogAppender" />
</root>
This is myAppender. Now it writes in common logtype Application.
<appender name="EventLogAppender" type="log4net.Appender.EventLogAppender" >
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%logger (%property{myContext}) [%level]- %message%newline" />
</layout>
</appender>
</log4net>

I think what you are looking for is this:
<param name="LogName" value="MyLog" />
Some more information can be found here. If you do something like this you may want to have a look at this issue as well. Basically this is about registering your application so that the eventlog knows about it.
Edit (configuration by code):
Did not test it but that ought to do the trick:
foreach (AppenderSkeleton appender in this.Logger.Repository.GetAppenders())
{
var eventlogAppender = appender as EventLogAppender;
if (eventlogAppender != null)
{
eventlogAppender.LogName = "MyLog";
eventlogAppender.ActivateOptions();
break;
}
}
You could add some tests to verify that there is only one EventLogAppender, but probably you do not need to bother to do so.

this works:
<param name="LogName" type="log4net.Util.PatternString" value="%property{LogName}" />