I have an application that listens for a hash input, processes that input, and serves it back. I would like to create a separate log directory for each input with the following structure:
- hashcode1
- input1.txt
- output1.txt
- log1.txt
- hashcode2
- input2.txt
- output2.txt
- log2.txt
I've looked into the NLog library but it seems the config requires a hardcoded log location, whereas my implementation requires the hashcode as an input. How can I achieve this with NLog, or any other logging library?
You can create the NLog configuration from code. This allows you to dynamically create targets in your processing loop for the appropriate log outputs:
using NLog;
using NLog.Targets;
// Init the config if we don't have a nlog.config
LogManager.Configuration = new();
// Do the work
var work = new List<string> { "5dd2ec2f", "24e1e841", "05137e70" };
work.ForEach(ProcessHash);
void ProcessHash(string hash)
{
// Create dynamic configuration and logger
var fileTarget = new FileTarget(hash) { FileName = $"{hash}/log.txt" };
LogManager.Configuration.AddRuleForAllLevels(fileTarget, hash);
LogManager.ReconfigExistingLoggers();
var logger = LogManager.GetLogger(hash);
// Process...
logger.Info($"Processing hash {hash}");
/* ... your code here ... */
// Remove the logging target
LogManager.Configuration.RemoveTarget(fileTarget.Name);
}
You can create a scope-property like this:
void ProcessHashCode(string hashInput)
{
using (NLog.MappedDiagnosticsLogicalContext.SetScoped("HashCode", hashInput))
{
NLog.LogManager.GetLogger("HashInput").Info(hashInput);
NLog.LogManager.GetCurrentClassLogger().Info("Received Hashcode");
var hashOutput = hashInput.GetHashCode();
NLog.LogManager.GetLogger("HashOuput").Info(hashOutput);
}
}
Then you can do this in NLog.config
<nlog>
<variable name="logdir" value="${mdlc:HashCode:whenEmpty=${basedir}}" />
<targets>
<target name="hashinput" type="file" filename="${logdir}/input.txt" />
<target name="hashouput" type="file" filename="${logdir}/output.txt" />
<target name="hashlog" type="file" filename="${logdir}/log.txt" />
</target>
<rules>
<logger name="HashInput" writeTo="hashinput" final="true" />
<logger name="HashOuput" writeTo="hashouput" final="true" />
<logger name="*" writeTo="hashlog" />
</rules>
</nlog>
See also: https://github.com/NLog/NLog/wiki/MDLC-Layout-Renderer
Alternative you can use Logger.WithProperty("HashCode", hashInput) like this:
void ProcessHashCode(string hashInput)
{
var inputLogger = NLog.LogManager.GetLogger("HashInput").WithProperty("HashCode", hashInput);
var outputLogger =
NLog.LogManager.GetLogger("HashOutput").WithProperty("HashCode", hashInput);
var hashLogger =
NLog.LogManager.GetCurrentClassLogger.WithProperty("HashCode", hashInput);
inputLogger.Info(hashCode);
hashLogger.Info("Received Hashcode");
var hashOutput = hashInput.GetHashCode();
outputLogger.Info(hashOutput);
}
And instead use variable like this:
<variable name="logdir" value="${event-properties:HashCode:whenEmpty=${basedir}}" />
See also: https://github.com/NLog/NLog/wiki/Context#logevent-properties
Related
I have the following code to Dependency Inject the NLog logger:
// startup.cs
[assembly: FunctionsStartup(typeof(MyApp.FunctionApp.Startup))]
namespace MyApp.FunctionApp {
public class Startup : FunctionsStartup {
public override void Configure(IFunctionsHostBuilder builder) {
var nLogConfigPath = GetLogPath("nlog.config");
builder.Services.AddLogging(loggingBuilder =>
{
var nLogOptions = new NLogAspNetCoreOptions
{
RegisterHttpContextAccessor = true,
IgnoreEmptyEventId = true,
IncludeScopes = true,
ShutdownOnDispose = true
};
var logFactory = NLogBuilder.ConfigureNLog(nLogConfigPath);
logFactory.AutoShutdown = false;
var nLogConfig = logFactory.Configuration;
loggingBuilder.AddNLog(nLogConfig, nLogOptions);
});
}
}
}
// actual function code
public class ActualFunctionClass {
public ActualFunctionClass (ILogger<ActualFunctionClass> logger) {
logger.LogInformation("log stuff");
}
}
In the nlog.config, I have several targets. How do I make sure that the logger parameter to the ActualFunctionClass is configured with the proper target?
These are the contents of the nlog.config. For the record, I would like to log using the locations-dataload-file target.
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
autoReload="true"
throwExceptions="true"
keepVariablesOnReload="true">
<variable name="logDirectory" value="${basedir}/logs/locations-dataload" />
<variable name="commonLayout" value="${longdate}|${logger}|${uppercase:${level}}|${message}, ${all-event-properties:format=[key]=[value]:separator=, } ${exception}" />
<targets>
<target xsi:type="ApplicationInsightsTarget"
name="locations-dataload-ai"
layout="${commonLayout}" />
<target xsi:type="File"
name="locations-dataload-file"
fileName="${logDirectory}/locations-dataload-file-${shortdate}.log"
layout="${commonLayout}" />
<target xsi:type="Null" name="blackhole" />
</targets>
<rules>
<!--All logs, including from Microsoft-->
<logger name="*" minlevel="Trace" writeTo="locations-dataload-ai" />
<!--All logs, including from Microsoft-->
<logger name="locations-dataload-local*" minlevel="Trace" writeTo="locations-dataload-ai,locations-dataload-file" />
<!-- Skip Microsoft logs and so log only own logs -->
<logger name="Microsoft.*" minlevel="Trace" writeTo="blackhole" final="true" />
</rules>
<extensions>
<add assembly="NLog.Web.AspNetCore"/>
<add assembly="MyApp.Logging" />
</extensions>
</nlog>
I think this is wrong:
<logger name="locations-dataload-local*" ...
The logger name is by default Namespace.ClassName
You could test this by:
change the rule it to name="*"
<logger name="*" minlevel="Trace" writeTo="locations-dataload-ai,locations-dataload-file" />
and add to your layout ${logger}, e.g.
layout="${logger}|${commonLayout}"
The name filter will be matched against the logger name ${logger). So if the logger name is YourNamespace.ActualFunctionClass then locations-dataload-local* won't match that.
I followed Diagnostic & Logging and install NLog.Config nuget package to Android & iOS project.
on Android project, Setup.cs
public override MvxLogProviderType GetDefaultLogProviderType() => MvxLogProviderType.NLog;
On Xamarin.Forms project,
private static readonly IMvxLog _logger = Mvx.IoCProvider.Resolve<IMvxLogProvider>().GetLogFor<CanvasContainer>();
_logger.Debug($"startScale: {startScale}, currentScale: {currentScale}");
I checked _logger is not null.
Also, I changed NLog.config's Build Action to Embedded Resource.
Here is Android Project's NLog.config,
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.nlog-project.org/schemas/NLog.xsd NLog.xsd"
autoReload="true"
throwExceptions="false"
internalLogFile="c:\temp\nlog-internal.log">
<targets>
<target xsi:type="File"
name="FileTarget"
fileName="${specialfolder:folder=MyDocuments}/logs/${shortdate}.log"
layout="${longdate}|${level:uppercase=true}|${logger}|${message}" />
<target xsi:type="ColoredConsole"
name="ConsoleTarget"
encoding="utf-8"
layout="${longdate}|${level:uppercase=true}|${logger}|${message}"
useDefaultRowHighlightingRules="true" />
</targets>
<rules>
<logger name="*" minlevel="Debug" writeTo="FileTarget" />
<logger name="*" minlevel="Debug" writeTo="ConsoleTarget" />
</rules>
</nlog>
Is there anything wrong with this set up? I can not find sample or tutorial for this...
Think you need to help NLog when having NLog.config as Embedded Resource:
public override MvxLogProviderType GetDefaultLogProviderType() => MvxLogProviderType.NLog;
protected override IMvxLogProvider CreateLogProvider()
{
var nlogConfigFile = GetEmbeddedResourceStream(myAssembly, "NLog.config");
if (nlogConfigFile != null)
{
var xmlReader = System.Xml.XmlReader.Create(nlogConfigFile);
NLog.LogManager.Configuration = new XmlLoggingConfiguration(xmlReader, null);
}
return base.CreateLogProvider();
}
private static Stream GetEmbeddedResourceStream(Assembly assembly, string resourceFileName)
{
var resourcePaths = assembly.GetManifestResourceNames()
.Where(x => x.EndsWith(resourceFileName, StringComparison.OrdinalIgnoreCase))
.ToList();
if (resourcePaths.Count == 1)
{
return assembly.GetManifestResourceStream(resourcePaths.Single());
}
return null;
}
See also: https://github.com/NLog/NLog/wiki/Explicit-NLog-configuration-loading#loading-nlog-configuration-from-xamarin-resource
I have a nlog.config file with the following entry:
<include file="${basedir}/ActiveConfig/NLog/*.config"/>
But I want to solve this via code behind and have not found a method to do so.
LogManager.LoadConfiguration()
is overwriting my existing configuration.
Did I miss something?
You could set it like this:
var config = XmlLoggingConfiguration
.CreateFromXmlString("<include file='${basedir}/ActiveConfig/NLog/*.config' />");
LogManager.Configuration = config; // apply
Think the easy solution is just to load all config-files into an in-memorystream, and then load that into a single XmlLoggingConfiguration:
var xmlReader = System.Xml.XmlReader.Create(memorystream);
NLog.LogManager.Configuration = new XmlLoggingConfiguration(xmlReader, null);
Something like this, where you put the contents of all config-files within the same <nlog>-root:
<nlog>
<!-- XML File 1 -->
<targets>
</targets>
<rules>
</rules>
<! -- XML File 2 -->
<targets>
</targets>
<rules>
</rules>
</nlog>
If you want the AutoReload-feature working also, then this might work:
class MySpecialLoggingConfiguration : XmlLoggingConfiguration
{
private string[] _fileNames;
public MySpecialLoggingConfiguration(string[] fileNames)
{
_fileNames = fileNames;
// Your special concat-logic in memory
}
public override LoggingConfiguration Reload()
{
return new MySpecialLoggingConfiguration(_fileNames);
}
public override IEnumerable<string> FileNamesToWatch => _fileNames;
}
Our tack is c# .net 4.7 logging through NLog library to a "Graylog" GELF input which of course is storing them in an elasticsearch cluster.
The application utilizes logging heavily, but EVERYTHING is sent through as strings - no matter if it was something other than a string in code.
I want to be able send my custom logger properties as their true data types so that I can aggregate on my statistical number fields.
I DO NOT want to constrain my developers and do this with scripted fields, or pre-mapping an index in elasticsearch - in fact, graylog manages my index for me. Nor do I want to utilize graylog pipeline processor - my poor graylog does enough work. I want them to be able to send stat data as their real data types and have them mapped as "long" or "number".
A code snippet from my stuff:
public void InfoExtended2(String Message, Dictionary<string, int> extrafields, [CallerMemberName] string callerMethodName = "", [CallerFilePath] string callerFilePath = "", [CallerLineNumber] int sourceLineNumber = 0)
{
var fileName = callerFilePath.Substring(callerFilePath.LastIndexOf(#"\") + 1).Replace(".cs", "");
var caller = $"{fileName}.{callerMethodName}()Line:{sourceLineNumber}";
var logger = LogManager.GetLogger(caller);
if (!(extrafields.Count == 0))
{
foreach (var field in extrafields)
{
string mykey = field.Key.ToString();
extrafields.TryGetValue(field.Key, out int myvalue);
//be careful whats in object!
if (!logger.Properties.ContainsKey(mykey)) { logger.Properties.Add(mykey, "{#myvalue}"); } else { logger.Properties[mykey] = "{#myvalue}"; }
}
}
logger.Info()
.Message(Message)
.Property("ComeOnNow3", 87)
.Write();
// Create the LogEventInfo object
//LogEventInfo logEvent = new LogEventInfo();
// Now add the event characteristics
//logEvent.Properties["EventCode"] = 2222;
//logEvent.Level = NLog.LogLevel.Info;
//logEvent.Properties["EventCode]."]
//logEvent.Message = "My Message";
////logEvent.Exception = ex;
//logger.Log(logEvent);
//logger.Info("{#Message}", new Dictionary<string, int> { { "key1", 1 }, { "key2", 2 } }); // dict. Result: Test "key1"=1, "key2"=2
}
I've tried a few different ways there with no luck.
If I make a POST GELF call myself with JSON fields representing ints instead of strings, it works. My field is brand new to the index and when I go check the mapping it comes back as a long. I want my code to do that.
Test GELF Call:
{
"version": "1.1",
"host": "example.org",
"short_message": "A short message that helps you identify what is going on",
"full_message": "Backtrace here\n\nmore stuff",
"timestamp": 1385053862.3072,
"level": 1,
"_user_id": 9001,
"_some_info": "foo",
"_some_env_var": "bar"
}
Resulting mapping for user id:
"user_id" : {
"type" : "long"
What our nlog target currently looks like:
<targets>
<target name="gelftcp"
type="gelftcp"
facility="Custom Facility"
remoteaddress="192.168.95.15"
remoteport="12201"
layout="${longdate}${newline}type: ${level:uppercase=true}${newline}class: ${logger}${newline}stacktrace: ${stacktrace}${newline}error message: ${message}${newline}exception: ${exception:format=tostring,data:maxinnerexceptionlevel=10}">
<parameter name="logged_at" layout="${longdate}" />
<parameter name="type" layout="${level:uppercase=true}" />
<parameter name="class" layout="${logger}" />
<parameter name="CustomInt" layout="${event-properties:item=CustomInt}" as="number" />
<parameter name="stacktrace" layout="${stacktrace}" />
<parameter name="error_message" layout="${message}" />
<parameter name="ComeOnNow3" layout="${event-properties:item=ComeOnNow3} " />
<parameter name="exception" layout="${exception:format=tostring,data:maxinnerexceptionlevel=10}" />
<attribute name="eventProperties" encode="false" >
<layout type='JsonLayout' includeAllProperties="true" maxRecursionLimit="2"/>
</attribute>
<variable name="ComeOnNow2" value ="${event-context:ComeOnNow2}" />
</target>
Take it easy on me, I'm not the original author of this code. Just someone tasked with picking up slack & deploying in its current state.
UPDATE:
Tried the first suggestion. Even added the field I'm testing:
<field name="ComeOnNow3" layout="${threadid}" type="System.Int32" />
Attempted this log entry:
logger.Info()
.Message(Message)
.Property("ComeOnNow3", 87)
.Write();
My mapping still comes back as "keyword".
SOLVED:
The trick was to switch to Http instead of Tcp.
Instead of using "gelftcp". Then you can try out GelfLayout instead:
https://www.nuget.org/packages/NLog.GelfLayout
You can combine it with the TCP network-target:
<nlog>
<extensions>
<add assembly="NLog.Layouts.GelfLayout" />
</extensions>
<targets async="true">
<target type="Network" name="gelftcp" address="tcp://192.168.95.15:12201" newLine="true" lineEnding="Null">
<layout type="GelfLayout" facility="MyFacility">
<field name="threadid" layout="${threadid}" type="System.Int32" />
</layout>
</target>
</targets>
<rules>
<logger name="*" minlevel="Debug" writeTo="gelftcp" />
</rules>
</nlog>
It also has some automatic conversion logic for simple NLog LogEventInfo.Properties (Decimal + Double + Integer + Boolean)
More examples can be found here: https://github.com/farzadpanahi/NLog.GelfLayout
I use a NLog.config, but some values I want to change from outside. E.g. target:adress could change so I want to set it each time the software starts.
I imagine some thing like
var logger = new LoggerFactory().AddNLog().CreateLogger<Program>();
logger.target.adress = "myNewAdress";
How can I set values to my NLog.config?
You could edit the config in C# like this:
var configuration = LogManager.Configuration;
var fileTarget = configuration.FindTargetByName<FileTarget>("myTargetName");
fileTarget.FileName = "${basedir}/file.log";
LogManager.Configuration = configuration; //apply
Please note that combining the config file (nlog.config) and changing it in code, the reload of nlog.config could undo your changes. If you combine both, then reapply the changes on the reload event. E.g.
public void UpdateConfig()
{
var configuration = LogManager.Configuration;
var fileTarget = configuration.FindTargetByName<FileTarget>("myTargetName");
fileTarget.FileName = "${basedir}/file.log";
LogManager.Configuration = configuration; //apply
}
// On start of your program
UpdateConfig();
LogManager.ConfigurationReloaded += (sender, e) =>
{
//Re apply if config reloaded
UpdateConfig();
};
See also: https://github.com/NLog/NLog/wiki/Configure-from-code
I recommend that one makes use of the NLog context layoutrenderers instead of modifying target properties at runtime. It ofcourse requires that the target-property supports NLog Layout.
Example of NLog.config:
<nlog>
<targets>
<target type="file" name="file" fileName="${gdc:item=LogDir}\LogFile.txt}" />
</targets>
<rules>
<logger minLevel="Trace" writeTo="file" />
</rules>
</nlog>
Then at runtime you can change the GDC-variables:
NLog.GlobalDiagnosticsContext.Set("LogDir", "C:\\Temp");
${gdc} Can also be combined with WhenEmpty, so one can provide a fallback default value, when nothing has been asigned from code.
See also: https://github.com/NLog/NLog/wiki/Context