Custom Column(s) with MsSqlServer Sink and AppSettings - c#

I've got an application that runs without problem with the File and Console Sinks and now I'm trying to add the MSSqlServer Sink.
Looking at the documentation on Github I've got my application to write to the SQL Database as well as the other sinks.
<add key="serilog:write-to:MSSqlServer.connectionString" value="Server=servername;Database=databasename;User Id=userid;Password=password;"/>
<add key="serilog:write-to:MSSqlServer.tableName" value="Logs"/>
<add key="serilog:write-to:MSSqlServer.autoCreateSqlTable" value="true"/>
One improvement is I'd like to add a custom column to the Logs table that Serilog uses to store a number indicating a unique RunId for my application. The idea being that a simple query would allow grouping by RunId to see all messages for that one run.
So I added the following, based on the documentation (and I haven't been able to find any other examples) as it seemed logical:
<add key="serilog:write-to:MSSqlServer.columnOptions.ColumnName" value="RunId"/>
<add key="serilog:write-to:MSSqlServer.columnOptions.PropertyName" value="RunId"/>
<add key="serilog:write-to:MSSqlServer.columnOptions.DataType" value="SqlDbType.Int"/>
<add key="serilog:write-to:MSSqlServer.columnOptions.DataLength" value="32"/>
and then in my code all I need to do is:
Log.Information("{RunId}{Message}", RunId, Message);
to see a new entry with {RunId} in the RunId column and {Message} in the Message column... however everytime I do this nothing is written to the RunId column, it remains as NULL whereas every log message the console/file has is also duplicated in the Table.
So it seems logging is working, it must be the keys wrong and I'm really not sure what should be used.
Would anyone be able to point me in the direction I need to be going or where I've gone wrong?
Thank you.

Logging definitely was working but some digging and finally I found out I had two issues:
Permissions on database where insuffident for the user serilog was using, and
AppSettings settings I was using were wrong
#1 was easy enough to fix, I created a dedicated Serilog account for this database and fixed the permissions as per documentation.
#2 however was very frustrating but eventually I was able to get the following to work:
<configSections>
<section name="MSSqlServerSettingsSection" type="Serilog.Configuration.MSSqlServerConfigurationSection, Serilog.Sinks.MSSqlServer"/>
</configSections>
<MSSqlServerSettingsSection DisableTriggers="false" ClusteredColumnstoreIndex="false" PrimaryKeyColumnName="Id">
<!-- SinkOptions parameters -->
<TableName Value="Logs"/>
<Columns>
<add ColumnName="RunId" DataType="int"/>
</Columns>
</MSSqlServerSettingsSection>
On compile and execution my program will now create the Logs table in the database and then create a RunId column which I have been able to populate with a {RunId} expression in my Log.Debug() calls.
FWIW: I hope this will be helpful and if I can work out how I'll see if I can add this to documentation as an example of using AppSettings for this use case. Searching this question most people seem to be using JSON and not AppSettings.

Related

FileTransform task in Azure DevOps throws error on transform

I am using the FileTransform#2 task to transform a web.config with a web.[environment].config in a Azure DevOps Pipeline (yaml). It seems to fail on one of the individual transforms which fails the whole job, though I'm not sure why.
Here is the error message from the task:
Executing SetAttributes (transform line 72, 48)
on /configuration/appSettings/add[#key='PCWSUser']
System.NullReferenceException: Object reference not set to an instance of an object.
Applying to 'add' element (no source line info)
at Microsoft.Web.XmlTransform.XmlTransformationLogger.ConvertUriToFileName(XmlDocument xmlDocument)
Set 'key' attribute
at Microsoft.Web.XmlTransform.XmlTransformationLogger.LogWarning(XmlNode referenceNode, String message, Object[] messageArgs)
Set 'value' attribute
at Microsoft.Web.XmlTransform.Transform.ApplyOnAllTargetNodes()
Set 2 attributes
Done executing SetAttributes
So it looks like it doesn't like the PCWSUser appSetting.
Here's the web.config snippet for PCWSUser:
...
<add key="PCWSUser" value="TheUserName" />
...
Here's the web.[environment].config (in this case web.qa.config) snippet for PCWSUser:
...
<add key="PCWSUser" value="TheUserNameQA" xdt:Transform="SetAttributes" xdt:Locator="Match(key)"/>
...
I'm not really sure what I'm doing wrong... When the transform is done locally in Visual Studio it doesn't have any problem with it. Another strange thing is I've ran this a few times and it seems to pick a different appSetting to error out on each time. Same error message and all, just different settings. All the settings are set up this way FYI.
Let me know if you need any more info.
EDIT 1
As per #Kevin Lu-MSFT suggestion, I added /p:TransformWebConfigEnabled=false to the build step and tried again.
Build stage logs:
##[debug]INPUT_MSBUILDARGS: '/t:rebuild /p:DeployOnBuild=true /p:PublishProfile="Dev" /p:PackageLocation="D:\agent\_work\283\a" /p:TransformWebConfigEnabled=false'
However, the transform still failed, although the error moved around again. This time the error is in between 2 steps so Im not even clear what went wrong.
Deploy stage logs:
Executing Replace (transform line 10, 105)
on /configuration/connectionStrings/add[#name='SqlConnectionString']
Applying to 'add' element (no source line info)
Replaced 'add' element
Done executing Replace
System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.Web.XmlTransform.XmlTransformationLogger.ConvertUriToFileName(XmlDocument xmlDocument)
at Microsoft.Web.XmlTransform.XmlTransformationLogger.LogWarning(XmlNode referenceNode, String message, Object[] messageArgs)
at Microsoft.Web.XmlTransform.Transform.ApplyOnAllTargetNodes()
Executing Replace (transform line 11, 105)
on /configuration/connectionStrings/add[#name='DB2ConnectionString']
Applying to 'add' element (no source line info)
Replaced 'add' element
Done executing Replace
Based on my test, the FileTransform task could transform the web.config file successfully.
Here are the steps (directly transform the file without build the project), you could refer to them.
Step1: File structure. You need to make sure that the files are in the same folder.
Web.config
<configuration>
<connectionStrings>
<appSettings>
....
<add key="PCWSUser" value="TheUserName" />
</appSettings>
</configuration>
web.qa.config
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
....
<appSettings>
<add key="PCWSUser" value="TheUserNameQA" xdt:Transform="SetAttributes" xdt:Locator="Match(key)"/>
</appSettings>
</configuration>
Step2: Use the FileTransform task with Yaml.
- task: FileTransform#2
inputs:
folderPath: 'configfolder'
xmlTransformationRules: '-transform **\web.qa.config -xml **\web.config'
Then the task could run successfully. But if the folder contains multiple transform files, it may cause error.
On the other hand, if the FileTransform task is after the build step ,you need to make sure the build task doesn't transform the web.config file.
You could add the Msbuild arguments /p:TransformWebConfigEnabled=false in the build task.
Here is a discussion about this issue.
Hope this helps.

Issue when saving app.config with heterogeneous ConfigurationElementCollection items

I have an application that reads the app.config of a separate application. The app.config contains a collection of heterogeneous elements. The application I'm building will generate a TreeView control that displays the app.config custom section. the Save() method of the configuration object works fine unless I drill down and read/display the contents of the custom section. I followed this as an example (also found here). The issue seems to be that when I'm saving (after viewing and regardless if I make changes to the underlying config) the GetElementKey method is called. This returns the Key value of the element collection. But then this value is passed to the CreateNewElement method. This causes a problem because CreateNewElement passes back a new type depending on what elementName is passed in. At this point the element name isn't being passed in, just the key value.
The other strange thing I've noticed, from stepping through the code several times, is that if I don't read the values from the config to build the UI to display them, I hit the GetElementKey twice per element in the collection and CreateNewElement is not called at all. What happens when I do display values is that GetElementKey is called the same number of times. However, it comes back and is called again and then CreateNewElement is called with the key value as described above. Hopefully someone can describe to me what's going on under the hood that's causing this sequence of events to occur. Is there a property I need to change when deserializing the xml to the configuration objects?
This is the way I have my app.config structured:
<configuration>
<configSections>
<section name="section1" type="example" />
</configSections>
<section1>
<node1>
<elementA name="anElement" />
</node1>
<node2>
<add key="1" value="foo" />
</node2>
<node3>
<elementB>
<elementC name="anElement"/>
<elementD >
<elementE>
<foo name="foo1" />
<bar name="bar1" />
</elementE>
<elementF>
<elementG />
</elementF>
</elementD>
</elementB>
</section1>
</configuration>
The issues is regarding the collection in node section1/node3/elementD/elementE. After GetElementKey is called on "foo" and "bar" (2 times each) then it is called again and "foo1" is passed to CreateNewElement and that's when things break down.

API WiktionaryNET

I am trying to use the Wiktionary's API trying to know if some words are defined or not.
I have seen the open source's WiktionaryNET and they use this code:
In my console code:
var word = Wiktionary.Define("clean");
foreach (var def in word.Definition)
Console.WriteLine(def);
In the app.config:
<system.net>
<defaultProxy useDefaultCredentials="true" />
</system.net>
I use the same things but the result is always "Definition.Count = 0"
Someone know, how can I use or set up to get results?
Thanks in advance for your help.
I'm the implementer of this library. Thanks for pointing this out. I'll try to explain what happened and what can you do if you really need this fixed right now.
The wiktionary response is in JSON format but it's terrible to parse. It's actually one single blub of text. What happened is that the JSON response from wiktionary was modified since the wiktionaryNET was implemented. It now contains an additional field. The wiktionaryNET parser mistakenly interprets this as the content it was supposed to parse in the first place. The result is an empty response from the library.
You can download the project from GitHub. Then go to WiktionaryJsonQuery.cs and modify the AddQuery statements to include the rawcontinue:
AddQuery("format=json");
AddQuery("rawcontinue"); // <-- add this line
AddQuery("action=query");
AddQuery("prop=revisions");
AddQuery("rvprop=content");
AddQuery("titles=" + word);
Build the project and add the resulting dll to your project.
Please note this is only in beta.

Best way to store associative array in app.config

I've been working on my software lately and I have been wondering what the best way is to store an associative array.
The only thing I could come up with out of the blue is to do something like this:
<add key="disks" value="C|1|10,D|2|20,E|1|5,Z|1|3"/>
But this doesn't offer a lot of readability in my config file and I want my config file to be readable as it is a console application.
The reason for this because I've written a program that checks the diskspace of the disks specified in the app.config file but I want different thresholds for different disks.
How would you solve it?
Here's a part of my current config file.
<!-- DISK FEATURE SETTINGS -->
<!-- Type 1 is threshold by percentage and type 2 is threshold by a certain limit -->
<add key="threshold_type" value="1" />
<add key="threshold_limit" value="0,1" />
<!-- Space_type defines if you want to limit using kilobytes (1), megabytes (2) or gigabytes (3) if using threshold_type 2 -->
<add key="space_type" value="3" />
<!-- Put the disks here delimited by a comma like this: C,D,E -->
<add key="disks" value="C,D,E,Z"/>
<!-- SERVICE FEATURE SETTINGS -->
<!-- Put the services here delimited by a comma like this: C,D,E -->
<add key="services" value="spooler,ekrn,RadeonPro Support Service,TeamViewer6"/>
<!-- Put this on 1 if you want to log your output to a text file -->
<add key="logging" value="1"/>
I want to use the same principle for my performancecounter program that uses the perfmon counters to get some data and store it in a text file.
I hope people can help me for a bit here :)
I suggesst you to create your own configuration section. Custom configuration gives more readability and type safety. Here are links to create custom configuration
http://msdn.microsoft.com/en-us/library/2tw134k3.aspx and
http://haacked.com/archive/2007/03/11/custom-configuration-sections-in-3-easy-steps.aspx (old one but easy to follow).
As far as standard configuration mechanism works with XML serialization, the best (and, IMHO, the wise) way to store dictionaries in App.config is a List<KeyValuePair<K,V>>.
you may want to use Hashtable from system.collection or List<>
below are few pointers for hashtable,
http://www.dotnetperls.com/hashtable
http://www.tutorialspoint.com/csharp/csharp_hashtable.htm
i hope this helps!! thanks :)

Web.Config encoding problems

Wanted to let you know that I've tried to solve my problem before turning to the community for help.
I need to send a request with a couple of query parameters to a web service from which I should get an XML in the response which I need to parse and populate image source and image link with the values.
let's assume the the url for the website is:
domain.com/user=1&passwd=2&param=u
When i type this in a browser I could see the XML i should get as a response.
I place the value in appSettings in a web.Config file like so:
<appSettings>
<add key="GetImageUrl" value="http://domain.com/user=1&passwd=2&param=u"/>
</appSettings>
However, this would not compile because the compiler cannot understand '&' so I used the encoded version like this:
<appSettings>
<add key="GetImageUrl" value="http://domain.com/user=1&passwd=2&param=u"/>
</appSettings>
However, when I type this I see the XML with "Error" values which is one of the expected values to be returned if wrong parameters are sent.
How should I proceed?
Thank you very much!
Edit:
Wanted to let you know that I've also tried "&;amp;" and "%26"
The web.config file is an XML file, so you must encode & as &.
When reading the value in your program, you will get a &.
I don't know wha tyou mean by:
when I type this I see the XML with "Error" values which is one of the expected values to be returned if wrong parameters are sent
Where is this happening? Sending where?

Categories

Resources