FileTransform task in Azure DevOps throws error on transform - c#

I am using the FileTransform#2 task to transform a web.config with a web.[environment].config in a Azure DevOps Pipeline (yaml). It seems to fail on one of the individual transforms which fails the whole job, though I'm not sure why.
Here is the error message from the task:
Executing SetAttributes (transform line 72, 48)
on /configuration/appSettings/add[#key='PCWSUser']
System.NullReferenceException: Object reference not set to an instance of an object.
Applying to 'add' element (no source line info)
at Microsoft.Web.XmlTransform.XmlTransformationLogger.ConvertUriToFileName(XmlDocument xmlDocument)
Set 'key' attribute
at Microsoft.Web.XmlTransform.XmlTransformationLogger.LogWarning(XmlNode referenceNode, String message, Object[] messageArgs)
Set 'value' attribute
at Microsoft.Web.XmlTransform.Transform.ApplyOnAllTargetNodes()
Set 2 attributes
Done executing SetAttributes
So it looks like it doesn't like the PCWSUser appSetting.
Here's the web.config snippet for PCWSUser:
...
<add key="PCWSUser" value="TheUserName" />
...
Here's the web.[environment].config (in this case web.qa.config) snippet for PCWSUser:
...
<add key="PCWSUser" value="TheUserNameQA" xdt:Transform="SetAttributes" xdt:Locator="Match(key)"/>
...
I'm not really sure what I'm doing wrong... When the transform is done locally in Visual Studio it doesn't have any problem with it. Another strange thing is I've ran this a few times and it seems to pick a different appSetting to error out on each time. Same error message and all, just different settings. All the settings are set up this way FYI.
Let me know if you need any more info.
EDIT 1
As per #Kevin Lu-MSFT suggestion, I added /p:TransformWebConfigEnabled=false to the build step and tried again.
Build stage logs:
##[debug]INPUT_MSBUILDARGS: '/t:rebuild /p:DeployOnBuild=true /p:PublishProfile="Dev" /p:PackageLocation="D:\agent\_work\283\a" /p:TransformWebConfigEnabled=false'
However, the transform still failed, although the error moved around again. This time the error is in between 2 steps so Im not even clear what went wrong.
Deploy stage logs:
Executing Replace (transform line 10, 105)
on /configuration/connectionStrings/add[#name='SqlConnectionString']
Applying to 'add' element (no source line info)
Replaced 'add' element
Done executing Replace
System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.Web.XmlTransform.XmlTransformationLogger.ConvertUriToFileName(XmlDocument xmlDocument)
at Microsoft.Web.XmlTransform.XmlTransformationLogger.LogWarning(XmlNode referenceNode, String message, Object[] messageArgs)
at Microsoft.Web.XmlTransform.Transform.ApplyOnAllTargetNodes()
Executing Replace (transform line 11, 105)
on /configuration/connectionStrings/add[#name='DB2ConnectionString']
Applying to 'add' element (no source line info)
Replaced 'add' element
Done executing Replace

Based on my test, the FileTransform task could transform the web.config file successfully.
Here are the steps (directly transform the file without build the project), you could refer to them.
Step1: File structure. You need to make sure that the files are in the same folder.
Web.config
<configuration>
<connectionStrings>
<appSettings>
....
<add key="PCWSUser" value="TheUserName" />
</appSettings>
</configuration>
web.qa.config
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
....
<appSettings>
<add key="PCWSUser" value="TheUserNameQA" xdt:Transform="SetAttributes" xdt:Locator="Match(key)"/>
</appSettings>
</configuration>
Step2: Use the FileTransform task with Yaml.
- task: FileTransform#2
inputs:
folderPath: 'configfolder'
xmlTransformationRules: '-transform **\web.qa.config -xml **\web.config'
Then the task could run successfully. But if the folder contains multiple transform files, it may cause error.
On the other hand, if the FileTransform task is after the build step ,you need to make sure the build task doesn't transform the web.config file.
You could add the Msbuild arguments /p:TransformWebConfigEnabled=false in the build task.
Here is a discussion about this issue.
Hope this helps.

Related

Custom Column(s) with MsSqlServer Sink and AppSettings

I've got an application that runs without problem with the File and Console Sinks and now I'm trying to add the MSSqlServer Sink.
Looking at the documentation on Github I've got my application to write to the SQL Database as well as the other sinks.
<add key="serilog:write-to:MSSqlServer.connectionString" value="Server=servername;Database=databasename;User Id=userid;Password=password;"/>
<add key="serilog:write-to:MSSqlServer.tableName" value="Logs"/>
<add key="serilog:write-to:MSSqlServer.autoCreateSqlTable" value="true"/>
One improvement is I'd like to add a custom column to the Logs table that Serilog uses to store a number indicating a unique RunId for my application. The idea being that a simple query would allow grouping by RunId to see all messages for that one run.
So I added the following, based on the documentation (and I haven't been able to find any other examples) as it seemed logical:
<add key="serilog:write-to:MSSqlServer.columnOptions.ColumnName" value="RunId"/>
<add key="serilog:write-to:MSSqlServer.columnOptions.PropertyName" value="RunId"/>
<add key="serilog:write-to:MSSqlServer.columnOptions.DataType" value="SqlDbType.Int"/>
<add key="serilog:write-to:MSSqlServer.columnOptions.DataLength" value="32"/>
and then in my code all I need to do is:
Log.Information("{RunId}{Message}", RunId, Message);
to see a new entry with {RunId} in the RunId column and {Message} in the Message column... however everytime I do this nothing is written to the RunId column, it remains as NULL whereas every log message the console/file has is also duplicated in the Table.
So it seems logging is working, it must be the keys wrong and I'm really not sure what should be used.
Would anyone be able to point me in the direction I need to be going or where I've gone wrong?
Thank you.
Logging definitely was working but some digging and finally I found out I had two issues:
Permissions on database where insuffident for the user serilog was using, and
AppSettings settings I was using were wrong
#1 was easy enough to fix, I created a dedicated Serilog account for this database and fixed the permissions as per documentation.
#2 however was very frustrating but eventually I was able to get the following to work:
<configSections>
<section name="MSSqlServerSettingsSection" type="Serilog.Configuration.MSSqlServerConfigurationSection, Serilog.Sinks.MSSqlServer"/>
</configSections>
<MSSqlServerSettingsSection DisableTriggers="false" ClusteredColumnstoreIndex="false" PrimaryKeyColumnName="Id">
<!-- SinkOptions parameters -->
<TableName Value="Logs"/>
<Columns>
<add ColumnName="RunId" DataType="int"/>
</Columns>
</MSSqlServerSettingsSection>
On compile and execution my program will now create the Logs table in the database and then create a RunId column which I have been able to populate with a {RunId} expression in my Log.Debug() calls.
FWIW: I hope this will be helpful and if I can work out how I'll see if I can add this to documentation as an example of using AppSettings for this use case. Searching this question most people seem to be using JSON and not AppSettings.

Issue when saving app.config with heterogeneous ConfigurationElementCollection items

I have an application that reads the app.config of a separate application. The app.config contains a collection of heterogeneous elements. The application I'm building will generate a TreeView control that displays the app.config custom section. the Save() method of the configuration object works fine unless I drill down and read/display the contents of the custom section. I followed this as an example (also found here). The issue seems to be that when I'm saving (after viewing and regardless if I make changes to the underlying config) the GetElementKey method is called. This returns the Key value of the element collection. But then this value is passed to the CreateNewElement method. This causes a problem because CreateNewElement passes back a new type depending on what elementName is passed in. At this point the element name isn't being passed in, just the key value.
The other strange thing I've noticed, from stepping through the code several times, is that if I don't read the values from the config to build the UI to display them, I hit the GetElementKey twice per element in the collection and CreateNewElement is not called at all. What happens when I do display values is that GetElementKey is called the same number of times. However, it comes back and is called again and then CreateNewElement is called with the key value as described above. Hopefully someone can describe to me what's going on under the hood that's causing this sequence of events to occur. Is there a property I need to change when deserializing the xml to the configuration objects?
This is the way I have my app.config structured:
<configuration>
<configSections>
<section name="section1" type="example" />
</configSections>
<section1>
<node1>
<elementA name="anElement" />
</node1>
<node2>
<add key="1" value="foo" />
</node2>
<node3>
<elementB>
<elementC name="anElement"/>
<elementD >
<elementE>
<foo name="foo1" />
<bar name="bar1" />
</elementE>
<elementF>
<elementG />
</elementF>
</elementD>
</elementB>
</section1>
</configuration>
The issues is regarding the collection in node section1/node3/elementD/elementE. After GetElementKey is called on "foo" and "bar" (2 times each) then it is called again and "foo1" is passed to CreateNewElement and that's when things break down.

API WiktionaryNET

I am trying to use the Wiktionary's API trying to know if some words are defined or not.
I have seen the open source's WiktionaryNET and they use this code:
In my console code:
var word = Wiktionary.Define("clean");
foreach (var def in word.Definition)
Console.WriteLine(def);
In the app.config:
<system.net>
<defaultProxy useDefaultCredentials="true" />
</system.net>
I use the same things but the result is always "Definition.Count = 0"
Someone know, how can I use or set up to get results?
Thanks in advance for your help.
I'm the implementer of this library. Thanks for pointing this out. I'll try to explain what happened and what can you do if you really need this fixed right now.
The wiktionary response is in JSON format but it's terrible to parse. It's actually one single blub of text. What happened is that the JSON response from wiktionary was modified since the wiktionaryNET was implemented. It now contains an additional field. The wiktionaryNET parser mistakenly interprets this as the content it was supposed to parse in the first place. The result is an empty response from the library.
You can download the project from GitHub. Then go to WiktionaryJsonQuery.cs and modify the AddQuery statements to include the rawcontinue:
AddQuery("format=json");
AddQuery("rawcontinue"); // <-- add this line
AddQuery("action=query");
AddQuery("prop=revisions");
AddQuery("rvprop=content");
AddQuery("titles=" + word);
Build the project and add the resulting dll to your project.
Please note this is only in beta.

Web.Config encoding problems

Wanted to let you know that I've tried to solve my problem before turning to the community for help.
I need to send a request with a couple of query parameters to a web service from which I should get an XML in the response which I need to parse and populate image source and image link with the values.
let's assume the the url for the website is:
domain.com/user=1&passwd=2&param=u
When i type this in a browser I could see the XML i should get as a response.
I place the value in appSettings in a web.Config file like so:
<appSettings>
<add key="GetImageUrl" value="http://domain.com/user=1&passwd=2&param=u"/>
</appSettings>
However, this would not compile because the compiler cannot understand '&' so I used the encoded version like this:
<appSettings>
<add key="GetImageUrl" value="http://domain.com/user=1&passwd=2&param=u"/>
</appSettings>
However, when I type this I see the XML with "Error" values which is one of the expected values to be returned if wrong parameters are sent.
How should I proceed?
Thank you very much!
Edit:
Wanted to let you know that I've also tried "&;amp;" and "%26"
The web.config file is an XML file, so you must encode & as &.
When reading the value in your program, you will get a &.
I don't know wha tyou mean by:
when I type this I see the XML with "Error" values which is one of the expected values to be returned if wrong parameters are sent
Where is this happening? Sending where?

How to change location of app.config

I want to change the location where my application looks for the app.config file.
I know that I can use ConfigurationManager.OpenExeConfiguration() to access an arbitrary config file - however, when the .Net Framework reads the config file (for ConnectionStrings or EventSources, for instance), it will look at the default location. I want to actually change the location, globally for the entire .Net Framework (for my application, of course).
I also know that I can use AppDomainSetup to change the location of the app.config for a new AppDomain. However, that doesn't apply to the primary AppDomain of the application.
I also know that I can override function Main() and create a new AppDomain as above and run my application in that new AppDomain. However, that has other side-effects - for instance, Assembly.GetEntryAssembly() will return a null reference.
Given how everything else works in .Net, I would expect there to be some way to configure the startup environment of my application - via a Application Manifest, or some such - but I have been unable to find even a glimmer of hope in that direction.
Any pointer would be helpful.
David Mullin
I used the approach with starting another AppDomain from Main(), specifying the "new" location of the configuration file.
No issues with GetEntryAssembly(); it only returns null, when being called from unmanaged code - or at least it doesn't for me, as I use ExecuteAssembly() to create/run the second AppDomain, much like this:
int Main(string[] args)
{
string currentExecutable = Assembly.GetExecutingAssembly().Location;
bool inChild = false;
List<string> xargs = new List<string>();
foreach (string arg in xargs)
{
if (arg.Equals("-child"))
{
inChild = true;
}
/* Parse other command line arguments */
else
{
xargs.Add(arg);
}
}
if (!inChild)
{
AppDomainSetup info = new AppDomainSetup();
info.ConfigurationFile = /* Path to desired App.Config File */;
Evidence evidence = AppDomain.CurrentDomain.Evidence;
AppDomain domain = AppDomain.CreateDomain(friendlyName, evidence, info);
xargs.Add("-child"); // Prevent recursion
return domain.ExecuteAssembly(currentExecutable, evidence, xargs.ToArray());
}
// Execute actual Main-Code, we are in the child domain with the custom app.config
return 0;
}
Note that we are effectively rerunning the EXE, just as a AppDomain and with a different config. Also note that you need to have some "magic" option that prevents this from going on endlessly.
I crafted this out from a bigger (real) chunk of code, so it might not work as is, but should illustrate the concept.
I am not sure why you want to change the location of your config file - perhaps there can be different approach for solving your actual problem. I had a requirement where I wanted to share configuration file across related applications - I had chosen to use own xml file as it had given me extra benefit of having complete control over the schema.
In your case, it's possible to externalize sections of your config file to a separate file using configSource property. See here under "Using External Configuration Files" to check how it has been done for connection strings section. Perhaps, this may help you.
var configPath = YOUR_PATH;
if (!Directory.Exists(ProductFolder))
{
Directory.CreateDirectory(ProductFolder);
}
if (!File.Exists(configPath))
{
File.WriteAllText(configPath, Resources.App);
}
var map = new ExeConfigurationFileMap
{
ExeConfigFilename = configPath,
LocalUserConfigFilename = configPath,
RoamingUserConfigFilename = configPath
};
Configuration config = ConfigurationManager.OpenMappedExeConfiguration(map, ConfigurationUserLevel.None);
Then use config member as you want.
Another approach is to leave the config file with the executable file and move the relevant changeable sections to external xml files which can be in whatever location you choose.
If you are using your config file in a readonly capacity, then you can add the relevant chunks to an XML file in a different location using XML Inlcude. This won't work if you are trying to write values back directly to app.config using the Configuration.Save method.
app.config:
<?xml version="1.0"?>
<configuration xmlns:xi="http://www.w3.org/2001/XInclude">
<appSettings>
<xi:include href="AppSettings.xml"/>
</appSettings>
<connectionStrings>
<xi:include href="ConnectionStrings.xml"/>
</connectionStrings>
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7"/></startup>
</configuration>
ConnectionStrings.xml:
<?xml version="1.0"?>
<add name="Example1ConnectionString"
connectionString="Data Source=(local)\SQLExpress;Initial Catalog=Example1DB;Persist Security Info=True;User ID=sa;Password=password"
providerName="System.Data.SqlClient" />
<add name="Example2ConnectionString"
connectionString="Data Source=(local)\SQLExpress;Initial Catalog=Example2DB;Persist Security Info=True;User ID=sa;Password=password"
providerName="System.Data.SqlClient" />
AppSettings.xml:
<?xml version="1.0"?>
<add key="Setting1" value="Value1"/>
<add key="Setting2" value="Value2"/>
A file URI looks like this:
file:///C:/whatever.txt
You can even define failover files in case the one you are trying to reference is missing. This pattern is from https://www.xml.com/pub/a/2002/07/31/xinclude.html:
<xi:include href="http://www.whitehouse.gov/malapropisms.xml">
<xi:fallback>
<para>
This administration is doing everything we can to end the stalemate in
an efficient way. We're making the right decisions to bring the solution
to an end.
</para>
</xi:fallback>

Categories

Resources