Develop Reference

Finding all XPaths in a XQuery using Saxon-HE with C#

Situational Background: XSD with SCH
XML Schema (XSD)
I have an XML schema definition ("the schema") that includes several other XSDs, all in the same namespace. Some of those import other XSDs from foreign namespaces. All in all, the schema declares several global elements that can be instantiated as XML documents. Let's call them Global_1, Global_2 and Global_3.
Business Rules (SCH)
The schema is augmented by a Schematron file that defines the "business rules". It defines a number of abstract rules, and each abstract rule contains a number of assertions using the data model defined via XSD. For instance:
<sch:pattern>
<sch:rule id="rule_A" abstract="true">
<sch:assert test="if (abc:a/abc:b = '123') then abc:x/abc:y = ('aaa', 'bbb', 'ccc') else true()" id="A-01">Error message</sch:assert>
<sch:assert test="not(abc:c = 'abcd' and abc:d = 'zz')" id="A-02">Some other error message</sch:assert>
</sch:rule>
<!-- (...) -->
</sch:pattern>
Each abstract rule is extended by one or more non-abstract (concrete) rule that defines a specific context in which the abstract rule's assertions are to be validated. For example:
<sch:pattern>
<!-- (...) -->
<sch:rule context="abc:Global_1/abc:x/abc:y">
<sch:extends rule="rule_A"/>
</sch:rule>
<sch:rule context="abc:Global_2/abc:j//abc:k/abc:l">
<sch:extends rule="rule_A"/>
</sch:rule>
<!-- (...) -->
</sch:pattern>
In other words, all the assertions defined within the abstract rule_A are being applied to their specific contexts.
Both "the schema" and "the business rules" are subject to change - my program gets them at run-time and I don't know their content at design-time. The only thing I can safely assume is that there are no endless recursive structures in the schema: There is always one definite leaf node for every type and no type contains itself. Put differently, there are no "infinite loops" possible in the instances.
The Problem I want To Solve
Basically, I want to evaluate programmatically if each of the defined rules is correct. Since correctness can be quite a problematic topic, here by correctness I simply mean: Each XPath used in a rule (i.e. its context and within the XQueries of its inherited assertions) is "possible", meaning it can exist according to the data model defined in the schema. If, for instance, a namespace prefix is forgotten (abc:a/b instead of abc:a/abc:b), this XPath will never return anything other than an empty node set. The same is true if one step in the XPath is accidentally omitted, or spelled wrong, etc. This is obviously not a very strong claim for "correctness" of such a rule, but it'll do for a first step.
My Approach Towards A Solution For This
At least to me it doesn't seem like a trivial problem to evaluate an XPath (not to speak of the entire XQuery!) designed for the instance of a schema against the actual schema, given how it may contain axis steps like //, ancestor::, sibling::, etc. So I decided to construct something I would call a "maximum instance": By recursively iterating through all global elements and their children (and the structure of their respective complex types etc.), I build an XML instance at run-time that contains every possible element and attribute where it would be in the normal instance, but all at once. So every optional element/attribute, every element within a choice block and so on. So, said maximum instance would look something like this:
<maximumInstance>
<Global_1>
<abc:a>
<abc:b additionalAttribute="some_fixed_value">
<abc:j/>
<abc:k/>
<abc:l/>
</abc:b>
</abc:a>
</Global_1>
<Global_2>
<abc:x>
<abc:y>
<abc:a/>
<abc:z>
<abc:l/>
</abc:z>
</abc:y>
</abc:x>
</Global_2>
<Global_3>
<!-- ... -->
</Global_3>
<!-- ... -->
</maximumInstance>
All it takes now is to iterate over all abstract rules: And for every assertion in each abstract rule it must be checked that for every context the respective abstract rule is extended by, every XPath within an assertion results in a non-empty node set when evaluated against the maximum instance.
Where I'm stuck
I have written a C# (.NET Framework 4.8) program that parses "the schema" into said "maximum instance" (which is an XDocument at run-time). It also parses the business rules into a structure that makes it easy to get each abstract rule, its assertions, and the contexts these assertions are to be validated against.
But currently, I only have each complete XQuery (just like they are in the Schematron file) which effectively creates an assertion. But I actually need to break the XQuery down into its components (I guess I'd need the abstract syntax tree) so that I would have all individual XPaths. For instance, when given the XQuery if (abc:a/abc:b = '123') then abc:x/abc:y = ('aaa', 'bbb', 'ccc') else true(), I would need to retrieve abc:a/abc:b and abc:x/abc:y.
I assume that this could be done using Saxon-HE (or maybe another Parser/Compiler currently available for C# I don't know about). Unfortunately, I have yet to understand how to make use of Saxon well enough to even find at least a valid starting point for what I want to achieve. I've been trying to use the abstract syntax tree (so I can access the respective XPaths in the XQuery) seemingly accessible via XQueryExecutable:
Processor processor = new Processor();
XQueryCompiler xqueryCompiler = processor.NewXQueryCompiler();
XQueryExecutable exe = xqueryCompiler.Compile(xquery);
var AST = exe.getUnderlyingCompiledQuery();
var st = new XDocument();
st.Add(new XElement("root"));
XdmNode node = processor.NewDocumentBuilder().Build(st.CreateReader());
AST.explain((node); // <-- this is an error!
But that doesn't get me anywhere: I don't find any properties exposed I could work with? And while VS offers me to use AST.explain(...) (which seems promising), I'm unable to figure out what to parametrize here. I tried using a XdmNode which I thought would be a Destination? But also, I am using Saxon 10 (via NuGet), while Destination seems to be from Saxon 9: net.sf.saxon.s9api.Destination?!
Does anybody who was kind enough to read through all of this have any advice for me on how to tackle this? :-) Or, maybe there's a better way to solve my problem I haven't thought of - I'm also grateful for suggestions.
TL;DR
Sorry for the wall of text! In short: I have Schematron rules that augment an XML schema with business logic. To evaluate these rules (not: validate instances against the rules!) without actual XML instances, I need to break down the XQueries which make up the Schematron's assertions into their components so that I can handle all XPaths used in them. I think it can be done with Saxon-HE, but my knowledge is too limited to even understand what a good starting point what be for that. I'm also open for suggestions regarding a possibly better approach to solve my actual problem (as described in detail above).
Thank you for taking the time to read this.

If this were an XSD schema rather than a Schematron schema, then Saxon-EE would do the job for you automatically: this is very similar what a schema-aware XQuery processor attempts to do. But another difference is that in schema-aware XQuery, you can't assume that every element named foo is a valid instance of the element declaration named foo in the schema; it's quite legitimate, for example, for a query to transform valid instances into invalid instances, or vice versa. The input and output, after all, might conform to different schemas.
Saxon uses path analysis to do this: it looks at path expressions to see "where they might lead". Path analysis is also used to assess streamability, and to support document projection (building a trimmed-down tree representation of the source document that leaves out the parts that the query cannot reach). The path analysis in Saxon is by no means complete, for example it doesn't attempt to handle recursive functions. Although all these operations require Saxon-EE, the basic path analysis code is actually present in Saxon-HE, but I would offer no guarantee that it works for any purpose other than those described.
You're basically right that this is a tough problem you've set yourself, and I wish you luck with it.
Another approach you could adopt that wouldn't involve grovelling around the Saxon internals is to convert the XQuery to XQueryX, which is an XML representation of the parse tree, and then inspect the XQueryX (presumably using XQuery) to find the parts you need.

While XQueryX (as pointed out by Michael Kay) would theoretically have been exactly what I was looking for, unfortunately I could not find anything useful regarding an implementation for .NET during my research.
So I eventually solved the whole thing by creating my own parser using the XPath3.1 grammar for ANTLR4 as an ideal starting point. This way, I am now able to retrieve a syntax tree of any Schematron rule expression, allowing me to extract each contained XPath expression (and its sub expressions) separately.
Note that another stumbling block has been the fact that .NET still (!) only handles XPath 1.0 genuinely: While my parser does everything as supposed to, for some of the found expressions .NET gave me "illegal token" errors when trying to evaluate them. Installing the XPath2 NuGet package by Chertkov/Heyenrath was the solution.

Best way to compare XML in C#

I'm looking for the best way to compare two XML files for difference using C#. Like say for example if I have two XMLs A and B like this:
XML A
<data:TR xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:pd="http://www.ascentn.com/bpm/XMLSchema">
<data:processFields />
<data:formFields>
<data:TextBox1>111</data:TextBox1>
<data:TextBox2>222</data:TextBox2>
<data:TextBox3>3333</data:TextBox3>
<data:Repeat1_Repeat>
<data:Repeat1>
<data:TextBox4>444</data:TextBox4>
<data:TextBox5>555</data:TextBox5>
</data:Repeat1>
</data:Repeat1_Repeat>
</data:formFields>
</data:TR>
XML B
<data:TR xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:pd="http://www.ascentn.com/bpm/XMLSchema">
<data:processFields />
<data:formFields>
<data:TextBox1>111</data:TextBox1>
<data:TextBox2>aaa</data:TextBox2>
<data:TextBox3>3333</data:TextBox3>
<data:Repeat1_Repeat>
<data:Repeat1>
<data:TextBox4>bbb</data:TextBox4>
<data:TextBox5>555</data:TextBox5>
</data:Repeat1>
<data:Repeat1>
<data:TextBox4>ccc</data:TextBox4>
<data:TextBox5>ddd</data:TextBox5>
</data:Repeat1>
</data:Repeat1_Repeat>
</data:formFields>
</data:TR>
I'm looking to get only the different between the two XML files, like in this case it would be TextBox2 and TextBox4 and one full node for Repeat1_Repeat.
Is there a easy way to get this? Maybe use some framework? I'm using .NET 4.5.2 so anything recent would work too!
Thanks!
EDIT : Oh and also, I need it to work for n-level of nesting.

I think XMLDiff is the best way. No framework needed. As seen on MSDN:
By using the XMLDiff class, the programmer is able to determine if the
two files are in fact different based on the conditions that are
important to their application. The programmer is able to ignore
changes that are only superficial (for example, different prefixes for
same namespace). XMLPatch then provides the ability to update the
original XML by applying only the changes that matter to the original
XML.
You should check it out:
https://msdn.microsoft.com/en-us/library/aa302294.aspx

You can use Paste Special to generate classes for your XML.
Then you can deserialise your xml to create an instance of your code generated class.
So for two xml files; you can create two objects like xmlObject1 and xmlObject2.
Then you can use CompareObject to identify the difference between two objects.

Parsing C# code for contextually aware semantic highlighting

I'm working on a semantic highlighting plugin for VS. Here you can see a web Example.
The goal:
Acquiring all variables and creating different Classifications for every one of them.
The problem:
Getting the variables from the code without writing a C# lexer.
My current approach uses an ITagger. I use an ITagAggregator to get the tags of all the spans that get passed to the ITagger. Then I filter those and get only spans with the "identifier" classification which includes varibles, methods names, class names, usings and properties.
public class Classifier : ITagger<ClassificationTag> {
public IEnumerable<ITagSpan<ClassificationTag>> GetTags(NormalizedSnapshotSpanCollection spans) {
ITextSnapshot snapshot = spans[0].Snapshot;
var tags = _aggregator.GetTags(spans).Where((span) => span.Tag.ClassificationType.Classification.Equals("identifier")).ToArray();
foreach(var classifiedSpan in tags) {
foreach(SnapshotSpan span in classifiedSpan.Span.GetSpans(snapshot)) {
//generate classification based on variable name
yield return new TagSpan<ClassificationTag>(span, new ClassificationTag(_classification));
}
}
}
}
It would be a lot easier to use the builtin C# Lexer to get a list of all variables bundled to a bunch of meta data. Is this data available for plugin development? Is there an alternative way I could acquire it, if not?

The problem: Getting the variables from the code without writing a C# lexer.
Roslyn can do this: https://roslyn.codeplex.com/
There's even a Syntax Visualizer sample that might interest you. I also found an example using Roslyn to create a Syntax Highlighter.

Visual Studio exposes that information as a code model.
Here is an example how you can access class, and then find attribute on the class, and parse attribute arguments:
Accessing attribute info from DTE
Here is more information about code models:
http://msdn.microsoft.com/en-us/library/ms228763.aspx
Here's also automation object model chart what I've been using quite few times: http://msdn.microsoft.com/en-us/library/za2b25t3.aspx
Also, as said, Roslyn is indeed also a possible option. Here is an example for VS2015 using roslyn: https://github.com/tomasr/roslyn-colorizer/blob/master/RoslynColorizer/RoslynColorizer.cs

For building language tools if may be better to use a parser generator for C#. The GOLD parsing system is one such toolkit which can handle LALR grammars. It has a .NET component based engine that you can use in your project and it can be used to integrate with any IDE. You can also find the grammars for various programming languages including C#.

Removing literal strings in .net code

What have you done to remove (Helpers/extension methods) string literal in code?
e.g. I have nhibernate search criteria code like this all over the place.
Expression.Eq("Status", LoginStatus.LoggedIn),
“Status” being the property of an entity object used as a string in the case.
Update: Primary reason in this case is to enable refactoring. If I write a helper method which reflects the object and gets the value, will make the above expression strongly typed.

This is what "Resources" and "Settings" is for. You can find this by right clicking your project in Visual Studio and clicking "Properties", then going to the "Resources" or "Settings" tab.
For pre-built resources that won't change often, use resources. For things you want to be configurable use Settings instead because it will auto-generate blocks of configuration for your App.config. You will still need to manually copy and paste these values if you do not want to use the defaults.
The nice thing about both is that VS will build a nice static class with properties you can use throughout your code. VS will maintain the class and resources for you dynamically as long as you continue to use the wizard.

I'll usually declare them as constants, or, if I have groups of related strings, I'll create an enum instead.
Either way, at least they have a descriptive name attached to them (instead of using "magic strings"), and their use will always be consistent.

In the past, I've used CodeRush (or your favourite refactoring tool) to convert to a const string in the class, and then moved said const strings to be public members of the entity class to which they apply.
The real answer here, if you're looking to get your code less brittle in the face of refactoring is to get out of the strings business, and use Linq 4/to NHibernate, but you'd have to research whether it's completeness is sufficeint for your purposes.

Realized that I could do this the Expression trees way. Using Code as data!
Something like this
protected IList<T> _FindByProperty<TResult>(Expression<Func<T, TResult>> expression, TResult value)
{
return _FindByProperty((expression.Body as MemberExpression).Member.Name, value);
}
IList<User> costCenters = _FindByProperty( user=> user.Name, "name");
Credits: http://suryagaddipati.wordpress.com/2009/03/14/code-as-data-in-c-taking-advantage-of-expression-trees/
This is related to a lot questions in the expression-trees tag.

I use a similar approach as Cherian. I got my idea from the FluentNhibernate's ReflectionHelper.
The principle is to use expression trees and then you could just put in a x => x.Status expression. The method would return the property name as string.
In fact, you could also just use FluentNHibernate? However, I don't know if their querying model is evenly extensive as their mapping interfaces...

Localization for Winforms from designmode?

I need to bind labels or items in a toolstrip to variables in Design Mode.
I don't use the buit-in resources not the settings, so the section Data is not useful. I am taking the values out from an XML that I map to a class.
I know there are many programs like:
http://www.jollans.com/tiki/tiki-index.php?page=MultilangVsNetQuickTourForms
but they work with compiled resx. I want to use not compiled XML.
I know that programatically i can do it, i create a method (for example, UpdateUI()), and there I assign the new values like this:
this.tsBtn.Text=Class.Texts.tsBtnText;
I would like something i could do from Design Mode or a more optimized way than the current one. Is there any Custom Control out there or Extension?

Aleksandar's response is one way to accomplish this, but in the long run it's going to be very time consuming and won't really provide much benefit. The bigger question that should be asked is why do you not want to use the tools and features built-in to .NET and Visual Studio or at least use a commercial third-party tool? It sounds like you are spending (have spent?) a lot of time to solve a problem that has already been solved.

Try with inheriting basic win controls and override OnPaint method. Example bellow is a button that has his text set on paint depending on value contained in his Tag property (let suppose that you will use Tag property to set the key that will be used to read matching resource). Then you can find some way to read all cache resource strings from xml files (e.g. fictional MyGlobalResources class.
public class LocalizedButton : Button
{
protected override void OnPaint(PaintEventArgs pevent)
{
base.OnPaint(pevent);
this.Text = MyGlobalResources.GetItem(this.Tag.ToString());
}
}

You can use satellite assemblies for localization and generate them using your XML file as a source for the translated entities.
more about satellites http://msdn.microsoft.com/en-us/library/21a15yht(VS.71).aspx
sure it's not from design mode, but there's no way to do it this way with your restrictions.