Develop Reference

How do I get the containing namespace of a called method using Roslyn when that method has no symbol info?

I have an application that allows users to write c-sharp code that gets saved as a class library for being called later.
A new requirement has been established that some namespaces (and the methods they contain and any variables or methods with their return types) are not allowed anymore. So I need to analyze the code and alert the user to any forbidden namespaces in their code so they can remove them.
Using Roslyn, I can access the InvocationExpressionSyntax nodes for the method calls. From that I then get the symbol info by calling var mySymbol = mySemanticModel.GetSymbolInfo(myInvocationExpressionSyntaxNode).Symbol.
Then calling mySymbol.ContainingType.ToDisplayString() returns the namespace type of the call.
However, it seems not all called methods have symbol information in Roslyn. For example, System.Math.Sqrt() has symbol information, so from that I can get the containing namespace of System.Math. On the other hand System.Net.WebRequest.Create() or System.Diagnostics.Process.Start() do not. How do I get System.Net.WebRequest or System.Dignostics.Process from those nodes? I can clearly see them using QuickWatch.
For example, the System.Diagnostics.Process.Start() node itself shows the following value in QuickWatch:
InvocationExpressionSyntax InvocationExpression System.Diagnostics.Process.Start("CMD.exe","")
And the node's expression has this value:
MemberAccessExpressionSyntax SimpleMemberAccessExpression System.Diagnostics.Process.Start
So obviously the namespace is there in the value itself. But the Symbol from the SymbolInfo and the Type from TypeInfo are both null.
Edit
In regards to my compilation, the C# Roslyn tools are set up as follows (we are supposed to support VB as well, hence the properties are interfaced):
private class CSharpRoslynTools : IRoslynTools
{
public CompilationUnitSyntax SyntaxTreeRoot { get; }
public SemanticModel SemanticModel { get; }
public CSharpRoslynTools(string code)
{
var mscorlib = MetadataReference.CreateFromFile(typeof(object).Assembly.Location);
var syntaxTree = CSharpSyntaxTree.ParseText(code);
var compilation = CSharpCompilation.Create(
"MyCompilation",
syntaxTrees: new[] { syntaxTree },
references: new[]
{
mscorlib
});
this.SemanticModel = compilation.GetSemanticModel(syntaxTree);
this.SyntaxTreeRoot = (CompilationUnitSyntax)syntaxTree.GetRoot();
}
}
One thing I did come to realize is that System Diagnostics isn't part of the mscorlib. Could that be why the symbol information is missing?
Honestly, I kind of view this as a bit of a waste of my time because the scripts were designed to be run in a WinForms desktop application that's probably 15 years old at this point. But then they decided to this desktop application needed to move to Citrix Cloud for certain customers. And as a result, we have to lock out anything that can access the filesystem if it's not an admin logged into the application. So we have this giant potential security hole with these scripts. The chance of someone getting access to the application and exploiting any of this is slim, though.
I pushed for a blacklist, which would be easy enough to do with a simple string search of the code. They want a whitelist which requires full out parsing of the symbols.

However, it seems not all called methods have symbol information in Roslyn.
This probably indicates that something went wrong with how you got your Compilation, and you should attempt to investigate that directly. Don't attempt to deal with it downstream. (Software: garbage in, garbage out!)
On the other hand System.Net.WebRequest.Create() or System.Diagnostics.Process.Start() do not. How do I get System.Net.WebRequest or System.Dignostics.Process from those nodes? I can clearly see them using QuickWatch.
Keep in mind that from the perspective of syntax only, System.Net.WebRequest.Create() could be:
A Create method on the WebRequest type that's in System.Net
A Create method on on the WebRequest type, which is a nested class of the Net type, in the System namespace
A Create method on the WebRequest type that's in MyApp.System.Net.WebRequest, because we of course don't require fully namespace names and if you decide to make a System namespace inside your MyApp, that could potentially work!
One thing I did come to realize is that System Diagnostics isn't part of the mscorlib. Could that be why the symbol information is missing?
Yep; we're only going to reference the assemblies you give us. It's up to you to know your context and if other references are included in what that code can reference, then you should include them in your production of the Compilation.
I pushed for a blacklist, which would be easy enough to do with a simple string search of the code. They want a whitelist which requires full out parsing of the symbols.
From a security perspective they may be right -- it's very difficult to block from a string search. See some thoughts at https://stackoverflow.com/a/66555319/972216 for how difficult that can be.

Is it possible to create a C# compile time Method? [duplicate]

I've been puzzling about this for a while and I've looked around a bit, unable to find any discussion about the subject.
Lets assume I wanted to implement a trivial example, like a new looping construct: do..until
Written very similarly to do..while
do {
//Things happen here
} until (i == 15)
This could be transformed into valid csharp by doing so:
do {
//Things happen here
} while (!(i == 15))
This is obviously a simple example, but is there any way to add something of this nature? Ideally as a Visual Studio extension to enable syntax highlighting etc.

Microsoft proposes Rolsyn API as an implementation of C# compiler with public API. It contains individual APIs for each of compiler pipeline stages: syntax analysis, symbol creation, binding, MSIL emission. You can provide your own implementation of syntax parser or extend existing one in order to get C# compiler w/ any features you would like.
Roslyn CTP
Let's extend C# language using Roslyn! In my example I'm replacing do-until statement w/ corresponding do-while:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Roslyn.Compilers.CSharp;
namespace RoslynTest
{
class Program
{
static void Main(string[] args)
{
var code = #"
using System;
class Program {
public void My() {
var i = 5;
do {
Console.WriteLine(""hello world"");
i++;
}
until (i > 10);
}
}
";
//Parsing input code into a SynaxTree object.
var syntaxTree = SyntaxTree.ParseCompilationUnit(code);
var syntaxRoot = syntaxTree.GetRoot();
//Here we will keep all nodes to replace
var replaceDictionary = new Dictionary<DoStatementSyntax, DoStatementSyntax>();
//Looking for do-until statements in all descendant nodes
foreach (var doStatement in syntaxRoot.DescendantNodes().OfType<DoStatementSyntax>())
{
//Until token is treated as an identifier by C# compiler. It doesn't know that in our case it is a keyword.
var untilNode = doStatement.Condition.ChildNodes().OfType<IdentifierNameSyntax>().FirstOrDefault((_node =>
{
return _node.Identifier.ValueText == "until";
}));
//Condition is treated as an argument list
var conditionNode = doStatement.Condition.ChildNodes().OfType<ArgumentListSyntax>().FirstOrDefault();
if (untilNode != null && conditionNode != null)
{
//Let's replace identifier w/ correct while keyword and condition
var whileNode = Syntax.ParseToken("while");
var condition = Syntax.ParseExpression("(!" + conditionNode.GetFullText() + ")");
var newDoStatement = doStatement.WithWhileKeyword(whileNode).WithCondition(condition);
//Accumulating all replacements
replaceDictionary.Add(doStatement, newDoStatement);
}
}
syntaxRoot = syntaxRoot.ReplaceNodes(replaceDictionary.Keys, (node1, node2) => replaceDictionary[node1]);
//Output preprocessed code
Console.WriteLine(syntaxRoot.GetFullText());
}
}
}
///////////
//OUTPUT://
///////////
// using System;
// class Program {
// public void My() {
// var i = 5;
// do {
// Console.WriteLine("hello world");
// i++;
// }
//while(!(i > 10));
// }
// }
Now we can compile updated syntax tree using Roslyn API or save syntaxRoot.GetFullText() to text file and pass it to csc.exe.

The big missing piece is hooking into the pipeline, otherwise you're not much further along than what .Emit provided. Don't misunderstand, Roslyn brings alot of great things, but for those of us who want to implement preprocessors and meta programming, it seems for now that was not on the plate. You can implement "code suggestions" or what they call "issues"/"actions" as an extension, but this is basically a one off transformation of code that acts as a suggested inline replacement and is not the way you would implement a new language feature. This is something you could always do with extensions, but Roslyn makes the code analysis/transformation tremendously easier:
From what I've read of comments from Roslyn developers on the codeplex forums, providing hooks into the pipeline has not been an initial goal. All of the new C# language features they've provided in C# 6 preview involved modifying Roslyn itself. So you'd essentially need to fork Roslyn. They have documentation on how to build Roslyn and test it with Visual Studio. This would be a heavy handed way to fork Roslyn and have Visual Studio use it. I say heavy-handed because now anyone who wants to use your new language features must replace the default compiler with yours. You could see where this would begin to get messy.
Building Roslyn and replacing Visual Studio 2015 Preview's compiler with your own build
Another approach would be to build a compiler that acts as a proxy to Roslyn. There are standard APIs for building compilers that VS can leverage. It's not a trivial task though. You'd read in the code files, call upon the Roslyn APIs to transform the syntax trees and emit the results.
The other challenge with the proxy approach is going to be getting intellisense to play nicely with any new language features you implement. You'd probably have to have your "new" variant of C#, use a different file extension, and implement all the APIs that Visual Studio requires for intellisense to work.
Lastly, consider the C# ecosystem, and what an extensible compiler would mean. Let's say Roslyn did support these hooks, and it was as easy as providing a Nuget package or a VS extension to support a new language feature. All of your C# leveraging the new Do-Until feature is essentially invalid C#, and will not compile without the use of your custom extension. If you go far enough down this road with enough people implementing new features, very quickly you will find incompatible language features. Maybe someone implements a preprocessor macro syntax, but it can't be used along side someone else's new syntax because they happened to use similar syntax to delineate the beginning of the macro. If you leverage alot of open source projects and find yourself digging into their code, you would encounter alot of strange syntax that would require you side track and research the particular language extensions that project is leveraging. It could be madness. I don't mean to sound like a naysayer, as I have alot of ideas for language features and am very interested in this, but one should consider the implications of this, and how maintainable it would be. Imagine if you got hired to work somewhere and they had implemented all kinds of new syntax that you had to learn, and without those features having been vetted the same way C#'s features have, you can bet some of them would be not well designed/implemented.

You can check www.metaprogramming.ninja (I am the developer), it provides an easy way to accomplish language extensions (I provide examples for constructors, properties, even js-style functions) as well as full-blown grammar based DSLs.
The project is open source as well. You can find documentations, examples, etc at github.
Hope it helps.

You can't create your own syntactic abstractions in C#, so the best you can do is to create your own higher-order function. You could create an Action extension method:
public static void DoUntil(this Action act, Func<bool> condition)
{
do
{
act();
} while (!condition());
}
Which you can use as:
int i = 1;
new Action(() => { Console.WriteLine(i); i++; }).DoUntil(() => i == 15);
although it's questionable whether this is preferable to using a do..while directly.

I found the easiest way to extend the C# language is to use the T4 text processor to preprocess my source. The T4 Script would read my C# and then call a Roslyn based parser, which would generate a new source with custom generated code.
During build time, all my T4 scripts would be executed, thus effectively working as an extended preprocessor.
In your case, the none-compliant C# code could be entered as follows:
#if ExtendedCSharp
do
#endif
{
Console.WriteLine("hello world");
i++;
}
#if ExtendedCSharp
until (i > 10);
#endif
This would allow syntax checking the rest of your (C# compliant) code during development of your program.

No there is no way to achieve what you'are talking about.
Cause what you're asking about is defining new language construct, so new lexical analysis, language parser, semantic analyzer, compilation and optimization of generated IL.
What you can do in such cases is use of some macros/functions.
public bool Until(int val, int check)
{
return !(val == check);
}
and use it like
do {
//Things happen here
} while (Until(i, 15))

How to access project code meta data?

In my VSPackage I need to replace reference to a property in code with its actual value. For example
public static void Main(string[] args) {
Console.WriteLine(Resource.HelloWorld);
}
What I want is to replace "Resource.HelloWorld" with its actual value - that is, find class Resource and get value of its static property HelloWorld. Does Visual Studio expose any API to handle code model of the project? It definitely has one, because this is very similar to common task of renaming variables. I don't want to use reflection on output assembly, because it's slow and it locks the file for a while.

There is no straight forward way to do this that I know of. Reliably getting an AST out of Visual Studio (and changes to it) has always been a big problem. Part of the goal of the Rosalyn project is to create an unified way of doing this, because many tool windows had their own way of doing this sort of stuff.
There are four ways to do this:
Symbols
FileCodeModel + CodeDOM
Rosalyn AST
Unexplored Method
Symbols
I believe most tool windows such as the CodeView and things like Code Element Search use the symbols created from a compiled build. This is not ideal as it is a little more heavy weight and hard to keep in sync. You'd have to cache symbols to make this not slow. Using reflector, you can see how CodeView implements this.
This approach uses private assemblies. The code for getting the symbols would look something like this:
var compilerHost = new IDECompilerHost();
var typeEnumerator = (from compiler in compilerHost.Compilers.Cast<IDECompiler>()
from type in compiler.GetCompilation().MainAssembly.Types
select new Tuple<IDECompiler, CSharpType>(compiler, type));
foreach (var typeTuple in typeEnumerator)
{
Trace.WriteLine(typeTuple.Item2.Name);
var csType = typeTuple.Item2;
foreach (var loc in csType.SourceLocations)
{
var file = loc.FileName.Value;
var line = loc.Position.Line;
var charPos = loc.Position.Character;
}
}
FileCodeModel + CodeDOM
You could try using the EnvDTE service to get the FileCodeModel associated with a Code Document. This will let you get classes and methods. But it does not support getting the method body. You're messing with buggy COM. This ugly because an COM object reference to a CodeFunction or CodeClass can get invalided without you knowing it, meaning you'd have to keep your own mirror.
Rosalyn AST
This allows provides the same capabilities as both FileCodeModel and Symbols. I've been playing with this and it's actually not too bad.
Unexplored Method
You could try getting the underlying LanguageServiceProvider that is associated with the Code Document. But this is really difficult to pull off, and leaves you with many issues.

ReSharper: how to remove "Possible 'System.NullReferenceException'" warning

Here is a piece of code:
IUser user = managerUser.GetUserById(UserId);
if ( user==null )
throw new Exception(...);
Quote quote = new Quote(user.FullName, user.Email);
Everything is fine here. But if I replace "if" line with the following one:
ComponentException<MyUserManagerException>.FailIfTrue(user == null, "Can't find user with Id=" + UserId);
where function implementation is following:
public abstract class ComponentException<T> : ComponentException
where T : ComponentException, new()
{
public static void FailIfTrue(bool expression, string message)
{
if (expression)
{
T t = new T();
t.SetErrorMessage(message);
throw t;
}
}
//...
}
Then ReSharper generates me a warning: Possible 'System.NullReferenceException' pointing on 1st usage of 'user' object.
Q1. Why it generates such exception? As far as I see if user==null then exception will be generated and execution will never reach the usage point.
Q2. How to remove that warning? Please note:
1. I don't want to suppress this warning with comments (I will have a lot of similar pieces and don't want to transform my source code in 'commented garbage);
2. I don't want to changes ReSharper settings to change this problem from warning to 'suggestion' of 'hint'.
Thanks.
Any thoughts are welcome!
P.S. I am using resharper 5.1, MVSV 2008, C#

Resharper only looks at the current method for its analysis, and does not recursively analyse other methods you call.
You can however direct Resharper a bit and give it meta-information about certain methods. It knows for example about "Assert.IsNotNull(a)", and will take that information into account for the analysis. It is possible to make an external annotations file for Resharper and give it extra information about a certain library to make its analysis better. Maybe this might offer a way to solve your problem.
More information can be found here.
An example showing how it's used for the library Microsoft.Contracts can be found here.

A new answer in old post...
Here a little sample of my code regarding how to use CodeContract via ContractAnnotation with Resharper:
[ContractAnnotation("value:null=>true")]
public static bool IsNullOrEmpty(this string value)
{
return string.IsNullOrEmpty(value);
}
It is very simple...if u find the breadcrumb in the wood. You can check other cases too.
Have a nice day

Q1: Because Resharper doesn't do path analysing. It just sees a possible null reference and flags that.
Q2: You can't without doing either of what you provided already.

You do know (or expect) that this code will throw an exception if there is a null reference:
ComponentException<MyUserManagerException>.FailIfTrue([...]);
However, since there is no contract specifying this, ReSharper has to assume that this is just a normal method call which may return without throwing any exception in any case.
Make this method implement the ReSharper contract, or as a simple workaround (which only affects debug mode, therefore no performance penalty for release mode), just after the FailIfTrue call:
Debug.Assert(user != null);
That will get rid of the warning, and as an added bonus do a runtime check in debug mode to ensure that the condition assumed by you after calling FailIfTrue is indeed met.

This is caused by the Resharper engine. These "possible NullReferenceException" happen because someone (probably at Resharper) has declared/configured somewhere an annotation on the method.
Here is how it works: ReSharper NullReferenceException Analysis and Its Contracts
Unfortunately, sometimes, these useful annotation are just wrong.
When you detect an error, you should report it to JetBrains and they will update the annotations on the next release. They're used to this.
Meanwhile, you can try to fix it by yourself. Read the article for more :)

Please check if you have any user==null if check above the given code. If there is, then ReSharper thinks that the variable "can be null" so recommends you to use a check/assert before referencing it. In some cases, that's the only way ReSharper can guess whether a variable can or cannot be null.

Passing a lambda to a secondary AppDomain as a stream of IL and assembling it back using DynamicMethod

Is it possible to pass a lambda expression to a secondary AppDomain as a stream of IL bytes and then assemble it back there using DynamicMethod so it can be called?
I'm not too sure this is the right way to go in the first place, so here's the (detailed) reason I ask this question...
In my applications, there are a lot of cases when I need to load a couple of assemblies for reflection, so I can determine what to do with them next. The problem part is I need to be able to unload the assemblies after I'm finished reflecting over them. This means I need to load them using another AppDomain.
Now, most of my cases are sort of similar, except not quite. For example, sometimes I need to return a simple confirmation, other times I need to serialize a resource stream from the assembly, and again other times I need to make a callback or two.
So I end up writing the same semi-complicated temporary AppDomain creation code over and over again and implementing custom MarshalByRefObject proxies to communicate between the new domain and the original one.
As this is not really acceptable anymore, I decided to code me an AssemblyReflector class that could be used this way:
using (var reflector = new AssemblyReflector(#"C:\MyAssembly.dll"))
{
bool isMyAssembly = reflector.Execute(assembly =>
{
return assembly.GetType("MyAssembly.MyType") != null;
});
}
AssemblyReflector would automize the AppDomain unloading by virtue of IDisposable, and allow me to execute a Func<Assembly,object>-type lambda holding the reflection code in another AppDomain transparently.
The problem is, lambdas cannot be passed to other domains so simply. So after searching around, I found what looks like a way to do just that: pass the lambda to the new AppDomain as an IL stream - and that brings me to the original question.
Here's what I tried, but didn't work (the problem was BadImageFormatException being thrown when trying to call the new delegate):
public delegate object AssemblyReflectorDelegate(Assembly reflectedAssembly);
public class AssemblyReflector : IDisposable
{
private AppDomain _domain;
private string _assemblyFile;
public AssemblyReflector(string fileName) { ... }
public void Dispose() { ... }
public object Execute(AssemblyReflectorDelegate reflector)
{
var body = reflector.Method.GetMethodBody();
_domain.SetData("IL", body.GetILAsByteArray());
_domain.SetData("MaxStackSize", body.MaxStackSize);
_domain.SetData("FileName", _assemblyFile);
_domain.DoCallBack(() =>
{
var il = (byte[])AppDomain.CurrentDomain.GetData("IL");
var stack = (int)AppDomain.CurrentDomain.GetData("MaxStackSize");
var fileName = (string)AppDomain.CurrentDomain.GetData("FileName");
var args = Assembly.ReflectionOnlyLoadFrom(fileName);
var pars = new Type[] { typeof(Assembly) };
var dm = new DynamicMethod("", typeof(object), pars,
typeof(string).Module);
dm.GetDynamicILInfo().SetCode(il, stack);
var clone = (AssemblyReflectorDelegate)dm.CreateDelegate(
typeof(AssemblyReflectorDelegate));
var result = clone(args); // <-- BadImageFormatException thrown.
AppDomain.CurrentDomain.SetData("Result", result);
});
// Result obviously needs to be serializable for this to work.
return _domain.GetData("Result");
}
}
Am I even close (what's missing?), or is this a pointless excercise all in all?
NOTE: I realize that if this worked, I'd still have to be carefull about what I put into lambda in regard to references. That's not a problem, though.
UPDATE: I managed to get a little further. It seems that simply calling SetCode(...) is not nearly enough to reconstruct the method. Here's what's needed:
// Build a method signature. Since we know which delegate this is, this simply
// means adding its argument types together.
var builder = SignatureHelper.GetLocalVarSigHelper();
builder.AddArgument(typeof(Assembly), false);
var signature = builder.GetSignature();
// This is the tricky part... See explanation below.
di.SetCode(ILTokenResolver.Resolve(il, di, module), stack);
dm.InitLocals = initLocals; // Value gotten from original method's MethodInfo.
di.SetLocalSignature(signature);
The trick is as follows. Original IL contains certain metadata tokens which are valid only in the context of the original method. I needed to parse the IL and replace those tokens with ones that are valid in the new context. I did this by using a special class, ILTokenResolver, which I adapted from these two sources: Drew Wilson and Haibo Luo.
There is still a small problem with this - the new IL doesn't seem to be exactly valid. Depending on the exact contents of the lambda, it may or may not throw an InvalidProgramException at runtime.
As a simple example, this works:
reflector.Execute(a => { return 5; });
while this doesn't:
reflector.Execute(a => { int a = 5; return a; });
There are also more complex examples that are either working or not, depending on some yet-to-be-determined difference. It could be I missed some small but important detail. But I'm reasonably confident I'll find it after a more detailed comparison of the ildasm outputs. I'll post my findings here, when I do.
EDIT: Oh, man. I completely forgot this question was still open. But as it probably became obvious in itself, I gave up on solving this. I'm not happy about it, that's for sure. It's really a shame, but I guess I'll wait for better support from the framework and/or CLR before I attempt this again. There're just to many hacks one has to do to make this work, and even then it's not reliable. Apologies to everyone interested.

I didn't get exactly what is the problem you are trying to solve, but I made a component in the past that may solve it.
Basically, its purpose was to generate a Lambda Expression from a string. It uses a separate AppDomain to run the CodeDOM compiler. The IL of a compiled method is serialized to the original AppDomain, and then rebuild to a delegate with DynamicMethod. Then, the delegate is called and an lambda expression is returned.
I posted a full explanation of it on my blog. Naturally, it's open source. So, if you get to use it, please send me any feedback you think is reasonable.

Probably not, because a lambda is more than just an expression in source code. lambda expressions also create closures which capture/hoist variables into their own hidden classes. The program is modified by the compiler so everywhere you use those variables you're actually talking to the class. So you'd have to not only pass the code for the lambda, but also any changes to closure variables over time.