Develop Reference

Allow Compiling With Errors

I'm used to being able to compile source code even if it has errors due to my Java (Eclipse) background. This is a tremendous help when developing test-first, which has become my modus operandi. Also it helps a lot with refactoring.
Is there any way to achieve something similar in C#? I have to use JetBrain's Rider, but I don't care much for the IDE.

The source code could not compiled when you violate the rules of writing syntax are known as Compile-Time errors.
Even in Java the source code could not compiled when incorrect syntax such as a missing semicolon at the end of a statement or a missing bracket, etc.

UWP debug/release error with ntdll.dll

I'm writing a UWP program to detect colors from LEDs, this program runs on a Raspberry Pi 3 with Windows 10 IoT with attached display.
What the program does is take a reference image with the LED turned off, then take a image from the LED turned on.
Both images are converted to the same pixelformat and then are cropped to a smaller size, in which only the LED is shown (both the reference and the lighted LED).
Then those picture parts are converted to grayscale wich is followed by creating a difference picture of the two, so that only pixels that changed from the reference to the lighted LED are shown.
To do so I use the NuGet-Package portable.AForge.imaging. The code is shown below.
LEDBildNeu = LEDBild.Clone(PixelFormat.Format24bppRgb);
ReferenzbildNeu = Referenzbild.Clone(PixelFormat.Format24bppRgb);
Crop cropping = new Crop(new System.Drawing.Rectangle(Convert.ToInt32(x), Convert.ToInt32(y), 100, 100));
CroppedLED = cropping.Apply(LEDBildNeu);
CroppedReferenz = cropping.Apply(ReferenzbildNeu);
Grayscale grayscale = new Grayscale(0.2125, 0.7154, 0.0721);
GrayscaleReferenz = grayscale.Apply(CroppedReferenz);
GrayscaleLED = grayscale.Apply(CroppedLED);
Difference difference = new Difference(GrayscaleReferenz);
Differenzbild = difference.Apply(GrayscaleLED);
This code works fine as long as im in debug mode, all of the functions are working.
However when i change to release mode, i get this error while building:
1>C:\Users\morsch.nuget\packages\microsoft.net.native.compiler\1.7.2\tools\Microsoft.NetNative.targets(697,5): warning : MCG : warning MCG0007: Unresolved P/Invoke method 'ntdll.dll!memcpy' for method 'System.Byte* AForge.SystemTools.memcpy(System.Byte*, System.Byte*, System.Int32)'. Calling this method would throw exception at runtime. Please make sure the P/Invoke either points to a Windows API allowed in UWP applications, or a native DLL that is part of the package. If for some reason your P/Invoke does not satisify those requirements, please use [DllImport(ExactSpelling=true) to indicate that you understand the implications of using non-UWP APIs.
1>C:\Users\morsch.nuget\packages\microsoft.net.native.compiler\1.7.2\tools\Microsoft.NetNative.targets(697,5): warning : MCG : warning MCG0007: Unresolved P/Invoke method 'ntdll.dll!memset' for method 'System.Byte* AForge.SystemTools.memset(System.Byte*, System.Int32, System.Int32)'. Calling this method would throw exception at runtime. Please make sure the P/Invoke either points to a Windows API allowed in UWP applications, or a native DLL that is part of the package. If for some reason your P/Invoke does not satisify those requirements, please use [DllImport(ExactSpelling=true) to indicate that you understand the implications of using non-UWP APIs.
When I run the code in release mode and get to the part where the difference picture is created, I get the exception
System.TypeLoadException: 'Unresolved P/Invoke method 'memcpy!ntdll.dll' from this method. Please look for this method in build warnings for more details.'
According to this 'memset' and 'memcpy' are not supported by UWP. My questions now are:
Why does the program run in debug mode without any problems even when those two entry points are not supported, but as soon as i turn to release mode i get the exceptions?
Is there a workaround for the problem?
I already tried to use
[DllImport("ntdll.dll", EntryPoint = "memset")]
and
[DllImport("ntdll.dll", EntryPoint = "memcpy")]
But either I did it wrong or it just don't work that way.
I know I could just program a workaround in which I check the pixels manually and create a new image, but I wanted to solve that problem if possible.

Finding the correct combination of directives can be a very frustrating and time consuming process. Here is additional information that I received from Microsoft via email, hope this helps:
Helpful links:
https://devblogs.microsoft.com/dotnet/net-native-deep-dive-dynamic-features-in-static-code/
https://learn.microsoft.com/en-us/dotnet/framework/net-native/runtime-directives-rd-xml-configuration-file-reference
https://learn.microsoft.com/en-us/dotnet/framework/net-native/runtime-directive-policy-settings
The analysis we do to get your application ready to be ahead of time compiled is quite extensive. We need to generate code for various generic types, reflection callable wrappers, serialization information, marshalling stubs etc etc. In come cases (as you could imagine) we end up generating more than is strictly necessary due to run away combinatorics. It’s completely possible that some fiddling with our heuristics can get you application to a place where it compiles without any loss of functionality.
Practically speaking, there’s two ways to manipulate the behavior of the compiler. One is through some of our compiler flags available through dropping elements into your csproj. The other is making edits to your applications Properties\Default.rd.xml file.
Compiler flags
There are a wide range of flags available but here’s a couple that may help out:
<ShortcutGenericAnalysis>true</ShortcutGenericAnalysis> - Can help stop runaway analysis of generic types and reduce overall generation requirements.
<UseDotNetNativeSharedAssemblyFrameworkPackage>false</UseDotNetNativeSharedAssemblyFrameworkPackage> - Eliminates one of the linking boundaries the compiler has to fight with. I actually suspect turning this off will make things worse not better but whole program optimizers are hard to reason about but rebuilds are cheap enough to try.
Runtime Directives
There’s lots of reading above but the tl;dr is that this file is read by the compiler and can contain lots of hints about what we want it to do or ignore etc. The overall syntax of the file is also included in the reading above but I don’t think we’re very clear about the one special directive that’s include by default:
<Assembly Name="*Application*" Dynamic="Required All" />
This directive says: “Please save/generate enough information so that all user types can be inspected and created via reflection.” Where ‘user types’ means any type in an assembly that isn’t signed with the .NET key token. So, basically everything that isn’t explicitly .NET Framework. This in leads to lots of bloat but also makes it so most folks don’t ever have to think about these things. In cases where we don’t have enough information, you’ll get runtime exceptions like MissingMetadataException or TypeLoadException or NullReferenceException. Each instance will require a bit of code inspection and fiddling with directives to get patched up. This can be an annoying a fragile process. All that said, the analysis engine is quite sophisticated and you’ll get lots and lots of things ‘for free’ without the special directive or any hassle. It’s entirely possible that your app runs great with just a little bit of tweaking.
Okay, the goal now is to remove this directive but still have a working application. There’s two approaches that have tradeoffs, so I’ll describe both and let you decide if either methodology suits you. Roughly here’s what the two workflows look like:
Start from nothing.
a. Remove the special Application directive
b. Build the app
c. If the build fails, email us, else…
d. Test the app and see if you hit any runtime errors
e. If you do you’ll need to look at the error location and see if adding some directives can help then head back to (b).
f. If you find no errors, you’re done! Hooray!
Start from everything
a. Remove the special Application directive
b. Get a list of the full set of dlls for your project, for example by inspecting here: obj[architecture]\Release\ilc\in
c. For each dll, add a Dynamic directive. They’ll look like: <Assembly Name="ASSEMBLYNAMEWITHOUTEXTENTION" Dynamic="Required All"/>
d. Comment out some subset of these libraries
e. Build the app
f. If the build fails again in RHBIND go to (d)
g. Test the app and see if you hit any runtime errors
h. If you do you’ll need to look at the error location and see if adding some directives can help then head back to (e)
i. If you find no errors, you’re done! Hooray!

I found a solution which worked:
Instead of downloading the portable.AForge package with NuGet i downloaded the portable.AForge from GitHub.
Find the .cs-file called SystemTools.cs (located in AForge/Sources/Core/).
Open it with any .cs editing porgram, now search for all code like
#if !MONO
...
#else
and remove it.
This clears the use of memcpu() or memset() from ntdll.dll.
Save the SystemTools.cs, create the library and add the AForge-Package manually to the application.
After the change it worked without any problems.

Does Java have 'Debug' and 'Release' build mode like C#?

In C#, we have 2 modes to build projects : Debug and Release, I wonder if Java has the same thing. I am using IntelliJ IDEA as Java IDE and so far I haven't seen anywhere to configure a build mode like in VS IDE.

javac
-g Generate all debugging info
-g:none Generate no debugging info
-g:{lines,vars,source} Generate only some debugging info
You can choose to include debug symbols in the compiled classes (this is the default) or to not do so. There is not much benefit to not doing that. The jar files will be a little smaller, but the performance benefit is minimal (if any). Without these symbols you no longer get line numbers in stack traces. You also have the option to include additional symbols with local variable names (by default there are only source file names and line numbers).
java
-ea[:<packagename>...|:<classname>]
-enableassertions[:<packagename>...|:<classname>]
enable assertions
You can also enable assertions at run-time (default is off), which is sometimes useful during development and testing. This does have a performance impact (if the code in question did indeed make use of assertions, which I think is uncommon).
Regardless of any of these settings, the JVM always allows you to attach a debugger.
What Java does not have is conditional compilation where completely different code would be compiled based on some external setting. The closest you can get is something like public static final boolean DEBUG_BUILD = true; somewhere in your code and use that in if statements. This will actually make the compiler exclude code that becomes unreachable, but you have to set this constant in the source code.

It is normal practice in Java to release everything is a manner which can be debugged. For some projects requiring obfuscation, they could have a release build, but I have never seen this in 12 years of developing Java.
Things such as assertions and debug messages are usually turned off at runtime for a production instance but can be turned on at any time (even dynamically) if required.
IMHO it is best practice to use the same build in every environment, not just the same source but the same JARs. This gives you the best chance that, if it works in test, it will work in production and if you have a problem in production, you can re-produce it in test.
As so much Java code is written this way, the JIT is very good at optimising dead code which is never called. So much so that IMHO most of the micro-"benchmarks" where Java out performs C++, is when the benchmark doesn't do any thing and the JIT is better at detecting this. IMHO, C++ assumes the developer is smart enough not to write code which doesn't do anything.

You're asking for different kinds of builds to compile in different things I guess. For example to have Debug.WriteLine and Console.WriteLine.
"No, Java doesn't have an exact match for that functionality. You could use aspects, or use an IOC container to inject different implementation classes." stole this from the following question: Conditional Java compilation
(there're other nice answers for you there)

Help me understand Resharper background compilation

So Jeff Atwood rightly complained about Visual Studio not performing background compilation see: http://www.codinghorror.com/blog/2007/05/c-and-the-compilation-tax.html
The solution from most sources seems to be Reshaper which will incrementally perform background compilation as you write. This leads to their great realtime re-factoring tips and error detection.
But what I don't understand is with R# continually compiling my code, why does it take so long when executing a compilation via VS (i.e. Ctrl + Shift + B or similar). What I mean by this is, if R# has already compiled my code then why would I need a recompilation?
My assumption is of course that R# is not overriding the assemblies in my bin directories but instead holding the compilation results in memory. In which case, is it possible to tell R# to simply override my assemblies when compilation is successful?

I don't know about "rightly complained" - that's an opinion I happen to disagree with:)
However, the VB.NET (and probably Resharper c#) background compilers do not actually compile full assemblies - they cannot! If you think about it, the natural state of your code while you are working is not compilable! Almost every keystroke puts your code in an invalid state. Think of this line:
var x = new Something();
As you type this, from the key "v" to the key ")", your code is "wrong". Or what if you are referencing a method you haven't defined yet? And if this code is in an assembly that another assembly requires, how would you compile that second assembly at all, background or not?
The background compilers get around this by compiling small chunks of your code into multiple transient "assemblies" that are actually just metadata holders - really, they don't care about the actual effects of the code as much as the symbols defined, used, etc. When you finally hit build, the actual full assemblies still need to be built.
So no, I don't believe it's possible because they're not built to do actual full compilation - they are built to check your code and interpret symbols on the fly.

Reshaper which will incrementally perform background compilation as you write
It doesn't, it just parses the source code. The exact same thing Visual Studio already does if you don't have Resharper, that's how it implements IntelliSense, its own refactoring features and commands like GoTo Definition and Find All References. Visual Studio also parses in the background, updating its data while you type. Resharper just implements more bells and whistles with that parsing data.
Going from parsing the code to actually generating the assembly is a pretty major step. The internal format of an assembly is too convoluted to allow this to happen in the background without affecting the responsiveness of the machine.
And the C# compiler is still a large chunk of unmanaged C++ code that is independent from the IDE. An inevitable consequence of having to have the compiler first. It is however a stated goal for the next version of C# to provide compile-on-demand services. Getting true background compilation is a possibility.

I don't really have an answer but I just wanted to say that I have been using Eclipse and Java for 4 months now and I love the automatic compilation. I have a very large java code base and compilation happens constantly as I save code changes. When I hit Run everything is ready to go! It's just awesome. It also deploys to the local web server instance (Tomcat in my case) automatically as I make code changes. All this is setup by default in Eclipse.
I hope Microsoft does something similar with .net in the near future.

Using reflection for code gen?

I'm writing a console tool to generate some C# code for objects in a class library. The best/easiest way I can actual generate the code is to use reflection after the library has been built. It works great, but this seems like a haphazard approch at best. Since the generated code will be compiled with the library, after making a change I'll need to build the solution twice to get the final result, etc. Some of these issues could be mitigated with a build script, but it still feels like a bit too much of a hack to me.
My question is, are there any high-level best practices for this sort of thing?

Its pretty unclear what you are doing, but what does seem clear is that you have some base line code, and based on some its properties, you want to generate more code.
So the key issue here are, given the base line code, how do you extract interesting properties, and how do you generate code from those properties?
Reflection is a way to extract properties of code running (well, at least loaded) into the same execution enviroment as the reflection user code. The problem with reflection is it only provides a very limited set of properties, typically lists of classes, methods, or perhaps names of arguments. IF all the code generation you want to do can be done with just that, well, then reflection seems just fine. But if you want more detailed properties about the code, reflection won't cut it.
In fact, the only artifact from which truly arbitrary code properties can be extracted is the the source code as a character string (how else could you answer, is the number of characters between the add operator and T in middle of the variable name is a prime number?). As a practical matter, properties you can get from character strings are generally not very helpful (see the example I just gave :).
The compiler guys have spent the last 60 years figuring out how to extract interesting program properties and you'd be a complete idiot to ignore what they've learned in that half century.
They have settled on a number of relatively standard "compiler data structures": abstract syntax trees (ASTs), symbol tables (STs), control flow graphs (CFGs), data flow facts (DFFs), program triples, ponter analyses, etc.
If you want to analyze or generate code, your best bet is to process it first into such standard compiler data structures and then do the job. If you have ASTs, you can answer all kinds of question about what operators and operands are used. If you have STs, you can answer questions about where-defined, where-visible and what-type. If you have CFGs, you can answer questions about "this-before-that", "what conditions does statement X depend upon". If you have DFFs, you can determine which assignments affect the actions at a point in the code. Reflection will never provide this IMHO, because it will always be limited to what the runtime system developers are willing to keep around when running a program. (Maybe someday they'll keep all the compiler data structures around, but then it won't be reflection; it will just finally be compiler support).
Now, after you have determined the properties of interest, what do you do for code generation? Here the compiler guys have been so focused on generation of machine code that they don't offer standard answers. The guys that do are the program transformation community (http://en.wikipedia.org/wiki/Program_transformation). Here the idea is to keep at least one representation of your program as ASTs, and to provide special support for matching source code syntax (by constructing pattern-match ASTs from the code fragments of interest), and provide "rewrite" rules that say in effect, "when you see this pattern, then replace it by that pattern under this condition".
By connecting the condition to various property-extracting mechanisms from the compiler guys, you get relatively easy way to say what you want backed up by that 50 years of experience. Such program transformation systems have the ability to read in source code,
carry out analysis and transformations, and generally to regenerate code after transformation.
For your code generation task, you'd read in the base line code into ASTs, apply analyses to determine properties of interesting, use transformations to generate new ASTs, and then spit out the answer.
For such a system to be useful, it also has to be able to parse and prettyprint a wide variety of source code langauges, so that folks other than C# lovers can also have the benefits of code analysis and generation.
These ideas are all reified in the
DMS Software Reengineering Toolkit. DMS handles C, C++, C#, Java, COBOL, JavaScript, PHP, Verilog, ... and a lot of other langauges.
(I'm the architect of DMS, so I have a rather biased view. YMMV).

Have you considered using T4 templates for performing the code generation? It looks like it's getting much more publicity and attention now and more support in VS2010.
This tutorial seems database centric but it may give you some pointers: http://www.olegsych.com/2008/09/t4-tutorial-creatating-your-first-code-generator/ in addition there was a recent Hanselminutes on T4 here: http://www.hanselminutes.com/default.aspx?showID=170.
Edit: Another great place is the T4 tag here on StackOverflow: https://stackoverflow.com/questions/tagged/t4
EDIT: (By asker, new developments)
As of VS2012, T4 now supports reflection over an active project in a single step. This means you can make a change to your code, and the compiled output of the T4 template will reflect the newest version, without requiring you to perform a second reflect/build step. With this capability, I'm marking this as the accepted answer.

You may wish to use CodeDom, so that you only have to build once.
First, I would read this CodeProject article to make sure there are not language-specific features you'd be unable to support without using Reflection.

From what I understand, you could use something like Common Compiler Infrastructure (http://ccimetadata.codeplex.com/) to programatically analyze your existing c# source.
This looks pretty involved to me though, and CCI apparently only has full support for C# language spec 2. A better strategy may be to streamline your existing method instead.

I'm not sure of the best way to do this, but you could do this
As a post-build step on your base dll, run the code generator
As another post-build step, run csc or msbuild to build the generated dll
Other things which depend on the generated dll will also need to depend on the base dll, so the build order remains correct