Develop Reference

UWP debug/release error with ntdll.dll

I'm writing a UWP program to detect colors from LEDs, this program runs on a Raspberry Pi 3 with Windows 10 IoT with attached display.
What the program does is take a reference image with the LED turned off, then take a image from the LED turned on.
Both images are converted to the same pixelformat and then are cropped to a smaller size, in which only the LED is shown (both the reference and the lighted LED).
Then those picture parts are converted to grayscale wich is followed by creating a difference picture of the two, so that only pixels that changed from the reference to the lighted LED are shown.
To do so I use the NuGet-Package portable.AForge.imaging. The code is shown below.
LEDBildNeu = LEDBild.Clone(PixelFormat.Format24bppRgb);
ReferenzbildNeu = Referenzbild.Clone(PixelFormat.Format24bppRgb);
Crop cropping = new Crop(new System.Drawing.Rectangle(Convert.ToInt32(x), Convert.ToInt32(y), 100, 100));
CroppedLED = cropping.Apply(LEDBildNeu);
CroppedReferenz = cropping.Apply(ReferenzbildNeu);
Grayscale grayscale = new Grayscale(0.2125, 0.7154, 0.0721);
GrayscaleReferenz = grayscale.Apply(CroppedReferenz);
GrayscaleLED = grayscale.Apply(CroppedLED);
Difference difference = new Difference(GrayscaleReferenz);
Differenzbild = difference.Apply(GrayscaleLED);
This code works fine as long as im in debug mode, all of the functions are working.
However when i change to release mode, i get this error while building:
1>C:\Users\morsch.nuget\packages\microsoft.net.native.compiler\1.7.2\tools\Microsoft.NetNative.targets(697,5): warning : MCG : warning MCG0007: Unresolved P/Invoke method 'ntdll.dll!memcpy' for method 'System.Byte* AForge.SystemTools.memcpy(System.Byte*, System.Byte*, System.Int32)'. Calling this method would throw exception at runtime. Please make sure the P/Invoke either points to a Windows API allowed in UWP applications, or a native DLL that is part of the package. If for some reason your P/Invoke does not satisify those requirements, please use [DllImport(ExactSpelling=true) to indicate that you understand the implications of using non-UWP APIs.
1>C:\Users\morsch.nuget\packages\microsoft.net.native.compiler\1.7.2\tools\Microsoft.NetNative.targets(697,5): warning : MCG : warning MCG0007: Unresolved P/Invoke method 'ntdll.dll!memset' for method 'System.Byte* AForge.SystemTools.memset(System.Byte*, System.Int32, System.Int32)'. Calling this method would throw exception at runtime. Please make sure the P/Invoke either points to a Windows API allowed in UWP applications, or a native DLL that is part of the package. If for some reason your P/Invoke does not satisify those requirements, please use [DllImport(ExactSpelling=true) to indicate that you understand the implications of using non-UWP APIs.
When I run the code in release mode and get to the part where the difference picture is created, I get the exception
System.TypeLoadException: 'Unresolved P/Invoke method 'memcpy!ntdll.dll' from this method. Please look for this method in build warnings for more details.'
According to this 'memset' and 'memcpy' are not supported by UWP. My questions now are:
Why does the program run in debug mode without any problems even when those two entry points are not supported, but as soon as i turn to release mode i get the exceptions?
Is there a workaround for the problem?
I already tried to use
[DllImport("ntdll.dll", EntryPoint = "memset")]
and
[DllImport("ntdll.dll", EntryPoint = "memcpy")]
But either I did it wrong or it just don't work that way.
I know I could just program a workaround in which I check the pixels manually and create a new image, but I wanted to solve that problem if possible.

Finding the correct combination of directives can be a very frustrating and time consuming process. Here is additional information that I received from Microsoft via email, hope this helps:
Helpful links:
https://devblogs.microsoft.com/dotnet/net-native-deep-dive-dynamic-features-in-static-code/
https://learn.microsoft.com/en-us/dotnet/framework/net-native/runtime-directives-rd-xml-configuration-file-reference
https://learn.microsoft.com/en-us/dotnet/framework/net-native/runtime-directive-policy-settings
The analysis we do to get your application ready to be ahead of time compiled is quite extensive. We need to generate code for various generic types, reflection callable wrappers, serialization information, marshalling stubs etc etc. In come cases (as you could imagine) we end up generating more than is strictly necessary due to run away combinatorics. It’s completely possible that some fiddling with our heuristics can get you application to a place where it compiles without any loss of functionality.
Practically speaking, there’s two ways to manipulate the behavior of the compiler. One is through some of our compiler flags available through dropping elements into your csproj. The other is making edits to your applications Properties\Default.rd.xml file.
Compiler flags
There are a wide range of flags available but here’s a couple that may help out:
<ShortcutGenericAnalysis>true</ShortcutGenericAnalysis> - Can help stop runaway analysis of generic types and reduce overall generation requirements.
<UseDotNetNativeSharedAssemblyFrameworkPackage>false</UseDotNetNativeSharedAssemblyFrameworkPackage> - Eliminates one of the linking boundaries the compiler has to fight with. I actually suspect turning this off will make things worse not better but whole program optimizers are hard to reason about but rebuilds are cheap enough to try.
Runtime Directives
There’s lots of reading above but the tl;dr is that this file is read by the compiler and can contain lots of hints about what we want it to do or ignore etc. The overall syntax of the file is also included in the reading above but I don’t think we’re very clear about the one special directive that’s include by default:
<Assembly Name="*Application*" Dynamic="Required All" />
This directive says: “Please save/generate enough information so that all user types can be inspected and created via reflection.” Where ‘user types’ means any type in an assembly that isn’t signed with the .NET key token. So, basically everything that isn’t explicitly .NET Framework. This in leads to lots of bloat but also makes it so most folks don’t ever have to think about these things. In cases where we don’t have enough information, you’ll get runtime exceptions like MissingMetadataException or TypeLoadException or NullReferenceException. Each instance will require a bit of code inspection and fiddling with directives to get patched up. This can be an annoying a fragile process. All that said, the analysis engine is quite sophisticated and you’ll get lots and lots of things ‘for free’ without the special directive or any hassle. It’s entirely possible that your app runs great with just a little bit of tweaking.
Okay, the goal now is to remove this directive but still have a working application. There’s two approaches that have tradeoffs, so I’ll describe both and let you decide if either methodology suits you. Roughly here’s what the two workflows look like:
Start from nothing.
a. Remove the special Application directive
b. Build the app
c. If the build fails, email us, else…
d. Test the app and see if you hit any runtime errors
e. If you do you’ll need to look at the error location and see if adding some directives can help then head back to (b).
f. If you find no errors, you’re done! Hooray!
Start from everything
a. Remove the special Application directive
b. Get a list of the full set of dlls for your project, for example by inspecting here: obj[architecture]\Release\ilc\in
c. For each dll, add a Dynamic directive. They’ll look like: <Assembly Name="ASSEMBLYNAMEWITHOUTEXTENTION" Dynamic="Required All"/>
d. Comment out some subset of these libraries
e. Build the app
f. If the build fails again in RHBIND go to (d)
g. Test the app and see if you hit any runtime errors
h. If you do you’ll need to look at the error location and see if adding some directives can help then head back to (e)
i. If you find no errors, you’re done! Hooray!

I found a solution which worked:
Instead of downloading the portable.AForge package with NuGet i downloaded the portable.AForge from GitHub.
Find the .cs-file called SystemTools.cs (located in AForge/Sources/Core/).
Open it with any .cs editing porgram, now search for all code like
#if !MONO
...
#else
and remove it.
This clears the use of memcpu() or memset() from ntdll.dll.
Save the SystemTools.cs, create the library and add the AForge-Package manually to the application.
After the change it worked without any problems.

Hash of source codes at compile time in C#

Having a server that other devs use, I currently log the version of the dll they use. I do that by having the client that use Reflection to retrieve its version:
Assembly.GetEntryAssembly().GetName().Version.ToString();
It's nice, but since it come from dev that uses TFS and do themself the build, I can not see if they have the latest version of the sources. Is there a trick, like a compilation tag, that would easily allow a hash of the generating source code?
Note: I have try to send the MD5 of the dll (using assembly.Location), but it is useless since the hash value changes between 2 compilations (I suppose there is some compilation timestamp inside the generated dll).

This is most collaboraton issue then a coding.
In the moment that you find out that the version is old one.notify them about it.
If the real version is not old one, that means that developers before making buold did not increment the version ID, which is mistake.
In other words, ordanize it among people, and not relly on these kind of tools (if there is any). You trying to create a complicated tool, that will help you avoid mistakes, but humans will find a way to make them again.
So it's better to create solid relation structure among you, imo.

Create a tool on pre build event to hash/last-write-time your code files.
Write the result to a cs file or a embedded resource file.
The result file must exclude in above action.
For prevent skip build (up-to-date) feature not work,Compare the file before write.
And if youre opening the file in IDE will get a prompt `changed from out side' when build.

Seem there is no easy way to do it.

Help me understand Resharper background compilation

So Jeff Atwood rightly complained about Visual Studio not performing background compilation see: http://www.codinghorror.com/blog/2007/05/c-and-the-compilation-tax.html
The solution from most sources seems to be Reshaper which will incrementally perform background compilation as you write. This leads to their great realtime re-factoring tips and error detection.
But what I don't understand is with R# continually compiling my code, why does it take so long when executing a compilation via VS (i.e. Ctrl + Shift + B or similar). What I mean by this is, if R# has already compiled my code then why would I need a recompilation?
My assumption is of course that R# is not overriding the assemblies in my bin directories but instead holding the compilation results in memory. In which case, is it possible to tell R# to simply override my assemblies when compilation is successful?

I don't know about "rightly complained" - that's an opinion I happen to disagree with:)
However, the VB.NET (and probably Resharper c#) background compilers do not actually compile full assemblies - they cannot! If you think about it, the natural state of your code while you are working is not compilable! Almost every keystroke puts your code in an invalid state. Think of this line:
var x = new Something();
As you type this, from the key "v" to the key ")", your code is "wrong". Or what if you are referencing a method you haven't defined yet? And if this code is in an assembly that another assembly requires, how would you compile that second assembly at all, background or not?
The background compilers get around this by compiling small chunks of your code into multiple transient "assemblies" that are actually just metadata holders - really, they don't care about the actual effects of the code as much as the symbols defined, used, etc. When you finally hit build, the actual full assemblies still need to be built.
So no, I don't believe it's possible because they're not built to do actual full compilation - they are built to check your code and interpret symbols on the fly.

Reshaper which will incrementally perform background compilation as you write
It doesn't, it just parses the source code. The exact same thing Visual Studio already does if you don't have Resharper, that's how it implements IntelliSense, its own refactoring features and commands like GoTo Definition and Find All References. Visual Studio also parses in the background, updating its data while you type. Resharper just implements more bells and whistles with that parsing data.
Going from parsing the code to actually generating the assembly is a pretty major step. The internal format of an assembly is too convoluted to allow this to happen in the background without affecting the responsiveness of the machine.
And the C# compiler is still a large chunk of unmanaged C++ code that is independent from the IDE. An inevitable consequence of having to have the compiler first. It is however a stated goal for the next version of C# to provide compile-on-demand services. Getting true background compilation is a possibility.

I don't really have an answer but I just wanted to say that I have been using Eclipse and Java for 4 months now and I love the automatic compilation. I have a very large java code base and compilation happens constantly as I save code changes. When I hit Run everything is ready to go! It's just awesome. It also deploys to the local web server instance (Tomcat in my case) automatically as I make code changes. All this is setup by default in Eclipse.
I hope Microsoft does something similar with .net in the near future.

How to decomplie / compile dll files?

I have an app written in C#. It was already compiled.
There are plenty of dll files there and i could not open them , decompile them cannot see and access C# code which build them, when i add a reference to dll files in Visual Studio i only can see what methods are there, but i cannot see / access code of those methods.
Is there a tool or is there a way how to see / access that code , modify it and then recompile it ?
Please if you have an idea give me a hand !
Thanks

The easiest way to do this is through Reflector - it is not perfect, but will give you a good idea.
It lets select an assembly and display it in either IL, vb.net or C#. You can copy/paste the result into Visual Studio and compile it with your changes.
Because it works on the raw optimized IL, things like foreach loops can end up as labels and gotos and variables names can end up as invalid C#/vb.net variables names (though they are valid IL).
I have used it in the past, and the biggest issue is in reconstructing foreach loops, though the pattern becomes apparent after several of those, and the variable names can be changed with a search/replace.

What happens if we add lines to IL code and add breakpoints to our program?

If I add let's say 1 line at the beggining of a method, if I set a breakpoint through Visual Studio on the first line, will it point to the first line or the second? If it will flag the wrong line, is there anything we could do when editing .exe files to ensure a regular debugging session later?
Isn't there something like setting line x to be Y? I remember seeing something like that somewhere, not sure if .NET related or not.

You'll need to update the debugging symbols in the PDB file if you want the debugging experience to remain unchanged.
The best option for this I've seen is to use Mono.Cecil, as it supports (limited) modification of the debugging symbols as well as the IL.

If you are modifying IL then the PDB files will contain outdated information. Note that there probably won't be a 1:1 between changes in IL lines to the C# line #s (eg inserting 3 IL statements won't offset the IDE breakpoint by 3 lines of C#).
You may want to separate the IL-modified portions of your code into separate methods to minimize the impact. Also, assuming you are the one doing the IL modification, you may find it convenient to switch between the C# & IL views while debugging.
You might need to muck a bit with the generated code to facilitate that. For example, if the injected IL can be in a wrapper method then you could tell the debugger to ignore it by usage of attrbiutes such as DebuggerStepThroughAttribute, DebuggerNonUserCodeAttribute, or DebuggerHiddenAttribute. (Look up the documentation for the nuances in behavior)

I expect that you get flagged as .pdb file might not match.

How are you adding the IL? If you are doing this via profiler instrumentation (SetILFunctionBody) then you need to supply a new IL code map as well (SetILInstrumentationCodeMap) so that the debugger becomes aware of the IL modification.