I'm writing a UWP program to detect colors from LEDs, this program runs on a Raspberry Pi 3 with Windows 10 IoT with attached display.
What the program does is take a reference image with the LED turned off, then take a image from the LED turned on.
Both images are converted to the same pixelformat and then are cropped to a smaller size, in which only the LED is shown (both the reference and the lighted LED).
Then those picture parts are converted to grayscale wich is followed by creating a difference picture of the two, so that only pixels that changed from the reference to the lighted LED are shown.
To do so I use the NuGet-Package portable.AForge.imaging. The code is shown below.
LEDBildNeu = LEDBild.Clone(PixelFormat.Format24bppRgb);
ReferenzbildNeu = Referenzbild.Clone(PixelFormat.Format24bppRgb);
Crop cropping = new Crop(new System.Drawing.Rectangle(Convert.ToInt32(x), Convert.ToInt32(y), 100, 100));
CroppedLED = cropping.Apply(LEDBildNeu);
CroppedReferenz = cropping.Apply(ReferenzbildNeu);
Grayscale grayscale = new Grayscale(0.2125, 0.7154, 0.0721);
GrayscaleReferenz = grayscale.Apply(CroppedReferenz);
GrayscaleLED = grayscale.Apply(CroppedLED);
Difference difference = new Difference(GrayscaleReferenz);
Differenzbild = difference.Apply(GrayscaleLED);
This code works fine as long as im in debug mode, all of the functions are working.
However when i change to release mode, i get this error while building:
1>C:\Users\morsch.nuget\packages\microsoft.net.native.compiler\1.7.2\tools\Microsoft.NetNative.targets(697,5): warning : MCG : warning MCG0007: Unresolved P/Invoke method 'ntdll.dll!memcpy' for method 'System.Byte* AForge.SystemTools.memcpy(System.Byte*, System.Byte*, System.Int32)'. Calling this method would throw exception at runtime. Please make sure the P/Invoke either points to a Windows API allowed in UWP applications, or a native DLL that is part of the package. If for some reason your P/Invoke does not satisify those requirements, please use [DllImport(ExactSpelling=true) to indicate that you understand the implications of using non-UWP APIs.
1>C:\Users\morsch.nuget\packages\microsoft.net.native.compiler\1.7.2\tools\Microsoft.NetNative.targets(697,5): warning : MCG : warning MCG0007: Unresolved P/Invoke method 'ntdll.dll!memset' for method 'System.Byte* AForge.SystemTools.memset(System.Byte*, System.Int32, System.Int32)'. Calling this method would throw exception at runtime. Please make sure the P/Invoke either points to a Windows API allowed in UWP applications, or a native DLL that is part of the package. If for some reason your P/Invoke does not satisify those requirements, please use [DllImport(ExactSpelling=true) to indicate that you understand the implications of using non-UWP APIs.
When I run the code in release mode and get to the part where the difference picture is created, I get the exception
System.TypeLoadException: 'Unresolved P/Invoke method 'memcpy!ntdll.dll' from this method. Please look for this method in build warnings for more details.'
According to this 'memset' and 'memcpy' are not supported by UWP. My questions now are:
Why does the program run in debug mode without any problems even when those two entry points are not supported, but as soon as i turn to release mode i get the exceptions?
Is there a workaround for the problem?
I already tried to use
[DllImport("ntdll.dll", EntryPoint = "memset")]
and
[DllImport("ntdll.dll", EntryPoint = "memcpy")]
But either I did it wrong or it just don't work that way.
I know I could just program a workaround in which I check the pixels manually and create a new image, but I wanted to solve that problem if possible.
Finding the correct combination of directives can be a very frustrating and time consuming process. Here is additional information that I received from Microsoft via email, hope this helps:
Helpful links:
https://devblogs.microsoft.com/dotnet/net-native-deep-dive-dynamic-features-in-static-code/
https://learn.microsoft.com/en-us/dotnet/framework/net-native/runtime-directives-rd-xml-configuration-file-reference
https://learn.microsoft.com/en-us/dotnet/framework/net-native/runtime-directive-policy-settings
The analysis we do to get your application ready to be ahead of time compiled is quite extensive. We need to generate code for various generic types, reflection callable wrappers, serialization information, marshalling stubs etc etc. In come cases (as you could imagine) we end up generating more than is strictly necessary due to run away combinatorics. It’s completely possible that some fiddling with our heuristics can get you application to a place where it compiles without any loss of functionality.
Practically speaking, there’s two ways to manipulate the behavior of the compiler. One is through some of our compiler flags available through dropping elements into your csproj. The other is making edits to your applications Properties\Default.rd.xml file.
Compiler flags
There are a wide range of flags available but here’s a couple that may help out:
<ShortcutGenericAnalysis>true</ShortcutGenericAnalysis> - Can help stop runaway analysis of generic types and reduce overall generation requirements.
<UseDotNetNativeSharedAssemblyFrameworkPackage>false</UseDotNetNativeSharedAssemblyFrameworkPackage> - Eliminates one of the linking boundaries the compiler has to fight with. I actually suspect turning this off will make things worse not better but whole program optimizers are hard to reason about but rebuilds are cheap enough to try.
Runtime Directives
There’s lots of reading above but the tl;dr is that this file is read by the compiler and can contain lots of hints about what we want it to do or ignore etc. The overall syntax of the file is also included in the reading above but I don’t think we’re very clear about the one special directive that’s include by default:
<Assembly Name="*Application*" Dynamic="Required All" />
This directive says: “Please save/generate enough information so that all user types can be inspected and created via reflection.” Where ‘user types’ means any type in an assembly that isn’t signed with the .NET key token. So, basically everything that isn’t explicitly .NET Framework. This in leads to lots of bloat but also makes it so most folks don’t ever have to think about these things. In cases where we don’t have enough information, you’ll get runtime exceptions like MissingMetadataException or TypeLoadException or NullReferenceException. Each instance will require a bit of code inspection and fiddling with directives to get patched up. This can be an annoying a fragile process. All that said, the analysis engine is quite sophisticated and you’ll get lots and lots of things ‘for free’ without the special directive or any hassle. It’s entirely possible that your app runs great with just a little bit of tweaking.
Okay, the goal now is to remove this directive but still have a working application. There’s two approaches that have tradeoffs, so I’ll describe both and let you decide if either methodology suits you. Roughly here’s what the two workflows look like:
Start from nothing.
a. Remove the special Application directive
b. Build the app
c. If the build fails, email us, else…
d. Test the app and see if you hit any runtime errors
e. If you do you’ll need to look at the error location and see if adding some directives can help then head back to (b).
f. If you find no errors, you’re done! Hooray!
Start from everything
a. Remove the special Application directive
b. Get a list of the full set of dlls for your project, for example by inspecting here: obj[architecture]\Release\ilc\in
c. For each dll, add a Dynamic directive. They’ll look like: <Assembly Name="ASSEMBLYNAMEWITHOUTEXTENTION" Dynamic="Required All"/>
d. Comment out some subset of these libraries
e. Build the app
f. If the build fails again in RHBIND go to (d)
g. Test the app and see if you hit any runtime errors
h. If you do you’ll need to look at the error location and see if adding some directives can help then head back to (e)
i. If you find no errors, you’re done! Hooray!
I found a solution which worked:
Instead of downloading the portable.AForge package with NuGet i downloaded the portable.AForge from GitHub.
Find the .cs-file called SystemTools.cs (located in AForge/Sources/Core/).
Open it with any .cs editing porgram, now search for all code like
#if !MONO
...
#else
and remove it.
This clears the use of memcpu() or memset() from ntdll.dll.
Save the SystemTools.cs, create the library and add the AForge-Package manually to the application.
After the change it worked without any problems.
Related
I will briefly tell you the situation.
I got a C# project which uses some DLL created in C++.
Now, separately, I also have a C++ project, which was used to create that DLL some time ago.
Now, I wanted to debug the C++ DLL during running the C# project.
I enabled "Enable Unmanaged Code Debugging" in my C# project.
I started debugging C# project and stepping into some functions alongside.
All seemed to be ok. When I reached a function which belonged to C++ DLL,
it asked for the source of the C++ file, I had to browse to my C++ project.
(Before I think it complained about some .pdb files).
Now, I managed to step into the C++ function also, but as I step over and over, some of the data structures in that function don't seem to be populated with the data, e.g., please see screenshot below
You can see the blob data structure is empty, and same happened with DataParser (it was showing it had 0 items inside, whereas in code above you can see there are multiple items being added to it).
I would really appreciate some help, what is going wrong here? And where I could have done mistake. How can I debug this C++ DLL so that I also see what values are assigned to its variables currently?
Maybe my way of debugging this C++ DLL is wrong? The fact that the C# project is using an already created DLL, and I have this C++ project which was used to create this DLL some time ago - the fact that they are separate, maybe that has to do something with it also?
PS Before I had to make changes like this to C++ project and lower toolset because I use VS2012 (strange if project was created using VS2013 though because I think it is old project). Also the project uses lot of manually written other C++ classes. Maybe that is the problem also and somehow the compiler can't retrieve their values and definitions?
What are the steps in general to debug a C++ DLL file in a setup like I have?
EDIT: PPS. Also some other interesting facts I have seen. If I click F11(Step into) on the DataParser.Add function for example, not necessarily I am taken to the body of that function, it shows me body of other function (which might be somehow related to it).
Also if I press F10 say after first time Request.Add is called, it jumps over multiple Request.Add lines, and moves to the fifth one for example.
EDIT2: Also before I step into C++ code it is showing me warning that "the source is different version than the one that was used to create a DLL". Is this a problem?
Module and PDB
There is a link between a module (.dll/.exe) and the debug database (.pdb). This link is established via a timestamp and a checksum that is present in both files. Visual Studio checks the correctness of those, otherwise it will complain and not stop at breakpoints at all.
While other debuggers such as WinDbg have commands to turn that feature off, Visual Studio doesn't have such a feature and requires active manipulation (such as Chkmatch) to turn off the checmsum verification. As long as you didn't use such a tool, your debugging symbols are fine.
PDB and source
There is also a link between the debug database (.pdb) and the source. This link is established by file name and line numbers. As you can guess, your source code will not modified during compilation, so the source code does not contain any checksum or timestamp that could be verified.
Therefore, the source may have changed and the line numbers may not even match roughly any more. There are several reasons for line numbers to get broken. I have answered a similar question before and listed the following reasons for line number changes although the code itself did not change:
code reformat, which e.g. sorts the methods by visibility, so complete methods are moved
code reformat, which e.g. breaks long lines at 80 characters, usually this moves things down
optimize usings (R#) which removes 30 lines of unneeded imports, so things move up
insertion of comments or newlines
How to debug
Restore the exact source code of that version, if you can.
Debug completely without source, just by PDB information. This way you can keep the binary components, if that's important (e.g. if a bug can only be reproduced with that version)
Rebuild all modules to make the code match the modules again. That way you lose the binary and the problem may not reproduce any more.
I have an application where I used compiler switches to control whether large chunks of code were included or not. Think car with automatic transmission or manual transmission.
Works fine in Debug mode but in Release mode it looks like both Auto Transmission and Manual Transmission are compiled so the car doesn't drive too well...
So am I wrong in trying to get option control functionality out of compiler switches?
More Detail:
My understanding of compiler switches was flawed.
Simple project in comments below.(Not too good at driving StackOverFlow)
To continue the analogy, The (Winform) project was built with manual transmission. It was deployed in debug mode using a setup project. (Mistake).
After some years Auto-transmission was required. (No commonality to give rise to a base class).
The thought was that one day manual transmission may be required again)
Seemed like a compiler switch was a good idea to decide which block of code to use.
Again the Auto-transmission version was deployed in debug mode.
After some more years of running I decided that it should be released. That was when I noticed the problem.
The simple example asked for (thanks) showed me that it wasn't both blocks that were being compiled, just the block in the #else. i.e. Release build regards the switch as off.
So unless I am missing something the answer is self-evident. Don't use compiler switches for deployment options.
I suggest you use composition instead and make use of object-orientation. Have two classes, one for AutoTransmission, and another for ManualTransmission. Each one will derive from a base class Transmission Then you can use either one as you see fit when you create the object that makes use of one of these transmissions.
Then, when you create the object that uses a Transmission, simply instantiate whatever transmission you need. That could be defined in config, or it could be compiler constants. But, I'd recommend against compiler constants because when you perform automatic refactorings blocks of could disabled by #ifdef will not be processed and may not build when you change your compiler options.
Trying to find a way to prove that my program is not running correctly because the version numbers of the dll's my interops are pointing to are different i.e. different GUIDs.
Works on my machine, not on "theirs" with the different dll's.
Can anyone recommend some debugging tools that let me watch the program as it starts up and see things like "looking for dll, not found, quitting"?
Is there logging tool available that would report these things to me?
If so I'm not aware of/using it.
You get an exception when a DLL isn't found. Or more commonly in your case, a COMException as soon as you try to use the interop library in your code. One drastic mistake you could make is catching such an exception. That's a very common mistake. But don't, undiagnosable failure is the result. There is rarely any point in letting your program continue running when an important chunk of code is just missing. Logging it isn't hard when you use AppDomain.UnhandledException.
This should at the very least provide you with decent diagnostics that help you to fix your code. You cannot get this started until you get good exception info. Pre-emptively fixing rather than waiting for the customer to get back to you with an exception trace usually requires you to recreate possible client configurations and testing your code. Highly advisable btw with 4 versions of IE in common use. You'll need a virtual machine so you can install the different OS and IE versions and test your code. Making the OS and IE version a system requirement is not unreasonable, ymmv.
You can try to do it yourself quick and dirty by enumerating all the assemblies loaded by your program via AppDomain.Current.GetAssemblies(). Also, check other questions about listing loaded assemblies, like this one
Read up on Assembly class in MSDN to see what information you can get about your assemblies.
So Jeff Atwood rightly complained about Visual Studio not performing background compilation see: http://www.codinghorror.com/blog/2007/05/c-and-the-compilation-tax.html
The solution from most sources seems to be Reshaper which will incrementally perform background compilation as you write. This leads to their great realtime re-factoring tips and error detection.
But what I don't understand is with R# continually compiling my code, why does it take so long when executing a compilation via VS (i.e. Ctrl + Shift + B or similar). What I mean by this is, if R# has already compiled my code then why would I need a recompilation?
My assumption is of course that R# is not overriding the assemblies in my bin directories but instead holding the compilation results in memory. In which case, is it possible to tell R# to simply override my assemblies when compilation is successful?
I don't know about "rightly complained" - that's an opinion I happen to disagree with:)
However, the VB.NET (and probably Resharper c#) background compilers do not actually compile full assemblies - they cannot! If you think about it, the natural state of your code while you are working is not compilable! Almost every keystroke puts your code in an invalid state. Think of this line:
var x = new Something();
As you type this, from the key "v" to the key ")", your code is "wrong". Or what if you are referencing a method you haven't defined yet? And if this code is in an assembly that another assembly requires, how would you compile that second assembly at all, background or not?
The background compilers get around this by compiling small chunks of your code into multiple transient "assemblies" that are actually just metadata holders - really, they don't care about the actual effects of the code as much as the symbols defined, used, etc. When you finally hit build, the actual full assemblies still need to be built.
So no, I don't believe it's possible because they're not built to do actual full compilation - they are built to check your code and interpret symbols on the fly.
Reshaper which will incrementally perform background compilation as you write
It doesn't, it just parses the source code. The exact same thing Visual Studio already does if you don't have Resharper, that's how it implements IntelliSense, its own refactoring features and commands like GoTo Definition and Find All References. Visual Studio also parses in the background, updating its data while you type. Resharper just implements more bells and whistles with that parsing data.
Going from parsing the code to actually generating the assembly is a pretty major step. The internal format of an assembly is too convoluted to allow this to happen in the background without affecting the responsiveness of the machine.
And the C# compiler is still a large chunk of unmanaged C++ code that is independent from the IDE. An inevitable consequence of having to have the compiler first. It is however a stated goal for the next version of C# to provide compile-on-demand services. Getting true background compilation is a possibility.
I don't really have an answer but I just wanted to say that I have been using Eclipse and Java for 4 months now and I love the automatic compilation. I have a very large java code base and compilation happens constantly as I save code changes. When I hit Run everything is ready to go! It's just awesome. It also deploys to the local web server instance (Tomcat in my case) automatically as I make code changes. All this is setup by default in Eclipse.
I hope Microsoft does something similar with .net in the near future.
OK so that title sucks a little but I could not think of anything better (maybe someone else can?).
So I have a few questions around a subject here. What I want to do is create a program that can take an object and use reflection to list all its properties, methods, constructors etc. I can then manipulate these objects at runtime to test, debug and figure out exactly what some of my classes / programs are doing whilst they are running, (some of them will be windows services and maybe installed on the machine rather than running in debug from VS).
So I would provide a hook to the program that from the local machine (only) this program could get an instance of the main object and therefore see all the sub objects running in it. (for security the program may need to be started with an arg to expose that hook).
The "reflection machine" would allow for runtime manipulation and interrogation.
Does this sound possible?
Would the program have to provide a hook or could the "reflection machine" take an EXE and (if it knew all the classes it was using), create an object to use?
I know you can import DLL's at runtime so that it knows about all sorts of classes, but can you import individual classes? I.E. Say I have project 'Y' that is not compiled to a DLL but I want to use the "reflection machine" on it, can I point at that directory and grab the files to be able to reference those classes?
EDIT: I would love to try and develop it my self but I already have a long list of projects I would like to do and have already started. Why reinvent the wheel when there is already a great selection to choose from.
Try looking at Crack.NET. It is used to do runtime manipulation and interrogation on WPF/WinForms but the source is available and might be a good start if it already doesn't meet your needs.
It sound as if Corneliu Tusnea's Hawkeye might be close to what you're looking for runtime interrogation of objects/properties/etc. He calls it the .NET Runtime Object Editor. I'm not sure if the homepage I linked to above or the CodePlex project is the best place to start.
It's a bit out of date now, I think, but there's an earlier version of it on CodeProject where you can see the source code for how and what he did.
Powershell actually does nearly all of this, if I properly understand what you are saying.
See this answer on how to build a "reflection engine".
All you need to do is to drop that set of machinery in the your set of available
runtime libraries and it does what you want, I think.
(It might not be as easy as I've made it sound in practice).
My guess is you'll also want a runtime compiler, so that you can
manufacture instrumented/transformed variants of the program under inspection
to collect the runtime data you want. You may find that such
machinery provide static analysis results that let you avoid
doing the runtime analysis in many cases.