Develop Reference

Passing a C++ object pointer to a C++ class through C++/CLI

I have a particular problem which I cannot seem to reproduce in a minimal working example.
I have to deal with a large framework of legacy code and modifying all of that out of my scope. To deal with it I have to apply some particular patterns.
Overview of the codebase
I have a managed C# application (.NET 5.0). In this appliation I need to run some C++ code.
For this, there is a CLI-wrapper project. This wrapper contains most of the legacy framework which is out of my control and is why I can only transfer strings to my C++ class (more on this later). Based on config, this legacy framework uses the wrapper to instantiate C++ classes and calls methods on them, processes the results and finally, destroys all the C++ classes afterwards.
This CLI-wrapper allows me ONLY to pass strings as parameters to the C++ classes it creates.
All of my libraries are dynamically linked (using DLL's). The C# is a project which references the C++/CLI wrapper which in turn referenced the C++ project with my C++-class. This project references the external LargeLibrary (more on this later).
The root of the problem
The C++ code is called repeatedly, every few seconds. It should respond fast.
My C++ code needs to load some large file from disk (about 400 MB) and process it which takes quite some time.
Since the C++ classes are recreated each time, loading the file each time consumes so much time which is unacceptable.
As this data is essentially constant, I try to load it once during initialisation of the program. Then I pass a pointer to my C++ class which then can use the object. The object then remains in memory when the C++ class is destroyed so it can be used again later.
To complicate things, I need quite a large library to read and process my file (I reference this library here as LargeLibrary). If I make the CLI-wrapper dependent on this, it won't compile.
I can imagine this is because of the CLI stuff. Therefore, I use a void pointer, so the wrapper does not have to be aware of the actual type of behind the pointer. The actual object is created using a function inside my C++-class (so the correct destructor is linked to the shared pointer).
This all compiles fine.
My solution
I made a small extension to the CLI-wrapper to create the object which read my file from disk and keeps the information in memory.
This object is created using the method CreateInformationObject(). ptr_native is a smart pointer for using native objects in managed code. It's type is: CAutoNativePtr<std::shared_ptr<void>> ptr_native.
Creating my object inside the wrapper looks like:
// Create a shared_ptr on dynamic memory (i.e. heap).
std::shared_ptr<void>* objectPointer = new std::shared_ptr<void>();
// Load the module and store a shared pointer pointing to it in the dynamic memory.
*objectPointer = CppConsumerStuff::CppConsumer::CreateInformationObject(value);
// Load the module and store a shared pointer pointing to it in the dynamic memory.
ptr_native.Attach(objectPointer);
The CreateInformationObject() method inside my C++ class (the CppConsumerStuff::CppConsumer) is:
std::shared_ptr<void> CppConsumer::CreateInformationObject(std::string pathToFile)
{
std::shared_ptr<LargeLibrary::ActualObjectType> objectPtr = std::make_shared<LargeLibrary::ActualObjectType>();
*objectPtr = LargeLibrary::FileLoader::load(pathToFile)
return objectPtr;
}
Then, because of the legacy framework, I tried this longshot: convert the pointer address to string, pass it via the framework to my C++ class and convert it back to a pointer to the actual type of the object.
This goes like (in my CLI-wrapper extension):
//Cast void pointer to string.
String^ CliStorage::GetPointerString()
{
std::stringstream ss;
ss << (*ptr_native).get(); // Pointer to hex string.
std::string ptr_string = ss.str();
return StringToManaged(ptr_string);
}
Finally, (in my C++ class), I convert this pointer-string back to a pointer to the actual object as:
void DoWorkOnLargeObject(std::string ptr_string)
{
// Cast pointer to usable type
uint64_t raw_ptr = 0; // Define int size depending on system architecture.
std::stringstream ss;
ss << std::hex << ptr_string;
ss >> raw_ptr; //Hex string to int.
cppObjectPtr = reinterpret_cast<void*>(raw_ptr);
LargeLibrary::ActualObjectType* cppObjectPtrCasted = static_cast<LargeLibrary::ActualObjectType*>(cppObjectPtr);
// Use the object.
cppObjectPtrCasted->GetDataStuff();
// Rest of code doing work...
}
My results
I build all of this in Visual Studio 2019.
When I create a Debug build, all works :).
However, when I create a Release build, it does not work and throws the following Exception: ``
Minimal working example
I tried to create a minimal working example.
Both with and without the large external library.
However, in my minimum working Examples it always works, no matter the build type (debug / release).
My question
So my question is: Do my minimum working examples work by accident and am I relying on undefined behavior? Or should this concept (no matter how ugly it is) indeed work?
If it is undefined behavior, please explain, I want to learn. If it should work, the problem resides in the legacy framework and I will make inquiries about this.
I know these are very ugly patterns, but I try to get something working with the means I have within my scope.
Thank you
EDIT, I added CreateInformationObject() method code to my question. I think my hazard may be inside here. Maybe I do some illegal pointer stuff which results in undefined behavior?

I am not an expert on this so take my advise with a grain of salt. In my opinion directly sharing the memory address between the processes will in general fail due to memory protection (which forbids programs to just access memory that was not allocated for them).
You could used shared memory. This is memory shared between processes. Normally one would use this to share memory between concurrent processes but this is in no way necessary (and not having competing accesses is actually beneficial). Wikipedia lists boost and Qt as examples for libraries implementing cross-platform support for shared memory.
Looking into the boost documentation for sharing memory, it says "As shared memory has kernel or filesystem persistence, the user must explicitly destroy it.", which is exactly what you want, since it should persist between calls of the same program. Note that you should remove the shared memory in some other way since it will persist.
Adapting the example from the documentation, it could look something like this:
#include <boost/interprocess/shared_memory_object.hpp>
#include <boost/interprocess/mapped_region.hpp>
#include <cstring>
#include <cstdlib>
#include <string>
constexpr auto shm_name = "SharedMemoryCLI";
using namespace boost::interprocess;
auto create_shared_memory() {
// Compute your data and calculate the size needed:
shared_memory_object shm {create_only, shm_name, read_write};
// Either use an upper bound for the size needed or compute your data before.
shm.truncate(data_size);
//Map the whole shared memory in this process
mapped_region region{shm, read_write};
// Either write your data directly to region.get_address():
compute_data_to(region.get_address());
// Or have the data already computed and memcopy it:
std::memcpy(region.get_address(), data_ptr, data_size);
return region;
}
auto obtain_memory_region() {
try {
shared_memory_object shm{open_only, shm_name, read_only};
return mapped_region {shm, read_only};
} catch(const std::exception &er) {
// One should probably check if this is the "right" exception but boost does not say what type it uses here...
return create_shared_memory();
}
}
int main(int argc, char *argv[])
{
region = obtain_memory_region();
static_cast<char*>(region.get_address()); // can be used as a to your data.
return 0;
}
Note that you maybe have to persist the exact size of your shared memory in some other way (or maybe just as the first 8 byte of the region). You can then have to somehow get the char* back to your wanted type, but I think that a reinterpret_cast should work here.
The above code is not tested and I give no guarantees but I am pretty confident that it should work roughly in this way and be about as fast as just sharing the pointer (if that would work). You really should read the entirety of https://www.boost.org/doc/libs/1_48_0/doc/html/interprocess/sharedmemorybetweenprocesses.html before applying this in any way.

Calling OpenGL functions in C# using DllImport?

I'm working on a C# utility library for OpenGL at the moment.
For cross platform window management I use GLFW and I got functions like glfwCreateWindow or glfwMakeContextCurrent loaded successfully with DllImport.
Now I want to use OpenGL and since I've mostly worked with GLEW in the past I decided to use it for my C# project as well.
So I tried to use DllImport for glewInit and that worked.
But when I try to do the same thing for OpenGL functions such as glClear I get an error saying that it can't find the function.
I looked up the exact function name and parameters in the glew header file, but I think I did that correctly.
Also I am creating an OpenGL context and calling glewInit before using it.
Here's the function declaration:
GLAPI void GLAPIENTRY glClear (GLbitfield mask);
And this is how I load the function from my C# program:
[DllImport("GLEW dll path here")]
internal static extern void glClear(int mask);
Is it because the function pointers get loaded at runtime?
Or am I doing something wrong loading the function?
I would appreciate any help / comments / hints.
If you need more code I can edit the post.
I'm quite new to stackoverflow, so please let me know if I'm doing something incorrectly.

For glClear specifically, the following should be enough:
[DllImport("opengl32.dll")]
public static extern void glClear(uint clearMask);
In Windows GL functions are defined in opengl32.dll and should be loaded from there. There is a caveat though: opengl32.dll is only compatible with very basic OpenGL version (1.1? not sure), and does not contain any fancy functions available in more recent OpenGL versions (the context created by default may very well be v. 4.x compatible, but you simply don't have proper pointers to newer functions).
In C or C++, you load those function pointers with wglGetProcAddress, but it gets very unwieldy very fast, that's where libraries like Glew come in: they contain all boilerplate to declare and load those newer GL functions. Unfortunately, it's a bit less useful for the C#: you still need to declare method you are going to call there, and Glew will not help you with it in any way.
So, suppose you want to call glCompileShader. For that you need to be able to call wglGetProcAddress first:
[DllImport("opengl32.dll", CharSet = CharSet.Ansi, SetLastError = true, ExactSpelling = true)]
public static extern IntPtr wglGetProcAddress(string functionName);
(you also need to make sure you call it on the thread that has GL context set as current after the context was set as current (that is, after wglMakeCurrent was called), otherwise the call will fail)
Then, declare delegate:
delegate void GlCompileShaderDelegate(uint shader);
Then, load function address:
IntPtr glCompileShaderPtr = GL.wglGetProcAddress("glCompileShader");
if (glCompileShaderPtr == IntPtr.Zero)
{
// load failed
}
Then, convert the address to delegate:
GlCompileShaderDelegate glCompileShader = Marshal.GetDelegateForFunctionPointer<GlCompileShaderDelegate>(glCompileShaderPtr);
Now you can call glCompileShader from your C# code. Rinse and repeat for every single GL function you want to call (except those that are readily available in opengl32.dll of course), and don't mess up character encoding if you pass or receive strings from the OpenGL functions.
Now that you know how to do that, save yourself time and find a library that does it for you.

OK, so the way handled this problem is by no means ideal and I'll try to find a different solution.
For now I wrote a wrapper around GLFW and GLEW using C++, in which I use
extern "C" _declspec(dllexport)
to wrap everything I need.
I link the libs statically so that I only have one dll in the end, and I can customize the dll's content to a higher degree.
Also this allows me to continue using DllImport to load the functions instead of doing everything at runtime once the function pointers are loaded.
But as said, it's not the best thing to do, and I have to manually wrap every function I want to use. For now it should get the job done and I can work on the library itself in the meantime.
Just wanted to let anyone know who cares about it.

Library requires reference to System.Windows.Forms

I'm having trouble finding information on this topic, possibly because I'm not sure how to phrase the question. Hopefully the braintrust here can help or at least advise. This situation might just be me being retentive but it's bugging me so I thought I'd ask for help on how to get around it.
I have a C# library filled with utility classes used in other assemblies. All of my extensions reside in this library and it comes in quite handy. Any of my other libraries or executables that need to use those classes must naturally reference the library.
But one of the extensions I have in there is an extension on the Control class to handle cross thread control updates in a less convoluted fashion. As a consequence the utility library must reference System.Windows.Forms.
The problem being that any library or executable that references the utilities library must now have a reference to System.Windows.Forms as well or I get a a build error for the missing reference. While this is not a big deal, it seems sort of stupid to have assemblies that have nothing to do with controls or forms having to reference System.Windows.Forms just because the utilities library does especially since most of them aren't actually using the InvokeAsRequired() extension I wrote.
I thought about moving the InvokeAsRequired() extension into it's own library, which would eliminate the System.Windows.forms problem as only assemblies that needed to use the InvokeAsRequired() extension would already have a reference to SWF.... but then I'd have a library with only one thing in it which will probably bother me more.
Is there a way around this requirement beyond separating out the 'offending' method and creating a nearly empty library? Maybe a compile setting or something?
It should be noted that the 'offending method' is actually used across multiple projects that have UI. A lot of the UI updates I do are as a result of events coming in and trying to update windows form controls from another thread causes various UI thread problems. Hence the method handling the Invoke when needed. (Though personally I think that whole InvokeRequired pattern should be wrapped up into the control itself rather than having something external do the thread alignment in the first place).

If it's just one function, then package it up as a source code file into a NuGet package and then add the NuGet package to your projects. Then, this code will be easily deployable to new projects (as well as easily updateable), but you don't need to create a separate assembly. Just compile it into your application.
You would then store your NuGet package in a local NuGet repository, or get a myget account, or even just store it somewhere on your network. Worst case, you can check it into your version control, but I would just check in the "project" that you build the nuget package from, so you can rebuild the package if need be.
Who knows, at some point, you may add more utility functions that require windows forms, and at that point you could justify a separate assembly.

It's easy: you have to move the offending code out. For now it might be a little concern, but in the end it might be a blast you did it now instead of at the moment you are forced to.
Even if it is just one method (for now), just move the method into another assembly. I didn't say a new one, it can be in the assembly that uses it if only one, or all others that need that moment derive from it.

You can solve your problem by switching the utility library code from the early-binding pattern to the late-binding pattern when it comes to types declared in the System.Windows.Forms namespace.
This article shows how to do it the short way: Stack Overflow: C#.NET - Type.GetType(“System.Windows.Forms.Form”) returns null
And this code snippet shows how the monoresgen tool from the Mono Project (open source ECMA CLI, C# and .NET implementation) solves the System.Windows.Forms dependency problem.
public const string AssemblySystem_Windows_Forms = "System.Windows.Forms, Version=" + FxVersion + ", Culture=neutral, PublicKeyToken=b77a5c561934e089";
// ...
static Assembly swf;
static Type resxr;
static Type resxw;
/*
* We load the ResX format stuff on demand, since the classes are in
* System.Windows.Forms (!!!) and we can't depend on that assembly in mono, yet.
*/
static void LoadResX () {
if (swf != null)
return;
try {
swf = Assembly.Load(Consts.AssemblySystem_Windows_Forms);
resxr = swf.GetType("System.Resources.ResXResourceReader");
resxw = swf.GetType("System.Resources.ResXResourceWriter");
} catch (Exception e) {
throw new Exception ("Cannot load support for ResX format: " + e.Message);
}
}
// ...
static IResourceReader GetReader (Stream stream, string name, bool useSourcePath) {
string format = Path.GetExtension (name);
switch (format.ToLower (System.Globalization.CultureInfo.InvariantCulture)) {
// ...
case ".resx":
LoadResX ();
IResourceReader reader = (IResourceReader) Activator.CreateInstance (
resxr, new object[] {stream});
if (useSourcePath) { // only possible on 2.0 profile, or higher
PropertyInfo p = reader.GetType ().GetProperty ("BasePath",
BindingFlags.Public | BindingFlags.Instance);
if (p != null && p.CanWrite) {
p.SetValue (reader, Path.GetDirectoryName (name), null);
}
}
return reader;
// ...
}
}
Snippet source: https://github.com/mono/mono/blob/mono-3.10.0/mcs/tools/resgen/monoresgen.cs#L30

Converting to C++ from C#

I have done pretty much all my programming using C# and very much a newbie to C++. However now I have to convert to C++ and is finding it a bit difficult. For example, I wrote a pretty simple program using C# to acquire a RegistryKey, then using a recursive function I iterate through my registry key to find a specific key and then get the values I want. No problem, I can write that program in 10 minutes using C#. Here is the code.
My primary function. It gets Bluetooth Registry Key and then call the recursive function.
private static void CheckOpenComPorts()
{
RegistryKey blueToothPorts = Registry.LocalMachine.OpenSubKey(#"SYSTEM\CurrentControlSet\Enum\Bluetooth");
List<string> foundPorts = new List<string>();
AddFoundPortsToList(blueToothPorts, ref foundPorts);
//Rest of the program; not relevant here.
}
Recursive Function. Iterates the passed Key to find out necessary values.
private static void AddFoundPortsToList(RegistryKey regKey, ref List<string> ports)
{
try
{
string[] subKeys = regKey.GetSubKeyNames();
if (subKeys != null)
{
foreach (string subKey in subKeys)
{
AddFoundPortsToList(regKey.OpenSubKey(subKey), ref ports);
}
}
if (regKey.Name.EndsWith("Device Parameters"))
{
string str = System.Convert.ToString(regKey.GetValue("PortName"));
if (String.IsNullOrEmpty(str) == false)
{
ports.Add(str);
}
}
}
catch (System.Security.SecurityException ex)
{
;
}
}
The above code works fine, but when I tried to convert it to C++, I'm pretty lost.
Note : I'm using a Win32 Console C++ Program.
I figured out that I can do something like the following to get the Bluetooth Registry Key.
RegOpenKeyEx(HKEY_LOCAL_MACHINE, L"SYSTEM\\CurrentControlSet\\Enum\\Bluetooth", 0, KEY_READ, &hKey)
But after that, I'm pretty lost about the recursive function. Specially, how do I get the available subkeys of the passed registry key when I do NOT know the subkey names?. Or in short, what is the equivalent behavior of RegistryKey.GetSubKeyNames() in C++?
As I am only beginning this thing a code sample with some explanations would be great.

Enumerate the subkeys of a key - RegEnumKeyEx
Enumerate the values of a key - RegEnumValue
You get all subkeys by calling RegEnumKeyEx in a loop until it returns ERROR_NO_MORE_ITEMS.
In the exact same way, you get all values by calling RegEnumValue in a loop until it returns ERROR_NO_MORE_ITEMS.

I'm assuming you want to transition from .NET to native C++ programming. (i.e. no CLI and no .NET framework, which you could still use if you enabled managed C++ compilation).
If you spent a ton of time in C# land, you are probably very used to a ton of very convenient classes for just about everything imaginable and all you have to do is hit "." and let the Intellisense list the methods. Well.... you can forget all those conveniences :)
There is no such (at least not complete) framework in C++ so often you have to turn to Win32 API. MSDN Library is YOUR FRIEND. If you want to get good at C++, learn how to read it and learn how to look things up (not just by name, but learn where different categories are). In this case, if you search for the function you found, you will find a whole set of functions that work on registry. So now, looking at other methods in the same category in MSDN library, you can find RegEnumKeyEx. (hint: make sure to switch MSDN library UI to classical view, that makes it much easier to navigate between topics. I don't know what MS was thinking with their "new" look and feel)
As you start using Win32 API you will realize what a pain it is, especially coming from C#. But you don't have to use it directly (or at least not every time). You can use other libraries, for example ATL provides you with CRegKey class which makes working with registry much simpler. If you can't find the class, do what C++ does best, write your own class. Constantly working directly with windows functions will make your code very, very long and a pain to read.
Another 2 libraries worth knowing as you get into C++ are STL (a must) and Boost (strong should). Boost especially has a lot of OS abstraction so you don't have to go directly to windows DLLs every time.

If you search for the MSDN help on RegOpenKeyEx and then go up in the contents you'll find all the related methods: Registry methods
You probably want to use RegEnumKeyEx to enumerate subkeys.

C# DllImport trouble

My question is a little general, so i'm not looking for an exact answer, but possibly some directions to look into that will help me...
At my work place I program mostly in C#.
We have this 3rd party company we work with, that gave us a Native C++ dll that we need to use.
Since the C++ method I needed wasn't exposed in a manner that was easy to reference from C#, I wrapped the dll in another Native C++ Dll.
So now i have 2 Native C++ dlls, one wrapping the other.
I created a small C# console application that calls the method I created in C++.
My method signature looks like this :
[DllImport("HashMethodWrapper.dll")]
[return: MarshalAs(UnmanagedType.LPStr)]
private static extern string CreateHash(
string input,
[MarshalAs(UnmanagedType.LPStr)]StringBuilder output);
In my console application, everything works fine, and i always receive the string im expecting in the result.
But when I move it to a web service or a Web Application i created (since this is where i really need it), I see that the string im receiving is garbage and not even consistent. It seems as if im getting just some reference to memory that is lost or something like that, but this is only a guess of mine...
I don't know why this happens, since in my console application everything works fine.
Does anyone have a direction that might help me ???...
Thanks in advance,
gillyb
Edit :
I thought it might have to do with some unpinned objects, so i tried calling the method in a fixed statement, something like :
unsafe public static string CreateHashWrap(string pass)
{
String bb;
StringBuilder outPass = new StringBuilder();
fixed (char* resultStr = CreateHash(pass, outPass))
{
bb = new String(resultStr);
}
return bb;
}
...but this still didn't do it for me. Is this the right way to pin objects ?
2nd Edit :
The method signature in C++ looks like this :
extern "C" __declspec(dllexport) char *CreateRsaHash(char *inputPass, char *hashPass);
3rd Edit :
I changed the signature of the method to be
extern "C" __declspec(dllexport) bool CreateRsaHash(char *inputPass, char *hashPass);
and the return value im looking for is placed in the *hashPass parameter.
Now, I created a simple Console application to test it. When insert the DllImport in my main class, and directly call the method everything works great, but when I move the DllImport and wrap the method in a different class and call that class from the Console 'Main' method, I get a StackOverflow exception!
Anyone got any ideas why this is happening ??

Try specifying the capacity of the StringBuilder before passing it to your interop method.

It's really hard to know from the sparse information but if I had to guess I would say you need to make sure you're pinning the output object. Also I would probably change the output parameter to some other type, it seems pretty strange that StringBuilder works at all frankly.
I do know that if you allocate an object, it will get a pointer but that doesn't mean that it won't move. So if you try to pass a pointer to a managed object into an unmanaged environment you need to make sure you tell the GC to "pin" the memory so it doesn't get moved out from under you.
Here is a really rough version of what I mean by pinning:
string input = "...";
StringBuilder output = new StringBuilder();
var handle = System.Runtime.InteropServices.GCHandle.Alloc(output, GCHandleType.Pinned);
try
{
CreateHash(input, output);
}
finally
{
handle.Free();
}

I would consider to warp inside a C# shared assembly/dll instead of a c++ dll and then try to get your console application to work with the dll. It is good practice to wrap external dependencies this way anyway.
Otherwise some traditional issues are 32 vs 64 bit, the load path to the shared library. Is it really only a string or something more complex?

I found the solution to my problem, and now i feel kinda (if not really!) stupid... :-|
I used the LoadLibrary() method in C++ to dynamically invoke a method from the other native dll. The problem was that I didn't give the method any path, and just the dll filename. In .net, it would've searched in the current folder, but seems like in native code this doesn't work this way.
The bigger problem in my programming practices is obviously the fact that i didn't fully cover error handling in my native C++ dll!
All the asnwers I received on this page weren't for nothing though...
Once I found out that i had problem with the directory path, I ran into different exceptions about trying to access corrupt memory, etc. And then I needed to create pinned objects, and declare a size for my StringBuilder object.
Thanks to everyone for your help!!
:)