Develop Reference

Passing a C++ object pointer to a C++ class through C++/CLI

I have a particular problem which I cannot seem to reproduce in a minimal working example.
I have to deal with a large framework of legacy code and modifying all of that out of my scope. To deal with it I have to apply some particular patterns.
Overview of the codebase
I have a managed C# application (.NET 5.0). In this appliation I need to run some C++ code.
For this, there is a CLI-wrapper project. This wrapper contains most of the legacy framework which is out of my control and is why I can only transfer strings to my C++ class (more on this later). Based on config, this legacy framework uses the wrapper to instantiate C++ classes and calls methods on them, processes the results and finally, destroys all the C++ classes afterwards.
This CLI-wrapper allows me ONLY to pass strings as parameters to the C++ classes it creates.
All of my libraries are dynamically linked (using DLL's). The C# is a project which references the C++/CLI wrapper which in turn referenced the C++ project with my C++-class. This project references the external LargeLibrary (more on this later).
The root of the problem
The C++ code is called repeatedly, every few seconds. It should respond fast.
My C++ code needs to load some large file from disk (about 400 MB) and process it which takes quite some time.
Since the C++ classes are recreated each time, loading the file each time consumes so much time which is unacceptable.
As this data is essentially constant, I try to load it once during initialisation of the program. Then I pass a pointer to my C++ class which then can use the object. The object then remains in memory when the C++ class is destroyed so it can be used again later.
To complicate things, I need quite a large library to read and process my file (I reference this library here as LargeLibrary). If I make the CLI-wrapper dependent on this, it won't compile.
I can imagine this is because of the CLI stuff. Therefore, I use a void pointer, so the wrapper does not have to be aware of the actual type of behind the pointer. The actual object is created using a function inside my C++-class (so the correct destructor is linked to the shared pointer).
This all compiles fine.
My solution
I made a small extension to the CLI-wrapper to create the object which read my file from disk and keeps the information in memory.
This object is created using the method CreateInformationObject(). ptr_native is a smart pointer for using native objects in managed code. It's type is: CAutoNativePtr<std::shared_ptr<void>> ptr_native.
Creating my object inside the wrapper looks like:
// Create a shared_ptr on dynamic memory (i.e. heap).
std::shared_ptr<void>* objectPointer = new std::shared_ptr<void>();
// Load the module and store a shared pointer pointing to it in the dynamic memory.
*objectPointer = CppConsumerStuff::CppConsumer::CreateInformationObject(value);
// Load the module and store a shared pointer pointing to it in the dynamic memory.
ptr_native.Attach(objectPointer);
The CreateInformationObject() method inside my C++ class (the CppConsumerStuff::CppConsumer) is:
std::shared_ptr<void> CppConsumer::CreateInformationObject(std::string pathToFile)
{
std::shared_ptr<LargeLibrary::ActualObjectType> objectPtr = std::make_shared<LargeLibrary::ActualObjectType>();
*objectPtr = LargeLibrary::FileLoader::load(pathToFile)
return objectPtr;
}
Then, because of the legacy framework, I tried this longshot: convert the pointer address to string, pass it via the framework to my C++ class and convert it back to a pointer to the actual type of the object.
This goes like (in my CLI-wrapper extension):
//Cast void pointer to string.
String^ CliStorage::GetPointerString()
{
std::stringstream ss;
ss << (*ptr_native).get(); // Pointer to hex string.
std::string ptr_string = ss.str();
return StringToManaged(ptr_string);
}
Finally, (in my C++ class), I convert this pointer-string back to a pointer to the actual object as:
void DoWorkOnLargeObject(std::string ptr_string)
{
// Cast pointer to usable type
uint64_t raw_ptr = 0; // Define int size depending on system architecture.
std::stringstream ss;
ss << std::hex << ptr_string;
ss >> raw_ptr; //Hex string to int.
cppObjectPtr = reinterpret_cast<void*>(raw_ptr);
LargeLibrary::ActualObjectType* cppObjectPtrCasted = static_cast<LargeLibrary::ActualObjectType*>(cppObjectPtr);
// Use the object.
cppObjectPtrCasted->GetDataStuff();
// Rest of code doing work...
}
My results
I build all of this in Visual Studio 2019.
When I create a Debug build, all works :).
However, when I create a Release build, it does not work and throws the following Exception: ``
Minimal working example
I tried to create a minimal working example.
Both with and without the large external library.
However, in my minimum working Examples it always works, no matter the build type (debug / release).
My question
So my question is: Do my minimum working examples work by accident and am I relying on undefined behavior? Or should this concept (no matter how ugly it is) indeed work?
If it is undefined behavior, please explain, I want to learn. If it should work, the problem resides in the legacy framework and I will make inquiries about this.
I know these are very ugly patterns, but I try to get something working with the means I have within my scope.
Thank you
EDIT, I added CreateInformationObject() method code to my question. I think my hazard may be inside here. Maybe I do some illegal pointer stuff which results in undefined behavior?

I am not an expert on this so take my advise with a grain of salt. In my opinion directly sharing the memory address between the processes will in general fail due to memory protection (which forbids programs to just access memory that was not allocated for them).
You could used shared memory. This is memory shared between processes. Normally one would use this to share memory between concurrent processes but this is in no way necessary (and not having competing accesses is actually beneficial). Wikipedia lists boost and Qt as examples for libraries implementing cross-platform support for shared memory.
Looking into the boost documentation for sharing memory, it says "As shared memory has kernel or filesystem persistence, the user must explicitly destroy it.", which is exactly what you want, since it should persist between calls of the same program. Note that you should remove the shared memory in some other way since it will persist.
Adapting the example from the documentation, it could look something like this:
#include <boost/interprocess/shared_memory_object.hpp>
#include <boost/interprocess/mapped_region.hpp>
#include <cstring>
#include <cstdlib>
#include <string>
constexpr auto shm_name = "SharedMemoryCLI";
using namespace boost::interprocess;
auto create_shared_memory() {
// Compute your data and calculate the size needed:
shared_memory_object shm {create_only, shm_name, read_write};
// Either use an upper bound for the size needed or compute your data before.
shm.truncate(data_size);
//Map the whole shared memory in this process
mapped_region region{shm, read_write};
// Either write your data directly to region.get_address():
compute_data_to(region.get_address());
// Or have the data already computed and memcopy it:
std::memcpy(region.get_address(), data_ptr, data_size);
return region;
}
auto obtain_memory_region() {
try {
shared_memory_object shm{open_only, shm_name, read_only};
return mapped_region {shm, read_only};
} catch(const std::exception &er) {
// One should probably check if this is the "right" exception but boost does not say what type it uses here...
return create_shared_memory();
}
}
int main(int argc, char *argv[])
{
region = obtain_memory_region();
static_cast<char*>(region.get_address()); // can be used as a to your data.
return 0;
}
Note that you maybe have to persist the exact size of your shared memory in some other way (or maybe just as the first 8 byte of the region). You can then have to somehow get the char* back to your wanted type, but I think that a reinterpret_cast should work here.
The above code is not tested and I give no guarantees but I am pretty confident that it should work roughly in this way and be about as fast as just sharing the pointer (if that would work). You really should read the entirety of https://www.boost.org/doc/libs/1_48_0/doc/html/interprocess/sharedmemorybetweenprocesses.html before applying this in any way.

Calling OpenGL functions in C# using DllImport?

I'm working on a C# utility library for OpenGL at the moment.
For cross platform window management I use GLFW and I got functions like glfwCreateWindow or glfwMakeContextCurrent loaded successfully with DllImport.
Now I want to use OpenGL and since I've mostly worked with GLEW in the past I decided to use it for my C# project as well.
So I tried to use DllImport for glewInit and that worked.
But when I try to do the same thing for OpenGL functions such as glClear I get an error saying that it can't find the function.
I looked up the exact function name and parameters in the glew header file, but I think I did that correctly.
Also I am creating an OpenGL context and calling glewInit before using it.
Here's the function declaration:
GLAPI void GLAPIENTRY glClear (GLbitfield mask);
And this is how I load the function from my C# program:
[DllImport("GLEW dll path here")]
internal static extern void glClear(int mask);
Is it because the function pointers get loaded at runtime?
Or am I doing something wrong loading the function?
I would appreciate any help / comments / hints.
If you need more code I can edit the post.
I'm quite new to stackoverflow, so please let me know if I'm doing something incorrectly.

For glClear specifically, the following should be enough:
[DllImport("opengl32.dll")]
public static extern void glClear(uint clearMask);
In Windows GL functions are defined in opengl32.dll and should be loaded from there. There is a caveat though: opengl32.dll is only compatible with very basic OpenGL version (1.1? not sure), and does not contain any fancy functions available in more recent OpenGL versions (the context created by default may very well be v. 4.x compatible, but you simply don't have proper pointers to newer functions).
In C or C++, you load those function pointers with wglGetProcAddress, but it gets very unwieldy very fast, that's where libraries like Glew come in: they contain all boilerplate to declare and load those newer GL functions. Unfortunately, it's a bit less useful for the C#: you still need to declare method you are going to call there, and Glew will not help you with it in any way.
So, suppose you want to call glCompileShader. For that you need to be able to call wglGetProcAddress first:
[DllImport("opengl32.dll", CharSet = CharSet.Ansi, SetLastError = true, ExactSpelling = true)]
public static extern IntPtr wglGetProcAddress(string functionName);
(you also need to make sure you call it on the thread that has GL context set as current after the context was set as current (that is, after wglMakeCurrent was called), otherwise the call will fail)
Then, declare delegate:
delegate void GlCompileShaderDelegate(uint shader);
Then, load function address:
IntPtr glCompileShaderPtr = GL.wglGetProcAddress("glCompileShader");
if (glCompileShaderPtr == IntPtr.Zero)
{
// load failed
}
Then, convert the address to delegate:
GlCompileShaderDelegate glCompileShader = Marshal.GetDelegateForFunctionPointer<GlCompileShaderDelegate>(glCompileShaderPtr);
Now you can call glCompileShader from your C# code. Rinse and repeat for every single GL function you want to call (except those that are readily available in opengl32.dll of course), and don't mess up character encoding if you pass or receive strings from the OpenGL functions.
Now that you know how to do that, save yourself time and find a library that does it for you.

OK, so the way handled this problem is by no means ideal and I'll try to find a different solution.
For now I wrote a wrapper around GLFW and GLEW using C++, in which I use
extern "C" _declspec(dllexport)
to wrap everything I need.
I link the libs statically so that I only have one dll in the end, and I can customize the dll's content to a higher degree.
Also this allows me to continue using DllImport to load the functions instead of doing everything at runtime once the function pointers are loaded.
But as said, it's not the best thing to do, and I have to manually wrap every function I want to use. For now it should get the job done and I can work on the library itself in the meantime.
Just wanted to let anyone know who cares about it.

How to see code of method which marked as MethodImplOptions.InternalCall?

When use ILSpy to check the code of System.String, I found there are some methods marked as MethodImplOptions.InternalCall such as:
[SecurityCritical]
[MethodImpl(MethodImplOptions.InternalCall)]
internal static extern int nativeCompareOrdinalEx(string strA, int indexA, string strB, int indexB, int count);
I know MethodImplOptions.InternalCall means this method is implemented natively by the common language runtime to optimized code to improve performance.
My question is: Is that anyway can enable us to see the code marked as MethodImplOptions.InternalCall?

You'll need the source code for the CLR to see the implementation of these methods. That's a bit hard to come by, Microsoft doesn't publish it and it is not covered by the Reference Source.
As long as the method is "old", available since .NET 2.0, then you'll have a shot at it from the SSCLI20 source code. With a non-zero risk that you will be looking at an outdated version of the code of course. But good enough to get an idea what it looks like and often still accurate.
The starting point to start searching for the code is the clr/src/vm/ecall.cpp source code file. It contains the tables that the jitter searches for internal methods. The section that's relevant for nativeCompareOrdinalEx() looks like this:
FCFuncStart(gStringFuncs)
FCDynamic("FastAllocateString", CORINFO_INTRINSIC_Illegal, ECall::FastAllocateString)
FCDynamicSig(COR_CTOR_METHOD_NAME, &gsig_IM_ArrChar_RetVoid, CORINFO_INTRINSIC_Illegal, ECall::CtorCharArrayManaged)
FCDynamicSig(COR_CTOR_METHOD_NAME, &gsig_IM_ArrChar_Int_Int_RetVoid, CORINFO_INTRINSIC_Illegal, ECall::CtorCharArrayStartLengthManaged)
FCDynamicSig(COR_CTOR_METHOD_NAME, &gsig_IM_PtrChar_RetVoid, CORINFO_INTRINSIC_Illegal, ECall::CtorCharPtrManaged)
FCDynamicSig(COR_CTOR_METHOD_NAME, &gsig_IM_PtrChar_Int_Int_RetVoid, CORINFO_INTRINSIC_Illegal, ECall::CtorCharPtrStartLengthManaged)
FCDynamicSig(COR_CTOR_METHOD_NAME, &gsig_IM_Char_Int_RetVoid, CORINFO_INTRINSIC_Illegal, ECall::CtorCharCountManaged)
FCFuncElement("nativeCompareOrdinal", COMString::FCCompareOrdinal) // <=== Here
FCFuncElement("nativeCompareOrdinalWC", COMString::FCCompareOrdinalWC)
FCIntrinsic("get_Length", COMString::Length, CORINFO_INTRINSIC_StringLength)
// etc..
}
Note how the FCFuncElement has the method name as a string and a function pointer to the C++ method that implements the internal call. Grepping the source code tree then takes you to clr/src/vm/comstring.cpp. I won't bore everybody with the C++ code, just have a look for yourself.
/*================================CompareOrdinal===============================*/
FCIMPL3(INT32, COMString::FCCompareOrdinal, StringObject* strA, StringObject* strB, CLR_BOOL bIgnoreCase) {
// Yadayada
//...
}
Searching for CaseInsensitiveCompHelper() and FastCompareStringHelperAligned() takes you to the actual implementations of the, respectively, case-insensitive and case-sensitive comparison functions in the same source code file.
The only other thing notable about this is that CLR version 4 made some changes to this mechanism. Adding lots of new internal methods and supporting an entirely different additional interop mechanism through a [DllImport] attribute for a fake DLL named "QCall". There's no good way to see source for these additions that I know of.
UPDATE: source is now available from the CoreCLR project. The table was moved from ecall.cpp to ecalllist.h, the mechanics are still the same. Do keep in mind that this is the .NETCore version of the CLR, the desktop version source is still closed-source. The two version are however likely to have much in common.

Since now CoreCLR is open source, so we could check out the internal code.
You could search COMString::CompareOrdinalEx keyword in stringnative.cpp to see the internal implementation.

As the help string says, they are "implemented in the CLR itself", so you'll need to consult its C++ sources or disassembly.
Generally, the files that comprise the CLR engine are a few native DLLs in the %WINDIR%\Microsoft.NET\Framework\<.NET engine version> folder, mostly mscor*.dll and clr.dll. The root .NET DLL, mscoree.dll, is in System32 but it seems to only act as a launcher.
Since the InternalCall method implementations are implementation details, there's no guarantee that those methods are implemented in a consistent way, e.g. that there's even some global registry of them.
E.g. disassembling shows that .NET 4 System.String's native methods are implemented in clr.dll and referenced in a directory-like structure while System.Deployment.Application.NativeMethods.IClrRuntimeInfo is backed by the CLRRuntimeInfoImpl COM class in mscoreei.dll, the methods simply being its virtual functions.

Loading multiple instance of DLL (C#/.NET)

I am developing C#/.NET 3.5 application. I am using legacy dll written in C, signals.dll. I invoke it from a .NET wrapper using P/Invoke. I am calling 2 types of processing functions, type A and B. When I call only one type of processing, all works fine. When I interleave calls to A and B processing, data result is corrupted. I believe that dll, signals.dll is using C style global variables, and data gets corrupted.
To resolve that, I created 2 copies of dll on disk, signals.dll and signals2.dll. Then I modified .NET wrapper using P/Invoke to direct type A processing to one dll, type B processing to another instance. And now, all works fine.
Then I saw similar problem on forums and solution there. (Supporting multiple instances of a plugin DLL with global data ).
Basically, that proposed solution is dynamic lay from code, creating a new instance of .dll on disk (based on need), and loads it and invokes functions from it. Key part of code looks like this:
private IntPtr dllHandle;
string myDllPath = Path.Combine(dllDir, String.Format("mylib-{0}.dll", GetHashCode()));
File.Copy(origDllPath, myDllPath);
dllPath = myDllPath;
dllHandle = LoadLibrary(dllPath);
_getVersion = GetProcEntryDelegate<_getVersionDelegate>(dllHandle, "GetVersion");
private delegate int _getVersionDelegate();
private readonly _getVersionDelegate _getVersion;
public int GetVersion()
{
return _getVersion();
}
private static D GetProcEntryDelegate<D>(IntPtr hModule, string name)
where D: class
{
IntPtr addr = _getProcAddress(hModule, name);
if (addr == IntPtr.Zero)
throw new Win32Exception();
return Marshal.GetDelegateForFunctionPointer(addr, typeof(D)) as D;
}
What is coming to my mind, would it be possible to modify above code to create copy of dll IN MEMORY, not on disk and load it from there. I think just that IntPtr dllHandle needs to be fooled into getting value from memory, not from LoadLibrary. How to do that?

Both LoadLibrary and LoadLibraryEx requires a file path. You'll need a custom loading procedure, including memory mappings and what-not. I've found a blog post ("Loading a DLL from memory") describing the procedure, and a matching GitHub project; MemoryModule.

There is nothing just about it :) - it's far more complex and involving
Here is a link that might help - Load Library/Module from Memory
And as #Hans Passant said I'd discourage you to go that way - even though it may be a tempting solution for some scenarios (but I don't see that you really need that honestly, nice maybe).
It involves dealing with the portable executable format - and I doubt that project covers all that needs to be done.
You could try making your C++/CLI wrapper - or exporting the MemoryLoadLibrary and try P/Invoking - but I doubt that'd work easily.

Converting to C++ from C#

I have done pretty much all my programming using C# and very much a newbie to C++. However now I have to convert to C++ and is finding it a bit difficult. For example, I wrote a pretty simple program using C# to acquire a RegistryKey, then using a recursive function I iterate through my registry key to find a specific key and then get the values I want. No problem, I can write that program in 10 minutes using C#. Here is the code.
My primary function. It gets Bluetooth Registry Key and then call the recursive function.
private static void CheckOpenComPorts()
{
RegistryKey blueToothPorts = Registry.LocalMachine.OpenSubKey(#"SYSTEM\CurrentControlSet\Enum\Bluetooth");
List<string> foundPorts = new List<string>();
AddFoundPortsToList(blueToothPorts, ref foundPorts);
//Rest of the program; not relevant here.
}
Recursive Function. Iterates the passed Key to find out necessary values.
private static void AddFoundPortsToList(RegistryKey regKey, ref List<string> ports)
{
try
{
string[] subKeys = regKey.GetSubKeyNames();
if (subKeys != null)
{
foreach (string subKey in subKeys)
{
AddFoundPortsToList(regKey.OpenSubKey(subKey), ref ports);
}
}
if (regKey.Name.EndsWith("Device Parameters"))
{
string str = System.Convert.ToString(regKey.GetValue("PortName"));
if (String.IsNullOrEmpty(str) == false)
{
ports.Add(str);
}
}
}
catch (System.Security.SecurityException ex)
{
;
}
}
The above code works fine, but when I tried to convert it to C++, I'm pretty lost.
Note : I'm using a Win32 Console C++ Program.
I figured out that I can do something like the following to get the Bluetooth Registry Key.
RegOpenKeyEx(HKEY_LOCAL_MACHINE, L"SYSTEM\\CurrentControlSet\\Enum\\Bluetooth", 0, KEY_READ, &hKey)
But after that, I'm pretty lost about the recursive function. Specially, how do I get the available subkeys of the passed registry key when I do NOT know the subkey names?. Or in short, what is the equivalent behavior of RegistryKey.GetSubKeyNames() in C++?
As I am only beginning this thing a code sample with some explanations would be great.

Enumerate the subkeys of a key - RegEnumKeyEx
Enumerate the values of a key - RegEnumValue
You get all subkeys by calling RegEnumKeyEx in a loop until it returns ERROR_NO_MORE_ITEMS.
In the exact same way, you get all values by calling RegEnumValue in a loop until it returns ERROR_NO_MORE_ITEMS.

I'm assuming you want to transition from .NET to native C++ programming. (i.e. no CLI and no .NET framework, which you could still use if you enabled managed C++ compilation).
If you spent a ton of time in C# land, you are probably very used to a ton of very convenient classes for just about everything imaginable and all you have to do is hit "." and let the Intellisense list the methods. Well.... you can forget all those conveniences :)
There is no such (at least not complete) framework in C++ so often you have to turn to Win32 API. MSDN Library is YOUR FRIEND. If you want to get good at C++, learn how to read it and learn how to look things up (not just by name, but learn where different categories are). In this case, if you search for the function you found, you will find a whole set of functions that work on registry. So now, looking at other methods in the same category in MSDN library, you can find RegEnumKeyEx. (hint: make sure to switch MSDN library UI to classical view, that makes it much easier to navigate between topics. I don't know what MS was thinking with their "new" look and feel)
As you start using Win32 API you will realize what a pain it is, especially coming from C#. But you don't have to use it directly (or at least not every time). You can use other libraries, for example ATL provides you with CRegKey class which makes working with registry much simpler. If you can't find the class, do what C++ does best, write your own class. Constantly working directly with windows functions will make your code very, very long and a pain to read.
Another 2 libraries worth knowing as you get into C++ are STL (a must) and Boost (strong should). Boost especially has a lot of OS abstraction so you don't have to go directly to windows DLLs every time.

If you search for the MSDN help on RegOpenKeyEx and then go up in the contents you'll find all the related methods: Registry methods
You probably want to use RegEnumKeyEx to enumerate subkeys.