Develop Reference

Passing a C++ object pointer to a C++ class through C++/CLI

I have a particular problem which I cannot seem to reproduce in a minimal working example.
I have to deal with a large framework of legacy code and modifying all of that out of my scope. To deal with it I have to apply some particular patterns.
Overview of the codebase
I have a managed C# application (.NET 5.0). In this appliation I need to run some C++ code.
For this, there is a CLI-wrapper project. This wrapper contains most of the legacy framework which is out of my control and is why I can only transfer strings to my C++ class (more on this later). Based on config, this legacy framework uses the wrapper to instantiate C++ classes and calls methods on them, processes the results and finally, destroys all the C++ classes afterwards.
This CLI-wrapper allows me ONLY to pass strings as parameters to the C++ classes it creates.
All of my libraries are dynamically linked (using DLL's). The C# is a project which references the C++/CLI wrapper which in turn referenced the C++ project with my C++-class. This project references the external LargeLibrary (more on this later).
The root of the problem
The C++ code is called repeatedly, every few seconds. It should respond fast.
My C++ code needs to load some large file from disk (about 400 MB) and process it which takes quite some time.
Since the C++ classes are recreated each time, loading the file each time consumes so much time which is unacceptable.
As this data is essentially constant, I try to load it once during initialisation of the program. Then I pass a pointer to my C++ class which then can use the object. The object then remains in memory when the C++ class is destroyed so it can be used again later.
To complicate things, I need quite a large library to read and process my file (I reference this library here as LargeLibrary). If I make the CLI-wrapper dependent on this, it won't compile.
I can imagine this is because of the CLI stuff. Therefore, I use a void pointer, so the wrapper does not have to be aware of the actual type of behind the pointer. The actual object is created using a function inside my C++-class (so the correct destructor is linked to the shared pointer).
This all compiles fine.
My solution
I made a small extension to the CLI-wrapper to create the object which read my file from disk and keeps the information in memory.
This object is created using the method CreateInformationObject(). ptr_native is a smart pointer for using native objects in managed code. It's type is: CAutoNativePtr<std::shared_ptr<void>> ptr_native.
Creating my object inside the wrapper looks like:
// Create a shared_ptr on dynamic memory (i.e. heap).
std::shared_ptr<void>* objectPointer = new std::shared_ptr<void>();
// Load the module and store a shared pointer pointing to it in the dynamic memory.
*objectPointer = CppConsumerStuff::CppConsumer::CreateInformationObject(value);
// Load the module and store a shared pointer pointing to it in the dynamic memory.
ptr_native.Attach(objectPointer);
The CreateInformationObject() method inside my C++ class (the CppConsumerStuff::CppConsumer) is:
std::shared_ptr<void> CppConsumer::CreateInformationObject(std::string pathToFile)
{
std::shared_ptr<LargeLibrary::ActualObjectType> objectPtr = std::make_shared<LargeLibrary::ActualObjectType>();
*objectPtr = LargeLibrary::FileLoader::load(pathToFile)
return objectPtr;
}
Then, because of the legacy framework, I tried this longshot: convert the pointer address to string, pass it via the framework to my C++ class and convert it back to a pointer to the actual type of the object.
This goes like (in my CLI-wrapper extension):
//Cast void pointer to string.
String^ CliStorage::GetPointerString()
{
std::stringstream ss;
ss << (*ptr_native).get(); // Pointer to hex string.
std::string ptr_string = ss.str();
return StringToManaged(ptr_string);
}
Finally, (in my C++ class), I convert this pointer-string back to a pointer to the actual object as:
void DoWorkOnLargeObject(std::string ptr_string)
{
// Cast pointer to usable type
uint64_t raw_ptr = 0; // Define int size depending on system architecture.
std::stringstream ss;
ss << std::hex << ptr_string;
ss >> raw_ptr; //Hex string to int.
cppObjectPtr = reinterpret_cast<void*>(raw_ptr);
LargeLibrary::ActualObjectType* cppObjectPtrCasted = static_cast<LargeLibrary::ActualObjectType*>(cppObjectPtr);
// Use the object.
cppObjectPtrCasted->GetDataStuff();
// Rest of code doing work...
}
My results
I build all of this in Visual Studio 2019.
When I create a Debug build, all works :).
However, when I create a Release build, it does not work and throws the following Exception: ``
Minimal working example
I tried to create a minimal working example.
Both with and without the large external library.
However, in my minimum working Examples it always works, no matter the build type (debug / release).
My question
So my question is: Do my minimum working examples work by accident and am I relying on undefined behavior? Or should this concept (no matter how ugly it is) indeed work?
If it is undefined behavior, please explain, I want to learn. If it should work, the problem resides in the legacy framework and I will make inquiries about this.
I know these are very ugly patterns, but I try to get something working with the means I have within my scope.
Thank you
EDIT, I added CreateInformationObject() method code to my question. I think my hazard may be inside here. Maybe I do some illegal pointer stuff which results in undefined behavior?

I am not an expert on this so take my advise with a grain of salt. In my opinion directly sharing the memory address between the processes will in general fail due to memory protection (which forbids programs to just access memory that was not allocated for them).
You could used shared memory. This is memory shared between processes. Normally one would use this to share memory between concurrent processes but this is in no way necessary (and not having competing accesses is actually beneficial). Wikipedia lists boost and Qt as examples for libraries implementing cross-platform support for shared memory.
Looking into the boost documentation for sharing memory, it says "As shared memory has kernel or filesystem persistence, the user must explicitly destroy it.", which is exactly what you want, since it should persist between calls of the same program. Note that you should remove the shared memory in some other way since it will persist.
Adapting the example from the documentation, it could look something like this:
#include <boost/interprocess/shared_memory_object.hpp>
#include <boost/interprocess/mapped_region.hpp>
#include <cstring>
#include <cstdlib>
#include <string>
constexpr auto shm_name = "SharedMemoryCLI";
using namespace boost::interprocess;
auto create_shared_memory() {
// Compute your data and calculate the size needed:
shared_memory_object shm {create_only, shm_name, read_write};
// Either use an upper bound for the size needed or compute your data before.
shm.truncate(data_size);
//Map the whole shared memory in this process
mapped_region region{shm, read_write};
// Either write your data directly to region.get_address():
compute_data_to(region.get_address());
// Or have the data already computed and memcopy it:
std::memcpy(region.get_address(), data_ptr, data_size);
return region;
}
auto obtain_memory_region() {
try {
shared_memory_object shm{open_only, shm_name, read_only};
return mapped_region {shm, read_only};
} catch(const std::exception &er) {
// One should probably check if this is the "right" exception but boost does not say what type it uses here...
return create_shared_memory();
}
}
int main(int argc, char *argv[])
{
region = obtain_memory_region();
static_cast<char*>(region.get_address()); // can be used as a to your data.
return 0;
}
Note that you maybe have to persist the exact size of your shared memory in some other way (or maybe just as the first 8 byte of the region). You can then have to somehow get the char* back to your wanted type, but I think that a reinterpret_cast should work here.
The above code is not tested and I give no guarantees but I am pretty confident that it should work roughly in this way and be about as fast as just sharing the pointer (if that would work). You really should read the entirety of https://www.boost.org/doc/libs/1_48_0/doc/html/interprocess/sharedmemorybetweenprocesses.html before applying this in any way.

Loading multiple instance of DLL (C#/.NET)

I am developing C#/.NET 3.5 application. I am using legacy dll written in C, signals.dll. I invoke it from a .NET wrapper using P/Invoke. I am calling 2 types of processing functions, type A and B. When I call only one type of processing, all works fine. When I interleave calls to A and B processing, data result is corrupted. I believe that dll, signals.dll is using C style global variables, and data gets corrupted.
To resolve that, I created 2 copies of dll on disk, signals.dll and signals2.dll. Then I modified .NET wrapper using P/Invoke to direct type A processing to one dll, type B processing to another instance. And now, all works fine.
Then I saw similar problem on forums and solution there. (Supporting multiple instances of a plugin DLL with global data ).
Basically, that proposed solution is dynamic lay from code, creating a new instance of .dll on disk (based on need), and loads it and invokes functions from it. Key part of code looks like this:
private IntPtr dllHandle;
string myDllPath = Path.Combine(dllDir, String.Format("mylib-{0}.dll", GetHashCode()));
File.Copy(origDllPath, myDllPath);
dllPath = myDllPath;
dllHandle = LoadLibrary(dllPath);
_getVersion = GetProcEntryDelegate<_getVersionDelegate>(dllHandle, "GetVersion");
private delegate int _getVersionDelegate();
private readonly _getVersionDelegate _getVersion;
public int GetVersion()
{
return _getVersion();
}
private static D GetProcEntryDelegate<D>(IntPtr hModule, string name)
where D: class
{
IntPtr addr = _getProcAddress(hModule, name);
if (addr == IntPtr.Zero)
throw new Win32Exception();
return Marshal.GetDelegateForFunctionPointer(addr, typeof(D)) as D;
}
What is coming to my mind, would it be possible to modify above code to create copy of dll IN MEMORY, not on disk and load it from there. I think just that IntPtr dllHandle needs to be fooled into getting value from memory, not from LoadLibrary. How to do that?

Both LoadLibrary and LoadLibraryEx requires a file path. You'll need a custom loading procedure, including memory mappings and what-not. I've found a blog post ("Loading a DLL from memory") describing the procedure, and a matching GitHub project; MemoryModule.

There is nothing just about it :) - it's far more complex and involving
Here is a link that might help - Load Library/Module from Memory
And as #Hans Passant said I'd discourage you to go that way - even though it may be a tempting solution for some scenarios (but I don't see that you really need that honestly, nice maybe).
It involves dealing with the portable executable format - and I doubt that project covers all that needs to be done.
You could try making your C++/CLI wrapper - or exporting the MemoryLoadLibrary and try P/Invoking - but I doubt that'd work easily.

What can be read from memory in a C# application

I know that any application running (whether it is built with C#, C, C++, Java, etc) will have elements exposed in memory. I'm curious as to how to control what and how it is exposed in memory?
I'm curious because I know that many games get hacked or modified by a user viewing the contents in memory of the game and altering them. I just want to know more details around how this works. I know special programs must be used to even dive into the memory and there are conversions and stuff that must happen for it to even be some what readable.
Let's take a extremely simple example and I'll ask some questions about it.
using System.Security;
static class Program2
{
private static SecureString fSecureString;
public static string fPublicString = "Test123";
private static string fPrivateString = "321tesT";
static void Main2()
{
}
}
class TestClass
{
private string fInstancedPrivateString;
public TestClass()
{
fInstancedPrivateString = "InstancedSet";
}
private string DoSomething()
{
return fInstancedPrivateString.ToLower();
}
}
}
Given the code above, I imagine that fPublicString is pretty visible to see. What elements can someone reading memory see? Can they read the variable name or do they just see an memory address and an assigned value (Test123). What about Functions like DoSomething that are inside an instanced class? Can someone see that in memory and write malicious code to execute it at their will?
I'm just curious as to how much of this I need to keep in mind while writing applications (or games). I understand the general idea of the accessor properties (public/private/etc) and their relation to other code having visibility to it, but I'm curious if they have any bearing on how it is represented in memory.
My final question will be very specific: EverQuest (game) has a hack called MacroQuest which from my understanding reads memory by having the proper offsets and can then execute code from the EQ client side or simply change values stored in memory for the client. How did EQ get this so wrong? Was it poor programming on their end? A technology limitation that is sort of resolved now? Or can this technically be done with virtually every piece of software that is written with the right amount of knowledge?
Over all I guess I could probably use a good tutorial, article, or book that provides some details on how code looks in memory etc.

Knowing that your application's memory can be read should not be something a "normal" developer needs to worry about. The number of users that are able to exploit this in a useful way are very few (in the grand scheme) and it only really matters for sensitive parts of your application anyway (licensing, passwords, and other personally identifiable information). Otherwise, the risk is really negligible.
If the effort of protecting it can't be justified by the cost of doing so then why should the person/group/etc paying to have it built worry. It isn't worth investing the time to care when there's always a ton of other things that could otherwise use the time investment.
Should Notepad or MS Word care that you can write a sniffer to listen to what is being typed? Probably not, and why? Because it really doesn't effect the bottom line or pose any realistic risk.

C# DllImport trouble

My question is a little general, so i'm not looking for an exact answer, but possibly some directions to look into that will help me...
At my work place I program mostly in C#.
We have this 3rd party company we work with, that gave us a Native C++ dll that we need to use.
Since the C++ method I needed wasn't exposed in a manner that was easy to reference from C#, I wrapped the dll in another Native C++ Dll.
So now i have 2 Native C++ dlls, one wrapping the other.
I created a small C# console application that calls the method I created in C++.
My method signature looks like this :
[DllImport("HashMethodWrapper.dll")]
[return: MarshalAs(UnmanagedType.LPStr)]
private static extern string CreateHash(
string input,
[MarshalAs(UnmanagedType.LPStr)]StringBuilder output);
In my console application, everything works fine, and i always receive the string im expecting in the result.
But when I move it to a web service or a Web Application i created (since this is where i really need it), I see that the string im receiving is garbage and not even consistent. It seems as if im getting just some reference to memory that is lost or something like that, but this is only a guess of mine...
I don't know why this happens, since in my console application everything works fine.
Does anyone have a direction that might help me ???...
Thanks in advance,
gillyb
Edit :
I thought it might have to do with some unpinned objects, so i tried calling the method in a fixed statement, something like :
unsafe public static string CreateHashWrap(string pass)
{
String bb;
StringBuilder outPass = new StringBuilder();
fixed (char* resultStr = CreateHash(pass, outPass))
{
bb = new String(resultStr);
}
return bb;
}
...but this still didn't do it for me. Is this the right way to pin objects ?
2nd Edit :
The method signature in C++ looks like this :
extern "C" __declspec(dllexport) char *CreateRsaHash(char *inputPass, char *hashPass);
3rd Edit :
I changed the signature of the method to be
extern "C" __declspec(dllexport) bool CreateRsaHash(char *inputPass, char *hashPass);
and the return value im looking for is placed in the *hashPass parameter.
Now, I created a simple Console application to test it. When insert the DllImport in my main class, and directly call the method everything works great, but when I move the DllImport and wrap the method in a different class and call that class from the Console 'Main' method, I get a StackOverflow exception!
Anyone got any ideas why this is happening ??

Try specifying the capacity of the StringBuilder before passing it to your interop method.

It's really hard to know from the sparse information but if I had to guess I would say you need to make sure you're pinning the output object. Also I would probably change the output parameter to some other type, it seems pretty strange that StringBuilder works at all frankly.
I do know that if you allocate an object, it will get a pointer but that doesn't mean that it won't move. So if you try to pass a pointer to a managed object into an unmanaged environment you need to make sure you tell the GC to "pin" the memory so it doesn't get moved out from under you.
Here is a really rough version of what I mean by pinning:
string input = "...";
StringBuilder output = new StringBuilder();
var handle = System.Runtime.InteropServices.GCHandle.Alloc(output, GCHandleType.Pinned);
try
{
CreateHash(input, output);
}
finally
{
handle.Free();
}

I would consider to warp inside a C# shared assembly/dll instead of a c++ dll and then try to get your console application to work with the dll. It is good practice to wrap external dependencies this way anyway.
Otherwise some traditional issues are 32 vs 64 bit, the load path to the shared library. Is it really only a string or something more complex?

I found the solution to my problem, and now i feel kinda (if not really!) stupid... :-|
I used the LoadLibrary() method in C++ to dynamically invoke a method from the other native dll. The problem was that I didn't give the method any path, and just the dll filename. In .net, it would've searched in the current folder, but seems like in native code this doesn't work this way.
The bigger problem in my programming practices is obviously the fact that i didn't fully cover error handling in my native C++ dll!
All the asnwers I received on this page weren't for nothing though...
Once I found out that i had problem with the directory path, I ran into different exceptions about trying to access corrupt memory, etc. And then I needed to create pinned objects, and declare a size for my StringBuilder object.
Thanks to everyone for your help!!
:)

Easily porting Lua code to C#

is there any easy way to port Lua code to C#?
The biggest problem would probably be to port tables neatly in some dictionaries.
And to prevent any misunderstanding: no I cannot use embedded Lua in my program.

Code designed in such a highly dynamic language like Lua would need substantial refactoring to make sense in a static language like C#- the two serve fundamentally different purposes. You would have to re-write such code again from scratch, realistically, unless it used only the most basic features of any language, like basic numerical/string ops.

There's no easy way to do that.

Universal-transpiler can translate a small subset of Lua into several other languages, including C#. This is an example, written for SWI-Prolog:
:- use_module(library(transpiler)).
:- set_prolog_flag(double_quotes,chars).
:- initialization(main).
main :-
translate("function add(a,b) return a + b end function squared(a) return a*a end function add_exclamation_point(parameter) return parameter .. \"!\" end",'lua','c#',X),
atom_chars(Y,X),
writeln(Y).
This is the C# source code that it generates:
public static int add(int a,int b){
return a+b;
}
public static int squared(int a){
return a*a;
}
public static string add_exclamation_point(string parameter){
return parameter+"!";
}

What do you want to achieve? Convert the lua files to C# code, where you want to work with them extensively, or you just want some code that does similar things than the original code.
For the first type of conversion the answer is that it is quite hard, but not impossible. You have to parse the code and re-create the same (dynamic) functionality in C#. Frameworks, like LinFu.Reflection can help here, because they will add some dynamic functionality to CLR.
For the second type, my idea is to convert the lua bytecode to C# instead of the original code. This shouldn't be too hard, mainly because lua doesn't have much opcodes (around 30 if I remember it correctly). From these opcodes the hardest to convert are the logic and jump operators (because you don't have goto in C#), but if you keep the flow operators intact (and convert them to C# - that is more or less accomplishable), and only compile the code between, and convert the result bytecode to C# should do the job. Of course this way you'll lose a lot from the readibility of the original code, and maintaining it will be much harder.
You might also try to find a solution between these two edge cases I've written here. Some constructs can be easily ported (mainly the loops, and simple arithmetic operators), but fall back to the opcode representation for table handling.