Develop Reference

Memory allocation/deallocation when working with C# and C++ unmanaged

I am working with some C# and C++ unmanaged code and there are two things I don't understand when dealing with memory. If someone can help me understand:
If a variable is dynamically allocated under C# (using new) and then is passed to the C++ unmanaged code. Does that variable memory needs to be deallocated manually under the C++ unmanaged code by the user ?
If a variable is dynamically allocated under C++ unmanaged (using new) and then passed to C#, is it safe to say the Garbage Collector will deallocate that memory ?

No, since the object is allocated on managed heap GC will handle deallocation as usual. The problem is you must tell him not to deallocate or change address of the object while it is used from unmanaged code because GC can't know how long you are going to use the object from the unmanaged code. This can be done by PINNING the object.
See answer to this question.
No, since the object is allocated on C++ unmanaged heap GC won't touch it. You have to deallocate it yourself using delete.
Edit:
If you need to allocate an object in managed code and deallocate in unmanaged code or vice versa, It's good to know there is OS heap for this purpose that you can use via Marshal.AllocHGlobal and Marshal.FreeHGlobal calls from C#, there will be similar calls in C++.

It's really simple!
Depends
Depends
Eh, Sorry about that.
Under typical conditions, C# will keep track of the memory and get rid of it any time after it's no longer used on the C# side. It has no way of tracking references on the C++ side, so one common mistake in interop is that the memory is deallocated before the unmanaged side is done with it (resulting in loads of FUN). This only applies for cases where the memory is directly referenced, not when its copied (the typical case being a byte[] that's pinned for the duration of the unmanaged call). Don't use automatic marshalling when the life-time of the object/pointer being passed to unmanaged code is supposed to be longer than the run of the invoked method.
Under typical conditions, C# has no way of tracking memory allocations in the C++ code, so you can't rely on automatic memory management. There are exceptions (e.g. some COM scenarios), but you'll almost always need to manage the memory manually. This usually means sending the pointer back to the C++ code to do the deallocation, unless it used a global allocator of some kind (e.g. CoMemoryInitialize). Remember that in the unmanaged world, there is no one memory manager that you can safely invoke to dispose of memory; you don't really have the necessary information anyway.
This only applies to pointers, of course. Passing integers is perfectly fine, and using automatic marshalling usually means the marshaller takes care of most of the subtleties (though still only in the simplest case, so be careful). Unmanaged code is unmanaged - you need to understand perfectly how the memory is allocated, and how, when and who is responsible for cleaning up the memory.

As a rule of thumb, whichever component/object allocates memory should deallocate memory. For every new a delete by the one which did new.
That is the ideal. If not followed for reasons such as you C++ program may terminate and not exists when allocated memory's lifecycle comes to an end, your C# should clean up and visa versa.

Serialize into native memory buffer from C# using flatbuffers

Is it possible with flatbuffers in C# to serialize objects to native (unmanaged) memory buffer?
So I want to do these steps:
Allocate a native memory buffer from native memory
Create objects in C# and serialize them into the allocated buffer
Send this memory buffer to C++ for deserialization
I'm thinking either of some custom memory buffer allocator in C#, or of some way of transferring ownership of a memory buffer form C# to C++.
In general I want to avoid copying memory when sending data from C# to C++ and vice versa. I want this memory buffer to be shared between C# and C++.
How do I do that?

No, the current FlatBuffers implementation is hard-coded to write to a regular byte array. You could copy this array to native memory afterwards, or like #pm100 says, pin it.
All serialization in FlatBuffers goes through an abstraction called the ByteBuffer, so if you made an implementation of that for native memory, it could be used directly relatively easily.

Yes, if you use C++/CLI. Basic data types such as bool, 32-bit int, short, etc are same. For other types check it out msclr::interop::marshal_as<>.
Similar post: C++/CLI Converting from System::String^ to std::string

Multi-threaded managed application that calls native code

I have a service that's using ASP WebApi. Each http request translates to a thread that needs to do some data manipulation (possibly changing the data). The API layer is written in C# and the data manipulation is written in C++. The C# layer calls the native library and supplies a pointer to some managed buffer.
Couple of questions:
How can I make sure there are no races? is std::mutex in the native library enough in this case? (do managed threads map to native threads? will they share the same std::mutex?)
How can I make sure that the GC doesn't release the pointer to the managed buffer while the native library is manipulating it?

Do you need a shared buffer? If the buffer is only ever used on one thread, you save yourself a lot of trouble. Managed threads do not map to native threads 1:1, but I'm not sure if that has any effect on your scenario.
You need to fix the buffer, and keep it fixed the whole time the native code has a pointer to it - releasing is the least of your worries, the .NET memory is moved around all the time. This is done using the fixed block.
Fixing managed memory:
byte[] theBuffer = new byte[256];
fixed (byte* ptr = &theBuffer[0])
{
// The pointer is now fixed - the GC is prohibited from moving the memory
TheNativeFunction(ptr);
}
// Unfixed again
However, note that prohibiting the GC from moving memory around can cause you quite a bit of trouble - it can prevent heap compaction altogether in a high-throughput server, for example.
If you don't need to work with the memory in the managed environment, you can simply allocate unmanaged memory for the task, such as by using Marshal.AllocHGlobal.

C++ DLL crashes when calling from managed code

there's a WinForms-application written in C# using .NET Framework 3.5. This application uses a C++ Dll which is imported using the following declaration:
[DllImport(DllName)]
public static unsafe extern int LoadDBData(String dsn, String userid, String password);
This method imports data from a given ODBC-DSN using a SQL Server database. The call crashes when there is too much data in database. The provider of this extern dll said this happens because the dll is unable to grab more heap size and my application should provide more heap memory.
How could I solve this problem? As far as I know the only possibility to exclude a component from automatic garbage collection is the unsafe keyword which I already used.
Any idea would be appreciated.
Thanks in advance
Martin

This seems like a problem with the vendor's library, rather than your code.
Managed and unmanaged memory should be considered to be completely separate. Managed memory is typically memory allocated on a garbage-collected
heap, while unmanaged memory is anything else: the ANSI C memory pool
allocated through malloc(3), custom memory pools, and
garbage-allocated heaps outside the control of the CLI implementation...
Note that the above quote is from the Mono documentation, but I believe (if I'm not mistaken) the same is true for .NET in general. If the data is being loaded in the DLL's internal data structures, then it should allocate its own memory. If you're providing a buffer which will get filled up with the data, then it will only get filled up with as much data as you've allocated for the buffer (and pinned before marshalling). So where is the data being loaded?

You can't increase the heap size in .NET.
You could create an EXE in c/c++ that your .NET app calls using Process.Start.
Your c/c++ EXE would just call the DLL function and return the result (or if you have more than one function it could take a command line parameter). If you don't want a separate EXE you could try using RunDll32 instead.

I doubt this is specific to .NET, managed memory, garbage collection etc. It's a native DLL so it uses regular, unmanaged memory. Of course, the .NET runtime will also use it's share of memory but a native application using the DLL would do the same.
If you're running in a 32 bit process, the total heap size for .NET and unmanaged code can be limited to 1.5 GB. It's difficult to tell without additional information, but you might have hit that limit.
So one option would be to ask your vendor, whether they have a 64 bit version of the library and switch to a 64 process. In a 64 bit process, memory is almost unlimited (according to today's standard).

Marshalling between C# and C++, and the Juggling of Responsibilities

what if I had a native C++ function in which, depending on the result of the function, the responsibility of deleting a certain pointer (delete[]) differs between the caller and the function. I would of course check for the return value and act accordingly in C++.
Question is, what if the function was marshalled between C++ and C#, will setting the pointer to null in C# be enough?

No. C# can't do what delete[] in C++ does. You'd have to use a shared memory allocation API, or write a C++ wrapper that handles the cleanup.

No, simply setting a pointer allocated in native code to null will not free the memory. The CLR can only garbage collect memory that it knows about (aka managed memory). It has no idea about native memory and hence can't collect it. Any native memory which has ownership in a managed type must be explicitly freed.
The most common way this is done is via the Alloc and Free functions on the Marshal class
http://msdn.microsoft.com/en-us/library/atxe881w.aspx