Short version:
Could passing the handle from GlobalAlloc(GMEM_MOVEABLE, Size) to Marshal.PtrToStructure() and Marshal.FreeHGlobal() cause memory corruption?
Long version:
I'm using Windows Global memory allocation to pass a data structure back and forth between a Delphi and C# application (the fact that it's Delphi isn't really significant to this question, because it's just Win32 API calls).
On the Delphi side, I pass in a record, it allocates the space, locks the memory, copies the structure into memory, and the unlocks the memory:
function MarshalRec(SourceRec: TInteropItemRec): THandle;
var
Size: integer;
Buffer: Pointer;
begin
Size := sizeof(SourceRec);
result := GlobalAlloc(GMEM_MOVEABLE and GMEM_ZEROINIT, Size);
Buffer := GlobalLock(result);
try
CopyMemory(Buffer, #SourceRec, Size);
finally
GlobalUnlock(result);
end;
end;
On the C# side, it gets that THandle (which is basically an unsigned int) into an IntPtr and uses Marshal.PtrToStructure to copy the data into the C# structure:
public void FromMemory(IntPtr Source)
{
Marshal.PtrToStructure(Source, this);
Marshal.FreeHGlobal(Source);
}
The problem I'm running into, is very rarely (as in 4 times over 6 months for me), the whole application goes down (This application has encountered an error and must close). If I try to pause execution in Visual Studio, I get "A fatal error has occurred and debugging needs to be terminated. For more details, please see the Microsoft Help and Support web site. HRESULT=0x80131c08."
Anyway, we managed to get a couple logs of it happening, and in both cases, it showed a recent call to that "MarshalRec" function above, a few other function calls, and then some processing of Windows messages on the event loop on the Delphi thread (yeah, it has its own thread and event loop to deal with a time sensitive device driver).
So my suspicion is falling on the GMEM_MOVEABLE flag to GlobalAlloc. I couldn't find anything in the Marshal class that does the GlobalLock and GlobalUnlock stuff, so I had assumed it was handled internally by PtrToStructure().
Does PtrToStructure deal properly with a handle, or does it need an actual pointer obtained from GlobalLock()? Is it possible that in very rare cases, Windows happens to move the memory I allocated, which means I need to call GlobalLock() to get an actual pointer to pass in? That FreeHGlobal is actually freeing something it shouldn't have, which brings down the whole app the next time that resource is accessed?
And if so, should changing the GMEM_MOVABLE to GMEM_FIXED prevent this from happening again? Or do I need to DllImport GlobalLock() and GlobalUnlock()?
I'm tempted to just blindly make those changes, but given the non-reproducibility of this issue, there's no way to tell if it's fixed until it happens again. So I'm looking for feedback on whether this code could lead to the symptoms I'm seeing, or if I need to start coming up with other theories.
Well, you are explicitly violating the contract for GlobalAlloc(). Your hope that Marshal.PtrToStructure() will call GlobalLock is unfounded, it has no way to tell whether the passed IntPtr is a handle or a pointer.
GlobalAlloc is a fairly hopelessly outdated legacy function from the Windows 3.x era. Yes, it is quite likely to return the address as the handle value with GlobalLock() being a no-op. But it is certainly not documented to do this. CoTaskMemAlloc() is the better mouse trap.
Related
I used native method call in C# with DllImport feature. I want to know that should I release memory for method paremeters manually in native-side.
Currently, I send double[] array to native method, and native method get paretmers as double* type. Should I release double* in native method?
No, you should let .NET handle the memory management itself. The native code marshaller follows the basic rules for COM interop, which also happen to work most of the time for P/Invoke, since Win32 also follows the rules. (There are exceptions, but they'd be called out in the Windows API documentation).
Since you wrote both ends of the P/Invoke call, you should follow those same rules to make your life easy. As far as memory allocations are concerned, most of the time the caller is responsible for freeing any memory that crosses the P/Invoke boundary, since the callee doesn't know if/when it's safe to do so. This includes:
If you allocate memory for a parameter and pass it in
If there is an out parameter or return value that is allocated by the callee and returned
In both cases, only the caller knows when the memory is no longer needed and is safe to be freed. In the case of a P/Invoke call, the run-time marshaller knows this, and it will allocate memory to marshal your double[] to a double * before it makes the call, then free that memory when the call returns. Depending on the combination of ref, out, [In] or [Out] attributes, it may or may not try to copy the data back into your double[], but it will always free that memory.
I tried to find out how pinned pointers defined with fixed keyword work. My idea was that internally GCHandle.Alloc(object, GCHandleType.Pinned) was used for that. But when I looked into the IL generated for the following C# code:
unsafe static void f1()
{
var arr = new MyObject[10];
fixed(MyObject * aptr = &arr[0])
{
Console.WriteLine(*aptr);
}
}
I couldn't find any traces of GCHandle.
The only hint I saw that the pinned pointer was used in the method was the following IL declaration:
.locals init ([0] valuetype TestPointerPinning.MyObject[] arr,
[1] valuetype TestPointerPinning.MyObject& pinned aptr)
So the pointer was declared as pinned, and that did not require any additional methods calls, to pin it.
My questions are
Is there any difference between using pinned pointers in the declaration and pinning the pointer by using GCHandle class?
Is there any way to declare a pinned pointer in C# without using fixed keyword? I need this to pin a bunch of pointers within a loop and there's no way I can do this using fixed keyword.
Well, sure there's a difference, you saw it. The CLR supports more than one way to pin an object. Only the GCHandleType.Pinned method is directly exposed to user code. But there are others, like "async pinned handles", a feature that keeps I/O buffers pinned while a driver performs an overlapped I/O operation. And the one that the fixed keyword uses, it doesn't use an explicit handle or method call at all. These extra ways were added to make unpinning the objects again as quick and reliable as possible, very important to GC health.
Fixed buffer pins are implemented by the jitter. Which performs two important jobs when it translates MSIL to machine code, the highly visible one is the machine code itself, you can easily see it with the debugger. But it also generates a data structure used by the garbage collector, completely invisible in the debugger. Required by the GC to reliably find object references back that are stored in the stack frame or a CPU register. More about that data structure in this answer.
The jitter uses the [pinned] attribute on the variable declaration in the metadata to set a bit in that data structure, indicating that the object that's referenced by the variable is temporarily pinned. The GC sees this and knows to not move the object. Very efficient because it doesn't require an explicit method call to allocate the handle and doesn't require any storage.
But no, these tricks are not available otherwise to C# code, you really do need to use the fixed keyword in your code. Or GCHandle.Alloc(). If you are finding yourself getting lost in the pins then high odds that you ought to be considering pinvoke or C++/CLI so you can easily call native code. The temporary pins that the pinvoke marshaller uses to keep objects stable while the native code is running are another example of automatic pinning that doesn't require explicit code.
.NET 4.5, 64bit Win8
I have two MSMQ running under WCF and hosted under Windows Activation Service.
Execute MSMQ: Responsible for calling an unmanaged function and obtaining a pointer IntPtr to 4GB array in addition to other limited size variables. This array is not that important and shouldn't be blocking the queue until it is passed to the DB and having this queue free is a priority. Passing the pointer to the Save MSMQ (Hypothesis).
Save MSMQ: Should save the array to the DB across the network then deallocate it from unmanaged memory.
The problem here would be:
Making sure the array is deallocated on poison messages and other queue errors. This case is the easy bit.
Passing an IntPtr cross app domains which might not be possible, as I recall, from the old good days, that pointer values are relative (and the word offsetting comes to mind, or maybe I am mixing processes and AppDomains) to app domains and for security reasons .NET might not allow this? Is this possible or am I dreaming, also is this a cross domain issue or does my problem have a different title?
N.B. I have used NetPipes communication before, but this has a queue in the middle, so NetPipes would not be a good solution.
I have a Windows Forms application running under .NET 4.0. This application imports a DLL which is available for:
32 bit
64 bit
Here is my code snippet:
[DllImport("my64Bit.dll"), EntryPoint="GetLastErrorText"]
private static extern string GetLastErrorText();
// Do some stuff...
string message = GetLastErrorText();
When calling this function (compiled for x64) the application just crashes. I can't even see any debug message in Visual Studio 2012. The identical code with the 32-bit-DLL (compiled for x86) works fine. The prototype is:
LPCSTR APIENTRY GetLastErrorText()
Unfortunately I don't have any further information about the DLL as it is a third-party product.
The function signature is quite troublesome. Whether your code will crash depends on what operating system you run. Nothing happens on XP, an AccessViolation exception is thrown on Vista and later.
At issue is that C functions returning strings need to typically do so by returning a pointer to a buffer that stores a string. That buffer needs to be allocated from the heap and the caller needs to release that buffer after using the string. The pinvoke marshaller implements that contract, it calls CoTaskMemFree() on the returned string pointer after converting it to a System.String.
That invariably turns out poorly, a C function almost never uses CoTaskMemAlloc() to allocate the buffer. The XP heap manager is very forgiving, it simply ignores bad pointers. Not the later Windows versions, they intentionally generate an exception. A strong enabler for the "Vista sucks" label btw, it took a while for programmers to get their pointer bugs fixed. If you have unmanaged debugging enabled then you'll get a diagnostic from the heap manager which warns that the pointer is invalid. Very nice feature but unmanaged debugging is invariably disabled when you debug managed code.
You can stop the pinvoke marshaller from trying to release the string by declaring the return value as IntPtr. You then have to marshal the string yourself with Marshal.PtrToStringAnsi() or one of its friends.
You still have the problem of having to release the string buffer. There is no way to do this reliably, you cannot call the proper deallocator. The only hope you have is that the C function actually returns a pointer to a string literal, one that's stored in the data segment and should not be released. That might work for a function that returns an error string, provided it doesn't implement anything fancy like localization. The const char* return type is encouraging.
You will need to test this to make sure there is no memory leak from not releasing the string buffer. Easy to do, call this function a billion times in a loop. If you don't get IntPtr.Zero as a return value and the program doesn't otherwise fall over with a out-of-memory exception then you're good. For 64-bit pinvoke you'll need to keep an eye on the test program's memory consumption.
Found it. The native function returns LPCSTR, i.e. the C# function cannot return a string. Instead an IntPtr must be returned like this:
[DllImport("my64Bit.dll"), EntryPoint="GetLastErrorText"]
private static extern IntPtr GetLastErrorText();
// Do some stuff...
IntPtr ptr = GetLastErrorText();
string s = Marshal.PtrToStringAnsi(ptr);
I understand that the CLR needs to do marshaling in some cases, but let's say I have:
using System.Runtime.InteropServices;
using System.Security;
[SuppressUnmanagedCodeSecurity]
static class Program
{
[DllImport("kernel32.dll", SetLastError = false)]
static extern int GetVersion();
static void Main()
{
for (; ; )
GetVersion();
}
}
When I break into this program with a debugger, I always see:
Given that there is no marshaling that needs to be done (right?), could someone please explain what's actually happening in this "managed-to-native transition", and why it is necessary?
First the call stack needs to be set up so that a STDCALL can happen. This is the calling convention for Win32.
Next the runtime will push a so called execution frame. There are many different types of frames: security asserts, GC protected regions, native code calls, ...
The runtime uses such a frame to track that currently native code is running. This has implications for a potentially concurrent garbage collection and probably other stuff. It also helps the debugger.
So not a lot is happening here actually. It is a pretty slim code path.
Besides the marshaling layer, which is responsible for converting parameters for you and figuring out calling conventions, the runtime needs to do a few other things to keep internal state consistent.
The security context needs to be checked, to make sure the calling code is allowed to access native methods. The current managed stack frame needs to be saved, so that the runtime can do a stack walk back for things like debugging and exception handling (not to mention native code that calls into a managed callback). Internal bits of state need to be set to indicate that we're currently running native code.
Additionally, registers may need to be saved, depending on what needs to be tracked and which are guaranteed to be restored by the calling convention. GC roots that are in registers (locals) might need to be marked in some way so that they don't get garbage collected during the native method.
So mainly it's stack handling and type marshaling, with some security stuff thrown in. Though it's not a huge amount of stuff, it will represent a significant barrier against calling smaller native methods. For example, trying to P/Invoke into an optimized math library rarely results in a performance win, since the overhead is enough to negate any of the potential benefits. Some performance profiling results are discussed here.
I realise that this has been answered, but I'm surprised that no one has suggested that you show the external code in the debug window. If you right click on the [Native to Managed Transition] line and tick the Show External Code option, you will see exactly which .NET methods are being called in the transition. This may give you a better idea. Here is an example:
I can't really see much that'd be necessary to do. I suspect that it is mainly informative, to indicate to you that part of your call stack shows native functions, and also to indicate that the IDE and debugger may behave differently across that transition (since managed code is handled very differently in the debugger, and some features you expect may not work)
But I guess you should be able to find out simply by inspecting the disassembly around the transition. See if it does anything unusual.
Since you are calling a dll. it needs to go out of the managed environment. It is going into windows core. You are breaking the .net barrier and going into windows code that doesn't run the same as .NET.