I am using DllImport to call method in c wrapper library from my own .net class. This method in c dll creates a string variable and returns the pointer of the string.
Something like this;
_declspec(dllexport) int ReturnString()
{
char* retval = (char *) malloc(125);
strcat(retval, "SOMETEXT");
strcat(retval, "SOMETEXT MORE");
return (int)retval;
}
Then i read the string using Marshall.PtrToStringAnsi(ptr). After i get a copy of the string, i simply call another c method HeapDestroy which is in c wrapper library that calls free(ptr).
Here is the question;
Recently while it is working like a charm, I started to get "Attempted to read or write protected memory area" exception. After a deeper analysis, i figured out, i beleive, although i call free method for this pointer, value of the pointer is not cleared, and this fills the heap unattended and makes my iis worker process to throw this exception. By the way, it is an web site project that calls this method in c library.
Would you kindly help me out on this issue?
Sure, here is C# code;
[DllImport("MyCWrapper.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
private extern static int ReturnString();
[DllImport("MyCWrapper.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
private extern static void HeapDestroy(int ptr);
public static string GetString()
{
try
{
int i = ReturnString();
string result = String.Empty;
if (i > 0)
{
IntPtr ptr = new IntPtr(i);
result = Marshal.PtrToStringAnsi(ptr);
HeapDestroy(i);
}
return result;
}
catch (Exception e)
{
return String.Empty;
}
}
What may be the problem is the underlying C code. You are not adding a NULL terminator to the string which strcat relies on (or checking for a NULL return from malloc). It's easy to get corrupted memory in that scenario. You can fix that by doing the following.
retval[0] = '\0';
strcat(retval, "SOMETEXT");
Also part of the problem is that you are playing tricks on the system. It's much better to write it correctly and let the system work on correctly functioning code. The first step is fixing up the native code to properly return the string. One thing you need to consider is that only certain types of memory can be natively freed by the CLR (HGlobal and CoTask allocations). So lets change the function signature to return a char* and use a different allocator.
_declspec(dllexport) char* ReturnString()
{
char* retval = (char *) CoTaskMemAlloc(125);
retval[0] = '\0';
strcat(retval, "SOMETEXT");
strcat(retval, "SOMETEXT MORE");
return retval;
}
Then you can use the following C# signature and free the IntPtr with Marshal.FreeCoTaskMem.
[DllImport("SomeDll.dll")]
public static extern IntPtr ReturnString();
Even better though. When marshalling, if the CLR ever thinks it needs to free memory it will use FreeCoTaskMem to do so. This is typically relevant for string returns. Since you allocated the memory with CoTaskMemAlloc you can save yourself the marshalling + freeing steps and do the following
[DllImport("SomeDll.dll", CharSet=Ansi)]
public static extern String ReturnString();
Freeing memory doesn't clear it, it just frees it up so it can be re-used. Some debug builds will write over the memory for you to make it easier to find problems with values such as 0xBAADFOOD
Callers should allocate memory, never pass back allocated memory:
_declspec(dllexport) int ReturnString(char*buffer, int bufferSize)
{
if (bufferSize < 125) {
return 125;
} else {
strcat(buffer, "SOMETEXT");
strcat(buffer, "SOMETEXT MORE");
return 0;
}
}
Although memory is allocated by the DLL in the same heap as your application, it MAY be using a different memory manager, depending on the library it was linked with. You need to either make sure you're using the same exact library, or add code to release the memory that the DLL allocates, in the DLL code itself.
I have a c# code which captures a fingerprint from a device. The code for capturing is written in c++ dll which returns the image through callback side by side to give live view. But the problem is that the application stops working after some time during capture. This happens because of callback method only. If I comment the code in callback method it works fine.
I want to know if the callback method can be put in another thread so that the coming images do not get crashed in callback method.
Following is the way how I have called the method.
/* c# code: */
[DllImport("abc.dll", EntryPoint = "capture", CharSet = CharSet.Auto, ExactSpelling = false, SetLastError = true, CallingConvention = CallingConvention.Cdecl)]
[return: MarshalAs(UnmanagedType.I4)]
static extern int capture(callback liveImageCallback);
private delegate void callback(IntPtr img);
void capture()
{
int ret= capture(callBackLiveImage);
}
public void callBackLiveImage(IntPtr imgPointer)
{
byte[] rawData = new byte[102400];
Marshal.Copy(imgPointer, rawData, 0, 102400);
}
/* c++ code: */
typedef void (__stdcall * Callback)(unsigned char* img);
Callback LiveImageHandler;
void capture(liveCallback)
{
LiveImageHandler = liveCallback;
someMethod();
}
someMethod()
{
LiveImageHandler(raw_img);
}
You have no error handling in the callBackLiveImage method. You're always copying 102400. My unmanaged sense is seeing an unhandled exception like AccessViolationException or such.
Try using a try-catch statement to catch the exception, and log it:
public void callBackLiveImage(IntPtr imgPointer)
{
try
{
byte[] rawData = new byte[102400];
Marshal.Copy(imgPointer, rawData, 0, 102400);
}
catch (Exception ex)
{
Debug.WriteLine(ex.ToString());
}
}
Unhandled .NET exceptions in unmanaged code don't really sound like a good idea :)
The specific exception will give you a better idea of what you're dealing with. It might be that the imgPointer is IntPtr.Zero, or it might be that there's less data than you expected. Also, are you sure the C++ library is taking care of disposing of the memory?
We usually throw exception when invalid input is passed to a method or when a object is about to enter invalid state. Let's consider the following example
private void SomeMethod(string value)
{
if(value == null)
throw new ArgumentNullException("value");
//Method logic goes here
}
In the above example I inserted a throw statement which throws ArgumentNullException. My question is how does runtime manages to throw ThreadAbortException. Obviously it is not possible to use a throw statement in all the methods, even runtime manages to throw ThreadAbortException in our custom methods too.
I was wondering how do they do it?
I was curious to know what is happening behind the scenes, I opened a reflector to open Thread.Abort and end up with this
[MethodImplAttribute(MethodImplOptions.InternalCall)]
private extern void AbortInternal();//Implemented in CLR
Then I googled and found this How does ThreadAbortException really work. This link says that runtime posts APC through QueueUserAPC function and that's how they do the trick.
I wasn't aware of QueueUserAPC method I just gave a try to see whether it is possible with some code. Following code shows my try.
[DllImport("kernel32.dll")]
static extern uint QueueUserAPC(ApcDelegate pfnAPC, IntPtr hThread, UIntPtr dwData);
delegate void ApcDelegate(UIntPtr dwParam);
Thread t = new Thread(Threadproc);
t.Start();
//wait for thread to start
uint result = QueueUserAPC(APC, new IntPtr(nativeId), (UIntPtr)0);//returns zero(fails)
int error = Marshal.GetLastWin32Error();// error also zero
private static void APC(UIntPtr data)
{
Console.WriteLine("Callback invoked");
}
private static void Threadproc()
{
//some infinite loop with a sleep
}
If am doing something wrong forgive me, I have no idea how to do it. Again back to question, Can somebody with knowledge about this or part of CLR team explain how it works internally?
If APC is the trick runtime follows what am doing wrong here?
Are you sure you read the page you were pointing to? In the end it boils down to:
The call to Thread.Abort boils down to .NET setting a flag on a thread to be aborted and then checking that flag during certain points in the thread’s lifetime, throwing the exception if the flag is set.
To get your APC callback to work, you need a thread handle (which is not the same as the thread ID). I've also updated the attributes on the PInvokes.
Also keep in mind that the thread needs to be in an "alert-able" wait state in order for the APC to be called (which Thread.Sleep will give us). So if the thread is busy doing stuff, it may not be called.
[DllImport("kernel32.dll", EntryPoint = "GetCurrentThread", CallingConvention = CallingConvention.StdCall)]
public static extern IntPtr GetCurrentThread();
[DllImport("kernel32.dll", EntryPoint = "QueueUserAPC", CallingConvention = CallingConvention.StdCall, SetLastError = true)]
public static extern uint QueueUserAPC(ApcDelegate pfnAPC, IntPtr hThread, UIntPtr dwData);
[UnmanagedFunctionPointerAttribute(CallingConvention.StdCall)]
public delegate void ApcDelegate(UIntPtr dwParam);
[DllImport("kernel32.dll", EntryPoint = "DuplicateHandle", CallingConvention = CallingConvention.StdCall, SetLastError = true)]
public static extern bool DuplicateHandle([In] System.IntPtr hSourceProcessHandle, [In] System.IntPtr hSourceHandle, [In] System.IntPtr hTargetProcessHandle, out System.IntPtr lpTargetHandle, uint dwDesiredAccess, [MarshalAsAttribute(UnmanagedType.Bool)] bool bInheritHandle, uint dwOptions);
[DllImport("kernel32.dll", EntryPoint = "GetCurrentProcess", CallingConvention = CallingConvention.StdCall, SetLastError = true)]
public static extern IntPtr GetCurrentProcess();
static IntPtr hThread;
public static void SomeMethod(object value)
{
DuplicateHandle(GetCurrentProcess(), GetCurrentThread(), GetCurrentProcess(), out hThread, 0, false, 2);
while (true)
{
Console.WriteLine(".");
Thread.Sleep(1000);
}
}
private static void APC(UIntPtr data)
{
Console.WriteLine("Callback invoked");
}
static void Main(string[] args)
{
Console.WriteLine("in Main\n");
Thread t = new Thread(Program.SomeMethod);
t.Start();
Thread.Sleep(1000); // wait until the thread fills out the hThread member -- don't do this at home, this isn't a good way to synchronize threads...
uint result = QueueUserAPC(APC, hThread, (UIntPtr)0);
Console.ReadLine();
}
Edit:
How the CLR injects the exception
Given this loop for the thread function:
while (true)
{
i = ((i + 7) * 3 ^ 0x73234) & 0xFFFF;
}
I then .Aborted the thread and looked at the native stack trace
...
ntdll!KiUserExceptionDispatcher
KERNELBASE!RaiseException
clr!RaiseComPlusException
clr!RedirectForThrowControl2
clr!RedirectForThrowControl_RspAligned
clr!RedirectForThrowControl_FixRsp
csTest.Program.SomeMethod(System.Object)
...
Looking at the return address of the RedirectForThrowControl_FixRsp call, it is pointing into the middle of my loop, for which there are no jumps or calls:
nop
mov eax,dword ptr [rbp+8]
add eax,7 // code flow would return to execute this line
lea eax,[rax+rax*2]
xor eax,73234h
and eax,0FFFFh
mov dword ptr [rbp+8],eax
nop
mov byte ptr [rbp+18h],1
jmp 000007fe`95ba02da // loop back to the top
So apparently the CLR is actually modifying the instruction pointer of the thread in question to physically yank control from the normal flow. They obviously needed to supply several wrappers to fixup and restore all the stack registers to make it work correctly (thus the aptly named _FixRsp and _RspAligned APIs.
In a separate test, I just had Console.Write() calls within my thread loop, and there it looked like the CLR injected a test just before the physical call out to WriteFile:
KERNELBASE!RaiseException
clr!RaiseTheExceptionInternalOnly
clr! ?? ::FNODOBFM::`string'
clr!HelperMethodFrame::PushSlowHelper
clr!JIT_RareDisableHelper
mscorlib_ni!DomainNeutralILStubClass.IL_STUB_PInvoke(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte*, Int32, Int32 ByRef, IntPtr)
mscorlib_ni!System.IO.__ConsoleStream.WriteFileNative(Microsoft.Win32.SafeHandles.SafeFileHandle, Byte[], Int32, Int32, Boolean)
To get the QueueUserAPC to work you have to do two things.
Acquire the target thread handle. Note that this is not the same thing as the native thread id.
Allow the target thread to go into an alertable state.
Here is a complete program that demonstrates this.
class Program
{
[DllImport("kernel32.dll", EntryPoint = "DuplicateHandle", CallingConvention = CallingConvention.StdCall, SetLastError = true)]
public static extern bool DuplicateHandle([In] System.IntPtr hSourceProcessHandle, [In] System.IntPtr hSourceHandle, [In] System.IntPtr hTargetProcessHandle, out System.IntPtr lpTargetHandle, uint dwDesiredAccess, [MarshalAsAttribute(UnmanagedType.Bool)] bool bInheritHandle, uint dwOptions);
[DllImport("kernel32.dll", EntryPoint = "GetCurrentProcess", CallingConvention = CallingConvention.StdCall, SetLastError = true)]
public static extern IntPtr GetCurrentProcess();
[DllImport("kernel32.dll")]
private static extern IntPtr GetCurrentThread();
[DllImport("kernel32.dll")]
private static extern uint QueueUserAPC(ApcMethod pfnAPC, IntPtr hThread, UIntPtr dwData);
private delegate void ApcMethod(UIntPtr dwParam);
static void Main(string[] args)
{
Console.WriteLine("Main: " + Thread.CurrentThread.ManagedThreadId);
IntPtr threadHandle = IntPtr.Zero;
var threadHandleSet = new ManualResetEvent(false);
var apcSet = new ManualResetEvent(false);
var thread = new Thread(
() =>
{
Console.WriteLine("thread started");
threadHandle = GetCurrentThread();
DuplicateHandle(GetCurrentProcess(), GetCurrentThread(), GetCurrentProcess(), out threadHandle, 0, false, 2);
threadHandleSet.Set();
apcSet.WaitOne();
for (int i = 0; i < 10; i++)
{
Console.WriteLine("thread waiting");
Thread.Sleep(1000);
Console.WriteLine("thread running");
}
Console.WriteLine("thread finished");
});
thread.Start();
threadHandleSet.WaitOne();
uint result = QueueUserAPC(DoApcCallback, threadHandle, UIntPtr.Zero);
apcSet.Set();
Console.ReadLine();
}
private static void DoApcCallback(UIntPtr dwParam)
{
Console.WriteLine("DoApcCallback: " + Thread.CurrentThread.ManagedThreadId);
}
}
This essentially allows a developer to inject the execution of a method into any arbitrary thread. The target thread does not have to have a message pump like would be necessary for the traditional approach. One problem with this approach though is that the target thread has to be in an alertable state. So basically the thread must call one of the canned .NET blocking calls like Thread.Sleep, WaitHandle.WaitOne, etc. for the APC queue to execute.
I downloaded the SSCLI code and started poking around. The code is difficult for me to follow (mostly because I am not a C++ or ASM expert), but I do see a lot of hooks where the aborts are injected semi-synchronously.
try/catch/finally/fault block flow control processing
GC activations (allocating memory)
proxied through soft interrupts (like with Thread.Interrupt) when in an alertable state
virtual call intercepts
JIT tail call preparations
unmanaged to managed transitions
That is just to name a few. What I wanted to know was how asynchronous aborts were injected. The general idea of hijacking the instruction pointer is part of how it happens. However, it is far more complex than what I described above. It does not appear that a Suspend-Modify-Resume idiom is always used. From the SSCLI code I can see that it does suspend and resume the thread in certain scenarios to prepare for the hijack, but this is not always the case. It looks to me that the hijack can occur while the thread is running full bore as well.
The article you linked to mentions that an abort flag is set on the target thread. This is technically correct. The flag is called TS_AbortRequested and there is a lot of logic that controls how this flag is set. There are checks for determining if a constrained execution region exists and whether the thread is currently in a try-catch-finally-fault block. Some of this work involves a stack crawl which means the thread must be suspended and resumed. However, how the change of the flag is detected is where the real magic happens. The article does not explain that very well.
I already mentioned several semi-synchronous injection points in the list above. Those should be pretty trivial to understand. But, how does the asynchronous injection happen exactly? Well, it appears to me that the JIT is the wizard behind by the curtain here. There is some kind of polling mechanism built into the JIT/GC that periodically determines if a collection should occur. This also provides an opportunity to check to see if any of the managed threads have changed state (like having the abort flag set). If TS_AbortRequested is set then the hijack happens then and there.
If you are looking at the SSCLI code here are some good functions to look at.
HandleThreadAbort
CommonTripThread
JIT_PollGC
JIT_TailCallHelper
COMPlusCheckForAbort
ThrowForFlowControl
JIT_RareDisableHelper
There are many other clues. Keep in mind that this is the SSCLI so the method names may not match exactly with call stacks observed in production (like what Josh Poley discovered), but there will be similarities. Also, a lot of the thread hijacking is done with assembly code so it is hard to follow at times. I highlighted JIT_PollGC because I believe this is where the interesting stuff happens. This is the hook that I believe the JIT will dynamically and strategically place into the executing thread. This is basically the mechanism for how those tight loops can still receive the abort injections. The target thread really is essentially polling for the abort request, but as part of a larger strategy to invoke the GC1
So clearly the JIT, GC, and thread aborts are intimately related. It is obvious when you look at the SSCLI code. As an example, the method used to determine the safe points for thread aborts is the same as the one used to determine if the GC is allowed to run.
1Shared Source CLI Essentials, David Stutz, 2003, pg. 249-250
It's easy, the underlying OS does it. If the thread is in any state except 'running on another core', there is no problem - it's state is set to 'never run again'. If the thread is runing on another core, the OS hardware-interrupts the other core via. it's interprocessor driver and so exterminates the thread.
Any mention of 'time-slice', 'quantum' etc. is just.....
Consider the following code which I recently changed to use FileStream.SafeFileHandle:
public static void FastWrite<T>(FileStream fs, T[] array, int offset, int count) where T: struct
{
int sizeOfT = Marshal.SizeOf(typeof(T));
GCHandle gcHandle = GCHandle.Alloc(array, GCHandleType.Pinned);
try
{
uint bytesWritten;
uint bytesToWrite = (uint)(count * sizeOfT);
var overlapped = new NativeOverlapped();
if
(
!WriteFile
(
fs.SafeFileHandle,
new IntPtr(gcHandle.AddrOfPinnedObject().ToInt64() + (offset*sizeOfT)),
bytesToWrite,
out bytesWritten,
ref overlapped
)
)
{
throw new IOException("Unable to write file.", new Win32Exception(Marshal.GetLastWin32Error()));
}
Debug.Assert(bytesWritten == bytesToWrite);
GC.KeepAlive(fs); // <--- Is this really not necessary?
}
finally
{
gcHandle.Free();
}
}
[DllImport("kernel32.dll", SetLastError=true)]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool WriteFile
(
SafeFileHandle hFile,
IntPtr lpBuffer,
uint nNumberOfBytesToWrite,
out uint lpNumberOfBytesWritten,
ref NativeOverlapped lpOverlapped
);
I had previously added the GC.KeepAlive(fs) to ensure that the FileStream won't be garbage-collected until after the Windows API WriteFile() call has returned.
However, after changing to use SafeFileHandle Code Analysis now tells me that it isn't necessary with warning CA2004: Remove calls to GC.KeepAlive:
If you are converting to SafeHandle usage, remove all calls to GC.KeepAlive (object).
I've consulted the documentation on FileStream.SafeFileHandle but it is unclear to me if it really is safe for me to remove the call to GC.KeepAlive().
Is it definitely safe to remove it? And am I using it correctly?
Also, can anyone point me at some decent documentation on the use of SafeHandle?
The point of using SafeHandle is that the handle won't be closed while the WriteFile() function is executing. Which is really what you want to achieve here. Do note however that the FileStream object may still be finalized. There are no obvious consequences from that happening in the posted code. So the FxCop warning is appropriate.
Do note the strings attached to using code like this. It is very unlikely to be any faster than FileStream.Write(). But you do add risk by not properly dealing with edge conditions. Including using Overlapped but not appropriately handling overlapped I/O, don't do that. If overlapped I/O is actually intended then check this answer for the way that's optimized in the CLR beyond what GCHandle can do. Have a good look at the Reference Source source code for FileStream, particularly focus on the _isAsync field and the error handling for ERROR_NO_DATA and ERROR_INVALID_HANDLE.
And you'll also see it using SafeFileHandle and not using GC.KeepAlive(), like FxCop demands.
I have the following function written in C#
public static string GetNominativeDeclension(string surnameNamePatronimic)
{
if(surnameNamePatronimic == null)
throw new ArgumentNullException("surnameNamePatronimic");
IntPtr[] ptrs = null;
try
{
ptrs = StringsToIntPtrArray(surnameNamePatronimic);
int resultLen = MaxResultBufSize;
int err = decGetNominativePadeg(ptrs[0], ptrs[1], ref resultLen);
ThrowException(err);
return IntPtrToString(ptrs, resultLen);
}
catch
{
return surnameNamePatronimic;
}
finally
{
FreeIntPtr(ptrs);
}
}
Function decGetNominativePadeg is in unmanaged dll
[DllImport("Padeg.dll", EntryPoint = "GetNominativePadeg")]
private static extern Int32 decGetNominativePadeg(IntPtr surnameNamePatronimic,
IntPtr result, ref Int32 resultLength);
and throws an exception:
Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
The catch that is in C# code doesn't actually catch it. Why? How to handle this exception?
Thank you for your help!
"The CLR no longer delivers exceptions for corrupted process state to exception handlers in managed code."
.NET Framework 4 Migration Issues.
Just add this to the config file:
http://msdn.microsoft.com/en-us/library/dd638517.aspx
If the IntPtr result parameter is to receive the value from within the function, it must be marked ref.
I don't see ptrs[1] being assigned any value before passing.
Try changing the definition to:
[DllImport("Padeg.dll", EntryPoint = "GetNominativePadeg")]
private static extern Int32 decGetNominativePadeg(IntPtr surnameNamePatronimic,
**ref** IntPtr result, ref Int32 resultLength);
The reason probably is that it's trying to write to "result" which is marked as input-only.
you have probably turned off debugging of unmanaged code.
"Enable unmanaged code debugging option" must be check in project properties under the Debug section. After this, the exception is shown in the debugging process.