.NET 4.5, Win 8, 64bit
I have an IntPtr to 4 GB unmanaged array. I want to pass it async to a WCF NetPipes Windows Service, which is supposed to read it, push it to DB then deallocate it.
I am aware that the caller and the callee are on two different appdomains, so this might not be possible.
Is this possible and how?
Ofcourse, I can copy the data to a managed array then pass the managed array to the service, but this is not good performance-wise.
N.B. I asked a question of how to pass an IntPtr on a queue Passing an unmanaged pointer between two AppDomains (By indirect call) , the answer to this question and the other question will help me taking a design decision.
Related
Okay, I messed something up. I've written in C++ a DLL which I call from the managed code (C# .NET). The library works like diamonds and is blazingly fast.
My DLL uses its internal state i.e. allocates heaps of memory and uses myriad of variables which are not cleared off between the calls from .NET. Instead they stay there and C# code is aware of that (there is preprocessing and building data structures), actually this is required for performance.
So what is the problem?
I want to add multi-threading, effectively by allowing each .NET thread access his own DLL. Without storing any data between the calls it would be easy achievable with just one DLL.
But in my case, do I have to copy the *.DLL the number of times equal to the number of threads and write pInvoke wrapper for each file separately?? :O I mean [DllImport(...)] for each out of like 40 functions?
No way, there must be something more clever. Help?
Simply put you need to stop sharing variables between threads.
Your global variables are the problem. Instead you need each different thread to have its own copy of the state that persists between calls. Typically you would put this state into a structure of some sort, perhaps a struct. Then an initial call to the DLL would return a new instance of this structure. You then pass that structure back into the DLL every time you call a function that requires access to the persistent state. When you are done, call back into the DLL to deallocate the structure. You don't need to declare the structure in the managed code. You can just treat it as an opaque pointer. Use IntPtr.
Of course, perhaps you'd just be better off with a C++/CLI assembly.
.NET 4.5, 64bit Win8
I have two MSMQ running under WCF and hosted under Windows Activation Service.
Execute MSMQ: Responsible for calling an unmanaged function and obtaining a pointer IntPtr to 4GB array in addition to other limited size variables. This array is not that important and shouldn't be blocking the queue until it is passed to the DB and having this queue free is a priority. Passing the pointer to the Save MSMQ (Hypothesis).
Save MSMQ: Should save the array to the DB across the network then deallocate it from unmanaged memory.
The problem here would be:
Making sure the array is deallocated on poison messages and other queue errors. This case is the easy bit.
Passing an IntPtr cross app domains which might not be possible, as I recall, from the old good days, that pointer values are relative (and the word offsetting comes to mind, or maybe I am mixing processes and AppDomains) to app domains and for security reasons .NET might not allow this? Is this possible or am I dreaming, also is this a cross domain issue or does my problem have a different title?
N.B. I have used NetPipes communication before, but this has a queue in the middle, so NetPipes would not be a good solution.
there's a WinForms-application written in C# using .NET Framework 3.5. This application uses a C++ Dll which is imported using the following declaration:
[DllImport(DllName)]
public static unsafe extern int LoadDBData(String dsn, String userid, String password);
This method imports data from a given ODBC-DSN using a SQL Server database. The call crashes when there is too much data in database. The provider of this extern dll said this happens because the dll is unable to grab more heap size and my application should provide more heap memory.
How could I solve this problem? As far as I know the only possibility to exclude a component from automatic garbage collection is the unsafe keyword which I already used.
Any idea would be appreciated.
Thanks in advance
Martin
This seems like a problem with the vendor's library, rather than your code.
Managed and unmanaged memory should be considered to be completely separate. Managed memory is typically memory allocated on a garbage-collected
heap, while unmanaged memory is anything else: the ANSI C memory pool
allocated through malloc(3), custom memory pools, and
garbage-allocated heaps outside the control of the CLI implementation...
Note that the above quote is from the Mono documentation, but I believe (if I'm not mistaken) the same is true for .NET in general. If the data is being loaded in the DLL's internal data structures, then it should allocate its own memory. If you're providing a buffer which will get filled up with the data, then it will only get filled up with as much data as you've allocated for the buffer (and pinned before marshalling). So where is the data being loaded?
You can't increase the heap size in .NET.
You could create an EXE in c/c++ that your .NET app calls using Process.Start.
Your c/c++ EXE would just call the DLL function and return the result (or if you have more than one function it could take a command line parameter). If you don't want a separate EXE you could try using RunDll32 instead.
I doubt this is specific to .NET, managed memory, garbage collection etc. It's a native DLL so it uses regular, unmanaged memory. Of course, the .NET runtime will also use it's share of memory but a native application using the DLL would do the same.
If you're running in a 32 bit process, the total heap size for .NET and unmanaged code can be limited to 1.5 GB. It's difficult to tell without additional information, but you might have hit that limit.
So one option would be to ask your vendor, whether they have a 64 bit version of the library and switch to a 64 process. In a 64 bit process, memory is almost unlimited (according to today's standard).
How long is the typical overhead added by calling a .dll written in C++ from a C# application using the following syntax?
[DllImport("abc.dll", EntryPoint = "xcFoo", CallingConvention = CallingConvention.Cdecl)]
public extern static Result Foo(out IntPtr session,
[MarshalAs(UnmanagedType.FunctionPtr)]ObjectCallback callback,
UInt64 turnKey,
string serverAddress,
string userId,
string password);
Is there a more efficient way to do it?
Check out this article on how to improve interop performance. What to do and what best to avoid.
http://msdn.microsoft.com/en-us/library/ms998551.aspx
Are you talking about the overhead of invoking the native method? If so, I dont think it is significant at all, as there are a lot of such calls in the .NET framework class libraries.
Now, whether the overhead is significant for your scenario can only be answered by doing performance measurements, and comparing them against what you expect.
The marshalling into the native method will cost three memory allocations from the NT heap, which is not so bad. It's the delegate back that gets worrysome.
A good method to check this sort of thing is throw in a break point where you make calls. Don't know when the library is loaded, so maybe only check the break point on the second call (unless loading cost is your main concern). Then open up the disassembly window in visual studio and see how many lines there are before your dll function is being invoked.
i know this question is old but ive managed to call native functions blazingly fast with not only using calli CIL instruction but also with special trick, but ofc you need to handle pinnig and/or marshalling arguments yourself if you deal with complex types including strings..
I have this problem, which I am dealing with for some time already. At start, I have two dlls with unmanaged code (let's say... dll_1, dll_2) and managed aplication in c#. What i'm supposed to do is to get pointers to unmanaged functions in dll_1 in managed code, pack it into structure, and send this structure as an argument to unmanaged function in dll_2. Have anyone dealt with that kind of problem before maybe?
Since you don't do anything in managed code but the DLLs live in the same process, just use an IntPtr (is automatically 32 or 64 bits depending on the platform) to pass the unmanaged pointer around. You can of course also insert the IntPtr into your struct and use it as argument or return value when using an external call (e.g. [DllImport('YourDll')] static extern IntPtr ImportedFunction();).
However, to give you more information, it would be necessary to know more about the DLL calls and their data structures.