I'm using a managed C# wrapper to access an unmanaged C++ library. The library does some time consumung caluclations (up to 6 seconds) that I need. But also in parallel to that I continuously need some data that is fast to get too.
To achieve that, I tried to get two instances of my wrapper, one for the quick stuff and the other one in a parallel thread to calculate the time consuming information. But, as soon as I instanciate the slow one of the Analyzers, even the quick one gets slow too.
fullAnalyzer = new Analyzer(FullAnalysis);
miniAnalyzer = new Analyzer(MinimalAnalysis);
It looks like both of them are sharing the same configuration in the back, because if I instanciate the quick one first, it is still fast.
Is it in general possible to have two or more seperate instances of a wrapper accessing an unmanaged library without interfering? I so - how is it done? Or is this behaviour just a local thing of this library?
Edit: This is the constructor and some part of the wrapper code
public class ScrWrapper
{
private const string DllName = #"Analyzer.dll";
public bool IsConfigLoaded { get; private set; }
public bool IsAnalyticsSuccessful { get; private set; }
public Analyzer()
{
IsConfigLoaded = false;
IsAnalyticsSuccessful = false;
}
public Analyzer(string configFileName, ScrProcLevel procLevel = ScrProcLevel.PL_NONE)
{
IsConfigLoaded = false;
IsAnalyticsSuccessful = false;
LoadConfig(configFileName, procLevel);
}
public void LoadConfig(string configFileName, ProcLevel procLevel = ScrProcLevel.PL_NONE)
{
if (configFileName.Length < 1)
throw new ArgumentException("Empty configFileName. Must contain valid file name.");
if (!System.IO.File.Exists(configFileName))
throw new ArgumentException(String.Format("Invalid configFileName. File not found: {0}",configFileName));
if (!System.IO.File.Exists(DllName))
throw new ArgumentException(String.Format("Invalid DllName. File not found: {0}", DllName));
bool b_config_status = false;
try
{
StringBuilder sb = new StringBuilder(configFileName);
ScanAuto_EnableWriteOut(true);
b_config_status = ScanAuto_LoadConfig(sb);
}
catch (Exception ex)
{
throw new ScrException("ERROR: Unmanaged Analyzer threw exception.", ex);
}
if (!b_config_status)
{
throw new ScrException(String.Format("ERROR: Failed to load the configurationfile, b_config_status=false"));
}
IsConfigLoaded = b_config_status;
_ProcLevel = procLevel;
Analyzer_SetProcLevel(_ProcLevel);
}
...
[DllImport(DllName, CallingConvention = CallConvention)]
[return: MarshalAs(UnmanagedType.I1)]
private extern static bool ScanAuto_LoadConfig(StringBuilder _pConfigFName);
}
Your wrapper looks fine from what I can tell, so its the fact that ScanAuto_LoadConfig method (and therefore the members it's initialising) is static, causing it to overwrite the same bit of config each time. see Static Data Members (C++)
Related
I have a .dll library, which I cannot modify, with classes which uses many static variables and singleton instances.
Now I need a second instance of all these classes and I need some solution which would isolate static variables between instances of some class without altering any other properties of the assembly.
Loading the same assembly second time doesn't actually load it again, but I found that reading it to byte array and then loading it, actually solves half of the problem:
lib.dll:
namespace lib
{
public class Class1 : ILib
{
private static int i;
public int DoSth()
{
return i++;
}
public string GetPath()
{
return typeof(Class1).Assembly.Location;
}
}
}
app.exe:
namespace test
{
public interface ILib
{
int DoSth();
string GetPath();
}
class Program
{
static void Main()
{
var assembly1 = Assembly.LoadFile(Path.GetFullPath(".\\lib.dll"));
var instance1 = (ILib)assembly1.CreateInstance("lib.Class1");
Console.WriteLine(instance1.GetPath());
Console.WriteLine(instance1.DoSth());
Console.WriteLine(instance1.DoSth());
var assembly2 = Assembly.LoadFile(Path.GetFullPath(".\\lib.dll"));
var instance2 = (ILib)assembly2.CreateInstance("lib.Class1");
Console.WriteLine(instance2.GetPath());
Console.WriteLine(instance2.DoSth());
Console.WriteLine(instance2.DoSth());
var assembly3 = AppDomain.CurrentDomain.Load(File.ReadAllBytes("lib.dll"));
var instance3 = (ILib)assembly3.CreateInstance("lib.Class1");
Console.WriteLine(instance3.GetPath());
Console.WriteLine(instance3.DoSth());
Console.WriteLine(instance3.DoSth());
Console.Read();
}
}
}
this returns:
C:\bin\lib.dll
0
1
C:\bin\lib.dll
2
3
0
1
Static variables got restarted but unfortunately the next problem is that assembly location which is used within the library is empty.
I would like to avoid loading the library to different AppDomain because it creates too many problems with cross domain code; some classes are not serializable.
I would like to avoid physically copying the library on disk.
I would like to avoid IL weaving and using Mono.Cecil or similar because it's an overkill.
Loading assembly into separate AppDomain or separate process are only sensible options you have. Either deal with cross-domain/cross-process communication or get version of library that does not have problems you trying to work around.
If you want to fix your load from bytes you'd need to read all articles around https://blogs.msdn.microsoft.com/suzcook/2003/09/19/loadfile-vs-loadfrom/.
I am experiencing a memory leak in context with AppDomains. I've stripped it down to the following:
I got 3 projects, two library projects and a console project: Shared, DynamicallyLoadable and RemotingTimeoutPrototype (the console program). Shared contains interfaces used by both DynamicallyLoadable and RemotingTimeoutPrototype. Both reference Shared at compile-time. There are no other compile-time references between any of the projects.
Shared contains this:
public interface IHostService
{
string GetStuff();
}
public interface IRemoteClass
{
IHostService Alpha { get; set; }
string CallHostServices();
}
DynamicallyLoaded contains only a single type:
public class RemoteClass : MarshalByRefObject, IRemoteClass
{
public IHostService Alpha { get; set; }
public string CallHostServices()
{
Console.WriteLine("Domain {0}, RemoteClass.CallHostServices():", AppDomain.CurrentDomain.Id);
return Alpha.GetStuff();
}
}
The console project contains an implementation of IHostService:
public class Alpha : MarshalByRefObject, IHostService
{
readonly byte[] mBuffer = new byte[100*1024*1024];
public Alpha()
{
for (var i = 0; i < mBuffer.Length; ++i)
mBuffer[i] = (byte) (i%256);
}
public string GetStuff()
{
return "Alpha";
}
}
Program.Main consists of this:
while (true)
{
var otherDomain = AppDomain.CreateDomain("OtherDomain");
var proxy =
(IRemoteClass)
otherDomain.CreateInstanceFromAndUnwrap("../../../DynamicallyLoadable/bin/debug/DynamicallyLoadable.dll",
"DynamicallyLoadable.RemoteClass");
var alpha = new Alpha();
proxy.Alpha = alpha;
Console.WriteLine(proxy.CallHostServices());
Thread.Sleep(TimeSpan.FromSeconds(2));
AppDomain.Unload(otherDomain);
RemotingServices.Disconnect(alpha); // this was just an attempt, doesn't change a thing whether it's there or not
GC.Collect(); // same here, this shouldn't really be necessary, I just tried it out of despair
}
Now when I let this run for a while, memory consumption increases constantly and eventually I hit an OutOfMemoryException.
I would expect that if the domain holding the proxy is unloaded, then the proxy is unloaded, too, and there are no references to the concrete Alpha anymore, so this would be collected. But obviously it is not.
Note that I also checked that the domain is really being unloaded by referencing mscoree and enumerating the loaded domains with code along the lines of:
var runtimeHost = new CorRuntimeHost();
runtimeHost.EnumDomains(out handle);
// etc.
Also, if I attach a handler to otherDomain.DomainUnload(), this handler is called just fine.
Can anyone shed some light on this, please?
It might be a typical problem of blocked finalization thread (as you have while-loop in, probably, STA thread and never give control to another threads explicitly). Even though you unload/dispose the domain it might be in the memory because it is not finalised. Read more here: http://alexatnet.com/articles/memory-leaks-in-net-applications - specifically the section "Blocked Finalization Thread" (I'm the author)
I want to keep a pointer to a managed Exception object in an unmanaged C assembly.
I've tried a bunch of ways. This is the only one I've found that passes my very preliminary tests.
Is there a better way?
What I'd really like to do is handle the alloc and free methods in the ExceptionWrapper constructor and destructor, but structs can't have constructors or destructors.
EDIT: Re: Why I would like this:
My C structure has a function pointer that is set with a managed delegate marshaled as an unmanaged function pointer. The managed delegate performs some complicated measurements using external equipment and an exceptions could occur during those measurements. I'd like to keep track of the last one that occurred and its stack trace. Right now, I'm only saving the exception message.
I should point out that the managed delegate has no idea it's interacting with a C DLL.
public class MyClass {
private IntPtr _LastErrorPtr;
private struct ExceptionWrapper
{
public Exception Exception { get; set; }
}
public Exception LastError
{
get
{
if (_LastErrorPtr == IntPtr.Zero) return null;
var wrapper = (ExceptionWrapper)Marshal.PtrToStructure(_LastErrorPtr, typeof(ExceptionWrapper));
return wrapper.Exception;
}
set
{
if (_LastErrorPtr == IntPtr.Zero)
{
_LastErrorPtr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(ExceptionWrapper)));
if (_LastErrorPtr == IntPtr.Zero) throw new Exception();
}
var wrapper = new ExceptionWrapper();
wrapper.Exception = value;
Marshal.StructureToPtr(wrapper, _LastErrorPtr, true);
}
}
~MyClass()
{
if (_LastErrorPtr != IntPtr.Zero) Marshal.FreeHGlobal(_LastErrorPtr);
}
}
This doesn't work. You are hiding a reference to the Exception object in unmanaged memory. The garbage collector cannot see it there so it cannot update the reference. When the C spits the pointer back out later, the reference won't point the object anymore after the GC has compacted the heap.
You'll need to pin the pointer with GCHandle.Alloc() so the garbage collector cannot move the object. And can pass the pointer returned by AddrOfPinnedObject() to the C code.
That's fairly painful if the C code holds on that pointer for a long time. The next approach is to give the C code a handle. Create a Dictionary<int, Exception> to store the exception. Have a static int that you increment. That's the 'handle' value you can pass to the C code. It is not perfect, you'll run into trouble when the program has added more than 4 billion exceptions and the counter overflows. Hopefully you'll never actually have that many exceptions.
You want serialization.
As a side note, your statement of:
What I'd really like to do is handle the alloc and free methods in the ExceptionWrapper constructor and destructor, but structs can't have constructors or destructors.
is not true. structs in C# can have and do have constructor(s), just not allowing user to declare parameterless constructor explicitly. That is, for example, you can declare a constructor which accepts an Exception. For destructors, which is not widely used in managed code, you should implement IDisposable if your class will hold some unmanaged resources.
Exception is non-blittable, you may not marshalling it the way you described, but it can be serialized as byte arry thus makes interop possible. I've read your another question:
Implications of throwing exception in delegate of unmanaged callback
and take the some of the usage from your code. Lets's to have two projects, one for managed and the other for the unmanaged code. You can create them with all the default settings but note the bitness of the executable images should be set the same. There are just one file per project need to be modified:
Managed console application - Program.cs:
using System.Diagnostics;
using System.Runtime.Serialization.Formatters.Binary;
using System.Runtime.InteropServices;
using System.IO;
using System;
namespace ConsoleApp1 {
class Program {
[DllImport(#"C:\Projects\ConsoleApp1\Debug\MyDll.dll", EntryPoint = "?return_callback_val##YGHP6AHXZ#Z")]
static extern int return_callback_val(IntPtr callback);
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
delegate int CallbackDelegate();
static int Callback() {
try {
throw new Exception("something went wrong");
}
catch(Exception e) {
UnmanagedHelper.SetLastException(e);
}
return 0;
}
static void Main() {
CallbackDelegate #delegate = new CallbackDelegate(Callback);
IntPtr callback = Marshal.GetFunctionPointerForDelegate(#delegate);
int returnedVal = return_callback_val(callback);
var e = UnmanagedHelper.GetLastException();
Console.WriteLine("exception: {0}", e);
}
}
}
namespace ConsoleApp1 {
public static class ExceptionSerializer {
public static byte[] Serialize(Exception x) {
using(var ms = new MemoryStream { }) {
m_formatter.Serialize(ms, x);
return ms.ToArray();
}
}
public static Exception Deserialize(byte[] bytes) {
using(var ms = new MemoryStream(bytes)) {
return (Exception)m_formatter.Deserialize(ms);
}
}
static readonly BinaryFormatter m_formatter = new BinaryFormatter { };
}
}
namespace ConsoleApp1 {
public static class UnmanagedHelper {
[DllImport(#"C:\Projects\ConsoleApp1\Debug\MyDll.dll", EntryPoint = "?StoreException##YGHHQAE#Z")]
static extern int StoreException(int length, byte[] bytes);
[DllImport(#"C:\Projects\ConsoleApp1\Debug\MyDll.dll", EntryPoint = "?RetrieveException##YGHHQAE#Z")]
static extern int RetrieveException(int length, byte[] bytes);
public static void SetLastException(Exception x) {
var bytes = ExceptionSerializer.Serialize(x);
var ret = StoreException(bytes.Length, bytes);
if(0!=ret) {
Console.WriteLine("bytes too long; max available size is {0}", ret);
}
}
public static Exception GetLastException() {
var bytes = new byte[1024];
var ret = RetrieveException(bytes.Length, bytes);
if(0==ret) {
return ExceptionSerializer.Deserialize(bytes);
}
else if(~0!=ret) {
Console.WriteLine("buffer too small; total {0} bytes are needed", ret);
}
return null;
}
}
}
Unnamaged class library - MyDll.cpp:
// MyDll.cpp : Defines the exported functions for the DLL application.
//
#include "stdafx.h"
#define DLLEXPORT __declspec(dllexport)
#define MAX_BUFFER_LENGTH 4096
BYTE buffer[MAX_BUFFER_LENGTH];
int buffer_length;
DLLEXPORT
int WINAPI return_callback_val(int(*callback)(void)) {
return callback();
}
DLLEXPORT
int WINAPI StoreException(int length, BYTE bytes[]) {
if (length<MAX_BUFFER_LENGTH) {
buffer_length=length;
memcpy(buffer, bytes, buffer_length);
return 0;
}
return MAX_BUFFER_LENGTH;
}
DLLEXPORT
int WINAPI RetrieveException(int length, BYTE bytes[]) {
if (buffer_length<1) {
return ~0;
}
if (buffer_length<length) {
memcpy(bytes, buffer, buffer_length);
return 0;
}
return buffer_length;
}
With these code you can serialize the exception first and then deserialize it at any later time for retrieving the object that represents the original exception - just the reference would not be as same as the original, so are the objects it referenced.
I'd add some note of the code:
The dll name and method entry point of DllImport should be modified as your real build, I didn't manipulate the mangled names.
Unmanaged Helper is just a demonstration of the interop with the example unmanaged methods StoreException and RetrieveException; you don't have to write the code like how I deal with them, you might want to design in your own way.
If you want to deserialize the exception within the unmanaged code, you might want to design your own formatter rather than BinaryFormatter. I don't know an existing implementation of Microsoft's specification in non-cli C++.
In case you want to build the code in C instead of C++, you'll have to change the Compile As option(Visual Studio) and rename MyDll.cpp to MyDll.c; also note the mangled names would be like _StoreException#8; use DLL Export Viewer or hex editors to find the exact name.
Basically what I need is being able to add new functionality to my application, without updating the application itself.
Lets say I have an application with two plugins.
Application has a function called generateNumber();
Plugin_1 would have the following code:
void init(){ }
int exec(){
int number = HOST_APPLICATION.generateNumber();
number = number * 2;
return number;
}
Plugin_2 would have the same structure, only different functionality:
void init(){ }
int exec(){
int number = HOST_APPLICATION.generateNumber();
number = (number * 10) + 13;
return number;
}
I need multiple instances of each 'plugin' to run simultaneously (each in its own thread), so 20 threads total at the same time. The code would be something like this:
Main Application:
void init_plugins() { }
void execute_plugins(){
for(int i=0; i<plugins.count; i++){
for(int z=0; z<10; z++){
Thread t = new Thread(plugin_research);
t.start(plugins[i]);
}
}
}
void plugin_research(PluginStructure Plugin){
Plugin.init();
int return_val = Plugin.exec();
// do something with return_val
}
I also need host application to call plugin's functions (the structure will be the same for all plugins) and plugin to be able to call host application's functions.
Each plugin would have different controls for configuration. I would need to show configuration in one place, but call plugin's functions multiple times simultaneously (using threads)
Is this possible? What would be the best way to accomplish this? can I do this with iPlugin?
Look into MEF, the managed extensibility framework at http://mef.codeplex.com/. It has a lot of built in support for runtime discovery of plugin components.
Basicly each plugin should be derived from a base class exposing methods you want.
public abstract class BasePlugin
{
protected int number;
public abstract void init_plugin();
public abstract int exec(int number);
public BasePlugin(int number)
{
this.number = number;
}
}
Then you have
public class Plugin1: BasePlugin
{
public override void init_plugin()
{
}
public override int exec(int number)
{
}
}
In your app you can create plugins and keep them in a list
List<BasePlugin> list = new List<BasePlugin>();
list.Add(new Plugin1(generateNumber()));
BasePlugin p2 = new Plugin2(generateNumber());
p2.init_plugin();
list.Add(p2);
and do with loaded plugins whatever you please.
Or (as I see from your edited question) you can create threads for every plugin...
To load plugins you could use functions like these:
public static List<T> GetFilePlugins<T>(string filename)
{
List<T> ret = new List<T>();
if (File.Exists(filename))
{
Type typeT = typeof(T);
Assembly ass = Assembly.LoadFrom(filename);
foreach (Type type in ass.GetTypes())
{
if (!type.IsClass || type.IsNotPublic) continue;
if (typeT.IsAssignableFrom(type))
{
T plugin = (T)Activator.CreateInstance(type);
ret.Add(plugin);
}
}
}
return ret;
}
public static List<T> GetDirectoryPlugins<T>(string dirname)
{
List<T> ret = new List<T>();
string[] dlls = Directory.GetFiles(dirname, "*.dll");
foreach (string dll in dlls)
{
List<T> dll_plugins = GetFilePlugins<T>(Path.GetFullPath(dll));
ret.AddRange(dll_plugins);
}
return ret;
}
Check out MEF - http://msdn.microsoft.com/en-us/library/dd460648.aspx. Even if you decide not to use it you can see how such architecture can be implemented and used.
Look into MEF, the Managed Extensibility Framework.
I'm trying to write a plugin system to provide some extensibility to an application of mine so someone can write a plugin(s) for the application without touching the main application's code (and risk breaking something).
I've got the base "IPlugin" interface written (atm, nothing is implemented yet)
Here is how I'm loading:
public static void Load()
{
// rawr: http://www.codeproject.com/KB/cs/c__plugin_architecture.aspx
String[] pluginFiles = Directory.GetFiles(Plugins.PluginsDirectory, "*.dll");
foreach (var plugin in pluginFiles)
{
Type objType = null;
try
{
//Assembly.GetExecutingAssembly().GetName().Name
MessageBox.Show(Directory.GetCurrentDirectory());
Assembly asm = Assembly.Load(plugin);
if (asm != null)
{
objType = asm.GetType(asm.FullName);
if (objType != null)
{
if (typeof(IPlugin).IsAssignableFrom(objType))
{
MessageBox.Show(Directory.GetCurrentDirectory());
IPlugin ipi = (IPlugin)Activator.CreateInstance(objType);
ipi.Host = Plugins.m_PluginsHost;
ipi.Assembly = asm;
}
}
}
}
catch (Exception e)
{
MessageBox.Show(e.ToString(), "Unhandled Exception! (Please Report!)", System.Windows.Forms.MessageBoxButtons.OK, System.Windows.Forms.MessageBoxIcon.Information);
}
}
}
A friend tried to help but I really didn't understand what was wrong.
The folder structure for plugins is the following:
\
\Plugins\
All plugins reference a .dll called "Lab.Core.dll" in the [root] directory and it is not present in the Plugins directory because of duplicate references being loaded.
The plugin system is loaded from Lab.Core.dll which is also referenced by my executable. Type "IPlugin" is in Lab.Core.dll as well. Lab.Core.dll is, exactly as named, the core of my application.
EDIT:
Question: Why/What is that exception I'm getting and how could I go about fixing it?
FINAL EDIT:
Ok so I decided to re-write it after looking at some source code a friend wrote for a TF2 regulator.
Here's what I got and it works:
public class TestPlugin : IPlugin {
#region Constructor
public TestPlugin() {
//
}
#endregion
#region IPlugin Members
public String Name {
get {
return "Test Plugin";
}
}
public String Version {
get {
return "1.0.0";
}
}
public String Author {
get {
return "Zack";
}
}
public Boolean OnLoad() {
MessageBox.Show("Loaded!");
return true;
}
public Boolean OnAllLoaded() {
MessageBox.Show("All loaded!");
return true;
}
#endregion
}
public static void Load(String file) {
if (!File.Exists(file) || !file.EndsWith(".dll", true, null))
return;
Assembly asm = null;
try {
asm = Assembly.LoadFile(file);
} catch (Exception) {
// unable to load
return;
}
Type pluginInfo = null;
try {
Type[] types = asm.GetTypes();
Assembly core = AppDomain.CurrentDomain.GetAssemblies().Single(x => x.GetName().Name.Equals("Lab.Core"));
Type type = core.GetType("Lab.Core.IPlugin");
foreach (var t in types)
if (type.IsAssignableFrom((Type)t)) {
pluginInfo = t;
break;
}
if (pluginInfo != null) {
Object o = Activator.CreateInstance(pluginInfo);
IPlugin plugin = (IPlugin)o;
Plugins.Register(plugin);
}
} catch (Exception) {
}
}
public static void LoadAll() {
String[] files = Directory.GetFiles("./Plugins/", "*.dll");
foreach (var s in files)
Load(Path.Combine(Environment.CurrentDirectory, s));
for (Int32 i = 0; i < Plugins.List.Count; ++i) {
IPlugin p = Plugins.List.ElementAt(i);
try {
if (!p.OnAllLoaded()) {
Plugins.List.RemoveAt(i);
--i;
}
} catch (Exception) {
Plugins.List.RemoveAt(i);
--i;
}
}
}
The Managed Extensibility Framework (MEF) is a new library in .NET that enables greater reuse of applications and components. Using MEF, .NET applications can make the shift from being statically compiled to dynamically composed. If you are building extensible applications, extensible frameworks and application extensions, then MEF is for you.
http://www.codeplex.com/MEF
Edit: CodePlex is going away - the code has been moved to Github for archival purposes only: https://github.com/MicrosoftArchive/mef
MEF is now a part of the Microsoft .NET Framework, with types primarily under the System.Composition. namespaces. There are two versions of MEF
System.ComponentModel.Composition, which has shipped with .NET 4.0 and higher. This provides the standard extension model that has been used in Visual Studio. The documentation for this version of MEF can be found here
System.Compostion is a lightweight version of MEF, which has been optimized for static composition scenarios and provides faster compositions. It is also the only version of MEF that is a portable class library and can be used on phone, store, desktop and web applications. This version of MEF is available via NuGet and is documentation is available here
It sounds like you have a circular reference. You said your plugins reference Lab.Core.DLL, but you also say the plugins are loaded from Lab.Core.DLL.
Am I misunderstanding what is happening here?
EDIT: OK now that you have added your question to the question...
You need to have Lab.Core.DLL accessible to the plugin being loaded since it is a dependency. Normally that would mean having it in the same directory or in the GAC.
I suspect there are deeper design issues at play here, but this is your immediate problem.
As a side answer, i use these 2 interfaces for implementing that
public interface IPlugin {
string Name { get; }
string Description { get; }
string Author { get; }
string Version { get; }
IPluginHost Host { get; set; }
void Init();
void Unload();
IDictionary<int, string> GetOptions();
void ExecuteOption(int option);
}
public interface IPluginHost {
IDictionary<string, object> Variables { get; }
void Register(IPlugin plugin);
}