I have an ActiveX (COM) DLL that makes windows system calls (such as ReadFile() and WriteFile()). My GUIs (in Python or C#) create an instance of the DLL in the main GUI thread. However, in order to make calls to it from threads, a new instance of the DLL must be created in each thread, regardless of using C# or Python. (As a side note, the original instance could be called from a thread in C#, but this blocks the main thread; doing this in Python crashes the GUI.) Is there any way to avoid creating a new instance of the DLL in threads?
The reason why using the original DLL instance is desired: The DLL allows connection to a HID microcontroller. The DLL provides an option to allow only one exclusive handle to the microcontroller. If the GUI designer chooses this option (which is necessary in certain situations), the second DLL instance would not work as desired, since only one instance of the DLL could make the connection.
I haven't worked with Phyton, but for C# I would suggest creating a helper class that contains the ActiveX as a static public property. Have the main thread create the ActiveX and then from there all threads access it as needed.
When you make an ActiveX/COM component you can specify threading model for your component, it could be e.g. "compartmentalized". Depending on which you choose ActiveX/COM takes care of serializing requests.
If you "open" and ActiveX/COM component multiple times (depending on threading model?) only one instance is actually created.
I'm assuming you use win32com.client.Dispatch(".") to "open" your ActiveX/COM component.
Also, don't forget pythoncom.CoInitialize() and CoUninitialize() pair of calls.
Google on what those actually do.
If you can't change given ActiveX/COM component and its threading model is unacceptable, you can wrap all "outbound" calls in one dedicated Python thread with monitor "interface."
Here's an outline for what code I wrote once faced with similar situation:
class Driver(threading.Thread):
quit = False # graceful exit
con = None
request = None
response = None
def __init__(self, **kw):
super(Driver, self).__init__(**kw)
self.setDaemon(True) # optional, helps termination
self.con = threading.Condition()
self.request = None
self.response = None
def run(self):
pythoncom.CoInitialize()
handle = win32com.client.Dispatch("SomeActiveX.SomeInterface")
try:
with self.con:
while not self.quit:
while not self.request: self.con.wait() # wait for work
method, args = self.request
try: self.response = getattr(handle, method)(*args), None # buffer result
except Exception, e: self.response = None, e # buffer exception
self.con.notifyAll() # result ready
finally:
pythoncom.CoUninitialize()
def call(method, *args):
with self.con:
while self.request: self.con.wait() # driver is busy
self.request = method, args
self.con.notifyAll() # driver can start
while not self.response: self.con.wait() # wait for driver
rv, ex = self.response
self.request = self.response = None # free driver
self.con.notifyAll() # other clients can continue
if ex: raise ex
else: return rv
Related
I'm using a C# Process to run the following Java code with the java command:
public class Test {
public static void main(String[] args) throws Exception {
Thread.sleep(2000);
System.out.print("Hello worl\nd 1!");
Thread.sleep(2000);
System.out.println("Hello world 2!");
}
}
I'm using the following code to listen to the output:
run.OutputDataReceived += (_, args) => { /* handle output */ };
run.Start();
run.BeginOutputReadLine();
Ideally, OutputDataReceived should be fired twice, with the coressponding values for args.Data:
"Hello worl\nd 1!"
"Hello world 2!\n"
Instead, the newlines are used to determine when OutputDataReceived is fired. This ends up giving 3 calls to the event, with the corresponding values to args.Data:
"Hello worl"
"d 1!Hello world 2!"
null
How would I run code to handle output according to my first scenario (each time stdout is updated) instead of what is currently happening/the second scenario (whenever stdout receives a new line)? In addition, how would I do the same for stderr as well?
For anyone else with this problem, you can use this solution described in this answer to get live output.
I think what you want is impossible, and this is by design.
These text streams ain’t made of discrete messages, it’s a continuous stream of bytes very similar to a TCP socket.
If you control both producing and consuming sides of the stream, you can workaround by implementing some framing protocol on top of that stream. An easy way which might be good enough for you —separate messages with double newlines. This way, on the C# side you can write code which reads lines accumulating in a buffer, search for double newlines, when found or end of stream raise an event to handle a complete message, and unless end of stream move the remaining portion of the buffer to the start of the buffer.
Another workaround is using some other IPC mechanism instead of standard output. On Windows, named pipes support PIPE_TYPE_MESSAGE flag which causes the pipe to preserve message boundaries. Similarly, Linux kernel supports SOCK_DGRAM for Unix domain sockets. In both cases, a common design pattern looks like that: 1. Generate unique pipe/socket name in the parent C# process, for instance by generating a new GUID and printing that GUID. 2. Create the pipe or socket of that name. 3. Pass that name to the child Java process through a command-line argument, or an environment variable. 4. In the Java process, connect the pipe/socket by the name, you then have a message oriented two way communication channel between your two processes.
I'm having a hard time with some code I have that apparently struggles when called from the second window created by the ShareTarget contract (when you share something to the app, it opens in a small standalone window).
This is my code so far:
// Blur and resize the image to get the average HSL color
// Assume that stream is an IRandomAccessStream pointing to valid image data
HslColor hslMean;
using (RandomAccessStreamImageSource imageProvider = new RandomAccessStreamImageSource(stream))
using (BlurEffect blurEffect = new BlurEffect(imageProvider) { KernelSize = 256 })
{
Color mean = await DispatcherHelper.GetFromUIThreadAsync(async () =>
{
WriteableBitmap
blurred = new WriteableBitmap((int)decoder.PixelWidth, (int)decoder.PixelHeight),
result = await blurEffect.GetBitmapAsync(blurred, OutputOption.Stretch),
resized = result.Resize(1, 1, WriteableBitmapExtensions.Interpolation.Bilinear);
return resized.GetPixel(0, 0);
});
hslMean = mean.ToHsl();
}
Note: that DispatcherHelper.GetFromUIThreadAsync method just checks the thread access to the UI thread, and if needed it schedules the code to a CoreDispatcher object that was obtained with CoreApplication.MainView.CoreWindow.Dispatcher.
Problem: this code works 100% fine if my app is already open, as at that point that CoreDispatcher object has already been created by previous calls to that DispatcherHelper class, so the method just uses the stored dispatcher to schedule the work and it works fine. But, if the app is closed when the ShareTarget window is opened (so that DispatcherHelper has to create the dispatcher for the first time) the CoreApplication.MainView.CoreWindow line throws an exception. A very weird one:
COMException:
A COM call to an ASTA was blocked because the call chain originated in or passed through another ASTA. This call pattern is deadlock-prone and disallowed by apartment call control.
A COM call (IID: {638BB2DB-451D-4661-B099-414F34FFB9F1}, method index: 6) to an ASTA (thread 10276) was blocked because the call chain originated in or passed through another ASTA (thread 4112). This call pattern is deadlock-prone and disallowed by apartment call control.
So, I needed a way to make that method reliable even when being called from different windows. I've tried different options:
#1: Just invoking that code without dispatching to a different thread, as in theory I should be on the UI thread at this point ---> FAIL (The application called an interface that was marshalled for a different thread. (Exception from HRESULT: 0x8001010E (RPC_E_WRONG_THREAD)))
#2: Manually calling CoreApplication.MainView.CoreWindow.Dispatcher to dispatch that code block ---> FAIL (I get that weird COMException mentioned above)
#3: Manually using CoreApplication.MainView.Dispatcher to dispatch the code block (as it was the .CoreWindow part that spawned the exception) ---> FAIL (COMException: item not found)
#4: Using CoreApplication.GetCurrentView().CoreWindow.Dispatcher, CoreApplication.GetCurrentView().Dispatcher, Window.Current.CoreWindow.Dispatcher and Window.Current.Content.Dispatcher to schedule that code ---> FAIL (wrong thread again, I get the usual marshalling exception)
All these marshalling exception are thrown at the line result = await blurEffect.GetBitmapAsync(blurred, OutputOption.Stretch), so I suspect it might be something related to the Lumia Imaging SDK.
I mean, I'm quite sure that I am in fact on the UI thread, or otherwise I wouldn't have managed to create an instance of the WriteableBitmap class, right?
Why is it that I can create WriteableBitmap objects (and they need to be created on the UI thread as far as I know), but that GetBitmapAsync method from the Lumia SDK always throws that marshalling exception? I'm using it everywhere in my app without any problems, why is it that it just won't work from the ShareTarget window? Is there anything I need to do?
Thanks for your help!
Looks like this is a bug in the Lumia Imaging SDK (that was originally written for WP8.1, which didn't have multiple windows/dispatchers), so unless the call to the library is made from the dispatcher associated with the main app window (which of course can only be retrieved if the app is open in the background when the ShareTarget window pops up), it will just fail.
The only solution at this point is to replace that call to the Lumia SDK with some other code that doesn't rely on that particular library (in this case for example, it is possible to just get the ARGB array from the WriteableBitmap object and calculate the mean color manually).
I have a C# application which in turn loads a C or C++ dll (which in turn loads other C/C++ dlls). Within the C# application I use a log4net logger to capture all the output into a series of log files. My application runs as a windows service so there is no console/output window for normal printfs or output written into stdout/stderr to go to.
Is there a way to setup the C# application to direct stdout/stderr (from the DLLs) and turn each line into a log4net output. Or is there some way within the C/C++ DLL to connect the stdout/stderr streams to the log4net output?
I found some solution (here : http://bytes.com/topic/c-sharp/answers/822341-dllimport-stdout-gets-eaten) that indicated I needed to put a call into my C DLL like this : setvbuf(stdout, NULL, _IONBF, 0); Though, I don't know what that does, it doesn't do what I want. I assume I'd also need a similar line for stderr. In either case, google seemed to think those lines simply take care of buffering and not redirection into log4net.
I assume I need some sort of function override which snags the console writes (from a loaded DLL in another language) and converts them into mLog.InfoFormat("{0}", consoleString); sorts of calls. I'm new to c# and not even sure what terms to google in order to find such an override (if its even possible).
Not sure if this complicates the problem, but my C# application is multithreaded and some of the DLLs have multiple threads as well. I assume that just means I need a lock of some sort inside the method that handles the console output and writes it into the log4net framework(maybe) or maybe the normal serialization of log4net will handle it for me.
Turns out those did the trick once I figured out how to use them. I setup two named pipes(or two ends of the same pipe?). One I connected to stdout and had it do a log message in log4net of whatever came through the pipe.
internal static void InfoLogWriter(Object threadContext)
{
mLog.Info("InfoLogWriterthread started");
int id = Process.GetCurrentProcess().Id; // make this instance unique
var serverPipe = new NamedPipeServerStream("consoleRedirect" + id, PipeDirection.In, 1);
NamedPipeClientStream clientPipe = new NamedPipeClientStream(".", "consoleRedirect" + id, PipeDirection.Out, PipeOptions.WriteThrough);
mLog.Info("Connecting Client Pipe.");
clientPipe.Connect();
mLog.Info("Connected Client Pipe, redirecting stdout");
HandleRef hr11 = new HandleRef(clientPipe, clientPipe.SafePipeHandle.DangerousGetHandle());
SetStdHandle(-11, hr11.Handle); // redirect stdout to my pipe
mLog.Info("Redirection of stdout complete.");
mLog.Info("Waiting for console connection");
serverPipe.WaitForConnection(); //blocking
mLog.Info("Console connection made.");
using (var stm = new StreamReader(serverPipe))
{
while (serverPipe.IsConnected)
{
try
{
string txt = stm.ReadLine();
if (!string.IsNullOrEmpty(txt))
mLog.InfoFormat("DLL MESSAGE : {0}", txt);
}
catch (IOException)
{
break; // normal disconnect
}
}
}
mLog.Info("Console connection broken. Thread Stopping.");
}
Also have a function to push all that to another thread so it doesn't block my main thread when it hits the various blocking calls.
internal static void RedirectConsole()
{
mLog.Info("RedirectConsole called.");
ThreadPool.QueueUserWorkItem(new System.Threading.WaitCallback(InfoLogWriter));
// TODO enqueue and item for error messages too.
}
I'm having trouble with it disconnecting and have to reconnect the pipes, but I'll figure out a reconnect solution. I'm guessing that happens when DLLs get swapped back out of memory or perhaps when I need to read but there isn't anything currently ready to be read? I've also got to setup another pair to snag stderr and redirect it as well, using Error logs for that one. Probably want to get rid of the magic number (-11) and use the normal enums as well (STD_ERROR_HANDLE, etc)
as I am new in multithreaded application I would like to have some advice from more experienced people before starting to write the code...
I need to queue data received on serial port in serial port event for further processing.
So I have the following event handler:
void jmPort_ReceivedEvent(object source, SerialEventArgs e)
{
SetStatusLabel("Iddle...", lbStatus);
SetPicVisibility(ledNotReceiving, true);
SetPicVisibility(ledReceiving, false);
String st = jmPort.ReadLine();
if (st != null)
{
lines.Enqueue(st); //"lines" is the ConcurrentQueue<string> object
StartDataProcessing(lines); //???
SetStatusLabel("Receiving data...", lbStatus);
SetPicVisibility(ledNotReceiving, false);
SetPicVisibility(ledReceiving, true);
}
else
{
jmPort.Close();
jmPort.Open();
}
}
Within the StartDataProcessing I need to dequeue strings and update MANY UI controlls (using the InvokeRequired...this I already know :-)).
What is the best approach and colision free (without deadlock) approach to achieve this?
How to call StartDataProcessing method in more threads and safely dequeue (TryDequeue) the lines queue, make all needed computations and update UI controlls?
I have to appoint that the communication is very fast and that I am not using the standard SerialPort class. If I simply write all received strings without further processing to console window it works just well.
I am working in .NET 4.5.
Thank you for any advice...
Updated question: Ok, so what will be the best way to run the task from the datareceived event using TPL? Is it necessary to create another class (object) that will process data and use callbacks to update UI or it is possible to load some form method from the event? I'll could be very happy if someone can give me the direction what exactly to do within the datareceived event. What to do as the first step because studying all possible ways is not the solution I have time for. I need to begin with some particular way... There is so many different possible multithreading approaches and after reading about them I am still more confused and I don't know what will be the best a fastest solution... Usual Thread(s), BackgroundWorker, TPL, async-await...? :-( Because my application uses .NET 4.5 I would like to use some state-of-the-art solution :-) Thank you for any advice...
So after a lot of trying it is working to my satisfaction now.
Finally I've used the standard .NET SerialPort class as the third-party Serial class causes somae problems with higher baudrates (115200). It uses WinAPI directly so the finall code was mixed - managed and unmanaged. Now, even the standard .NET 4.5 SerialPort class works well (I've let my application successfully running through a whole night).
So, for everyone that need to deal with C#, SerialPort and higher rates (only for clarification - the device sending messages to PC is the STM32F407 /using USART 2/. I've tried it also with Arduino Due and it works as well) my datareceived event is in the following form now:
private void serialPort1_DataReceived(object sender, System.IO.Ports.SerialDataReceivedEventArgs e)
{
//the SetXXXXX functions are using the .InvokeRequired approach
//because the UI components are updated from another thread than
//the thread they were created in
SetStatusLabel("Iddle...", lbStatus);
SetPicVisibility(Form1.frm.ledNotReceiving, true);
SetPicVisibility(Form1.frm.ledReceiving, false);
String st = serialPort1.ReadLine();
if (st != null)
{
lines.Enqueue(st);
Task.Factory.StartNew(() => StartDataProcessing(lines)); // lines is global ConcurrentQueue object so in fact there is no need to pass it as parameter
SetStatusLabel("Receiving data...", lbStatus);
SetPicVisibility(Form1.frm.ledNotReceiving, false);
SetPicVisibility(Form1.frm.ledReceiving, true);
}
}
Within the StartDataProcessing function:
1. TryDequeue(lines, out str)
2. Use the ThreadPool.QueueUserWorkItem(lCallBack1, tmp); where tmp is needed part of the str (without EOF, without the message number etc.)
lCallBack1 = new WaitCallback(DisplayData);
Within the DisplayData function all the UI controls are updated
This approach mixes the ThreadPool and TPL ways but it is not a problem because the ThreadPool is used by TPL in background operation anyway.
Another working method I've tried was the following:
ThreadPool.QueueUserWorkItem(lCallBack, lines);
instead of :
Task.Factory.StartNew(() => StartDataProcessing(lines));
This method was working well but I've not tested it in over night run.
By my subjective perception the Task.... method updated the controls more smoothly but it can be only my personal feeling :-)
So, I hope this answer will help someone as I know from forums that many people are dealing with with unreliable communication based on the micocontroller <--> PC
My (surprising :-) ) conclusion is that the standard .NET SerialPort is able to handle messages even at higher baudrates. If you still run into troubles with buffer overrun then try to play with the SerialPort buffer size and SerialPort threshold. For me the settings 1024/500 are satisfactory (max size of the message send by microcontroller is 255 bytes so 500 bytes means that 2 messages are in buffer before the event is fired.)
You can also remove all SetXXXX calls from the datareceived event as they are not really needed and they can slow down the communication a little...
I am very close to real-time data capturing now and it is exactly what I've needed.
Good luck to everyone :-)
Within the StartDataProcessing I need to dequeue strings and update MANY UI controlls
No, you do not. You need to dequeue strings and then enqueue them again into the multiple queues for the different segments of the UI.
If you want to be fast, you scatter all operations and definitely the UI into separate windows that run their own separate message pumps and thus can update independently in separate UI threads.
The general process would be:
1 thread handles the serial port and takes the data and queues it.
Another one dequeues it and distributes it to separate processing threads from which
the data goes to multiple output queues all responsible for one part of the UI (depending on whether the UI Will turn a bottleneck).
There is no need to be thread safe in dequeuing. How serial is the data? Can you skip data when another update for the same piece arrives?
Read up on TPL and tasks - there are base libraries for parallel processing which come with a ton of documentation.
As Ive stated with a few other questions, Ive been using a new SSH .NET library to connect to a Unix server and run various scripts and commands. Well, I've finally attempted to use it to run a Unix tail -f on a live log file and display the tail in a Winforms RichTextBox.
Since the library is not fully-fleshed out, the only kinda-sorta solution I've come up with seems lacking... like the feeling you get when you know there has to be a better way. I have the connection/tailing code in a separate thread as to avoid UI thread lock-ups. This thread supports cancellation request (which will allow the connection to gracefully exit, the only way to ensure the process Unix side is killed). Here's my code thus far (which for the record seems to work, I would just like some thoughts on if this is the right way to go about it):
PasswordConnectionInfo connectionInfo = new PasswordConnectionInfo(lineIP, userName, password);
string command = "cd /logs; tail -f " + BuildFileName() + " \r\n";
using (var ssh = new SshClient(connectionInfo))
{
ssh.Connect();
var output = new MemoryStream();
var shell = ssh.CreateShell(Encoding.ASCII, command, output, output);
shell.Start();
long positionLastWrite = 0;
while (!TestBackgroundWorker.CancellationPending) //checks for cancel request
{
output.Position = positionLastWrite;
var result = new StreamReader(output, Encoding.ASCII).ReadToEnd();
positionLastWrite = output.Position;
UpdateTextBox(result);
Thread.Sleep(1000);
}
shell.Stop();
e.Cancel = true;
}
The UpdateTextBox() function is a thread-safe way of updating the RichTextBox used to display the tail from a different thread. The positionLastWrite stuff is an attempt to make sure I don’t loose any data in between the Thread.Sleep(1000).
Now Im not sure about 2 things, first being that I have the feeling I might be missing out on some data each time with the whole changing MemoryStream position thing (due to my lack of experiance with MemoryStreams, and the second being that the whole sleep for 1 second then update again thing seems pretty archaic and inefficient... any thoughts?
Mh, I just realized that you are not the creator of the SSH library (although it's on codeplex so you could submit patches), anyway: You might want to wrap your loop into a try {} finally {} and call shell.Stop() in the finally block to make sure it is always cleaned up.
Depending on the available interfaces polling might be the only way to go and it is not inherently bad. Whether or not you loose data depends on what the shell object is doing for buffering: Does it buffer all output in memory, does it throw away some output after a certain time?
My original points still stand:
One thing which comes to mind is that it looks like the shell object is buffering the whole output in memory all the time which poses a potential resource problem (out of memory). One option of changing the interface is to use something like a BlockingQueue in the shell object. The shell is then enqueuing the output from the remote host in there and in your client you can just sit there and dequeue which will block if nothing is there to read.
Also: I would consider making the shell object (whatever type CreateShell returns) IDisposable. From your description it sounds shell.Stop() is required to clean up which won't happen in case some exception is thrown in the while loop.