Is there a way to handle exceptions in using the (Py.GIL()) block?
For example:
using System;
using Python.Runtime;
public class Test{
public static void Main(){
using(Py.GIL()){
try{
dynamic module = Py.Import("module");
dynamic result = module.function("argument");
Console.WriteLine(result);
}
catch(Excepiton ex){
// Handled Exception
}
}
}
}
I asked this question because I call a C# function which uses the using(Py.GIL()) block. It is executed with a new thread which the Main thread waits for to finish.
It works for the first round, but for the next, it stops on the using block, and the application freezes without showing any exception.
I even tried to stop the Main thread from waiting for the execution, but the worker thread still stops at the using block of Py.GIL() after the first round.
For the thread execution, I am using the Thread pool.
Thread.Factory.StartNew(FunctionName);
This issue was caused by the way Python handles threads. The main thread must start the Python engine and enable threading.
PythonEngine.Initialize();
PythonEngine.BeginAllowThreads();
Run the above code on the Main thread before using worker thread that uses using(Py.GIL()) block.
Related
I have a C# program that starts a long running python program on a separate (C#) thread. It does this via "await Task.Run(() => cntl.doTest());".
public void doTest()
{
PythonEngine.Initialize();
using (Py.GIL())
{
test = Py.Import("tester"); //ok
test.TestLoop.loop(); //ok
}
}
The main thread has a timer that periodically calls a function to get a python variable:
public int getCntr()
{
using (Py.GIL())
{
int foo = test.TestLoop.cntr; //ok
test.TestLoop.stopLoop = true; //ok -- causes python to exit normally
return foo; //first time through, ok; never gets past "using" on second try
}
}
Once called, the function stops the python program by setting python variable "test.TestLoop.stopLoop".
Everything works continuously if I comment out the "stopLoop=true".
If include that statement, the first call works fine, but when I call "getCntr" the second time the "using (Py.GIL())" statement hangs the C# program (i.e. in VS-debugger, the thread appears to be executing but does not move past the "using GIL" statement). I suspect this is because the python program has already exited normally (via setting the "stopLoop" in python).
So, is there a test I should do before executing the "using GIL()" statement to determine if the python program has exited? I looked into testing something on Python.Runtime and on PythonEngine, but did not see what to do.
I thought the problem might be because the "PythonEngine.Initialize" was on a thread that had ended before the second call to "getCntr". However I cannot initialize the engine outside of the doTest thread (because then the "using Py.GIL" block is skipped over in doTest and the "import" is never executed).
This problem might be similar to Python.Net: Do I need "using (Py.GIL())" in a callback from .Net to Python?.
I consulted *http://pythonnet.github.io/ and https://github.com/pythonnet/pythonnet/wiki/Threading
but found no answer.
This is my solution which works, but seems cumbersome:
Add a variable (string aLock = "";) that will indicate when the doTest thread has finished.
Set this variable to "done" when the doTest thread has returned (i.e. after the "await Task.Run()" statement).
Test this variable (aLock !="done") before the "using GIL" block in getCntr.
Lock aLock before reading and setting.
Please, see https://github.com/pythonnet/pythonnet/blob/16f04e9c0281cd1e4c266399b35ee89069eb36fe/src/embed_tests/TestInterrupt.cs
TL;DR; You should call PythonEngine.Initialize() followed by PythonEngine.BeginAllowThreads() in your app main thread. This will solve "because then the using Py.GIL block is skipped over".
Actually, in your scenario with PythonEngine.Initialize outside of doTest you should have seen Py.GIL hang in doTest because GIL is being held by the thread, that called PythonEngine.Initialize.
The reason for all above is very simple: when a thread calls PythonEngine.Initialize(), it has GIL lock when call completes. My guess is it behaves this way for convenience of writing single-threaded programs. BeginAllowThreads explicitly releases GIL so that other threads could run Python code. When you do this, main thread must also use Py.GIL.
UPD I just noticed, that Threading page you linked actually mentions that.
I am reading multithreading in Albahari's c# in a nutshell. He says that if a thread (for example Main thread) creates and starts a worker thread , then an exception thrown by the worker thread cannot directly be caught and handled by the creating thread. Let me quote him verbatim:
"Any try / catch / finally blocks in effect when a thread is created are of no
relevance to the thread when it starts executing. Consider the following
program:
public static void Main()
{
try
{
Thread workerThread = new Thread (Go);
workerThread.Start();
}
catch (Exception ex)
{
// We'll never get here!
Console.WriteLine ("Exception!");
}
}
static void Go()
{
throw null; // Throws a NullReferenceException
}
Albahari goes on to say that:
"The try / catch statement in this example is ineffective, and the newly
created thread will be encumbered with an unhandled
NullReferenceException . This behavior makes sense when you consider
that each thread has an independent execution path."
So here is the crux of my question:
I don't get the relevance of "each thread has an independent execution path". I mean why should it matter if the execution paths are independent ? I mean when the workerThread throws an unhandled exception -- why can't the CLR just halt the Main thread and hand over the exception object to the catch block in the Main? What's stopping it ??
[NOTES:
The other related question How do I handle exceptions from worker threads and main thread in a single catch block? is NOT asking the same question --- and none of the elaborate answers express WHY can't the CLR marshal an unhandled exception from a worker thread to the Main thread
Similar question is asked about another language , C++, -- where the answers suggest that it has something to do with the two threads having different stacks and the logical impossibility of mixing the two while unwinding the stack during exception handling. I'm not sure whether those answers apply here to a managed execution environment , like that of c#.
]
The main thread might have already finished executing.
I know it is hard to understand at first, but the worker thread is not executed like a method call even if it looks like that. The thread is created inside that try-block, but the execution might happen much later or not at all. Source code in text form can not make this visible. There seems to be some kind of "spacial" proximity of the try-block and the thread, but the moment the thread is created, the spacial proximity is gone. The try block only handles any exceptions that happen while the thread is created, but the it is "detached" from it.
Analogy: The main thread is the manager (team lead) and the worker thread(s) are the workers (team members) reporting to that manager. The workers are working in their home office. In the morning the manager is sending them emails with tasks for the day ("execute method Go"). Since the manager can not see the workers doing their work, she can only notice any progress or lack of it, if the workers send progress reports from time to time. If workers fall off their chairs or down the stairs (exceptions), the manager would not know. The workers need to make sure to catch such exceptions and send an appropriate message to the manager to let her know. The manager is (in this case) not waiting around after sending the initial emails, but is doing other work in the meantime.
You are asking why the CLR can't halt the Main thread, and handle the exception of the child thread. Do you mean the main thread should always suspend execution automatically, immediately after launching a new Thread? This would be very unhelpful, since a program could then have at a maximum one and only one active thread. It would be the end of multithreading as we know it! So you probably mean that the CLR should offer a mechanism to do this suspension of execution on demand. Although such a mechanism is indeed not available, you can do something very close to it using tasks:
public static void Main()
{
try
{
Task workerTask = new Task(() =>
{
throw null;
}, TaskCreationOptions.LongRunning); // Use a dedicated thread
workerTask.Start();
workerTask.Wait();
}
catch (Exception ex)
{
Console.WriteLine("Exception!"); // It happens!
}
}
I have some problem with ThreadStart delegate. After I provide a function and start the thread nothing is actually happening. I need to add Console.Readline() to write messages to a file. Why it behaves like that?
using System;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
namespace ThreadStart
{
class Program
{
static void Main(string[] args)
{
Thread thread = new Thread(new System.Threading.ThreadStart(() =>
{
int messageSeq = 0;
while (messageSeq < 5)
{
File.AppendAllText(#"c:\Test\write.txt", DateTime.Now.ToString() + Environment.NewLine);
messageSeq++;
Thread.Sleep(TimeSpan.FromMinutes(1));
}
}));
thread.IsBackground = true;
thread.Start();
//Console.ReadLine();
}
}
}
I don't have experience in multi-threaded applications so i might be missing something simple
Thread.IsBackground Property
A thread is either a background thread or a foreground thread. Background threads are identical to foreground threads, except that background threads do not prevent a process from terminating.
You're telling the thread not to force the application to stay running, and then you're letting the application close by returning from the Main method.
Console.ReadLine(); will stop the application returning from Main and will give the thread time to do it's work.
Thread.Start Method
Note that the call to Start does not block the calling thread.
The Start method of the Thread doesn't block the calling thread. That means it returns ~right away and the calling thread continues to execute.
Console.ReadLine Method
If the standard input device is the keyboard, the ReadLine method blocks until the user presses the Enter key.
Console.ReadLine() does block the calling thread until the user hits enter/return in the console (causing a new line).
I'd like to implement timeout mechanism for scripts, executed through Roslyn, so I need a way to abort a script execution. The only solution I found so far, is executing the script inside a thread and terminating it, but it's obviously a flawed solution.
Is there a better way to do this?
Other than you launching a separate process and killing that instead (to avoid the standard pitfalls of Thread.Abort()), that's the best you can do. Given the script code can be more or less arbitrary, there really couldn't be a better option. All it takes is your script to do Thread.Sleep(Timeout.Infinite) and there's nothing the scripting engine could do to recover that.
First create a new thread and put your code inside the thread as an action. Then call the Thread.Start method to begin execution. Use Thread.Join method to wait until the thread completes its execution.
In the below code, if the thread execution does not complete in 6 seconds then the thread is interrupted.
Here is the code:
Thread thread = new Thread(() => {
try
{
// your code related to Roslyn is here
// ...
}
catch (Exception ex)
{
}
});
thread.Start();
if (!thread.Join(TimeSpan.FromSeconds(6)))
{
thread.Interrupt();
}
The above code ends compilation after 6 seconds.
I've searched around to find the best way to terminate a non UI thread. What I found is that, in short, the thread is in a while loop and you simply set the while loop condition to false to make the thread exit the loop:
https://msdn.microsoft.com/en-us/library/7a2f3ay4(v=vs.80).aspx
Fine but my thread is not in a while loop. The thread is started and then its dispatcher is used. It's actually a thread in a C# dll. The dll itself is using a second C++ dll through a SWIG interface. Long story short, I need a dispatcher.
Now before going further, I would like to mention that everything works fine. But the only way I found to stop the dll thread is by calling Thread.Abort(). I would like to this in a more graceful way.
From the UI thread, that uses the C# dll, an instance of the dll class is created:
public CSomethingDll()
{
// This is the dll class constructor.
using (m_startEvent = new ManualResetEvent(false))
{
m_mainSdkThread = new Thread(DllThreadEntryPoint);
m_mainSdkThread.Name = "DLL THREAD";
m_mainSdkThread.Start(m_startEvent);
m_startEvent.WaitOne();
}
}
[STAThreadAttribute]
public void DllThreadEntryPoint (object o)
{
m_dispatcher = System.Windows.Threading.Dispatcher.CurrentDispatcher;
(o as ManualResetEvent).Set();
System.Windows.Threading.Dispatcher.Run();
// The dispatcher is now running and ready for calls to BeginInvoke().
}
The dispatcher of the dll thread is now running. The UI thread can now initialize the dll by calling the InitializeA method of the dll:
public void InitializeA()
{
m_dispatcher.BeginInvoke(EvInitialize);
}
Now, when the UI thread is terminating, when the application is closing, the UI thread calls the FinalizeA method:
public void FinalizeA()
{
m_dispatcher.BeginInvoke(EvFinalize);
}
Inside the dll thread, stuff is done to terminate everything and release the resources. Everything works fine. But the only way I found to stop the dll thread is by calling Thread.Abort():
m_dispatcher.Thread.Abort();
Now, is there some trick to stop the dll thread through the Dispatcher? Some way to stop the dispatcher that will then terminate the associated thread?
Thanks!
How do you terminate your UI thread, do you properly exit your dispatcher?
You really need to call Dispatcher.ExitAllFrames so that the Run method exits.