Background info
I am writing an integration test that spawns a child process (c# console app). The test is counting some rows in the database after the process is spun up and after the process is closed. The process is closed via process.Kill()
When the process is killed in this manner, it doesn't hit the Stop method within the process. I need to call this stop method to stop threads and remove entries from the database in order for the test to pass.
Original Code
The console app process that I am spawning in my test:
static void Main(string[] args)
{
TaskManager tm = new TaskManagerProcess();
if (Environment.UserInteractive ||
(args.EmptyForNull().Any(a => a.Equals("-RunInteractive", StringComparison.OrdinalIgnoreCase) || a.Equals("/RunInteractive"))))
{
tm.ConsoleStart(args);
Console.WriteLine("Press [Enter] to shut down, any other key to mark");
while (true)
{
ConsoleKeyInfo key = Console.ReadKey(true);
if (key.Key == ConsoleKey.Enter)
break;
Console.WriteLine("========================================================");
Console.Out.Flush();
}
Console.WriteLine("Shutting down...");
tm.ConsoleStop();
}
else
{
ServiceBase.Run(tm);
}
}
}
The test code:
//count before starting child proc
int preCount;
//count after process is spun up
int runningsCount;
//count after stopped
int postCount;
//Get an initial count of the logged in modules before svc host is started
user = ApiMethod.GetLoggedInUsers().Where(x => x.RecId == userRecID).FirstOrDefault();
preCount = user.LoggedInModules.Count;
Process proc = Helper.StartProcess(ConnectionBundle);
//Give process time to spin up leaders and workers
await Task.Delay(TimeSpan.FromSeconds(30));
//Get a count of modules after process is spun up
user = ApiMethod.GetLoggedInUsers().Where(x => x.RecId == userRecID).FirstOrDefault();
runningCount = user.LoggedInModules.Count;
//Write a line terminator to the child svc host process -
//this allows it to shutdown normally
Helper.ProcessInput.WriteLine();
Helper.ProcessInput.Close();
Helper.KillProcess(proc);
await Task.Delay(TimeSpan.FromSeconds(5));
//Get count of logged in modules after process is closed
user = ApiMethod.GetLoggedInUsers().Where(x => x.RecId == userRecID).FirstOrDefault();
postCount = user.LoggedInModules.Count;
Helper is a static class that sets up the process start info(including args) and starts the process. In helper I've redirected the StandardInput and added a property ProcessInput which is set to the StandardInput of the created process.
My goal is to send input of "Enter" from the test to the spawned process so that it will break from the loop and call tm.ConsoleStop()
TaskManagerProcess is a private custom class that controls the process. It does not inherit from System.Diagnostics.Process. As an alternate approach, my test could interact with TaskManagerProcess directly. However, I can't make TaskManagerProcess public and I need to run TaskManagerProcess in its own AppDomain because calling ConsoleStop is disposing objects in the API that I need to finish the test.
Things I've Tried
[DllImport("Kernel32")]
private static extern bool SetConsoleCtrlHandler(CloseProcDelgate handler, bool add);
I tried adding a call to Kernel32.SetConsoleCtrlHandler (and the necessary delegate) to call ConsoleStop when the process is exited. This doesn't seem to work when the process is killed via process.Kill()
With the original process code, I noticed an exception when I wrote to the StandardInput. The exception message told me to use Console.Read instead of Console.ReadKey(). This actually works intermittently! I can sometimes get a breakpoint on int cKey = Console.Read() (with debugger attached to child process) but other times it doesn't hit the breakpoint.
while (true)
{
//Changing this to Console.Read instead of Console.ReadKey
//Allows us to send redirected input to process?
int cKey = Console.Read();
if ((ConsoleKey)cKey == ConsoleKey.Enter)
break;
Console.WriteLine("========================================================");
Console.Out.Flush();
}
Finally, I tried interacting with TaskManagerProcess directly. I made the private class internal, and marked the internals visible to my test assembly. I cannot make the class public.
When I go this route, calling tm.ConsoleStop() blows away some objects in my API so I can't check the count after this method is called. For this reason, I thought I would create a new AppDomain and call AppDomain.CreateInstanceAndUnwrap() on the TaskManagerProcess class. However, I get an exception here, I believe its due to the the fact that the class is internal.
I am really stuck at this point! Any help is appreciated and thanks for taking the time to read this!
Edit
I created a demo project here
that shows what I am trying to do and has both approaches in the Test method.
Initially I thought I couldn't call AppDomain.CreateInstanceAndUnwrap() because the TaskManagerProcess class was internal. However, after playing with my demo project, I think I just can't load the assembly.
I'm guessing here, but I believe your TaskManagerProcess is a service application. If it is not, please ignore this. If it is, be advised of including details like this in your question. Debugging service applications can be complicated, believe me, I've been there. But before proceed, more advise.
Test the methods in your modules, no whole running programs, as Michael Randall just said.
Unless absolutely necessary, don't do tests against a database. Mock whatever you need to test your code.
You should go back to your alternate approach of interact with TaskManagerProcess directly. From the code of your console app, the only working method I see called is tm.ConsoleStart(args), the rest inside the loop is console writing and reading. So you can't change the acces level of that class, again, I've been there. What I have done in the past to overcome this is to use conditional compilation to create a kind of public facade in my private or internal modules.
Suppose you have:
internal class TaskManagerContainer
{
private class TaskManagerProcess
{
internal void Start()
{
// stuff
}
private void DoSomething(int arg)
{
// more stuff
}
}
}
Change it like this:
#define TEST
// Symbol TEST can also be defined using the GUI of your IDE or compiler /define option
internal class TaskManagerContainer
{
//
#if TEST
public class TaskManagerProcess
#else
private class TaskManagerProcess
#endif
{
internal void Start()
{
// stuff
}
private void DoSomething(int arg)
{
// more stuff
}
#region Methods Facade for Testing
#if TEST
public void Start_Test()
{
Start();
}
private void DoSomething_Test(int arg)
{
DoSomething(arg);
}
#endif
#endregion
}
}
I really hope it will help you making the methods visible to the test assembly and it won't blow objects in you API.
I think I got it with a brute force approach.
while (!testProcess.HasExited)
{
testProcess.StandardInput.WriteLine();
}
Thanks everyone for the input!
Related
I would like to use PyCharm's debugger to stop at a breakpoint set in a Python method which is called from C# via Python.NET. However, the breakpoint is never hit, even though the code clearly gets executed.
The issue seems to be that the thread on which the Python method is called was created in C#. I am aware that for a thread to be visible to the debugger, settrace must be called. I thought, calling it once per thread should do the trick, but it does not. Because after calling settrace at the first execution of a Python method, my breakpoints in that method and any methods called by it, are hit. But after control passed from that method back to C#, another call of a Python method on the same thread does not hit the breakpoint.
Here is the Python side of things; imagine breakpoints set at the two print statements.
import sys
import thread
import threading
sys.path.append(r'C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2018.2.4\helpers\pydev')
import pydevd
import clr
from System.Threading import Thread
from System import Action
sys.path.append(r'C:\Code\PythonNetLib\bin\Debug')
clr.AddReference('MyLib')
from MyLib import MyThread, MyThreadBase
def init_debugging():
pydevd.settrace(suspend=False)
# breakpoint here is hit
print 'debugging initialized in thread {0}, managed thread ID {1}'.format(thread.get_ident(), Thread.CurrentThread.ManagedThreadId)
def hello():
# pydevd.settrace(suspend=False)
# breakpoint here is not hit unless pydevd.settrace is called again
print 'Hello from thread {0}, managed thread ID {1}'.format(thread.get_ident(), Thread.CurrentThread.ManagedThreadId))
t = MyThread()
t.ExecuteAsync(Action(init_debugging))
t.ExecuteAsync(Action(hello))
t.ExecuteAsync(Action(t.Stop))
t.Join()
For illustration purposes here is a simplified version of the C# library referenced above containing the MyThread class:
using System;
using System.Collections.Concurrent;
using System.Threading;
namespace MyLib
{
public class MyThread: MyThreadBase
{
private Thread _thread;
public MyThread()
{
_thread = new Thread(Execute);
_thread.Start();
}
public void Join()
{
_thread.Join();
}
}
public class MyThreadBase
{
private ConcurrentQueue<Action> _queue;
private bool _stopped;
public MyThreadBase()
{
_queue = new ConcurrentQueue<Action>();
}
public void Execute()
{
while (!_stopped)
{
if (_queue.TryDequeue(out Action action))
{
action();
}
else
{
Thread.Sleep(TimeSpan.FromSeconds(1));
}
}
}
public void ExecuteAsync(Action action)
{
_queue.Enqueue(action);
}
public void Stop()
{
_stopped = true;
}
}
}
Workaround: I noticed that my breakpoints are hit as expected when I don't create the thread in C# but use a Python thread instead like so:
t = MyThreadBase()
tPy = threading.Thread(target=lambda: t.Execute())
tPy.start()
t.ExecuteAsync(Action(init_debugging))
t.ExecuteAsync(Action(hello))
t.ExecuteAsync(Action(t.Stop))
tPy.join()
This is an option in some cases, but I do not control all thread creations in the C# libraries I want to use.
Question: What would I have to do make C# threads known to the Python debugger permanently such that the debugger doesn't lose sight of the thread after control passes back to C#. Repeatedly calling settrace seems a bit much, particularly since I can't know definitively which methods might all get called from C#.
I am using Python 2.7 and .NET Framework 4.6.1 in case that makes a difference.
I'm using Pythonnet to embed a Python script launcher into a C# WPF application. I can pass variable to python scripts using Scope and i get the result on a console using MVVM pattern.
Now I want to allow the user to stop a script execution at anytime. I couldn't find how to make that work in order to close the Thread properly.
class PythonRuntime
{
private static NLog.Logger logger = NLog.LogManager.GetCurrentClassLogger();
private MainViewModel viewModel;
private string pythonCode;
private bool runtimeThreadLock = false;
Thread thread;
private PyScope scope;
private dynamic pyThread;
private dynamic pyLock;
ConsoleWriter consoleWriter;
public PythonRuntime(MainViewModel viewModel, ConsoleWriter consoleWriter)
{
this.viewModel = viewModel;
this.consoleWriter = consoleWriter;
SetUpPython();
}
public string PythonCode { get => pythonCode; set => pythonCode = value; }
private void SetUpPython()
{
PythonEngine.Initialize(true);
scope = Py.CreateScope();
// consoleWriter to make python prints into C# UI
scope.Set("Console", consoleWriter);
}
public void LaunchScript()
{
if (!runtimeThreadLock)
{
thread = new Thread(PythonNetTest);
thread.Start();
}
}
public void StopScript()
{
// ???
}
[HandleProcessCorruptedStateExceptions]
private void PythonNetTest()
{
runtimeThreadLock = true;
pyThread = PythonEngine.BeginAllowThreads();
pyLock = PythonEngine.AcquireLock();
using (Py.GIL())
{
try
{
scope.Exec(pythonCode);
}
catch (PythonException exception)
{
consoleWriter.WriteError(exception.ToString());
}
}
PythonEngine.ReleaseLock(pyLock);
PythonEngine.EndAllowThreads(pyThread);
runtimeThreadLock = false;
}
}
Besides my question, I was wondering what is the purpose of wrapping code in using(Py.GIL()). Because with or whithout it my script runs the same way.
Pythonnet : 2.4.0
Python : 2.7.2 32bit
NetFramework : 4.7.1
OK, I'm just beginning work on embedding CPython and may know only a little more than you. What that caveat...
First, you need to get your script to terminate. When it does the call to .Exec() will return, and the thread will exit. If your script runs for a finite amount of time then you just wait for it. Otherwise, you must arrange some signal that it should exit.
Second, mainline will wait for thread to complete using one of several .NET patterns described at: How to wait for thread to finish with .NET?
using(Py.GIL()) is shorthand for PythonEngine.AcquireLock(); and PythonEngine.ReleaseLock(pyLock); It creates an IDisposable object that acquires the lock and then releases it on Dispose(). So, in your sample, it is redundant.
I'm unsure effects of your call to BeginAllowThreads(). Documentation says that it releases the lock to allow other threads. When you call it you don't have the GIL. Next line acquires the GIL. Therefore, it appears to have no function to me.
See https://docs.python.org/3/c-api/init.html for details on threading. This seems more related to python threading and saving thread state so that other non-python things can be done. This is python 3. Python 2 did not seem to support the equivalent.
I am creating a C# console application of showroom.
I want something like that:
{
customer comes and places an order.
when order completse then I want the main page back.
where customer can re order or re use this application.
}
Actually I want to re run application during runtime.
Is there any specific function or syntax in C#?
I dont know any function that reruns your program. But I also think that its not the best idea to just rerun the program if you want to get back to the programs starting point.
Here is a sample implementation of the loop:
class Program
{
private bool _running = true;
static void Main(string[] args)
{
Program program = new Program();
while (program._running)
{
PlaceOrders();
...
if (exitCondition)
{
program._running = false;
}
}
}
}
You should store all your orders in a database or something similar. At the start of the application you can then load all the orders and give the user the possibility of changing orders, ...
I need a operations which needs to run every x seconds forever, and to achieve this I did:
protected void Application_Start()
{
InitialieOnce.Initialize();
}
public static class InitialieOnce
{
private static bool initialized = false;
public static void Initialize()
{
if (initialized == false)
{
initialized = true;
Thread t = new Thread(x => CheckStatus());
t.IsBackground = true;
t.Start();
}
}
private static void CheckStatus()
{
//My script goes here.
Thread.Sleep(8000);
CheckStatus();
}
}
After some time (about 5 minutes) I get this error:
"An unhandled exception of type 'System.StackOverflowException' occurred in mscorlib.dll"
Can this error be related to how I made my infinite loop?
If yes, is there a better way to achieve this, can I fix it, or is this code ok?
You are calling "CheckStatus" recursively. So every 8 seconds there will be one more entry on your call stack:
CheckStatus() -> CheckStatus() -> CheckStatus() -> and so on.. until you get a StackOverflowException.
Instead you should use
while (true)
{
/* Your Code */
Thread.Sleep(8000);
}
Please also note that by default IIS will unload your application if there was no request in 15 minutes, resulting in your thread being killed.
Consider creating a Windows service for such a thing, instead of abusing IIs..
I don't know where, but I read that IIs isn't good for creating long living services like in javas servers.
I would also suggest creating a Windows service for that, something like a daemon. You can create a service that will just call a special action on your application on regular intervals. The rest of the work will be done within your MVC application. Have a look at this post for an example.
the problem only appears when making Release build and running exe file ( not from visual studio )
in all other combination either it's running from visual studio or running exe everything works fine
I'm running Function Load using backgroundWorker
Load:
while (!Request.GAMELIST.XMLReceived) ;
GameEngine.ParseGameList( Request.GAMELIST.XML );
Request.GAMELIST.XMLReceived = false;
while loop in this fragment works like delay
it should wait till XML is received from server and then continue
but it stucks in above specified situation
if I'll put MessageBox.show("here we go"); after while loop
messageBox will not appear
but if I'll put MessageBox.show("here we go"); before while loop
application will receive data until I click messagebox ok
and then everything will work fine
here is GAMELIST class implementation
public class RequestGAMELIST
{
public string XML;
public bool XMLReceived = false;
public void ParseRequest( string request )
{
int index = request.IndexOf(':') + 2;
XML = request.Substring(index, request.Length - index);
XMLReceived = true;
}
}
please provide help if you can
this is really strange thing which I can't figure out by my self
Thanks.
Yes, this code has very good odds to hang in the Release build. The JIT optimizer doesn't know that the variable might be set to true by code outside of the method. You need to tell it that, like this:
public class RequestGAMELIST
{
public volatile bool XMLReceived = false;
// etc..
}
The volatile keyword ensures that the jitter won't store the variable value in a CPU register.
That solves your problem, it is still not the right way to do it. You should use an AutoResetEvent instead. It ensures that the thread responds to the variable change is quickly as possible. And most importantly, it lets the thread block so it doesn't burn any cpu cycles.
public class RequestGAMELIST
{
public AutoResetEvent XMLReceived = new AutoResetEvent();
public void ParseRequest( string request )
{
int index = request.IndexOf(':') + 2;
XML = request.Substring(index, request.Length - index);
XMLReceived.Set();
}
}
In your thread:
XMLReceived.WaitOne();
GameEngine.ParseGameList( Request.GAMELIST.XML );
This is a bad idea:
while (!Request.GAMELIST.XMLReceived) ;
At least you should be doing something like:
while (!Request.GAMELIST.XMLReceived) {
System.Threading.Thread.Sleep(100); // Don't hog the CPU!
}
Your program runs fine in debug mode perhaps due to certain debug routines added inside the while loop which makes it run slower...