Develop Reference

C# - Send input to child process - console app

Background info
I am writing an integration test that spawns a child process (c# console app). The test is counting some rows in the database after the process is spun up and after the process is closed. The process is closed via process.Kill()
When the process is killed in this manner, it doesn't hit the Stop method within the process. I need to call this stop method to stop threads and remove entries from the database in order for the test to pass.
Original Code
The console app process that I am spawning in my test:
static void Main(string[] args)
{
TaskManager tm = new TaskManagerProcess();
if (Environment.UserInteractive ||
(args.EmptyForNull().Any(a => a.Equals("-RunInteractive", StringComparison.OrdinalIgnoreCase) || a.Equals("/RunInteractive"))))
{
tm.ConsoleStart(args);
Console.WriteLine("Press [Enter] to shut down, any other key to mark");
while (true)
{
ConsoleKeyInfo key = Console.ReadKey(true);
if (key.Key == ConsoleKey.Enter)
break;
Console.WriteLine("========================================================");
Console.Out.Flush();
}
Console.WriteLine("Shutting down...");
tm.ConsoleStop();
}
else
{
ServiceBase.Run(tm);
}
}
}
The test code:
//count before starting child proc
int preCount;
//count after process is spun up
int runningsCount;
//count after stopped
int postCount;
//Get an initial count of the logged in modules before svc host is started
user = ApiMethod.GetLoggedInUsers().Where(x => x.RecId == userRecID).FirstOrDefault();
preCount = user.LoggedInModules.Count;
Process proc = Helper.StartProcess(ConnectionBundle);
//Give process time to spin up leaders and workers
await Task.Delay(TimeSpan.FromSeconds(30));
//Get a count of modules after process is spun up
user = ApiMethod.GetLoggedInUsers().Where(x => x.RecId == userRecID).FirstOrDefault();
runningCount = user.LoggedInModules.Count;
//Write a line terminator to the child svc host process -
//this allows it to shutdown normally
Helper.ProcessInput.WriteLine();
Helper.ProcessInput.Close();
Helper.KillProcess(proc);
await Task.Delay(TimeSpan.FromSeconds(5));
//Get count of logged in modules after process is closed
user = ApiMethod.GetLoggedInUsers().Where(x => x.RecId == userRecID).FirstOrDefault();
postCount = user.LoggedInModules.Count;
Helper is a static class that sets up the process start info(including args) and starts the process. In helper I've redirected the StandardInput and added a property ProcessInput which is set to the StandardInput of the created process.
My goal is to send input of "Enter" from the test to the spawned process so that it will break from the loop and call tm.ConsoleStop()
TaskManagerProcess is a private custom class that controls the process. It does not inherit from System.Diagnostics.Process. As an alternate approach, my test could interact with TaskManagerProcess directly. However, I can't make TaskManagerProcess public and I need to run TaskManagerProcess in its own AppDomain because calling ConsoleStop is disposing objects in the API that I need to finish the test.
Things I've Tried
[DllImport("Kernel32")]
private static extern bool SetConsoleCtrlHandler(CloseProcDelgate handler, bool add);
I tried adding a call to Kernel32.SetConsoleCtrlHandler (and the necessary delegate) to call ConsoleStop when the process is exited. This doesn't seem to work when the process is killed via process.Kill()
With the original process code, I noticed an exception when I wrote to the StandardInput. The exception message told me to use Console.Read instead of Console.ReadKey(). This actually works intermittently! I can sometimes get a breakpoint on int cKey = Console.Read() (with debugger attached to child process) but other times it doesn't hit the breakpoint.
while (true)
{
//Changing this to Console.Read instead of Console.ReadKey
//Allows us to send redirected input to process?
int cKey = Console.Read();
if ((ConsoleKey)cKey == ConsoleKey.Enter)
break;
Console.WriteLine("========================================================");
Console.Out.Flush();
}
Finally, I tried interacting with TaskManagerProcess directly. I made the private class internal, and marked the internals visible to my test assembly. I cannot make the class public.
When I go this route, calling tm.ConsoleStop() blows away some objects in my API so I can't check the count after this method is called. For this reason, I thought I would create a new AppDomain and call AppDomain.CreateInstanceAndUnwrap() on the TaskManagerProcess class. However, I get an exception here, I believe its due to the the fact that the class is internal.
I am really stuck at this point! Any help is appreciated and thanks for taking the time to read this!
Edit
I created a demo project here
that shows what I am trying to do and has both approaches in the Test method.
Initially I thought I couldn't call AppDomain.CreateInstanceAndUnwrap() because the TaskManagerProcess class was internal. However, after playing with my demo project, I think I just can't load the assembly.

I'm guessing here, but I believe your TaskManagerProcess is a service application. If it is not, please ignore this. If it is, be advised of including details like this in your question. Debugging service applications can be complicated, believe me, I've been there. But before proceed, more advise.
Test the methods in your modules, no whole running programs, as Michael Randall just said.
Unless absolutely necessary, don't do tests against a database. Mock whatever you need to test your code.
You should go back to your alternate approach of interact with TaskManagerProcess directly. From the code of your console app, the only working method I see called is tm.ConsoleStart(args), the rest inside the loop is console writing and reading. So you can't change the acces level of that class, again, I've been there. What I have done in the past to overcome this is to use conditional compilation to create a kind of public facade in my private or internal modules.
Suppose you have:
internal class TaskManagerContainer
{
private class TaskManagerProcess
{
internal void Start()
{
// stuff
}
private void DoSomething(int arg)
{
// more stuff
}
}
}
Change it like this:
#define TEST
// Symbol TEST can also be defined using the GUI of your IDE or compiler /define option
internal class TaskManagerContainer
{
//
#if TEST
public class TaskManagerProcess
#else
private class TaskManagerProcess
#endif
{
internal void Start()
{
// stuff
}
private void DoSomething(int arg)
{
// more stuff
}
#region Methods Facade for Testing
#if TEST
public void Start_Test()
{
Start();
}
private void DoSomething_Test(int arg)
{
DoSomething(arg);
}
#endif
#endregion
}
}
I really hope it will help you making the methods visible to the test assembly and it won't blow objects in you API.

I think I got it with a brute force approach.
while (!testProcess.HasExited)
{
testProcess.StandardInput.WriteLine();
}
Thanks everyone for the input!

Pythonnet how to stop script execution properly

I'm using Pythonnet to embed a Python script launcher into a C# WPF application. I can pass variable to python scripts using Scope and i get the result on a console using MVVM pattern.
Now I want to allow the user to stop a script execution at anytime. I couldn't find how to make that work in order to close the Thread properly.
class PythonRuntime
{
private static NLog.Logger logger = NLog.LogManager.GetCurrentClassLogger();
private MainViewModel viewModel;
private string pythonCode;
private bool runtimeThreadLock = false;
Thread thread;
private PyScope scope;
private dynamic pyThread;
private dynamic pyLock;
ConsoleWriter consoleWriter;
public PythonRuntime(MainViewModel viewModel, ConsoleWriter consoleWriter)
{
this.viewModel = viewModel;
this.consoleWriter = consoleWriter;
SetUpPython();
}
public string PythonCode { get => pythonCode; set => pythonCode = value; }
private void SetUpPython()
{
PythonEngine.Initialize(true);
scope = Py.CreateScope();
// consoleWriter to make python prints into C# UI
scope.Set("Console", consoleWriter);
}
public void LaunchScript()
{
if (!runtimeThreadLock)
{
thread = new Thread(PythonNetTest);
thread.Start();
}
}
public void StopScript()
{
// ???
}
[HandleProcessCorruptedStateExceptions]
private void PythonNetTest()
{
runtimeThreadLock = true;
pyThread = PythonEngine.BeginAllowThreads();
pyLock = PythonEngine.AcquireLock();
using (Py.GIL())
{
try
{
scope.Exec(pythonCode);
}
catch (PythonException exception)
{
consoleWriter.WriteError(exception.ToString());
}
}
PythonEngine.ReleaseLock(pyLock);
PythonEngine.EndAllowThreads(pyThread);
runtimeThreadLock = false;
}
}
Besides my question, I was wondering what is the purpose of wrapping code in using(Py.GIL()). Because with or whithout it my script runs the same way.
Pythonnet : 2.4.0
Python : 2.7.2 32bit
NetFramework : 4.7.1

OK, I'm just beginning work on embedding CPython and may know only a little more than you. What that caveat...
First, you need to get your script to terminate. When it does the call to .Exec() will return, and the thread will exit. If your script runs for a finite amount of time then you just wait for it. Otherwise, you must arrange some signal that it should exit.
Second, mainline will wait for thread to complete using one of several .NET patterns described at: How to wait for thread to finish with .NET?
using(Py.GIL()) is shorthand for PythonEngine.AcquireLock(); and PythonEngine.ReleaseLock(pyLock); It creates an IDisposable object that acquires the lock and then releases it on Dispose(). So, in your sample, it is redundant.
I'm unsure effects of your call to BeginAllowThreads(). Documentation says that it releases the lock to allow other threads. When you call it you don't have the GIL. Next line acquires the GIL. Therefore, it appears to have no function to me.
See https://docs.python.org/3/c-api/init.html for details on threading. This seems more related to python threading and saving thread state so that other non-python things can be done. This is python 3. Python 2 did not seem to support the equivalent.

Random message "The thread has exited with code 0" in managed threads

I've a got a problem with the infamous message "The thread xxx has exited with code 0 (0x0)".
In my code I have a main class called "Load" that starts with a Windows Form load event:
public class Load
{
public Load()
{
Device[] devices = GetDevices(); // Get an array of devices from an external source
for (int i = 0; i < devices.Length; i++)
{
DeviceDiagnosticCtrl deviceDiagnostic = new DeviceDiagnosticCtrl(devices[i].name);
}
}
}
Inside the constructor, for each generic device read from an external source, I initialize a custom diagnostic class that runs a thread:
public class DeviceDiagnosticCtrl
{
private Thread diagnosticController;
private volatile bool diagnosticControllerIsRunning = false;
public DeviceDiagnosticCtrl(string _name)
{
// Thread initialization
this.diagnosticController = new Thread(new ThreadStart(this.CheckDiagnostic));
this.diagnosticController.Start();
this.diagnosticControllerIsRunning = true;
}
private void CheckDiagnostic()
{
while (this.diagnosticControllerIsRunning)
{
try
{
// Custom 'Poll' message class used to request diagnostic to specific device
Poll poll = new Poll();
// Generic Message result to diagnostic request
IGenericMessage genericResult;
// Use a custom driver to send diagnostic request
SendSyncMsgResult res = this.customDriver.SendSyncMessage(poll, out genericResult);
switch (res)
{
case SendSyncMessageResult.GOOD:
{
// Log result
}
break;
case SendSyncMessageResult.EXCEPTION:
{
// Log result
}
break;
}
Thread.Sleep(this.customDriver.PollScantime);
}
catch (Exception ex)
{
// Loggo exception
}
}
}
}
When I run the above code in debug mode I always read 8 devices from external source, and for each of them I continuously run a managed thread to retrieve diagnostic.
My problem is that randomly one or more of the 8 threads I expect from the code above exit with code 0, without any exception.
I've started/restarted the code in debug mode a lot of time, and almost everytime one of the thread exits.
I've read somewhere (i.e. this SO question) that it could depends of Garbage Collector action, but I'm not too sure if this is my case - and how to prevent it.
Do someone see something strange/wrong in the sample code I posted above? Any suggestion?

'while (this.diagnosticControllerIsRunning)' is quite likely to fail immediate, in which case the thread drops out. It's no good starting the thread and THEN setting 'this.diagnosticControllerIsRunning = true;' - you're quite likely to be too late.
Bolt/stable-door. Something like:
do{
lengthyStuff with Sleep() in it
}
while (this.diagnosticControllerRun);

Copied from Here
Right click in the Output window when you're running your program and
uncheck all of the messages you don't want to see (like Thread Exit
messages).

c# Subthread stuck with Thread.join()

We have a piece of code that previously worked fine and now it gets stuck, the problem seems to be runtime updates or something like it maybe?
We launch a subthread that shows a Form Dialog and gets a value, this value is returned to the main thread by a static variable in an static class and in that statement the thread gets stuck and visual studio doesn't say anything about what's happening... is there another way to return the value (in some cases there are more than one string to return)
We need to do this way specifically because of how the software we do the add-on programs for work.
Example code:
public static Cons
{
public static string inputvalue;
}
public static Funs
{
public static string GetValueString()
{
Thread threadx = new Thread(GetValueStringx);
threadx.SetApartmentState(ApartmentState.STA);
threadx.Start();
if (threadx.Join(new TimeSpan(0, 3, 0)))
{
return ComprobarLicencia(Cons.inputvalue);
}
/*because the subthread is stuck the wait time (3mins) is always
reached and the code continues here, not getting the input value :/ */
try
{
threadx.Abort();
}
catch
{
}
return "";
}
public static string GetValueStringx()
{
WindowWrapper window = new WindowWrapper(Fun.GetForegroundWindow());
FormInput formlicencia = new FormLicencia();
formlicencia.ShowDialog(window);
Cons.inputvalue = formlicencia.inputvalue; //thread gets stuck here
/*even if i comment all the above lines and i put
directly Cons.inputvalue="valx"; it gets stuck too :s */
}
}
In the end I solved it using this response in this question. Is there some way to give credit to that user? If not i would give the anwser or upvotes to someone that can explain why the subthread gets stuck accesing the static variable...

Why does parallel this code work sometimes?

I wanted to parallelize a piece of code, but the code actually got slower probably because of overhead of Barrier and BlockCollection. There would be 2 threads, where the first would find pieces of work wich the second one would operate on. Both operations are not much work so the overhead of switching safely would quickly outweigh the two threads.
So I thought I would try to write some code myself to be as lean as possible, without using Barrier etc. It does not behave consistent however. Sometimes it works, sometimes it does not and I can't figure out why.
This code is just the mechanism I use to try to synchronize the two threads. It doesn't do anything useful, just the minimum amount of code you need to reproduce the bug.
So here's the code:
// node in linkedlist of work elements
class WorkItem {
public int Value;
public WorkItem Next;
}
static void Test() {
WorkItem fst = null; // first element
Action create = () => {
WorkItem cur=null;
for (int i = 0; i < 1000; i++) {
WorkItem tmp = new WorkItem { Value = i }; // create new comm class
if (fst == null) fst = tmp; // if it's the first add it there
else cur.Next = tmp; // else add to back of list
cur = tmp; // this is the current one
}
cur.Next = new WorkItem { Value = -1 }; // -1 means stop element
#if VERBOSE
Console.WriteLine("Create is done");
#endif
};
Action consume = () => {
//Thread.Sleep(1); // this also seems to cure it
#if VERBOSE
Console.WriteLine("Consume starts"); // especially this one seems to matter
#endif
WorkItem cur = null;
int tot = 0;
while (fst == null) { } // busy wait for first one
cur = fst;
#if VERBOSE
Console.WriteLine("Consume found first");
#endif
while (true) {
if (cur.Value == -1) break; // if stop element break;
tot += cur.Value;
while (cur.Next == null) { } // busy wait for next to be set
cur = cur.Next; // move to next
}
Console.WriteLine(tot);
};
try { Parallel.Invoke(create, consume); }
catch (AggregateException e) {
Console.WriteLine(e.Message);
foreach (var ie in e.InnerExceptions) Console.WriteLine(ie.Message);
}
Console.WriteLine("Consume done..");
Console.ReadKey();
}
The idea is to have a Linkedlist of workitems. One thread adds items to the back of that list, and another thread reads them, does something, and polls the Next field to see if it is set. As soon as it is set it will move to the new one and process it. It polls the Next field in a tight busy loop because it should be set very quickly. Going to sleep, context switching etc would kill the benefit of parallizing the code.
The time it takes to create a workitem would be quite comparable to executing it, so the cycles wasted should be quite small.
When I run the code in release mode, sometimes it works, sometimes it does nothing. The problem seems to be in the 'Consumer' thread, the 'Create' thread always seems to finish. (You can check by fiddling with the Console.WriteLines).
It has always worked in debug mode. In release it about 50% hit and miss. Adding a few Console.Writelines helps the succes ratio, but even then it's not 100%. (the #define VERBOSE stuff).
When I add the Thread.Sleep(1) in the 'Consumer' thread it also seems to fix it. But not being able to reproduce a bug is not the same thing as knowing for sure it's fixed.
Does anyone here have a clue as to what goes wrong here? Is it some optimization that creates a local copy or something that does not get updated? Something like that?
There's no such thing as a partial update right? like a datarace, but then that one thread is half doen writing and the other thread reads the partially written memory? Just checking..
Looking at it I think it should just work.. I guess once every few times the threads arrive in different order and that makes it fail, but I don't get how. And how I could fix this without adding slowing it down?
Thanks in advance for any tips,
Gert-Jan

I do my damn best to avoid the utter minefield of closure/stack interaction at all costs.
This is PROBABLY a (language-level) race condition, but without reflecting Parallel.Invoke i can't be sure. Basically, sometimes fst is being changed by create() and sometimes not. Ideally, it should NEVER be changed (if c# had good closure behaviour). It could be due to which thread Parallel.Invoke chooses to run create() and consume() on. If create() runs on the main thread, it might change fst before consume() takes a copy of it. Or create() might be running on a separate thread and taking a copy of fst. Basically, as much as i love c#, it is an utter pain in this regard, so just work around it and treat all variables involved in a closure as immutable.
To get it working:
//Replace
WorkItem fst = null
//with
WorkItem fst = WorkItem.GetSpecialBlankFirstItem();
//And
if (fst == null) fst = tmp;
//with
if (fst.Next == null) fst.Next = tmp;

A thread is allowed by the spec to cache a value indefinitely.
see Can a C# thread really cache a value and ignore changes to that value on other threads? and also http://www.yoda.arachsys.com/csharp/threads/volatility.shtml