Develop Reference

How to capture the output of a process (from OR Tools solver) and display it through console

I have a C# winform application in which an optimization model is solved by OR-Tools. The optimization solver has the capability of sending the whole optimization process as stdout.This is done by:
Slvr.EnableOutput();
Solver.ResultStatus restatus = Slvr.Solve();
However, the solver does not automatically open up the console.
Currently, what I have done is:
Projects properties --> Application --> Output type --> Console Application
and the console is ready from the beginning till the end of the application run. Hence, that process stdout is automatically displayed.
What I want is to open the console exactly when the above part of code is run and display the stdout from the solver. Then wait for a key from the user to close the console and continue with the main application.

I guess your problem is you are trying to run the solver as part of the Winforms application, inside the GUI process right? But Console output is usually disabled in a Winforms application. You have basically two options:
use one of the options described here in this older SO answer to attach a console window for output to a Winforms application
split the application into two exe files: one command line program which runs the solver, and a Winforms part, just containing the UI. Then run the command line part as a separate process by System.Diagnostics.Process.Start, which allows finegrained control about output redirection. You may need the UI to pass parameters to the command line program, for example, by using a temporary file.
The second option is more work, especially for the communication between the GUI and the command line tool, but can be implemented easier in a way the GUI is not blocked, is more robust against bugs / program crashes in the solver part and performs usually better in case you want to introduce parallelization / run multiple solver processes at once.

Doc Brown has already answered your question, I'm only adding this to provide some code of how we implemented it here-- it's exactly what he suggests. We have a separate testPlugin.exe that get's started here. The communication is via files read and written on the file system. The console output gets captured in the "output handlers"
using System;
using System.Diagnostics;
using System.IO;
...
private void startTest()
{
int result = 2;
setFormStatus("working..."); // My method to inform the user with the form to wait.
getFormData(); // My method to get the data from the form
string errorMessage = null;
System.Diagnostics.Process testPlugInProcess = new System.Diagnostics.Process();
try
{
using (testPlugInProcess)
{
testPlugInProcess.StartInfo.UseShellExecute = false;
testPlugInProcess.StartInfo.FileName = System.IO.Path.Combine(assemblyDirectory, TestPlugInExe); // The name of the exe file
testPlugInProcess.StartInfo.CreateNoWindow = false;
testPlugInProcess.StartInfo.Arguments = getModelTestCommandLineArgs(); // My method to create the command line arguments
testPlugInProcess.StartInfo.RedirectStandardError = true;
testPlugInProcess.StartInfo.RedirectStandardOutput = true;
testPlugInProcess.OutputDataReceived += pluginTestOutputHandler;
testPlugInProcess.ErrorDataReceived += pluginTestOutputHandler;
testPlugInProcess.Start();
testPlugInProcess.BeginErrorReadLine();
testPlugInProcess.BeginOutputReadLine();
testPlugInProcess.WaitForExit();
result = testPlugInProcess.ExitCode;
}
setFormStatus("");
}
catch (Exception ex)
{
errorMessage = ex.Message;
}
testPlugInProcess = null;
}
Both the console and error output get written to the same file here, but you could separate them.
The plug-in handler looks like this:
private static void pluginTestOutputHandler(object sendingProcess,
DataReceivedEventArgs outLine)
{
if (!String.IsNullOrEmpty(outLine.Data))
{
for (int i = 0; i < numberOfTriesForWriting; i++)
{
try
{
using (StreamWriter sw = File.AppendText(lastPlugInTestTraceFilePath)) // The file name where the data is written.
{
sw.WriteLine(outLine.Data);
sw.Flush();
return;
}
}
catch (IOException)
{
System.Threading.Thread.Sleep(msToWaitBetweenTries);
}
}
}
}

"Resource Monitor" shows me more threads than the two that I wrote on my program

I am learning about Threads. Im using C# with .NET Framework 4.5.2 and Windows 10 x64.
I wrote a simple program with two threads and one large loop in each one:
class Program
{
static void Main(string[] args)
{
Thread t = new Thread(foo);
t.Start();
for (int i = 0; i < 99999999; i++)
{
Console.WriteLine("x");
}
}
static void foo()
{
for (int i = 0; i < 99999999; i++)
{
Console.WriteLine("y");
}
}
}
And when I run the final release of the program, in "Resource Monitor" I read it is running more than two threads.
It leads me to understand that we can't have a real control of how our application will be executed, only we can say "I want to run X at the same time than Y", but no a strict (real) control of number of threads that will be created. Is that correct?
I want to know the explanation of this behaviour.
Here a image of what I've just explained:

You have at least three threads when you run your application without a debugger attached and without creating any additional thread.
Remember that the garbage collector works on a separate thread. Also the finalizer works on a separate thread. The Main Thread is Trivial in this discussion.
When you see more threads, you need to keep in mind that when debugging using Visual Studio, there are debug-related threads running.
To test that, create a simple program like the below :
class Program
{
static void Main(string[] args)
{
Console.ReadKey();
}
}
Build your application, and run it using the Executable (Without Visual Studio Debugger Attached), you would see exactly 3 threads in the resource monitor.

Logic for Windows Services & Getting Updates

I developed a Windows service using vb.net which does the following using OnStart Even...
Grabs all Entries from a SQL Table
Creates Schedules from returned rows
it works fine, schedules fire on their time and stuff.
Problem: Whenever I have to ADD a new row to my Table, I have to restart the Service, So it can Grab the Newly created rows. This is causing problems for me...there could be a task which is running already and restarting service might break the system.
So what is the best way to handle this? Can the new rows be loaded into Service without a restart?
Thanks

Use the concept of Polling into the Database. Use the System.Threading.Timer class, set some interval after which a callback method will be invoked and that will be responsible to Poll the Database for new entries.

This OnStart was provided here by Marc Gravell:
public void OnStart(string[] args) // should this be override?
{
var worker = new Thread(DoWork);
worker.Name = "MyWorker";
worker.IsBackground = false;
worker.Start();
}
void DoWork()
{
// do long-running stuff
}
Note that OnStart can launch multiple threads or the first thread started may be used to start additional threads as needed. This allows you to set up either database polling or a thread that waits on a message queue for data.
A useful tip:
Adding a Main to your service allows you to run it as a console application in Visual Studio. This greatly simplifies debugging.
static void Main(string[] args)
{
ServiceTemplate service = new ServiceTemplate();
if (Environment.UserInteractive)
{
// The application is running from a console window, perhaps creating by Visual Studio.
try
{
if (Console.WindowHeight < 10)
Console.WindowHeight = 10;
if (Console.WindowWidth < 240) // Maximum supported width.
Console.WindowWidth = 240;
}
catch (Exception)
{
// We couldn't resize the console window. So it goes.
}
service.OnStart(args);
Console.Write("Press any key to stop program: ");
Console.ReadKey();
Console.Write("\r\nInvoking OnStop() ...");
service.OnStop();
Console.Write("Press any key to exit: ");
Console.ReadKey();
}
else
{
// The application is running as a service.
// Misnomer. The following registers the services with the Service Control Manager (SCM). It doesn't run anything.
ServiceBase.Run(service);
}
}

C#-C Interprocess communication

Okay, so I have a program written in c# that starts up a process running another program, this one written in C. Now I already worked out redirecting the stdout from the C program and picking it up on the C# side, and it works great. But I need a way for the C# program to tell the C program to end. My first thought, was that since the C program originally was controlled through testing kbhit from conio.h, that I could just redirect stdin and streamwrite something and it would detect a key hit, but that didn't work (I'm guessing kbhit doesn't lend itself to redirection), so I searched around and a lot of people kept suggesting PeekConsoleInput(), because I'm writing this program exclusively for windows (in fact a specific machine). So I changed the c program to use PeekConsoleInput instead of kbhit and redirected stdin and it still didn't detect anything I sent to it. For example in C...
#include <stdio.h>
#include <windows.h>
int main() {
BOOL peeked = 0;
HANDLE input_handle = GetStdHandle(STD_INPUT_HANDLE);
DWORD events = 0; // how many events took place
INPUT_RECORD input_record; // a record of input events
DWORD input_size = 1; // how many characters to read
printf("Hello World\n");
fflush(stdout);
while (1) {
FlushConsoleInputBuffer(input_handle);
peeked = PeekConsoleInput(input_handle, &input_record, input_size, &events);
if (peeked && events>0) {
break;
}
}
printf("Hit Detected\n");
return 0;
}
and a snippet from the C# code...
ProcessStartInfo info = new ProcessStartInfo("CTest.exe");
info.CreateNoWindow = true;
info.UseShellExecute = false;
info.RedirectStandardOutput = true;
info.RedirectStandardInput = true;
pro = new Process();
pro.StartInfo = info;
pro.Start();
StreamWriter sw = pro.StandardInput;
sw.AutoFlush = true;
pro.BeginOutputReadLine();
pro.OutputDataReceived += new DataReceivedEventHandler(pro_OutputDataReceived);
sw.WriteLine("Q");
The C program executes by itself, but when the C# program runs it, it never reaches "Hit Detected". I also delayed the s.WriteLine("Q") by a few seconds to see if it was a timing issue and it still didn't work.
Ideally, I wanted the C program to be able to run by itself, or be run by the C# program, but even if you couldn't run the C program by itself, that wouldn't be so bad.
One thing I tried that worked, was to have the C# program just write a file, and have the C program just poll with fopen's until it successfully opens it, but one of the main purposes of this C program is to write to data to disk really fast, and I worry that polling the disk may be slowing it down.
Another thing that kinda worked was just closing the process. But that's messy because the C program needs to clean up some stuff before it closes (unless there's some way to get the C program to execute some code before it shuts down, but I'm not sure if you can do that).
Other ways to get this done would be sockets, pipes, etc. but it seems like a lot of work for just a one bit signal. Also, all the examples I could find about how to do that all seemed to be how to get two C# programs to communicate or how to get two C programs to communicate, never two different programs in two different languages.
So, firstly, is there any way to get this stdin redirection thing working? and if not, what's the easiest solution to tell the C process that it needs to exit?

I ran into the same problem and was able to get it to work using SendKeys. Just add a reference to System.Windows.Forms if you have a WPF app.
For the processes StartInfo I used:
newProcess.StartInfo.UserShellExecute = false;
newProcess.StartInfo.CreateNoWindow = false;
newProcess.StartInfo.RedirectStandardInput = false;
newProcess.StartInfo.RedirectStandardOutput = true;
newProcess.StartInfo.RedirectStatndardError = true;
newProcess.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
newProcess.OutputDataReceived += new DataReceivedHandler(gotData);
The reason for setting CreatNoWindow to false is that you can find the Window in Spy++. ProcessWindowStyle.Hidden hides the window after it pops up for a brief moment.
RedirectStatndardInput must be false because we are using SendKeys to simulated keyboard input to the hidden window.
Using SendKeys.waitSend("command{ENTER}"); I was able to send a command to the console app that uses _kbhit() and getc() for input.

Have you tried to see if and by how much the C program is faster at writing files?
There are other ways (in Windows) to do interprocess communication.
Named Pipes (similar to TCP)
Memory mapped files (might be a good option here - similar to your
other experiment which worked)
TCP/IP (as you've suggested.)
As far as TCP/IP is concerned it is not constrained to works with software build using the same language and tool. Your C program will be the TCP server and your C# program will be the TCP client. If you know how to do this in either of the languages you've got your answer.

You could use the Process object to kill the spawned process:
(assuming that the C program has a handler to do its cleanup after receiving the "CloseMainWindow()" signal)
/// <summary>
/// Close the shelled process, if there is any
/// </summary>
/// <param name="reason">why closing?</param>
public static void shellTerminate(string reason)
{
Process p = shelledProcess;
try
{
if (p == null)
{
o3(reason + ". No shelled process.");
}
else if (p.HasExited)
{
o3(reason + ". Process has exited already.");
}
else
{
//Process is still running.
//Test to see if the process is hung up.
if (p.Responding)
{
//Process was responding; close the main window.
o3(reason + ". Process is being closed gracefully.");
if (p.CloseMainWindow())
{
//Process was not responding; force the process to close.
o3(reason + ". Process is being killed.");
p.Kill();
p.Close();
p = null;
}
}
if (p != null)
{
//Process was not responding; force the process to close.
o3(reason + ". Process is being killed.");
p.Kill();
}
p.Close();
}
if (shellOut != null)
{
shellOut.Close();
shellOut = null;
}
shelledProcess = null;
}
catch (Exception ex)
{
o("Exception in shellTerminate: " + ex.Message);
}
}

Checking if a console application is still running using the Process class

I'm making an application that will monitor the state of another process and restart it when it stops responding, exits, or throws an error.
However, I'm having trouble to make it reliably check if the process (Being a C++ Console window) has stopped responding.
My code looks like this:
public void monitorserver()
{
while (true)
{
server.StartInfo = new ProcessStartInfo(textbox_srcdsexe.Text, startstring);
server.Start();
log("server started");
log("Monitor started.");
while (server.Responding)
{
if (server.HasExited)
{
log("server exitted, Restarting.");
break;
}
log("server is running: " + server.Responding.ToString());
Thread.Sleep(1000);
}
log("Server stopped responding, terminating..");
try
{ server.Kill(); }
catch (Exception) { }
}
}
The application I'm monitoring is Valve's Source Dedicated Server, running Garry's Mod, and I'm over stressing the physics engine to simulate it stopping responding.
However, this never triggers the process class recognizing it as 'stopped responding'.
I know there are ways to directly query the source server using their own protocol, but i'd like to keep it simple and universal (So that i can maybe use it for different applications in the future).
Any help appreciated

The Responding property indicates whether the process is running a Windows message loop which isn't hung.
As the documentation states,
If the process does not have a MainWindowHandle, this property returns true.
It is not possible to check whether an arbitrary process is doing an arbitrary thing, as you're trying to.