Develop Reference

Why is OutputDataReceived not being raised for prompt, preventing AutoResetEvent from being signaled?

I'm making an app that will spawn up a process that has a command-line interpreter. I need to supply commands to this CLI from another machine. Now, I have to detect when the command is finished, so I'm checking for when the CLI's prompt appears in the standard output of the process I'm spawning. Here's a snippet of code:
private string StartProcess(string input)
{
try
{
StringBuilder output = new StringBuilder();
StringBuilder error = new StringBuilder();
AutoResetEvent commandFinished = new AutoResetEvent(false);
ProcessStartInfo startInfo = new ProcessStartInfo()
{
FileName = "myprocess.exe",
Arguments = "",
UseShellExecute = false,
CreateNoWindow = true,
RedirectStandardOutput = true,
RedirectStandardError = true,
RedirectStandardInput = true,
UserName = System.Environment.UserName
};
Process myProcess = new Process()
{
StartInfo = startInfo,
EnableRaisingEvents = true
};
myProcess.OutputDataReceived += new DataReceivedEventHandler((sender, e) =>
{
if (e.Data != null)
{
string prompt = "user >";
if (e.Data.Substring(e.Data.Length - prompt.Length).Equals(prompt))
{
Console.WriteLine("Received prompt! Sending CommandFinished signal!");
commandFinished.Set();
Console.WriteLine("CommandFinished signal set!");
}
else
{
output.AppendLine(e.Data);
}
}
else
{
// Data finished
Console.WriteLine("StdOut data finished! Sending CommandFinished signal!");
commandFinished.Set();
Console.WriteLine("CommandFinished signal set!");
}
});
myProcess.ErrorDataReceived += new DataReceivedEventHandler((sender, e) =>
{
if (e.Data != null)
{
Console.WriteLine("Error Data received: " + e.Data.ToString());
error.AppendLine(e.Data);
}
});
myProcess.Start();
myProcess.BeginOutputReadLine();
myProcess.BeginErrorReadLine();
Console.WriteLine("Executing input command: " + input);
myProcess.StandardInput.WriteLine(input);
Console.WriteLine("Waiting for input command to complete...");
commandFinished.WaitOne();
Console.WriteLine("Command complete!");
return output.ToString();
}
catch (Exception ex)
{
Console.WriteLine("EXCEPTION: " + ex.ToString());
throw ex;
}
}
Now, the code is hanging on the call to WaitOne(). I'm confused as to why - I don't detect the CLI prompt in the output, and I never get any WriteLines telling me the prompt was received in the OutputDataReceived event, or that the Data received was null. I.e. the OutputDataReceived event isn't being raised when the previous command has completed and the prompt is displayed.
The input command I'm supplying does take a while, but it does finish. Am I using AutoResetEvent wrong here?

Without a good, minimal, complete code example it's impossible to know for sure what might be wrong with the code. But I can see one obvious explanation:
The OutputDataReceived event is raised for each line of output that is received. That is, a string terminated by a newline character.
Without specifics about your external process, I can't say for sure. But most CLI type scenarios involve the display of a prompt without a newline character. I.e. the prompt is written to the console and user input is expected to be echoed to the console immediately following the prompt, rather than on the next line.
If that's the case for your scenario, then almost certainly you are failing to detect the prompt because the event won't be raised for the prompt alone. After the final line of output from the previous command's operation, the next time the event will be raised is after the command has been sent to the process via standard input. This is obviously too late to be useful in knowing when to send that command. :)
To get this to work, you will have to read input from the process via one of the other mechanisms available that are not line-based.

Thread is stuck when executing SerialPort.Close()

I am facing a problem that a thread in my application runs into a deadlock when I want to close the serial port of a gsm terminal. This problem is well known here and here but all advices in these threads didn't help me.
/// <summary>
/// Closes the COM port, prevents reading and writing
/// </summary>
public void Stop()
{
Debug.WriteLine("stop called");
var block = true;
var bgw = new BackgroundWorker
{
WorkerReportsProgress = false,
WorkerSupportsCancellation = false,
};
bgw.DoWork += (s, e) =>
{
if (!CanAccessPort())
return;
try
{
_serialPort.DataReceived -= Read;
GC.ReRegisterForFinalize(_serialPort.BaseStream);
_serialPort.Close();
_isOpen = false;
}
catch (Exception ex)
{
throw new Exception(PORTERROR, ex);
}
};
bgw.RunWorkerCompleted += (s, e) =>
{
Debug.WriteLine("block is set to false =)");
block = false;
};
bgw.RunWorkerAsync();
while (block)
Thread.Sleep(250);
}
The code above runs forever when _serialPort.Close() is executed. As a recommended advice I read about running the close operation in a separate thread. I tried with BackgroundWorker and Thread classes but nothing did work. Using AutoResetEvent as suggested in another thread did not work either. Before closing the port I am sending some commands to it and receive several results but it won't close. When I run a simple command line program that starts the port, reads data and tries to close it, everything works and even without threads.
What could cause the deadlock? I am not doing any GUI related stuff as mentioned in almost all other answers.
DataReceived event handler code here:
/// <summary>
/// Reads input from the COM interface and invokes the corresponding event depending on the input
/// </summary>
private void Read(object sender, SerialDataReceivedEventArgs e)
{
var buffer = new char[1024];
var counter = 0;
_keepRunning = true;
if (_timeout == null)
{
// timeout must be at least 3 seconds because when sending a sms to the terminal the receive notification (+CSDI) can be lost with less timeout
_timeout = new Timer(3000);
_timeout.Elapsed += (s, ev) =>
{
_keepRunning = false;
_timeout.Stop();
};
}
_timeout.Start();
// cancel condition: no more new data for 3 seconds or "OK"/"ERROR" found within the result
while (_keepRunning)
{
var toRead = _serialPort.BytesToRead;
if (toRead == 0)
{
Thread.Sleep(100);
continue;
}
_timeout.Stop();
_timeout.Start();
counter += _serialPort.Read(buffer, counter, toRead);
// ok or error found in result string
var tmp = new string(buffer).Replace("\0", "").Trim();
if (tmp.EndsWith("OK") || tmp.EndsWith("ERROR"))
{
_timeout.Stop();
_keepRunning = false;
}
}
// remove empty array slots from the back
var nullTerminalCounter = 0;
for (var i = buffer.Length - 1; i != 0; i--)
{
if (buffer[i] == '\0')
{
nullTerminalCounter++;
continue;
}
break;
}
Array.Resize(ref buffer, buffer.Length - nullTerminalCounter);
var str = new String(buffer).Trim();
// result must be something different than incoming messages (+CMTI: \"MT\", 25)
if (!((str.StartsWith("+CMTI") || str.StartsWith("+CSDI")) && str.Length < 20))
{
// when an incoming message is received, it does not belong to the command issued, so result has not yet arrived, hence port is still blocked!
_isBlocked = false;
Debug.WriteLine("port is unblocked");
}
var args = new CommandReturnValueReceivedEventArgs
{
ResultString = str
};
OnCommandReturnValueReceived(this, args);
}

if (toRead == 0)
{
Thread.Sleep(100);
continue;
}
This code is the basic source of the deadlock. The rule for SerialPort.Close() is that it can only close the serial port when none of the event handlers for SerialPort are active. Problem is, your DataReceived event handler is almost always active, waiting for data. This was not the intended use for the event. You are supposed to read whatever is available from the serial port, typically appending bytes to a buffer and get out. The event fires again when more bytes are available.
while (_keepRunning)
Looks like you discovered this problem and tried to fix it with the Timer. That doesn't work either, in a very ratty way that's very difficult to debug. A bool variable is not a proper synchronization primitive, like ManualResetEvent. The while() loop will not see the _keepRunning variable turn to false when you target x86 and run the Release build of your program. Which enables the jitter optimizer, it is apt to store the variable in a cpu register. Declaring the variable volatile is required to suppress that optimization.
I suspect, but can't guarantee, that using volatile will solve your problem. And you probably want to set _keepRunning to false before calling Close() so you don't get the timeout delay.
A more structural fix is however indicated. Rewrite the DataReceived event handler so it never loops waiting for data. Given that this appears to be code to talk to a modem, a simple _serialPort.ReadLine() call should be all that's required.

The issues I had went away if I ensured that Open and Close never were called at the same time by any thread. I did this by using a lock.
The essence is the following.
lock(_port)
_port.Open(.....);
...
lock(_port)
_port.Close(.....);

Process takes longer to finish than in CMD

I am running a program that runs driverquery.exe command and gets back the ouput
pProcess.StartInfo.CreateNoWindow = true;
debugLog.WriteToLog("");
pProcess.StartInfo.UseShellExecute = false;
pProcess.StartInfo.RedirectStandardOutput = true;
pProcess.StartInfo.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden;
debugLog.WriteToLog("before start method");
pProcess.Start();
debugLog.WriteToLog("after start method");
if (windowTitleToHide.Length > 0)
{
WindowsUtil.HideWindow(windowTitleToHide);
}
if (null != attributeName && !"".Equals(attributeName))
Console.WriteLine(attributeName + " : ");
debugLog.WriteToLog("before read to end");
StreamReader reader = pProcess.StandardOutput;
String strOutput = string.Empty;
while (reader.Peek() > -1)
strOutput += reader.ReadLine();
debugLog.WriteToLog("after read to end");
Console.WriteLine(strOutput);
debugLog.WriteToLog("before wait for exit");
pProcess.WaitForExit();
debugLog.WriteToLog("after wait for exit");
pProcess.Close();
The process takes about 30 minutes to finish.If i run the same process via cmd it always finishes in 2 minutes. I have tried using readtoend instead of readline but that also did not help. Can some tell what is wrong here? in my logs i can see the last line getting printed as before wait for exit
PS: When i see the processes in the taskmanager the driverquery.exe is running but not consuming any CPU cycles. The process calling this code is consuming about 99% CPU. I know for sure that the calling code is not doing any other task while running this code.

I might guess the problem is related to :
while (reader.Peek() > -1)
strOutput += reader.ReadLine();
You probably are not reading the full output of your application. If there is any pause in its output then reader.Peek() will return -1 before the application is finished its full output. If it is outputting a lot of data you may even be overflowing the output stream since your process will have given up reading after emptying the stream once. If this is the case, the child process may be generating a lot of exceptions outputting to a full stream (which will dramatically increase execution time). Profiling and debugging would tell you more about what is actually going on.
You might try an async approach like this :
pProcess.StartInfo.RedirectStandardOutput = true;
pProcess.EnableRaisingEvents = true; // Enable events
pProcess.OutputDataReceived += outputRedirection; // hook up
pProcess.Start();
pProcess.BeginOutputReadLine(); // use async BeginOutputReadLine
pProcess.WaitForExit();
where
static void outputRedirection(object sendingProcess,
DataReceivedEventArgs outLine)
{
try
{
if (outLine.Data != null)
Console.WriteLine(outLine.Data);
// or collect the data, etc
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
return;
}
}
Rather than polling in a tight loop (which is likely to empty the output stream faster than the child process will fill it, and therefore fail), this waits for data to come in. Your main process will still block on the call to WaitForExit but a threadpool thread will handle the incoming events.
EDIT
Here is an SSCCE that works just fine :
static void Main(string[] args)
{
Stopwatch spw = new Stopwatch();
spw.Start();
Process pProcess = new Process();
pProcess.StartInfo.FileName = "driverquery.exe";
pProcess.StartInfo.CreateNoWindow = true;
pProcess.StartInfo.UseShellExecute = false;
pProcess.StartInfo.RedirectStandardOutput = true;
pProcess.EnableRaisingEvents = true;
pProcess.OutputDataReceived += outputRedirection;
pProcess.Start();
pProcess.BeginOutputReadLine();
pProcess.WaitForExit();
pProcess.Close();
spw.Stop();
Console.WriteLine();
Console.WriteLine("Completed in : " +
spw.ElapsedMilliseconds.ToString()
+ "ms");
}
Using outputRedirection as define above --> output :
If this isn't working for you, then please show us your complete, real code. Something else you are doing is wrong.

C# - Accessing the STDIN and STDOUT handles of a process without redirecting them

I'm trying to access the input and output streams of a process my application starts. My code is:
this.App = new Process();
this.App.StartInfo.FileName = this.AppPath;
this.App.StartInfo.Arguments = this.AppArgs;
this.App.StartInfo.RedirectStandardOutput = true;
this.App.StartInfo.UseShellExecute = false;
this.App.Start();
ThreadStart ts = new ThreadStart(() =>
{
while (true)
{
if (this.App.StandardOutput != null && !this.App.StandardOutput.EndOfStream)
{
Console.Write((char)this.App.StandardOutput.Read());
}
}
}
this.listener = new Thread(ts);
this.listener.IsBackground = true;
this.listener.Start();
this.App.WaitForExit();
The code does work perfectly. It prints any output of the spawned process into console. The problem is, that the spawned process is a Source Dedicated Server, which doesn't like when it's STDIN/STDERR/STDOUT handles are redirected - throws up an error MessageBox and dies once it's closed. I have spent hours trying to mess with kernel32.dll, but failed.
I'm looking for a way to access the handles/output of the spawned process without redirecting them, as I'm obviously not allowed to do that.
Anyone?
Please.

Deadlock on out/error buffer

Here is the code:
var output = new StringWriter();
var error = new StringWriter();
var p = new Process();
p.StartInfo = new ProcessStartInfo
{
WorkingDirectory = workingDir,
FileName = file,
Arguments = args,
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
};
p.OutputDataReceived += (o, e) => { output.WriteLine(e.Data); };
p.ErrorDataReceived += (o, e) => { error.WriteLine(e.Data); };
p.Start();
if (!p.WaitForExit((int)timeout.TotalMilliseconds))
{
p.Kill();
throw new Exception("Timeout expired.");
}
string errorData = error.ToString();
if (errorData.Length > 0)
throw new Exception(errorData);
return output.ToString();
I would expect OutputDataReceived and ErrorDataReceived handlers to be called whenever data becomes available, however that is not happening. For a process that outputs ton of stuff into stdout this code eventually exits on throw new Exception("Timeout expired.");. I tried throwing some debug code into handlers, but it is never called. Isn't the whole point of specifying out/err handlers is to have them called asynchronously?

From the docs:
The event is enabled during asynchronous read operations on StandardOutput. To start asynchronous read operations, you must redirect the StandardOutput stream of a Process, add your event handler to the OutputDataReceived event, and call BeginOutputReadLine. Thereafter, the OutputDataReceived event signals each time the process writes a line to the redirected StandardOutput stream, until the process exits or calls CancelOutputRead.
You are not performing any asynchronous reads, therefore, the event will not be fired.
If you simply add a call to p.BeginOutputReadline() after starting the process it will work. Similarly, to read the error stream, call BeginErrorReadLine().

To start receiving the events from standard output and standard error, you need to call BeginOutputReadLine() and BeginErrorReadLine(), respectively.