Develop Reference

Trigger Apache Nutch Crawl Programmatically

I'm trying to create a ASP.NET web api to trigger a crawl event to happen. I can't seem to get cygwin to process any of the commands I give it. The only thing I can really do is get it to open a terminal. Once the terminal is open I'd have to redirect the pwd to another location and then trigger my command I want.
Process p = new Process();
ProcessStartInfo info = new ProcessStartInfo();
info.CreateNoWindow = false;
info.RedirectStandardInput = true;
info.UseShellExecute = false;
info.FileName = "C:\\cygwin64\\bin\\mintty.exe";
p.StartInfo = info;
p.Start();
StreamWriter sw = p.StandardInput;
if (sw.BaseStream.CanWrite)
{
sw.WriteLine(#"cd C:\Users\UName\Desktop\apache-nutch-2.3-mongodb\runtime\local\");
sw.WriteLine("bin/autoCrawl");
}
sw.Close();
p.WaitForExit();
I've tried many approaches, this is the last one I've tried but it just does nothing. Is there a way to launch this crawl from my .NET application? I've looked into the NutchApi about creating a new job with a type of crawl but I'm not sure if that applies here or not.

I ended up figuring out how to use the NutchApi to answer my question.

Interactive c# System.Process not Echoing input

Given the following code running in Mono on Linux, I can successfully run ssh from C# and get a shell prompt on a remote box. I can type commands and get output. However I can't figure out how to get what I type into that shell to echo back. When I type ls and hit enter you don't see the ls or the newline from hitting the enter key, you only see it's output. I've verified ssh is assigning a tty. The destination shell is bash in interactive mode so readline is enabled there. The problem has to be in how C# is wiring up the STDIN and STDOUT up to the Console. Google is no help so I'm hoping someone on here can help.
var process_info = new ProcessStartInfo("/usr/bin/ssh");
process_info.Arguments = "-ttt hostname";
Console.Out.WriteLine("Arguments: [" + process_info.Arguments + "]");
process_info.CreateNoWindow = true;
process_info.UseShellExecute = true;
var process = new Process();
process.StartInfo = process_info;
try {
process.Start();
process.WaitForExit();
exitCode = process.ExitCode;
}
catch (Exception e)
{
exitCode = this.ExitCode == 0 ? 255 : exitCode;
Console.WriteLine(e.ToString());
}
Console.Out.WriteLine("ExitCode: " + exitCode);

Maybe this is what you're trying to do:
using System;
using System.Diagnostics;
using System.IO;
using System.Threading;
namespace Echo
{
class Program
{
private static void Read(StreamReader reader)
{
new Thread(() =>
{
while (true)
{
int current;
while ((current = reader.Read()) >= 0)
Console.Write((char)current);
}
}).Start();
}
static void Main(string[] args)
{
ProcessStartInfo startInfo = new ProcessStartInfo(#"/usr/bin/ssh");
startInfo.Arguments = "-ttty localhost";
startInfo.CreateNoWindow = true;
startInfo.ErrorDialog = false;
startInfo.RedirectStandardError = true;
startInfo.RedirectStandardInput = true;
startInfo.RedirectStandardOutput = true;
startInfo.UseShellExecute = false;
startInfo.CreateNoWindow = true;
Process process = new Process();
process.StartInfo = startInfo;
process.Start();
Thread.Sleep(15000); //time to login
Read(process.StandardOutput);
Read(process.StandardError);
process.StandardInput.WriteLine("echoing your input now");
while (!process.HasExited)
try { process.StandardInput.WriteLine(Console.ReadLine()); }
catch {}
Console.WriteLine(process.ExitCode.ToString());
}
}
}
EDIT 1
You need to redirect the StandardInput in order to echo it, but then the cmd in windows will elaborate it line by line (even if you use Console.ReadKey() => process.StandardInput.Write), so you can't have shell support while typing (look at this question/answer if you want to dig in).
But mono with linux ssh behaves differently from windows cmd, so the following could be acceptable maybe:
commands are echoed and even the tab while typing a dir is managed (look at my screenshot below)! Finally please notice that the tty is correctly set.
using System;
using System.Diagnostics;
using System.IO;
using System.Text;
using System.Threading;
namespace Echo
{
class Program
{
private static Process process;
private static void Read(StreamReader reader)
{
new Thread(() =>
{
while (!process.HasExited)
{
int current;
while ((current = reader.Read()) >= 0)
Console.Write((char)current);
}
}).Start();
}
static void Main(string[] args)
{
ProcessStartInfo startInfo = new ProcessStartInfo(#"/usr/bin/ssh");
startInfo.Arguments = "-ttty localhost";
startInfo.CreateNoWindow = true;
startInfo.ErrorDialog = false;
startInfo.RedirectStandardError = true;
startInfo.RedirectStandardInput = true;
startInfo.RedirectStandardOutput = true;
startInfo.UseShellExecute = false;
startInfo.CreateNoWindow = true;
process = new Process();
process.StartInfo = startInfo;
process.Start();
Thread.Sleep(15000); //time to login
Read(process.StandardOutput);
Read(process.StandardError);
process.StandardInput.WriteLine("echo echoing your input now");
//Console.ReadLine();
string theLine = "\n";
while (!process.HasExited)
try {
ConsoleKeyInfo kinfo = Console.ReadKey(true);
char theKey = kinfo.KeyChar;
theLine += theKey;
process.StandardInput.Write(theKey) ;
process.StandardInput.Flush();
if (theKey.Equals('\n'))
{
Console.WriteLine(theLine);
theLine = "\n";
}
}
catch { }
Console.WriteLine(process.ExitCode.ToString());
}
}
}
EDIT 2
If you want to also manage the terminal escape sequences for the UpArrow/DownArrow, here is the code (tested on my Ubuntu terminal)
string theLine = "\n";
string theEsc = ((char)27).ToString();
while (!process.HasExited)
try {
//byte[] bytes = new byte[1];
ConsoleKeyInfo kinfo = Console.ReadKey(true);
char theKey = kinfo.KeyChar;
theLine += theKey;
switch (kinfo.Key)
{
case ConsoleKey.DownArrow:
process.StandardInput.Write(theEsc+"[B");
break;
case ConsoleKey.UpArrow:
process.StandardInput.Write(theEsc+"[A");
break;
default:
process.StandardInput.Write(theKey);
break;
}
process.StandardInput.Flush();
if (theKey.Equals('\n'))
{
Console.Write(theLine);
theLine = "\n";
}
}
EDIT 3
Just a follow up to my comments with a suggested command to restore echo (reference here).
This is the change to the code:
process.StandardInput.WriteLine("stty -a");
process.StandardInput.WriteLine("stty echo"); // or "reset" insted of "stty echo"
process.StandardInput.WriteLine("echo echoing your input now");
Back to your original code (since you're not redirecting the standard input), you could do something like the following
process_info.Arguments = "-ttt hostname 'stty echo; '$SHELL' -i'"; // or reset insted of stty echo
Look at this answer too.
In conclusion the source you're showing - and more specifically c# System.Process - is not supposed to echo anything (unless one intentionally redirects the standard I/O, as I've done here in my first example and in edit 1&2).
Echoing is a behavior of the shell, in Linux as well as in Windows: that can be managed as shown in edit 3.

I stumbled upon the same problem and the analysis of user4569980 helped a lot.
The root cause of this behavior is that mono disables the echo functionality of the currently used tty.
See http://www.linusakesson.net/programming/tty/ and https://github.com/mono/mono/blob/master/mcs/class/corlib/System/TermInfoDriver.cs#L204
I used the following workaround:
// mono sets echo off for some reason, therefore interactive mode
// doesn't work as expected this enables this tty feature which
// makes the interactive mode work as expected
let private setEcho (b:bool) =
// See https://github.com/mono/mono/blob/master/mcs/class/corlib/System/ConsoleDriver.cs#L289
let t = System.Type.GetType("System.ConsoleDriver")
if Env.isMono then
let flags = System.Reflection.BindingFlags.Static ||| System.Reflection.BindingFlags.NonPublic
if isNull t then
eprintfn "Expected to find System.ConsoleDriver.SetEcho"
false
else
let setEchoMethod = t.GetMethod("SetEcho", flags)
if isNull setEchoMethod then
eprintfn "Expected to find System.ConsoleDriver.SetEcho"
false
else
setEchoMethod.Invoke(null, [| b :> obj |]) :?> bool
else false
I'l leave it to the interested reader to convert this F# code to C#. It's basically simple reflection to bool System.ConsoleDriver.SetEcho(bool enable).
Now use the following pseudo code:
setEcho(true)
var p = startProcess ()
p.WaitForExit()
setEcho(false)

https://msdn.microsoft.com/de-de/library/system.diagnostics.processstartinfo.redirectstandardinput%28v=vs.110%29.aspx
Just capture and duplicate STDIN.
process.Start();
StreamWriter processInputStream = process.StandardInput;
do {
String inputText = Console.ReadLine():
processInputStream.write(inputText);
while(!process.HasExited)
process.WaitForExit();
Now the SSH process is no longer capturing your input, so it should be displayed locally. If it doesn't, just add Console.writeLine(inputText) to the loop, and done.
If you want better control, consider reading and writing byte wise. Just beware that TAB and other control characters might not be that easy to handle.
If you really need these as well, use ReadKey() instead and pass along whatever control characters you need. Remember to set Console.TreatControlCAsInput = true; or you won't be able to send CMD + c at all without killing your application.
Oh, but sending control sequences (keys with CMD or ALT modifier) from .NET outside of Windows?
Uh... I think that one is actually of limits. That is part of System.Window.Forms and I have no clue how to replicate that behavior with pure C# outside of Windows.
As for the other, and probably much easier option:
Just don't invoke the naked ssh executable. Open a shell instead and run SSH inside of that thing. /usr/bin/bash and "-c 'ssh -ttt hostname'". Your problem is TTY emulation, so just let the shell handle that for you. Console.TreatControlCAsInput = true; still applies though, at least if you want to be able to pass through Ctrl+C as a command sequence.

Calling jhead from a Console Application via Process does nothing

[Updated]
Turns out it couldn't not find the path to jpegtran. When you start a new process it does not seem to use the system's path variable.
I figured it out. I needed to add proc.StartInfo.WorkingDirectory = #"c:\ImageMagick";
This way it knows where to find the other program.
I've been banging my head on the desk. Based on all the examples I've seen it should be so simple.
I'm trying to run jhead.exe to auto rotate a bunch of images. I don't care if I do it one at a time or all at once. So I'm basically executing the
jhead.exe -autorot c:\RotateMe\imagename001.jpg
So to do this I've implemented the following code.
private void button3_Click(object sender, EventArgs e)
{
string strTarget = #"C:\RotateMe";
DirectoryInfo dir = new DirectoryInfo(strTarget);
int maxFiles = dir.GetFiles("*.jpg").Length;
System.Diagnostics.Process proc = new System.Diagnostics.Process();
foreach (FileInfo f in dir.GetFiles("*.jpg"))
{
proc.EnableRaisingEvents = false;
proc.StartInfo.FileName = #"c:\ImageMagick\jhead.exe";
//added this line
proc.StartInfo.WorkingDirectory = #"c:\ImageMagick";
proc.StartInfo.Arguments = string.Format(#"-autorot {0}", f.FullName.ToString());
proc.StartInfo.UseShellExecute = false;
proc.StartInfo.RedirectStandardOutput = true;
proc.StartInfo.CreateNoWindow = true;
proc.Start();
proc.WaitForExit();
}
}
Nothing happens.. If I set CreateNoWindow to False I get cmd boxes popping up. But too fast to see anything.
I've tried getting the StandardOutput to write to the console so I can at least see what happens. But I get nothing there either.
I know this isn't a whole lot to go on. But it seems simple enough... and I'm not getting it.

Redirecting standard input/output/error streams with .NET's Process class

I'm trying to write a wrapper for an interactive console-based application. For this I use C# and the Process class. I'm trying to redirect stdin/out/err, but it doesn't work.
Example code:
ProcessStartInfo startInfo = new ProcessStartInfo("admin.exe");
startInfo.RedirectStandardOutput = true;
startInfo.RedirectStandardError = true;
startInfo.RedirectStandardInput = true;
startInfo.UseShellExecute = false;
Process process = Process.Start(startInfo);
process.BeginOutputReadLine();
process.BeginErrorReadLine();
process.OutputDataReceived += (s, e) => Console.WriteLine(e.Data);
process.ErrorDataReceived += (s, e) => Console.WriteLine(e.Data);
while (true)
{
process.StandardInput.Write("uptime" + Environment.NewLine);
process.StandardInput.Flush();
Thread.Sleep(1000);
}
Console.ReadKey();
Nothing happens. But if I start admin.exe and write uptime, output is printed.
All solutions in the internet use ReadToEnd, but I can't use this because i have a dynamic communication on which I have to read stdout/err and write to stdin.
Has anyone an idea?
Update
I played with the posted zip on the linked thread. And then i tried to create a small 'proof-of-concept'-code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.IO;
using System.Threading;
namespace ConsoleApplication3
{
class Program
{
private static void Read(StreamReader reader)
{
new Thread(() =>
{
while (true)
{
int current;
while ((current = reader.Read()) >= 0)
Console.Write((char)current);
}
}).Start();
}
static void Main(string[] args)
{
ProcessStartInfo startInfo = new ProcessStartInfo(#"cmd.exe");
startInfo.CreateNoWindow = true;
startInfo.ErrorDialog = false;
startInfo.RedirectStandardError = true;
startInfo.RedirectStandardInput = true;
startInfo.RedirectStandardOutput = true;
startInfo.UseShellExecute = false;
startInfo.CreateNoWindow = true;
Process process = new Process();
process.StartInfo = startInfo;
process.Start();
Read(process.StandardOutput);
Read(process.StandardError);
while (true)
process.StandardInput.WriteLine(Console.ReadLine());
}
}
}
It works perfectly:-)
But with the admin.exe it doesn't work? The admin.exe doesn't use any tricky input-method and it doesn't need an inputed password.
I know the admin.exe is written in c and compiled with mingw on linux. So i created a small dummy-tool:
#include <stdio.h>
int main() {
int readed;
while ((readed = fgetc(stdin)) >= 0)
fputc((char)readed, stdout);
}
This tool does only echo the inputed text/line. I compiled it with i586-mingw32msvc-gcc and copied it to my windows machine. There i used the program on the top of this post to communicate with the dummy.exe. It doesn't work. No echo is shown. But why?
I compiled the dummy-code also with the Microsoft C++ Compiler, same effect.
Update2
(btw: Thanks to Tim Post)
I'm trying and trying. I tried to create a c-Tool, which does the same as my c# tool. I used _popen, but the effect was, that the output were shown at the end of the process. Hm, not good.
I found this alternative command shell for windows:
https://stackoverflow.com/questions/440269/whats-a-good-alternative-windows-console
http://sourceforge.net/projects/console/
It seems to work. It gets the stdout/err in realtime, can redirect the stdin and admin.exe works. And it is opensource. May be i'll find the solution inside the C++-Code.
I'm not well in C++, so it's hard, but i'll try it. May be i have to write a "clear" redirect-wrapper in C/C++ and use it in C#.
If someone has an idea please say it, because the other way can be very hard (for me^^):-)
Thanks.
best regards
Update 3
Hm, i think this happens because the child-process (admin.exe) uses a few threads...
But how to solve it?

The problem could be because of you are using Readline, where the data are output from admin.exe application are sequentially and not in new lines.., try to use Read instead, and build the desirable string from it..
Also you don't have to use Environment.NewLine to write string followed by new line, use WriteLine instead, so:
process.StandardInput.WriteLine("uptime");

Control console application from Windows form application C#

I have 2 applications.
One of them is console application, the other is normal form application - both written in C#. I want to open (hidden from view) the console application form the windows form application and be able to send a command lines to the console application.
How can i do that?

You can start the background process
ProcessStartInfo startInfo = new ProcessStartInfo();
startInfo.FileName = "Myapplication.exe";
startInfo.WindowStyle = ProcessWindowStyle.Hidden;
Process process = new Process();
process.StartInfo = startInfo;
process.Start();
and after that use the Process.StandardOutput property
// This is the code for the base process
Process myProcess = new Process();
// Start a new instance of this program but specify the 'spawned' version.
ProcessStartInfo myProcessStartInfo = new ProcessStartInfo(args[0], "spawn");
myProcessStartInfo.UseShellExecute = false;
myProcessStartInfo.RedirectStandardOutput = true;
myProcess.StartInfo = myProcessStartInfo;
myProcess.Start();
StreamReader myStreamReader = myProcess.StandardOutput;
// Read the standard output of the spawned process.
string myString = myStreamReader.ReadLine();
Console.WriteLine(myString);
myProcess.WaitForExit();
myProcess.Close();
If you want to send commands to this process, just use Process.StandardInput Property
// Start the Sort.exe process with redirected input.
// Use the sort command to sort the input text.
Process myProcess = new Process();
myProcess.StartInfo.FileName = "Sort.exe";
myProcess.StartInfo.UseShellExecute = false;
myProcess.StartInfo.RedirectStandardInput = true;
myProcess.Start();
StreamWriter myStreamWriter = myProcess.StandardInput;
// Prompt the user for input text lines to sort.
// Write each line to the StandardInput stream of
// the sort command.
String inputText;
int numLines = 0;
do
{
Console.WriteLine("Enter a line of text (or press the Enter key to stop):");
inputText = Console.ReadLine();
if (inputText.Length > 0)
{
numLines ++;
myStreamWriter.WriteLine(inputText);
}
} while (inputText.Length != 0);

One of possible solutions can be IPC, in particularly
NamedPipes
That is already wrapped in .NET 4.0.
Regards.

To start the console application, use the System.Diagnostics.Process class.
To send commands to the console application you need something that is called Interprocess Communication. One way to do it is by using WCF. A simple tutorial can be found here.