Develop Reference

How to combine multiple gz files into one from Process in C# program when one is missing EOF

I have multiple .gz files in a directory (2 or more), with at least one file missing the end of file marker. Our C# process is unable to read the file with missing end of file, but since they are coming from a third party we do not have control over how they are created.
As such, we've been running the following Linux command manually:
cat file1.gz file2.gz > newFile.gz
In order to automate this, I am looking for a way to leverage the Process functionality in C# to trigger the same command, but this would only be available in Cygwin or some other Linux shell. In my example, I'm using git bash but it could be Powershell or Cygwin or any other available Linux shell that runs on a Windows box.
The following code does not fail, but it does not work as expected. I am wondering if anyone has recommendations about how to do this or any suggestions on a different approach to consider?
Assume that the working directory is set and initialized successfully, so the files exist where the process is run from.
Process bashProcess = new Process();
bashProcess.StartInfo.FileName = #"..\Programs\Git\git-bash.exe";
bashProcess.StartInfo.UseShellExecute = false;
bashProcess.StartInfo.RedirectStandardInput = true;
bashProcess.StartInfo.RedirectStandardOutput = true;
bashProcess.Start();
bashProcess.StandardInput.WriteLine("cat file1.gz file2.gz > newFile.gz");
bashProcess.StandardInput.WriteLine("exit");
bashProcess.StandardInput.Flush();
.
.
.
bashProcess.WaitForExit();
My expectation is that newFile.gz is created

I was able to find a solution to my problem using a DOS command, and spawning a cmd Process from CSharp.
My code now looks like this, avoids having to launch a linux-based shell from Windows, and the copy command in windows does the same thing as cat:
Process proc = new Process();
proc.EnableRaisingEvents = false;
proc.StartInfo.FileName = "cmd";
proc.StartInfo.Arguments = #"/C pushd \\server\folder && copy *.txt.gz /b
combined.gz";
proc.StartInfo.RedirectStandardOutput = true;
proc.StartInfo.RedirectStandardError = true;
proc.StartInfo.UseShellExecute = false;
proc.Start();
proc.WaitForExit();
System.Threading.Thread.Sleep(2000);
string line = proc.StandardOutput.ReadLine();
while (line != null)
{
output.Append(line);
line = proc.StandardOutput.ReadLine();
}

Calling Batch File From C#

I am hoping that this is an easy question, but i have the following code in my C# application and for some reason it will not execute the batch file I am pointing to.
private void filesystemwatcher_Renamed(object sender, System.IO.RenamedEventArgs e)
{
if (File.Exists("C:\\Watcher\\File.txt"))
{
System.Diagnostics.Process proc = new System.Diagnostics.Process();
proc.EnableRaisingEvents = false;
proc.StartInfo.FileName = "C:\\Watcher\\Cleanup.bat";
proc.Start();
MessageBox.Show("Cleaned up files, your welcome.");
}
else
{
label4.Text = "Error: No file found";
}
}
It will display the messagebox correctly so I know that it is reaching that area of code, but I do not see a cmd box pop up or anything that would show that it just ran the batch file. I can also tell because cleanup.bat just renames a file and that's it. After I get the messagebox the file name hasn't changed.
If I double click the batch file manually it works just fine. I have also adjusted the permissions of the batch file to Full Control for everyone (just for testing purposes)

This should work
System.Diagnostics.Process proc = new System.Diagnostics.Process();
proc.StartInfo.FileName = "C:\\Watcher\\Cleanup.bat";
proc.StartInfo.WorkingDirectory = "C:\\Watcher";
proc.Start();
You need to set the WorkingDirectory otherwise the command will be executed in what is the current directory of the calling application

Try setting the proc.StartInfo.UseShellExecute to true; this tells the OS to perform a lookup of the file extension to find the correct handler in the registry.

Diagnostics.Process - Dump output to file

Hi I need to write the result of a mysqldump to a file with a standard windows commands.
System.Diagnostics.Process proc = new System.Diagnostics.Process();
proc.EnableRaisingEvents = false;
proc.StartInfo.WorkingDirectory = "sample directory";
proc.StartInfo.FileName = "mysqldump";
proc.StartInfo.Arguments = "-u root -pPassword --all-databases > db.sql";
proc.StartInfo.RedirectStandardOutput = false;
proc.StartInfo.UseShellExecute = false;
proc.Start();
proc.WaitForExit();
But it doesn't write to file this way...
I don't want to read the output and then write it to file, since mysqldump output can become really big...
Any solutions?

Try executing through cmd.exe and enquote the command to keep your program from gobbling up the redirect:
proc.StartInfo.FileName = "cmd.exe";
proc.startinfo.Arguments =
"/c \"mysqldump -u root -pPassword --all-databases\" > db.sql"

If it's a lot of output you can use the proc.OutputDataReceived event, in the event handler just write the output to your file.
Read the MSDN article here

For piped output you may need ShellExecute to be true. If that doesnt work, you may had to pipe it yourself (i.e. either handle the data events, or have an async read/write loop), but the size shouldn't matter since you should only be reading small chunks of it at any time.

Pretty much on similar lines, but written in VB.NET. Convert it and you should be good... Link

Open txt file from C# application

The following code is suppose to open CMD from my C# application and open the file text.txt.
I tried to set the file path as an environment variable but when notepad opens it looks for %file%.txt instead of text.txt
Any idea why?
System.Diagnostics.Process proc = new System.Diagnostics.Process();
proc.EnableRaisingEvents=false;
proc.StartInfo.EnvironmentVariables.Add("file", "c:\\text.txt");
proc.StartInfo.UseShellExecute = false;
proc.StartInfo.FileName = "notepad";
proc.StartInfo.Arguments="%file%";
proc.Start();
proc.WaitForExit();
Console.WriteLine(proc.ExitCode);

If your purpose is to start the editor with a .txt file (like the title of the question says) just use:
Process.Start("C:\\text.txt")

The short version is that I suspect you are going to have to pass the arg more directly, i.e.
proc.StartInfo.Arguments = #"""c:\text.txt""";
Although you can set environment variables (for use within the process), I don't think you can use them during the process start.

What are you trying to accomplish with %file%? The command line argument for notepad.exe is the file you want to open. You need to do something like this:
proc.StartInfo.Arguments = "c:\\text.txt";

set UseShellExecute = true
that way it should use the cmd.exe processor to expand the %file% variable

One obvious problem is that you have UseShellExecute set false. This means you are executing notepad directly without passing via the command shell cmd.exe. Therefore environment variables aren't being expanded.
I'm not sure what you're trying to achieve (why do you need to add an environment variable?) but the following would work:
System.Diagnostics.Process proc =
new System.Diagnostics.Process();
proc.EnableRaisingEvents = false;
proc.StartInfo.EnvironmentVariables.Add("file", "c:\\text.txt");
proc.StartInfo.UseShellExecute = false;
proc.StartInfo.FileName = "cmd.exe";
proc.StartInfo.Arguments = "/c notepad %file%";
proc.Start();
proc.WaitForExit();

Try this:
proc.StartInfo.Arguments = System.Environment.GetEnvironmentVariable("file");

Perhaps it has to do with how the StartInfo.Arguments work. If you can't find anything better, this worked for me:
proc.StartInfo.FileName = "cmd";
proc.StartInfo.Arguments="/c notepad %my_file%";

I am willing to bet you need to set WorkingDirectory to get this to work. NOTEPAD.exe is usually located in %SYSTEMROOT% (C:\windows) however the default is %SYSTEMROOT%\system32. Try out the below.
System.Diagnostics.Process proc = new System.Diagnostics.Process();
proc.EnableRaisingEvents=false;
proc.StartInfo.WorkingDirectory = "%SYSTEMROOT%";
proc.StartInfo.EnvironmentVariables.Add("file", "c:\\text.txt");
proc.StartInfo.UseShellExecute = false;
proc.StartInfo.FileName = "notepad";
proc.StartInfo.Arguments="%file%";
proc.Start();
proc.WaitForExit();
Console.WriteLine(proc.ExitCode);

Service hangs up at WaitForExit after calling batch file

I have a service that sometimes calls a batch file. The batch file takes 5-10 seconds to execute:
System.Diagnostics.Process proc = new System.Diagnostics.Process(); // Declare New Process
proc.StartInfo.FileName = fileName;
proc.StartInfo.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden;
proc.StartInfo.CreateNoWindow = true;
proc.Start();
proc.WaitForExit();
The file does exist and the code works when I run the same code in-console. However when it runs inside the service, it hangs up at WaitForExit(). I have to kill the batch file from the Process in order to continue. (I am certain the file exists, as I can see it in the processes list.)
How can I fix this hang-up?
Update #1:
Kevin's code allows me to get output. One of my batch files is still hanging.
"C:\EnterpriseDB\Postgres\8.3\bin\pg_dump.exe" -i -h localhost -p 5432 -U postgres -F p -a -D -v -f "c:\backupcasecocher\backupdateevent2008.sql" -t "\"public\".\"dateevent\"" "DbTest"
The other batch file is:
"C:\EnterpriseDB\Postgres\8.3\bin\vacuumdb.exe" -U postgres -d DbTest
I have checked the path and the postgresql path is fine. The output directory does exist and still works outside the service. Any ideas?
Update #2:
Instead of the path of the batch file, I wrote the "C:\EnterpriseDB\Postgres\8.3\bin\pg_dump.exe" for the proc.StartInfo.FileName and added all parameters to proc.StartInfo.Arguments. The results are unchanged, but I see the pg_dump.exe in the process window. Again this only happens inside the service.
Update #3:
I have run the service with a user in the administrator group, to no avail. I restored null for the service's username and password
Update #4:
I created a simple service to write a trace in the event log and execute a batch file that contains "dir" in it. It will now hang at proc.Start(); - I tried changing the Account from LocalSystem to User and I set the admnistrator user and password, still nothing.

Here is what i use to execute batch files:
proc.StartInfo.FileName = target;
proc.StartInfo.RedirectStandardError = true;
proc.StartInfo.RedirectStandardOutput = true;
proc.StartInfo.UseShellExecute = false;
proc.Start();
proc.WaitForExit
(
(timeout <= 0)
? int.MaxValue : timeout * NO_MILLISECONDS_IN_A_SECOND *
NO_SECONDS_IN_A_MINUTE
);
errorMessage = proc.StandardError.ReadToEnd();
proc.WaitForExit();
outputMessage = proc.StandardOutput.ReadToEnd();
proc.WaitForExit();
I don't know if that will do the trick for you, but I don't have the problem of it hanging.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
namespace VG
{
class VGe
{
[STAThread]
static void Main(string[] args)
{
Process proc = null;
try
{
string targetDir = string.Format(#"D:\adapters\setup");//this is where mybatch.bat lies
proc = new Process();
proc.StartInfo.WorkingDirectory = targetDir;
proc.StartInfo.FileName = "mybatch.bat";
proc.StartInfo.Arguments = string.Format("10");//this is argument
proc.StartInfo.CreateNoWindow = false;
proc.Start();
proc.WaitForExit();
}
catch (Exception ex)
{
Console.WriteLine("Exception Occurred :{0},{1}", ex.Message,ex.StackTrace.ToString());
}
}
}
}

string targetDir = string.Format(#"D:\");//PATH
proc = new Process();
proc.StartInfo.WorkingDirectory = targetDir;
proc.StartInfo.FileName = "GetFiles.bat";
proc.StartInfo.Arguments = string.Format("10");//argument
proc.StartInfo.CreateNoWindow = false;
proc.Start();
proc.WaitForExit();
Tested,works clear.

What does the batch file do? Are you certain the process is getting launched with enough privs to execute the batch file? Services can be limited in what they are allowed to do.
Also make sure if you are doing something like usin the copy command to overwrite a file that you do something like:
echo Y | copy foo.log c:\backup\
Also, make sure you are using full paths for the batch commands, etc. If the batch file is launching a GUI app in some sort of "Console" mode, that may be an issue too. Remember, services don't have a "Desktop" (unless you enable the "interact with desktop") to draw any kind of windows or message boxes to. In your program, you might want to open the stdout and stderr pipes and read from them during execution in case you are getting any error messages or anything.
WebServices are probably executing as the IUSR account, or the anonymous account, which ever, so that might be an issue for you. If it works when you run it in console, that's just the first step. :)
I don't recall if System.Diagnostics. are available only in debug or not. Probably not, but some of them might be. I'll have to check up on that for ya.
Hope this gives you some ideas.
Larry

pg_dump.exe is probably prompting for user input. Does this database require authentication? Are you relying on any ENVIRONMENT variables that won't be present for the service? I don't know pg_dump but what are the other possible reasons it would prompt for input?

The next step I would take is to fire up the debugger, and see if you can tell what the program is waiting on. If you are expierenced at debugging in assembly, you may be able to get an IDEA of what's happening using tools like ProcExp, FileMon, etc.
Being a windows SERVICE, and not a web service, makes quite a bit of difference. Anyways, have you tried my suggestion of setting the "Allow Service to interact with desktop"?
If you are desperate, you might try launching cmd.exe instead of your batch file. Then, using the cmd.exe's cmd line parameters, you can have IT start the batch file. This would probably give you a cmd prompt window to view the actual output, if you turn on the interact with desktop.
For complete help on cmd.exe, just type cmd /? at any command prompt.
Larry

Here is the solution. The solution is not clear because I have changed so many time the code and now it's working!
I have tried to use a Account of User, and it's not what worked. Use LocalSystem. Here is the code that execute, mostly what Kevin gave me.
System.Diagnostics.Process proc = new System.Diagnostics.Process();
proc.StartInfo.FileName = fileName;
proc.StartInfo.RedirectStandardError = true;
proc.StartInfo.RedirectStandardOutput = true;
proc.StartInfo.UseShellExecute = false;
proc.Start();
proc.WaitForExit();
output1 = proc.StandardError.ReadToEnd();
proc.WaitForExit();
output2 = proc.StandardOutput.ReadToEnd();
proc.WaitForExit();
Thank you all, I'll up-vote everybody and accept Kevin since he helps me since the beginning. Very weird because it works now...

Daok, it looks as if the only thing you changed was the timeout period on the initial WaitForExit(). You need to be VERY careful of that. If something DOES hang your service, it will NEVER return (and well, pretty much work like it has been for you thus far.. heh), but it won't be good for the end users...
Now, perhaps that you know what's causing this to hang, you can debug it further and find the full solution...
That, or spin this off in some thread that you can monitor, and kill if it hangs too long.
Just my 2 cents worth, which usually isn't a whole lot. ;)