Use wget command to download multiple files at different locations - c#

I want to download multiple files say www.google.com, yahoo.com and gmail.com at 3 different locations using wget. How should i go about it? Please help me out..
I am doing all this through c#:
ProcessStartInfo startInfo = new ProcessStartInfo("CMD.exe");
Process p = new Process();
startInfo.RedirectStandardInput = true;
startInfo.UseShellExecute = false;
startInfo.RedirectStandardOutput = true;
startInfo.RedirectStandardError = true;
p = Process.Start(startInfo);
p.StandardInput.WriteLine(#"wget --output-document=C:\1.xml xyz.com/a.xml");
p.StandardInput.WriteLine(#"wget --output-document=C:\2.xml xyz.com/b.xml");
p.StandardInput.WriteLine(#"wget --output-document=C:\3.xml xyz.com/c.xml");
p.StandardInput.WriteLine(#"EXIT");
string output = p.StandardOutput.ReadToEnd();
string error = p.StandardError.ReadToEnd();
p.WaitForExit();
p.Close();
This is not working. would like to know if there r any othe ways of downloading multiple files using wget..

If you're just talking about retrieving each file from a different location, but still doing it sequentially, you just change the URI in the wget command to point to a different location.
If you want concurrent downloads rather than sequential, you would have to start three separate processes and have them download one file each. These ptocesses could run side by side but I'd probably only consider this for large files (of which an XML file is probably not).
If you're having troubles getting the commands to run at all, the first thing I would do is ditch cmd.exe and its standard input. There's no reason why you can't have a process run wget directly. Or, if you really only want to start the one process, you could output them to a temporary file and use a single process cmd /c tempfile.cmd to run it.
However, there may be a totally different problem you're having unrelated to what you've shown, because that exact code with three echo statements in place of your wget ones runs fine, generating the correct output, at least in Visual C# Express 2010.
And, in fact, once I got my GnuWin32 wget on to the path, the following worked as well, getting real documents off the net and placing them in my top-level directory:
using System;
using System.Diagnostics;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
ProcessStartInfo startInfo = new ProcessStartInfo("cmd.exe");
Process p = new Process();
startInfo.RedirectStandardInput = true;
startInfo.UseShellExecute = false;
startInfo.RedirectStandardOutput = true;
startInfo.RedirectStandardError = true;
p = Process.Start(startInfo);
p.StandardInput.WriteLine(
#"wget --output-document=c:\q1.txt http://www.ibm.com");
p.StandardInput.WriteLine(
#"wget --output-document=c:\q2.txt http://www.microsoft.com");
p.StandardInput.WriteLine(
#"wget --output-document=c:\q3.txt http://www.borland.com");
p.StandardInput.WriteLine(#"exit");
string output = p.StandardOutput.ReadToEnd();
string error = p.StandardError.ReadToEnd();
p.WaitForExit();
p.Close();
}
}
}
Here's the proof, the single window partway through the Microsoft download:
So, bottom line, what you have shown us is not inherently unworkable as evidenced by the image above. My only suggestion is to start looking around at other things such as the version of wget you're using, GnuWin32 or CygWin.
Now, things get interesting with larger files, as you've stated in one of your comments. If I change all three URIs to http://download.microsoft.com/download/5/F/C/5FC4F80C-242D-423B-9A11-9510A013152D/Dolphins.themepack, a file of 12,889,103 bytes, the code above hangs at about 18% of the first download (around the 2.3M mark).
However, if I change the commands so that they have >nul: 2>nul: on the end, the download goes through without issue, so I suspect it's most likely an issue with the way wget writes its output (without newlines). It also works fully if you don't use redirection on the output and error streams, which strengthens that assertion.

Well, first of all, you're on Windows. wget is part of the GNU Operating System. Unless you've installed a "clone" of wget for Windows, this is impossible. You are probably better off downloading the pages yourself, with something like the HTTPClient class.
But if you have a form of wget installed, what is not working? And how do you want it to work? Your question is not very detailed, you just ask how to go about it, and provide a seemingly fine solution.

Related

How to print a text file content in .Net using c#

I am having a text file with few lines in them. I just want to send this file to printer of my choice. Why is this so hard? MSDN and google search points me to use : https://learn.microsoft.com/en-us/dotnet/api/system.drawing.printing.printdocument.print?view=dotnet-plat-ext-6.0
But when I use the above code in my VS Code it giving me errors when building because "PagePageEventArgs" is available only in System.Drawing.Common. However, I couldnt install that in my VS Code even through NuGet Package manager.
First of all codes in above lnik looks like it is designed for a windows form and it has too much details. I get so many other errors with all those codes. I am new to programming and these is overwhelming for me and I am stuck with for a week. Can anyone please tell me how to just print a text file (all the lines in text file) using printer of my choice.
If they are text files you could cheat and use Notepad:
var fileToPrint = "C:\\Junk\\Junk.txt";
var p = new System.Diagnostics.Process();
p.StartInfo.FileName = "notepad";
p.StartInfo.Arguments = $"/p {fileToPrint}";
p.StartInfo.UseShellExecute = false;
p.Start();
p.Dispose();
Console.WriteLine("FINISHED");
Based on code from: https://kimsereyblog.blogspot.com/2018/01/start-processes-from-c-in-dotnet-core.html
and Notepad command line options from: https://answers.microsoft.com/en-us/windows/forum/all/notepadexe-command-line-options/810760c1-a45a-4013-9544-1c1208e1b389?auth=1

How to combine multiple gz files into one from Process in C# program when one is missing EOF

I have multiple .gz files in a directory (2 or more), with at least one file missing the end of file marker. Our C# process is unable to read the file with missing end of file, but since they are coming from a third party we do not have control over how they are created.
As such, we've been running the following Linux command manually:
cat file1.gz file2.gz > newFile.gz
In order to automate this, I am looking for a way to leverage the Process functionality in C# to trigger the same command, but this would only be available in Cygwin or some other Linux shell. In my example, I'm using git bash but it could be Powershell or Cygwin or any other available Linux shell that runs on a Windows box.
The following code does not fail, but it does not work as expected. I am wondering if anyone has recommendations about how to do this or any suggestions on a different approach to consider?
Assume that the working directory is set and initialized successfully, so the files exist where the process is run from.
Process bashProcess = new Process();
bashProcess.StartInfo.FileName = #"..\Programs\Git\git-bash.exe";
bashProcess.StartInfo.UseShellExecute = false;
bashProcess.StartInfo.RedirectStandardInput = true;
bashProcess.StartInfo.RedirectStandardOutput = true;
bashProcess.Start();
bashProcess.StandardInput.WriteLine("cat file1.gz file2.gz > newFile.gz");
bashProcess.StandardInput.WriteLine("exit");
bashProcess.StandardInput.Flush();
.
.
.
bashProcess.WaitForExit();
My expectation is that newFile.gz is created
I was able to find a solution to my problem using a DOS command, and spawning a cmd Process from CSharp.
My code now looks like this, avoids having to launch a linux-based shell from Windows, and the copy command in windows does the same thing as cat:
Process proc = new Process();
proc.EnableRaisingEvents = false;
proc.StartInfo.FileName = "cmd";
proc.StartInfo.Arguments = #"/C pushd \\server\folder && copy *.txt.gz /b
combined.gz";
proc.StartInfo.RedirectStandardOutput = true;
proc.StartInfo.RedirectStandardError = true;
proc.StartInfo.UseShellExecute = false;
proc.Start();
proc.WaitForExit();
System.Threading.Thread.Sleep(2000);
string line = proc.StandardOutput.ReadLine();
while (line != null)
{
output.Append(line);
line = proc.StandardOutput.ReadLine();
}

C# - Open a file with the associated program without invoking the command line

Is it possible to open a file with the default program without invoking the command line? I want to run a unit test and have the unit test open the file (PDF) at completion for visual inspection.
Just call Process.Start(filePath).
This will open the file in the user's default program.
I think this should work:
System.Diagnostics.Process.Start(#"c:\file.pdf"); //i.e provide the full path!
Simply use the following syntax:
System.Diagnostics.Process.Start(#"c:\yourfile.txt");
Process process = new System.Diagnostics.Process();
process.EnableRaisingEvents = false;
process.StartInfo.CreateNoWindow = true;
process.StartInfo.FileName = filePath;
string arguments = fileArguments;
process.StartInfo.Arguments = fileArguments;
process.Start();
process.WaitForExit();
This way you can invoke and put the file name in the pdf with parameters/arguments. You can also specify different programs and put it in the path, then the pdf name in the fileArguments. It's up to you.
if you use this code:
System.Diagnostics.Process.Start( "C:\...\...\myfile.pdf" );
the pdf should get opened by the default program associated to the .pdf extension.
is this what you wanted? I would be careful in putting this inside the unit test in case you include those tests in an automated build on the server, which runs with no logged in user, this could be an issue if it fails and if it does not fail, who is there to close Acrobat Reader? :D

how to create command application like run command for vista

how to create application like window run command using C#. When i insert any command (for example: ipconfig) , this return result (for example: 192.168.1.1) on the textbox.
how to get windows command list?
how to get command result?
how to get installed application list on the machine?
(1) The command list will most likely come from whatever executables are found in your %PATH%. You can figure out your list by finding all .exe/.bat/other executable files in every folder specified by %PATH%. You may not even need to know which apps are available, the Process.Start method will find them for you. (see below)
(2) You can run a command-line tool programmatically using:
System.Diagnostics.Process.Start("notepad.exe"); // located using %PATH%
To capture the output, you have to redirect it like this:
System.Diagnostics.ProcessStartInfo psi =
new System.Diagnostics.ProcessStartInfo(#"ipconfig");
psi.RedirectStandardOutput = true;
psi.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden;
psi.UseShellExecute = false;
System.Diagnostics.Process myProcess;
myProcess = System.Diagnostics.Process.Start(psi);
System.IO.StreamReader myOutput = myProcess.StandardOutput; // Capture output
myProcess.WaitForExit(2000);
if (myProcess.HasExited)
{
string output = myOutput.ReadToEnd();
Console.WriteLine(output);
}
(3) Probably the same answer as 1
Create a Windows Forms application using the wizard. Draw a text box and a button. Add a Click handler to the button which takes the contents of the text box and launches a process. Use the Process class. That class also has a StandardOutput property that you can read the output from so you can put it into the text box.
You may find that to use many Command Prompt commands, you need to type CMD /C in front, because they aren't separate programs from the command interpreter.
As for discovering a list of commands, that's not generally possible. A command is just a program (or a feature of the CMD command interpreter). You could search the hard drive for .exe files, but then many of them wouldn't be suitable as "commands".

Service hangs up at WaitForExit after calling batch file

I have a service that sometimes calls a batch file. The batch file takes 5-10 seconds to execute:
System.Diagnostics.Process proc = new System.Diagnostics.Process(); // Declare New Process
proc.StartInfo.FileName = fileName;
proc.StartInfo.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden;
proc.StartInfo.CreateNoWindow = true;
proc.Start();
proc.WaitForExit();
The file does exist and the code works when I run the same code in-console. However when it runs inside the service, it hangs up at WaitForExit(). I have to kill the batch file from the Process in order to continue. (I am certain the file exists, as I can see it in the processes list.)
How can I fix this hang-up?
Update #1:
Kevin's code allows me to get output. One of my batch files is still hanging.
"C:\EnterpriseDB\Postgres\8.3\bin\pg_dump.exe" -i -h localhost -p 5432 -U postgres -F p -a -D -v -f "c:\backupcasecocher\backupdateevent2008.sql" -t "\"public\".\"dateevent\"" "DbTest"
The other batch file is:
"C:\EnterpriseDB\Postgres\8.3\bin\vacuumdb.exe" -U postgres -d DbTest
I have checked the path and the postgresql path is fine. The output directory does exist and still works outside the service. Any ideas?
Update #2:
Instead of the path of the batch file, I wrote the "C:\EnterpriseDB\Postgres\8.3\bin\pg_dump.exe" for the proc.StartInfo.FileName and added all parameters to proc.StartInfo.Arguments. The results are unchanged, but I see the pg_dump.exe in the process window. Again this only happens inside the service.
Update #3:
I have run the service with a user in the administrator group, to no avail. I restored null for the service's username and password
Update #4:
I created a simple service to write a trace in the event log and execute a batch file that contains "dir" in it. It will now hang at proc.Start(); - I tried changing the Account from LocalSystem to User and I set the admnistrator user and password, still nothing.
Here is what i use to execute batch files:
proc.StartInfo.FileName = target;
proc.StartInfo.RedirectStandardError = true;
proc.StartInfo.RedirectStandardOutput = true;
proc.StartInfo.UseShellExecute = false;
proc.Start();
proc.WaitForExit
(
(timeout <= 0)
? int.MaxValue : timeout * NO_MILLISECONDS_IN_A_SECOND *
NO_SECONDS_IN_A_MINUTE
);
errorMessage = proc.StandardError.ReadToEnd();
proc.WaitForExit();
outputMessage = proc.StandardOutput.ReadToEnd();
proc.WaitForExit();
I don't know if that will do the trick for you, but I don't have the problem of it hanging.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
namespace VG
{
class VGe
{
[STAThread]
static void Main(string[] args)
{
Process proc = null;
try
{
string targetDir = string.Format(#"D:\adapters\setup");//this is where mybatch.bat lies
proc = new Process();
proc.StartInfo.WorkingDirectory = targetDir;
proc.StartInfo.FileName = "mybatch.bat";
proc.StartInfo.Arguments = string.Format("10");//this is argument
proc.StartInfo.CreateNoWindow = false;
proc.Start();
proc.WaitForExit();
}
catch (Exception ex)
{
Console.WriteLine("Exception Occurred :{0},{1}", ex.Message,ex.StackTrace.ToString());
}
}
}
}
string targetDir = string.Format(#"D:\");//PATH
proc = new Process();
proc.StartInfo.WorkingDirectory = targetDir;
proc.StartInfo.FileName = "GetFiles.bat";
proc.StartInfo.Arguments = string.Format("10");//argument
proc.StartInfo.CreateNoWindow = false;
proc.Start();
proc.WaitForExit();
Tested,works clear.
What does the batch file do? Are you certain the process is getting launched with enough privs to execute the batch file? Services can be limited in what they are allowed to do.
Also make sure if you are doing something like usin the copy command to overwrite a file that you do something like:
echo Y | copy foo.log c:\backup\
Also, make sure you are using full paths for the batch commands, etc. If the batch file is launching a GUI app in some sort of "Console" mode, that may be an issue too. Remember, services don't have a "Desktop" (unless you enable the "interact with desktop") to draw any kind of windows or message boxes to. In your program, you might want to open the stdout and stderr pipes and read from them during execution in case you are getting any error messages or anything.
WebServices are probably executing as the IUSR account, or the anonymous account, which ever, so that might be an issue for you. If it works when you run it in console, that's just the first step. :)
I don't recall if System.Diagnostics. are available only in debug or not. Probably not, but some of them might be. I'll have to check up on that for ya.
Hope this gives you some ideas.
Larry
pg_dump.exe is probably prompting for user input. Does this database require authentication? Are you relying on any ENVIRONMENT variables that won't be present for the service? I don't know pg_dump but what are the other possible reasons it would prompt for input?
The next step I would take is to fire up the debugger, and see if you can tell what the program is waiting on. If you are expierenced at debugging in assembly, you may be able to get an IDEA of what's happening using tools like ProcExp, FileMon, etc.
Being a windows SERVICE, and not a web service, makes quite a bit of difference. Anyways, have you tried my suggestion of setting the "Allow Service to interact with desktop"?
If you are desperate, you might try launching cmd.exe instead of your batch file. Then, using the cmd.exe's cmd line parameters, you can have IT start the batch file. This would probably give you a cmd prompt window to view the actual output, if you turn on the interact with desktop.
For complete help on cmd.exe, just type cmd /? at any command prompt.
Larry
Here is the solution. The solution is not clear because I have changed so many time the code and now it's working!
I have tried to use a Account of User, and it's not what worked. Use LocalSystem. Here is the code that execute, mostly what Kevin gave me.
System.Diagnostics.Process proc = new System.Diagnostics.Process();
proc.StartInfo.FileName = fileName;
proc.StartInfo.RedirectStandardError = true;
proc.StartInfo.RedirectStandardOutput = true;
proc.StartInfo.UseShellExecute = false;
proc.Start();
proc.WaitForExit();
output1 = proc.StandardError.ReadToEnd();
proc.WaitForExit();
output2 = proc.StandardOutput.ReadToEnd();
proc.WaitForExit();
Thank you all, I'll up-vote everybody and accept Kevin since he helps me since the beginning. Very weird because it works now...
Daok, it looks as if the only thing you changed was the timeout period on the initial WaitForExit(). You need to be VERY careful of that. If something DOES hang your service, it will NEVER return (and well, pretty much work like it has been for you thus far.. heh), but it won't be good for the end users...
Now, perhaps that you know what's causing this to hang, you can debug it further and find the full solution...
That, or spin this off in some thread that you can monitor, and kill if it hangs too long.
Just my 2 cents worth, which usually isn't a whole lot. ;)

Categories

Resources