I am a developer on the horn OSS project that sets out to ease the pain of building other OSS projects. We are attempting to make horn a ruby gems like experience. One of the many challenges of horn is having to deal with all the various build engines like Nant, powershell, msbuild and rake which is the point of this post.
Horn has 2 manifestations, it runs as a cmd line tool and it also runs as a windows service where it builds all the various packages which can be downloaded from this website.
Certain OSS projects use rake to build their source code which has eventually brought me to the point of this post.
I cannot get the rake process to run from the windows service while the exact same code can start the rake process without any issues when running from the command line. The reason that rake does run from the cmd line tool could be because it is associated with a window although I cannot rightly say. No exception is thrown but the process just does not start.
The funny thing is that every other .exe works fine and it is only rake that is causing the problem.
Here is the code to start that creates the process:
public IProcess GetProcess(string pathToBuildTool, string cmdLineArguments, string workingDirectoryPath)
{
var psi = new ProcessStartInfo(pathToBuildTool, cmdLineArguments)
{
UseShellExecute = false,
RedirectStandardOutput = true,
WorkingDirectory = workingDirectoryPath,
Arguments = cmdLineArguments
};
return new DiagnosticsProcess(Process.Start(psi));
}
Does anyone have any suggestions as to what the problem is?
I compiled and investigated a bit myself.
On my machine, the Horn service actually starts a Ruby process to run Rake, but the process exits immediately with an error. I used Process Monitor to monitor process creation while filtering for the path containing "ruby". The end result is that Horn was not able to build with the Rakefile.
After some more investigation I was playing a bit with how Horn creates build processes. I found that the build actually runs on my machine when I do not only redirect StandardOutput but also StandardError.
public IProcess GetProcess(string pathToBuildTool, string cmdLineArguments, string workingDirectoryPath)
{
var psi = new ProcessStartInfo(pathToBuildTool, cmdLineArguments)
{
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
WorkingDirectory = workingDirectoryPath,
Arguments = cmdLineArguments
};
return new DiagnosticsProcess(Process.Start(psi));
}
I also changed DiagnosticProcess to output both messages from StandardOutput and StandardError:
public string GetLineOrOutput()
{
return process.StandardOutput.ReadLine() ?? process.StandardError.ReadLine();
}
The net result is the following line being the last message in the Horn service log:
HORN HAS FINISHED INSTALLING mspec.
Apparently, Rake on Windows should be invoked through the supplied batch file (typically c:\ruby\bin\rake.bat), instead of running the .EXE directly. See Instant Badger: Rake gotcha on Windows for details.
If switching to using the supplied batch file doesn't fix the problem, please let me know & I'll d/l the horn source and take a closer look.
Related
I have an app that runs DirectX 11 that plays a scene and generates an mp4.
I am trying to launch it through Process.Start so that I can manage the process and force it to timeout if it crashed or doesn't close correctly.
When I test the function on my local Win10 machine it works perfectly, and when I run it through CL or a .BAT file on the WinServ2012R2 machine it works perfectly too.
However when I try to run it through the Process.Start function on the server machine it fails to open DirectX
var startInfo = new System.Diagnostics.ProcessStartInfo($"{AppLocation}", $"{Parameters}")
{
WindowStyle = System.Diagnostics.ProcessWindowStyle.Normal,
UseShellExecute = false,
RedirectStandardOutput = true,
CreateNoWindow = false,
WorkingDirectory = $"{DirectoryName}",
Verb = "runas"
};
var loop = 0;
while (!System.IO.File.Exists($"{FileLocation}"))
{
loop++;
using (var p = System.Diagnostics.Process.Start(startInfo))
{
Logger.Info("Process Running....");
if (!p.WaitForExit(300000))
{
p.Kill();
}
if (loop >= 5)
break;
}
}
Edit: The DirectX error is: DXGI_ERROR_NOT_CURRENTLY_AVAILABLE
0x887A0022
It’s probably something about the environment.
If the parent process is a normal Win32 console or GUI app running inside a desktop of that server, put something like Sleep( 60000 ); in the first line of your main() or WinMain function, and use Process Explorer to find differences between manual launch which works, and programmatic launch which fails. Check the “Image”, “Security” and “Environment” tabs of the processes.
If the parent process is a system service, it’s more complicated. Services generally run under another user account and you gonna need some setup to allow the service, or a child process launched by the service, to access GPU.
Another possible reason is anti-virus or anti-malware breaking things.
P.S. Note you have minor bugs in your code.
One thing, when you detect timeouts, Process.Kill is asynchronous, you need to wait afterwards.
Another one, you specifying RedirectStandardOutput = true but you don't consume that stream. If the child process prints a lot of text it will eventually stall waiting for the parent process to consume the data buffered in that pipe. If you don’t care about output, don’t redirect these streams. If you do care, redirect and consume the data as soon as it printed, either on a separate thread or with async/await.
A requirement has arisen that I need to start a Node.js server from a C# application, this is as simple as running a server.js script within the Node.js console. However, I'm not entirely certain how exactly to achieve that.
Here's what I've looked into so far:
In the Node.js installation, there's a file called C:\Program Files (x86)\nodejs\nodevars.bat, this is the command prompt window for Node.js. To start the server, I could possibly be using the following steps:
Execute the nodevars.bat file.
SendKeys to the new process console window to start the server.
This approach feels a bit fragile. There's no guarantee that the target user will have their Node.js installation in the same place, also sending keys to a process may not be an ideal solution.
Another method could be:
Write a batch file that executes nodevars.bat.
Execute the batch file from the C# application.
This seems like a better approach. However, the only problem here is that the nodevars.bat opens in a new console window.
So to the question(s), is there a way I can start a node.js server script using functionality built into the node.js installation? Perhaps sending arguments to the node.exe?
If it is to serve multiple users, i.e. as a server, then you can use the os-service package, and install a Windows service. You can then start and stop the service using the standard API.
If you are to start the server as a "single purpose" server, i.e. to serve only the current user, then os-service is the wrong approach. (Typically when using this approach you will specify a unique port for the service to use, which will only be used by your application).
To start a batch file or other Console application, from C#, without showing a console window, use the standard method, but be sure to specify:
ProcessStartInfo psi = new ProcessStartInfo();
psi.UseShellExecute = false; // This is important
psi.CreateNoWindow = true; // This is what hides the command window.
psi.FileName = #"c:\Path\to\your\batchfile.cmd";
psi.Arguments = #"-any -arguments -go Here"; // Probably you will pass the port number here
using(var process = Process.Start(psi)){
// Do something with process if you want.
}
There are a few different ones but I recommend the os-service package.
I am trying to figure out how to run a bash command from C# running on IIS 7/.Net 4.5.
I've been searching the web and a lot of answers presume you have certain things installed/in place.
I already have Git 1.9.4.msysgit.2 installed with Git Bash and Git Giu. I'm looking for some help as to what else I need installed to run even the simplest of bash commands. And how to run it.
I've looked at posts like bash pipes - I am trying to call script from c# but that uses cygwin. Can I do the same without it and if so, how do I go about it?
Goal
If what I'm asking above doesn't make sense or seems to ask separate questions, here my ultimate goal. I'm trying to write my own server-side git hook. When a developer pushes their commits to our GitHub repo, I want GitHub to call our callback url. I want my callback url to run a git pull command to update our staging server with what was just pushed.
I got to this question based on a previous question I asked at GitHub - setup auto deployment with remote server. based on answers there I'm trying to run a simple command, either but hard coding the command, or putting it in a script and running it, e.g.: cd $REPO_DIR && git pull origin $branch_name.
I am aware of Jenkins and other software, but I want to perform these commands myself vs. installing another software.
If further information is needed please feel free to ask.
Update 1
So based on a few answers below I've come up with the following
using System.Diagnostics;
Process process = new Process();
ProcessStartInfo processStartInfo = new ProcessStartInfo();
processStartInfo.WindowStyle = ProcessWindowStyle.Hidden;
processStartInfo.FileName = #"C:\Program Files (x86)\Git\bin\bash.exe";
processStartInfo.WorkingDirectory = #"C:\myrepo\mysite";
processStartInfo.Arguments = "git status";
processStartInfo.RedirectStandardOutput = true;
processStartInfo.RedirectStandardError = true;
processStartInfo.UseShellExecute = false;
process.StartInfo = processStartInfo;
process.Start();
String error = process.StandardError.ReadToEnd();
String output = process.StandardOutput.ReadToEnd();
ViewBag.Error = error;
ViewBag.Ouput = output;
With the code above I am getting "C:/Program Files (x86)/Git/bin/bash.exe": git: No such file or directory. I know the exe is there. What's am I doing wrong?
Update 2
As per #SurgeonofDeath comment I followed this post http://blog.countableset.ch/2012/06/07/adding-git-to-windows-7-path/ and added the paths of Git to my environmental variables. However I still am getting the same issues. Any ideas?
Thanks.
Instead of calling the bash.exe, simply call git and pass the status as argument:
processStartInfo.FileName = "git";
processStartInfo.Arguments = "status";
perhaps i misunderstood your question but what about execve?
here is an excerpt of it's man page.
NAME
execve - execute program
SYNOPSIS
#include <unistd.h>
int execve(const char *filename, char *const argv[],
char *const envp[]);
DESCRIPTION
execve() executes the program pointed to by filename. filename must > be
either a binary executable, or a script starting with a line of > the
form:
#! interpreter [optional-arg]
Check your PATH environment variable and update it
C:/Program Files (x86)/Git/bin/bash.exe": git: No such file or directory
means that it's git which is not found by bash.
1. Check the PATH environment variable in bash (which is and should remain different from Windows one)
Adjust this
processStartInfo.WindowStyle = ProcessWindowStyle.Hidden;
to make the terminal visible.
In the terminal you will create with the Process.Start()
Type:
echo ${PATH}
2. Update your path
You could update the global path of windows (which requires a restart)
You could update the user path of windows (which should require a logoff, but I'm not sure).
You just set Path to what you like with System.Environment.SetEnvironmentVariable before starting the Process
Additional note:
If you have like me several versions of bash, command interpreters, git and so on, it could be really messy if you try to concatenate all the paths and hope to find the ideal order. You could see some weird behavior of you beloved commands until you realize it's not the one you intend to run ... think of FIND.exe... And I didn't even think of the user-friendly interface of windows to edit environment variables ...
Part of a project I'm working on (Windows, C#, MVC4 WebAPI) requires some integration with git. None of the existing C# git libraries supported remote cloning, so I wound up porting the parts of the JavaGit project we needed (checkout, fetch, status), and writing clone myself. All it really is is a wrapper to the git command line executable. The relevant code it calls is here:
public static void RunGitCommand(string repositoryPath, string gitArguments)
{
// GitCommand is the full path to git.exe (currently using Github for Windows)
if (null == GitCommand || String.IsNullOrWhiteSpace(GitCommand))
{
throw new FileNotFoundException("Unable to find git.exe on your system PATH.");
}
// gitArguments contains the command to run (e.g. "clone -- git#repo:projectName c:\repositories\repo_a8c0dd321f")
var startInfo = new ProcessStartInfo(GitCommand, gitArguments)
{
WorkingDirectory = (null != repositoryPath && Directory.Exists(repositoryPath)) ? repositoryPath : String.Empty,
CreateNoWindow = true,
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardInput = true,
RedirectStandardError = true
};
using (var p = new Process
{
EnableRaisingEvents = true,
StartInfo = startInfo
})
{
p.OutputDataReceived += (sender, args) => Log.Debug(args.Data);
p.ErrorDataReceived += (sender, args) => Log.Debug(args.Data);
p.Start();
p.BeginOutputReadLine();
p.BeginErrorReadLine();
p.WaitForExit();
}
}
Which would be called by the code as:
// names changed to protect the innocent
string localRepo = #"c:\repositories\repo_a8c0dd321f";
string gitArgs = "clone -- git#repo:projectName c:\repositories\repo_a8c0dd321f";
GitConfiguration.RunGitCommand(localRepo, gitArgs);
From within the MVC API, we use impersonation to make sure it's running as a user with a valid git login and key (no passphrase). The command above works perfectly from a command line as myself, and in an quick unit test, as well (I know it's really an integration test).
However, when it's actually called from the API as shown above, it hangs. Looking in Task Manager shows git.exe running, with the command line showing the full path to git.exe followed by the arguments above. It's not using any processor time, and only 2604K of RAM, but it claims to be running. Likewise, there is an ssh.exe process running, also with no processor usage, and 1212K of RAM, with the command line:
ssh git#repo "git-upload-pack 'projectName'"
Both processes are listed as running under my username, so it appears impersonation is working correctly.
Looking in the localRepo directory, it creates the .git directory, then hangs, leaving around 13K worth of git files in there, but none of our code. Thinking it was due to our repo being huge, I let it run overnight. Still no movement as of this morning.
Brought up LINQPad, ran:
Process.GetProcessById($gitPID).Dump()
Did the same for the SSH process as well. The threads showed them as being in the Wait state, and the WaitReason was Executive (waiting for thread scheduler). I initially assumed it was waiting for a passphrase, as my key had one. I switched to a working key without a passphrase, same result.
Git/SSH versions (from latest GitHub for Windows):
git version
git version 1.7.11.msysgit.1
ssh -v
OpenSSH_4.6p1, OpenSSL 0.9.8e 23 Feb 2007
The only idea I have left is that maybe it can't communicate with ssh-agent, which is running. It is impersonating me properly, though, so I don't know why it wouldn't work from the WebApi framework, but works fine from the "unit" test and Git Shell. I tried making sure the HOME, PLINK_PROTOCOL, TERM, SSH_AUTH_SOCK, and SSH_AGENT_PID environment variables were set, after looking through the Github for Windows setup scripts, just to make sure I wasn't missing anything.
I'm at a total loss. Here's a snippet of the log file, with some comments afterward:
2012-11-20 13:42:59.5898 Info Initializing repo at path: c:\repositories\repo_a8c0dd321f
2012-11-20 13:42:59.5898 Debug Working Directory: c:\repositories\repo_a8c0dd321f
2012-11-20 13:42:59.6053 Debug C:\Users\christian.doggett\AppData\Local\GitHub\PortableGit_8810fd5c2c79c73adcc73fd0825f3b32fdb816e7\bin\git.exe status --branch
2012-11-20 13:42:59.6209 Debug HOME=H:\
2012-11-20 13:42:59.6209 Debug PLINK_PROTOCOL=ssh
2012-11-20 13:42:59.6365 Debug TERM=msys
2012-11-20 13:42:59.6365 Debug SSH_AGENT_PID=58416
2012-11-20 13:42:59.6365 Debug SSH_AUTH_SOCK=/tmp/ssh-IgTHj19056/agent.19056
2012-11-20 13:42:59.6521 Info git status --branch
Exit code: 128
2012-11-20 13:42:59.6521 Error
2012-11-20 13:42:59.6677 Info Cloning repo from origin: git#repo:projectName
2012-11-20 13:43:01.8674 Debug Cloning into 'c:\repositories\repo_a8c0dd321f'...
2012-11-20 13:43:03.2090 Debug Could not create directory 'h/.ssh'.
2012-11-20 13:43:03.2870 Debug Warning: Permanently added 'repo,359.33.9.234' (RSA) to the list of known hosts.
2012-11-20 13:44:41.4593 Debug fatal: The remote end hung up unexpectedly
I always get the "Could not create directory 'h/.ssh'" and "Warning: Permanently added * to the list of known hosts." messages, even on the command line. My H:.ssh\known_hosts remains empty, but my keys are in that directory, and git finds those just fine. The "remote end hung up unexpectedly" error was when I killed the git and ssh processes.
I may wind up switching to LibGit2Sharp for most of my needs, but that still doesn't solve my clone problem. Is something screwed up with my key setup, which again, works perfectly outside of the w3wp.exe process? Does it need to be able to communicate with ssh-agent.exe, and is not able to? Has anyone cloned a remote git repository via System.Diagnostics.Process and lived to tell the tale?
UPDATE (11/25/2012 6:54PM):
mvp was correct in pointing out that impersonation and mapped network drives don't play well together. I added the following before starting the process:
var userProfile = Environment.GetFolderPath(Environment.SpecialFolder.UserProfile);
Environment.SetEnvironmentVariable("HOME", userProfile);
I ran it again, and am now at least seeing a new git process in Task Manager that's taking 0 processor time, but is using an increasing amount of memory, I believe as part of the clone process. The command line was:
git index-pack --stdin --fix-thin "--keep=fetch-pack 6024 on MACHINENAME"
It just finally finished after 10 minutes (It's a huge repository), and threw the exception:
fatal: git checkout: updating paths is incompatible with switching branches.
Did you intend to checkout 'origin/8b243b8d9a5140673fc552ef7da8f0dfe9039d50' which can not be resolved as commit?
Looks like the clone worked after changing into the directory, though! The other problem is something to do with immediately calling checkout after the clone operation finishes, but is unrelated to the hanging problem.
I just need to verify my/mvp's solution on a production server, then I'll award the bounty.
I believe your main problem is that home directory for impersonated account is not what you think it is, mostly because network mapped drives for impersonated accounts don't really work.
As a workaround, you should set HOME environment variable for impersonated user to point to some local directory (say on drive C:) which should contain your ssh keys (without passphrases). You should test this by running git clone manually (while having fake HOME in effect) and accept known_host keys, so it would not prevent background git command from working automatically.
Home being h:\ is a little broad, but other than that, I would say your next step is to create a .ssh directory (h:.ssh) with the correct permissions, should be read only for the web user and no access for any other user.
Processes launched via Process.Start seems to have around a 26-second delay when the spawned process (the "child") launches more new processes (the "grandchildren") - I'm trying to find a way to solve this issue. Specifically, this is occurring when the original process (the "parent") is an ASP.Net website or a Windows Service (tried both).
We're attempting to run a server-side command-line tool to gather information, make modifications in the file system, and continue with other processes when the "child" is finished. When creating the "child" directly via command-line, there is no delay, and with certain command-line parameters, the "child" does not spawn new processes, and there is no delay. However, with other parameters, the "child" spawns "grandchildren" (the same executable as itself, but we can't modify its code) and seems to have a 25-30 second (usually 26 seconds) delay before the first process is started, and then runs normally.
I've tried modifying the UseShellExecute property, the CreateNoWindow property, and the WindowStyle property, to no effect. ErrorDialog and the RedirectStandard* properties are false.
The code I'm using is as follows:
using (Process p = new Process())
{
p.StartInfo = new ProcessStartInfo(exePath, args)
{
WorkingDirectory = workingDirectory,
UseShellExecute = true,
CreateNoWindow = true,
};
p.Start();
p.WaitForExit();
}
Oh, I don't think it matters as I've seen the issue referenced elsewhere (but no solutions), but the exePath I'm using points to msysgit's git.exe.
I had this same exact problem executing a .bat file which in turn made a call to git.cmd using Process.Start from a windows service. The git command would execute immediately if the .bat file was ran directly from the command line, but would delay exactly 50 seconds any time it was called from the windows service.
It came down to a permissions issue. After configuring my windows service to run as a user (administrator in my case), the git process ran immediately. You can probably modify your service installer to run the service as "User", but you can just modify the service properties after it's installed to the same effect.
There may be ways to enable "Local Service" to get around the delay, but I wouldn't know how.
Hard to tell a reason why this might happen, you need to do further troubleshooting.
I would suggest that you use Process Explorer and Process Monitor to look for potential problems.
I would guess that the problem is not directly in your code but more related to the environment of the user. For example, the w3wp.exe process runs in a non-GUI session (session 0) and the user might not be configured to have web access (proxy configuration) so that you might see a timeout issue here.