Part of a project I'm working on (Windows, C#, MVC4 WebAPI) requires some integration with git. None of the existing C# git libraries supported remote cloning, so I wound up porting the parts of the JavaGit project we needed (checkout, fetch, status), and writing clone myself. All it really is is a wrapper to the git command line executable. The relevant code it calls is here:
public static void RunGitCommand(string repositoryPath, string gitArguments)
{
// GitCommand is the full path to git.exe (currently using Github for Windows)
if (null == GitCommand || String.IsNullOrWhiteSpace(GitCommand))
{
throw new FileNotFoundException("Unable to find git.exe on your system PATH.");
}
// gitArguments contains the command to run (e.g. "clone -- git#repo:projectName c:\repositories\repo_a8c0dd321f")
var startInfo = new ProcessStartInfo(GitCommand, gitArguments)
{
WorkingDirectory = (null != repositoryPath && Directory.Exists(repositoryPath)) ? repositoryPath : String.Empty,
CreateNoWindow = true,
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardInput = true,
RedirectStandardError = true
};
using (var p = new Process
{
EnableRaisingEvents = true,
StartInfo = startInfo
})
{
p.OutputDataReceived += (sender, args) => Log.Debug(args.Data);
p.ErrorDataReceived += (sender, args) => Log.Debug(args.Data);
p.Start();
p.BeginOutputReadLine();
p.BeginErrorReadLine();
p.WaitForExit();
}
}
Which would be called by the code as:
// names changed to protect the innocent
string localRepo = #"c:\repositories\repo_a8c0dd321f";
string gitArgs = "clone -- git#repo:projectName c:\repositories\repo_a8c0dd321f";
GitConfiguration.RunGitCommand(localRepo, gitArgs);
From within the MVC API, we use impersonation to make sure it's running as a user with a valid git login and key (no passphrase). The command above works perfectly from a command line as myself, and in an quick unit test, as well (I know it's really an integration test).
However, when it's actually called from the API as shown above, it hangs. Looking in Task Manager shows git.exe running, with the command line showing the full path to git.exe followed by the arguments above. It's not using any processor time, and only 2604K of RAM, but it claims to be running. Likewise, there is an ssh.exe process running, also with no processor usage, and 1212K of RAM, with the command line:
ssh git#repo "git-upload-pack 'projectName'"
Both processes are listed as running under my username, so it appears impersonation is working correctly.
Looking in the localRepo directory, it creates the .git directory, then hangs, leaving around 13K worth of git files in there, but none of our code. Thinking it was due to our repo being huge, I let it run overnight. Still no movement as of this morning.
Brought up LINQPad, ran:
Process.GetProcessById($gitPID).Dump()
Did the same for the SSH process as well. The threads showed them as being in the Wait state, and the WaitReason was Executive (waiting for thread scheduler). I initially assumed it was waiting for a passphrase, as my key had one. I switched to a working key without a passphrase, same result.
Git/SSH versions (from latest GitHub for Windows):
git version
git version 1.7.11.msysgit.1
ssh -v
OpenSSH_4.6p1, OpenSSL 0.9.8e 23 Feb 2007
The only idea I have left is that maybe it can't communicate with ssh-agent, which is running. It is impersonating me properly, though, so I don't know why it wouldn't work from the WebApi framework, but works fine from the "unit" test and Git Shell. I tried making sure the HOME, PLINK_PROTOCOL, TERM, SSH_AUTH_SOCK, and SSH_AGENT_PID environment variables were set, after looking through the Github for Windows setup scripts, just to make sure I wasn't missing anything.
I'm at a total loss. Here's a snippet of the log file, with some comments afterward:
2012-11-20 13:42:59.5898 Info Initializing repo at path: c:\repositories\repo_a8c0dd321f
2012-11-20 13:42:59.5898 Debug Working Directory: c:\repositories\repo_a8c0dd321f
2012-11-20 13:42:59.6053 Debug C:\Users\christian.doggett\AppData\Local\GitHub\PortableGit_8810fd5c2c79c73adcc73fd0825f3b32fdb816e7\bin\git.exe status --branch
2012-11-20 13:42:59.6209 Debug HOME=H:\
2012-11-20 13:42:59.6209 Debug PLINK_PROTOCOL=ssh
2012-11-20 13:42:59.6365 Debug TERM=msys
2012-11-20 13:42:59.6365 Debug SSH_AGENT_PID=58416
2012-11-20 13:42:59.6365 Debug SSH_AUTH_SOCK=/tmp/ssh-IgTHj19056/agent.19056
2012-11-20 13:42:59.6521 Info git status --branch
Exit code: 128
2012-11-20 13:42:59.6521 Error
2012-11-20 13:42:59.6677 Info Cloning repo from origin: git#repo:projectName
2012-11-20 13:43:01.8674 Debug Cloning into 'c:\repositories\repo_a8c0dd321f'...
2012-11-20 13:43:03.2090 Debug Could not create directory 'h/.ssh'.
2012-11-20 13:43:03.2870 Debug Warning: Permanently added 'repo,359.33.9.234' (RSA) to the list of known hosts.
2012-11-20 13:44:41.4593 Debug fatal: The remote end hung up unexpectedly
I always get the "Could not create directory 'h/.ssh'" and "Warning: Permanently added * to the list of known hosts." messages, even on the command line. My H:.ssh\known_hosts remains empty, but my keys are in that directory, and git finds those just fine. The "remote end hung up unexpectedly" error was when I killed the git and ssh processes.
I may wind up switching to LibGit2Sharp for most of my needs, but that still doesn't solve my clone problem. Is something screwed up with my key setup, which again, works perfectly outside of the w3wp.exe process? Does it need to be able to communicate with ssh-agent.exe, and is not able to? Has anyone cloned a remote git repository via System.Diagnostics.Process and lived to tell the tale?
UPDATE (11/25/2012 6:54PM):
mvp was correct in pointing out that impersonation and mapped network drives don't play well together. I added the following before starting the process:
var userProfile = Environment.GetFolderPath(Environment.SpecialFolder.UserProfile);
Environment.SetEnvironmentVariable("HOME", userProfile);
I ran it again, and am now at least seeing a new git process in Task Manager that's taking 0 processor time, but is using an increasing amount of memory, I believe as part of the clone process. The command line was:
git index-pack --stdin --fix-thin "--keep=fetch-pack 6024 on MACHINENAME"
It just finally finished after 10 minutes (It's a huge repository), and threw the exception:
fatal: git checkout: updating paths is incompatible with switching branches.
Did you intend to checkout 'origin/8b243b8d9a5140673fc552ef7da8f0dfe9039d50' which can not be resolved as commit?
Looks like the clone worked after changing into the directory, though! The other problem is something to do with immediately calling checkout after the clone operation finishes, but is unrelated to the hanging problem.
I just need to verify my/mvp's solution on a production server, then I'll award the bounty.
I believe your main problem is that home directory for impersonated account is not what you think it is, mostly because network mapped drives for impersonated accounts don't really work.
As a workaround, you should set HOME environment variable for impersonated user to point to some local directory (say on drive C:) which should contain your ssh keys (without passphrases). You should test this by running git clone manually (while having fake HOME in effect) and accept known_host keys, so it would not prevent background git command from working automatically.
Home being h:\ is a little broad, but other than that, I would say your next step is to create a .ssh directory (h:.ssh) with the correct permissions, should be read only for the web user and no access for any other user.
Related
I work with emergency services and they have an application that uses map files to let them know where they need to go and it uses GPS to let them know where they are. We have to update the map files as things change and before I started here they were being done through VB scripts which started to fail. I decided to code my own app in C# to do this which works fine.
I created a package in SCCM 2012 that caches all of the files locally and then it compares the files in the cache to what is on the machine and then replaces any older files. This all works fine but the application they use called MobileCAD locks the files so I have to kill this process and then do the file copy and start the application again. We never know when an emergency happens so this update may start when they are on the road so it is important that it starts the application again as soon as possible. If it does not start the application then the emergency services people may try to do so manually but if core files are being updated then it may not start or cause issues.
I coded my application which uses an app manifest to force it to run as an administrator for the file copy. This application is run through SCCM which uses the local 'System' account to do all of the work and killing MobileCAD and copying files which works great. What I originally found was that it does start MobileCAD but it does so under the System account and the process would be there but it was not visible. I think this is the same problem they were originally having so the emergency services people would need to reboot the computer and wait for it to log back in and then start the wireless service so they could get back into MobileCAD.
To address this issue I did research and found that I could use the ProcessStartInfo in .NET and force it to use another account. As we use an automatic logon for these machines the users name, password, and domain are all in the registry so it was easy to pull it out and inject it into the code. Awesome, looks like it is easy enough so I code it up and sure enough it works perfectly when run under my admin account. In my basic testing everything worked perfectly until I try the same in SCCM, now it fails with the following error message.
System.ComponentModel.Win32Exception (0x80004005): Access is denied
at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
at System.Diagnostics.Process.Start()
at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
at UpdateFDM.Program.StartProcess(String processName)
I am sorry for all of the words but I believe it helps if you have a good understanding of the issue and what I am trying to do. I have also hard coded the user information into the code instead of pulling it from the registry but I get the same error. Again, this works fine under my admin account but fails when it is pushed through SCCM and it is only launching MobileCAD that fails.
This is the code I am using for launching MobleCAD, do you see where my issue may lie? I know SCCM confuses it but SCCM basically runs things just as you would from the command line but it uses the local System account.
Thanks for any help.
// Declares the new start instance
ProcessStartInfo process = new ProcessStartInfo();
// Gets the process to start
process.FileName = processName;
// Maximizes the process windows at start-up
process.WindowStyle = ProcessWindowStyle.Maximized;
// Gets the user name from the autologon information
process.UserName = GetDefaultUserInfo("DefaultUserName");
// Gets the domain for the user
process.Domain = GetDefaultUserInfo("DefaultDomainName");
// Holds the password for the default user
SecureString password = new SecureString();
// Gets the raw password from the registry
string rawPassword = GetDefaultUserInfo("DefaultPassword");
// Copies the password in a secure string
foreach (char ch in rawPassword)
{
password.AppendChar(ch);
}
// Sets the password
process.Password = password;
// Needed to launch the app as the logged on user
process.LoadUserProfile = true;
process.UseShellExecute = false;
// Starts the process
Process.Start(process);
// Process started, return true
return true;
I am trying to figure out how to run a bash command from C# running on IIS 7/.Net 4.5.
I've been searching the web and a lot of answers presume you have certain things installed/in place.
I already have Git 1.9.4.msysgit.2 installed with Git Bash and Git Giu. I'm looking for some help as to what else I need installed to run even the simplest of bash commands. And how to run it.
I've looked at posts like bash pipes - I am trying to call script from c# but that uses cygwin. Can I do the same without it and if so, how do I go about it?
Goal
If what I'm asking above doesn't make sense or seems to ask separate questions, here my ultimate goal. I'm trying to write my own server-side git hook. When a developer pushes their commits to our GitHub repo, I want GitHub to call our callback url. I want my callback url to run a git pull command to update our staging server with what was just pushed.
I got to this question based on a previous question I asked at GitHub - setup auto deployment with remote server. based on answers there I'm trying to run a simple command, either but hard coding the command, or putting it in a script and running it, e.g.: cd $REPO_DIR && git pull origin $branch_name.
I am aware of Jenkins and other software, but I want to perform these commands myself vs. installing another software.
If further information is needed please feel free to ask.
Update 1
So based on a few answers below I've come up with the following
using System.Diagnostics;
Process process = new Process();
ProcessStartInfo processStartInfo = new ProcessStartInfo();
processStartInfo.WindowStyle = ProcessWindowStyle.Hidden;
processStartInfo.FileName = #"C:\Program Files (x86)\Git\bin\bash.exe";
processStartInfo.WorkingDirectory = #"C:\myrepo\mysite";
processStartInfo.Arguments = "git status";
processStartInfo.RedirectStandardOutput = true;
processStartInfo.RedirectStandardError = true;
processStartInfo.UseShellExecute = false;
process.StartInfo = processStartInfo;
process.Start();
String error = process.StandardError.ReadToEnd();
String output = process.StandardOutput.ReadToEnd();
ViewBag.Error = error;
ViewBag.Ouput = output;
With the code above I am getting "C:/Program Files (x86)/Git/bin/bash.exe": git: No such file or directory. I know the exe is there. What's am I doing wrong?
Update 2
As per #SurgeonofDeath comment I followed this post http://blog.countableset.ch/2012/06/07/adding-git-to-windows-7-path/ and added the paths of Git to my environmental variables. However I still am getting the same issues. Any ideas?
Thanks.
Instead of calling the bash.exe, simply call git and pass the status as argument:
processStartInfo.FileName = "git";
processStartInfo.Arguments = "status";
perhaps i misunderstood your question but what about execve?
here is an excerpt of it's man page.
NAME
execve - execute program
SYNOPSIS
#include <unistd.h>
int execve(const char *filename, char *const argv[],
char *const envp[]);
DESCRIPTION
execve() executes the program pointed to by filename. filename must > be
either a binary executable, or a script starting with a line of > the
form:
#! interpreter [optional-arg]
Check your PATH environment variable and update it
C:/Program Files (x86)/Git/bin/bash.exe": git: No such file or directory
means that it's git which is not found by bash.
1. Check the PATH environment variable in bash (which is and should remain different from Windows one)
Adjust this
processStartInfo.WindowStyle = ProcessWindowStyle.Hidden;
to make the terminal visible.
In the terminal you will create with the Process.Start()
Type:
echo ${PATH}
2. Update your path
You could update the global path of windows (which requires a restart)
You could update the user path of windows (which should require a logoff, but I'm not sure).
You just set Path to what you like with System.Environment.SetEnvironmentVariable before starting the Process
Additional note:
If you have like me several versions of bash, command interpreters, git and so on, it could be really messy if you try to concatenate all the paths and hope to find the ideal order. You could see some weird behavior of you beloved commands until you realize it's not the one you intend to run ... think of FIND.exe... And I didn't even think of the user-friendly interface of windows to edit environment variables ...
I want to automate a process, which is invoked after a successful build on TFS. The process will RDP to a test server, then call a C# application on that server, and reset IIS on that server. Each step will return the result so whether or not to call next step is based on the previous step.
There are a few obstacles in implementing it. Below is what I want to know if it is possible, and how to code it.
1) Invoking the process via a build on TFS
There is an option in Build definition to invoke automated test. I assume that the process can be invoked by implementing it as a test.
2) RDP to remote server
I found the links below, which might be a solution
Process rdcProcess = new Process();
rdcProcess.StartInfo.FileName = Environment.ExpandEnvironmentVariables(#"%SystemRoot%\system32\cmdkey.exe");
rdcProcess.StartInfo.Arguments = "/generic:TERMSRV/192.168.0.217 /user:" + "username" + " /pass:" + "password";
rdcProcess.Start();
rdcProcess.StartInfo.FileName = Environment.ExpandEnvironmentVariables(#"%SystemRoot%\system32\mstsc.exe");
rdcProcess.StartInfo.Arguments = "/v " + "192.168.0.217"; // ip or name of computer to connect
rdcProcess.Start();
Run mstsc.exe with specified username and password
Automating remote desktop connection
3) IISReset
I think it should be simply invoke "IISRESET" after RDP, but the problem is that, HOW to captrue the result of running IISRESET.
The tools that might be suitable are:
1) Powershell - I don't know much about Powershell but am willing to learn if required
2) C#
My question is that how to implement it, any code example, and idea would be very much appreciated.
Check my answer here which is somewhat related: Answer
If the user which runs the TFSBuild Service on the build server have enough rights on the test server then you can use psexec or powershell to run your commands remotely. Read the below links:
PSEXEC
PowerShell Remote commands
There is no inbuilt activity/process which can help you run scripts on remote machines in TFS build workflow.
Step 1 for you is to identify how you are going to run scripts on the remote machine, as mentioned above you can either use PSEXEC or Powershell (though running PowerShell on remote computers may be a little more complicated to set up).
Step2, write the actual scripts to do the work, stop services, install MSI etc.
Step3, Edit your current build defintion - create a new custom activity or make use of InvokeProcess activity from within your build definition to invoke the script that you have created in Step 2. InvokeProcess Activity
in most cases you do not need to run iisreset
if you want to upgrade an asp.net application, try to put app_offline.htm in the application folder, it will stop an application and application files will be unlocked
after upgrading an application, it will restart automatically, or you can "touch" web.config to force restart
You might be better using the Lab Build to run the scripts as part of an environment ob the target computer. It can run any powershell against that machine as well as deploy and execute applications....
Question: HOW to capture the result of running IISRESET
I believe the old fashioned way, Hope this is what you are looking for
c:> IISRESET >> C:\temp.log
You can use the above either from CMD or powershell
In the past I have used Psexec to run commands against a remote server and where ever we need to control flow on the result of that command, we simply piped the console out to a shared folder and checked for our success flag.
I am not sure if TFS can run commands in this manner but we implemented it on hudson/jenkins.
This won't answer your question directly but it may offer a better way forward
An Example:
psexec.exe \remoteserver "iisreset > h:\iisreset.log"
Then run a grep or similar against the iisreset.log with your success flag as a condition to run the next step.
I have some VB6 .ocx files that I would like to register. These .ocx files would be on a remote machine.
What is the best way to register these .ocx files programatically?
string arg_fileinfo = "/s" + " " + "\"" + "\\<remotemachine>\\<directory>\\<ocx>" + "\"";
Process reg = new Process();
//This file registers .dll files as command components in the registry.
reg.StartInfo.FileName = "regsvr32.exe";
reg.StartInfo.Arguments = arg_fileinfo;
reg.StartInfo.UseShellExecute = false;
reg.StartInfo.CreateNoWindow = true;
reg.StartInfo.RedirectStandardOutput = true;
reg.Start();
reg.WaitForExit();
reg.Close();
I'm not getting any errors but it isn't registering the .ocx either. Any ideas?
If you want to register a remote file for use on a local machine, there is nothing special required for registering a file on a UNC path, but you do need to make sure that the UNC path or mapped drive is still available to all users, especially the user that is running regsvr32. Presumably, this will be the local admin which (by default on Windows Vista+) will require elevation which can disconnect network connections.
Also note that your example is missing the extra \ from the beginning of the UNC path. Your code will result in arg_fileinfo containing /s "\<remotemachine>\<directory>\<ocx>".
You can add the extra \, or use the # decorator which makes it a lot clearer when entering Windows paths:
string arg_fileinfo = "/s \"" + #"\\<remotemachine>\<directory>\<ocx>" + "\"";
Or just use it for the entire string and the alternative quote escaping method:
string arg_fileinfo = #"/s ""\\<remotemachine>\<directory>\<ocx>""";
Take this as a warning you're free to ignore (because I know you will anyway):
Doing this isn't a good practice. Just to begin with "run from network" PE files (EXE, DLL, OCX) need to be specially linked for it or you risk high network activity and crashes due to intermittent network interruptions. And registering anything not on the boot drive or at least a local hard drive isn't sensible anyway. Doing any of this ranks high on the "poor practices" list even though it might seem to work most of the time.
Why not just do normal deployment following accepted practices?
My guess would be that you are doing a lot of Mort development, throwing together version after version of some program hoping one of them will eventually "stick." So you want to dump some or all of it onto a network share, thinking "Installation? Installation? We don't need no steenking installation. I can just plop new files out there and have everything magically work with no effort."
I'll assume you don't have the luxury of a managed network you can use to push out updates via Group Policy, and that you aren't creating the necessary MSI installer packages handling the Product and Upgrade Codes in them.
One alternative would be to use reg-free COM, which will solve a lot of small issues for you.
Now, you could do this and still ignore the hazards of PE files run from a network share, or you could bypass that using a small launcher program. That launcher could check a network share for a new version, and if found copy the newer files to the local PC before starting the actual application and terminating. This is basically an auto-updated XCopy Deployment technique.
You can get as fancy as need be. For example if your application accepts command line parameters it might do the new version check itself and if found then start the small updater (passing it the command line parameters), then terminate. The updater app could restart and pass those parameters to the new version.
But yes, life as Mort (or even an official on-the-payroll developer) can be a pain. It can be extremely difficult to get the attention of your friendly neighborhood box jockeys to do things properly even if you are working in a managed corporate LAN environment. That goes double if your application isn't part of some highly engineered sanctioned Major Project.
I had to do this several years ago. As best I can remember, UNC names wouldn't work, a mapped drive letter was required. Whether it was strictly a regsvr32 issue, or was caused by something else (e.g. Windows 95) is lost in the fog of time.
If you want to register the file for use on the remote machine, you you need to run the code on that remote machine.
You can either do this by physically sitting in front of the computer, using remote control software, or a remote admin tool like psexec.exe.
I am a developer on the horn OSS project that sets out to ease the pain of building other OSS projects. We are attempting to make horn a ruby gems like experience. One of the many challenges of horn is having to deal with all the various build engines like Nant, powershell, msbuild and rake which is the point of this post.
Horn has 2 manifestations, it runs as a cmd line tool and it also runs as a windows service where it builds all the various packages which can be downloaded from this website.
Certain OSS projects use rake to build their source code which has eventually brought me to the point of this post.
I cannot get the rake process to run from the windows service while the exact same code can start the rake process without any issues when running from the command line. The reason that rake does run from the cmd line tool could be because it is associated with a window although I cannot rightly say. No exception is thrown but the process just does not start.
The funny thing is that every other .exe works fine and it is only rake that is causing the problem.
Here is the code to start that creates the process:
public IProcess GetProcess(string pathToBuildTool, string cmdLineArguments, string workingDirectoryPath)
{
var psi = new ProcessStartInfo(pathToBuildTool, cmdLineArguments)
{
UseShellExecute = false,
RedirectStandardOutput = true,
WorkingDirectory = workingDirectoryPath,
Arguments = cmdLineArguments
};
return new DiagnosticsProcess(Process.Start(psi));
}
Does anyone have any suggestions as to what the problem is?
I compiled and investigated a bit myself.
On my machine, the Horn service actually starts a Ruby process to run Rake, but the process exits immediately with an error. I used Process Monitor to monitor process creation while filtering for the path containing "ruby". The end result is that Horn was not able to build with the Rakefile.
After some more investigation I was playing a bit with how Horn creates build processes. I found that the build actually runs on my machine when I do not only redirect StandardOutput but also StandardError.
public IProcess GetProcess(string pathToBuildTool, string cmdLineArguments, string workingDirectoryPath)
{
var psi = new ProcessStartInfo(pathToBuildTool, cmdLineArguments)
{
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
WorkingDirectory = workingDirectoryPath,
Arguments = cmdLineArguments
};
return new DiagnosticsProcess(Process.Start(psi));
}
I also changed DiagnosticProcess to output both messages from StandardOutput and StandardError:
public string GetLineOrOutput()
{
return process.StandardOutput.ReadLine() ?? process.StandardError.ReadLine();
}
The net result is the following line being the last message in the Horn service log:
HORN HAS FINISHED INSTALLING mspec.
Apparently, Rake on Windows should be invoked through the supplied batch file (typically c:\ruby\bin\rake.bat), instead of running the .EXE directly. See Instant Badger: Rake gotcha on Windows for details.
If switching to using the supplied batch file doesn't fix the problem, please let me know & I'll d/l the horn source and take a closer look.