Run docker-compose command from C# .Net 4.7 - c#

I'm wishing to run a command from C# to a container set up via docker-compose. In Powershell, I run this command and the file is created:
docker-compose exec database sh -c "mysqldump -u((username)) -p((password)) ((databasename)) > /backups/test.sql"
When I run the following code it seems to ignore the environment variables, even though I have them set. It only creates a file named backup.sql and the SQL outputted to the file indicates that no database was selected. I've verified the env variables are set by outputting the last parameters string to the console.
var exportPath = $"/backups/backup.sql {DateTime.Now}";
using (var runspace = RunspaceFactory.CreateRunspace())
{
runspace.Open();
runspace.SessionStateProxy.Path.SetLocation(Path.GetFullPath(".."));
using (var pipeline = runspace.CreatePipeline())
{
var cmd = new Command("docker-compose");
cmd.Parameters.Add("exec");
cmd.Parameters.Add("database");
cmd.Parameters.Add($"sh -c \"mysqldump - u{GetEnv("MYSQL_USERNAME")} " +
$"-p{GetEnv("MYSQL_PASSWORD")} {GetEnv("MYSQL_DATABASE")} > {exportPath} \"");
pipeline.Commands.Add(cmd);
pipeline.Invoke();
}
}
GetEnv is just a convenience method for Environment.GetEnvironmentVariable
I'm fairly certain that I am not setting the parameters right, but I don't know where to go from here.

I came across to FluentDocker which seems like a nice way to run docker-compose from c#.
Here is an example from the project page:
using (
var container =
new Builder().UseContainer()
.UseImage("kiasaki/alpine-postgres")
.ExposePort(5432)
.WithEnvironment("POSTGRES_PASSWORD=mysecretpassword")
.WaitForPort("5432/tcp", 30000 /*30s*/)
.Build()
.Start())
{
var config = container.GetConfiguration(true);
Assert.AreEqual(ServiceRunningState.Running, config.State.ToServiceState());
}
Disclaimer: I am not affiliated with the project in any way nor have attempted to use it yet.

I gave up and used CliWrap because it is easier to debug. I couldn't figure out how to set the current directory, but fortunately I could modify the rest of the program to look for stuff in the current directory.
using CliWrap;
using CliWrap.Buffered;
var now = DateTime.Now.ToString().Replace("/", "-").Replace(" ", "-");
var exportPath = $"/backups/backup.sql-{now}";
var cmd = $"exec -T database sh -c \"mysqldump -u{GetEnv("MYSQL_USER")} " +
$"-p{GetEnv("MYSQL_PASSWORD")} {GetEnv("MYSQL_DATABASE")} > {exportPath}\"";
Console.WriteLine(cmd);
var result = await Cli.Wrap("docker-compose")
.WithArguments(cmd)
.ExecuteBufferedAsync();
Console.WriteLine(result.StandardOutput);
Console.WriteLine(result.StandardError);
The library can be found here: https://github.com/Tyrrrz/CliWrap

Related

PExec doesn't exit after execute

I'm writing a program with C# , that can create Users on remote Computers.
Actually it's done and working.
But I have one little problem.
In C# I use PowerShell to run a Script which runs then an Pexec, which executes a Batch file on a remote Computer.
C# :
private void executeScripts()
{
string _dirPath = System.Reflection.Assembly.GetExecutingAssembly().Location;
string _sPath = Path.GetDirectoryName(_dirPath) + #"\ExecuteScripts\FileToExecute.ps1";
string _scriptPath = "& '" + _sPath + "'";
using (PowerShellProcessInstance pspi = new PowerShellProcessInstance())
{
string psfn = pspi.Process.StartInfo.FileName;
psfn = psfn.ToLowerInvariant().Replace("\\syswow64\\", "\\sysnative\\");
pspi.Process.StartInfo.FileName = psfn;
using (Runspace r = RunspaceFactory.CreateOutOfProcessRunspace(null, pspi))
{
r.Open();
using (PowerShell ps = PowerShell.Create())
{
ps.Runspace = r;
ps.AddScript(_scriptPath);
ps.Invoke();
}
}
}
}
PS Script :
#
# First there are some Copy-Items to the remote Computer
#
# Execute Above copied Bat File on remote Computer
[string] $IPAddress = "\\" + $XmlFile.ComputerSettings.LastChild.ChildNodes[1].InnerText
$PsTools = "\PsTools"
$PsToolsPath = Join-Path -path $ScriptParent -childpath $PsTools
& $PsToolsPath\PsExec.exe $IPAddress /accepteula -i -s -u $Login -p $LoginPassword Powershell C:\Path\ToBatFile\Execute.bat > log.txt
Exit
I use this PExec 3 other times in my Program, creating a User, updating a User and removing a User, i just execute different files, scripts or batch files.
And it works perfectly.
But with the Script above, the PExec executes everything but doesn't exit. Neiter does it log something.
I tried it also with the -d switch, but that didn't work either. I also put an exit /b in the batch file but no luck.
When running the script manually from Powershell it works, it executes and it exits, but when running it from my Program it doesn't.
After some waiting my C# returns a timed-out Exception end exits.
Anyone seeing what I'm doing wrong ?
Powershell class itself has a method called Stop() which makes it pretty easy to stop this.
If you want to do it asynchronously here is an example of implementation:
using(cancellationToken.Register(() => powershell.Stop())
{
await Task.Run(() => powershell.Invoke(powershellCommand), cancellationToken);
}

How to Debug Remote Command

I need to execute this command on our remote Skype server:
SEFAUtil.exe /server:lyncserver.domain1.co.uk sip:MySelf#domain.com /addteammember:sip:OtherUser#domain.com /delayringteam:10
which adds a colleague to my team call group.
I am able to run the command on the server itself, and the code below works when sending other commands to that server:
var processToRun = new[] { process };
var connection = new ConnectionOptions();
var wmiScope = new ManagementScope(String.Format("\\\\{0}\\root\\cimv2", LyncServer), connection);
var wmiProcess = new ManagementClass(wmiScope, new ManagementPath("Win32_Process"), new ObjectGetOptions());
var reason = wmiProcess.InvokeMethod("Create", processToRun);
However, when process is the string:
"cmd /c cd /d C:\\Program Files\\Microsoft Lync Server 2013\\ResKit && SEFAUtil.exe /server:lyncserver.domain1.co.uk sip:MySelf#domain.com /addteammember:sip:OtherUser#domain.com /delayringteam:10"
Then the user is not added to the team call group.
I can see that reason contains the uint 0, which usually indicates success - but the actual command is clearly failing.
I also tried adding > C:\users\user.name\desktop\output.txt and 2> C:\users\user.name\desktop\output.txt to the end of the command, but they just created empty text files, so not very useful!
Update
I tried changing the command to the following:
const string LyncServer = "server.domain1.co.uk";
const string ResKitPath = #"C:\Program Files\Microsoft Lync Server 2013\ResKit";
var command = "SEFAUtil.exe /server:{LyncServer} sip:MySelf#domain.com /addteammember:sip:OtherUser#domain.com /delayringteam:10";
var process = $"cmd /c cd /d \"{ResKitPath}\" && {command}";
So that the path containing spaces is double-quoted and the slashes are not being escaped, but with the same results.
Does anyone know of another way of debugging this or retrieving the output for the newly created process?
I've had a similar issue, mine was that the command shell needed to run elevated. SEFA is a bit naff at giving good error messages, and fails silently.

Using Command Line from C# to copy container using AzCopy

I'm trying to copy containers in Azure from one storage location to another. I'm using the AzCopy command for this. First I get a list of all the containers and then run AzCopy based on the container name from the command line, using c# code.
The problem that I was running into is that it does copy the containers from one location to another but after 4 containers, it seems to get stuck. And the test keeps running forever. When I cancel the test I see all the other containers get copied as well.
I was wondering how can I solve this issue of having the test be complete and all the folders get copied over. I tried to wait after each call to make sure there is enough time for the call to complete. Also tried using cmd.WaitForExit(); after each call but that just gets stuck.
Any suggestions on what I could be missing, one thing I wanted to do was get the output after each call, because right now it only outputs the result once all the commands are finished. Also was thinking of how to run the command lines call sequentially so run only after the first has finished.
Any help would be appreciated!
namespace Test2
{
[TestFixture]
class ContainerList
{
[Test]
public void CopyingContainerData()
{
CloudStorageAccount sourceCloudStorageAccount =
CloudStorageAccount.Parse("StorageAccountKey");
CloudBlobClient sourceCloudBlobClient = sourceCloudStorageAccount.CreateCloudBlobClient();
List<string> outputLines = new List<string>();
IEnumerable<CloudBlobContainer> containers = sourceCloudBlobClient.ListContainers();
Process cmd = new Process();
cmd.StartInfo.FileName = "cmd.exe";
cmd.StartInfo.RedirectStandardInput = true;
cmd.StartInfo.RedirectStandardOutput = true;
cmd.StartInfo.CreateNoWindow = false;
cmd.StartInfo.UseShellExecute = false;
cmd.Start();
int i = 0;
foreach (CloudBlobContainer oneContainer in containers)
{
string outputLine = oneContainer.Name;
outputLines.Add(outputLine);
string container = oneContainer.Name;
string strCmdText = #"AzCopy /Source:https://location1.blob.core.windows.net/" + container + #" /Dest:https://location2.blob.core.windows.net/" + container + #" /SourceKey:abc /DestKey:abc123 /S /NC:8 /XO /Y";
string location = #"cd C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy";
cmd.StandardInput.WriteLine(location);
cmd.StandardInput.WriteLine(strCmdText);
//System.Threading.Thread.Sleep(20000);
//cmd.WaitForExit();
i++;
if (i == 15)
{
break;
}
}
string[] outputText = outputLines.ToArray();
File.WriteAllLines(#"C:\AzureTests\CopyData.txt", outputText);
cmd.StandardInput.Flush();
cmd.StandardInput.Close();
Console.WriteLine(cmd.StandardOutput.ReadToEnd());
}
}
}
I suggest you use powershell to do this:
$SourceStorageAccount = "sourceStorageAccount"
$SourceStorageKey = "sourceKey"
$DestStorageAccount = "destStorageAccount"
$DestStorageKey = "destKey"
$SourceStorageContext = New-AzureStorageContext –StorageAccountName $SourceStorageAccount -StorageAccountKey $SourceStorageKey
$DestStorageContext = New-AzureStorageContext –StorageAccountName $DestStorageAccount -StorageAccountKey $DestStorageKey
$containers = Get-AzureStorageContainer -Context $SourceStorageContext
foreach($container in $containers) {
New-AzureStorageContainer -Context $DestStorageContext -Name $container.name -Permission Off
$Blobs = Get-AzureStorageBlob -Context $SourceStorageContext -Container $container.name
#Do the copy of everything
foreach ($Blob in $Blobs) {
Write-Output "Moving $Blob.Name"
Start-CopyAzureStorageBlob -Context $SourceStorageContext -SrcContainer $container.name -SrcBlob $Blob.Name `
-DestContext $DestStorageContext -DestContainer $container.name -DestBlob $Blob.Name
}
}
We had similar scenario of AzCopy.exe invocation not returning control to .Net. The reason was parallel execution of AzCopy.exe without specifying the journal files. It share the journal files by default and gets access violation. When we issued different journal files to different instances of AzCopy it stared working.
Running multiple instances of AzCopy.exe from .Net

I am trying to run powershell script in c# . program runs successfully but does not show any output

I am trying to run powershell script in c# . program runs successfully but does not show any output.
try
{
string fileName = "D:\\Script\\script.psm1";
RunspaceConfiguration config = RunspaceConfiguration.Create();
Runspace myRs = RunspaceFactory.CreateRunspace(config);
myRs.Open();
RunspaceInvoke scriptInvoker = new RunspaceInvoke(myRs);
scriptInvoker.Invoke("Set-ExecutionPolicy Unrestricted");
/*using (new Impersonator("myUsername", "myDomainname", "myPassword"))
{
using (RunspaceInvoke invoker = new RunspaceInvoke())
{
invoker.Invoke("Set-ExecutionPolicy Unrestricted");
}
} */
Pipeline pipeline = myRs.CreatePipeline();
pipeline.Commands.AddScript(fileName);
//...
pipeline.Invoke();
var error = pipeline.Error.ReadToEnd();
myRs.Close();
string errors = "";
if (error.Count >= 1)
{
foreach (var Error in error)
{
errors = errors + " " + Error.ToString();
}
}
return errors;
}
Your program is only checking for error output. You typically get the "standard" output as the return value of the Invoke method e.g.
Collection<PSObject> results = pipeline.Invoke();
string output = "";
foreach (var result in results)
{
output += result.ToString();
}
You aren't doing yourself any favors with that big try {} block wrapped around everything, as you can't see the exceptions that are happening.
You will need to run Visual Studio as a local administrator in order for "Set-ExecutionPolicy Unrestricted" to work, and the final executable will also have that requirement since issuing that command requires access to a protected registry key.
The pipeline.Invoke() method returns a type of Collection<PSObject>.
Collection<PSObject> results = pipeLine.Invoke();
If your intent is to ignore the output of the pipeline and only look at errors, that is fine; but if there are no errors in the script, it would be normal not to see anything.
With the .psm1 file extension on the script, you will probably get null results. The proper extension should be .ps1. The .psm1 extension is for modules that are stored in special locations on the file system and which are loaded automatically (in PowerShell 3.0+).
By default, 'Stop' type errors in PowerShell will generate an Exception in the C# program, so wrapping with try/catch is one way to see them.
Collection<PSObject> results = null;
try
{
results = pipeline.Invoke();
// results returned from PowerShell can be accessed here but may not
// necessarily be valid since a 'Continue' error could have occurred
// which would not generate an exception
}
catch (RuntimeException e)
{
Debug.WriteLine("Error: " + e.Message);
}
You can test this by adding for example the following to your script.ps1:
throw "This is an error"
Working Example:
Note:
1. You will need to add a reference to System.Management.Automation.dll in order to run this code sample. If you are using Visual Studio, you can select Add Reference then the Browse... button and in the search box of the Browse dialog enter the name of the assembly and it will likely show up in the search results. If not you may need to download the .NET portion of the Windows SDK.
2. PowerShell scripts are disabled by default in Windows, and this is code that runs PowerShell scripts. There is plenty of information on the 'Net, but the standard way to enable scripts is to open a PowerShell command prompt as a local administrator and run Set-ExecutionPolicy RemoteSigned (if needed, Unrestricted can be used instead of RemoteSigned).
3. In some environments, you will need to unblock scripts downloaded from the Internet by right-clicking on the file in Windows Explorer, going to Properties, and clicking Unblock. If there is no Unblock button then the file is OK.
4. The first thing to try if you get access errors is to run Visual Studio and/or the executable as a local administrator. Please do not attempt to impersonate an administrator and embed a password in the executable. If you are in a corporate setting, group policy can be configured to allow PowerShell scripts to run. If you are at home, you should be a local administrator.
using System.Management.Automation;
using System.Collections.ObjectModel;
using System.Management.Automation.Runspaces;
using System.Diagnostics;
namespace PowerShell
{
class Program
{
static void Main(string[] args)
{
// Create and Open a Runspace
string fileName = #"D:\script.ps1";
RunspaceConfiguration config = RunspaceConfiguration.Create();
Runspace myRs = RunspaceFactory.CreateRunspace(config);
myRs.Open();
// Attempt to configure PowerShell so we can forcefully run a script.
RunspaceInvoke scriptInvoker = new RunspaceInvoke(myRs);
scriptInvoker.Invoke("Set-ExecutionPolicy Unrestricted -Scope Process -Force");
Pipeline pipeline = myRs.CreatePipeline();
pipeline.Commands.AddScript(fileName);
Collection<PSObject> results = null;
try
{
results = pipeline.Invoke();
// Read standard output from the PowerShell script here...
foreach (var item in results)
{
Debug.WriteLine("Normal Output: " + item.ToString());
}
}
catch (System.Management.Automation.RuntimeException e)
{
Debug.WriteLine("PowerShell Script 'Stop' Error: " + e.Message);
}
myRs.Close();
}
}
}

How to create variables using Powershell in C#

Trying to do the following in powershell in C#
$certThumbrint = "someLocationToACert"
$cert = get-item $certThumbrint
Get-RoleInstanceCount -ServiceName "someServiceName" -DeploymentSlot "someSlot" -RoleName "someRole" -SubscriptionId "someId" -Certificate $cert
This works perfectly when running them one by one in the powershell comandline. But I cannot figure out how to do this by code. So far Ive done this.
Pipeline pipeline = runspace.CreatePipeline();
pipeline.Commands.Add("$certThumbrint = \"someLocationToACert\"");
pipeline.Commands.Add(#"$cert = get-item $certThumbrint");
Command instanceCount = new Command("Get-RoleInstanceCount");
instanceCount.Parameters.Add(new CommandParameter("ServiceName", "someServiceName"));
....
instanceCount.Parameters.Add(new CommandParameter("Certificate", "$cert"));
I then get the following exception:
"The term '$certThumbrint = "someLocation"' is not recognized as the
name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the
path is correct and try again.
I've tried to add the varibles as "AddScrips" and I also used
SessionStateVariableEntry var2 = new SessionStateVariableEntry("cert", "get-item $certThumbrint", "Initial session state MyVar1 test");
initialSessionState.Variables.Add(var2);
Before creating the runspace. Nothing is working. Also added all the code into a string and tried to run it as a script.
I actually have no way of doing this and it feels like it's a really simple thing that must be able to do... thanks.
Edit: also tried the following:
const string getInstanceCountScript = "$certThumbrint = \"somecert\" \n " +
"$cert = get-item $certThumbrint \n " +
"Get-RoleInstanceCount -ServiceName someservicename" +
...
" -Certificate $cert";
pipeline.Commands.AddScript(getInstanceCountScript);
It runs but returns an empty string. If I put the same code into a ps1 file that I call with "Add()" it runs and gives me the right output. But I really dont want to have a load of ps1 files in my project just for 3 lines of code or less.
This code perfectly works for me. Are you shure, that PS1 file contains exact same code?
static void Main(string[] args)
{
using (Runspace rs = RunspaceFactory.CreateRunspace())
{
rs.Open();
var pipeline = rs.CreatePipeline();
pipeline.Commands.AddScript("$certThumbrint = \"c:\\1.txt\"\n" +
"$cert = get-item $certThumbrint\n" +
"Get-Content $cert");
foreach (var s in pipeline.Invoke())
{
Console.WriteLine(s);
}
}
Console.ReadLine();
}
Take a look at the New-Variable commandlet.

Categories

Resources