I am trying to work out an efficient way of calling a Powershell cmdlet for 20-30 files asynchronously.
Although the below code is working, the Import-Module step is run for every file which is processed. Unfortunately this Module takes between 3 or 4 seconds to import.
Searching on the web I can find references to RunspacePools & InitialSessionState, but have had issues trying to create a PSHost object which is required in the CreateRunspacePool overload.
Any help would be appreciated.
Thanks
Gavin
.
.
Code sample from my application:
I am using a Parallel ForEach to distribute the files between threads.
Parallel.ForEach(files, (currentFile) =>
{
ProcessFile(currentfile);
});
private void ProcessFile(string filepath)
{
//
// Some non powershell related code removed for simplicity
//
// Start PS Session, Import-Module and Process file
using (PowerShell PowerShellInstance = PowerShell.Create())
{
PowerShellInstance.AddScript("param($path) Import-Module MyModule; Process-File -Path $path");
PowerShellInstance.AddParameter("path", filepath);
PowerShellInstance.Invoke();
}
}
As it has already been explained in the comments, this won't work with PSJobs because objects are serialized and the jobs themselves run in a separate process.
What you can do is create a RunspacePool with an InitialSessionState that has the module imported:
private RunspacePool rsPool;
public void ProcessFiles(string[] files)
{
// Set up InitialSessionState
InitialSessionState initState = InitialSessionState.Create();
initState.ImportPSModule(new string[] { "MyModule" });
initState.LanguageMode = PSLanguageMode.FullLanguage;
// Set up the RunspacePool
rsPool = RunspaceFactory.CreateRunspacePool(initialSessionState: initState);
rsPool.SetMinRunspaces(1);
rsPool.SetMaxRunspaces(8);
rsPool.Open();
// Run ForEach()
Parallel.ForEach(files, ProcessFile);
}
private void ProcessFile(string filepath)
{
// Start PS Session and Process file
using (PowerShell PowerShellInstance = PowerShell.Create())
{
// Assign the instance to the RunspacePool
PowerShellInstance.RunspacePool = rsPool;
// Run your script, MyModule has already been imported
PowerShellInstance.AddScript("param($path) Process-File #PSBoundParameters").AddParameter("path", filepath);
PowerShellInstance.Invoke();
}
}
Related
This is in continuation to this Question here, I have a PowerShell command which I have created and am able to call the command in a PowerShell window, but when trying to call from C# method, I am getting error as the cmdlet is not recognized, I tried with other existing commands and get same error, so I suspect issue in Importing the Module, though I don't get that error in streams. Error. The only error I get is "Get-RowAndPartitionKey is not a recognized cmndlt, please check the spelling.....".
Would like to know if there is any other way, I should try it or if I can debug more here to see if my Module fetches all command or not. right now I am clueless how to fix this.
public string RunScript( string contentScript, Dictionary<string, EntityProperty> parameters )
{
List<string> parameterList = new List<string>();
foreach( var item in parameters )
{
parameterList.Add( item.Value.ToString() );
}
using( PowerShell ps = PowerShell.Create() )
{
IAsyncResult async =
ps.AddCommand( "Import-Module" ).AddArgument( #"C:\Users\...\.D.PowerShell.dll" )
.AddStatement()
.AddCommand( "Get-RowAndPartitionKey" ).AddParameter( "Properties", "test" )
.BeginInvoke();
StringBuilder stringBuilder = new StringBuilder();
foreach( PSObject result in ps.EndInvoke( async ) )
{
stringBuilder.AppendLine( result.ToString() );
}
return stringBuilder.ToString();
}
}
}
Below method do not return any error in Streams.Error or Verbose but no output also:
public async Task<IEnumerable<object>> RunScript( string scriptContents, List<string> scriptParameters )
{
// create a new hosted PowerShell instance using the default runspace.
// wrap in a using statement to ensure resources are cleaned up.
using( PowerShell ps = PowerShell.Create() )
{
// specify the script code to run.
ps.AddScript( scriptContents );
// specify the parameters to pass into the script.
ps.AddParameter( "Properties" ,"test") ;
// execute the script and await the result.
var pipelineObjects = await ps.InvokeAsync().ConfigureAwait( false );
return pipelineObjects;
}
}
scriptContent
"\"$path = 'C:\\Users...\\.TabularData.PowerShell.dll'\\r\\nImport-Module $path\\r\\nGet-RowAndPartitionKeys\""
The following is self-contained PowerShell sample code that uses on-demand compilation of C# code:
It shows that the approach works in principle, as described in this answer to your original question.
Prerequisites: The PowerShell SDK package and .NET runtime used in the C# project that calls your custom Get-RowAndPartitionKey" cmdlet must be compatible with the PowerShell SDK and .NET runtime that you used to compile the assembly DLL that houses that cmdlet, to be imported via Import-Module.
The sample code below ensures that implicitly, by running directly from PowerShell, using the Add-Type cmdlet to compile C# code on demand - it works in Windows PowerShell as well as in PowerShell (Core) 7+.
In practice I've found that a .NET Framework-compiled DLL (from Windows PowerShell) also works in PowerShell (Core) (.NET (Core) 5.0), but not vice versa.
It shows troubleshooting techniques, namely:
Adding the -Verbose switch to the Import-Module call to produce verbose output that lists the commands being imported from the given module (DLL).
Printing these verbose messages (look for // --- TROUBLESHOOTING CODE)
Printing any non-terminating PowerShell errors that occurred (as opposed to exceptions that you'd have to handle in C# code).
# Create a (temporary) assembly containing cmdlet "Get-RowAndPartitionKey".
# This assembly can directly be imported as a module from PowerShell.
# The cmdlet simply outputs "Hi from Get-RowAndPartitionKey" and
# echoes the elements of the list passed to -Properties, one by one.
$tempModuleDll = Join-Path ([IO.Path]::GetTempPath()) "TempModule_$PID.dll"
Remove-Item -ErrorAction Ignore $tempModuleDll
Add-Type #'
using System.Management.Automation;
using System.Collections.Generic;
[Cmdlet("Get", "RowAndPartitionKey")]
public class GetRowAndPartitionKeyCmdlet : PSCmdlet {
[Parameter] public List<string> Properties { get; set; }
protected override void ProcessRecord() {
WriteObject("Hi from Get-RowAndPartitionKey: ");
WriteObject(Properties, true);
}
}
'# -ErrorAction Stop -OutputAssembly $tempModuleDll
# Compile a C# class ad hoc to simulate your project, and call its static
# method, which imports the module and effectively calls
# Get-RowAndPartitionKey -Properties "foo", "bar"
(Add-Type #"
using System;
using System.Management.Automation;
using System.Collections.Generic;
using System.Text;
public static class Foo {
public static string RunScript(List<string> parameterList)
{
using (System.Management.Automation.PowerShell ps = PowerShell.Create())
{
IAsyncResult async =
// Add -Verbose to the Import-Module call, so that the list of
// commands being imported is written to the verbose output stream.
ps.AddCommand("Import-Module").AddArgument(#"$tempModuleDll").AddParameter("Verbose", true)
.AddStatement()
.AddCommand("Get-RowAndPartitionKey").AddParameter("Properties", parameterList)
.BeginInvoke();
StringBuilder stringBuilder = new StringBuilder();
foreach (PSObject result in ps.EndInvoke(async))
{
stringBuilder.AppendLine(result.ToString());
}
// --- TROUBLESHOOTING CODE
// Print verbose output from the Import-Module call
foreach (var v in ps.Streams.Verbose) { Console.WriteLine("VERBOSE: " + v.ToString()); }
// Print any errors.
foreach (var e in ps.Streams.Error) { Console.WriteLine("ERROR: " + e.ToString()); }
// ---
return stringBuilder.ToString();
}
}
}
"# -ErrorAction Stop -PassThru)::RunScript(("foo", "bar"))
# Clean-up instructions:
if ($env:OS -eq 'Windows_NT') {
Write-Verbose -vb "NOTE: Re-running this code requires you to start a NEW SESSION."
Write-Verbose -vb "After exiting this session, you can delete the temporary module DLL(s) with:`n`n Remove-Item $($tempModuleDll -replace '_.+', '_*.dll')`n "
} else {
Write-Verbose -vb "NOTE: Re-running this code after modifying the embedded C# code requires you to start a NEW SESSION."
Remove-Item $tempModuleDll
}
On my Windows 10 machine, both from PowerShell (Core) 7.0.5 and Windows PowerShell 5.1, the above yields (clean-up instructions omitted) the following, showing that everything worked as intended:
VERBOSE: Loading module from path 'C:\Users\jdoe\AppData\Local\Temp\TempModule_11876.dll'.
VERBOSE: Importing cmdlet 'Get-RowAndPartitionKey'.
Hi from Get-RowAndPartitionKey:
foo
bar
Specifically, line VERBOSE: Importing cmdlet 'Get-RowAndPartitionKey'. indicates that the custom cmdlet was successfully imported into the session.
I am attempting to write a C# core program to run powershell scripts on remote linux systems. Running on .net core is a requirement for this project. I am trying to loosely follow a guide I found on CodeProject.
This is my code:
using System;
using System.Management.Automation;
using System.Management.Automation.Runspaces;
namespace ConsoleApp1
{
class Program
{
static void Main(string[] args)
{
SSHConnectionInfo connectionInfo = new SSHConnectionInfo(userName: "user", computerName: "server", keyFilePath: "id_rsa");
using (Runspace runspace = RunspaceFactory.CreateRunspace(connectionInfo))
{
runspace.Open(); // The program errors out here
runspace.Close();
}
}
}
}
I have the "id_rsa" file located int the same folder as the program. I have verified that openssh for windows, powershell core 6.0.2, and .net core 2 SDK are installed and working. I am using the following nuget packages from the Microsoft Powershell Core repository: Microsoft.PowerShell.SDK (6.0.2) and Sytem.Managment.Automation (6.0.2)
This is the error I am receiving:
Unhandled Exception: System.Management.Automation.Remoting.PSRemotingDataStructureException: An error has occurred which PowerShell cannot handle. A remote session might have ended. ---> System.ArgumentException: The path is not of a legal form.
Parameter name: path
at System.IO.Path.GetDirectoryName(String path)
at System.Management.Automation.Runspaces.SSHConnectionInfo.StartSSHProcess(StreamWriter& stdInWriterVar, StreamReader& stdOutReaderVar, StreamReader& stdErrReaderVar)
at System.Management.Automation.Remoting.Client.SSHClientSessionTransportManager.CreateAsync()
at System.Management.Automation.Remoting.ClientRemoteSessionDSHandlerImpl.SendNegotiationAsync(RemoteSessionState sessionState)
at System.Management.Automation.Remoting.ClientRemoteSessionDSHandlerImpl.HandleStateChanged(Object sender, RemoteSessionStateEventArgs arg)
at System.Management.Automation.ExtensionMethods.SafeInvoke[T](EventHandler`1 eventHandler, Object sender, T eventArgs)
at System.Management.Automation.Remoting.ClientRemoteSessionDSHandlerStateMachine.RaiseStateMachineEvents()
at System.Management.Automation.Remoting.ClientRemoteSessionDSHandlerStateMachine.ProcessEvents()
--- End of inner exception stack trace ---
at System.Management.Automation.Runspaces.AsyncResult.EndInvoke()
at System.Management.Automation.Runspaces.Internal.RunspacePoolInternal.EndOpen(IAsyncResult asyncResult)
at System.Management.Automation.Runspaces.Internal.RemoteRunspacePoolInternal.Open()
at System.Management.Automation.RemoteRunspace.Open()
at ConsoleApp1.Program.Main(String[] args) in C:\ConsoleApp1\ConsoleApp1\Program.cs:line 18
Press any key to continue . . .
At this point I am not sure what I am missing.
I ended up opening an issue on github for this error. To work around this issue currently you will need to add the following code to your program until this issue gets resolved in a future version of powershell core (>6.0.4 or >6.1.0-rc.1). Here is the specific post regarding the issue.
if (System.Management.Automation.Runspaces.Runspace.DefaultRunspace == null)
{
var defaultRunspace = RunspaceFactory.CreateRunspace();
defaultRunspace.Open();
System.Management.Automation.Runspaces.Runspace.DefaultRunspace = defaultRunspace;
}
The following answer is derived from the Bruc3 work-around answer above and the RemoteRunspace Sample 01 from the Powershell SDK.
namespace Sample.PowerShell.Runspace
{
using System;
using System.Management.Automation;
using System.Management.Automation.Runspaces;
/// <summary>
/// This class contains the Main enrty point for the application.
/// </summary>
internal class SshRemoteRunspace
{
public static void Main(string[] args)
{
SSHConnectionInfo sshConnectionInfo = new
SSHConnectionInfo("Administrator", "remote-hyper-v-server.mydomain.com", #"C:\Users\myself\.ssh\id_ed25519.pub");
// Bruc3 Workaround
if (System.Management.Automation.Runspaces.Runspace.DefaultRunspace is null)
{
var defaultRunspace = RunspaceFactory.CreateRunspace();
defaultRunspace.Open();
System.Management.Automation.Runspaces.Runspace.DefaultRunspace = defaultRunspace;
}
// Create a remote runspace using the connection information.
using (Runspace remoteRunspace = RunspaceFactory.CreateRunspace(sshConnectionInfo))
{
remoteRunspace.Open();
// Powershell command
using (PowerShell powershell = PowerShell.Create().AddCommand("Get-VMReplication"))
{
// makes the Powershell command run in the remote runspace instead of locally
powershell.Runspace = remoteRunspace;
// display the results in the local console
foreach (PSObject result in powershell.Invoke())
{
Console.WriteLine(result.ToString());
}
}
Console.WriteLine("Press 'Enter' to exit > ");
Console.ReadKey();
remoteRunspace.Close();
}
}
}
}
We want to store and manage PowerShell scripts in a database an execute them via C#.
How can we discover the parameters of such a script before executing it? so we can set them to known values or prompt a user for values.
Some clarification:
We create a management system MS.
An admin stores a PowerShell script in the MS database.
Later a different admin selects this script from a list offered by the MS.
MS discovers the parameters of the script.
MS prompts the admin for values.
MS executes the script with the parameters supplied.
string testScript = #"
{
param(
[ValidateNotNullOrEmpty()]
[string]$Name
)
get-process $name
";
Dictionary<string,object> DiscoverParameters()
{
using (PowerShell psi = PowerShell.Create())
{
psi.AddScript(testScript);
var pars = new Dictionary<string,object>();
//How do we get at the parameters
return pars;
}
}
void ExecuteScript(Dictionary<string,object> pars)
{
using (PowerShell psi = PowerShell.Create())
{
psi.AddScript(testScript);
pars.ToList().ForEach(p => psi.AddParameter(p.Key, p.Value));
Collection<PSObject> PSOutput = psi.Invoke();
//...
}
}
mjolinor is correct that using the PowerShell parser is probably the best way to get the parameters. That example is in PowerShell, below is an example in C#. I'm not quite sure what you are looking for with the parameters being Dictionary<string, object>. Here we just stick the names into the list although there is other info you could pull out like the static type.
using System.Management.Automation;
using System.Management.Automation.Language;
static void Main(string[] args)
{
const string testScript = #"
param(
[ValidateNotNullOrEmpty()]
[string]$Name
)
get-process $name
";
foreach(var parameter in GetScriptParameters(testScript))
{
Console.WriteLine(parameter);
}
}
private static List<string> GetScriptParameters(string script)
{
Token[] tokens;
ParseError[] errors;
var ast = Parser.ParseInput(script, out tokens, out errors);
if (errors.Length != 0)
{
Console.WriteLine("Errors: {0}", errors.Length);
foreach (var error in errors)
{
Console.WriteLine(error);
}
return null;
}
return ast.ParamBlock.Parameters.Select(p => p.Name.ToString()).ToList();
}
You can use the PS parser, and access the parameter information via AST:
$scriptfile = '<full path to script file>'
$AST = [System.Management.Automation.Language.Parser]::ParseFile( $scriptfile,[ref]$null,[ref]$null)
$AST.ParamBlock.Parameters | ft
I found a C# example here of invoking a PowerShell script asynchronously from a host application (in folder Chapter 6 - Reading With Events) and am trying to use it in a Windows Forms application.
I have a button (button1) to start the PowerShell script, textBox1 is to enter the script and textBox2 displays the script output. Here is my current code:
using System;
using System.Management.Automation;
using System.Management.Automation.Runspaces;
using System.Windows.Forms;
namespace PSTestApp
{
delegate void SetTextDelegate(string text);
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void button1_Click(object sender, EventArgs e)
{
textBox2.Text = "";
Runspace runspace =
RunspaceFactory.CreateRunspace();
runspace.Open();
Pipeline pipeline =
runspace.CreatePipeline(textBox1.Text);
pipeline.Output.DataReady +=
new EventHandler(HandleDataReady);
pipeline.Error.DataReady +=
new EventHandler(HandleDataReady);
pipeline.InvokeAsync();
pipeline.Input.Close();
}
private void HandleDataReady(object sender, EventArgs e)
{
PipelineReader<PSObject> output =
sender as PipelineReader<PSObject>;
if (output != null)
{
while (output.Count > 0)
{
SetText(output.Read().ToString());
}
return;
}
PipelineReader<object> error =
sender as PipelineReader<object>;
if (error != null)
{
while (error.Count > 0)
{
SetText(error.Read().ToString());
}
return;
}
}
private void SetText(string text)
{
if (textBox2.InvokeRequired)
{
SetTextDelegate d = new SetTextDelegate(SetText);
this.Invoke(d, new Object[] { text });
}
else
{
textBox2.Text += (text + Environment.NewLine);
}
}
}
}
The code works, but I have a problem handling the output. Pipeline.Output.Read() returns an instance of PSObject so ToString() returns different things for different objects. For example, if I use this PowerShell command:
Get-ChildItem
the output is:
PSTestApp.exe
PSTestApp.pdb
PSTestApp.vshost.exe
PSTestApp.vshost.exe.manifest
and if I use:
Get-Process
the output is:
...
System.Diagnostics.Process (csrss)
System.Diagnostics.Process (ctfmon)
System.Diagnostics.Process (devenv)
System.Diagnostics.Process (devenv)
...
I could use the returned PSObject instances to construct the output, but it would be nice If I could use the existing PowerShell formatting and get the same output as in the console. When I run the application and check Runspace.RunspaceConfiguration.Formats, the count is 9, and DotNetTypes.format.ps1xml is present, but I don't know how to apply the format.
I have noticed that if I add Out-String at the end of the script:
...
Pipeline pipeline =
runspace.CreatePipeline(textBox1.Text);
pipeline.Commands.Add("Out-String");
...
the output is formatted just as in the standard PowerShell console. This works, but if I run a script with a long output that takes some time to execute:
gci d:\ -recurse
Pipeline.Output.DataReady event is raised only once (after the ending Out-String is executed) and only then is the output added to the text box.
Is there a way to use standard PowerShell output formatting in a hosted PowerShell instance?
If you use the -stream parameter on out-string, I think you'll find that it doesn't block.
Also, if you actually build a host (implement the host interface, including the UI and possibly the rawui), you'll implement methods to handle the "standard" host output.
You might also try using out-default instead of out-string. I know in self-hosted environments I usually use that.
BACKGROUND
I am using Powershell 2.0 on Windows 7.
I am writing a cmdlet in a Powershell module ("module" is new to Powershell 2.0).
To test the cmdlet I am writing Unit tests in Visual Studio 2008 that programmatically invoke the cmdlet.
REFERENCE
This Article on MSDN called "How to Invoke a Cmdlet from Within a Cmdlet" shows how to call a cmdlet from C#.
THE SOURCE CODE
This is a distilled version of my actual code — I've made it as small as possible so that you can see the problem I am having clearly:
using System;
using System.Management.Automation;
namespace DemoCmdLet1
{
class Program
{
static void Main(string[] args)
{
var cmd = new GetColorsCommand();
foreach ( var i in cmd.Invoke<string>())
{
Console.WriteLine("- " + i );
}
}
}
[Cmdlet("Get", "Colors")]
public class GetColorsCommand : Cmdlet
{
protected override void ProcessRecord()
{
this.WriteObject("Hello");
this.WriteVerbose("World");
}
}
}
COMMENTS
I understand how to enable and capture verbose output from the Powershell command line; that's not the problem.
In this case I am programmatically invoking the cmdlet from C#.
Nothing I've found addresses my specific scenario. Some articles suggest I should implement my own PSHost, but seems expensive and also it seems like a have to call the cmdlet as text, which I would like to avoid because that is not as strongly typed.
UPDATE ON 2009-07-20
Here is is the source code based on the answer below.
Some things are still not clear to me:
* How to call the "Get-Colors" cmdlet (ideally without having to pass it as a string to the ps objet)
* How to get the verbose output as it is generated instead of getting an collection of them at the end.
using System;
using System.Management.Automation;
namespace DemoCmdLet1
{
class Program
{
static void Main(string[] args)
{
var ps = System.Management.Automation.PowerShell.Create();
ps.Commands.AddScript("$verbosepreference='continue'; write-verbose 42");
foreach ( var i in ps.Invoke<string>())
{
Console.WriteLine("normal output: {0}" , i );
}
foreach (var i in ps.Streams.Verbose)
{
Console.WriteLine("verbose output: {0}" , i);
}
}
}
[Cmdlet("Get", "Colors")]
public class GetColorsCommand : Cmdlet
{
protected override void ProcessRecord()
{
this.WriteObject("Red");
this.WriteVerbose("r");
this.WriteObject("Green");
this.WriteVerbose("g");
this.WriteObject("Blue");
this.WriteVerbose("b");
}
}
}
The code above generates this output:
d:\DemoCmdLet1\DemoCmdLet1>bin\Debug\DemoCmdLet1.exe
verbose output: 42
UPDATE ON 2010-01-16
by using the Powershell class (found in System.Management.Automation but only in the version of the assembly that comes with the powershell 2.0 SDK, not what comes out-of-the-box on Windows 7) I can programmatically call the cmdlet and get the verbose output. The remaining part is to actually add a custom cmdlet to that powershell instance - because that was my original goal - to unit test my cmdlets not those that come with powershell.
class Program
{
static void Main(string[] args)
{
var ps = System.Management.Automation.PowerShell.Create();
ps.AddCommand("Get-Process");
ps.AddParameter("Verbose");
ps.Streams.Verbose.DataAdded += Verbose_DataAdded;
foreach (PSObject result in ps.Invoke())
{
Console.WriteLine(
"output: {0,-24}{1}",
result.Members["ProcessName"].Value,
result.Members["Id"].Value);
}
Console.ReadKey();
}
static void Verbose_DataAdded(object sender, DataAddedEventArgs e)
{
Console.WriteLine( "verbose output: {0}", e.Index);
}
}
[Cmdlet("Get", "Colors")]
public class GetColorsCommand : Cmdlet
{
protected override void ProcessRecord()
{
this.WriteObject("Hello");
this.WriteVerbose("World");
}
}
Verbose output is not actually output unless $VerbosePreference is set at least to "Continue."
Use the PowerShell type to run your cmdlet, and read VerboseRecord instances from the Streams.Verbose propery
Example in powershell script:
ps> $ps = [powershell]::create()
ps> $ps.Commands.AddScript("`$verbosepreference='continue'; write-verbose 42")
ps> $ps.invoke()
ps> $ps.streams.verbose
Message InvocationInfo PipelineIterationInfo
------- -------------- ---------------------
42 System.Management.Automation.Invocat... {0, 0}
This should be easy to translate into C#.
1. string scriptFile = "Test.ps1";
2. using (PowerShell ps = PowerShell.Create())
3. {
4. const string getverbose = "$verbosepreference='continue'";
5. ps.AddScript(string.Format(getverbose));
6. ps.Invoke();
7. ps.Commands.Clear();
8. ps.AddScript(#".\" + scriptFile);
9. ps.Invoke();
10. foreach (var v in ps.Streams.Verbose)
11. {
12. Console.WriteLine(v.Message);
13. }
14. }
Important lines are line 5 and 6. This basically set the $verbosepreference for the session and for upcoming new commands and scripts.
First off, if you are unit testing cmdlets, likely Pester is a better (and easier) option.
As per your many updates, all you are likely missing at this point is a strongly typed approach to reference the C# cmdlet
ps.AddCommand(new CmdletInfo("Get-MyCS", typeof(GetMyCS)));
DISCLAIMER: I know this works for PowerShell 5.0, but don't have experience with the older PowerShell 2.0.