My application is written in c# and uses Powershell class of .net library. It is multi-threaded and can execute a bunch of powershell scripts simultaneously on a runspacepool.
I use a runspacepool size of 100. Before the execution of each script I create a session to the target computer, by using the code below.
I use BeginInvoke to start execution in a pipeline and end-invoke to get the results, which include the session.
ps = PowerShell.Create()
var sessionScript = new PSCommand();
sessionScript.AddCommand(New-PSSession);
sessionScript.AddParameter("ConnectionUri", "http://192.xxx.x.xxx:5985";);
sessionScript.AddParameter("AllowRedirection");
var sessOpt = new System.Management.Automation.Remoting.PSSessionOption();
sessOpt.OpenTimeout = new TimeSpan(0,0,0,0, 120000);
sessOpt.CancelTimeout = new TimeSpan(0, 0, 0, 0,120000);
sessionScript.AddParameter("SessionOption", sessOpt);
var cred = new PSCredential(uName, password);
sessionScript.AddParameter("Credential", cred);
ps.Commands = sessionScript
ps.RunSpacePool = mypool ## size 100 created earlier
asyncResult = ps.BeginInvoke();
resultsOut = ps.EndInvoke(asyncResult);
All is fine when I keep my number of parallel executing sessions below 100. Once in a while when I hit 100 and above, I notice one or two threads (.net tasks) just getting stuck. I did some debugging and realized that the thread was getting stuck for EndInvoke. Even after a week the call just didn't return and the thread was still alive.
So, I replaced execution code by adding a Timer to this operation
Timer taskTimer = new Timer(TaskHitTimeout, null , 120000, System.Threading.Timeout.Infinite);
asyncResult = ps.BeginInvoke();
while (true)
{
if (asyncResult.IsCompleted || this.isCancelled)
break;
Thread.Sleep(200);
}
if (!this.isCancelled)
resultsOut = ps.EndInvoke(asyncResult);
and in the TimerHandler I set isCancelled to True
private void TaskHitTimeout(Object statusCode)
{ this.isCancelled = True;}
This did fix my problem. But I would like to understand why the call to EndInvoke just doesn't return. Is there anything else I can do to improve this. And If anyone else has seen a similar issue.
Related
I get cItems which is a IReadOnlyCollection<IWebElement> provided by Selenium.
However. I make the function Single Thread based, it works nice. But now I wanted to increase performance and so I choosed the .Net ThreadPool.
ThreadPool.SetMaxThreads(16, 16);
IEnumerator<IWebElement> iter = cItems.GetEnumerator();
while(iter.MoveNext()) {
cThreadJobObj.Item = iter.Current;
ThreadPool.QueueUserWorkItem(new WaitCallback(GetThreadJob), cThreadJobObj.Clone()); // .Clone() is a deep clone
}
The Problem now is, that the Threads disappear, in Line 5. There isn't any Exception thrown, I think because the } catch (Exception Ex) { block isn't called.
public static void GetThreadJob(object ThreadJob) {
try {
var cThreadJob = ThreadJob as ThreadJobObj;
var IWebElement = cThreadJob.Item;
var cElem = cItem.FindElement(By.CssSelector("span.im_message_date_text"));
} catch (Exception Ex) {
Rupert.Logger.E("{Thread.GetCurrentProcessorId()} on Obj {cThreadJob.iCount}", Ex, false);
}
}
The GetThreadJob is called by all 16 Threads, and all disappear at the same line.
Update
I separated the line and the Threads disappear in this method ISearchContext.FindElement Method.
IWebElement.FindElement(By)
If I make a quickwatch on this line, this is the value:
cItem.FindElement(cSel) Function evaluation disabled because a previous function evaluation timed out. You must continue execution to reenable function evaluation. OpenQA.Selenium.IWebElement
Ok, after a long night I found out, that the .Net Limits the Sockets which could be opened, during the Selenium Process.
The default value for ServicePointManager.DefaultConnectionLimit is 2 and 10 for Asp.net. More Info
This can be avoided through this in the init of the script:
ThreadPool.SetMinThreads(Environment.ProcessorCount / 2, 25);
ThreadPool.SetMaxThreads(Environment.ProcessorCount * 4, 100);
ServicePointManager.UseNagleAlgorithm = true;
ServicePointManager.Expect100Continue = true;
ServicePointManager.DefaultConnectionLimit = 65000;
ServicePointManager.MaxServicePointIdleTime = 500;
I'm tasked with creating a tool to help set up customers systems easily. I've created a function that calls a chocolatey script through powershell in c# and I use Task.run to create a new thread so it doesn't affect the UI thread, The system works fine, but I'm having problems with some computers. It's not helped that I have no access to these computers and do not know much about their system, and due to time constraints do not have access to these computers. I do know they have windows 8.1. I was given a windows 10 virtual machine to test on (which I still don't understand as it was known that this was a windows 8 problem)
Here is the code.
I know for a fact(due to the one time I was given access to these computers) that it stops on Task.Run(() => task)
Does anyone know if there are any problems with either chocolatey or Tasks on windows 8.1?
Task callTask = Task.Run(() => ExecuteAsynchronouslyAsync("chocolatey string", CheckBox box, string logName));
public async Task<PowerShellAction> ExecuteAsynchronouslyAsync(String commandStr, CheckBox box, string logName)
{
powerShellAction = new PowerShellAction();
powerShellAction.isFinished = false;
using (PowerShell ps = PowerShell.Create())
{
ps.AddScript(commandStr); // adding the script to the powershell script.
outputCollection = new PSDataCollection<PSObject>();
outputCollection.DataAdded += OutputData;
IAsyncResult result = ps.BeginInvoke<PSObject, PSObject>(null, outputCollection);
PSDataCollection<PSObject> execRes = await Task.Factory.FromAsync(result, ps.EndInvoke);
}
return powerShellAction;
}
Working right now on trying to get a virtual machine of 8.1 to continue trying to debug myself. Any other suggestions would be welcome.
Unfortunately I cannot ensure that my suggestions are correct. The main reason is, that i can't figure out what PowerShellAction is supposed to be. I'm assuming here that PowerShell is System.Management.Automation.PowerShell.
I'm suggesting several things:
Your code does not compile for several reasons: you have no var or type-declaration on the first line of your method and the method-call would not work because of the addition string keyword. Try to avoid pasting in code like yours in the future please because it's pretty hard to rebuild your sample.
Don't bypass a UI control to an async method but use the needed value (e.g. box.IsChecked as a bool) instead.
Add ConfigureAwait(false) to your await to prevent .NET from trying to sync back to the context.
Take more care about exception handling insude of your method.
Dont' return anything if you don't need it in your method.
The code (untestet) could be something like this:
var task = Task.Run(() => ExecutePowerShellAsync("chocolatey string", box.IsChecked, "NameOfTheLog"));
public async Task<PowerShellAction> ExecutePowerShellAsync(String commandStr, bool checkBoxValue, string logName)
{
var powerShellAction = new PowerShellAction();
powerShellAction.isFinished = false;
using (PowerShell ps = PowerShell.Create())
{
ps.AddScript(commandStr); // adding the script to the powershell script.
var outputCollection = new PSDataCollection<PSObject>();
outputCollection.DataAdded += OutputData;
IAsyncResult result = ps.BeginInvoke<PSObject, PSObject>(null, outputCollection);
PSDataCollection<PSObject> execRes = await Task.Factory.FromAsync(result, ps.EndInvoke).ContinueWith(t => {
if (t.IsFaulted)
{
System.Diagnostics.Trace.TraceError("Task faulted with exception: " + t.Exception?.Message);
}
return t.Result;
}).ConfigureAwait(false);
}
return powerShellAction;
}
I use ContinueWith in order to be able to react to any exception that might occur inside the original task.
I'm suggesting this because your description smells like you have a typical thread-lock which means the code simple does not come back due to an exception or context-syncing-problems.
I am working on a web interface which manage virtual machines on vsphere Esx 5.5. My program is developed with .net web forms (not MVC).
I followed jeffpaton posts (using VMware.Vim ) which helped me (Thanks to you, Jeff) https://communities.vmware.com/thread/434579.
But now i freeze on this subject. I do not know how to wait a task after cloning VM. My web site launch vsphere command to the Vsphere Esx with vmware.vim. I need to know when vpshere finished his work to launch an another instruction.
I try to use PropertyCollector but i do not know how to use it :
i red this post but without success :
Here is a part of my code with a try but i am blocked. i use jeffpaton functions.
using VMware.Vim;
...
VimClient client;
string serverUrl = "..."
client.Connect("https://" + serverUrl + "/sdk");
client.Login(userLogin, userPassword);
...
ManagedObjectReference cloneTask_MoRef = null;
//1 waiting the cloning task
cloneTask_MoRef = sourceVm.cloneVM_Task(sourceVm.Parent, "cloneName", mySpec);
if (cloneTask_MoRef == null) {
//error
}else
{
PropertyCollector pc = new PropertyCollector(client, cloneTask_MoRef);
PropertyFilterSpec[] pfs = null;
RetrieveOptions ro = new RetrieveOptions();
RetrieveResult rResult = new RetrieveResult();
//PropertySpec
//pc.CreateFilter(pfs, true);
//rResult = pc.RetrievePropertiesEx(pfs,ro);
//
//2 PowerOn the CloneVM
cloneVM = this.vimClientTools.getVirtualMachines(selectedDC, cloneName)[0];
//3 waiting the powerOn Task...
//What could i do to know if the task is over or in progress ? :-(
I need some help. if somebody has a suggestion beginning...
Thanks for all.
This is probably too late, but here goes.
VimClient has a WaitForTask method;
client.WaitForTask(cloneTask_MoRef);
Alternatively, you could get the task and view its progress;
var task = (Task) client.GetView(cloneTask_MoRef, null);
while (task.Info.State != TaskInfoState.success)
{
Thread.Sleep(5000);
task.UpdateViewData();
if (task.Info.State == TaskInfoState.error)
throw new Exception($"The clone failed: {task.Info.Error.LocalizedMessage}");
Console.WriteLine(task.Info.Progress);
}
I have a Windows service that spawns different tasks in threads at different intervals. Some happen once a day, some happen every couple minutes (like the one that is giving me an issue).
My goal was to code it so that a new thread will be created for the task, but only if the previous time it was spawned has completed (or it has never been created / first run). There should never be more than one "checkThread" thread running at a time.
I have other areas where I use the same code as below to check if I should spawn the new thread or not, and those all work without issue, but with one of them I get an exception very infrequently:
System.Threading.ThreadStateException: Thread is running or terminated; it cannot restart.
The following are defined outside of the main for{} loop and are used to track what's going on:
const int MINUTES_CHECK = 1;
DateTime lastCheck = DateTime.Now.AddMinutes(-MINUTES_CHECK); //force this to run immediately on first run
Thread checkThread; //used to poll status of thread
const int DIFFERENT_MINUTES_CHECK = 5; //maybe this one only runs every 5 minutes
DateTime different_LastCheck = DateTime.Now.AddMinutes(-DIFFERENT_MINUTES_CHECK); //force this to run immediately on first run
Thread different_checkThread; //used to poll status of thread
This is inside the service's main for{} loop and is how I check if the thread should be created or not. The goal is only to start a new checkThread if it has never been started before, or of the previous one is not still running:
// Worker thread loop
for (; ; )
{
if ((DateTime.Now - lastCheck).TotalMinutes >= MINUTES_CHECK)
{
if (checkThread == null ||
(checkThread.ThreadState != System.Threading.ThreadState.Running &&
checkThread.ThreadState != System.Threading.ThreadState.WaitSleepJoin)
)
{
checkThread = new Thread(DoCheck);
checkThread.Start();
Console.WriteLine("Checking for items...");
lastCheck = DateTime.Now;
}
}
if ((DateTime.Now - different_LastCheck).TotalMinutes >= DIFFERENT_MINUTES_CHECK)
{
if (different_checkThread== null ||
(different_checkThread.ThreadState != System.Threading.ThreadState.Running &&
different_checkThread.ThreadState != System.Threading.ThreadState.WaitSleepJoin)
)
{
different_checkThread= new Thread(DoSomethingElse);
different_checkThread.Start();
Console.WriteLine("Checking for something else...");
different_LastCheck = DateTime.Now;
}
}
//// Run this code once every N milliseconds
var wait_for = new TimeSpan(0, 0, 0, 0, 1000);
if (mStop.WaitOne(wait_for)) return;
}
It works 99.9% of the time, and so far I have been unable to get it to occur in Visual Studio. I have run it for a couple hours in VS with no issue, and even loaded the CPU to 99% and ran if for half an hour or so and still did not get the exception.
I did the above because my suspicion is that it's trying to start the new thread and assign it to "checkThread" before the previous one has finished (although this does seem unlikely since there's really not all that much going on inside "DoCheck" and 1 minute should be more than enough for it to complete).
Am I missing another possible ThreadState, or is there something else going on?
Edit: Exception occurs on checkThread.Start();
This error occurs when multiple attempts (more than 1 thread) are trying to execute the .Start(), you need to sync it.
Are there ANY other places that issue a START()?
Try this:
Add a static _syncRoot:
private static _syncRoot = new object();
Synchronize your start:
if ((DateTime.Now - lastCheck).TotalMinutes >= MINUTES_CHECK)
{
if (checkThread == null ||
(checkThread.ThreadState != System.Threading.ThreadState.Running &&
checkThread.ThreadState != System.Threading.ThreadState.WaitSleepJoin)
)
{
lock( _syncRoot ) {
checkThread = new Thread(DoCheck);
checkThread.Start();
Console.WriteLine("Checking for items...");
lastCheck = DateTime.Now;
}
}
}
I had a general question about extracting code from a visual studio web test. I used the "generate code" option of a web test and got the following:
using System;
using System.Collections.Generic;
using Microsoft.VisualStudio.TestTools.WebTesting;
using Microsoft.VisualStudio.TestTools.WebTesting.Rules;
namespace SignsOfLife
{
public class SignsOfLifeCoded : WebTest {
public SignsOfLifeCoded() {
this.PreAuthenticate = true;
}
public override IEnumerator<WebTestRequest> GetRequestEnumerator() {
// Initialize validation rules that apply to all requests in the WebTest
if ((this.Context.ValidationLevel >= Microsoft.VisualStudio.TestTools.WebTesting.ValidationLevel.Low)) {
ValidateResponseUrl validationRule1 = new ValidateResponseUrl();
this.ValidateResponse += new EventHandler<ValidationEventArgs>(validationRule1.Validate);
}
WebTestRequest request1 = new WebTestRequest("http://localhost/site/client/default.aspx");
yield return request1;
request1 = null;
WebTestRequest request2 = new WebTestRequest("http://localhost/site/login.aspx");
request2.ThinkTime = 5;
request2.QueryStringParameters.Add("c", "clientcode", false, false);
ExtractHiddenFields extractionRule1 = new ExtractHiddenFields();
extractionRule1.Required = true;
extractionRule1.HtmlDecode = true;
extractionRule1.ContextParameterName = "1";
request2.ExtractValues += new EventHandler<ExtractionEventArgs>(extractionRule1.Extract);
yield return request2;
request2 = null;
WebTestRequest request3 = new WebTestRequest("http://localhost/site/login.aspx");
request3.Method = "POST";
request3.ExpectedResponseUrl = "http://localhost/site/user/home.aspx";
request3.QueryStringParameters.Add("c", "clientcode", false, false);
FormPostHttpBody request3Body = new FormPostHttpBody();
request3Body.FormPostParameters.Add("__LASTFOCUS", this.Context["$HIDDEN1.__LASTFOCUS"].ToString());
request3Body.FormPostParameters.Add("__EVENTTARGET", this.Context["$HIDDEN1.__EVENTTARGET"].ToString());
request3Body.FormPostParameters.Add("__EVENTARGUMENT", this.Context["$HIDDEN1.__EVENTARGUMENT"].ToString());
request3Body.FormPostParameters.Add("__VIEWSTATE", this.Context["$HIDDEN1.__VIEWSTATE"].ToString());
request3Body.FormPostParameters.Add("__EVENTVALIDATION", this.Context["$HIDDEN1.__EVENTVALIDATION"].ToString());
request3Body.FormPostParameters.Add("Login1$UserName", "username");
request3Body.FormPostParameters.Add("Login1$Password", "password");
request3Body.FormPostParameters.Add("Login1$LoginButton", "Log In");
request3.Body = request3Body;
yield return request3;
request3 = null;
}
}
}
What I wanted to do is basically put this test into a separate service to run throughout the day for purposes of health checking. I basically want to make sure that users are able to log in throughout the day. What process should I use to get the test into something like a console app? I was running into issues with debugging the webtest code. Like how would I correctly call the GetRequestEnumerator method and respond to it from code?
First suggestion:
assumption: you want to run this on a daily basis and make it reusable/extensible for new tests
Throw your test code into a web service and link it appropriately.
Then create a windows service that consumes the service on a daily basis based on whatever interval of time and store the results in some fashion.
If my assumption is incorrect then this is an unnecessary amount of work for something that will not be reused later.
Second suggestion:
assumption: your tests will only be executed for a period of one day
Simply write a console app to execute the tests at a given interval
write the console app to run a timer class and register a callback event for when the time interval has elapsed. In the callback method you defined, perform the tests needed and record the results.
There is a threading issue that can be handled in a number of ways ... you want you app to continue running all day so you could just use something like this
public class tester
{
public tester()
{
//setup timer logic here
//..
//..
var ts = new ThreadStart(run);
var thread = new Thread(ts);
thread.start();
}
public void run()
{
while (ShouldContinue);
{
//do nothing execute keep app going
}
}
}
then toggle ShouldContinue when you meet your exiting condition ... i would assume the condition would be if the app is running for 24 hours + in this scenario
The second solution really shouldn't take you very long to design and manufacture. But the first will be more time consuming.
hopefully that answers your question or sparks another idea =D