Develop Reference

TNonblockingServerTransport not implemented with Thrift C# library

When using Apache Thrift [https://github.com/apache/thrift] to create a non blocking server in C#, the following Classes/Types cannot be recognized:
TNonblockingServerTransport
TNonblockingServer
I want to send command from my win10 laptop to control a time-consuming calculation performed on a high performance server (ubuntu). That's why I came to Apache Thrift. I have found the official C# version tutorial [https://github.com/apache/thrift/tree/master/tutorial/csharp] and it works well. This tutorial uses the so-called Blocking Mode (TSimpleServer). But in my situation, the time-consuming calculation procedure should be interrupt-able. Consequently, I must use a non-blocking server.
The logic is simple. For the server, I used a private flag forceStop. If the Client call Stop(), forceStop will set to true and the calculation loop will break.
// #Server#
// Server Set-Up
private void SetUp()
{
try
{
CalculatorHandler handler = new CalculatorHandler();
Calculator.Processor processor = new
Calculator.Processor(handler);
var serverTransport = new TServerSocket(9090);
TServer server = new TSimpleServer(processor, serverTransport);
// Use this for a multithreaded server
// server = new TThreadPoolServer(processor, serverTransport);
Console.WriteLine("Starting the server...");
server.Serve();
}
catch (Exception x)
{
Console.WriteLine(x.StackTrace);
}
}
private bool forceStop;
public int TimeConsumingOperation(int n1, int n2)
{
Console.WriteLine("add({0},{1})", n1, n2);
for (int i = 0; i < 10; i++)
{
//calculating
Thread.Sleep(500);
if (forceStop)
{
Quit();
}
}
return n1 + n2;
}
public void Stop()
{
forceStop = true;
}
// Client
// Button#1 Click callback
private void Button_Start_Click()
{
client.TimeConsumingOperation(0,0);
}
// Button#2 Click callback
private void Button_Stop_Click()
{
client.Stop();
}
//
I've found some useful examples in java [https://chamibuddhika.wordpress.com/2011/10/02/apache-thrift-quickstart-tutorial/]. I've try my best to convert the java code of non-block server to the corresponding C# code but I found that there seems to be no TNonblockingServerTransport in C#. Anyone could help me with this probelm?
// Java Code
public class NonblockingServer {
private void start() {
try {
TNonblockingServerTransport serverTransport = new TNonblockingServerSocket(7911);
ArithmeticService.Processor processor = new ArithmeticService.Processor(new ArithmeticServiceImpl());
TServer server = new TNonblockingServer(new TNonblockingServer.Args(serverTransport).
processor(processor));
System.out.println("Starting server on port 7911 ...");
server.serve();
} catch (TTransportException e) {
e.printStackTrace();
}
}
public static void main(String[] args) {
NonblockingServer srv = new NonblockingServer();
srv.start();
}
}

There's actually two answers to that question.
Possible implementation
Your setup is flawed, to begin with.
// #Server#
// Server Set-Up
private bool forceStop;
public void Stop()
{
forceStop = true;
}
Let's assume, we have two clients, both starting a new calculation. Now one client wants to abort. What happens?
The solution would be to structure it in a way where the calculation is a separate business logic object that is instantiated at TimeConsumingOperation() and made available to the client by some means, typically by returning some sort of ID.
When the client now wants to abort, it calls Stop(calcualtionID). The server side logic now routes that call to the implementation and triggers whatever the abort mechanism might be, with C# is is probably a CancellationToken.
A third call would be needed to query the final results from the server end once the calculation has finished. Note that we still work with a TSimpleServer and the reason it works is because we avoid blocking calls by means of the API design.
Nonblocking server
Yes, there is no implementation yet for C#. Since Thrift is Open Source, that probably simply means that there were nobody running into that use case so far and wanted to spend time on an implementation. That is not to say that such a use case may not exist.
What exists are
Threaded and Threadpool servers
Task.Run(() => { your code })
which may help to solve your use case. Also, when used with ASP.NET there is no need for a nonblocking server since the runtime offers enough support already for multiple connections.
Bottom line
There are certain ways to work around that limitation you experierenced. One additional alternative could be to become a contributor by porting one of the existing (e.g. Java) nonblocking implementation to NetStd (preferred, since C# and NetCore will mature into "deprecated" state with the next release and both be replaced by NetStd eventually)

Problem with installing chocolatey files with task.run on windows 8.1

I'm tasked with creating a tool to help set up customers systems easily. I've created a function that calls a chocolatey script through powershell in c# and I use Task.run to create a new thread so it doesn't affect the UI thread, The system works fine, but I'm having problems with some computers. It's not helped that I have no access to these computers and do not know much about their system, and due to time constraints do not have access to these computers. I do know they have windows 8.1. I was given a windows 10 virtual machine to test on (which I still don't understand as it was known that this was a windows 8 problem)
Here is the code.
I know for a fact(due to the one time I was given access to these computers) that it stops on Task.Run(() => task)
Does anyone know if there are any problems with either chocolatey or Tasks on windows 8.1?
Task callTask = Task.Run(() => ExecuteAsynchronouslyAsync("chocolatey string", CheckBox box, string logName));
public async Task<PowerShellAction> ExecuteAsynchronouslyAsync(String commandStr, CheckBox box, string logName)
{
powerShellAction = new PowerShellAction();
powerShellAction.isFinished = false;
using (PowerShell ps = PowerShell.Create())
{
ps.AddScript(commandStr); // adding the script to the powershell script.
outputCollection = new PSDataCollection<PSObject>();
outputCollection.DataAdded += OutputData;
IAsyncResult result = ps.BeginInvoke<PSObject, PSObject>(null, outputCollection);
PSDataCollection<PSObject> execRes = await Task.Factory.FromAsync(result, ps.EndInvoke);
}
return powerShellAction;
}
Working right now on trying to get a virtual machine of 8.1 to continue trying to debug myself. Any other suggestions would be welcome.

Unfortunately I cannot ensure that my suggestions are correct. The main reason is, that i can't figure out what PowerShellAction is supposed to be. I'm assuming here that PowerShell is System.Management.Automation.PowerShell.
I'm suggesting several things:
Your code does not compile for several reasons: you have no var or type-declaration on the first line of your method and the method-call would not work because of the addition string keyword. Try to avoid pasting in code like yours in the future please because it's pretty hard to rebuild your sample.
Don't bypass a UI control to an async method but use the needed value (e.g. box.IsChecked as a bool) instead.
Add ConfigureAwait(false) to your await to prevent .NET from trying to sync back to the context.
Take more care about exception handling insude of your method.
Dont' return anything if you don't need it in your method.
The code (untestet) could be something like this:
var task = Task.Run(() => ExecutePowerShellAsync("chocolatey string", box.IsChecked, "NameOfTheLog"));
public async Task<PowerShellAction> ExecutePowerShellAsync(String commandStr, bool checkBoxValue, string logName)
{
var powerShellAction = new PowerShellAction();
powerShellAction.isFinished = false;
using (PowerShell ps = PowerShell.Create())
{
ps.AddScript(commandStr); // adding the script to the powershell script.
var outputCollection = new PSDataCollection<PSObject>();
outputCollection.DataAdded += OutputData;
IAsyncResult result = ps.BeginInvoke<PSObject, PSObject>(null, outputCollection);
PSDataCollection<PSObject> execRes = await Task.Factory.FromAsync(result, ps.EndInvoke).ContinueWith(t => {
if (t.IsFaulted)
{
System.Diagnostics.Trace.TraceError("Task faulted with exception: " + t.Exception?.Message);
}
return t.Result;
}).ConfigureAwait(false);
}
return powerShellAction;
}
I use ContinueWith in order to be able to react to any exception that might occur inside the original task.
I'm suggesting this because your description smells like you have a typical thread-lock which means the code simple does not come back due to an exception or context-syncing-problems.

Why did I have to use Task<T> to acheive Synchronous calls?

Im working on a website that integrates with Dynamics 365 with the Dynamics SDK. We have seen errors in the logs such as “Cannot access a disposed object”. Upon further investigation we found out that the SDK methods are not thread safe so needed to refactor the code to take this into account.
We had a method such as follows that would create or update a Contact entity depending on whether it already exists:
public Guid? SetProfile(IProfile profile)
{
using (var xrm = new XrmServiceContext(_organizationService))
{
//check whether account already exists
var crmProfile = GetContact(xrm, profile.UserId);
if (crmProfile == null)
{
//create new account if required
{
crmProfile = new Contact
{
EMailAddress1 = profile.Username,
//lots of properties hidden to make for easier code example
};
}
xrm.AddObject(crmProfile);
}
else
{
//update existing account
crmProfile.new_Title = profile.Title.HasValue ? new OptionSetValue(profile.Title.Value) : null;
//lots of properties hidden to make for easier code example
xrm.UpdateObject(crmProfile);
}
var response = xrm.SaveChanges();
return crmProfile.Id;
}
}
When this method was executed concurrently by 2 or more users the error "Cannot access a disposed object" would be thrown, referring to the XrmServiceContext object.
I therefore knew that I needed to make this method thread-safe, but also it needs to be Synchronous as our UI depends on having the return value of the method. I played around with different threading methods:
Task.Factory.StartNew(() => delegate
new Thread()
However, with both of these methods I wasn't able to get the method to execute synchronously, so I ended up with:
public Guid? SetProfile(IProfile profile)
{
var task = new Task<Guid?>(() =>
{
using (var xrm = new XrmServiceContext(_organizationService))
{
//check whether account already exists
var crmProfile = GetContact(xrm, profile.UserId);
if (crmProfile == null)
{
//create new account if required
{
crmProfile = new Contact
{
EMailAddress1 = profile.Username,
//lots of properties hidden to make for easier code example
};
}
xrm.AddObject(crmProfile);
}
else
{
//update existing account
crmProfile.new_Title = profile.Title.HasValue ? new OptionSetValue(profile.Title.Value) : null;
//lots of properties hidden to make for easier code example
xrm.UpdateObject(crmProfile);
}
var response = xrm.SaveChanges();
return crmProfile.Id;
}
});
task.RunSynchronously();
return task.Result;
}
Everything I seemed to read online suggested I should use the StartNew method, however this is geared towards Asynchronous calls with I could not allow, and it also seemed that it doesnt guarantee a new thread - from what I've read I understand it is clever enough to know when it needs to create a new thread - however in my instance I have to be certain a new thread is used for the call to Dynamics.
Questions:
Anything wrong with the approach I've taken for a Web application?
If I can't use Asynchronous calls, is there any advantage whatsoever to using the StartNew method?
Many thanks for your time in advance
Kind regards
dotdev

c# How to load test a webservice

I need to test if there's any memory leak in our application and monitor to see if memory usage increases too much while processing the requests.
I'm trying to develop some code to make multiple simultaneous calls to our api/webservice method. This api method is not asynchronous and takes some time to complete its operation.
I've made a lot of research about Tasks, Threads and Parallelism, but so far I had no luck. The problem is, even after trying all the below solutions, the result is always the same, it appears to be processing only two requests at the time.
Tried:
-> Creating tasks inside a simple for loop and starting them with and without setting them with TaskCreationOptions.LongRunning
-> Creating threads inside a simple for loop and starting them with and without high priority
-> Creating a list of actions on a simple for loop and starting them using
Parallel.Foreach(list, options, item => item.Invoke)
-> Running directly inside a Parallel.For loop (below)
-> Running TPL methods with and without Options and TaskScheduler
-> Tried with different values for MaxParallelism and maximum threads
-> Checked this post too, but it didn't help either. (Could I be missing something?)
-> Checked some other posts here in Stackoverflow, but with F# solutions that I don't know how to properly translate them to C#. (I never used F#...)
(Task Scheduler class taken from msdn)
Here's the basic structure that I have:
public class Test
{
Data _data;
String _url;
public Test(Data data, string url)
{
_data = data;
_url = url;
}
public ReturnData Execute()
{
ReturnData returnData;
using(var ws = new WebService())
{
ws.Url = _url;
ws.Timeout = 600000;
var wsReturn = ws.LongRunningMethod(data);
// Basically convert wsReturn to my method return, with some logic if/else etc
}
return returnData;
}
}
sealed class ThreadTaskScheduler : TaskScheduler, IDisposable
{
// The runtime decides how many tasks to create for the given set of iterations, loop options, and scheduler's max concurrency level.
// Tasks will be queued in this collection
private BlockingCollection<Task> _tasks = new BlockingCollection<Task>();
// Maintain an array of threads. (Feel free to bump up _n.)
private readonly int _n = 100;
private Thread[] _threads;
public TwoThreadTaskScheduler()
{
_threads = new Thread[_n];
// Create unstarted threads based on the same inline delegate
for (int i = 0; i < _n; i++)
{
_threads[i] = new Thread(() =>
{
// The following loop blocks until items become available in the blocking collection.
// Then one thread is unblocked to consume that item.
foreach (var task in _tasks.GetConsumingEnumerable())
{
TryExecuteTask(task);
}
});
// Start each thread
_threads[i].IsBackground = true;
_threads[i].Start();
}
}
// This method is invoked by the runtime to schedule a task
protected override void QueueTask(Task task)
{
_tasks.Add(task);
}
// The runtime will probe if a task can be executed in the current thread.
// By returning false, we direct all tasks to be queued up.
protected override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued)
{
return false;
}
public override int MaximumConcurrencyLevel { get { return _n; } }
protected override IEnumerable<Task> GetScheduledTasks()
{
return _tasks.ToArray();
}
// Dispose is not thread-safe with other members.
// It may only be used when no more tasks will be queued
// to the scheduler. This implementation will block
// until all previously queued tasks have completed.
public void Dispose()
{
if (_threads != null)
{
_tasks.CompleteAdding();
for (int i = 0; i < _n; i++)
{
_threads[i].Join();
_threads[i] = null;
}
_threads = null;
_tasks.Dispose();
_tasks = null;
}
}
}
And the test code itself:
private void button2_Click(object sender, EventArgs e)
{
var maximum = 100;
var options = new ParallelOptions
{
MaxDegreeOfParallelism = 100,
TaskScheduler = new ThreadTaskScheduler()
};
// To prevent UI blocking
Task.Factory.StartNew(() =>
{
Parallel.For(0, maximum, options, i =>
{
var data = new Data();
// Fill data
var test = new Test(data, _url); //_url is pre-defined
var ret = test.Execute();
// Check return and display on screen
var now = DateTime.Now.ToString("HH:mm:ss");
var newText = $"{Environment.NewLine}[{now}] - {ret.ReturnId}) {ret.ReturnDescription}";
AppendTextBox(newText, ref resultTextBox);
}
}
public void AppendTextBox(string value, ref TextBox textBox)
{
if (InvokeRequired)
{
this.Invoke(new ActionRef<string, TextBox>(AppendTextBox), value, textBox);
return;
}
textBox.Text += value;
}
And the result that I get is basically this:
[10:08:56] - (0) OK
[10:08:56] - (0) OK
[10:09:23] - (0) OK
[10:09:23] - (0) OK
[10:09:49] - (0) OK
[10:09:50] - (0) OK
[10:10:15] - (0) OK
[10:10:16] - (0) OK
etc
As far as I know there's no limitation on the server side. I'm relatively new to the Parallel/Multitasking world. Is there any other way to do this? Am I missing something?
(I simplified all the code for clearness and I believe that the provided code is enough to picture the mentioned scenarios. I also didn't post the application code, but it's a simple WinForms screen just to call and show results. If any code is somehow relevant, please let me know, I can edit and post it too.)
Thanks in advance!
EDIT1: I checked on the server logs that it's receiving the requests two by two, so it's indeed something related to sending them, not receiving.
Could it be a network problem/limitation related to how the framework manages the requests/connections? Or something with the network at all (unrelated to .net)?
EDIT2: Forgot to mention, it's a SOAP webservice.
EDIT3: One of the properties that I send (inside data) needs to change for each request.
EDIT4: I noticed that there's always an interval of ~25 secs between each pair of request, if it's relevant.

I would recommend not to reinvent the wheel and just use one of the existing solutions:
Most obvious choice: if your Visual Studio license allows you can use MS Load Testing Framework, most likely you won't even have to write a single line of code: How to: Create a Web Service Test
SoapUI is a free and open source web services testing tool, it has some limited load testing capabilities
If for some reasons SoapUI is not suitable (i.e. you need to run load tests in clustered mode from several hosts or you need more enhanced reporting) you can use Apache JMeter - free and open source multiprotocol load testing tool which supports web services load testing as well.

A good solution to create load tests without write a own project is use this service https://loader.io/targets
It is free for small tests, you can POST Parameters, Header,... and you have a nice reporting.

Isnt the "two requests at a time" the result of the default maxconnection=2 limit on connectionManagement?
<configuration>
<system.net>
<connectionManagement>
<add address = "http://www.contoso.com" maxconnection = "4" />
<add address = "*" maxconnection = "2" />
</connectionManagement>
</system.net>
</configuration>

My favorite load testing library is NBomber. It has an easy and powerful API, realistic user simulations, and provides you with nice HTML reports about latency and requests per second.
I used it to test my API and wrote an article about how I did it.

WP7 + Waiting for an operation to complete before proceeding

I have a Windows Phone 7 application that is using Silverlight with C#. This application has a method that fires off multiple web service requests. At another point in my code, I have some code that looks like the following:
myProgressBar.Visibility = Visibility.Visible;
while (AreWebServiceCallsDone() == false)
{
// Need "waiting" code here
}
// Proceed
How do I make my UI "wait" a small amount of time, without locking up the UI, and then check again to see if my web service calls are done?
Thank you!

The answer is to not make the UI wait at all but to "go with the (asynchronous) flow", as it were.
Ultimately, the async stuff in C# 5 will solve this particular problem, but there's no timeline for it's release for the desktop CLR let alone Silverlight or WP7.
I'd personally recommend looking into Microsoft.Phone.Reactive, which is the WP7 version of the Reactive Extensions (Rx) and ships with the SDK. It's a pretty big subject that takes a fair amount of time to get your head around, but can really simplify how you deal with asynchronous scenarios.
Assuming each of your web services return different types of data, I would:
Wrap each web service call in an IObservable1
Use Do to "peek" at the message and perform your side effects (like assigning the value locally)
Use Select to "normalize" the types so that they are all of the same type (required for the next step)
Use ForkJoin to execute each of the requests in parallel and to handle when each has completed
1 Creating an IObservable for your request really depends on how asynchronous pattern you are using. Assuming you're using WebClient, here's an extension method that creates an Observable from DownloadStringAsync as a sample (it may look complex but it's just handling errors and cancellation):
public static class ObservableWebClient
{
public static IObservable<string> DownloadStringObservable(
this WebClient webClient, Uri uri)
{
return Observable.Create(observer =>
{
var disposable = new CompositeDisposable();
var completedObservable = Observable.FromEvent<
DownloadStringCompletedEventHandler,
DownloadStringCompletedEventArgs
>(
h => new DownloadStringCompletedEventHandler(h),
h => webClient.DownloadStringCompleted += h,
h => webClient.DownloadStringCompleted h= h
);
disposable.Add(completedObservable
.SelectMany(ev =>
{
return (ev.EventArgs.Error != null)
? Observable.Throw<string>(ev.EventArgs.Error)
: Observable.Return(ev.EventArgs.Result);
})
.Subscribe(observer));
disposable.Add(Disposable.Create(
() => webClient.CancelAsync()));
return disposable;
});
}
}
You can then use it like so:
Note that I've skipped the Do + "normalizing" steps because my data types are all the same (String). As such, I can subscribe to them all as an array (it's a subtlety of how ForkJoin works, if you were wondering)
var webClientA = new WebClient();
var webClientB = new WebClient();
var webClientC = new WebClient();
Observable.ForkJoin(
webClientA.DownloadStringObservable(uriA),
webClientB.DownloadStringObservable(uriB),
webClientC.DownloadStringObservable(uriC),
)
.ObserveOnDispatcher()
.Subscribe(dataArray =>
{
// All three have completed
this.DataA = dataArray[0];
this.DataB = dataArray[1];
this.DataC = dataArray[2];
});

You should be using an async call back method and handle the progress bar's visibility on the call back event.
By using a while, you are making the UI wait for the thread to be executed.
I used this method in my blog post here: http://www.infopoint.com/News-Events/EntryId/29/Building-a-WP7-RSS-reader-Part-1-Basics.aspx