Develop Reference

c# How to load test a webservice

I need to test if there's any memory leak in our application and monitor to see if memory usage increases too much while processing the requests.
I'm trying to develop some code to make multiple simultaneous calls to our api/webservice method. This api method is not asynchronous and takes some time to complete its operation.
I've made a lot of research about Tasks, Threads and Parallelism, but so far I had no luck. The problem is, even after trying all the below solutions, the result is always the same, it appears to be processing only two requests at the time.
Tried:
-> Creating tasks inside a simple for loop and starting them with and without setting them with TaskCreationOptions.LongRunning
-> Creating threads inside a simple for loop and starting them with and without high priority
-> Creating a list of actions on a simple for loop and starting them using
Parallel.Foreach(list, options, item => item.Invoke)
-> Running directly inside a Parallel.For loop (below)
-> Running TPL methods with and without Options and TaskScheduler
-> Tried with different values for MaxParallelism and maximum threads
-> Checked this post too, but it didn't help either. (Could I be missing something?)
-> Checked some other posts here in Stackoverflow, but with F# solutions that I don't know how to properly translate them to C#. (I never used F#...)
(Task Scheduler class taken from msdn)
Here's the basic structure that I have:
public class Test
{
Data _data;
String _url;
public Test(Data data, string url)
{
_data = data;
_url = url;
}
public ReturnData Execute()
{
ReturnData returnData;
using(var ws = new WebService())
{
ws.Url = _url;
ws.Timeout = 600000;
var wsReturn = ws.LongRunningMethod(data);
// Basically convert wsReturn to my method return, with some logic if/else etc
}
return returnData;
}
}
sealed class ThreadTaskScheduler : TaskScheduler, IDisposable
{
// The runtime decides how many tasks to create for the given set of iterations, loop options, and scheduler's max concurrency level.
// Tasks will be queued in this collection
private BlockingCollection<Task> _tasks = new BlockingCollection<Task>();
// Maintain an array of threads. (Feel free to bump up _n.)
private readonly int _n = 100;
private Thread[] _threads;
public TwoThreadTaskScheduler()
{
_threads = new Thread[_n];
// Create unstarted threads based on the same inline delegate
for (int i = 0; i < _n; i++)
{
_threads[i] = new Thread(() =>
{
// The following loop blocks until items become available in the blocking collection.
// Then one thread is unblocked to consume that item.
foreach (var task in _tasks.GetConsumingEnumerable())
{
TryExecuteTask(task);
}
});
// Start each thread
_threads[i].IsBackground = true;
_threads[i].Start();
}
}
// This method is invoked by the runtime to schedule a task
protected override void QueueTask(Task task)
{
_tasks.Add(task);
}
// The runtime will probe if a task can be executed in the current thread.
// By returning false, we direct all tasks to be queued up.
protected override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued)
{
return false;
}
public override int MaximumConcurrencyLevel { get { return _n; } }
protected override IEnumerable<Task> GetScheduledTasks()
{
return _tasks.ToArray();
}
// Dispose is not thread-safe with other members.
// It may only be used when no more tasks will be queued
// to the scheduler. This implementation will block
// until all previously queued tasks have completed.
public void Dispose()
{
if (_threads != null)
{
_tasks.CompleteAdding();
for (int i = 0; i < _n; i++)
{
_threads[i].Join();
_threads[i] = null;
}
_threads = null;
_tasks.Dispose();
_tasks = null;
}
}
}
And the test code itself:
private void button2_Click(object sender, EventArgs e)
{
var maximum = 100;
var options = new ParallelOptions
{
MaxDegreeOfParallelism = 100,
TaskScheduler = new ThreadTaskScheduler()
};
// To prevent UI blocking
Task.Factory.StartNew(() =>
{
Parallel.For(0, maximum, options, i =>
{
var data = new Data();
// Fill data
var test = new Test(data, _url); //_url is pre-defined
var ret = test.Execute();
// Check return and display on screen
var now = DateTime.Now.ToString("HH:mm:ss");
var newText = $"{Environment.NewLine}[{now}] - {ret.ReturnId}) {ret.ReturnDescription}";
AppendTextBox(newText, ref resultTextBox);
}
}
public void AppendTextBox(string value, ref TextBox textBox)
{
if (InvokeRequired)
{
this.Invoke(new ActionRef<string, TextBox>(AppendTextBox), value, textBox);
return;
}
textBox.Text += value;
}
And the result that I get is basically this:
[10:08:56] - (0) OK
[10:08:56] - (0) OK
[10:09:23] - (0) OK
[10:09:23] - (0) OK
[10:09:49] - (0) OK
[10:09:50] - (0) OK
[10:10:15] - (0) OK
[10:10:16] - (0) OK
etc
As far as I know there's no limitation on the server side. I'm relatively new to the Parallel/Multitasking world. Is there any other way to do this? Am I missing something?
(I simplified all the code for clearness and I believe that the provided code is enough to picture the mentioned scenarios. I also didn't post the application code, but it's a simple WinForms screen just to call and show results. If any code is somehow relevant, please let me know, I can edit and post it too.)
Thanks in advance!
EDIT1: I checked on the server logs that it's receiving the requests two by two, so it's indeed something related to sending them, not receiving.
Could it be a network problem/limitation related to how the framework manages the requests/connections? Or something with the network at all (unrelated to .net)?
EDIT2: Forgot to mention, it's a SOAP webservice.
EDIT3: One of the properties that I send (inside data) needs to change for each request.
EDIT4: I noticed that there's always an interval of ~25 secs between each pair of request, if it's relevant.

I would recommend not to reinvent the wheel and just use one of the existing solutions:
Most obvious choice: if your Visual Studio license allows you can use MS Load Testing Framework, most likely you won't even have to write a single line of code: How to: Create a Web Service Test
SoapUI is a free and open source web services testing tool, it has some limited load testing capabilities
If for some reasons SoapUI is not suitable (i.e. you need to run load tests in clustered mode from several hosts or you need more enhanced reporting) you can use Apache JMeter - free and open source multiprotocol load testing tool which supports web services load testing as well.

A good solution to create load tests without write a own project is use this service https://loader.io/targets
It is free for small tests, you can POST Parameters, Header,... and you have a nice reporting.

Isnt the "two requests at a time" the result of the default maxconnection=2 limit on connectionManagement?
<configuration>
<system.net>
<connectionManagement>
<add address = "http://www.contoso.com" maxconnection = "4" />
<add address = "*" maxconnection = "2" />
</connectionManagement>
</system.net>
</configuration>

My favorite load testing library is NBomber. It has an easy and powerful API, realistic user simulations, and provides you with nice HTML reports about latency and requests per second.
I used it to test my API and wrote an article about how I did it.

Using multiple BlockingCollection<T> buffers in implementing pipeline validation

So, I have a requirement to read each record(line) of a large data file and then application various validation rules on each of these lines. So, rather than just apply sequential validation, I decided to see if I could use some pipelining to help speed things up. So, I need to apply the same set of Business validation rules(5 at the moment) to all items in my collection. As there is no need to return output from each validation process, I don't need to worry about passing values from one validation routine to the other. I do however need to make the same data available to all my validation steps and to do this, I came up with coping the same data(records) to 5 different buffers, which will be used by each of the validation stages.
Below is the code I have going. But I have little confidence in this applied and wanted to know if there is a better way of doing this please? I appreciate any help you can give on this please. Thanks in advance.
public static void LoadBuffers(List<BlockingCollection<FlattenedLoadDetail>> outputs,
BlockingCollection<StudentDetail> students)
{
try
{
foreach (var student in students)
{
foreach (var stub in student.RecordYearDetails)
foreach (var buffer in outputs)
buffer.Add(stub);
}
}
finally
{
foreach (var buffer in outputs)
buffer.CompleteAdding();
}
}
public void Process(BlockingCollection<StudentRecordDetail> StudentRecords)
{
//Validate header record before proceeding
if(! IsHeaderRecordValid)
throw new Exception("Invalid Header Record Found.");
const int buffersize = 20;
var buffer1 = new BlockingCollection<FlattenedLoadDetail>(buffersize);
var buffer2 = new BlockingCollection<FlattenedLoadDetail>(buffersize);
var buffer3 = new BlockingCollection<FlattenedLoadDetail>(buffersize);
var buffer4 = new BlockingCollection<FlattenedLoadDetail>(buffersize);
var taskmonitor = new TaskFactory(TaskCreationOptions.LongRunning, TaskContinuationOptions.NotOnCanceled);
using (var loadUpStartBuffer = taskmonitor.StartNew(() => LoadBuffers(
new List<BlockingCollection<FlattenedLoadDetail>>
{buffer1, buffer2, buffer3, buffer4}, StudentRecords)))
{
var recordcreateDateValidationStage = taskmonitor.StartNew(() => ValidateRecordCreateDateActivity.Validate(buffer1));
var uniqueStudentIDValidationStage =
taskmonitor.StartNew(() => ValidateUniqueStudentIDActivity.Validate(buffer2));
var SSNNumberRangeValidationStage =
taskmonitor.StartNew(() => ValidateDocSequenceNumberActivity.Validate(buffer3));
var SSNRecordNumberMatchValidationStage =
taskmonitor.StartNew(() => ValidateStudentSSNRecordNumberActivity.Validate(buffer4));
Task.WaitAll(loadUpStartBuffer, recordcreateDateValidationStage, uniqueStudentIDValidationStage,
SSNNumberRangeValidationStage, SSNRecordNumberMatchValidationStage);
}
}
In fact, if I could tie up the tasks in such a way that once one fails, all the others stop, that would help me a lot but I am a newbie to this pattern and kind of trying to figure out best way to handle this problem I have here. Should I just throw caution to the wind and have each of the validation steps load an output buffer to be passed on to subsequent task? Is that a better way to go with this?

The first question you need to answer for yourself is whether you want to improve latency or throughput.
The strategy you depicted takes a single item and perform parallel calculation on it. This means that an item is serviced very fast, but at the expense of other items that are left waiting for their turn to enter.
Consider an alternative concurrent approach. You can treat the entire validation process as a sequential operation, but simultaneously service more than one item in parallel.
It seems to me that in your case you will benefit more from the latter approach, especially from the perspective of simplicity and since I am guessing that latency is not as important here.

Multithreading in a Single Transaction

According to this MSDN article, you should be able to multithread a process with each thread enlisted in a single root transaction.
I created a sample based on that article where I expect atransaction to be rolled-back (bool[] results should be all false in the foreach loop). Unfortunately, this is not the case, and the outcome is predictably unpredictable (run the example enough times and you will see any combination of bool values in the array).
In addition, I've tried both DependentCloneOption.BlockCommitUntilComplete and DependentCloneOption.RollbackIfNotComplete neither of which produce the expected result.
Secondly, I think ThreadPool.QueueUserWorkItem is ugly code at best, and it would be nice to see something like this using Parallel.ForEach instead.
And finally, my question :) Why the heck is this not working? What am I doing wrong? Is it just flat-out impossible to wrap multiple threads in a single transaction?
namespace Playing
{
class Program
{
static bool[] results = new bool[] { false, false, false };
static void Main(string[] args)
{
try
{
using (var outer = new TransactionScope(
TransactionScopeOption.Required))
{
for (var i = 0; i < 3; i++ )
{
ThreadPool.QueueUserWorkItem(WorkerItem,
new Tuple<int, object>(
i, Transaction.Current.DependentClone(
DependentCloneOption.BlockCommitUntilComplete)));
}
outer.Complete();
}
}
catch { /* Suppress exceptions */ }
// Expect all to be false
foreach (var r in results)
Console.WriteLine(r);
}
private static void WorkerItem(object state)
{
var tup = (Tuple<int, object>)state;
var i = tup.Item1;
var dependent = (DependentTransaction)tup.Item2;
using (var inner = new TransactionScope(dependent))
{
// Intentionally throw exception to force roll-back
if (i == 2)
throw new Exception();
results[i] = true;
inner.Complete();
}
dependent.Complete();
}
}
}

Yours results[] members that have been set to true won't magically set themselves back to false (sadly). That's what Transaction Managers do. Look at the EnlistXXX methods to get an idea of what's involved.
Basically, you'll need to compensate in the event of a rollback. For example, you could subscribe to the root Transaction's TransactionCompleted event and check if the transaction was rolled back. If it was you'll need to restore the previous values for the child workers that completed.
You can also handle the TransactionAbortedException thrown that you are suppressing, or handle it at the worker level (see an example of catching it on this page: http://msdn.microsoft.com/en-us/library/ms973865.aspx)
Typically, with in-memory "transactions" you are better off using the Task library to have the workers batch up results and then "commit" them in a continuation of a parent Task. It's easier than messing about with Transactions, which you only need to do if you are coordinating between memory and some other Transaction Manager (like SQL Server or other processes).

RX: How to concat a Snapshot stream and an Update stream?

I've been trying to create an observable which streams a state-of-the-world (snapshot) from a repository cache, followed by live updates from a separate feed. The catch is that the snapshot call is blocking, so the updates have to be buffered during that time.
This is what I've come up with, a little simplified. The GetStream() method is the one I'm concerned with. I'm wondering whether there is a more elegant solution. Assume GetDataFeed() pulses updates to the cache all day long.
private static readonly IConnectableObservable<long> _updateStream;
public static Constructor()
{
_updateStream = GetDataFeed().Publish();
_updateStream.Connect();
}
static void Main(string[] args)
{
_updateStream.Subscribe(Console.WriteLine);
Console.ReadLine();
GetStream().Subscribe(l => Console.WriteLine("Stream: " + l));
Console.ReadLine();
}
public static IObservable<long> GetStream()
{
return Observable.Create<long>(observer =>
{
var bufferedStream = new ReplaySubject<long>();
_updateStream.Subscribe(bufferedStream);
var data = GetSnapshot();
// This returns the ticks from GetSnapshot
// followed by the buffered ticks from _updateStream
// followed by any subsequent ticks from _updateStream
data.ToObservable().Concat(bufferedStream).Subscribe(observer);
return Disposable.Empty;
});
}
private static IObservable<long> GetDataFeed()
{
var feed = Observable.Interval(TimeSpan.FromSeconds(1));
return Observable.Create<long>(observer =>
{
feed.Subscribe(observer);
return Disposable.Empty;
});
}
Popular opinion opposes Subjects as they are not 'functional', but I can't find a way of doing this without a ReplaySubject. The Replay filter on a hot observable wouldn't work because it would replay everything (potentially a whole day's worth of stale updates).
I'm also concerned about race conditions. Is there a way to guarantee sequencing of some sort, should an earlier update be buffered before the snapshot? Can the whole thing be done more safely and elegantly with other RX operators?
Thanks.
-Will

Whether you use a ReplaySubject or the Replay function really makes no difference. Replay uses a ReplaySubject under the hood. I'll also note that you are leaking subscriptions like mad, which can cause a resource leak. Also, you put no limit on the size of the replay buffer. If you watch the observable all day long, then that replay buffer will keep growing and growing. You should put a limit on it to prevent that.
Here is an updated version of GetStream. In this version I take the simplistic approach of just limitting the Replay to the most recent 1 minute of data. This assumes that GetData will always complete and the observer will observe the results within that 1 minute. Your mileage may vary and you can probably improve upon this scheme. But at least this way when you have watched the observable all day long, that buffer will not have grown unbounded and will still only contain a minute's worth of updates.
public static IObservable<long> GetStream()
{
return Observable.Create<long>(observer =>
{
var updateStreamSubscription = new SingleAssignmentDisposable();
var sequenceDisposable = new SingleAssignmentDisposable();
var subscriptions = new CompositeDisposable(updateStreamDisposable, sequenceDisposable);
// start buffering the updates
var bufferedStream = _updateStream.Replay(TimeSpan.FromMinutes(1));
updateStreamSubscription.Disposable = bufferedStream.Connect();
// now retrieve the initial snapshot data
var data = GetSnapshot();
// subscribe to the snapshot followed by the buffered data
sequenceDisposable.Disposable = data.ToObservable().Concat(bufferedStream).subscribe(observer);
// return the composite disposable which will unsubscribe when the observer wishes
return subscriptions;
});
}
As for your questions about race conditions and filtering out "old" updates...if your snapshot data includes some sort of version information, and your update stream also providers version information, then you can effectively measure the latest version returned by your snapshot query and then filter the buffered stream to ignore values for older versions. Here is a rough example:
public static IObservable<long> GetStream()
{
return Observable.Create<long>(observer =>
{
var updateStreamSubscription = new SingleAssignmentDisposable();
var sequenceDisposable = new SingleAssignmentDisposable();
var subscriptions = new CompositeDisposable(updateStreamDisposable, sequenceDisposable);
// start buffering the updates
var bufferedStream = _updateStream.Replay(TimeSpan.FromMinutes(1));
updateStreamSubscription.Disposable = bufferedStream.Connect();
// now retrieve the initial snapshot data
var data = GetSnapshot();
var snapshotVersion = data.Length > 0 ? data[data.Length - 1].Version : 0;
var filteredUpdates = bufferedStream.Where(update => update.Version > snapshotVersion);
// subscribe to the snapshot followed by the buffered data
sequenceDisposable.Disposable = data.ToObservable().Concat(filteredUpdates).subscribe(observer);
// return the composite disposable which will unsubscribe when the observer wishes
return subscriptions;
});
}
I have successfully used this pattern when merging live updates with a stored snapshot. I haven't yet found an elegant Rx operator that already does this without any race conditions. But the above method could probably be turned into such. :)
Edit: Note I have left out error handling in the examples above. In theory the call to GetSnapshot could fail and you'd leak the subscription to the update stream. I suggest wrapping everything after the CompositeDisposable declaration in a try/catch block, and in the catch handler, ensure call subscriptions.Dispose() before re-throwing the exception.

How to restrict number of concurrent processes?

My situation is as follow:
I have an application that can be started only a fixed number of times (less than 50).
A separate central process to manage other processes is not allowed due to business requirement. (ie. if a nice solution that involves ONLY the application processes is still acceptable)
I am using C# for developing the application and therefore managed solution is preferred.
I have to deal with "unexpected" cases such as the processes can be terminated by using TaskManager.
I am thinking of solution that make use of system-wide mutex. However, it doesn't survive the "Unexpected" cases very well in the way that it leaves "abandoned" mutex. If this is a good way, may I ask what is the catch of "ignoring" the mutex abandoned?

One approach would be to query for the process list and count the number of instances currently alive. Another approach, more comples, would be to broadcast UDP and count the number of responses. I have used this pattern for distributed scenarios related to job processors.
HTH
Colby Africa

You could use a shared memory segment and increment a count each time an application is opened, and decrement when the application is closed. A more simple approach may be to use an interprocess semaphore which you alluded to in your question.

When a process is terminated via an "unexpected" event such as task manager process killing it should throw a ThreadAbortException. You should really try to wrap your mutex holding in some sort of try / finally that will allow you to release it as the thread is aborting.
I'm not 100% sure that's true but there aught to be some way to respond to situations like that.

Expanding on the Process List approach, using WMI.NET with C# may look like this:
using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Text;
using System.Management;
namespace WmiProc
{
class Program
{
static void Main(string[] args)
{
ManagementScope ms = new System.Management.ManagementScope(
#"\\myserver\root\cimv2");
var oq = new System.Management.ObjectQuery(
"SELECT * FROM Win32_Process where Name='myprocname'");
ManagementObjectSearcher query1 = new ManagementObjectSearcher(ms, oq);
ManagementObjectCollection procsCollection = query1.Get();
Console.WriteLine("process count:{0}", procsCollection.Count);
}
}
}
EDIT: There will be some separation of starting times, such that letting too many processes to run at once is not likely. You'll have to test for specific behavior in your environment.
Maybe you can periodically check the process count from a separate (long running) process and terminate excess processes according to some criterion (e.g. newest).

Well, you could work with named Mutex-instances.
Use a personal naming-scheme for Mutexes, request this name and check the result of a mutex with this name already was created.
If you use a naming scheme with an incremental element, you can try all mutex-names ascending you incremental element, and count like this, how many mutex were created.
Needs some improvement on handling with released mutexes still, but that seems trivial.
class Program
{
private static Mutex mutex = null;
static void Main(string[] args)
{
AppDomain.CurrentDomain.ProcessExit += new EventHandler(CurrentDomain_ProcessExit);
int count = Program.CheckInstanceCount();
Console.WriteLine("This is instance {0} running.", count);
Console.Read();
}
static void CurrentDomain_ProcessExit(object sender, EventArgs e)
{
Program.mutex.ReleaseMutex();
Program.mutex.Close();
}
private static int CheckInstanceCount()
{
int result = 0;
bool created = false;
for (int i = 0; i < 50; i++)
{
/* try to create a mutex with this name,
* if it does exist, another instance
* of this program is running
*/
mutex = new Mutex(true, string.Concat(AppDomain.CurrentDomain.FriendlyName, i.ToString()), out created);
if (created)
{
// this instance is instance #i currently running
result = i;
break;
}
}
return result;
}
}

I couldn't add comments to an above answer, but from reading the above answers, and comments, it seems like you should be able to combine a mutex with the Process Instance check.
// You can use any System wide mutual exclusion mechanism here
bool waitAndLockMutex();
void unlockMutex();
// returns the number of processes who use the specified command
int getProcessCount();
void main() {
try {
waitAndLockMutex();
if (getProcessCount() > MAX_ALLOWED)
return;
doUsualWork();
} finally {
unlockMutex();
}
}
Note that the above code is simply for illustrative purposes, and the body for the declared functions calls can be easily written using .NET
EDIT:
If you do not want to go the route of counting the processes of interest, you can use global mutex for it. Not sure if .NET exposes that. But the gist is that you can acquire all the mutexes till the MAX, and in the process if you get a Mutex that has not yet been created or is ABANDONED, then you go ahead and let the process launch, else exit saying exceeding max count
void main() {
for (int i = 0; i < MAX; ++i) {
int status = TryToAcquireMutex("mutex" + i);
continue if (status == locked);
if (status == success || status == WAIT_ABANDONED) {
doUsusalWork();
}
}
}