Develop Reference

Why did I have to use Task<T> to acheive Synchronous calls?

Im working on a website that integrates with Dynamics 365 with the Dynamics SDK. We have seen errors in the logs such as “Cannot access a disposed object”. Upon further investigation we found out that the SDK methods are not thread safe so needed to refactor the code to take this into account.
We had a method such as follows that would create or update a Contact entity depending on whether it already exists:
public Guid? SetProfile(IProfile profile)
{
using (var xrm = new XrmServiceContext(_organizationService))
{
//check whether account already exists
var crmProfile = GetContact(xrm, profile.UserId);
if (crmProfile == null)
{
//create new account if required
{
crmProfile = new Contact
{
EMailAddress1 = profile.Username,
//lots of properties hidden to make for easier code example
};
}
xrm.AddObject(crmProfile);
}
else
{
//update existing account
crmProfile.new_Title = profile.Title.HasValue ? new OptionSetValue(profile.Title.Value) : null;
//lots of properties hidden to make for easier code example
xrm.UpdateObject(crmProfile);
}
var response = xrm.SaveChanges();
return crmProfile.Id;
}
}
When this method was executed concurrently by 2 or more users the error "Cannot access a disposed object" would be thrown, referring to the XrmServiceContext object.
I therefore knew that I needed to make this method thread-safe, but also it needs to be Synchronous as our UI depends on having the return value of the method. I played around with different threading methods:
Task.Factory.StartNew(() => delegate
new Thread()
However, with both of these methods I wasn't able to get the method to execute synchronously, so I ended up with:
public Guid? SetProfile(IProfile profile)
{
var task = new Task<Guid?>(() =>
{
using (var xrm = new XrmServiceContext(_organizationService))
{
//check whether account already exists
var crmProfile = GetContact(xrm, profile.UserId);
if (crmProfile == null)
{
//create new account if required
{
crmProfile = new Contact
{
EMailAddress1 = profile.Username,
//lots of properties hidden to make for easier code example
};
}
xrm.AddObject(crmProfile);
}
else
{
//update existing account
crmProfile.new_Title = profile.Title.HasValue ? new OptionSetValue(profile.Title.Value) : null;
//lots of properties hidden to make for easier code example
xrm.UpdateObject(crmProfile);
}
var response = xrm.SaveChanges();
return crmProfile.Id;
}
});
task.RunSynchronously();
return task.Result;
}
Everything I seemed to read online suggested I should use the StartNew method, however this is geared towards Asynchronous calls with I could not allow, and it also seemed that it doesnt guarantee a new thread - from what I've read I understand it is clever enough to know when it needs to create a new thread - however in my instance I have to be certain a new thread is used for the call to Dynamics.
Questions:
Anything wrong with the approach I've taken for a Web application?
If I can't use Asynchronous calls, is there any advantage whatsoever to using the StartNew method?
Many thanks for your time in advance
Kind regards
dotdev

c# How to load test a webservice

I need to test if there's any memory leak in our application and monitor to see if memory usage increases too much while processing the requests.
I'm trying to develop some code to make multiple simultaneous calls to our api/webservice method. This api method is not asynchronous and takes some time to complete its operation.
I've made a lot of research about Tasks, Threads and Parallelism, but so far I had no luck. The problem is, even after trying all the below solutions, the result is always the same, it appears to be processing only two requests at the time.
Tried:
-> Creating tasks inside a simple for loop and starting them with and without setting them with TaskCreationOptions.LongRunning
-> Creating threads inside a simple for loop and starting them with and without high priority
-> Creating a list of actions on a simple for loop and starting them using
Parallel.Foreach(list, options, item => item.Invoke)
-> Running directly inside a Parallel.For loop (below)
-> Running TPL methods with and without Options and TaskScheduler
-> Tried with different values for MaxParallelism and maximum threads
-> Checked this post too, but it didn't help either. (Could I be missing something?)
-> Checked some other posts here in Stackoverflow, but with F# solutions that I don't know how to properly translate them to C#. (I never used F#...)
(Task Scheduler class taken from msdn)
Here's the basic structure that I have:
public class Test
{
Data _data;
String _url;
public Test(Data data, string url)
{
_data = data;
_url = url;
}
public ReturnData Execute()
{
ReturnData returnData;
using(var ws = new WebService())
{
ws.Url = _url;
ws.Timeout = 600000;
var wsReturn = ws.LongRunningMethod(data);
// Basically convert wsReturn to my method return, with some logic if/else etc
}
return returnData;
}
}
sealed class ThreadTaskScheduler : TaskScheduler, IDisposable
{
// The runtime decides how many tasks to create for the given set of iterations, loop options, and scheduler's max concurrency level.
// Tasks will be queued in this collection
private BlockingCollection<Task> _tasks = new BlockingCollection<Task>();
// Maintain an array of threads. (Feel free to bump up _n.)
private readonly int _n = 100;
private Thread[] _threads;
public TwoThreadTaskScheduler()
{
_threads = new Thread[_n];
// Create unstarted threads based on the same inline delegate
for (int i = 0; i < _n; i++)
{
_threads[i] = new Thread(() =>
{
// The following loop blocks until items become available in the blocking collection.
// Then one thread is unblocked to consume that item.
foreach (var task in _tasks.GetConsumingEnumerable())
{
TryExecuteTask(task);
}
});
// Start each thread
_threads[i].IsBackground = true;
_threads[i].Start();
}
}
// This method is invoked by the runtime to schedule a task
protected override void QueueTask(Task task)
{
_tasks.Add(task);
}
// The runtime will probe if a task can be executed in the current thread.
// By returning false, we direct all tasks to be queued up.
protected override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued)
{
return false;
}
public override int MaximumConcurrencyLevel { get { return _n; } }
protected override IEnumerable<Task> GetScheduledTasks()
{
return _tasks.ToArray();
}
// Dispose is not thread-safe with other members.
// It may only be used when no more tasks will be queued
// to the scheduler. This implementation will block
// until all previously queued tasks have completed.
public void Dispose()
{
if (_threads != null)
{
_tasks.CompleteAdding();
for (int i = 0; i < _n; i++)
{
_threads[i].Join();
_threads[i] = null;
}
_threads = null;
_tasks.Dispose();
_tasks = null;
}
}
}
And the test code itself:
private void button2_Click(object sender, EventArgs e)
{
var maximum = 100;
var options = new ParallelOptions
{
MaxDegreeOfParallelism = 100,
TaskScheduler = new ThreadTaskScheduler()
};
// To prevent UI blocking
Task.Factory.StartNew(() =>
{
Parallel.For(0, maximum, options, i =>
{
var data = new Data();
// Fill data
var test = new Test(data, _url); //_url is pre-defined
var ret = test.Execute();
// Check return and display on screen
var now = DateTime.Now.ToString("HH:mm:ss");
var newText = $"{Environment.NewLine}[{now}] - {ret.ReturnId}) {ret.ReturnDescription}";
AppendTextBox(newText, ref resultTextBox);
}
}
public void AppendTextBox(string value, ref TextBox textBox)
{
if (InvokeRequired)
{
this.Invoke(new ActionRef<string, TextBox>(AppendTextBox), value, textBox);
return;
}
textBox.Text += value;
}
And the result that I get is basically this:
[10:08:56] - (0) OK
[10:08:56] - (0) OK
[10:09:23] - (0) OK
[10:09:23] - (0) OK
[10:09:49] - (0) OK
[10:09:50] - (0) OK
[10:10:15] - (0) OK
[10:10:16] - (0) OK
etc
As far as I know there's no limitation on the server side. I'm relatively new to the Parallel/Multitasking world. Is there any other way to do this? Am I missing something?
(I simplified all the code for clearness and I believe that the provided code is enough to picture the mentioned scenarios. I also didn't post the application code, but it's a simple WinForms screen just to call and show results. If any code is somehow relevant, please let me know, I can edit and post it too.)
Thanks in advance!
EDIT1: I checked on the server logs that it's receiving the requests two by two, so it's indeed something related to sending them, not receiving.
Could it be a network problem/limitation related to how the framework manages the requests/connections? Or something with the network at all (unrelated to .net)?
EDIT2: Forgot to mention, it's a SOAP webservice.
EDIT3: One of the properties that I send (inside data) needs to change for each request.
EDIT4: I noticed that there's always an interval of ~25 secs between each pair of request, if it's relevant.

I would recommend not to reinvent the wheel and just use one of the existing solutions:
Most obvious choice: if your Visual Studio license allows you can use MS Load Testing Framework, most likely you won't even have to write a single line of code: How to: Create a Web Service Test
SoapUI is a free and open source web services testing tool, it has some limited load testing capabilities
If for some reasons SoapUI is not suitable (i.e. you need to run load tests in clustered mode from several hosts or you need more enhanced reporting) you can use Apache JMeter - free and open source multiprotocol load testing tool which supports web services load testing as well.

A good solution to create load tests without write a own project is use this service https://loader.io/targets
It is free for small tests, you can POST Parameters, Header,... and you have a nice reporting.

Isnt the "two requests at a time" the result of the default maxconnection=2 limit on connectionManagement?
<configuration>
<system.net>
<connectionManagement>
<add address = "http://www.contoso.com" maxconnection = "4" />
<add address = "*" maxconnection = "2" />
</connectionManagement>
</system.net>
</configuration>

My favorite load testing library is NBomber. It has an easy and powerful API, realistic user simulations, and provides you with nice HTML reports about latency and requests per second.
I used it to test my API and wrote an article about how I did it.

Better performance of a service call than a library was not expected

I was trying to figure out some performance values of two scenarios. I thought I was only going to declare the obvious at first. But when I got the results I got a little confused. And now I am looking for a justification for the case.
I have a library which makes couple of queries through a MongoDb database and Active Directory services, then returns the results to client, which are:
GetUserType - to MongoDb - there is a collection which has username and type fields in its all documents. In the query I give the username and ask for the type field.
LoginCheck - to Active Directory - given the username and the password from the client, I create a PrincipalContext object to access to AD server and call ValidateCredentials upon it.
This job is performing on an existing MVC application at the moment. And we are going to create a new desktop application and employ it with the same job.
We were curios about how different can these two scenarios perform? We thought that a direct call to a library without any http connection would perform better than a service request without an hesitation. But we still wondered how much difference is there, and if it was acceptable we are going to make it work through the rest MVC service - because of reasons :)
Hence we tested out the following architectures:
Scenario 1:
Scenario 2:
Basically, what I do for performance test is this:
For scenario 1:
for(var i = 0; i<10000; i++)
{
new Class1().HeavyMethod();
}
For scenario 2:
// client side
for(var i = 0; i<10000; i++)
{
using ( var client = new HttpClient() )
{
var values = new Dictionary<string, string>();
var content = new FormUrlEncodedContent(values);
var response = client.PostAsync("http://localhost:654/Home/HeavyLift", content).Result;
var responseString = response.Content.ReadAsStringAsync().Result;
}
}
// MVC rest service
public class HomeController : Controller
{
public JsonResult HeavyLift()
{
return Json(new Class1().HeavyMethod(), JsonRequestBehavior.AllowGet);
}
}
Common Class:
public class Class1
{
public string HeavyMethod ()
{
var userName = "asdfasdfasd";
var password = "asdfasdfasdf";
try
{
// this call is to MongoDB
var userType = Personnel.GetPersonnelsType(userName).Result;
// this call is to Active Directory
var user = new ADUser(new Session
{
UserType = userType.Type,
UserName = userName,
Password = password
});
return userType.Type + "-" + user.Auth();
}
catch ( Exception e )
{
return e.Message;
}
}
}
The results for 10000 consecutive calls are confusingly shocking:
Scenario 1: 159181 ms
Scenario 2: 13952 ms
Scenario 1 starts off pretty quicly for the first few dosens of calls, then it starts to slow down.
Scenario 2 though offers a constant response time through 10k calls.
What is actually happening here?
Note: I checked the memory and cpu usages of the server that this scenarios runs on(everything runs on the same server) but there is nothing interesting actually, they are behaving just the same in terms of memory and cpu resources.

using EWS Managed API with ASP.NET MVC

I'm trying to create a web app which does many things but the one that I'm currently focused in is the inbox count. I want to use EWS StreamSubscription so that I can get notification for each event and returns the total count of items in the inbox. How can I use this in terms of MVC? I did find some code from Microsoft tutorial that I was gonna test, but I just couldn't figure how I could use it in MVC world i.e. What's the model going to be, if model is the count then how does it get notified every time an event occurs in Exchange Server, etc.
Here's the code I downloaded from Microsoft, but just couldn't understand how I can convert the count to json and push it to client as soon as a new change event occurs. NOTE: This code is unchanged, so it doesn't return count, yet.
using System;
using System.Linq;
using System.Net;
using System.Threading;
using Microsoft.Exchange.WebServices.Data;
namespace StreamingNotificationsSample
{
internal class Program
{
private static AutoResetEvent _Signal;
private static ExchangeService _ExchangeService;
private static string _SynchronizationState;
private static Thread _BackroundSyncThread;
private static StreamingSubscriptionConnection CreateStreamingSubscription(ExchangeService service,
StreamingSubscription subscription)
{
var connection = new StreamingSubscriptionConnection(service, 30);
connection.AddSubscription(subscription);
connection.OnNotificationEvent += OnNotificationEvent;
connection.OnSubscriptionError += OnSubscriptionError;
connection.OnDisconnect += OnDisconnect;
connection.Open();
return connection;
}
private static void SynchronizeChangesPeriodically()
{
while (true)
{
try
{
// Get all changes from the server and process them according to the business
// rules.
SynchronizeChanges(new FolderId(WellKnownFolderName.Inbox));
}
catch (Exception ex)
{
Console.WriteLine("Failed to synchronize items. Error: {0}", ex);
}
// Since the SyncFolderItems operation is a
// rather expensive operation, only do this every 10 minutes
Thread.Sleep(TimeSpan.FromMinutes(10));
}
}
public static void SynchronizeChanges(FolderId folderId)
{
bool moreChangesAvailable;
do
{
Console.WriteLine("Synchronizing changes...");
// Get all changes since the last call. The synchronization cookie is stored in the _SynchronizationState field.
// Only the the ids are requested. Additional properties should be fetched via GetItem calls.
var changes = _ExchangeService.SyncFolderItems(folderId, PropertySet.IdOnly, null, 512,
SyncFolderItemsScope.NormalItems, _SynchronizationState);
// Update the synchronization cookie
_SynchronizationState = changes.SyncState;
// Process all changes
foreach (var itemChange in changes)
{
// This example just prints the ChangeType and ItemId to the console
// LOB application would apply business rules to each item.
Console.Out.WriteLine("ChangeType = {0}", itemChange.ChangeType);
Console.Out.WriteLine("ChangeType = {0}", itemChange.ItemId);
}
// If more changes are available, issue additional SyncFolderItems requests.
moreChangesAvailable = changes.MoreChangesAvailable;
} while (moreChangesAvailable);
}
public static void Main(string[] args)
{
// Create new exchange service binding
// Important point: Specify Exchange 2010 with SP1 as the requested version.
_ExchangeService = new ExchangeService(ExchangeVersion.Exchange2010_SP1)
{
Credentials = new NetworkCredential("user", "password"),
Url = new Uri("URL to the Exchange Web Services")
};
// Process all items in the folder on a background-thread.
// A real-world LOB application would retrieve the last synchronization state first
// and write it to the _SynchronizationState field.
_BackroundSyncThread = new Thread(SynchronizeChangesPeriodically);
_BackroundSyncThread.Start();
// Create a new subscription
var subscription = _ExchangeService.SubscribeToStreamingNotifications(new FolderId[] {WellKnownFolderName.Inbox},
EventType.NewMail);
// Create new streaming notification conection
var connection = CreateStreamingSubscription(_ExchangeService, subscription);
Console.Out.WriteLine("Subscription created.");
_Signal = new AutoResetEvent(false);
// Wait for the application to exit
_Signal.WaitOne();
// Finally, unsubscribe from the Exchange server
subscription.Unsubscribe();
// Close the connection
connection.Close();
}
private static void OnDisconnect(object sender, SubscriptionErrorEventArgs args)
{
// Cast the sender as a StreamingSubscriptionConnection object.
var connection = (StreamingSubscriptionConnection) sender;
// Ask the user if they want to reconnect or close the subscription.
Console.WriteLine("The connection has been aborted; probably because it timed out.");
Console.WriteLine("Do you want to reconnect to the subscription? Y/N");
while (true)
{
var keyInfo = Console.ReadKey(true);
{
switch (keyInfo.Key)
{
case ConsoleKey.Y:
// Reconnect the connection
connection.Open();
Console.WriteLine("Connection has been reopened.");
break;
case ConsoleKey.N:
// Signal the main thread to exit.
Console.WriteLine("Terminating.");
_Signal.Set();
break;
}
}
}
}
private static void OnNotificationEvent(object sender, NotificationEventArgs args)
{
// Extract the item ids for all NewMail Events in the list.
var newMails = from e in args.Events.OfType<ItemEvent>()
where e.EventType == EventType.NewMail
select e.ItemId;
// Note: For the sake of simplicity, error handling is ommited here.
// Just assume everything went fine
var response = _ExchangeService.BindToItems(newMails,
new PropertySet(BasePropertySet.IdOnly, ItemSchema.DateTimeReceived,
ItemSchema.Subject));
var items = response.Select(itemResponse => itemResponse.Item);
foreach (var item in items)
{
Console.Out.WriteLine("A new mail has been created. Received on {0}", item.DateTimeReceived);
Console.Out.WriteLine("Subject: {0}", item.Subject);
}
}
private static void OnSubscriptionError(object sender, SubscriptionErrorEventArgs args)
{
// Handle error conditions.
var e = args.Exception;
Console.Out.WriteLine("The following error occured:");
Console.Out.WriteLine(e.ToString());
Console.Out.WriteLine();
}
}
}
I just want to understand the basic concept as in what can be model, and where can I use other functions.

Your problem is that you are confusing a service (EWS) with your applications model. They are two different things. Your model is entirely in your control, and you can do whatever you want with it. EWS is outside of your control, and is merely a service you call to get data.
In your controller, you call the EWS service and get the count. Then you populate your model with that count, then in your view, you render that model property. It's really that simple.
A web page has no state. It doesn't get notified when things change. You just reload the page and get whatever the current state is (ie, whatever the current count is).
In more advanced applications, like Single Page Apps, with Ajax, you might periodically query the service in the background. Or, you might have a special notification service that uses something like SignalR to notify your SPA of a change, but these concepts are far more advanced than you currently are. You should probably develop your app as a simple stateless app first, then improve it to add ajax functionality or what not once you have a better grasp of things.

That's a very broad question without a clear-cut answer. Your model could certainly have a "Count" property that you could update. The sample code you found would likely be used by your controller.

I need ideas to unit test this Tasked method

This is the method in question:
public void StartBatchProcessing(IFileBatch fileBatch)
{
var dataWarehouseFactsMerger = m_dataWarehouseFactsMergerFactory.Create(fileBatch);
dataWarehouseFactsMerger.Merge();
if(!m_isTaskStarted)
{
m_isTaskStarted = true;
m_lastQueuedBatchProcessingTask = new TaskFactory().StartNew(() => ProcessBatch(dataWarehouseFactsMerger));
}
else
{
m_lastQueuedBatchProcessingTask = m_lastQueuedBatchProcessingTask.ContinueWith(previous => ProcessBatch(dataWarehouseFactsMerger));
}
}
As you can see I'm using TPL to queue tasks one after the other and I would like to test that the tasks will execute in the order they arrive as soon as the previous one finishes.
The ProcessBatch method is protected so I think it could be overwritten in a derived class and be used to set some flag or something and assert that.
All ideas are welcome and appreciated.

You could create an implementation of DataWarehouseFactsMergerFactory that creates implementations of DataWarehouseFactsMerger that are capable of logging which fileBatch was entered and the start time of each task, but for the rest don't really do anything.