Develop Reference

Best practice for threading in a Web API

I have a simple Web API with a single GET controller only. The GET controller reads XML data and returns it in JSON format.
The GET API is supposed to be excessed every minute but in case if there is no GET request for more than 1 minute then I need to start backing up the data (simply in XML format..nothing fancy).
My GET Api
//GET api/
public IHttpActionResult Get()
{
try
{
//Read XML
XDocument xDoc = XDocument.Load(#"D:\myfile.xml");
//Convert XML data into JSON string
string jsonStr = JsonConvert.SerializeXNode(xDoc);
JObject json = JObject.Parse(jsonStr);
return Ok(json);
}
catch (Exception ex)
{
return NotFound();
}
}
Sample XML:
<CurrentStatus>
<Time Stamp= "20181011133631244">
<price>12</price>
<amount>100</amount>
</Time>
</CurrentStatus>
Sample Backup in XML:
<CurrentStatus>
<Time Stamp= "20181011133631244">
<price>12</price>
<amount>100</amount>
</Time>
<Time Stamp= "20181011133633354">
<price>11</price>
<amount>120</amount>
</Time>
<Time Stamp= "20181011133633204">
<price>15</price>
<amount>90</amount>
</Time>
</CurrentStatus>
My Planned Logic: I am planning to declare a static variable _lastAccessedTimestamp, which will be monitored by a separate thread. If currentTime - _lastAccessedTimestamp > 1 min then start backing-up the data.
Question: Good-practice to implement threading in Web API for this kind of scenario (or any other better option)?
PS: Web Service will run on a localhost IIS server.

After each request I would set/reset a timer for 1 min. Having a variable that you poll from a different thread is not efficient and it raises other problems.
Even more, you can use a proper job handling lib (like Hangfire - https://www.hangfire.io/) so you can just schedule a job (after 1 min) and reschedule it after each request, in case a new request comes.
Be aware that IIS can stop your process at any time, you might want to look into that.
http://docs.hangfire.io/en/latest/deployment-to-production/making-aspnet-app-always-running.html
For the implementation, you can use a middleware/filter that does this at the end of each request automatically. No need to add this in the controller. It's also easier to add multiple controllers without duplicating the code.

You can use something like Hangfire to fire off a task, and then that task can be handled by something. That could be a console app, another web application, an Azure Function, whatever. The point is that it's external to your main web application. Then, you can immediately return, without waiting for that thing, whatever it is, to finish. You can, also, utilize something like SignalR and web workers to push status updates to the client.

I would do this using System.Timers.Timer. Further reading here -> Time Class
Please know that my answer uses dependency injection but could easily be refactored to instantiating normal classes.
I would Implement a timer service which is configured using an interval in your case 1 minute. Im going to share my timer implementation below , it is registered in the dependency injection container as a singleton but should give you something to work with.
public class TimerService : ITimerService
{
private readonly System.Timers.Timer _timer;
private DateTime _startTime = DateTime.Now;
private double _timerSettings;
public TimerService()
{
_timer = new System.Timers.Timer
{
AutoReset = true,
Interval = 60000,
};
_timer.Elapsed += (sender, args) =>
{
//Backup Data method here
_startTime = DateTime.Now;
};
}
public double GetTimerInterval()
{
return _timer.Interval;
}
public void StopTimer()
{
if (_timer == null)
{
throw new ApplicationException("Timer not primed.");
}
_timer.Stop();
}
public void StartTimer()
{
if (_timer == null)
{
throw new ApplicationException("Timer not primed.");
}
_startTime = DateTime.Now;
_timer.Start();
}
}
On application start you must kick off the timer , after every minute it will start doing the backup of data. Due to the AutoReset = true.
Now simply reset the timer in your controller
[Route("api/[controller]/[action]")]
public class XmlController : Controller
{
private readonly ITimerService _timerService;
public XmlController(ITimerService timerService)
{
//Injected in
_timerService = timerService;
}
[HttpGet]
public IActionResult ProccessXML(object someXMLObject)
{
_timerService.StopTimer();
SomeMethodWithXml(someXMLObject)
//Reset Timer
_timerService.StartTimer();
return Ok();
}
}

How to make a static class update its own variables constantly?

I have a user control that displays information from the database. This user control has to update these information constantly(let's say every 5 seconds). A few instances of this user control is generated programmatically during run time in a single page. In the code behind of this user control I added a code that sends a query to the database to get the needed information (which means every single instance of the user control is doing this). But this seems to slow down the processing of queries so I am making a static class that will do the querying and store the information in its variables and let the instances of my user control access those variables. Now I need this static class to do queries every 5 seconds to update its variables. I tried using a new thread to do this but the variables don't seem to be updated since I always get a NullReferenceException whenever I access them from a different class.
Here's my static class:
public static class SessionManager
{
public static volatile List<int> activeSessionsPCIDs;
public static volatile List<int> sessionsThatChangedStatus;
public static volatile List<SessionObject> allSessions;
public static void Initialize() {
Thread t = new Thread(SetProperties);
t.Start();
}
public static void SetProperties() {
SessionDataAccess sd = new SessionDataAccess();
while (true) {
allSessions = sd.GetAllSessions();
activeSessionsPCIDs = new List<int>();
sessionsThatChangedStatus = new List<int>();
foreach (SessionObject session in allSessions) {
if (session.status == 1) { //if session is active
activeSessionsPCIDs.Add(session.pcid);
}
if (session.status != session.prevStat) { //if current status doesn't match the previous status
sessionsThatChangedStatus.Add(session.pcid);
}
}
Thread.Sleep(5000);
}
}
And this is how I am trying to access the variables in my static class:
protected void Page_Load(object sender, EventArgs e)
{
SessionManager.Initialize();
loadSessions();
}
private void loadSessions()
{ // refresh the current_sessions table
List<int> pcIds = pcl.GetPCIds(); //get the ids of all computers
foreach (SessionObject s in SessionManager.allSessions)
{
SessionInfo sesInf = (SessionInfo)LoadControl("~/UserControls/SessionInfo.ascx");
sesInf.session = s;
pnlMonitoring.Controls.Add(sesInf);
}
}
Any help, please? Thanks

Multiple threads problem
You have one thread that gets created for each and every call to SessionManager.Initialize.
That happens more than once in the lifetime of the process.
IIS recycles your app at some point, after a period of time should you have absolutely no requests.
Until that happens, all your created threads continue to run.
After the first PageLoad you will have one thread which updates stuff every 5 seconds.
If you refresh the page again you'll have two threads, possibly with different offsets in time but each of which, doing the same thing at 5 second intervals.
You should atomically check to see if your background thread is started already. You need at least an extra bool static field and a object static field which you should use like a Monitor (using the lock keyword).
You should also stop relying on volatile and simply using lock to make sure that other threads "observe" updated values for your static List<..> fields.
It may be the case that the other threads don't observe a change field and thusly, for them, the field is still null - therefore you get the NullReferenceException.
About volatile
Using volatile is bad, at least in .NET. There is a 90% chance that you think you know what it is doing and it's not true and there's a 99% chance that you feel relief because you used volatile and you aren't checking for other multitasking hazards the way you should.
RX to the rescue
I strongly suggest you take a look at this wonderful thing called Reactive Extensions.
Believe me, a couple of days' research combined with the fact that you're in a perfect position to use RX will pay of, not just now but in the future as well.
You get to keep your static class, but instead of materialised values that get stored within that class you create pipes that carry information. The information flows when you want it to flow. You get to have subscribers to those pipes. The number of subscribers does not affect the overall performance of your app.
Your app will be more scalable, and more robust.
Good luck!

There are few solution for this approach:
One of them is:
It's better in Global.asax in Application_start or Session_Start (depends on your case) create Thread to call your method:
Use below code :
var t = Task.Factory.StartNew(() => {
while(true)
{
SessionManager.SetProperties();
Task.Delay(5);
}
});
Second solution is using Job Scheduler for ASP.NET (that's my ideal solution).
for more info you can check this link How to run Background Tasks in ASP.NET
and third solution is rewrite your static class as follow:
public static class SessionManager
{
public static volatile List<int> activeSessionsPCIDs;
public static volatile List<int> sessionsThatChangedStatus;
public static volatile List<SessionObject> allSessions;
static SessionManager()
{
Initialize();
}
public static void Initialize() {
var t = Task.Factory.StartNew(() => {
while(true)
{
SetProperties();
Task.Delay(5);
}
});
}
public static void SetProperties() {
SessionDataAccess sd = new SessionDataAccess();
while (true) {
allSessions = sd.GetAllSessions();
activeSessionsPCIDs = new List<int>();
sessionsThatChangedStatus = new List<int>();
foreach (SessionObject session in allSessions) {
if (session.status == 1) { //if session is active
activeSessionsPCIDs.Add(session.pcid);
}
if (session.status != session.prevStat) { //if current status doesn't match the previous status
sessionsThatChangedStatus.Add(session.pcid);
}
}
Thread.Sleep(5000);
}
}

This is a solution that is a change in approach, but I kept the solution in Web Forms, to make it more directly applicable to your use case.
SignalR is a technology that enables real-time, two way communication between server and clients (browsers), which can replace your static session data class. Below, I have implemented a simple example to demonstrate the concept.
As a sample, create a new ASP.NET Web Forms application and add the SignalR package from nuget.
Install-Package Microsoft.AspNet.SignalR
You will need to add a new Owin Startup class and add these 2 lines:
using Microsoft.AspNet.SignalR;
... and within the method
app.MapSignalR();
Add some UI elements to Default.aspx:
<div class="jumbotron">
<H3 class="MyName">Loading...</H3>
<p class="stats">
</p>
</div>
Add the following JavaScript to the Site.Master. This code references signalr, and implement client-side event handlers and initiates contact with the signalr hub from the browser. here's the code:
<script src="Scripts/jquery.signalR-2.2.0.min.js"></script>
<script src="signalr/hubs"></script>
<script >
var hub = $.connection.sessiondata;
hub.client.someOneJoined = function (name) {
var current = $(".stats").text();
current = current + '\nuser ' + name + ' joined.';
$(".stats").text(current);
};
hub.client.myNameIs = function (name) {
$(".MyName").text("Your user id: " + name);
};
$.connection.hub.start().done(function () { });
</script>
Finally, add a SignalR Hub to the solution and use this code for the SessionDataHub implementation:
[HubName("sessiondata")]
public class SessionDataHub : Hub
{
private ObservableCollection<string> sessions = new ObservableCollection<string>();
public SessionDataHub()
{
sessions.CollectionChanged += sessions_CollectionChanged;
}
private void sessions_CollectionChanged(object sender, NotifyCollectionChangedEventArgs e)
{
if (e.Action == NotifyCollectionChangedAction.Add)
{
Clients.All.someOneJoined(e.NewItems.Cast<string>().First());
}
}
public override Task OnConnected()
{
return Task.Factory.StartNew(() =>
{
var youAre = Context.ConnectionId;
Clients.Caller.myNameIs(youAre);
sessions.Add(youAre);
});
}
public override Task OnDisconnected(bool stopCalled)
{
// TODO: implement this as well.
return base.OnDisconnected(stopCalled);
}
}
For more information about SignalR, go to http://asp.net/signalr
Link to source code: https://lsscloud.blob.core.windows.net/downloads/WebApplication1.zip

using EWS Managed API with ASP.NET MVC

I'm trying to create a web app which does many things but the one that I'm currently focused in is the inbox count. I want to use EWS StreamSubscription so that I can get notification for each event and returns the total count of items in the inbox. How can I use this in terms of MVC? I did find some code from Microsoft tutorial that I was gonna test, but I just couldn't figure how I could use it in MVC world i.e. What's the model going to be, if model is the count then how does it get notified every time an event occurs in Exchange Server, etc.
Here's the code I downloaded from Microsoft, but just couldn't understand how I can convert the count to json and push it to client as soon as a new change event occurs. NOTE: This code is unchanged, so it doesn't return count, yet.
using System;
using System.Linq;
using System.Net;
using System.Threading;
using Microsoft.Exchange.WebServices.Data;
namespace StreamingNotificationsSample
{
internal class Program
{
private static AutoResetEvent _Signal;
private static ExchangeService _ExchangeService;
private static string _SynchronizationState;
private static Thread _BackroundSyncThread;
private static StreamingSubscriptionConnection CreateStreamingSubscription(ExchangeService service,
StreamingSubscription subscription)
{
var connection = new StreamingSubscriptionConnection(service, 30);
connection.AddSubscription(subscription);
connection.OnNotificationEvent += OnNotificationEvent;
connection.OnSubscriptionError += OnSubscriptionError;
connection.OnDisconnect += OnDisconnect;
connection.Open();
return connection;
}
private static void SynchronizeChangesPeriodically()
{
while (true)
{
try
{
// Get all changes from the server and process them according to the business
// rules.
SynchronizeChanges(new FolderId(WellKnownFolderName.Inbox));
}
catch (Exception ex)
{
Console.WriteLine("Failed to synchronize items. Error: {0}", ex);
}
// Since the SyncFolderItems operation is a
// rather expensive operation, only do this every 10 minutes
Thread.Sleep(TimeSpan.FromMinutes(10));
}
}
public static void SynchronizeChanges(FolderId folderId)
{
bool moreChangesAvailable;
do
{
Console.WriteLine("Synchronizing changes...");
// Get all changes since the last call. The synchronization cookie is stored in the _SynchronizationState field.
// Only the the ids are requested. Additional properties should be fetched via GetItem calls.
var changes = _ExchangeService.SyncFolderItems(folderId, PropertySet.IdOnly, null, 512,
SyncFolderItemsScope.NormalItems, _SynchronizationState);
// Update the synchronization cookie
_SynchronizationState = changes.SyncState;
// Process all changes
foreach (var itemChange in changes)
{
// This example just prints the ChangeType and ItemId to the console
// LOB application would apply business rules to each item.
Console.Out.WriteLine("ChangeType = {0}", itemChange.ChangeType);
Console.Out.WriteLine("ChangeType = {0}", itemChange.ItemId);
}
// If more changes are available, issue additional SyncFolderItems requests.
moreChangesAvailable = changes.MoreChangesAvailable;
} while (moreChangesAvailable);
}
public static void Main(string[] args)
{
// Create new exchange service binding
// Important point: Specify Exchange 2010 with SP1 as the requested version.
_ExchangeService = new ExchangeService(ExchangeVersion.Exchange2010_SP1)
{
Credentials = new NetworkCredential("user", "password"),
Url = new Uri("URL to the Exchange Web Services")
};
// Process all items in the folder on a background-thread.
// A real-world LOB application would retrieve the last synchronization state first
// and write it to the _SynchronizationState field.
_BackroundSyncThread = new Thread(SynchronizeChangesPeriodically);
_BackroundSyncThread.Start();
// Create a new subscription
var subscription = _ExchangeService.SubscribeToStreamingNotifications(new FolderId[] {WellKnownFolderName.Inbox},
EventType.NewMail);
// Create new streaming notification conection
var connection = CreateStreamingSubscription(_ExchangeService, subscription);
Console.Out.WriteLine("Subscription created.");
_Signal = new AutoResetEvent(false);
// Wait for the application to exit
_Signal.WaitOne();
// Finally, unsubscribe from the Exchange server
subscription.Unsubscribe();
// Close the connection
connection.Close();
}
private static void OnDisconnect(object sender, SubscriptionErrorEventArgs args)
{
// Cast the sender as a StreamingSubscriptionConnection object.
var connection = (StreamingSubscriptionConnection) sender;
// Ask the user if they want to reconnect or close the subscription.
Console.WriteLine("The connection has been aborted; probably because it timed out.");
Console.WriteLine("Do you want to reconnect to the subscription? Y/N");
while (true)
{
var keyInfo = Console.ReadKey(true);
{
switch (keyInfo.Key)
{
case ConsoleKey.Y:
// Reconnect the connection
connection.Open();
Console.WriteLine("Connection has been reopened.");
break;
case ConsoleKey.N:
// Signal the main thread to exit.
Console.WriteLine("Terminating.");
_Signal.Set();
break;
}
}
}
}
private static void OnNotificationEvent(object sender, NotificationEventArgs args)
{
// Extract the item ids for all NewMail Events in the list.
var newMails = from e in args.Events.OfType<ItemEvent>()
where e.EventType == EventType.NewMail
select e.ItemId;
// Note: For the sake of simplicity, error handling is ommited here.
// Just assume everything went fine
var response = _ExchangeService.BindToItems(newMails,
new PropertySet(BasePropertySet.IdOnly, ItemSchema.DateTimeReceived,
ItemSchema.Subject));
var items = response.Select(itemResponse => itemResponse.Item);
foreach (var item in items)
{
Console.Out.WriteLine("A new mail has been created. Received on {0}", item.DateTimeReceived);
Console.Out.WriteLine("Subject: {0}", item.Subject);
}
}
private static void OnSubscriptionError(object sender, SubscriptionErrorEventArgs args)
{
// Handle error conditions.
var e = args.Exception;
Console.Out.WriteLine("The following error occured:");
Console.Out.WriteLine(e.ToString());
Console.Out.WriteLine();
}
}
}
I just want to understand the basic concept as in what can be model, and where can I use other functions.

Your problem is that you are confusing a service (EWS) with your applications model. They are two different things. Your model is entirely in your control, and you can do whatever you want with it. EWS is outside of your control, and is merely a service you call to get data.
In your controller, you call the EWS service and get the count. Then you populate your model with that count, then in your view, you render that model property. It's really that simple.
A web page has no state. It doesn't get notified when things change. You just reload the page and get whatever the current state is (ie, whatever the current count is).
In more advanced applications, like Single Page Apps, with Ajax, you might periodically query the service in the background. Or, you might have a special notification service that uses something like SignalR to notify your SPA of a change, but these concepts are far more advanced than you currently are. You should probably develop your app as a simple stateless app first, then improve it to add ajax functionality or what not once you have a better grasp of things.

That's a very broad question without a clear-cut answer. Your model could certainly have a "Count" property that you could update. The sample code you found would likely be used by your controller.

Testing with MemoryCache UpdateCallback

The following test fails intermittently. It caches an item in MemoryCache with an absolute expiration time, and an update callback that should be called before the item is removed. However sometimes the callback is invoked before the test finishes, and sometimes not at all.
With a large enough buffer time it will always be invoked at least once. But that does not serve my purposes, since I require that the cache always attempts to update the data before it expired.
Now in my real world scenario I will not have 10 second expiration time and granularity, but it still bothers me that this test fails intermittently.
Anyone have thoughts on why this is happening?
Note: Also intermittently fails with 60 second expiry and 5 second buffer.
using System;
using System.Runtime.Caching;
using System.Threading.Tasks;
using Microsoft.VisualStudio.TestTools.UnitTesting;
[TestClass]
public class MemoryCacheTest
{
private const double ExpiryInSeconds = 10;
private const double ExpiryBufferInSeconds = 5;
private readonly object updateItemCounterLock = new object();
private int updateItemCounter = 0;
[TestMethod]
public async Task MemoryCacheUpdateTest()
{
// Set item in cache with absolute expiration defined above
MemoryCache cache = MemoryCache.Default;
CacheItem cacheItem = new CacheItem("key", "value");
CacheItemPolicy cacheItemPolicy = new CacheItemPolicy
{
AbsoluteExpiration = DateTimeOffset.Now + TimeSpan.FromSeconds(ExpiryInSeconds),
UpdateCallback = new CacheEntryUpdateCallback(this.UpdateItem)
};
cache.Set(cacheItem, cacheItemPolicy);
// Delay for absolute expiration time + buffer
await Task.Delay(TimeSpan.FromSeconds(ExpiryInSeconds) + TimeSpan.FromSeconds(ExpiryBufferInSeconds));
// Test that the update callback was invoked once
Assert.AreEqual(1, updateItemCounter);
}
// Incrememnts the updateItemCounter
private void UpdateItem(CacheEntryUpdateArguments args)
{
lock (updateItemCounterLock)
{
updateItemCounter++;
}
}
}

I suppose calling new CacheEntryUpdateCallback is redundant. You can call:
UpdateCallback = new CacheEntryUpdateCallback(this.UpdateItem) instead

Since there was no solution to this question, I abstracted the MemoryCache methods that I needed into an interface and tested against that. At that point the test became invalid because I would have just been testing my own implementation of the interface.

how to support big size content in asp.net mvc

Often I have to create csv export files from the db in my apps. When I have to create 15 mbytes export files I have problems with mvc, because to use the return Content(); , I need to create in memory big strings with all the data, with frequent memory exception, so I have to use .aspx pages where I can user Response.Write while I'm processing the data.
The second problem trying to use mvc is that if I have to do long processing, the client see nothing come until I haven't completed the elaboration and starting to send it, and often thinks that is having connection or server problem.
Is it possible to send data to content during the processing of the action in mvc, without to end the completing of the processing?

Here is a piece of code that will address this issue. Note though that you can use Web API and just create a CSV formatter (and disable output buffering) or use pushtreamcontent directly.
For MVC here is a sample code, note this sample is using a closure, but you can just the same use an IEnumerable. The key is to make the evaluation lazy, so you don't create the whole string in memory.
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading;
using System.Web;
using System.Web.Mvc;
namespace SampleApplication
{
public class HomeController : Controller
{
public ActionResult LargeCsv()
{
// TODO: Replace the func with real data access, or alternatively use an IQueryable/IEnumerable/IEnumerator implementation
// to access the data dynamically.
int i = 0;
Func<string> func = () =>
{
while (i < 100)
{
i++;
return "Name" + i + ", " + i;
}
return null;
};
return new CsvActionResult(func);
}
}
public class CsvActionResult : ActionResult
{
private readonly Func<string> _getNextCsvLine;
public CsvActionResult(Func<string> getNextCsvLine)
{
_getNextCsvLine = getNextCsvLine;
}
public override void ExecuteResult(ControllerContext context)
{
context.HttpContext.Response.Headers.Add("Content-Type", "text/csv");
context.HttpContext.Response.BufferOutput = false;
// StreamWriter has inherent buffering so this operation is reasonably performant, it
// is going to write buffers to the wire rather than each writeline.
using (StreamWriter sw = new StreamWriter(context.HttpContext.Response.OutputStream))
{
string csvLine;
do
{
csvLine = _getNextCsvLine();
sw.WriteLine(csvLine);
} while (csvLine != null);
}
}
}
}

I think you have a problem with your application logic. I mean look into state machines to make your application to get into different states. So when processing the data happens, spawn a task running and will notify when a specific task is finished.
You can use JS to ask from time to time the state on the server and only when the task is finished, you can send it.
Also, make sure you use compression at the end when you send big data of results (and this may help your memory consumption also).