public interface IEventDismiss
{
[OperationContract]
[return:MessageParameter(Name="response")]
[XmlSerializerFormat]
Response ProcessRequest(Request request);
}
Hello,
Above is my WCF implementation in C# and it is pretty straight forward. However, it becomes a little more complicated when I receive the request and pass it on to another thread to process to produce the response and finally send this response back.
My algorithm is:
Get the request.
Pass it on to a separate thread to process by putting onto a static queue for other thread.
Once thread finish processing, it put the response object onto a static queue.
In my function ProcessRequest I have a while loop that dequeue this response and send it back to the requester.
public Response ProcessRequest (Request request)
{
bool sWait = true;
Response sRes = new Response();
ResponseProcessor.eventIDQueue.Enqueue(request.EventID);
while (sWait)
{
if (ResponseProcessor.repQ.Count > 0)
{
sRes = ResponseProcessor.repQ.Dequeue();
sWait = false;
}
}
return sRes;
}
Now, before everyone start to grill me, I am too realized this is bad practice and that's why I ask the question here in hoping to get better way to do this. I realized with the current code I have the following issues:
My while loops maybe in a continue loop and thus eating up the CPU if it has no sleep() in between.
My response queue may contains the wrong response back due to the nature of async call.
So I have two questions:
Is there a way to put sleep in the while loop to eliminate the high CPU usage?
Is there a better way to do this?
There's not point in doing this in the first place. Rather than having the current thread sitting around doing nothing while it waits for another queue to compute the work (while eating up tons of CPU cycles anyway), just compute the response in the current thread and send it back. You are gaining nothing by queuing it for another thread to handle.
You are also using queue objects that cannot be safely accessed from multiple threads, so in addition to being extremely inefficient, it's also subject to race conditions that can mean it won't even work.
Related
I'm getting a little lost between processes, threads etc.
Essentially, I'm just trying to find a standard way of doing something on an incoming request, that might take 5-300 seconds without the client having to wait that long for a response.
Given the code below, can any of the numbered ways of calling the long running method be swallowed?
As in, can this ever happen?
Enter endpoint (second 0)
Start long running task
Return response (second 2)
At second 2? 5? 30? this... instance/thread/call/execution environment just shuts down because its "done". I'm "done" because my response is sent.
OR is the long running task itself keeping the... instance/thread/call alive? Or is an api just always alive?
public async Task<ActionResult<int>> Get()
{
// second 0
//...
var threadId = Thread.CurrentThread.ManagedThreadId;
DoSomething60sAsync(); // #1 Because this call is not awaited, execution of the current method continues before the call is completed. Consider applying the 'await' operator to the result of the call.
_ = DoSomething60sAsync(); // #2 No Warning
Task.Run(DoSomething60sAsync); // #3 Same Warning as #1
_ = Task.Run(DoSomething60sAsync); // # 4 No Warning
return Ok(threadId); // second 2
}
private async Task DoSomething60sAsync()
{
for (int i = 0; i < 60; i++)
{
var threadId = Thread.CurrentThread.ManagedThreadId;
Console.WriteLine(threadId + ": Second " + i);
await Task.Delay(1000);
}
}
As you can see, I tried to look at the threads. I noticed they're ALL background threads. I also noticed that the threadId seems completely random. It even changes within that for loop.
I read that aspnetcore threading is complicated.
Given that they're all background threads anyway, using Task.Run in #3/#4 seems pointless.
Another way of solving this problem I read about is implementing a IHostedService and some kind of TaskRunner-BackgroundQueue. https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-5.0&tabs=visual-studio
Also there is Hangfire.
I tried all of these and they all seem to work. So I would like to use the simplest solution, but I'm worried that just "calling tasks without await" will be lost somehow. Is this a valid concern? Can this ever happen?
Essentially, I'm just trying to find a standard way of doing something on an incoming request, that might take 5-300 seconds without the client having to wait that long for a response.
There are a few names for this. I've called it "basic distributed architecture", but "asynchronous messaging" is also common. The core idea is that you need two things:
A durable queue.
A backend process.
At second 2? 5? 30? this... instance/thread/call/execution environment just shuts down because its "done". I'm "done" because my response is sent.
OR is the long running task itself keeping the... instance/thread/call alive? Or is an api just always alive?
Loss of work is certainly possible. Shutdowns are normal, and when the hosting process is shut down, work can be lost.
Given that they're all background threads anyway, using Task.Run in #3/#4 seems pointless.
On ASP.NET Core, yes, that is true. Also, the discard (_ = ) is literally just telling the compiler "I know what I'm doing; don't give me a warning", so you're not getting a warning on those lines because the warning is silenced - the code is still just as dangerous (i.e., you'll still lose work).
Another way of solving this problem I read about is implementing a IHostedService and some kind of TaskRunner-BackgroundQueue.
To prevent loss of work, your queue has to be durable (saved on disk).
Also there is Hangfire.
Hangfire (or an IHostedService connected to a durable queue) is one way to implement asynchronous messaging. I tend to prefer background services in separate processes for reasons described on my blog, but a background service inside the ASP.NET Core process would also work.
Background
We have a service operation that can receive concurrent asynchronous requests and must process those requests one at a time.
In the following example, the UploadAndImport(...) method receives concurrent requests on multiple threads, but its calls to the ImportFile(...) method must happen one at a time.
Layperson Description
Imagine a warehouse with many workers (multiple threads). People (clients) can send the warehouse many packages (requests) at the same time (concurrently). When a package comes in a worker takes responsibility for it from start to finish, and the person who dropped off the package can leave (fire-and-forget). The workers' job is to put each package down a small chute, and only one worker can put a package down a chute at a time, otherwise chaos ensues. If the person who dropped off the package checks in later (polling endpoint), the warehouse should be able to report on whether the package went down the chute or not.
Question
The question then is how to write a service operation that...
can receive concurrent client requests,
receives and processes those requests on multiple threads,
processes requests on the same thread that received the request,
processes requests one at a time,
is a one way fire-and-forget operation, and
has a separate polling endpoint that reports on request completion.
We've tried the following and are wondering two things:
Are there any race conditions that we have not considered?
Is there a more canonical way to code this scenario in C#.NET with a service oriented architecture (we happen to be using WCF)?
Example: What We Have Tried?
This is the service code that we have tried. It works though it feels like somewhat of a hack or kludge.
static ImportFileInfo _inProgressRequest = null;
static readonly ConcurrentDictionary<Guid, ImportFileInfo> WaitingRequests =
new ConcurrentDictionary<Guid, ImportFileInfo>();
public void UploadAndImport(ImportFileInfo request)
{
// Receive the incoming request
WaitingRequests.TryAdd(request.OperationId, request);
while (null != Interlocked.CompareExchange(ref _inProgressRequest, request, null))
{
// Wait for any previous processing to complete
Thread.Sleep(500);
}
// Process the incoming request
ImportFile(request);
Interlocked.Exchange(ref _inProgressRequest, null);
WaitingRequests.TryRemove(request.OperationId, out _);
}
public bool UploadAndImportIsComplete(Guid operationId) =>
!WaitingRequests.ContainsKey(operationId);
This is example client code.
private static async Task UploadFile(FileInfo fileInfo, ImportFileInfo importFileInfo)
{
using (var proxy = new Proxy())
using (var stream = new FileStream(fileInfo.FullName, FileMode.Open, FileAccess.Read))
{
importFileInfo.FileByteStream = stream;
proxy.UploadAndImport(importFileInfo);
}
await Task.Run(() => Poller.Poll(timeoutSeconds: 90, intervalSeconds: 1, func: () =>
{
using (var proxy = new Proxy())
{
return proxy.UploadAndImportIsComplete(importFileInfo.OperationId);
}
}));
}
It's hard to write a minimum viable example of this in a Fiddle, but here is a start that give a sense and that compiles.
As before, the above seems like a hack/kludge, and we are asking both about potential pitfalls in its approach and for alternative patterns that are more appropriate/canonical.
Simple solution using Producer-Consumer pattern to pipe requests in case of thread count restrictions.
You still have to implement a simple progress reporter or event. I suggest to replace the expensive polling approach with an asynchronous communication which is offered by Microsoft's SignalR library. It uses WebSocket to enable async behavior. The client and server can register their callbacks on a hub. Using RPC the client can now invoke server side methods and vice versa. You would post progress to the client by using the hub (client side). In my experience SignalR is very simple to use and very good documented. It has a library for all famous server side languages (e.g. Java).
Polling in my understanding is the totally opposite of fire-and-forget. You can't forget, because you have to check something based on an interval. Event based communication, like SignalR, is fire-and-forget since you fire and will get a reminder (cause you forgot). The "event side" will invoke your callback instead of you waiting to do it yourself!
Requirement 5 is ignored since I didn't get any reason. Waiting for a thread to complete would eliminate the fire and forget character.
private BlockingCollection<ImportFileInfo> requestQueue = new BlockingCollection<ImportFileInfo>();
private bool isServiceEnabled;
private readonly int maxNumberOfThreads = 8;
private Semaphore semaphore = new Semaphore(numberOfThreads);
private readonly object syncLock = new object();
public void UploadAndImport(ImportFileInfo request)
{
// Start the request handler background loop
if (!this.isServiceEnabled)
{
this.requestQueue?.Dispose();
this.requestQueue = new BlockingCollection<ImportFileInfo>();
// Fire and forget (requirement 4)
Task.Run(() => HandleRequests());
this.isServiceEnabled = true;
}
// Cache multiple incoming client requests (requirement 1) (and enable throttling)
this.requestQueue.Add(request);
}
private void HandleRequests()
{
while (!this.requestQueue.IsCompleted)
{
// Wait while thread limit is exceeded (some throttling)
this.semaphore.WaitOne();
// Process the incoming requests in a dedicated thread (requirement 2) until the BlockingCollection is marked completed.
Task.Run(() => ProcessRequest());
}
// Reset the request handler after BlockingCollection was marked completed
this.isServiceEnabled = false;
this.requestQueue.Dispose();
}
private void ProcessRequest()
{
ImportFileInfo request = this.requestQueue.Take();
UploadFile(request);
// You updated your question saying the method "ImportFile()" requires synchronization.
// This a bottleneck and will significantly drop performance, when this method is long running.
lock (this.syncLock)
{
ImportFile(request);
}
this.semaphore.Release();
}
Remarks:
BlockingCollection is a IDisposable
TODO: You have to "close" the BlockingCollection by marking it completed:
"BlockingCollection.CompleteAdding()" or it will loop indeterminately waiting for further requests. Maybe you introduce a additional request methods for the client to cancel and/ or to update the process and to mark adding to the BlockingCollection as completed. Or a timer that waits an idle time before marking it as completed. Or make your request handler thread block or spin.
Replace Take() and Add(...) with TryTake(...) and TryAdd(...) if you want cancellation support
Code is not tested
Your "ImportFile()" method is a bottleneck in your multi threading environment. I suggest to make it thread safe. In case of I/O that requires synchronization, I would cache the data in a BlockingCollection and then write them to I/O one by one.
The problem is that your total bandwidth is very small-- only one job can run at a time-- and you want to handle parallel requests. That means that queue time could vary wildly. It may not be the best choice to implement your job queue in-memory, as it would make your system much more brittle, and more difficult to scale out when your business grows.
A traditional, scaleable way to architect this would be:
An HTTP service to accept requests, load balanced/redundant, with no session state.
A SQL Server database to persist the requests in a queue, returning a persistent unique job ID.
A Windows service to process the queue, one job at a time, and mark jobs as complete. The worker process for the service would probably be single-threaded.
This solution requires you to choose a web server. A common choice is IIS running ASP.NET. On that platform, each request is guaranteed to be handled in a single-threaded manner (i.e. you don't need to worry about race conditions too much), but due to a feature called thread agility the request might end with a different thread, but in the original synchronization context, which means you will probably never notice unless you are debugging and inspecting thread IDs.
Given the constraints context of our system, this is the implementation we ended up using:
static ImportFileInfo _importInProgressItem = null;
static readonly ConcurrentQueue<ImportFileInfo> ImportQueue =
new ConcurrentQueue<ImportFileInfo>();
public void UploadAndImport(ImportFileInfo request) {
UploadFile(request);
ImportFileSynchronized(request);
}
// Synchronize the file import,
// because the database allows a user to perform only one write at a time.
private void ImportFileSynchronized(ImportFileInfo request) {
ImportQueue.Enqueue(request);
do {
ImportQueue.TryPeek(out var next);
if (null != Interlocked.CompareExchange(ref _importInProgressItem, next, null)) {
// Queue processing is already under way in another thread.
return;
}
ImportFile(next);
ImportQueue.TryDequeue(out _);
Interlocked.Exchange(ref _importInProgressItem, null);
}
while (ImportQueue.Any());
}
public bool UploadAndImportIsComplete(Guid operationId) =>
ImportQueue.All(waiting => waiting.OperationId != operationId);
This solution works well for the loads we are expecting. That load involves a maximum of about 15-20 concurrent PDF file uploads. The batch of up to 15-20 files tends to arrive all at once and then to go quiet for several hours until the next batch arrives.
Criticism and feedback is most welcome.
I'm getting confused by await/async as I may still not get the point of its correct usage.
I have a simple WPF-UI and a ViewModel-method to start listening for clients which want to connect.
The following method is executed when the user clicks the button to start listening:
public void StartListening()
{
_tcpListener.Start(); // TcpListener
IsListening = true; // bool
Task.Factory.StartNew(DoStartListeningAsync, TaskCreationOptions.LongRunning);
}
The method DoStartListeningAsync which is called is defined like
private async Task DoStartListeningAsync()
{
while (IsListening)
{
using (var newClient = await _tcpListener.AcceptTcpClientAsync() /*.WithWaitCancellation(_cts.Token)*/)
{
apiClient = new ApiClient();
if(await apiClient.InitClientAsync()) // <-- here is the problem
{
// ... apiClient is now initialized
}
// ... do more and go back to await _tcpListener.AcceptTcpClientAsync()
}
}
}
The ApiClient class' InitClientAsync method is defined like:
public async Task<bool> InitClientAsync()
{
using (var requestStream = await _apiWebRequest.GetRequestStreamAsync())
{
_apiStreamWriter = new StreamWriter(requestStream);
}
// ... do somehing with the _apiStreamWriter
return true;
}
However, sometimes the InitClientAsync-call will get stuck at await _apiWebRequest.GetRequestStreamAsync() which then will freeze the execution of the DoStartListeningAsync-method at // <-- here is the problem.
In case the DoStartListeningAsync is stuck, no new connections will be handled which destroys my whole concept of handling multiple clients asynchronously.
Since you are using "await" keyword along the code path, you won't actually serve
multiple clients asynchronously.
The thing is, your code in the background thread will serve clients one by one. Take a deeper look - in the while loop you are getting request stream, wait till it is loaded, serve it, and then wait for other request stream.
async/await principle doesn't itself provide ability to serve multiple actions at the time. The only thing it is doing - prevents blocking current thread from being reusable by other code. So if going with async/await, you allow system yo use your current task thread, while it is waiting to other async action to complete (like _apiWebRequest.GetRequestStreamAsync()).
But since you are having one task, and you are waiting on every iteration of while loop - your code will work the same way, if you wrote it completely synchronous. The only profit is that you are using Task, and so .Net can reuse it's thread from thread pool while you are waiting for async actions to complete.
If you wan't to serve multiple clients asynchronously, you should either start multiple tasks, or don't wait till request is completely served - so actually remove some awaits from your code.
So you should move towards design, there you have one listening task/thread, that does nothing exept reading requests and putting it to the some queue. And having other tasks, that serve requests, reading it from the queue.
If I understood you correctly, you are using TcpListener under the hood. So what you need, is the loop where you accept new clients, and start serving them in the different thread/task without any waiting, so going directly to accepting other clients. But you can and should use async/await inside those handlers that serve clients.
Take a look at this answer - not completely your case (since I don't know all details of implementation), but just to get the idea.
I'm trying to write to Azure Table Storage asynchronously using BeginExecute but have been getting inconsistent results. When I change BeginExecute to Execute, then everything gets written properly, but I'm guessing I have something wrong in my threads that they are either cancelling each other or something depending on how fast the main thread sends the messages. Here's what I'm doing now:
TableOperation op = TableOperation.Insert(entity);
_table.BeginExecute(op, new AsyncCallback(onTableExecuteComplete), entity);
private void onTableExecuteComplete(IAsyncResult result)
{
TableResult tr = _table.EndExecute(result);
if ((tr.HttpStatusCode < 200) || (tr.HttpStatusCode > 202))
{
Console.WriteLine("Error writing to table.");
}
}
I'm testing it with a few entries, and I'll get one or two entries in the table, but not all of them. Any ideas on how to catch errors and make sure that all the entries are written properly?
Update: I found that when I put Thread.Sleep(5000); at the end of my main thread, everything finishes writing. Is there a way to pause the main thread before it ends to ensure all other threads have finished so they don't get cancelled before they're done?
Likely what is happening is that your main thread ends, and destroys all active child threads. When you are doing asynchronous programming, your main thread either needs to be running long enough to wait for completion (such as a service), or it needs to wait for asynchronous tasks to finish:
var result = _table.BeginExecute(op,
new AsyncCallback(onTableExecuteComplete), entity);
result.AsyncWaitHandle.WaitOne();
Source: http://msdn.microsoft.com/en-us/library/system.iasyncresult.aspx
This of course begs the question: if you are not needing to do anything else while you are waiting for the "asynchronous" task to complete, then you might as well do it synchronously to keep things simpler. The purpose of the asynchronous pattern is for threads that shouldn't be blocked while waiting for some other process to finish - at a cost of increased complexity.
I enjoyed the new C# 5's async and await and I want to set it up to one of my previous Tcp application which I used the async whay (not async & await, in fact Begin*, End*)
in my application every message have a response, so every time you use BeginSend, you will receive a message related the message you first sent. (lets suppose command and it's report)
I want to add a new function called RequestAsync that you send your message and wait till the response comes. let me show the usage
string response = await RequestAsync("hi");
on this occasion, you will send a "hi" message and then code waits till the response related to this comes.
I had problems using Task.Factory.FromAsync, so I tried the following code, i want to make sure the performance is roughly the same as TaskFactory way.
public async Task<IRequestResult> SendRequestAsync(IRequest request, string message)
{
IRequestResult result = BeginSendRequest(request, message);
while (!result.IsCompleted)
{
await Task.Delay(1);
}
return result;
}
and BeginRequest is my own method.
Sincerely yours,
Peyman Mortazavi
No, that's going to be pretty awful in terms of efficiency - you'll be waking up and requiring rescheduling every 5 milliseconds. You've added latency while you effectively sleep between cycles as well.
You should go back to trying to use Task.Factory.FromAsync - you haven't told us what went wrong when you tried that, so it's hard to help you fix it... It's possible that you haven't implemented the asynchronous programming model pattern correctly to start with. (The fact that you're not providing a callback to your BeginSendRequest method is suggestive of that.)
I hope your real code doesn't have catch {} either...
Poling can never be efficient as IO completion ports, and why even use an async method if you are only waiting for it and not doing anything in the mean time, it just ads needles overhead.