We are developing an application where in UI calls a Service(WCF) layer and a component that resides inside service layer calls external webservices asynchronously. For instance: UI sends says calls WCF layer by uploading a file, and if file has 1000 entries, currently we call external services asynchronously in a loop (BeginxxxMethod) 1000 times for response. Is this the right way of doing so? Second question: what is the maximum number of asynchronous connections that can be made? Our technology is ASP.NEt and C# 4.0.
Two suggestions:
If you control the web service API add another method that lets you pass all 1000 args and returns all results. This is chunky vs chatty so you only go through the cross process pain once.
If you do not control the web service API, come up with a wrapper that makes n number of remote calls synchonously, then call this wrapper asynchonously n times. Play around until you find the best balance between number of async calls and number of sequential remote calls per async call.
Is this the right way of doing so?
Async calls are often the best way to get large batch jobs done. I think you're looking good with this approach. As a developer, it's often our job to see where cutting new threads no longer optimizes response times. Myles mentioned using a wrapper around a service. I have often done this when calling 1000's of calls at a time... calling a few thousand async calls actually hurt performance (in my application), so I created functionality to group calls a few hundred (asynchronously) at a time until all x-thousand calls were finished. This gave me the best of both worlds... I was able to find the point number of async calls at once gave me the best performance and went from there.
As far as max threads, it depends on resources, but here is a link that explains the defaults... I'll post below for ease
1023 in Framework 4.0 (32-bit environment)
32768 in Framework 4.0 (64-bit environment)
250 per core in Framework 3.5
25 per core in Framework 2.0
WE have resolved this by implementing Task Parallel library.
Below is the pseudo code.
Read the file contents in to generic list.
Use Parallel for to process each request and set MaxDOP (Degree of parallelism) according to processor count in the server to process the request.
Thanks
Related
I am in need of some guidance on a project I am working on. We are looking for a replacement for CORBA server setup. In a nutshell we currently run a CORBA deamon service that hosts 10 instances of a C++ exe that is the entry point into our calculation process. The C++ code hooks into a bunch of different .net and C++ dlls and OCXs via COM. We also have another version of the executable that is compiled as a .dll that we are able to call in a similar fashion but it is only a single instance system so all is well there.
We are now looking to replace the CORBA components with a WebAPI so I have put together a basic ASP.net webAPI project that is able to process the requests into this C++ dll. Again, this works great when it only needs to handle 1 request at a time. Things start going sideways when I start testing concurrent requests. The request come into my handler just fine and I can see the 5 requests (I have logging everywhere tracking whats going on,) and each thread creates an instance of the dll but they are run synchronously.
What I have figured out is that even though there are multiple threads going in the ASP.net handler, the dll is STAThreaded (this is confirmed in the code) so the calls are queued up and only processed 1 at a time. My guess here is because the threads are all inside the same process the dll treats all the threads as the same apartment (STAThread) and causes the queue.
I have tried different async/await and task.run code and I can see different threads but it still comes down to the same process which makes the dll run synchronously. I did try change the dll to be MTA by changing the CoInitializeEx(NULL,0x2) to CoInitializeEx(NULL,0x0) but that didn't seem to change anything.
I am now running out of ideas and I don't think changing to use the .exe version and spawning multiple process is going to work because there is the CORBA stuff that allows a return object to be created and communicated back to the calling code. I need to be able to get the objects that are created in the exe to send back in the request.
Sorry for the long post, hopefully someone will take the time to read this wall of text and have some ideas of what I can try.
Thank you!
I would suggest that the WebAPI architecture is a poor solution to your problem. Typically you do not want to spawn long-running or blocking processes from ASP.NET, because it's quite easy to exhaust the threadpool and prevent the server from being able to handle new requests.
If you do want to continue having a WebAPI endpoint, I would start with taking the requests and putting them in a queue, and having the client poll or subscribe for the completed result.
You may be interested in looking a what they're doing with gRPC in dotnetcore 3.0 - if you want to keep that kind of architecture, but update the platform.
You can create multiple app domains. App domain is "It can be considered as a Lightweight process which is both a container and boundary" ref. Load your DLLs into that different domains. This way every app domain you create will load your COM DLLs separately. Create proxies using MarshalByRefObject as used here. Write an orchestrator that distributes requests to app domains and get the results from appdomains and send responses. Keeps tracks of which domain is busy which is not or create new domains for the request.
Also different methods mentioned in this link
I'm new to .Net WebApi, as I know it's preferable to use async APIs instead of sync ones, but what is the difference?
if the API is sync and has been called from a client, and another call from another client, as I checked, no interruption will happen, and both calls will go through simultaneously. So, what's the benefit of making it Async?
Update: as I understand, if the number of requests are huge, if I use async, the waiting time for some calls will be less, cause there are more threads available to run tasks(as some of them are released waiting for database call, or network call etc.) is it true?
I case of SYNC what happens is that for each request a thread is assigned exclusively and this thread is released only upon completion of particular request.
While in case of ASYNC the thread may be reused by other request.
So if your application is I/O Bound then you can see significant improvement in your application by using ASYNC, if your application is CPU Bound then ASYNC will not be that much useful.
https://en.wikipedia.org/wiki/I/O_bound
https://en.wikipedia.org/wiki/CPU-bound
First of all re-iterating the difference between sync and async.
{------------- Sync Task1-------------}{-------------- Sync Task 2------------}
{---------------------- async task 1 --------------------}
{---------------------- async task 2 --------------------}
I hope you got your answer by this point to why its beneficial. Imaging a situaion where your API serving list of 1000 basketball payers and their details while requests comings in for list of cities. I bet your client app would look neater if you get something while player list is been served wouldn't it?
Secondly, APIs don't prefer Aync as such. Its a choice of programming you have. If you utilise the full language and operating system capabilities, It's your application and the users going to get the benefit out of it.
CACHING,
Using Async does help caching if you are use new In memory Cache or custom server-level cache. Afterall your client is looking for 304 return and while a long request is been served a small requests can be served ie cache checks.
In my server application I want to process lots of coming from client tasks. Client application submits tasks for processing and processing each task requires calling a web service for pre-processing and then actual processing happens.
I was suggested having a queue into which I'll put all tasks that server receives. Then a thread picks up a task from a queue and calls a web service. There will be maybe 40 threads doing this if one blocks on web service, other can do calls as well, picking up items from queue. After a thread receives response from web service it puts pre-processed item on a second queue from which another thread takes tasks for processing. And there will be 1 thread for this queue (will be scaled further per processor - so probably 4 (or more) threads on a 4 core machine).
I believe this can be accomplished more efficiently without having 40 predefined threads doing web service calls and maybe having 1 queue. I think there are multiple options for doing this is .NET more efficiently. Any suggestions?
It's probably broader question how to implement better such a system rather then a .net specific.
I think you should learn about async/await construction awailable in .net 4.5. It is hard to say if it meets all your requirements but you should check it.
I recommend looking into TPL Dataflow, a library that allows you to define a "pipeline" or "mesh" for data processing and then you put the data through it. TPL Dataflow works very well with both asynchronous (e.g., web request) blocks and synchronous (e.g., processing) blocks and has lots of options for parallelism.
In case for some reason you are not up to version 4.5 of the framework look into one-way WCF calls as a kind of "fire and forget" method.
Async has become a buzzword in .net and MS have introduced it in Web API 2 so that more requests can be handled whilst others are waiting on IO to finish.
Whilst I can see the benefit of this, is it really a concern? A x64 architecture has 30000+ threads in the Thread Pool so unless you have that many concurrent users on your website is async really required? Even if you have that many concurrent users without caching I'm pretty sure SQL Server will fall over with that many requests?
Apart from it being shiny when is there a real need to have async routing on a web framework?
Many of the other answers here are coming from a UI (desktop/mobile app) perspective, not a web server perspective.
Async has become a buzzword in .net and MS have introduced it in Web API 2 so that more requests can be handled whilst others are waiting on IO to finish.
async and await were introduced in .NET 4.5 / VS 2012. However, ASP.NET has had asynchronous request capability since .NET 2.0 - a very long time ago. And there have been people using it.
What async and await bring to the table is asynchronous code that is easy to maintain.
Whilst I can see the benefit of this, is it really a concern?
The key benefit of async on the server is scalability. Simply put, async tasks scale far better than threads.
#Joshua's comment is key regarding the memory; a thread takes a significant amount of memory (and don't forget the kernel-mode stack which cannot be paged out), while an async request literally only takes a few hundred bytes.
There's also bursting to consider. The .NET threadpool has a limited injection rate, so unless you set your minWorkerThread count to a value much higher than you normally need, then when you get a burst of traffic some requests will 503 before .NET can spin up enough threads to handle them. async keeps your threads free (as much as possible) so it handles bursting traffic better.
A x64 architecture has 30000+ threads in the Thread Pool so unless you have that many concurrent users on your website is async really required?
#Joshua is again correct when he points out that you're probably thinking of a request queue limit (which defaults to 1000 for the IIS queue and 5000 for the ASP.NET request limit). It's important to note that once this queue is filled (during bursty traffic), new requests are rejected with 503.
Even if you have that many concurrent users without caching I'm pretty sure SQL Server will fall over with that many requests?
Ah, now that's another question entirely.
I'm giving a talk at ThatConference 2013 specifically on async servers. One part of that talk is situations where async doesn't help (my Twitter update).
There's an excellent blog post here that takes the position that asynchronous db calls are just not worth the effort. It's important to note the assumptions in this post:
At the time that post was written, asynchronous web servers were difficult. These days we have async and more and more libraries are offering asynchronous APIs (e.g., Entity Framework).
The architecture assumes a single web server with a single SQL Server backend. This was a very common setup traditionally, but is quickly changing today.
Where async servers really shine is when your backend can also scale. E.g., a web service, Azure SQL, NoSQL cluster, etc. Example: I'm writing an MVC/WebAPI server that uses Azure SQL and Storage for its backend (for all practical purposes, I can act like they have infinite scalability); in that case, I'm going to make my server async. In situations like this, you can scale your server 10x or more by using async.
But if you just have a single SQL Server backend (and have no plans to change to Azure SQL), then there's no point in making your web server async because you're limited by your backend anyway.
When long operations can be efficiently executed in parallel. For instance, you have to execute two SQLs and load three pictures - do all five operations as async and await them all. In this case the overall time will be the longest duration of five operations, but not the sum of the durations.
Pre-fetch. If you can predict (with good probability) what user will do (e.g. almost certainly, (s)he will want to see the details...) you may start preparing the next page (frame, window) while user's reading the previous.
where did you get 30000 from. i dont remember exactly but I think Asp.net uses 12 x number of cores threads.
I have to use async, when operation take too long time (upload, export, processing) and user have to know about progress.
You need async in following scenarios
1) When you are performing a very long operation and you don't want to freeze your UI.
2) When you designed some task that needs to be completed in background.
For example, You are rendering images from database. But you don't want your page to be freeze at that time async is really helpful.
I need to build a web application (MVC) that uses a third-party win32 dll as gateway for all business logic (contains logon-mechanism, functions and maintains some state).
This dll is not designed for multithreading. In an MTA scenario, the dll stumbles after a certain time.
The recommended solution is to run ASP.NET MVC in ASP-Classic Mode (STA using an Asp-CompatHandler). I tried this with success - everything runs stable.
The probblem is, there will be a lot concurrent users and some of the function calls takes some seconds (up to 10 secodns!). This will get horrible if all users block each other.
What would be the best approach to minimize the blocking-effects within the same application? Say only ten users should block each other?
It would be nice if:
...the web runs in MTA
...the web is just deployed once
...everything runs within the same process
Can anyone give me some advice for a good concept solving this?
Thank you! Martin
Update - Found a Solution:
Thanks to the "Smart Thread Pool" from Ami Bar I could accomplish the behavior I was looking for (easily). I implemented a worker concept (a specific amout of users share a worker and block each other in this worker), and for each worker, I have now my own thread pool instance with a max and min number of one thread. Well, it's not the idea of a thread pool, but it makes it very easy to handle the work-items and it also has some nice other features. The web application is running on MTA now.
I'm going to prepare some load tests to see if its stable over hours.
see here: http://www.codeproject.com/Articles/7933/Smart-Thread-Pool
The answer is very simple, although I don't think you will like much: you cannot use something that is not designed to be used in a multithreaded environment in a multithreaded environment.
2 possibilities:
Live with STA
Replace the single threaded COM object with something that is intended to be used in a web application
Unfortunately, if the dll is not designed to be used with parallel requests, you cannot use it with parallel requests.
The only solution I see for you to increase the number of concurrent users without having the application running parallely is to have multiple instances of the application running at the same time, with maybe a load balancer in front of them to dispatch the queries.