I am implementing a windows service which will be subscribing to an external data source. Whenever there is some update in the external data source it will activate an eventhandler lets say OnDataReceived method. The number of updates received from the data source could be huge at a particular time of day. For example, in evening, within 30-40 mins I could get ~50K event handler calls. This event handler gets the data as XML and does some processing onxml before saving it to Database.
I am trying to figure out if so many eventhandler call in small time window will overwhelm my service by creating too many threads or the .Net framework will manage the number of threads running in parallel. If this can cause issues, so I am planning to use ThreadPool to put the request on a thread in the EventHandler thus reducing the time request spends in EventHandler method code.
So looking out for suggestions from fellow coders if I need to worry about thread management at all in my scenario. If yes, do you see any problems with the approach I am planning to take or if you can suggest some better approach to handle it.
Would appreciate any help on this.
Related
I need to implement the following requirement for my job.
When a user starts a new application, a 5-minute timer begins.
If the user makes any edits to the application before the 5 minutes is up, the timer is canceled.
If the timer runs to completion, an email is sent to our company ("an application was created but abandoned").
The web server for this project is a .NET MVC project, though other than the Home Controller, all controllers inherit from System.Web.Http.ApiController rather than System.Web.Mvc.Controller. The front end is Angular 6.
It seems easy enough to start a 5-minute timer that will execute the "email send" method after 5 minutes. I'm stuck on how to implement the ability to cancel the timer if the user edits the application before the timer has run out. The command to start the application and any subsequent edits will come as separate queries to the API, so I don't have any state maintained from call to call.
My current idea is to create the timer via System.Timers.Timer when the application is started and store the timer in an ObjectCache under a unique ID representing that particular application. Then when the edit action is called, I can check the cache to see if a timer is stored that matches the application being edited, and if so, cancel the timer. If such a call doesn't come within 5 minutes, the timer will fire and the email be sent.
Will this work? (Both being able to access the timer to cancel it, and the timer firing as expected if not canceled?) Is there a better or more .NET-appropriate way to implement this requirement? Apologies for the vague scope of this question; I've had no luck with Google or searching SO, though my unfamiliarity with working with timers might be hindering my searches.
Thank you!
The root of your problem is architectural. You should probably give more thought to how your server-side is designed and how the client-side design and the server-side designs compliment one another. For starters, persistent state, the ability to run some background tasks, and the utilization of locking functionality (such as C#'s lock keyword) when accessing that persistent state would help in producing a more extensible and flexible design. How you design those features and how your client-side interacts with is up to you. One approach would be to have the API controller write to the persistent state, using locking to prevent concurrent writing, and then using a background task to monitor that persistent state and fire certain actions when necessary. Play around with designs and figure out what works for your needs. Good luck with your application.
Currently I am about to develop logging for a c# application into a SQL server table.
I have a designated class called Logger that has a static method writeToLog().
I just want to call that static function without block the calling thread.
How is this possible in C# clean and fast?
The functions don't return anything they are just fire and forget.
Thanks for you advice
There is the chance that the serveral available logging libraries out there have spent some thoughts about performance.
If you still need to develop a light weight solution yourself I think of two ways.
1.) Create a Task that runs the logging function, without awaiting them
2.) The Log-Method saves the Log-Information in a queue that is written to the SQL-database with a background thread.
I would recommend the second way, because the logging itself can be done in a synchronous function call without the Task creation overhead.
Another argument for this approach is that it is possible to gurantee the order of log messages. By using a Task for each single message the order of execution is not defined.
And a last argument: It might be more performant to write blocks of messages to the SQL table. By using the queue you will be able write messages in a bulk operation.
Let me briefly explain my problem. I want to read data from a sensor with my c#
solution. In order to do that I use an event, which pulls the data at a very fast rate from the sensor. The data is then saved to a database in sql.
To achieve that and maximize performance, I registered the event in Task A, which now frequently polls the Data from my sensor, lets say 1000 samples per second to give you an idea.
The data is saved in a blocking collection object with a queue.
What i want to do now is to stark a second task, which saves my data to an sql database, but only if there are more than 5000 samples in my blocking collection queue. How do i achieve that?
I tried to start the second task in my event which runs on the first task, but came over a few problems.
A) Sometimes the second task would not start in my event
B) I got an exception that the second task is still runnig (Because the event triggered so fast and it was not done i guess)
Is there any good way to do this?
Best regards
I've a pretty simple question/issue. I want to use 0MQ for some pretty basic Pub/sub functionality. My subscriber app is a windows GUI based app using plain winforms.
As there seems to be no explicit reference in 0MQ for handling this scenario, I am assuming that worst case I'd have to use a BeginInvoke(...) on the windows GUI thread once the 0MQ thread has recieved any subscription message. This seems pretty straightforward but if anyone has any insight/opinion/heads up in terms of the best way to handle this I'd very much appreciate it.
Thanks in advance
David
For your ZeroMQ subscriber in a WinForms application you have at least a few options:
Use a background worker thread to receive your 0MQ message. When you receive a message pass it to your background worker progressChanged event to update your UI.
Use a system thread to receive the message and invoke to update your UI.
Another suggestion on SO also suggest using a system thread to queue the message upon receive and a timer event to dequeue the message. If you used a Forms Timer you could update the UI without invoking or if System Timer you would need an invoke.
Which method you choose depends on your design criteria.
As pointed out, there are several ways to hook up ZeroMq into a WinForms app. It really does not differ much from using ZeroMq and C# in other settings, like in a console application. The main difference is as you point out that you have to get the messages into the Gui thread in order to update your Gui controls.
An example of using a separate thread + queues (producer/consumer pattern) can be found here: Examples of zeromq pub/sub with C# winform
Another way could be to use (again) a separate thread to do the ZeroMq work and set up an event that gets fired each time a message is received. The Gui could then hook up an event handler and process the message. You would of course also have to invoke any updates to the Gui. A drawback to this approach is that the event processing will interfere with the ZeroMq handler thread a bit, especially if the Gui updates takes a while, and if you receive lots of data.
All solutions have their pros and cons, it all depends on how your particular problem domain looks. All-in-all, integrating ZeroMq into a C# application, be it WinForms or other, is IMO pretty straightforward. The clrzmq wrapper is well-written and easy to use. The problems you may run into has more to do with the limited error feedback in ZeroMq itself. If you encounter strange crashes; try to run the same logic in a console application, that will give you better error messages.
First of all, I wanted to thank the community. You've been of great support lately ! Usually i don't even need to ask the questions because they're already there. Now i have an issue that's not directly related to code but programming itself.
I'm working with a FTDI Chip and C# programming a communication protocol in which a PC application acts like the Master (will send requests) and there is also Slave device who will answer to them, not immediately, maybe a couple of millisecs, but anyway, will take some time. I'm stuck in a conceptual/philosophical code design question.
After sending a request, should I ask right away for an answer (checking also a timeout) or should I constantly monitor the input (BackgroundWorker powered) and raise an event after receiving a data input ? What would you recommend, what is on your experience. What factors should i consider for making my choice ?
I never studied software design of programming itself so i think i lack the basic on this, but this is a personal project i'm working on and sure i'd love some feedback/pointers on this from you guys.
Thanks !
My preferred solution in this scenario would be to issue the request in async mode (such that you get called back by an event that fires when it completes), and also implement an async time out using standard .Net mechanisms, which calls you back if it appears the slave is unresponsive. This way you just start the request and the timer and then can continue doing more work, without needing any other threads to process results.
You would have to make sure that concurrent time out and response arrival is handled cleanly using a locking mechanism, so that you know for sure whether you are timing out or handling the response.
Try to avoid polling and input monitoring, unless your slave's API does not allow for deterministic generation of response events.
Normally i would work with the asynchronous approach on the low level site and maybe put some synchronization mechanism on top of this. Here is some example approach if you get data fragments and you have to put these fragments together to a whole message.
So on the low level site implement a BackgroundWorker that checks constantly for incoming data and raise some kind of event if you got something and put this into the event.
Above this is someone listening to the events of incoming data and puts all this (maybe) fragments into an internal queue. There it checks if it already has a enough data for a complete message, maybe does some error checking, etc. If it has a complete message it will raise an event to send this message to all listeners out there.
On top of this put another class that watches for messages and reacts on them. This class maybe implements some sync mechanism to watch out if an incoming message matches to something that should happened beforehand.
I think this design makes it easier to react on data that comes in when you don't expect something. And when you like to shutdown you don't have to wait for any timeouts to happen (maybe a very small one the low level BackgroundWorker is using to pull the data out of the source that doesn't support an event mechanism).