I have many Windows Services (written in C# 4.0) that at various intervals connect to a database and do various complex tasks. Some of these tasks only occur every X hour intervals per day. However, the server support team would like to know if the Windows Service is actually running as there can be a big interval between tasks. They would like essentially a heartbeat for each Windows Service. Every 5 minutes the Windows Service would do something that could be monitored by Microsoft System Center Operation Manager (SCOM). Any solution must being easily monitored by SCOM as the server support team rely on it.
What is the best way to achieve this type of funcationality? I thought of using Performance Counters and have SCOM listen for those but not sure if that is that best use of perf counters.
UPDATE (Totally forgot to include): Currently each service writes to a database this action but SCOM is not very good at monitoring databases records and differentiating which service is doing what etc.
If your case here is limited to service health check, I am pretty sure this can be done by windows tools such as windows performance monitor. Collecting their data should be an easier task than creating heartbeats in a usually disconnected environment.
There are plenty of server and service monitoring tools. Few of them are also open-source, which you can choose as an initial step.
What about simply writing to a DB or event log?
You could add either SNMP or WMI instrumentation to your services - this incidentally will allow you to raise more than just heartbeat messages - for example you could also raise events when you encounter transient faults that require an operation to be retried.
There are plenty of SNMP libraries out there and WMI support is built into the .NET framework.
Just FYI, I'm researching this question because how I do it is that the Service application logs a hearbeat message every X seconds when it polls the Db for a new job. At midnight, all heartbeat messages over a week old are deleted from the log. I do have an application scheduled at 7 AM that checks when the last heartbeat message was sent, but it looks like I'm going to have to run it constantly to periodically check ... and notify Solar Winds to send an alert if there is a problem.
Related
I am considering creating a Windows service that collects data over TCP/IP, in this case a hundred or so bytes of Modbus data from PLC type hardware. I currently do this inside the application on a thread and would like to separate it. The thread runs with an elevated priority and polls the hardware on a timed interval. It works well.
This is used in near real-time environments and it would be nice if multiple applications can run without having to duplicate the polling. Making the thread a Windows service seems like a possibility.
My idea is to have the application subscribe to the registers (array items) that it is interested in. When the data changes in a subscribed register the service will send a notification to the subscriber along with the current value.
I need the event communication to be as fast as possible so I’m not sure what tool to use with the service. I don’t want to do any polling at the application level.
If this has been asked before my apologies, and this is .NET 2.0 ASMX Web services, again my apologies =D
A .NET Application that only exposes web services. Roughly 10 million messages per day load balanced between multiple IIS Servers. Each incoming messages is XML, and an outgoing message is XML. (XMLElement) (we have beefy servers that run on steroids).
I have a SLA that all messages are processed in under X Seconds.
One function, Linking Methods, in the process is now taking 10-20 seconds, it is required for every transaction, however is not critical that it happens before the web service returns the results. Because of this I made a suggestion to throw it on another thread, but now realize that my words and the eager developers behind them might have not fully thought this through.
The below example shows on the left the current flow. On the right what is being attempted
Effectively what I'm looking for is to have a web service spawn a long running (10-20 second) thread that will execute even after the web service is completed.
This is what, effectively, is going on:
Thread linkThread= new Thread(delegate()
{
Linkmembers(GetContext(), ID1, ID2, SomeOtherThing, XMLOrSomething);
});
linkThread.Start();
Using this we've reduced the time from 19 seconds to 2.1 seconds on our dev boxes, which is quite substantial.
I am worried that with the amount of traffic we get, and if a vendor/outside party decides to throttle us, IIS might decide to recycle/kill those threads before they're done processing. I agree our solution might not be the "best" however we don't have the time to build in a Queue system or another Windows Service to handle this.
Is there a better way to do this? Any caveats that should be considered?
Thanks.
Apart from the issues you've described, I cannot think of any. That being said, there are ways to fix the problem that do not involve building your own solution from scratch.
Use MSMQ with WCF: Create a WCF service with an MSMQ endpoint that is IIS hosted (no need to use a windows service as long as WAS is enabled) and make calls to the service from within your ASMX service. You reap all the benefits of reliable queueing without having to build your own.
Plus, if your MSMQ service fails or throws an exception, it will reprocess automatically. If you use DTC and are hitting a database, you can even have the MSMQ transaction flow to the DB.
We have a number of different old school client-server C# WinForm client-side apps that are essentially front-ends for the database. Then there is a C# server-side windows service that waits on the client apps to submit orders and then it processes them.
The way the server-side service finds out whether there is work to do is that it polls the database. Over the years the logic of polling for waiting orders has gotten a lot more complicated due to the myriad of business rules. So because of this, the polling stored proc itself uses quite a bit of SQL Server resources even if there is nothing to do. Add to this the requirement that the orders be processed the moment they are submitted and you got yourself a performance problem, as the database is being polled constantly.
The setup actually works fine right now, but the load is about to go through the roof and, it is obvious, that it won't hold up.
What are some effective ways to communicate between a bunch of different client-side apps and a server-side windows service, that will be more future-proof than the current method?
The database server is SQL Server 2005. I can probably get the powers that be to pony up for latest SQL Server if it really comes to that, but I'd rather not fight that battle.
There are numerous options ways you can notify the clients.
You can use a ready-made solution like NServiceBus, to publish information from the server to the clients or other servers. NServiceBus uses MSMQ to publish one message to multiple subscribers in a very easy and durable way.
You can use MSMQ or another queuing product to publish messages from the server that will be delivered to the clients.
You can host a WCF service on the Windows service and connect to it from each client using a Duplex channel. Each time there is a change the service will notify the appropriate clients or even all of them. This is more complex to code but also much more flexible. You could probably send enough information back to the clients that they wouldn't need to poll the database at all.
You can have the service broadcast a UDP packet to all clients to notify them there are changes they need to pull. You can probably add enough information in the packet to allow the clients to decide whether they need to pull data from the server or not. This is a very lightweight for the server and the network, but it assumes that all clients are in the same LAN.
Perhaps you can leverage SqlDependency to receive notifications only when the data actually changes.
You can use any messaging middleware like MSMQ, JMS or TIBCO to communicate between your client and the service.
By far the easiest, and most likely the cheapest, answer is to simply buy a bigger server.
Barring that, you are in for a development effort that has a high probability of early failure. By failure I don't mean that you end up scraping whatever it is you end up building. Rather, I mean you launch the changes and orders will be screwed up while you are debugging your myriad of business rules.
Quite frankly, I wouldn't consider approaching a communications change under pressure; presuming your statement about load going "through the roof" in the near term.
If your risk exposure is such that it has to be 100% functional day one (which is normal when you are expecting a large increase in orders), with no hiccups then just upsize the DB server. Heck, I wouldn't even install the latest sql server on it. Instead, just buy a larger machine, install the exact same OS and DB server (and patch levels) and move your database.
Then look at your architecture to determine what needs to go away and what can be salvaged.
If everybody connects to SQL Server then there is also the option of Service Broker. Unlike other messaging/queueing solution recommended so far it is entirely contained in your database (no separate product to deploy, administer and configure), it offers a single story vis-a-vis your backup/recovery and high availability needs ( no separate backup for message store, no separate DR/HA, whatever is your DB solution is also your messaging solution) and overs a uniform programming API (SQL).
Even when everything is within one single SQL Server instance (ie. there is no need to communicate over network between multiple SQL Service instances) Service Broker still has an ace that no one can match: activation. With activation you eliminate completely the need to poll because the system itself will launch your processing code (will 'activate') when there are events to process. The processing code can be internal (T-SQL procedure or SQLCLR .Net procedure) or external (see external activator).
I am working on one Asp.Net application and need to send mails periodically based on some event. First I thought of creating a thread in global.asax and start thread in application_start. But that becomes a bit of problem when application pool crashes or something. So I implemented a windows service and started thread in that and log any errors in windows event log. This works fine. But I need to know whether I am implementing it correctly or is there a better way of doing it?
I think you are moving (or moved already) to the right direction.
We have similar architectures as well, in some cases we used MSMQ to queue outgoing notifications from the ASP.NET application then the Windows Service, usually called Messaging Manager, can grab asynchronously the incoming messages and send the emails or alerts out.
this proves to be effective and robust, if anything crashes after the message has been queued, nothing will be lost because the windows service will always process the messages in the queue, so you can have ASP.NET recycling or the machine with the windows service being rebooted, nothing is lost ever. And in fact in normal production mode, messages are sent out instantly, the decoupling or loose of sync is mostly hidden when everything is working smooth and servers are not overloaded or suffering anything.
In a later project we are now implementing something similar using TIBCO technologies, EMS for the queues and Business Works for queue subscribers.
Using a Windows Service for this kind of tasks is the preferred way instead of doing it in the ASP.NET application. You may also take a look at Quartz.NET which could simply your code for scheduling the task execution and dealing with threads. But if you don't want to write Windows Services probably the simplest would be to have a console application that will do the job of sending emails and then simply use the Windows Task Scheduler to run it at regular intervals.
Another option is a message-based approach. You could have a Windows Service/Console Application reading messages of a message queue (like msmq) and send email when a message is recieved. You can then have your ASP.NET application publish messages to this queue.
Minibuss is a lightweight client for msmq which is very easy to work with. Another options is NServiceBus.
I'm currently working on a Windows service (my first) and I'm wondering how to handle disconnect events and the like. In essence, this Windows service polls our Exchange servers for new emails. Once an email is received we parse it and insert it into a database. Now, I have everything working so long as everything is working in my favour. Since that is impossible to maintain I need to look for ways to ensure my service stays on line regardless of what may happen that is out of my control (minus the server hosting the service that is).
The main issues I can foresee are our Exchange servers going down for whatever reason or losing internet connectivity. Two problems which can happen several times a year.
Currently, if an exception is thrown regarding connectivity issues I keep attempting to connect every n minutes with a 30 second time out. So say our Exchange servers go down (either planned maintenance or unforeseen events) for 2 hours then the service would try and reconnect every n minutes until a connection is made.
Is this a sustainable strategy to ensure my service always stays online? If not, what is a better way?
What I want to avoid is my service going down because Exchange had issues making me have to manually restart my Windows service.
Thank you.
Your strategy sounds like the only thing that's practical.
It may also be worth considering adding the ability to view event logs from the service remotely so you can diagnose issues that you don't currently know about. If you're really paranoid, a second "watcher" service could be used to periodically check the primary service and report if it fails.