Building C# console app for multiple instances - c#

I'm building a console application which imports data into databases. This is to run every hour depending on an input CSV file being present. The application also needs to be reused for other database imports on the same server, e.g. there could be up to 20 instances of the same .exe file with each instance having their own separate configuration.
At the moment I have the base application which passes a location of config file via args, so it can be tweaked depending on which application needs to use it. It also undertakes the import via a transaction, which all works fine.
I'm concerned that having 20 instances of the same .exe file running on the same box, every hour, may cause the CPU to max out?
What can I do to resolve this? Would threading help?

Why not make a single instance that can handle multiple configurations? Seems a lot easier to maintain and control.

Each executable will be running in it's own process, and therefore, with it's own thread(s). Depending on how processor intensive each task is, the CPU may well max out but this is not necessarily something to be concerned about. If you are concerned about concurrent load then the best way may be to stagger the scheduling of your processes so that you have the minimum number of them running simultaneously.

No, this isn't a threading issue.
Just create a system-wide named Mutex at the start of the application. When creating that Mutex, see if it already exists. If it does, it means that there is another instance of your application running. At this point you can give the user a message (via the console or message box) to say that another instance is already running, then you can terminate the application.

I realize this thread is very old but I had the very same issues on my project. I suggest using MSMQ to process jobs in sequence.

Related

Runing and monitoring several instances of a custom C++/C# application automatically, possible?

Ok, I guess I need to make myself a bit clearer, sorry.
I DO NOT have problem with the app it self. it runs single or multiple instances just fine. we have mechanism built in to prevent cross-instance interference, file/record blocking etc..
the issue is that sometimes when running several instances, (some one go and click on myapp.exe several times manually ) one or more instances can crush.
be it from a bad data in the file, loss db connection, what ever. I am still trying to figure out some of the unexplained crushes and hanged instances sources.
what I am looking for is a way for me to setup a monitoring process that
A. can identify each instance of the app running as separate entity.
B. check if that one instance is running and not hanged / crushed , this maybe mediated but me changing the code and force the app to quit if a fatal error is detected, as in I crushed and can not recover I quit, kind of setup.
C. start new additional instances up to the total count of desired instances running. that is I want to have a minimal count or app running at the same time. if current count is less start up new until desired count is reached.
what would be the best approach to have this setup.
I have a custom inhouse application written on C++ and C# mixed API.
it is a file processing parser for EDI process, that takes data from flat files or Flat file representation from a DB table (we have other modules that store a contents of flat file in DB varchar field) and process and store data in appropriate DB tables.
it works fine for the most part, but several clients have a need to run multiple instances of it to speed up the process.
problem is that when the app crashes, there is no automatic way I know to identify the instance that crashed , shut it down and restart.
need help to identify my options here.
how can I run the same EXE multiple time yet monitor and manage each in the event of instance misbehaving.
a code change is possible but C++ is not my main language so it will be a nightmare for me to do any extensive changes.
PS> the app is a mix of C++ and C# modules with MSSQL DB backend
thanks.
the process is monitored with MSSQL SPs. an issue can be identified by luck of new records processed and other factors
my needs are to be able to
1. start several instances of an app at the same time.
2. monitor each instance of running EXE and if crush is detected kill and restart that one instance. I have a kill switch in the app that can shut it down gracefully, but need a way to monitor and restart it. but only that one.

C# What exactly is application domain?

I understand that an application domain forms:
an isolation boundary for security,
versioning,
reliability,
and unloading of managed code,
but so does a process
Can someone please help me understand the practical benefits of an application domain?
I assumed app domain provides you a container to load one version of an assembly but recently I discovered that multiple versions of strong key assembly can be loaded in an app domain.
My concept of application domain is still not clear. And I am struggling to understand why this concept was implemented when the concept of process is present.
Thank you.
I can't tell if you are talking in general or specifically .NET's AppDomain.
I am going to assume .NET's AppDomain and why it can be really useful when you need that isolation inside of a single process.
For instance:
Say you are dealing with a library that had certain worker classes and you have no choice, but to use those workers and can't modify the code. It's your job to build a Windows Service that manages said workers and makes sure they all stay up and running and need to work in parallel.
Easy enough right? Well, you hoped. It turns out your worker library is prone to throwing exceptions, uses a static configuration, and is generally just a real PITA.
You could try to launch them in their own process, but monitor them, you'll need to implement namedpipes or try to thoughtfully parse the STDIN and STDOUT of the process.
What else can you do? Well AppDomain actually solves this. I can spawn an AppDomain for each worker, give them their own configuration, they can't screw each other up by changing static properties because they are isolated, and on top of that, if the library bombs out and I failed to catch the exception, it doesn't bother the workers in their domain. And during all of this, I can still communicate with those workers easily.
Sadly, I have had to do this before
EDIT: Started to write this as a comment response, but got too large
Individual processes can work great in many scenarios, however, there are just times where they can become a pain. I am not saying one should use an AppDomain over another process. I think it's uncommon you would need a separate process or AppDomain, but once you need it, you'll definitely know.
The main problem I see with processes in the scenario I've given above is that processes have their own downfalls that are easier to mitigate with the AppDomain.
A process can go rogue, become unresponsive, and crash or be killed at any point.
If you're managing processes, you need to keep track of the process ID and monitor the status of it. IPCs are great, but it does take time to get proper communication going back and forth as needed.
As an example let's say your process just dies. What do you do? Depending on the mechanism you chose to monitor, maybe the communication thread died, perhaps the work finished and you still show it as "processing". What do you do?
Now what happens when you have 20 processes and your management app dies. You don't have any real information, all you have is 20 "myprocess.exe" and maybe now have to start parsing the command line arguments they were started with to see which workers you actually have. Obviously with an AppDomain all 20 would have died too, but did you really gain anything with the process? You still have to code the ability to recover, however, now you have to also code all of the recovery for your processes instead of just firing the workers back up.
As with anything in programming, there's 1,000 different ways to achieve the same goal. It's up to you to decide which solution you feel is most appropriate.
Some practical benefits using app domain:
Multiple app domains can be run in a process. You can also stop individual app domain without stopping the entire process. This alone drastically increases the server scalability.
Managing app domain life cycle is done programmatically by runtime hosts (you can override it as well). For processes & threads, you have to explicitly manage their life cycle. Initialization, execution, termination, inter-process/multithread communication is complex and that's why it's easier to defer that to CLR management.
Source: https://learn.microsoft.com/en-us/dotnet/framework/app-domains/application-domains

Wrapping unmanaged code using multiple processors

I have an existing application written in c++ that does a number of tasks currently, reading transactiosn from a database for all customers, processing them and writing the results back.
What I want to do is have multiple versions of this running in parallel on separate machines to increase transaction capacity, by assigning a certain subset of customers to each version of the app so that there is no contention or data sharing required, hence no locking or synchronisation.
What I want to do though is have multiple versions running on the same machine aswell as distributed across other machines, so if I have a quad core box, there would be four instances of the application running, each utilising one of the CPU's.
I will be wrapping the c++ code in a .NET c# interface and managing all these processes - local and distributed from a parent c# management service responsible for creating, starting and stopping the processes, aswell as all communication and management between them.
What I want to know is if I create four instances each on a separate background thread on a quad core box, whether or not the CLR and .NET will automatically take care of spreading the load across the four CPUs on each box or whether I need to do something to make use of the parallel processing capability?
If you mean that you will be running your application in four processes on the same box, then it is the operating system (Windows) which controls how these processes are allocated CPU time. If the processes are doing similar work, then generally they will get roughly equal processor time.
But, have you considered using four threads within a single process? Threads are much more lightweight than processes, and you wouldn't then need a separate management service, i.e., you would have one process (with four threads) instead of 5 processes. Do you come from a unix background by any chance?
You can set the process affinity when launching the process via the Process object (or ProcessThread depending on how you are launching the app).
Here is an SO post which covers the subject (I didn't vote to close as a duplicate (yet) because I'm not 100% sure if this is exactly what you are after).

.NET - limiting the number of instances of an execution unit

Let's say I have an application written in C# called EquipCtrl.exe which runs as a local process on a PC to control a piece of equipment.
Obviously, I would wish to have only one instance of Equipctrl to run on each PC. If I had two equip to control per PC, I would then limit it to two instances per PC.
The way I did is was either one of
1. Process name. I name the process EqCtrl and at process start-up, it would count the number processes with the name "EqCtrl".
2. Execution name. At start-up count the number of processes with the execution name EquipCtrl.exe.
3. Registry record.
4. SQL Server database record.
To me, process name or execution name detection is the simplest and what I do most (if not all) of the time. But, they are susceptible to name clashing. Even if I go further to find out the execution path, the limit could be circumvented by copying the execution file to another folder.
What is the best way to perform execution limiting on .NET? Why? Is registry record the best way?
Try a system-wide Semaphore:
http://msdn.microsoft.com/en-us/library/system.threading.semaphore.aspx
Scott Hanselman's Blog has a great post on this...
http://www.hanselman.com/blog/TheWeeklySourceCode31SingleInstanceWinFormsAndMicrosoftVisualBasicdll.aspx
Hmm .. when we want to allow only a single instance of an application we use a named mutex
That doesn't exactly enable the scenario you desire.
Another point, to avoid collisions, in addition to the executable name and directory, use an MD5 hash of the executable file.
You can do this with counting named mutex\semaphore, also make sure the mutex/semaphore prefix you create starts with GLOBAL\ and LOCAL\ otherwise remote desktop connection won't read these named mutex and count will be lost.
http://msdn.microsoft.com/en-us/library/ms682411(VS.85).aspx

Hints and tips for a Windows service I am creating in C# and Quartz.NET

I have a project ongoing at the moment which is create a Windows Service that essentially moves files around multiple paths. A job may be to, every 60 seconds, get all files matching a regular expression from an FTP server and transfer them to a Network Path, and so on. These jobs are stored in an SQL database.
Currently, the service takes the form of a console application, for ease of development. Jobs are added using an ASP.NET page, and can be editted using another ASP.NET page.
I have some issues though, some relating to Quartz.NET and some general issues.
Quartz.NET:
1: This is the biggest issue I have. Seeing as I'm developing the application as a console application for the time being, I'm having to create a new Quartz.NET scheduler on all my files/pages. This is causing multiple confusing errors, but I just don't know how to institate the scheduler in one global file, and access these in my ASP.NET pages (so I can get details into a grid view to edit, for example)
2: My manager would suggested I could look into having multiple 'configurations' inside Quartz.NET. By this, I mean that at any given time, an administrator can change the applications configuration so that only specifically chosen applications run. What'd be the easiest way of doing this in Quartz.NET?
General:
1: One thing that that's crucial in this application is assurance that the file has been moved and it's actually on the target path (after the move the original file is deleted, so it would be disastrous if the file is deleted when it hasn't actually been copied!). I also need to make sure that the files contents match on the initial path, and the target path to give peace of mind that what has been copied is right. I'm currently doing this by MD5 hashing the initial file, copying the file, and before deleting it make sure that the file exists on the server. Then I hash the file on the server and make sure the hashes match up. Is there a simpler way of doing this? I'm concerned that the hashing may put strain on the system.
2: This relates to the above question, but isn't as important as not even my manager has any idea how I'd do this, but I'd love to implement this. An issue would arise if a job is executed when a file is being written to, which may be that a half written file will be transferred, thus making it totally useless, and it would also be bad as the the initial file would be destroyed while it's being written to! Is there a way of checking of this?
As you've discovered, running the Quartz scheduler inside an ASP.NET presents many problems. Check out Marko Lahma's response to your question about running the scheduler inside of an ASP.NET web app:
Quartz.Net scheduler works locally but not on remote host
As far as preventing race conditions between your jobs (eg. trying to delete a file that hasn't actually been copied to the file system yet), what you need to implement is some sort of job-chaining:
http://quartznet.sourceforge.net/faq.html#howtochainjobs
In the past I've used the TriggerListeners and JobListeners to do something similar to what you need. Basically, you register event listeners that wait to execute certain jobs until after another job is completed. It's important that you test out those listeners, and understand what's happening when those events are fired. You can easily find yourself implementing a solution that seems to work fine in development (false positive) and then fails to work in production, without understanding how and when the scheduler does certain things with regards to asynchronous job execution.
Good luck! Schedulers are fun!

Categories

Resources