I have spent days trying to solve this problem and still stuck with this and I have posted some questions already on this website, but didn't get satisfactory answers. I am trying to be more clear this time and hope to get a better answer. I have gone through this article already http://blogs.msdn.com/b/michen/archive/2007/03/22/running-ssis-package-programmatically.aspx
and here are my issues (I need to run the SSIS package from ASP.NET)
option 1 is not suitable for me, because it may recycle worker process if it consumes memory
option 2 is also not suitable because of security issues in creating a new process and passing the context to new process looks very complicated for me (according to the support article)
option 3 is not suitable because using SQL Server Agent to run SSIS package is not allowed by the company I am working for(I guesss it requires installation of db engine on application server, not sure). but SSIS is installed on the application server.
option 4&5 will have the same issues as options 1&2.
I guess the only option left now is to create a windows service and start the service from ASP.NET. but will this allow running multiple packages in parallel? OR is there a better alternate solution for this? please let me know. Thanks.
I have built something like this in the past with
a windows service that monitors a database table
an asp.net page that inserts jobs in the database table
each job is a (set of) ssis package(s)
the windows service is configured at startup with a number of threads in a pool
each time the windows service sees a new job, it checks if there is a thread available and launches the thread with the job to be run with dtexec.exe using System.Diagnostics.Process (you can use the SSIS class libraries, but I found dtexec.exe more useful
the thread (and the job) runs in the security context of the windows service, thus using the windows credentials used by the windows service
when a job is finished, the windows service updates the database table
You could change this to do without the table, and expose a wcf service from the windows service with asynch methods that return when the job is finished. I'm not sure how to use multiple threads in that case, but I think by making it asynch you also make it inherently multi-threaded.
You are on the right track with the windows service.
Instead of running the SSIS package from asp.net, get asp.net to place an entry in a queue or a list. Then get the windows service to monitor the list.
When the windows service sees that a new item is in the list it runs the SSIS package that the list item refers to.
The benefit of this is that the web page does not have to wait while the SSIS package runs.
Related
I have a couple of questions regarding a project I recently started working on. This project requires me to discover devices on the network (broadcasting UDP packets on a specific port), store device information into a database, and then display all the data on a local web page. My questions are:
Can I have a C# program that runs in the background that deals with the device discovery and database management while also running a local webpage for users to interact with?
If so, which kind of Visual Studio project would be the best to
implement this project?
If not, how else could I do this?
We had a problem where we needed a long running process that was initiated by a user but which timed out. We resolved this by running a background Windows service which interacted with the database backing the website. We also have other tasks that we wish to run asynchronously following user action. We run three Windows services on the web server; this solution has been running successfully for a number of years. So, in answer to our specific points:
Yes, as a Windows service separate from your website project
There is a specific project type for this; you will also need to set up an installation project
There may be other ways but I would recommend this approach
I am developing an email marketing WinForm application. And for the scheduling of campaigns, I decided to use Quartz.NET. I need it to be running as a Windows Service. but I also want the user to be able to add a job (e.g. a campaign that needs to be run everyday at 8 AM which is basically running a .bat file) to the service through the program.
I am also saving all the job schedules in the database so that when you stop/start the OS or Windows Service it can still read from all the jobs that need to be run.
How can I add a job to the service while the service is running? The dynamic addition/removal of the job to the service is of course much preferred. Stopping the service and reading all the jobs again from the database is, to be frank, my last resort.
You can install Quartz.Net as a windows service and configure it to use AdoJobStore.
Your WinForm application will be the interface where you can add, suspend and remove jobs and triggers.
You do not have to stop your windows services while you're doing that.
You do not need to read your database to figure out what's happening in Quartz but you must use the APIs provided.
I've done something similar long time ago; my quartz.net "manager" was a web application.
You can read my answer here.
I guess the steps to do what you're looking for a pretty much the same.
Reference to Quartz.net samples and a free book can be found here and here.
I want to Scheduling in Asp.net
I have following options to implement this
To write SQLServer JOB(I dont want to do this.Dont want to go outside of .Net environment)
Second option is I will write windows service and this window service will call asp.net
webservice then this webservice calls asp.net method
(I also dont need to do this because my hosting provider might not be allow me to install
window service)
Third option is I call my scheduling method in Application_Start event in global class
(Drawback is, webserver will kill thread any time )
To call Scheduling Code in Page_Load event of Home Page(Might be nobody visits my website for hours
,Also page execution might be slow due to scheduling code)
I also found some online services that calls your page at given interval,some are given below
http://www.cronservice.co.uk/new/
http://scheduler.codeeffects.com
Anybody give me bettor solution of this and also explain why it is bettor?
Thanks in Advance
The ASP.NET application isn't the right place to implement scheduling. I would suggest creating a service or a scheduled task that runs in short intervals.
You don't have many options in a shared hosting environment. My host (WinHost) allows remote access to their database, so I was able to create an executable that ran on a local server with Task Scheduler.
The performance isn't great since the database is accessed over the internet, but it's still better than attempting to run pseudo scheduled tasks with ASP.NET.
Some hosts also offer a service that will request a url within your site on a scheduled basis. However, this didn't work for me because the task I had to run took longer than the request timeout.
There is no one solution that fits all. SQL jobs and windows jobs (scheduled thru windows task scheduler) are very widely used. In one of my previous work places they had jobs that ran on multiple platforms (mainframe,windows,sql server). Failure in some of these jobs, would cost in thousands by the day. So they employed something called ESP. This software monitored jobs on all platforms and sent a message to the control room in case of a failure.
If you throw some more light on the requirement, we might be able to help you better.
ASP.NET is not the right place to house your Scheduled Tasks. I'm using Quartz.net when I have to create Scheduled Tasks.
Create a page that launches your task and place it at the URL http://www.mydomain.com/runtask.
Create a scheduled task on your home PC that sends a request to http://www.mydomain.com/runtask.
You'll need to keep your home PC on all the time.
Ideally I would go with number 1 as you get full control/history/error reporting etc. You can write an SSIS job in .NET and have SQL server schedule it.
However, I have had a similar problem with shared hosting that is very restrictive. What I did was create a page which runs the process on page load (using validation in the querystring for security). I then used a machine I have which is always on to schedule a Windows Task Scheduler (this is part of Windows as standard) to call a bit of VB script that opens the browser and then shuts it.
I am developing a solution in .Net utilising the VMWare Web Service API to create new isolated virtualised development environments.
The front end will be web based. Requestors input details for the specific environment which will be persisted to SQL. So I need to write an engine of some sort to pull the data from SQL and work on the long running task of creating resource pools, switch port groups and cloning existing VM templates etc. As work progresses, events will be raised to write logs and update info back to SQL. This allows requestors to pull data back into a webpage to see how it's progressing or if it's completed.
The thing I am struggling with is how to engineer the engine which will exec the long running task of creating the environment. I cannot write a windows service (which I would like) as we work in a very secure environment and it's not possible (group policy etc). I could write a web service to execute the tasks, extending the httpRuntime executionTimeout to allow the task to complete. But I'm keen to hear what you guys think may be a better solution to use (based on .Net 3.5). The solution needs to be service oriented as we may be using it on other projects within our org. What about WWF, WCF? I have not used any of the newer technologies available since .Net 2.0 as we've only just been approved to move up from .Net 2.0.
First of all, a Windows Service isn't insecure. One of software development devils is discarding a solution by ignorance or because a lack of investigation, requirement analysis and taking decisions collaborately.
Anyway, if you want to do it in a SOA way, WCF is going to be your best friend.
But maybe you can mix a WCF service hosted by Internet Information Services and Message Queue Server (MSMQ).
A client calls a WCF service operation and enqueues a message to some Message Queue Server queue. Later, a Windows scheduled task executed overtime, checks if your queue has at least an incoming message, and task processes this and others until you dequeue all messages.
Doing this way, IIS WCF host will never need to increase its execution time and it'll serve more requests, while the heavy work of executing this long tasks is performed by a Windows scheduled task, for example a console application or a PowerShell cmdlet (or just a PowerShell script, why not?).
Check this old but useful MSDN article about MSMQ: http://msdn.microsoft.com/en-us/library/ms978430.aspx
I don't understand your comment regarding services being insecure, I think that is false.
However I would look into creating something that uses a messaging bus/daemon.
RabbitMQ: http://www.rabbitmq.com/
MassTransit: http://masstransit-project.com/
NServiceBus: http://nservicebus.com/
Then you can basically use something like WCF, Console App or Service as your daemon.
I'm building a web application that will need to import data from other database servers when it starts.
I would like to have this import done automatically at regular intervals. I would also like to be able to start and stop the import process from my web application.
What would be the best implementation for the import agent - a Windows Service? Something else?
If your web application needs to have this data in memory, you can use the Cache class.
Set it to expire every X hours, as you need and when it expires, re-fetch the data..
You could create a Windows Service that uses Quartz.Net to run the scheduled tasks.
You should not run scheduled task from your web app, since you don't have any guarantee that your web app is running. You're at IIS app pool management's mercy.
You might want to look at Best way to run scheduled tasks.
Of what I heard this looks like a description for Microsoft Sync Framework. I have just few information about it for myself but will be pleased to see you pointed into that direction.
I'm not sure about your question because you are talking about hourly syncing. When talking web applications, there can't be a nice way to do such a task. You have to create a console app or best task would be a Windows Service Process (which are easier then it sounds)?
Sync Framework Intro
http://msdn.microsoft.com/en-us/sync/bb821992
Sync Framework Tutorial
http://alexduggleby.com/2007/12/16/sync-framework-tutorial-part-1-introduction/
Sync Framework Samples
http://archive.msdn.microsoft.com/sync
And, when I'm editing the answer with links
Nice guide to create a Windows Service (and setup)
http://www.codeproject.com/KB/dotnet/simplewindowsservice.aspx
(if first time, try it on a test project before the production project)
This might be an oversimplification, but can you create a class that does all of this work using a Timer, and then in the application_start of the global.asax, create a BackgroundWorker that kicks off this process?
Your web application could then control the BackgroundWorker object, starting/stopping as necessary.