Develop Reference

SSIS Package Created and Executed via Code Requires Service to be Restarted after Multiple Runs

we have existing C# code to dynamically create SSIS packages that reads from different files and into a SQL Server 2016 database. It does what we need it to, but stumbled into an issue that we remain unable to resolve: being unable to keep running the .Execute command without having to restart our custom Windows service.
Right now the limit is at 2 files. If we run our code that calls the Execute command the third time, it will get stuck up on post validate based from what we're logging via the Component and Run Event handlers, and we couldn't proceed until we restart the Windows service and run again. We cannot just always restart the service because there could be other processes that can be disrupted if we go with that approach.
Things we've tried so far:
Extract the dtsx package that the code creates and attempt to run it using dtexec or BIDS locally. No errors / warnings raised, and we can keep re-running the same package over and over without fail.
Run the same code locally. Same result as #1.
Run a SQL trace with the help of our DBA. Confirmed that we have no queries locking the destination table and the database itself. We did observe some SPIDs being retained after the third run but all are in a sleeping state.
Modify our RunEventHandler and ComponentEventHandler to log which step is the process in. Also tried enabling logging via Event Viewer. No errors, really just getting stuck at post-validate as mentioned earlier come the third run.
Explicitly call the SSIS dispose methods, even tried explicitly disposing the connection managers themselves.
Play around the DelayValidations and ValidateExternalMetadata properties.
Any chance others have encountered this before and were able figure out what's causing this?

To expound on the latest comment, we've found that the issue stems from the fact that we're calling a separate AppDomain to take care of running the job and consequently executing the package. The way this separate AppDomain is created is via CreateInstanceAndUnwrap,using the following code:
private AppDomain CreateAppDomain(string applicationName)
{
var uid = Guid.NewGuid();
var fi = new FileInfo(Assembly.GetExecutingAssembly().Location);
var info = new AppDomainSetup
{
ConfigurationFile = #"D:\Deploy\App\web.config",
ApplicationBase = #"D:\Deploy\App",
ApplicationName = applicationName,
CachePath = fi.Directory.Parent.FullName + #"\Temp",
ShadowCopyDirectories = #"D:\Deploy\App\bin",
PrivateBinPath = "bin",
ShadowCopyFiles = "true"
};
var domain = AppDomain.CreateDomain(uid.ToString(), null, info);
return domain;
}
How we're able to test out this theory is by calling AppDomain.Unload(domain) against that created domain, and while we get an DomainUnloadException, this does prevent the job from freezing after being run twice.
We still haven't determined exactly what's within the domain that's getting locked up and preventing us to run the job more than twice, and guidance to learn more about that will be helpful. In the meantime, we're using this workaround of unloading the app domain for now.

c# : Create windows scheduled task as a different user

I'm creating Windows Scheduled Tasks dynamically from c# using the build-in TaskService and TaskDefinition libraries.
But for some of them, we need to create then to run as a different user (Local Service or Network Service). As the tasks are created and removed dynamically we cannot edit all of them manually to change the user. We need to do it via code.
Is is possible?
I've tried the following settings:
TaskDefinition.Principal.Id = "NETWORK SERVICE";
TaskDefinition.Principal.LogonType = TaskLogonType.ServiceAccount;
but this gives me the very descript error when creating the task:
System.Runtime.InteropServices.COMException: '(52,4):Task:'
Without those 2 lines, it works but creates them as the logged in user.

I've played around with the Task Scheduler stuff a bit and have replicated your problem. I believe I’ve found some things out, maybe they can help.
1. Firstly if your making Tasks in the debugger using Services Accounts, you'll want to ensure your Visual Studio or other IDE is launched as administrator to ensure you have the correct privileges to do this task.
2. I'm not sure if you do this later in your code but to make the task save and run as NETWORK SERVICE, I had to Identify Network Service as NT AUTHORITY\\NETWORKSERVICE in both the principle and on the RegisterTaskDefinition method:
TaskService tService = new TaskService();
TaskDefinition tDefinition = tService.NewTask();
tDefinition.Principal.Id = "NT AUTHORITY\\NETWORKSERVICE";
tDefinition.Principal.LogonType = TaskLogonType.ServiceAccount;
tDefinition.RegistrationInfo.Description = "Testing";
tDefinition.Triggers.Add(new DailyTrigger {DaysInterval = 2});
tDefinition.Actions.Add(new ExecAction("notepad.exe"));
tService.RootFolder.RegisterTaskDefinition(#"Test", tDefinition, TaskCreation.CreateOrUpdate,
"NT AUTHORITY\\NETWORKSERVICE", null,
TaskLogonType.ServiceAccount);
I used the above code to make a test Task that got successfully added to my scheduler as Network Service as shown below:
I'm guessing that one or both of the above points may have stopped the task from being added, hope that helps

Coded UI Application Under Test

Just started implementing a coded ui test automation solution but keep running into an issue when starting the application.
The application seems to start just fine but no matter what I always get an exception stating:
Microsoft.VisualStudio.TestTools.UITest.Extension.FailedToLaunchApplicationException: "The application cannot be started. This could be due to one of the following reasons:
1) Another instance of the application is already running and only one instance can be running at a time.
2) The application started another process and has now stopped. You may need to launch the process directly.
3) You do not have sufficient privileges for this application."
The application is a little strange as it currently is setup to run off of a setup.exe so the user always has the latest version.
Am I missing something in my code (sample below)? Or does the application need to be better set up before I start writing the automation tests. EXE is located in a network location.
ApplicationUnderTest aut = ApplicationUnderTest.Launch(#"\\test.com\\applicationdir\\testenv\\application\\setup.exe");
WpfEdit userName = new WpfEdit(aut);
userName.SearchProperties.Add(WpfEdit.PropertyNames.AutomationId, "PART_UserName");
userName.Text = "TEST";

Currently using a workaround where I start the app via Process and then pass it to the application under test FromProcess(). Seemed to fix the issue.
Probably not the best solution and have to use a Thread.Sleep() but it works for now.
Example:
var process = Process.Start(#"pathToApplication");
Thread.Sleep(2000);
process = Process.GetProcessesByName("process.Name")[0];
ApplicationUnderTest aut = ApplicationUnderTest.FromProcess(process);

Creating new AppDomain calling method in same class as the AppDomain in made

I want to start below potentially long running thread in it's own AppDomain to prevent the webserver from aborting it during recycling. It compiles fine, however during runtime I get this cryptic error
Type is not resolved for member 'MyCore.MyWebService,MyCore,
Version=5.0.0.0, Culture=neutral, PublicKeyToken=null'.
How do I find out what member is not resolved?
Are there any better ways running a long standing thread in a MVC business service layer, that does not get aborted by the server recycling mechanism?
Here is the code:
namespace MyCore
{
[Serializable]
public class MyWebService : IMyWebService
{
AppDomain domain = AppDomain.CreateDomain("Domain");
Thread.CurrentThread.Name = "MVCThread";
domain.SetData("lDatabaseID", lDatabaseID);
domain.DoCallBack(() =>
{
long lID = Convert.ToInt64(AppDomain.CurrentDomain.GetData("lDatabaseID"));
Thread thread = new Thread(
(() =>
{
PopulateTables(lID );
}));
thread.Name = "DomThread";
thread.Start();
});
}
}

IIS is heavily optimised to respond very quickly to hundreds of small simultaneous requests and just isn't the right tool for what you're attempting. You can try to work around that but in the long term you'll be better off building a tool that is designed for long-running tasks. You've then got a pre-packaged solution the next time this problem arises.
The basic idea is to create an external application that does your background processing with some way to pass tasks to it and get results back. I like using the database to communicate as most web applications that need baground processing already use a database. Add a 'tasks' table with {status, startedDateTime, finishedDateTime, parameters, etc}, then write an external application that will periodically look for a new task, complete it and update the database. Your web site can poll the database for status or your application could make an AJAX call to notify the web site when a job has completed (a small iframe in the web site header that shows waiting / completed tasks can be useful if someone will be waiting for the job to complete and is easy to do).
EDIT: Before you do the above review HangFire (which works inside IIS, as a Windows Service or as a console app). Same principles, but a pre-packaged solution. Note that I haven't implemented this yet but it looks good.
Although it's a bit of work to set up, handing this task off to a Windows Service is a good approach if you might have multiple tasks and need them responded to quickly. There are a lot of tutorials on the web that will help you create a Windows Service, such as http://www.codeproject.com/Articles/106742/Creating-a-simple-Windows-Service but you'll have to build a simple task executor on top of that so if that's the way you want to go I'd look for a pre-built task engine (I couldn't find one quickly but I'm probably using the wrong search phrase).
But that's overkill if turn-around time isn't important and a better approach for you might be to create a small console application that will be started every five minutes by task scheduler. It would connect to the database, execute any waiting tasks then shut down again. This is easier to debug and install than a Windows service and achieves the same goal of moving the task execution out of IIS.
Remember that you still have to detect and handle Windows shutdown so that you don't get half-finished orphaned jobs - at the very least just tag that task as aborted and exit cleanly.

Alright after having mucked with Hangfire, I finally got it to work in .Net 4.0 and MVC 3. Had to install Common.Logging.Core 2.2.0, since the NuGet installed the wrong version (3.3.0)
In my Initial controller I added the following
namespace Core.Controllers
{
...
public void Configuration(IAppBuilder app)
{
app.UseHangfire(config =>
{
config.UseSqlServerStorage(ConnectionString.GetTVConnectionString());
config.UseServer();
});
}
...
}
ConnectionString.GetTVConnectionString() gets the connection string from the config file.
Up top I added the following
[assembly: OwinStartup(typeof(Core.Controllers.BaseController))]
In the code that starts the background thread I added the following, passing in a long instead of the class and having the job load the POCO class from the db.
BackgroundJob.Enqueue(() => PopulateTables(lDatabaseID, JobCancellationToken.Null));
The Enqueue() function returns a job id, that later can be used to cancel the job if needed, through the BackgroundJob.Delete(jobid) function.
In the job method I then have this
while (idxMin < max)
{
try
{
cancellationToken.ThrowIfCancellationRequested();
....
}
catch (JobAbortedException jobEx)
{
....
}
}
It's important to use dependency injection, so my class had a parameter less constructor added that re-reads the connection string rather than have it passed in.
public MyWebService ()
: this(ConnectionString.GetTVConnectionString())
{
}
public MyWebService (string sConnStr)
{
msConnStr = sConnStr;
}
After that it seems to run pretty well. A number of tables are added to the database specified in the connection string. So far it seems like the jobs survive recycling on the webserver.

MVC 3 - Do some task at interval time

I am developing an web app with ASP.NET MVC3.
The requirement is: At some time interval (0:00AM or 10:00pm), the app must automatically do some task such as: Change order status in DB, send notify email to customer, clear temp folder....
I'm thinking of using a thread in Global.aspx, let it run and sleep for 24 hours, but I don't think that is a good resolution.
#Thank to patryk-wiek comment, I find a solution with " Task Scheduler MVC" keywork here.
https://github.com/jgeurts/FluentScheduler
Is that a good solution or not? I think writing a window service is a bit overkill for me?

Don't do it in your web application as you may not be in control of when the host process times out and goes!
Create a windows service to perform these tasks.
http://msdn.microsoft.com/en-us/library/d56de412.aspx

you can use RX for the same by creating a window service for the same.. as i have attached one sample code below which would suscibe first and will do the operation in regular interval of 2 minutes.:-
Observable.Generate(
true,
_ => true,
i =>
{
////your code...
return i;
},
i => i,
_ => TimeSpan.FromMinutes(2)).Subscribe();

Don't do it inside your web app, becasue the AppPool can be terminated at any time. There are ways how to prevent this, but it's not very clean solution.
Use windows scheduler, WCF service or some workflow solution for log running tasks.