Sorry for abstract question, but I'm looking for some samples/advices/articles on type of applications which does some equivalent operations in cycle, and every iteration of cycle should expose its result in certain portion of time (for instance, 10 seconds).
My application does synchronization of data between external WCF service and local database. In every iteration an application retrieves changes of data passing request to WCF service and puts changes to database and vice versa. One of most hard requirement for this application is that iterations should fire every ten seconds.
So here is the issues arises. How can I guarantee that iteration will finish for no more than 10 seconds?
I guess this type of applications called real-time applications (in maner of real-time OS).
DAL components that we use acts randomly on connection timeout behavior. So DB operations may take longer time than 10 seconds.
Here is the estimated code of one iteration:
Stopwatch s1 = new Stopwatch();
s1.Start();
Parallel.ForEach(Global.config.databases, new ParallelOptions { MaxDegreeOfParallelism = -1 }, (l) =>
{
Console.WriteLine("Started for {0}", l.key.name);
DB db = new DB(l.connectionString);
DateTime lastIterationTS = GetPreviousIterationTS(l.id);
ExternalService serv = new ExternalService(l.id);
List<ChangedData> ChangedDataDb = db.GetChangedData(DateTime.Now.AddSeconds((lastIterationTS == DateTime.MinValue) ? -300 : -1 * (DateTime.Now - lastIterationTS).Seconds));
List<Data> ChangedDataService = serv.GetModifiedData();
Action syncDBChanges = new Action(() =>
{
// Изменения в БД
foreach (ChangedData d in ChangedDataDb)
{
try
{
// ...
// analyzing & syncing
}
catch (Exception e)
{
logger.InfoEx("Exception_SyncDatabase", e.ToString());
}
}
}
);
Action syncService = new Action(() =>
{
foreach (Data d in ChangedDataService)
{
try
{
// ...
// analyzing & syncing
}
catch (Exception e)
{
logger.InfoEx("Exception_SyncService", e.ToString());
}
}
});
List<WaitHandle> handles = new List<WaitHandle>();
IAsyncResult ar1 = syncDBChanges.BeginInvoke(syncDBChanges.EndInvoke, null);
IAsyncResult ar2 = syncService.BeginInvoke(syncService.EndInvoke, null);
handles.Add(ar1.AsyncWaitHandle);
handles.Add(ar2.AsyncWaitHandle);
WaitHandle.WaitAll(handles.ToArray(), (int)((Global.config.syncModifiedInterval - 1) * 1000));
SetCurrentIterationTS(l.id);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
logger.InfoEx("Exception_Iteration", e.ToString());
continue;
}
}
logger.InfoEx("end_Iteration", IterationContextParams);
}
);
s1.Stop();
Console.WriteLine("Main iteration done for {0}...", s1.Elapsed);
You can consider a couple of options...
Kill the iteration if it exceeds more than 10 seconds and hope that the next iteration can complete process. The issue with this approach is that there is a good possibility that the none of the iterations will complete and therefore the synchronization process will never occur. I would recommend the following option...
If the iteration takes more than 10 seconds, wait for it to complete and skip the next iteration(s). This way you ensure the process completes atleast once. The following is a simplified code sample for reference...
class Updater
{
Timer timer = new Timer();
public object StateLock = new object();
public string State;
public Updater()
{
timer.Elapsed += timer_Elapsed;
timer.Interval = 10000;
timer.AutoReset = true;
timer.Start();
}
void timer_Elapsed(object sender, ElapsedEventArgs e)
{
if (State != "Running")
{
Process();
}
}
private void Process()
{
try
{
lock (StateLock)
{
State = "Running";
}
// Process
lock (StateLock)
{
State = "";
}
}
catch
{
throw;
}
}
}
...
class Program
{
static void Main(string[] args)
{
Updater updater = new Updater();
Console.ReadLine();
}
}
Quartz.net is an excellent scheduler for the .NET platform, which I think could suit your needs.
If you want to kill a job, you can implement IInterruptableJob. You should be able to add some cleanup code in the Interupt method to dispose of any db connections.
If you want finish a job, but only start another job if the last one is completed (which I think is the better option), you can implement IStatefulJob interface
I usually separate the update cycle from the actual timer
The timer does two things:
1) if the update is not running starts it.
2) if the service is already running set a flag for it to continue running.
The update cycle:
1)set running flag
2) do the update
3) set running flag to false
4) if continue running is set go to 1).
You might want to read up on the variety of Timer objects available in .Net: http://msdn.microsoft.com/en-us/magazine/cc164015.aspx
I personally like System.Threading.Timer because you can easily use lambdas, and it allows a state object to be passed if you create a separate callback.
I would also recommend using the System.Threading.Tasks library, because it allows you to gracefully handle cancellations in the case that the timer elapses before your work is completed. Msdn example: http://msdn.microsoft.com/en-us/library/dd537607.aspx
Here's an example of using these components together in a 10 minute timer:
Note: to do this with your sql database you'll need to set Asynchronous Processing=true; and MultipleActiveResultSets=True;
CancellationTokenSource cancelSource = new CancellationTokenSource();
System.Threading.Timer timer = new System.Threading.Timer(callback =>
{
//start sync
Task syncTask = Task.Factory.StartNew(syncAction =>
{
using (SqlConnection conn =
new SqlConnection(
ConfigurationManager.ConnectionStrings["db"].ConnectionString))
{
conn.Open();
using (SqlCommand syncCommand = new SqlCommand
{
CommandText = "SELECT getdate() \n WAITFOR DELAY '00:11'; ",
CommandTimeout = 600,
Transaction = conn.BeginTransaction(),
Connection = conn
})
{
try
{
IAsyncResult t = syncCommand.BeginExecuteNonQuery();
SpinWait.SpinUntil(() =>
(t.IsCompleted || cancelSource.Token.IsCancellationRequested));
if (cancelSource.Token.IsCancellationRequested && !t.IsCompleted)
syncCommand.Transaction.Rollback();
}
catch (TimeoutException timeoutException)
{
syncCommand.Transaction.Rollback();
//log a failed sync attepmt here
Console.WriteLine(timeoutException.ToString());
}
finally
{
syncCommand.Connection.Close();
}
}
}
}, null, cancelSource.Token);
//set up a timer for processing in the interim, save some time for rollback
System.Threading.Timer spinTimer = new System.Threading.Timer(c => {
cancelSource.Cancel();
}, null, TimeSpan.FromMinutes(9), TimeSpan.FromSeconds(5));
//spin here until the spintimer elapses;
//this is optional, but would be useful for debugging.
SpinWait.SpinUntil(()=>(syncTask.IsCompleted || cancelSource.Token.IsCancellationRequested));
if (syncTask.IsCompleted || cancelSource.Token.IsCancellationRequested)
spinTimer.Dispose();
}, null, TimeSpan.FromMinutes(1), TimeSpan.FromMinutes(10));
Perhaps try this. Please make sure you do not create and use any new threads in DoWork() method.
class DatabaseUpdater
{
private readonly Timer _timer;
private List<Thread> _threads;
private readonly List<DatabaseConfig> _dbConfigs;
public DatabaseUpdater(int seconds, List<DatabaseConfig> dbConfigs)
{
_timer = new Timer(seconds * 1000);
_timer.Elapsed += TimerElapsed;
_dbConfigs = dbConfigs;
}
public void Start()
{
StartThreads();
_timer.Start();
}
public void Stop()
{
_timer.Stop();
StopThreads();
}
void TimerElapsed(object sender, ElapsedEventArgs e)
{
StopThreads();
StartThreads();
}
private void StartThreads()
{
var newThreads = new List<Thread>();
foreach (var config in _dbConfigs)
{
var thread = new Thread(DoWork);
thread.Start(config);
newThreads.Add(thread);
}
_threads = newThreads;
}
private void StopThreads()
{
if (_threads == null) return;
var oldThreads = _threads;
foreach (var thread in oldThreads)
{
thread.Abort();
}
}
static void DoWork(object objConfig)
{
var dbConfig = objConfig as DatabaseConfig;
if (null == dbConfig) return;
var n = GetRandomNumber();
try
{
ConsoleWriteLine("Sync started for : {0} - {1} sec work.", dbConfig.Id, n);
// update/sync db
Thread.Sleep(1000 * n);
ConsoleWriteLine("Sync finished for : {0} - {1} sec work.", dbConfig.Id, n);
}
catch (Exception ex)
{
// cancel/rollback db transaction
ConsoleWriteLine("Sync cancelled for : {0} - {1} sec work.",
dbConfig.Id, n);
}
}
static readonly Random Random = new Random();
[MethodImpl(MethodImplOptions.Synchronized)]
static int GetRandomNumber()
{
return Random.Next(5, 20);
}
[MethodImpl(MethodImplOptions.Synchronized)]
static void ConsoleWriteLine(string format, params object[] arg)
{
Console.WriteLine(format, arg);
}
}
static void Main(string[] args)
{
var configs = new List<DatabaseConfig>();
for (var i = 1; i <= 3; i++)
{
configs.Add(new DatabaseConfig() { Id = i });
}
var databaseUpdater = new DatabaseUpdater(10, configs);
databaseUpdater.Start();
Console.ReadKey();
databaseUpdater.Stop();
}
Related
I am trying to develop a windows service to send notifications to subscriptions.
The data is saved in a SQL server database.
Notifications are created by making a web POST request to a REST API endpoint and saved in a database table.
The service starts one Task that keeps reading notifications from this db table and add them to a queue.
Also the service starts few Tasks that keep reading from the queue and do the actual send process.
The code is working good and doing the job needed, but the problem is that CPU usage is 100% when running the service.
I tried to use Thread.Sleep or Task.Delay but neither helped me to reduce the CPU usage.
I have read in this codeprojct page, that I need to use wait handlers and should wait on some condition. I could not have this working properly.
so can anyone advise what can I do to reduce CPU usage for EnqueueTask and DequeueTask ?
Here is the sender code:
static class NotificationSender
{
static ConcurrentQueue<NotificationDelivery> deliveryQueue = null;
static Task enqueueTask = null;
static Task[] dequeueTasks = null;
public static void StartSending(ServiceState serviceState)
{
PushService.InitServices();
enqueueTask = Task.Factory.StartNew(EnqueueTask, serviceState);
deliveryQueue = new ConcurrentQueue<NotificationDelivery>();
int dequeueTasksCount = 10;
dequeueTasks = new Task[dequeueTasksCount];
for (int i = 0; i < dequeueTasksCount; i++)
{
dequeueTasks[i] = Task.Factory.StartNew(DequeueTask, serviceState);
}
}
public static void EnqueueTask(object state)
{
ServiceState serviceState = (ServiceState)state;
using (DSTeckWebPushNotificationsContext db = new DSTeckWebPushNotificationsContext())
{
while (!serviceState.CancellationTokenSource.Token.IsCancellationRequested)
{
int toEnqueue = 100 - deliveryQueue.Count;
if (toEnqueue > 0)
{
// fetch some records from db to be enqueued
NotificationDelivery[] deliveries = db.NotificationDeliveries
.Include("Subscription")
.Include("Notification")
.Include("Notification.NotificationLanguages")
.Include("Notification.NotificationLanguages.Language")
.Where(nd => nd.Status == NotificationDeliveryStatus.Pending && DateTime.Now >= nd.StartSendingAt)
.OrderBy(nd => nd.StartSendingAt)
.Take(toEnqueue)
.ToArray();
foreach (NotificationDelivery delivery in deliveries)
{
delivery.Status = NotificationDeliveryStatus.Queued;
deliveryQueue.Enqueue(delivery);
}
if (deliveries.Length > 0)
{
db.SaveChanges(); // save Queued state, so not fetched again the next loop
}
}
// save any changes made by the DequeueTask
// an event may be used here to know if any changes made
db.SaveChanges();
}
Task.WaitAll(dequeueTasks);
db.SaveChanges();
}
}
public async static void DequeueTask(object state)
{
ServiceState serviceState = (ServiceState)state;
while (!serviceState.CancellationTokenSource.Token.IsCancellationRequested)
{
NotificationDelivery delivery = null;
if (deliveryQueue.TryDequeue(out delivery))
{
NotificationDeliveryStatus ns = NotificationDeliveryStatus.Pending;
if (delivery.Subscription.Status == SubscriptionStatus.Subscribed)
{
PushResult result = await PushService.DoPushAsync(delivery);
switch (result)
{
case PushResult.Pushed:
ns = NotificationDeliveryStatus.Delivered;
break;
case PushResult.Error:
ns = NotificationDeliveryStatus.FailureError;
break;
case PushResult.NotSupported:
ns = NotificationDeliveryStatus.FailureNotSupported;
break;
case PushResult.UnSubscribed:
ns = NotificationDeliveryStatus.FailureUnSubscribed;
delivery.Subscription.Status = SubscriptionStatus.UnSubscribed;
break;
}
}
else
{
ns = NotificationDeliveryStatus.FailureUnSubscribed;
}
delivery.Status = ns;
delivery.DeliveredAt = DateTime.Now;
}
}
}
public static void Wait()
{
Task.WaitAll(enqueueTask);
Task.WaitAll(dequeueTasks);
enqueueTask.Dispose();
for(int i = 0; i < dequeueTasks.Length; i++)
{
dequeueTasks[i].Dispose();
}
}
}
An object of type ServiceState is used to maintain starting and stopping the service, and here is the code for this type:
class ServiceState
{
public CancellationTokenSource CancellationTokenSource { get; set; }
public void Start()
{
CancellationTokenSource = new CancellationTokenSource();
NotificationSender.StartSending(this);
}
public void Stop()
{
CancellationTokenSource.Cancel();
NotificationSender.Wait();
CancellationTokenSource.Dispose();
}
}
Here is the service start and stop code:
protected override void OnStart(string[] args)
{
_serviceState = new ServiceState();
_serviceState.Start();
}
protected override void OnStop()
{
_serviceState.Stop();
}
I think I could finally do good changes to maintain the CPU usage using wait handlers and a timer.
EnqueueTask will wait 5 seconds before trying to fetch data again from the notifications table if no notifications fetched. If no notifications fetched, it will start the timer and reset the wait handle. The timer elapsed callback will then set the wait handle.
Also DequeueTask is now using a wait handle. If no more items in the queue, it will reset the wait handle to stop dequeue-ing empty queue. EnqueueTask will set this wait handle when it adds items to the queue.
CPU usage is now <= 10%
And here is the updated NotificationSender code:
static class NotificationSender
{
static ConcurrentQueue<NotificationDelivery> deliveryQueue = null;
static Task enqueueTask = null;
static Task[] dequeueTasks = null;
static ManualResetEvent enqueueSignal = null;
static ManualResetEvent dequeueSignal = null;
static System.Timers.Timer enqueueTimer = null;
public static void StartSending(CancellationToken token)
{
PushService.InitServices();
using (DSTeckWebPushNotificationsContext db = new DSTeckWebPushNotificationsContext())
{
NotificationDelivery[] queuedDeliveries = db.NotificationDeliveries
.Where(nd => nd.Status == NotificationDeliveryStatus.Queued)
.ToArray();
foreach (NotificationDelivery delivery in queuedDeliveries)
{
delivery.Status = NotificationDeliveryStatus.Pending;
}
db.SaveChanges();
}
enqueueSignal = new ManualResetEvent(true);
dequeueSignal = new ManualResetEvent(false);
enqueueTimer = new System.Timers.Timer();
enqueueTimer.Elapsed += EnqueueTimerCallback;
enqueueTimer.Interval = 5000;
enqueueTimer.AutoReset = false;
enqueueTimer.Stop();
enqueueTask = new Task(EnqueueTask, token, TaskCreationOptions.LongRunning);
enqueueTask.Start();
deliveryQueue = new ConcurrentQueue<NotificationDelivery>();
int dequeueTasksCount = 10;
dequeueTasks = new Task[dequeueTasksCount];
for (int i = 0; i < dequeueTasksCount; i++)
{
dequeueTasks[i] = new Task(DequeueTask, token, TaskCreationOptions.LongRunning);
dequeueTasks[i].Start();
}
}
public static void EnqueueTimerCallback(Object source, ElapsedEventArgs e)
{
enqueueSignal.Set();
enqueueTimer.Stop();
}
public static void EnqueueTask(object state)
{
CancellationToken token = (CancellationToken)state;
using (DSTeckWebPushNotificationsContext db = new DSTeckWebPushNotificationsContext())
{
while (!token.IsCancellationRequested)
{
if (enqueueSignal.WaitOne())
{
int toEnqueue = 100 - deliveryQueue.Count;
if (toEnqueue > 0)
{
// fetch some records from db to be enqueued
NotificationDelivery[] deliveries = db.NotificationDeliveries
.Include("Subscription")
.Include("Notification")
.Include("Notification.NotificationLanguages")
.Include("Notification.NotificationLanguages.Language")
.Where(nd => nd.Status == NotificationDeliveryStatus.Pending && DateTime.Now >= nd.StartSendingAt)
.OrderBy(nd => nd.StartSendingAt)
.Take(toEnqueue)
.ToArray();
foreach (NotificationDelivery delivery in deliveries)
{
delivery.Status = NotificationDeliveryStatus.Queued;
deliveryQueue.Enqueue(delivery);
}
if (deliveries.Length > 0)
{
// save Queued state, so not fetched again the next loop
db.SaveChanges();
// signal the DequeueTask
dequeueSignal.Set();
}
else
{
// no more notifications, wait 5 seconds before try fetching again
enqueueSignal.Reset();
enqueueTimer.Start();
}
}
// save any changes made by the DequeueTask
// an event may be used here to know if any changes made
db.SaveChanges();
}
}
Task.WaitAll(dequeueTasks);
db.SaveChanges();
}
}
public async static void DequeueTask(object state)
{
CancellationToken token = (CancellationToken)state;
while (!token.IsCancellationRequested)
{
if (dequeueSignal.WaitOne()) // block untill we have items in the queue
{
NotificationDelivery delivery = null;
if (deliveryQueue.TryDequeue(out delivery))
{
NotificationDeliveryStatus ns = NotificationDeliveryStatus.Pending;
if (delivery.Subscription.Status == SubscriptionStatus.Subscribed)
{
PushResult result = await PushService.DoPushAsync(delivery);
switch (result)
{
case PushResult.Pushed:
ns = NotificationDeliveryStatus.Delivered;
break;
case PushResult.Error:
ns = NotificationDeliveryStatus.FailureError;
break;
case PushResult.NotSupported:
ns = NotificationDeliveryStatus.FailureNotSupported;
break;
case PushResult.UnSubscribed:
ns = NotificationDeliveryStatus.FailureUnSubscribed;
delivery.Subscription.Status = SubscriptionStatus.UnSubscribed;
break;
}
}
else
{
ns = NotificationDeliveryStatus.FailureUnSubscribed;
}
delivery.Status = ns;
delivery.DeliveredAt = DateTime.Now;
}
else
{
// empty queue, no more items
// stop dequeueing untill new items added by EnqueueTask
dequeueSignal.Reset();
}
}
}
}
public static void Wait()
{
Task.WaitAll(enqueueTask);
Task.WaitAll(dequeueTasks);
enqueueTask.Dispose();
for(int i = 0; i < dequeueTasks.Length; i++)
{
dequeueTasks[i].Dispose();
}
}
}
I know Entity Framework has a 'Connection Resiliency and Retry Logic' mechanism to work with unreliable networks.
Is there also such a thing for the System.Net.Http.HttpClient?
So a sort of an automatic retry mechanism so when the connections gets lost or a timeout occurs, then it will retry for x times.
Is that possible?
As far as I know, there is no such mechanism. But you can write yourself an event, checking the connectivity all x milliseconds.
like
public class CheckInternet
{
public CheckInternet()
{
Timer t1 = new Timer();
t1.Interval = 1000;
t1.Tick += T1_Tick;
t1.Start();
}
private void T1_Tick(object sender, EventArgs e)
{
var ping = new System.Net.NetworkInformation.Ping();
var result = ping.Send("www.google.com");
if (result.Status != System.Net.NetworkInformation.IPStatus.Success)
{
// Your code to reconnect here
}
}
public event EventHandler CheckConnectivity;
private void OnCheckConnectivity()
{
CheckConnectivity?.Invoke(this, new EventArgs());
}
}
Now just implement this to your other class use
CheckInternet cki = new CheckInternet();
cki.CheckConnectivity += new cki_CheckConnectivity;
I'm using System.Timers.Timer to backup my SQL Server Express Database once a day. It seems to work fine, most of the time. Occasionally, the ElapsedEventHandler gets called multiple times at 1 or 4 minute intervals. It should just be hit once a day. I have AutoReset as false and I call Start at the end of the ElapsedEventHandler. Also, possibly relevant is that I do recalculate the interval so that the timer always goes off as close to 1 am. as possible. The backing up of the database can take a few minutes, and if I didn't change the interval, the time might drift unacceptably. I mention this because these links suggest there might be a problem with resetting the interval:
Thread-safety of System.Timers.Timer vs System.Threading.Timer
Multiple timer elapsed issue
See in particular the answers by Hans Passant
However, I don't see how I can avoid resetting the interval. Also, I looked into the code for System.Timers.Timer. It didn't seem like just resetting the interval would Start the timer again. I'm not opposed to using a different timer (System.Threading.Timer?) but I'd like to know what is going on first.
I've pasted all the code below. I would think the truly relevant part is the method: DatabaseBackupTimerOnElapsed
Finally, I'll mention that the program is sometimes stopped and restarted (if there are uncaught exceptions in other parts of the code). I would assume though that all timers are killed at the point of exiting the program even if Dispose is not called? That is Timers don't live on in the operating system?
EDIT
I was requested to put down a small, complete, verifiable example. I do so here. I've kept the full example as someone might claim (quite correctly!) that I took out an important detail. I've run this code and have NOT seen the problem but then, it only happens very occasionally with the original code.
public class DatabaseCleanupManager
{
private const int MaxRetries = 5;
private const int DatabaseBackupHourOneAm = 1;
private Timer _databaseBackupTimer;
public DatabaseCleanupManager()
{ }
public void Initialize()
{
Console.WriteLine("Initialize");
TimeSpan spanTimer = GetDBBackupTimeSpan(1);
_databaseBackupTimer = new Timer(spanTimer.TotalMilliseconds)
{
AutoReset = false,
};
_databaseBackupTimer.Elapsed += DatabaseBackupTimerOnElapsed;
_databaseBackupTimer.Start();
}
private TimeSpan GetDBBackupTimeSpan(int databaseBackupFrequencyInDays)
{
Console.WriteLine("GetDBBackupTimeSpan");
DateTime dt1 = DateTime.Now;
DateTime dt2 = new DateTime(dt1.Year, dt1.Month,
dt1.Day, 1, 0, 0);
// I'm really interested in a timer once a day. I'm just trying to get it to happen quicker!
//dt2 = dt2.AddDays(databaseBackupFrequencyInDays);
dt2 = dt1.AddMinutes(4);
TimeSpan spanTimer = dt2 - dt1;
if (spanTimer.TotalMilliseconds < 0) // This could conceivably happen if the have 0 or a negative number (erroneously) for DatabaseBackupFrequencyInDays
{
dt2 = new DateTime(dt1.Year, dt1.Month,
dt1.Day, 1, 0, 0);
//dt2 = dt2.AddDays(databaseBackupFrequencyInDays);
dt2 = dt1.AddMinutes(4);
spanTimer = dt2 - dt1;
}
return spanTimer;
}
public void PerformDatabaseMaintenance()
{
if (BackupCurrentDatabase())
{
var success = CleanupExpiredData();
if (success)
{
Console.WriteLine("Database Maintenance Finished");
}
}
}
public void Dispose()
{
_databaseBackupTimer.Elapsed -= DatabaseBackupTimerOnElapsed;
_databaseBackupTimer.Stop();
_databaseBackupTimer.Dispose();
}
private void DatabaseBackupTimerOnElapsed(object sender, ElapsedEventArgs elapsedEventArgs)
{
try
{
Console.WriteLine("DatabaseBackupTimerOnElapsed at: " + DateTime.Now);
PerformDatabaseMaintenance();
TimeSpan spanTimer = GetDBBackupTimeSpan(1);
// NOTICE I'm calculating Interval again. Some posts suggested that this restarts timer
_databaseBackupTimer.Interval = Math.Max(spanTimer.TotalMilliseconds, TimeSpan.FromMinutes(1).TotalMilliseconds);
_databaseBackupTimer.Start();
}
catch (Exception )
{
// something went wrong - log problem and start timer again.
_databaseBackupTimer.Start();
}
}
private bool BackupCurrentDatabase()
{
// actually backup database but here I'll just sleep for 1 minute...
Thread.Sleep(1000);
Console.WriteLine("Backed up DB at: " + DateTime.Now);
return true;
}
private bool CleanupExpiredData()
{
// Actually remove old SQL Server Express DB .bak files but here just sleep
Thread.Sleep(1000);
Console.WriteLine("Cleaned up old .Bak files at: " + DateTime.Now);
return true;
}
}
class Program
{
static void Main(string[] args)
{
DatabaseCleanupManager mgr = new DatabaseCleanupManager();
mgr.Initialize();
// here we'd normally be running other threads etc., but for here...
Thread.Sleep(24*60*60*1000); // sleep for 1 day
}
}
END EDIT
public class DatabaseCleanupManager : IDatabaseCleanupManager
{
private const int MaxRetries = 5;
private const int DatabaseBackupHourOneAm = 1;
private readonly ISystemConfiguration _systemConfiguration;
private readonly IPopsicleRepository _repository;
private readonly ISystemErrorFactory _systemErrorFactory;
private readonly IAuthorizationManager _authorizationManager;
private readonly IReportRobotState _robotStateReporter;
private Timer _databaseBackupTimer;
public DatabaseCleanupManager(
IPopsicleRepository repository,
ISystemConfiguration configuration,
ISystemErrorFactory systemErrorFactory,
IAuthorizationManager authorizationManager,
IReportRobotState robotStateReporter)
{
if (repository == null)
throw new ArgumentNullException("repository");
if (configuration == null)
throw new ArgumentNullException("configuration");
if (systemErrorFactory == null)
throw new ArgumentNullException("systemErrorFactory");
if (authorizationManager == null)
throw new ArgumentNullException("authorizationManager");
if (robotStateReporter == null)
throw new ArgumentNullException("robotStateReporter");
_repository = repository;
_systemConfiguration = configuration;
_systemErrorFactory = systemErrorFactory;
_authorizationManager = authorizationManager;
_robotStateReporter = robotStateReporter;
}
public event EventHandler<SystemErrorEventArgs> SystemError;
public event EventHandler<SystemErrorClearedEventArgs> SystemErrorCleared;
public void Initialize()
{
TimeSpan spanTimer = GetDBBackupTimeSpan(_systemConfiguration.DatabaseBackupFrequencyInDays);
_databaseBackupTimer = new Timer(spanTimer.TotalMilliseconds)
{
AutoReset = false,
};
_databaseBackupTimer.Elapsed += DatabaseBackupTimerOnElapsed;
_databaseBackupTimer.Start();
}
private TimeSpan GetDBBackupTimeSpan(int databaseBackupFrequencyInDays)
{
DateTime dt1 = DateTime.Now;
DateTime dt2 = new DateTime(dt1.Year, dt1.Month,
dt1.Day, 1, 0, 0);
dt2 = dt2.AddDays(_systemConfiguration.DatabaseBackupFrequencyInDays);
TimeSpan spanTimer = dt2 - dt1;
if (spanTimer.TotalMilliseconds < 0) // This could conceivably happen if the have 0 or a negative number (erroneously) for DatabaseBackupFrequencyInDays in configuration.json
{
dt2 = new DateTime(dt1.Year, dt1.Month,
dt1.Day, 1, 0, 0);
dt2 = dt2.AddDays(1);
spanTimer = dt2 - dt1;
}
return spanTimer;
}
public void PerformDatabaseMaintenance()
{
if (BackupCurrentDatabase())
{
var success = CleanupExpiredData();
if (success)
{
Logger.Log(LogLevel.Info, string.Format("Database Maintenance succeeded"));
NotifySystemError(ErrorLevel.Log, ErrorCode.DatabaseBackupComplete, "Database backup completed");
}
}
}
public void Dispose()
{
_databaseBackupTimer.Elapsed -= DatabaseBackupTimerOnElapsed;
_databaseBackupTimer.Stop();
_databaseBackupTimer.Dispose();
}
private void DatabaseBackupTimerOnElapsed(object sender, ElapsedEventArgs elapsedEventArgs)
{
try
{
PerformDatabaseMaintenance();
TimeSpan spanTimer = GetDBBackupTimeSpan(_systemConfiguration.DatabaseBackupFrequencyInDays);
_databaseBackupTimer.Interval = Math.Max(spanTimer.TotalMilliseconds, TimeSpan.FromMinutes(10).TotalMilliseconds);
_databaseBackupTimer.Start();
}
catch (Exception e)
{
Logger.Log(LogLevel.Warning,
string.Format("Database Backup Failed: {0}, ",
e.Message));
NotifySystemError(ErrorLevel.Log, ErrorCode.DatabaseBackupFailed,
"Database backup failed ");
_databaseBackupTimer.Start();
}
}
private bool BackupCurrentDatabase()
{
try
{
_repository.Alerts.Count();
}
catch (Exception ex)
{
NotifySystemError(ErrorLevel.Log, ErrorCode.DatabaseBackupFailed, "Database backup failed - the database server does not respond or the database does not exist");
throw new InvalidOperationException(string.Format("The DB does not exist : {0} Error {1}", _systemConfiguration.LocalDbPath, ex.Message));
}
if (!Directory.Exists(_systemConfiguration.LocalBackupFolderPath))
Directory.CreateDirectory(_systemConfiguration.LocalBackupFolderPath);
var tries = 0;
var success = false;
while (!success && tries < MaxRetries)
{
try
{
_repository.BackupDatabase(_systemConfiguration.LocalBackupFolderPath);
success = true;
}
catch (Exception e)
{
Logger.Log(LogLevel.Warning, string.Format("Database Backup Failed: {0}, retrying backup", e.Message));
Thread.Sleep(TimeSpan.FromSeconds(1));
tries++;
if (tries == MaxRetries)
{
NotifySystemError(ErrorLevel.Log, ErrorCode.DatabaseBackupFailed, string.Format("Database backup failed - {0}", e.Message));
}
}
}
var backupDirectory = new DirectoryInfo(_systemConfiguration.LocalBackupFolderPath);
var files = backupDirectory.GetFiles().OrderBy(f => f.CreationTime).ToArray();
if (files.Length > _systemConfiguration.MaxDatabaseBackups)
{
for (var i = 0; i < (files.Length - _systemConfiguration.MaxDatabaseBackups); i++)
{
try
{
files[i].Delete();
}
catch (Exception e)
{
Logger.Log(LogLevel.Warning, string.Format("Failed to delete old backup: {0}", e.Message));
}
}
}
Logger.Log(LogLevel.Info, success ?
"Database Backup succeeded" :
string.Format("Database Backup failed after {0} retries", MaxRetries));
return success;
}
private bool CleanupExpiredData()
{
var success = false;
try
{
var expirationTime = DateTime.Now - TimeSpan.FromDays(_systemConfiguration.DatabaseDataExpirationInDays);
_repository.DeleteTemperatureReadingsBeforeDate(expirationTime);
_repository.DeleteTransactionsBeforeDate(expirationTime);
success = true;
}
catch (Exception e)
{
Logger.Log(LogLevel.Warning, string.Format("Failed to cleanup expired data: {0}", e.Message));
NotifySystemError(ErrorLevel.Log, ErrorCode.DatabaseBackupFailed, string.Format("Database cleanup of expired data failed - {0}", e.Message));
}
Logger.Log(LogLevel.Info, success ?
string.Format("Database clean up expired data succeeded") :
string.Format("Database clean up expired data failed"));
return success;
}
private void NotifySystemError(ErrorLevel errorLevel, ErrorCode errorCode, string description)
{
var handler = SystemError;
if (handler != null)
{
var systemError = _systemErrorFactory.CreateSystemError(errorLevel, errorCode, description);
handler(this, new SystemErrorEventArgs(systemError));
}
}
}
what I suspect is that Initialize is called multiple time, what you could do is check that the _databaseBackupTimer is null before creating a new instance
if it's not null, just skip the whole code in that method
I think the solution is overly complicated.
The timer interval will fire the Elapsed event independent of when the fact that a previous run had finished or not, unless you explicitly stop the timer during the event. This should not be necessary. You could simply keep track of whether you are running or not when you enter the method.
Set the interval to 59999. This is a millisecond short of a minute. Then at entry in the event handler check if the current hour and minute correspond to the time when you want to backup.
private bool running = false;
private Timer timer = new Timer();
//other code.
private void Initialize()
{
timer.Interval = 59999;
myTimer.Elapsed += TimerElapsed;
timer.Start();
}
public void TimerElapsed(object sender, ElapsedEventArgs e)
{
if (running) return;
DateTime dt = DateTime.Now();
if (!(dt.Hour.Equals(1) && dt.Minute.Equals(0))) return;
running = true;
//other code
running = false;
}
In addition, I would keep the time of running the backup in a configuration file or registry so that if I want to change it, I could do that without re-compiling my service.
When the code is working so laggy it would be very good so that it is not laggy when it works.
How the code works:
It searches the computer for a file is then when it find it to change the file but if the file is running line will loop until it manages to do its job.
Main class
public Form1(string[] Args)
{
InitializeComponent();
backgroundWorker1.RunWorkerAsync();
}
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
Thread.Sleep(1000); // One second.Thread.Sleep(1000); // One second.
MessageBox.Show("Testing");
Fille mc = new Fille();
mc.Search();
}
Fille clss
private static ArrayList list2 = new ArrayList();
private static ArrayList listRemove = new ArrayList();
public void Search()
{
try
{
foreach (string file in Directory.EnumerateFiles(#"C:\Users\user\Downloads\MCFILE\trrtrt\", "*.exe", SearchOption.AllDirectories))
{
// Display file path.
if (SHA1Hash.GetSHA1Hash(file) == "1233456") // fake SHA1Hash
{
try
{
COPYWithReplace(#"C:\Users\user\Downloads\MCFILE\Fake2\Test.exe", file);
}
catch (IOException)
{
// log errors
if (list2.Count == 0)
{
list2.Add(file);
Thread thread = new Thread(new ThreadStart(Test2));
thread.Start();
}
else
{
Thread thread = new Thread(new ThreadStart(Test2));
thread.Abort();
list2.Add(file);
thread.Join();
}
}
}
}
}
catch (Exception ex)
{
// log errors
}
}
private void Test2()
{
if (list2.Count == 0)
{
}
else
{
foreach (string _item in list2)
{
try
{
//Run
COPYWithReplace(#"C:\Users\user\Downloads\MCFILE\Fake2\Test.exe", _item);
listRemove.Add(_item);
}
catch (IOException)
{
//error
}
}
foreach (var Remove in listRemove)
{
list2.Remove(Remove);
}
listRemove.Clear();
if (list2.Count == 0)
{
}
else
{
Thread thread = new Thread(new ThreadStart(Test2));
thread.Start();
}
}
}
I made a new thread because I found the problem. But now it's just that it lags.
I suspect the reason it's "lagging" is because you have the system in a very convoluted but rather processor intensive and I/O intensive loop. If a file fails the first test, your code starts a thread that tries it again. And if that fails then you start another thread to try it again, lather, rinse, repeat.
That's going to absolutely kill performance. You're basically doing this:
while (forever)
{
if I can overwrite the file
{
break;
}
}
Except if you have multiple files that you're trying to write, then you're doing that loop for every file. Concurrently. And you're not just using a loop. Instead, you're starting and stopping threads like nobody's business.
Yeah, that's going to slow down your computer.
A more reasonable way to do this would be with a thread to do the first check, and a timer that will limit how often you do the other checks. Communication is with a simple queue, because only one thread will be accessing it at a time.
Here's what I would recommend:
private static Queue<string> filesToCheck = new Queue<string>();
private System.Timers.Timer copyTimer;
public void Search()
{
try
{
foreach (string file in Directory.EnumerateFiles(#"C:\Users\user\Downloads\MCFILE\trrtrt\", "*.exe", SearchOption.AllDirectories))
{
// Display file path.
if (SHA1Hash.GetSHA1Hash(file) == "1233456") // fake SHA1Hash
{
if (!TryToCopy(file)) // See function below
{
filesToCheck.Enqueue(file);
}
}
}
// Checked all the files once.
// If there are any in the queue, start the timer.
if (filesToCheck.Count > 0)
{
copyTimer = new System.Timers.Timer(CopyTimerProc, null, 1000, Timeout.Infinite);
}
}
catch (Exception)
{
// do your error handling
}
}
private void CopyTimerProc(object state)
{
string filename = filesToCheck.Dequeue();
if (TryToCopy(filename))
{
// success. If the queue is empty, kill the timer.
if (filesToCheck.Count == 0)
{
copyTimer.Dispose();
}
}
else
{
// File still locked.
// Put it back on the queue and reset the timer.
filesToCheck.Enqueue(filename);
copyTimer.Change(1000, 0);
}
}
private bool TryToCopy(string filename)
{
try
{
COPYWithReplace(#"C:\Users\user\Downloads\MCFILE\Fake2\Test.exe", filename);
return true;
}
catch (IOException)
{
// log error
return false;
}
}
The timer is a one-shot that is reset after each time it ticks. The reason I did it this way is to prevent another tick coming along while the previous tick is still processing. After all, it takes time to copy a file.
There's no reason to do this with a bunch of threads. The file system can only do one thing at a time, anyway, and it's not like an extra second or two while you wait for a file to become available is going to hurt anything.
Trying to convert XML files using XSL and printing the output. However, receiving the following message: The calling thread cannot access this object because a different thread owns it.
To set an interval for checking files, added a timer to the OnStart.
if (findPrinter() > 0)
{
System.Timers.Timer printNetterCheck = new System.Timers.Timer();
printNetterCheck.Elapsed += new ElapsedEventHandler(OnTimedEvent);
printNetterCheck.Interval = 30000;
printNetterCheck.Enabled = true;
}
The OnTimedEvent:
private void OnTimedEvent(object source, ElapsedEventArgs e)
{
getFiles();
}
If any files available, call print:
foreach (string file in files)
{
try
{
StringWriter xslTransformResult = new StringWriter();
XslCompiledTransform xslt = new XslCompiledTransform();
xslt.Load(xslPath);
xslt.Transform(file, null, xslTransformResult);
if (print(xslTransformResult) == 1)
{
//do some things
The print function:
private int print(StringWriter transformedXML)
{
//assume OK
int rc = 1;
try
{
StringReader printNetterStreamReader = new StringReader(transformedXML.ToString());
PrintSystemJobInfo printNetterJob = printer.AddJob("PrintNetterPrint");
Stream printNetterStream = printNetterJob.JobStream;
Byte[] printNetterByteBuffer = UnicodeEncoding.Unicode.GetBytes(printNetterStreamReader.ReadToEnd());
printNetterStream.Write(printNetterByteBuffer, 0, printNetterByteBuffer.Length);
printNetterStream.Close();
}
catch (Exception e)
{
//return fail
rc = -1;
eventLog.WriteEntry("Error printing: " + e.Message, EventLogEntryType.Error);
}
return rc;
}
When calling print I receive the thread error. Found some stuff about Dispatchers etc.. but those are not available when using services.
Check PrintQueue.AddJob.
The method makes a COM call which requires the application be running in a single apartment (STA). The easiest way to fix that is to add STAThreadAttribute to Main which will force the application to run in a single thread. If you need multithreading in your application then you will need to implement the necessary plumbing to run the PrintQueue separately on an STA Thread.
// Create a factory to hold your printer configuration
// So that it can be retrieved on demand
// You might need to move the findPrinter() logic
public class PrintQueueFactory
{
private static PrintQueue _instance = new PrinterQueue(/* Details */);
public static PrintQueue PrintQueue { get { return _instance; } }
}
private int print(StringWriter transformedXML)
{
//assume OK
int rc = 1;
try
{
var printer = PrintQueueFactory.PrintQueue;
var staThread = new Thread(() => Print(printer, transformedXML.ToString()));
staThread.SetApartmentState(ApartmentState.STA);
staThread.Start();
staThread.Join();
}
catch (Exception e)
{
//return fail
rc = -1;
eventLog.WriteEntry("Error printing: " + e.Message, EventLogEntryType.Error);
}
return rc;
}
private static void Print(PrintQueue printer, string lines)
{
using(var printNetterJob = printer.AddJob("PrintNetterPrint"))
using(var printNetterStreamReader = new StringReader(lines))
using(var printNetterStream = printNetterJob.JobStream)
{
Byte[] printNetterByteBuffer = UnicodeEncoding.Unicode.GetBytes(printNetterStreamReader.ReadToEnd());
printNetterStream.Write(printNetterByteBuffer, 0, printNetterByteBuffer.Length);
}
}
maybe, as you are using a Timer control, it is related with multi-threading, maybe you should check if an Invoke is Required (InvokeRequired) in the Timer.Elapsed event handler.
If so, you should create a delegate to call this function, so it can be executed in the right thread.
Check this Invoke-Required question