I have a service that processes files. Sometimes they process very quickly and sometimes they take a very long time. I can't control the process that sends the files to me. The are randomly dropped on me throughout the day and night. When I use the timer, it seems like the "ProcessFiles" method is abandoned whereever it is when the time has elapsed and ProcessFiles is called again. Since the files contain sensitive information, they can't sit on the server for a long time so I can't set the timer for any longer than 5 minutes and, still, at 5 minutes, the process sometimes interrupts itself. As a result, I have partially processed files. I would appreciate any thoughts and input on this quandary.
System.Timers.Timer _timer;
// As the files come in, massage them and encrypt them
public const string InPath = #"c:\input";
public const string OutPath = #"\c:\output";
public FileMassaging()
{
InitializeComponent();
}
public EventLog MyEventLog = new EventLog();
public string sSource = "FileMassaging";
public string sLog = "FileMassaging";
protected override void OnStart(string[] args)
{
// Create the source, if it does not already exist.
if (!EventLog.SourceExists(sSource))
EventLog.CreateEventSource(sSource, sLog);
// set up the service
ServiceStatus serviceStatus = new ServiceStatus();
serviceStatus.dwCurrentState = ServiceState.SERVICE_START_PENDING;
serviceStatus.dwWaitHint = 100000;
SetServiceStatus(this.ServiceHandle, ref serviceStatus);
// set up the service
_timer = new System.Timers.Timer();
_timer.Elapsed += new System.Timers.ElapsedEventHandler(this.OnTimer);
_timer.Interval = 5000;
_timer.Start();
// Update the service state to Running.
serviceStatus.dwCurrentState = ServiceState.SERVICE_RUNNING;
SetServiceStatus(this.ServiceHandle, ref serviceStatus);
}
public void OnTimer(object sender, System.Timers.ElapsedEventArgs args)
{
ProcessFiles();
}
public void ProcessFiles()
{
string[] originalFiles = Directory.GetFiles(InPath + #"\", "*.txt");
foreach (string fileName in originalFiles)
{
// Check and process the file
CheckFile(InPath, OutPath, fileName);
}
}
public void CheckFile(string InPath,Outpath, fileName)
{
// for example only -- actual file processing is much longer
//
string infile = InPath+fileName;
string outfile= OutPath+fileName;
File.Move(infile,outfile,true);
}
For testing and extensiblity I would recommend a different overall structure.
First let's seperate out this work into classes where it makes sense. Lets start with a class called FolderWatcher:
public class FolderWatcher
{
private readonly string _inPath;
private readonly string _outPath;
public bool CurrentlyRunning { get; set; }
public FolderWatcher(string inPath, string outPath)
{
_inPath = inPath;
_outPath = outPath;
}
public void TryProcessFiles(object sender, ElapsedEventArgs e)
{
try
{
this.CurrentlyRunning = true;
ProcessFiles(sender, e);
}
catch (Exception)
{
throw;
}
finally
{
this.CurrentlyRunning = false;
}
}
public void ProcessFiles(object sender, ElapsedEventArgs e)
{
string[] originalFiles = GetFilesInDirectory();
foreach (var originalFile in originalFiles)
{
CheckFile(originalFile);
}
}
// Internal/Virtual so that this can mocked in unit testing.
internal virtual string[] GetFilesInDirectory()
{
return Directory.GetFiles(_inPath + #"\", "*.txt");
}
// Internal/Virtual so that this can mocked in unit testing.
internal virtual void CheckFile(string fileName)
{
string infile = $"{_inPath}{fileName}";
string outfile = $"{_outPath}{fileName}";
File.Move(infile, outfile);
}
}
This class has a single responsibility, to move files in response to an event.
Next let's build a class to wrap the FolderWatcher class that handles the timer functionality:
public class TimedFolderWatcher
{
private readonly FolderWatcher _folderWatcher;
private readonly Timer _timer;
public TimedFolderWatcher(FolderWatcher folderWatcher)
{
_folderWatcher = folderWatcher;
InitTimer();
}
private void InitTimer()
{
_timer.Elapsed += new System.Timers.ElapsedEventHandler(this.ProcessFiles);
_timer.Interval = 5000;
_timer.Start();
}
private void ProcessFiles(object sender, ElapsedEventArgs e)
{
_folderWatcher.TryProcessFiles(sender, e);
}
}
This class also has a single responsibility to call the ProcessFiles method every 5000 milliseconds.
Lastly we can init and call these classes this way:
var fileMassageService = new TimedFolderWatcher(new FolderWatcher(#"c:\input", #"c:\output"));
This approach lends itself to testing and follows the best practices of Dependency Injection which will allow you to use an IOC framework in the future if you need to.
From the MSDN Page on Timers try this (https://msdn.microsoft.com/en-us/library/system.timers.timer.interval(v=vs.110).aspx)
_timer = new System.Timers.Timer(5);
_timer.Elapsed += OnTimer;
_timer.AutoReset = true;
// _timer.Start();
_timer.Enable = true;
private static void OnTimer(object sender, System.Timers.ElapsedEventArgs args) { ProcessFiles(); }
Related
I have a windows service that basically monitors a folder, copies files to a local directory processes the files. The service starts up fine enough calls the Watcher Class and does all of the configuration like it should. So my code works for the setup. However I get 0 events firing. I am wondering if when my StartFolderWatcher() method goes out of scope that my object - even though declared at the class level is somehow getting disposed.
So do I need to run the object continuously in a separate thread ? If so can I get an example of how I should do that Task.Run(()=> myObject)
Relevant Code is below - if more is needed just ask and I will post it.
static class Program
{
static void Main(string[] args)
{
JobProcessor service = new JobProcessor();
if (Environment.UserInteractive)
{
service.RunAsConsole(args);
}
}
private static bool IsDebugMode(string[] args)
{
if (args == null) return false;
if (args[0].ToLower() == "/debug") return true;
return false;
}
}
public partial class JobProcessor : ServiceBase
{
string[] folders = new string[] {"InBoxFolder"};
HotFolderWatch HFW = new HotFolderWatch();
public JobProcessor()
{
InitializeComponent();
}
protected override void OnStart(string[] args)
{
StartFolderWatcher();
}
public void StartFolderWatcher()
{
FileWatcherInfo[] ServiceWatchers = new FileWatcherInfo[4];
ServiceConfiguration sc = new ServiceConfiguration();
for (int i = 0; i < folders.Length; i++)
{
ServiceWatchers[i] = sc.GetWatchFolderSettings(folders[i]);
}
HFW = new HotFolderWatch(ServiceWatchers[0]);
HFW.ReadyToProcess += ReadyToProcess;
HFW.InBoxFolderDisconnected += OnInBoxFolderDisconnected;
HFW.LocalFolderDisconnected += OnLocalFolderDisconnected;
HFW.ProcessFolderDisconnected += OnProcessFolderDisconnected;
}
public void RunAsConsole(string[] args)
{
OnStart(args);
Console.WriteLine("Press any key to exit...");
Console.ReadLine();
OnStop();
}
}
public HotFolderWatch(FileWatcherInfo inbox)
{
this.InboxCacheTimeMilliseconds = inbox.CacheTimeMilliseconds;
this.InBoxFolder = inbox.Folder.Trim();
this.InboxFileFilter = inbox.Filter.Trim();
SetInboxWatcher();
}
private void SetInboxWatcher()
{
InBoxWatcher = new FileSystemWatcher(InBoxFolder, InboxFileFilter);
InBoxWatcher.IncludeSubdirectories = false;
InBoxWatcher.NotifyFilter =
NotifyFilters.LastAccess | NotifyFilters.LastWrite;
InboxCache = MemoryCache.Default;
InboxCachePolicy = new CacheItemPolicy()
{
RemovedCallback = OnRemovedFromInBoxCache
};
InBoxWatcher.Created += new FileSystemEventHandler(OnInBoxChanged);
InBoxWatcher.EnableRaisingEvents = true;
}
private void OnInBoxChanged(object source, FileSystemEventArgs e)
{
InboxCachePolicy.AbsoluteExpiration =
DateTimeOffset.Now.AddMilliseconds(InboxCacheTimeMilliseconds);
InboxCache.AddOrGetExisting(e.Name, e, InboxCachePolicy);
}
}
If in my HotFolderWatch Class I create a FileSystemWatcher in a method as below the code works!
private void CreateWatcher()
{
FileSystemWatcher fsw = new FileSystemWatcher(#"C:\Tests\JobQueue\InFolder","*.txt");
fsw.Created += Fsw_Created;
fsw.EnableRaisingEvents = true;
}
private void Fsw_Created(object sender, FileSystemEventArgs e)
{
string ex = e.FullPath;
WatcherChangeTypes ctmp = e.ChangeType;
// throw new NotImplementedException();
}
Turns out the default constructor of FileSystemWatcher uses the filter FileName by default and I was thinking I would get a last Write or Last Access with out concerning myself with the Name.
I received good information from here :
https://www.codeproject.com/Articles/1220094/NotifyFilters-Enumeration-Explained-FileSystemWatc
The article posted in the link is what helped me to solve my issue.
I had to Ste the NotifyFilters.FileName as one of the filters.
I am implementing a winform application that is supposed to do the following:
OnLoad, open an XML file reading required information while watching for any new changes to the file (incoming new information)
A timer performing certain actions based on the nature of the provided information from the XML file. Please note that the nature of the action depends on the content of the XML file
What is the best way to achieve this? Two threads? Async? Some starting point would be much appreciated.
I think you need like this structure. Firstly you read XML file and configure your custom MyConfigurationClass object. After this point you can configure FileSystemWatcher object. And finally, you can configure your task schedular object with your desired time interval.
public partial class MainWindow : Window
{
MyConfigurationClass configuration;
string filePath = #"./some.xml";
FileSystemWatcher fileWatcher = new FileSystemWatcher();
System.Timers.Timer timer = new System.Timers.Timer();
public MainWindow()
{
// First read action action for
var task = Task.Run(() => ReadXML());
InitializeComponent();
FileWatherConfigure();
TimerConfigure(task.Result);
}
private void FileWatherConfigure()
{
fileWatcher.Path = System.IO.Path.GetDirectoryName(filePath);
fileWatcher.Filter = System.IO.Path.GetFileName(filePath);
fileWatcher.Changed += FileWatcher_Changed;
fileWatcher.EnableRaisingEvents = true;
}
private void TimerConfigure(SomeClass someClass)
{
timer.Elapsed += new ElapsedEventHandler(OnTimedEvent);
timer.Interval = configuration.TimeInterval.TotalMilliseconds;
timer.Enabled = true;
}
private void FileWatcher_Changed(object sender, FileSystemEventArgs e)
{
Thread.Sleep(TimeSpan.FromSeconds(1));
SomeClass someClass = ReadXML();
// Do whatever you want file change
}
private void OnTimedEvent(object sender, ElapsedEventArgs e)
{
timer.Stop();
try
{
// Schedule Operation
}
catch (Exception ex)
{
}
timer.Start();
}
private SomeClass ReadXML()
{
// Read file and do what ever you want
}
}
public class SomeClass
{
// Data from XML
}
public class MyConfigurationClass
{
public TimeSpan TimeInterval { get; set; }
}
I have the following code:
namespace SSS.RemoteTruckService
{
public partial class Startup : Form
{
private Timer _gpsTimer;
private Timer _ppsTimer;
private Timer _creditCardTimer;
private Timer _iniTimer;
public string Message
{
get { return richTextBox_Message.Text; }
set
{
richTextBox_Message.Invoke((MethodInvoker)(()
=> richTextBox_Message.Text = DateTime.Now + " " +
value + Environment.NewLine + richTextBox_Message.Text));
}
}
public Startup()
{
InitializeComponent();
}
private void ButtonStartClick(object sender, EventArgs e)
{
StartRemoteTruck();
}
private void ButtonPauseClick(object sender, EventArgs e)
{
if (_gpsTimer.Enabled) _gpsTimer.Enabled = false;
if (_ppsTimer.Enabled) _ppsTimer.Enabled = false;
if (_creditCardTimer.Enabled) _creditCardTimer.Enabled = false;
if (_iniTimer.Enabled) _iniTimer.Enabled = false;
ProcessIniFile.StopProcess();
}
public void StartRemoteTruck()
{
Message = "RemoteTruck started.";
if (Settings.GlobalSettings == null)
{
Message = "GlobalSettings was null or not loaded. Cannot continue.";
Logging.Log("GlobalSettings was null or not loaded. Cannot continue.", "RemoteTruck", Apps.RemoteTruckService);
Environment.Exit(0);
}
if (Settings.GlobalSettings.IniFileWatcherEnabled)
{
ProcessIniFile.StartProcess();
}
CreateTimers();
}
And in the ProcessIniFile.StartProcess() I have the code:
namespace SSS.RemoteTruckService.inifile
{
public static class ProcessIniFile
{
private static DateTime _iniLastWriteTime;
private static readonly string Inifile = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Windows), "sss.ini");
private static FileSystemWatcher _watcher;
public static void StartProcess()
{
ReadIniFile();
SaveCurrentIniReadings();
CreateIniFileWatcher();
}
public static void StopProcess()
{
if (_watcher != null)
{
_watcher.EnableRaisingEvents = false;
_watcher = null;
}
}
private static void CreateIniFileWatcher()
{
_watcher = new FileSystemWatcher
{
Path = Environment.GetFolderPath(Environment.SpecialFolder.Windows),
NotifyFilter = NotifyFilters.LastWrite,
Filter = "sss.ini"
};
_watcher.Changed += SssIniWatcherChanged;
_watcher.EnableRaisingEvents = true;
}
I'd like to pass back the the calling form the status of the reads of the file watcher.
Maybe I'm overthinking this, but if I want to add to the Message on the main form, how do I get to it?
You can use Events for that. Your process can send events and your form can handle them.
More info: http://msdn.microsoft.com/en-us/library/awbftdfh.aspx
The simple but not pretty way I like to use is to make that part of the form static as well. For example, creating a static variable WriteMessage, and in your Form Load or Startup(), you can set it:
WriteMessage = (s) => Message = s;
Sure this has some issues, but it's a quick way to get it done. One of those issues is that, you may need to use Dispatcher.invoke if you're not on the UI thread.
I have implemented a SqlListener class that uses SqlDependency to wait for changes in SQL database. At one point in my business workflow I need to wait for a record turning up in the database. The SqlListener triggers an event when requested record is found. This works fine. I can make it work by entering a While-loop and wait until I detect the event being returned. But this is not ideal design. It makes the processor spin a lot in vain.
I would like to wait for the event in a more intelligent manner. I read a lot of suggestions on using Task, NotificationDelegate, ManualResetEvent, etc. .... but I was not able to get it all together.
A simplified example will probably make it easier to understand. This is my current setup that works. But if possible I would like to get rid of the ugly while loop.
private const int MaxWaitTime = 5;
private SqlListener<RecordType> _recordListener;
private RecordType _record;
/// <summary>
/// Request a record and wait until it is found.
/// </summary>
public RecordType GetRecordAwait(int requestedId)
{
// Initiate listening for record
_recordListener = new SqlListener<RecordType>();
_recordListener.SqlModified += SqlListener_SqlModified;
_recordListener.StartListening(requestedId);
// Wait until record is found
var startTime = DateTime.Now;
while (_record == null &&
DateTime.Now.Subtract(startTime).TotalSeconds < MaxWaitTime)
{
Thread.Sleep(1);
}
// Stop listening
_recordListener.SqlModified -= SqlListener_SqlModified;
_recordListener.Dispose();
_recordListener = null;
// Return record
return _record;
}
private void SqlListener_SqlModified(object sender, SqlModifiedArgs args)
{
_record = (RecordType)args.Record;
}
Instead of using While, you could go with Timer and events. Something like:
public class ListenerWaiting
{
public ListenerWaiting(int waitingTimeSeconds)
{
_waitSeconds = waitingTimeSeconds;
}
private int _waitSeconds;
private System.Timers.Timer _timer;
private Listener _listener;
public event EventHandler<string> ListenerDone;
public void Listen(int listeningPeriodSeconds)
{
_listener = new Listener(listeningPeriodSeconds * 1000);
_listener.ListenerCompleted += ListenerListenerCompleted;
_timer = new System.Timers.Timer(_waitSeconds * 1000) {Enabled = true};
_timer.Elapsed += TimerElapsed;
}
void ListenerListenerCompleted(object sender, string e)
{
StopTimer();
StopListener();
if (ListenerDone != null)
ListenerDone(this, "Waiting success! Message was: " + e);
}
void TimerElapsed(object sender, System.Timers.ElapsedEventArgs e)
{
StopTimer();
StopListener();
if (ListenerDone != null)
ListenerDone(this, "Waited longer than set, aborted waiting...");
}
private void StopTimer()
{
_timer.Stop();
_timer.Elapsed -= TimerElapsed;
_timer = null;
}
private void StopListener()
{
_listener.ListenerCompleted -= ListenerListenerCompleted;
_listener = null;
}
}
public class Listener
{
private System.Timers.Timer _timer;
private string _listeningPeriodSeconds;
public event EventHandler<string> ListenerCompleted;
public Listener(int listeningPeriodSeconds)
{
_listeningPeriodSeconds = listeningPeriodSeconds.ToString();
_timer = new System.Timers.Timer(listeningPeriodSeconds) { Enabled = true };
_timer.Elapsed += TimerElapsed;
}
private void TimerElapsed(object sender, System.Timers.ElapsedEventArgs e)
{
_timer.Elapsed -= TimerElapsed;
_timer = null;
if (ListenerCompleted != null)
ListenerCompleted(this, _listeningPeriodSeconds);
}
}
...and then consume it with:
static void Main(string[] args)
{
var wait = new ListenerWaiting(5);
wait.ListenerDone += WaitListenerDone;
wait.Listen(3);
Console.ReadLine();
}
static void WaitListenerDone(object sender, string e)
{
Console.WriteLine(e);
}
I guess I could find better names for classes, but you'll get the idea ;)
In fact the solution was more simple than I first thought. When I rephrased my question and searched again I found it. The ManualResetEvent as mentioned already in my question turned out to be the simplest way to solve it.
All I had to do was to add a ManualResetEvent and set it to wait ;-)
private const int MaxWaitTime = 5000;
private SqlListener<RecordType> _recordListener;
private RecordType _record;
private readonly ManualResetEvent _recordWaiter = new ManualResetEvent(false);
/// <summary>
/// Request a record and wait until it is found.
/// </summary>
public RecordType GetRecordAwait(int requestedId)
{
// Initiate listening for record
_recordListener = new SqlListener<RecordType>();
_recordListener.SqlModified += SqlListener_SqlModified;
_recordListener.StartListening(requestedId);
// Wait synchronously until record is found
_recordWaiter.WaitOne(MaxWaitTime);
// Stop listening
_recordListener.SqlModified -= SqlListener_SqlModified;
_recordListener.Dispose();
_recordListener = null;
// Return record
return _record;
}
private void SqlListener_SqlModified(object sender, SqlModifiedArgs args)
{
_record = (RecordType)args.Record;
_recordWaiter.Set();
}
I'm trying to create a simple app, what moves all files writed to some directory to other directory. That's my problem: if i write other than 10000 files at once in my directory(small .txt files over 1KB) - some of them not handling to move on output directory. I'm using FileSystemWatcher events handler to solve this problem. Here is my code example:
Class MyProgramm
{
void Process(Object o, FileSystemEventArgs e)
{
//do something with e.Name file
}
void main()
{
var FSW = New FileSystemWatcher{Path = "C:\\InputDir"};
FSW.Created += Process;
FSW.EnableRisingEvents = true;
Thread.Sleep(Timeout.Infinite);
}
}
Finally, we got some files processed, but some of written files stays unprocessed..
Any suggestions?
I had similar issues with the FileSystemWatcher. It seemed to "drop the ball" quite regularly. I went for an alternative solution by extending "ServiceBase", which has been working consistently since it went live as a windows service. Hope this helps:
public partial class FileMonitor : ServiceBase
{
private Timer timer;
private long interval;
private const string ACCEPTED_FILE_TYPE = "*.csv";
public FileMonitor()
{
this.ServiceName = "Service";
this.CanStop = true;
this.CanPauseAndContinue = true;
this.AutoLog = true;
this.interval = long.Parse(1000);
}
public static void Main()
{
ServiceBase.Run(new FileMonitor());
}
protected override void OnStop()
{
base.OnStop();
this.timer.Dispose();
}
protected override void OnStart(string[] args)
{
AutoResetEvent autoEvent = new AutoResetEvent(false);
this.timer = new Timer(new TimerCallback(ProcessNewFiles), autoEvent, interval, Timeout.Infinite);
}
private void ProcessNewFiles(Object stateInfo)
{
DateTime start = DateTime.Now;
DateTime complete = DateTime.Now;
try
{
string directoryToMonitor = "c:\mydirectory";
DirectoryInfo feedDir = new DirectoryInfo(directoryToMonitor);
FileInfo[] feeds = feedDir.GetFiles(ACCEPTED_FILE_TYPE, SearchOption.TopDirectoryOnly);
foreach (FileInfo feed in feeds.OrderBy(m => m.LastWriteTime))
{
// Do whatever you want to do with the file here
}
}
finally
{
TimeSpan t = complete.Subtract(start);
long calculatedInterval = interval - t.Milliseconds < 0 ? 0 : interval - t.Milliseconds;
this.timer.Change(calculatedInterval, Timeout.Infinite);
}
}
}