Out-of-mem-Exception in c# GUI when using FileSystemWatcher - multithreading - c#

I build a windows-forms-app where I (try to) do extensive calculations on images whenever they are created in a specific directory which I watch using the FileSystemWatcher.
private void OnNewFileInDir(object source, FileSystemEventArgs evtArgs)
{
//Load the actual image:
imageFilepath = evtArgs.FullPath; //imageFilepath is a private class string var
Image currentImage = Image.FromFile(imageFilepath);
//Display the image in the picture box:
UpdatePictureBox(currentImage); //Method to update the GUI with invoking for the UI thread
//Extensive Calculation on the images
Image currentResultImage = DoExtensiveWork(currentImage);
// Put the current result in the picture box
UpdatePictureBox(currentResultImage );
//dispose the current/temporary image
currentImage.Dispose();
}
The event is fired correctly when pasting a new file into the directory. But I get a "System.OutOfMemoryException" on the line
Image currentImage = Image.FromFile(imageFilepath);
When I put exactly this code (using the same filepath) in a button event (so not using the FileSystemWatcher) everything works fine. So I thought there is some issue regarding the thread since the extensive calculation is then called by the FileSystemWatcher-Thread not by the UI thread.
I tried things like:
//TRY 1: By executing a button click method containg the code
pb_Calculate_Click(this, new EventArgs()); //This does not work eigther --> seems to be a problem with "Who is calling the method"
//TRY 2: Open a new dedicated thread for doing the work of the HistoCAD calculations
Thread newThread_OnNewFile = new Thread(autoCalcAndDisplay);
newThread_OnNewFile.Start();
//TRY 3: Use a background worker as a more safe threading method(?)
using (BackgroundWorker bw = new BackgroundWorker())
{
bw.DoWork += new DoWorkEventHandler(bw_DoWork);
if (bw.IsBusy == false)
{
bw.RunWorkerAsync();
}
}
Unfortunalty none of them worked reliable. 1st not at all. 2nd works only from time to time and 3rd one as well.
Do some of you know whats going on there? What can I do to make it work correctly? Thanks!
EDIT:
Thanks for the comments:
I also tried to call GC.collect() on every event and tried to include using() and dispose() wherever I can. When I'm doing the process manually (with buttons) it works even when processing a lot of files one after another. But when done with the eventhandler I sometimes get the outOfMem-Exception even on the very first file I copy in the folder. File is always the same BMP with 32MB. This is the memory usage for processing one image:
EDIT 2:
I created a minimal example (GUI with one picture Box and one Checkbox in buttonstyle). It turns out that the same thing is happening. The OutOfMemException occured at the same line (Image...). Especially for large BMPs the exception occours nearly always:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
namespace MinimalExampleTesting
{
public partial class Form1 : Form
{
private string imageFilepath;
private string autoModePath = #"C:\Users\Tim\Desktop\bmpordner";
//Define a filesystem watcher object
private FileSystemWatcher watcher;
public Form1()
{
InitializeComponent();
/*** Creating as FileSystemEventArgs watcher in order to monitor a specific folder ***/
watcher = new FileSystemWatcher();
Console.WriteLine(watcher.Path);
// set the path if already exists, otherwise we have to wait for it to be set
if (autoModePath != null)
watcher.Path = autoModePath;
// Watch for changes in LastAccess and LastWrite times and renaming of files or directories.
watcher.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite
| NotifyFilters.FileName | NotifyFilters.DirectoryName;
// Only watch for BMP files.
watcher.Filter = "*.bmp";
// Add event handler. Only on created, not for renamed, changed or something
// Get into the list of the watcher. Watcher fires event and "OnNewFileCreatedInDir" will be called
watcher.Created += new FileSystemEventHandler(OnNewFileInDir);
}
private void tb_AutoMode_CheckedChanged(object sender, EventArgs e)
{
//First of all test if the auto mode path is set and correctly exists currently:
if (!Directory.Exists(autoModePath) || autoModePath == null)
{
MessageBox.Show("Check if Auto Mode path is correctly set and if path exists",
"Error: Auto Mode Path not found");
return;
}
// Begin watching if the AutoModePath was at least set
if (autoModePath != null)
{
watcher.EnableRaisingEvents = tb_AutoMode.Checked; //Since we have a toogle butten, we can use the 'checked' state to enable or disable the automode
}
}
private void OnNewFileInDir(object source, FileSystemEventArgs evtArgs)
{
Console.WriteLine("New file in detected: " + evtArgs.FullPath);
//Force a garbage collection on every new event to free memory and also compact mem by removing fragmentation.
GC.Collect();
//Set the current filepath in the class with path of the file added to the folder:
imageFilepath = evtArgs.FullPath;
//Load the actual image:
Image currentImage = Image.FromFile(imageFilepath);
UpdatePictureBox(currentImage);
}
private void UpdatePictureBox(Image img)
{
if (pictureBox_Main.InvokeRequired)
{
MethodInvoker mi = delegate
{
pictureBox_Main.Image = img;
pictureBox_Main.Refresh();
};
pictureBox_Main.Invoke(mi);
}
else { //Otherwise (when the calculation is perfomed by the GUI-thread itself) no invoke necessary
pictureBox_Main.Image = img;
pictureBox_Main.Refresh();
}
img.Dispose();
}
}
}
Thanks in advance for further hints :)

SOLVED:
The issue seems to be, that event is fired immediately but the file is not yet finally copied. That means we have to wait until the file is free.
A Thread.Sleep(100) at the start of the event does the job. As I now know what to google for, I found two links:
This and this where you can find:
The OnCreated event is raised as soon as a file is created. If a file is being copied or transferred into a watched directory, the OnCreated event will be raised immediately, followed by one or more OnChanged events
So, what works best for my case, was to include a method to test if the file is still locked and than wait at the beginning of the event for an unlock of the file. No need for an additional thread or a BackgroundWorker.
See the code:
private void OnNewFileInDir(object source, FileSystemEventArgs evtArgs)
{
Console.WriteLine("New file detected: " + evtArgs.FullPath);
//Wait for the file to be free
FileInfo fInfo = new FileInfo(evtArgs.FullPath);
while (IsFileLocked(fInfo))
{
Console.WriteLine("File not ready to use yet (copy process ongoing)");
Thread.Sleep(5); //Wait for 5ms
}
//Set the current filepath in the class with path of the file added to the folder:
imageFilepath = evtArgs.FullPath;
//Load the actual image:
Image currentImage = Image.FromFile(imageFilepath);
UpdatePictureBox(currentImage);
}
private static bool IsFileLocked(FileInfo file)
{
FileStream stream = null;
try
{
//try to get a file lock
stream = file.Open(FileMode.Open, FileAccess.ReadWrite, FileShare.None);
}
catch (IOException)
{
//File isn't ready yet, so return true as it is still looked --> we need to keep on waiting
return true;
}
finally
{
if (stream != null){
stream.Close();
stream.Dispose();
}
}
// At the end, when stream is closed and disposed and no exception occured, return false --> File is not locked anymore
return false;
}
Nevertheless: Thanks for your help...it got me on the right track;)

As MSDN says about FileSystemWatcher:
Common file system operations might raise more than one event. For example, when a file is moved from one directory to another, several OnChanged and some OnCreated and OnDeleted events might be raised. Moving a file is a complex operation that consists of multiple simple operations, therefore raising multiple events. Likewise, some applications (for example, antivirus software) might cause additional file system events that are detected by FileSystemWatcher.
Maybe your Image is loaded severals times.
To test it, you can add this line after imageFilepath = evtArgs.FullPath;
imageFilepath = evtArgs.FullPath;
Task.Run(()=>{MessageBox.Show(imageFilepath);});
This will inform you about the fact that the Created event is fired, and will not hold up your program.
Edit
Put your line of code that give the OutOfMemory in a Try Catch.
Like this and this questions describes, you can get this error if your image is corrupt.

Related

How can I make GUI responsive during long iteration?

To begin with, I'm relatively new to programming. I went through some introductory C# training for my new job, and it's the first language I've worked with.
I recently had a business problem that I decided to solve using C#, both to save time (I had hoped) and to learn more C# in the process. The business problem I mentioned was this: I had 600+ Word files that I needed to audit. For each document, I had to make sure that...
There was no text with strike-through anywhere in the document.
Track Changes was disabled.
There were no pending changes (as in changes that were made while
Track Changes was enabled and have yet to be accepted or
rejected).
There were no comments.
It would have been fastest to have my program iterate through all of the documents, making changes as it went along. But because of the nature of this assignment I wanted to make the changes manually, limiting the program's use to generating a list of files (out of the 600) where changes were necessary, and detailing what changes needed to be made for each of those files.
So, I have a button that calls up a FolderBrowserDialog.
private void AddFolderButtonClick(object sender, EventArgs e)
{
var folderBrowser = new FolderBrowserDialog();
if (folderBrowser.ShowDialog() != DialogResult.OK)
{
return;
}
this.progressBar1.Visible = true;
this.progressBar1.Style = ProgressBarStyle.Marquee;
this.Cursor = Cursors.WaitCursor;
var args = new List<string>(Directory.EnumerateDirectories(folderBrowser.SelectedPath));
// Get list of files in selected directory, adding to list of directories
args.AddRange(Directory.EnumerateFiles(folderBrowser.SelectedPath));
this.displayListBox.BeginUpdate();
foreach (string path in args)
{
if (File.Exists(path))
{
// This path is a file
this.ProcessFile(Path.GetFullPath(path));
}
else if (Directory.Exists(path))
{
// This path is a directory
this.ProcessDirectory((Path.GetFullPath(path)));
}
else
{
Console.WriteLine(Resources.Finder_Invalid_File_Or_Directory, path);
}
}
this.displayListBox.EndUpdate();
this.progressBar1.Visible = false;
this.progressBar1.Style = ProgressBarStyle.Continuous;
this.Cursor = Cursors.Default;
}
Together, the following two methods iterate through all subdirectories and files to create a full list of all files below the top level directory selected through the FolderBrowserDialog:
private void ProcessDirectory(string targetDirectory)
{
// Process the list of files found in the directory.
string[] fileEntries = Directory.GetFiles(targetDirectory);
foreach (string fileName in fileEntries)
{
this.ProcessFile(fileName);
}
// Recurse into subdirectories of this directory.
string[] subdirectoryEntries = Directory.GetDirectories(targetDirectory);
foreach (string subdirectory in subdirectoryEntries)
{
this.ProcessDirectory(subdirectory);
}
}
private void ProcessFile(string path)
{
Console.WriteLine(Resources.Finder_File_Processed, path);
string fileName = Path.GetFileName(path);
if (fileName == null || fileName.StartsWith(#"~$") || this.selectedFilesList.Contains(path))
{
return;
}
this.selectedFilesList.Add(path);
this.filePathsCountLabel.Text = (#"Count: " + this.selectedFilesList.Count);
this.displayListBox.Items.Add(path);
}
Once all this code has run, I get a full list of documents. I click a button and the program does what it's supposed to from here on out. Okay, cool. I mentioned before that half of the reason I chose to use C# to solve this was for the sake of learning. At this point I've got everything I need but what I really want to know is how can I implement threading to make the GUI responsive while the list of files is being generated? I've looked through several examples. They made sense. For some reason I just can't get my head around it for this application though. How can I make the whole process of processing subdirectories and files happen without locking up the GUI?
I believe what you need could be found here.
In short, to use a backgroundworker which does all the work on a separate thread thus prevents GUI freezes, first you instantiate BackgroundWorker and handle the DoWork event. Then you call RunWorkerAsync, optionally with an object argument.
As a skeleton code:
class myClass
{
static BackgroundWorker myBw = new BackgroundWorker();
static void Main()
{
myBw .DoWork += myBw_DoWork;
myBw .RunWorkerAsync ("an argument here");
Console.ReadLine();
}
static void myBw_DoWork (object sender, DoWorkEventArgs e)
{
// This is called on the separate thread, argument is called as e.Argument
// Perform heavy task...
}
}
You have to create a separate thread to process your work. Look at this if you are using .NET 4.0+ or this for older versions.
With Task, you can write
Task.Factory.StartNew(() => DoAction()
where DoAction is your function that starts to process data.
But do not forget to use Invoke, if you want to act with GUI from separate thread. For example, if you want to update some Label text from separate thread, you have to do this
label1.Invoke(() => label1.Text = "Some Text");

DeleteFile throwing IsolatedStorageFileException within a ContextMenu

Writing a WP7.1 (Mango) app in Silverlight for Windows Phone (XAML / C#). I have an exception that I just cannot shake. I have a ListBox displaying an ObservableCollection<string> which displays the file names of XML files from IsolatedStorage. The Listbox control also contains a ContextMenu from the Windows Phone Toolkit. I use it to delete items from my ListBox, and from disk.
The ContextMenu and my delete method works fine normally... But if, whilst on the same page without navigating away, I create a new file (let's call it File1) then create another file (File2), if I then attempt to delete File1, the IsolateStorageFile.DeleteFile method throws an exception stating "An error occurred while accessing IsolatedStorage" with an inner message of null. HOWEVER, if I create File1, then File2. Then delete File2 then File1, it works just fine! ARGH!
If I leave the Page or restart the app again, I can delete the file no problems.
I've stripped back the code to hopefully make it a bit easier to read.
UI Binding Collection field in the code behind.
ObservableCollection<string> Subjects;
Click event calls the write method.
private void Button_Click_AddNewSubject(object sender, RoutedEventArgs e)
{
if (TryWriteNewSubject(NewSubjectNameTextBox.Text))
{
... Manipulate UI
}
}
Method to Add file to IsoStore and Subjects collection. Returns a bool for conditional UI manipulation.
private bool TryWriteNewSubject(string subjectName)
{
... file name error checking
using (var store = IsolatedStorageFile.GetUserStoreForApplication())
{
store.OpenFile(subjectName + ".xml", FileMode.CreateNew);
store.Dispose();
}
Subjects.Add(subjectName);
return true;
}
else return false;
}
else return false;
}
The ContextMenu click event calls the delete file method
private void ContextMenuButton_Click(object sender, RoutedEventArgs e)
{
string subjectName = (sender as MenuItem).DataContext as string;
DeleteFile(subjectName);
}
And my delete method
private void DeleteFile(string subjectName)
{
using (var store = IsolatedStorageFile.GetUserStoreForApplication())
{
string fileName = subjectName + ".xml";
store.DeleteFile(fileName);
Subjects.Remove(subjectName);
}
}
The code is straight forward, I just don't know what I'm missing. :(
You get a IsolatedStorageFileStream from OpenFile. You need to dispose it before another operation can manipulate it.
Btw, a using statement calls dispose for you, so there's no need to call dispose at the end of a using statement.

Application just closes (no error or smth) when run from VS2010

Problem is that application closes without any error, VS stays opened.
I have multiple dynamically created FileSystemWatchers, all of them have eventhandler on "Created" event. So this eventhandler method looks like this :
void watcher_FileCreated(object sender, FileSystemEventArgs e)
{
FileInfo f1 = new FileInfo(e.FullPath);
filesDataGrid.Rows.Add(f1.Name);
foreach (TLPclass table in parameterForm.getParameters)
{
//uses some funcion form another class
}
}
Line which causes program to close is the one where I'm adding File name to DataGridView - filesDataGrid.Rows.Add(f1.Name);
Also runs OK without that line.
Weird thing is that application runs normally, when launched from .exe file in projects folder. I can't see error in my code, but I guess theres something awfully wrong with it, if it doesn't even show error message.
And - what are the most common reasons why program could just shut down with no warnings?
The FileSystemWatcher will trigger the events in a separate thread. The logic inside the event handlers will need to take that fact in consideration and perform any synchronization needed. So you'll need something like this:
private void watcher_FileCreated(object sender, FileSystemEventArgs e)
{
if (filesDataGrid.InvokeRequired)
{
filesDataGrid.Invoke((MethodInvoker)delegate { watcher_FileCreated(sender, e); });
}
else
{
FileInfo f1 = new FileInfo(e.FullPath);
filesDataGrid.Rows.Add(f1.Name);
foreach (TLPclass table in parameterForm.getParameters)
{
//uses some funcion form another class
}
}
}
Wrap your function with a Try{}catch (Exception ex) { } block.
i Would think it is because DataGridRow needs to be created with the variables from the FileSystemWatcher first.
e.g
DataGridViewRow row = filesDataGrid.NewRow();
row["columnname"] = f1.name;
filesDataGrid.Rows.Add(row);

A little help needed catching events being raised inside a seperate thread

To begin, I'm working on a pretty high level file system where I need to be able to (with very pin point accuracy) identify and process files in a timely manner. With that said, I am working on a system that will be using the FileSystemWatcher. The whole problem with the watcher though is the fact that it tends to have issues when throwing events when there are large files involved.
To remedy this I'm working on an abstract class that can handle the files individualy once they are created by the file system watcher.
The current road block that I am running into is that my out of process validation on the file is throwing an event, I'm just having problems catching it.
public abstract class absFile
{
public delegate void FileAvailable(object sender, FileEventArgs e);
public event FileAvailable OnFileAvailable;
public void isAvailable()
{
// Create a new threaded instance of the AvailableCheck private void
// This method will be run out of process to allow other operations to continue.
Thread toAvailableCheck = new Thread(new ThreadStart(AvailableCheck));
// Once the threaded object is created, start it.
toAvailableCheck.Start();
}
private void AvailableCheck()
{
// Declaring the file stream toThisFile to be used later with the File.Open
FileStream toThisFile;
// Declaring and instantiating the attempt counter for the loop
int tiAttemptNumber = 0;
// Declaring the event args for returning the events that are
// used by this object.
FileEventArgs toFileEventArgs = new FileEventArgs();
do {
try
{
// Attempt to open the file. If this fails the try
// will interrupt the processing and it will be caught.
toThisFile = File.Open(this.FilePath, FileMode.Open);
// If the file.open method does not fail, the isFileAvailable
// property will be set to true by updating the mbIsAvailable
// private boolean.
mbIsAvailable = true;
// populate the file event args to send back
// the number of attempts made at the file and the pause lenght
toFileEventArgs.Attempts = tiAttemptNumber;
toFileEventArgs.Pause = this.AttemptPause / 1000;
// This event is called when the file is complete.
// The client application will be able to handle this event.
OnFileAvailable(this, toFileEventArgs);
// close and dispose of the filestream.
toThisFile.Close();
toThisFile.Dispose();
}
catch (Exception toException)
{
// Since the open failed, add 1 to the counter so that
// it will eventually time out.
tiAttemptNumber++;
// Set the isFileAvailable property to false. This property
// will default as false, but as a part of standard, make sure that
// if the open fails that the flag IS set to false by updating the
// mbIsAvailable private boolean.
mbIsAvailable = false;
// Put the thread to sleep for the ammount of time specified
// by the AttemptPause. This will give the file time to finish
// whatever process it is involved in.
Thread.Sleep(this.AttemptPause);
}
// Continue to make attempts until either the file is marked as available
// or the number of current attempts is the same as or greater than the
// AccessAttempts property.
} while (!this.isFileAvailable && this.AccessAttempts > tiAttemptNumber);
}
this is the code that I am running as you can see in the private void AvailableCheck, OnfileAvailable is the delegate called passing back this and the file event args.
now i have inherited this abstract class, and i need to be able to catch that event.
toWatcher.Created += new FileSystemEventHandler(OnCreated);
is called in the main and farther down the code is the following method
private void OnCreated(object source, FileSystemEventArgs e)
{
lstStatus.Invoke(new MethodInvoker(delegate {
lstStatus.Items.Add(DateTime.Now.ToString("g") + " - " + e.Name + " - File Created Event Detected for: " + e.FullPath);
lstStatus.TopIndex = lstStatus.Items.Count - 1;
tgtFile ThisFile = new tgtFile(e.FullPath);
lstStatus.Items.Add(DateTime.Now.ToString("g") + " - " + e.Name + " - Creating tgtFile Object");
}));
}
The instatiation of the tgtFile object is passed the path which makes its way down the pike to the is available method.
as you can see the chance exists for the OnFileAvailable event to be fired from the tgtFile object.
Also as you can see, the the possibility for multiple tgtFile objects to exist in memory at the same time is there as well based on the threading design of the filesystemwatcher.
in my main application then i want to be able to do something like:
public tgtFile ThisFile;
ThisFile.OnFileAvailable += new EventHandler(OnFileAvailable);
but the EventHandler errors out, and that is where I am stuck.
If it is giving you a compiler error, it's probably because your "OnFileAvailable" method referenced in this line (from the bottom of your post):
ThisFile.OnFileAvailable += new EventHandler(OnFileAvailable);
is not expecting an EventHandler - it's expecting a FileAvailable delegate. Change it to:
ThisFile.OnFileAvailable += new absFile.FileAvailable(OnFileAvailable);
//note that this can also just be ThisFile.OnFileAvailable += OnFileAvailable;
and make sure OnFileAvailable looks like this:
public void OnFileAvailable(object sender, FileEventArgs e)
{
//...
}
First things first... Ensure the event is subscribed, before calling it. Wrap the event call in an IF statement:
if (OnFileAvailable != null)
OnFileAvailable(this, toFileEventArgs);

How to detect a truncated file in C#

If I'm reading a text file in shared access mode and another process truncates it, what is the easiest way to detect that? (I'm excluding the obvious choice of refreshing a FileInfo object periodically to check its size) Is there some convenient way to capture an event? (Filewatcher?)
There is, It's called FileSystemWatcher.
If you are developing a windows forms application, you can drag-and-drop it from the toolbox.
Here's some usage example:
private void myForm_Load(object sender, EventArgs e)
{
var fileWatcher = new System.IO.FileSystemWatcher();
// Monitor changes to PNG files in C:\temp and subdirectories
fileWatcher.Path = #"C:\temp";
fileWatcher.IncludeSubdirectories = true;
fileWatcher.Filter = #"*.png";
// Attach event handlers to handle each file system events
fileWatcher.Changed += fileChanged;
fileWatcher.Created += fileCreated;
fileWatcher.Renamed += fileRenamed;
// Start monitoring!
fileWatcher.EnableRaisingEvents = true;
}
void fileRenamed(object sender, System.IO.FileSystemEventArgs e)
{
// a file has been renamed!
}
void fileCreated(object sender, System.IO.FileSystemEventArgs e)
{
// a file has been created!
}
void fileChanged(object sender, System.IO.FileSystemEventArgs e)
{
// a file is modified!
}
It's in System.IO and System.dll so you should be able to use it in most type of projects.
FSW cannot work reliably, it is asynchronous. Assuming you don't get an exception, StreamReader.ReadLine() will return null when the file got truncated. Then check if the size changed. Beware of the unavoidable race condition, you'll need to verify assumptions about timing.
Just something to chew on; it may not apply to your situation:
chakrit's solution is correct for what you asked for, but I have to ask -- why are you reading a file while another process truncates it?
In particular, if you don't have some synchronization, reading/writing files concurrently is not particularly safe, and you may find you have other mysterious problems.

Categories

Resources