To begin with, I'm relatively new to programming. I went through some introductory C# training for my new job, and it's the first language I've worked with.
I recently had a business problem that I decided to solve using C#, both to save time (I had hoped) and to learn more C# in the process. The business problem I mentioned was this: I had 600+ Word files that I needed to audit. For each document, I had to make sure that...
There was no text with strike-through anywhere in the document.
Track Changes was disabled.
There were no pending changes (as in changes that were made while
Track Changes was enabled and have yet to be accepted or
rejected).
There were no comments.
It would have been fastest to have my program iterate through all of the documents, making changes as it went along. But because of the nature of this assignment I wanted to make the changes manually, limiting the program's use to generating a list of files (out of the 600) where changes were necessary, and detailing what changes needed to be made for each of those files.
So, I have a button that calls up a FolderBrowserDialog.
private void AddFolderButtonClick(object sender, EventArgs e)
{
var folderBrowser = new FolderBrowserDialog();
if (folderBrowser.ShowDialog() != DialogResult.OK)
{
return;
}
this.progressBar1.Visible = true;
this.progressBar1.Style = ProgressBarStyle.Marquee;
this.Cursor = Cursors.WaitCursor;
var args = new List<string>(Directory.EnumerateDirectories(folderBrowser.SelectedPath));
// Get list of files in selected directory, adding to list of directories
args.AddRange(Directory.EnumerateFiles(folderBrowser.SelectedPath));
this.displayListBox.BeginUpdate();
foreach (string path in args)
{
if (File.Exists(path))
{
// This path is a file
this.ProcessFile(Path.GetFullPath(path));
}
else if (Directory.Exists(path))
{
// This path is a directory
this.ProcessDirectory((Path.GetFullPath(path)));
}
else
{
Console.WriteLine(Resources.Finder_Invalid_File_Or_Directory, path);
}
}
this.displayListBox.EndUpdate();
this.progressBar1.Visible = false;
this.progressBar1.Style = ProgressBarStyle.Continuous;
this.Cursor = Cursors.Default;
}
Together, the following two methods iterate through all subdirectories and files to create a full list of all files below the top level directory selected through the FolderBrowserDialog:
private void ProcessDirectory(string targetDirectory)
{
// Process the list of files found in the directory.
string[] fileEntries = Directory.GetFiles(targetDirectory);
foreach (string fileName in fileEntries)
{
this.ProcessFile(fileName);
}
// Recurse into subdirectories of this directory.
string[] subdirectoryEntries = Directory.GetDirectories(targetDirectory);
foreach (string subdirectory in subdirectoryEntries)
{
this.ProcessDirectory(subdirectory);
}
}
private void ProcessFile(string path)
{
Console.WriteLine(Resources.Finder_File_Processed, path);
string fileName = Path.GetFileName(path);
if (fileName == null || fileName.StartsWith(#"~$") || this.selectedFilesList.Contains(path))
{
return;
}
this.selectedFilesList.Add(path);
this.filePathsCountLabel.Text = (#"Count: " + this.selectedFilesList.Count);
this.displayListBox.Items.Add(path);
}
Once all this code has run, I get a full list of documents. I click a button and the program does what it's supposed to from here on out. Okay, cool. I mentioned before that half of the reason I chose to use C# to solve this was for the sake of learning. At this point I've got everything I need but what I really want to know is how can I implement threading to make the GUI responsive while the list of files is being generated? I've looked through several examples. They made sense. For some reason I just can't get my head around it for this application though. How can I make the whole process of processing subdirectories and files happen without locking up the GUI?
I believe what you need could be found here.
In short, to use a backgroundworker which does all the work on a separate thread thus prevents GUI freezes, first you instantiate BackgroundWorker and handle the DoWork event. Then you call RunWorkerAsync, optionally with an object argument.
As a skeleton code:
class myClass
{
static BackgroundWorker myBw = new BackgroundWorker();
static void Main()
{
myBw .DoWork += myBw_DoWork;
myBw .RunWorkerAsync ("an argument here");
Console.ReadLine();
}
static void myBw_DoWork (object sender, DoWorkEventArgs e)
{
// This is called on the separate thread, argument is called as e.Argument
// Perform heavy task...
}
}
You have to create a separate thread to process your work. Look at this if you are using .NET 4.0+ or this for older versions.
With Task, you can write
Task.Factory.StartNew(() => DoAction()
where DoAction is your function that starts to process data.
But do not forget to use Invoke, if you want to act with GUI from separate thread. For example, if you want to update some Label text from separate thread, you have to do this
label1.Invoke(() => label1.Text = "Some Text");
Related
I build a windows-forms-app where I (try to) do extensive calculations on images whenever they are created in a specific directory which I watch using the FileSystemWatcher.
private void OnNewFileInDir(object source, FileSystemEventArgs evtArgs)
{
//Load the actual image:
imageFilepath = evtArgs.FullPath; //imageFilepath is a private class string var
Image currentImage = Image.FromFile(imageFilepath);
//Display the image in the picture box:
UpdatePictureBox(currentImage); //Method to update the GUI with invoking for the UI thread
//Extensive Calculation on the images
Image currentResultImage = DoExtensiveWork(currentImage);
// Put the current result in the picture box
UpdatePictureBox(currentResultImage );
//dispose the current/temporary image
currentImage.Dispose();
}
The event is fired correctly when pasting a new file into the directory. But I get a "System.OutOfMemoryException" on the line
Image currentImage = Image.FromFile(imageFilepath);
When I put exactly this code (using the same filepath) in a button event (so not using the FileSystemWatcher) everything works fine. So I thought there is some issue regarding the thread since the extensive calculation is then called by the FileSystemWatcher-Thread not by the UI thread.
I tried things like:
//TRY 1: By executing a button click method containg the code
pb_Calculate_Click(this, new EventArgs()); //This does not work eigther --> seems to be a problem with "Who is calling the method"
//TRY 2: Open a new dedicated thread for doing the work of the HistoCAD calculations
Thread newThread_OnNewFile = new Thread(autoCalcAndDisplay);
newThread_OnNewFile.Start();
//TRY 3: Use a background worker as a more safe threading method(?)
using (BackgroundWorker bw = new BackgroundWorker())
{
bw.DoWork += new DoWorkEventHandler(bw_DoWork);
if (bw.IsBusy == false)
{
bw.RunWorkerAsync();
}
}
Unfortunalty none of them worked reliable. 1st not at all. 2nd works only from time to time and 3rd one as well.
Do some of you know whats going on there? What can I do to make it work correctly? Thanks!
EDIT:
Thanks for the comments:
I also tried to call GC.collect() on every event and tried to include using() and dispose() wherever I can. When I'm doing the process manually (with buttons) it works even when processing a lot of files one after another. But when done with the eventhandler I sometimes get the outOfMem-Exception even on the very first file I copy in the folder. File is always the same BMP with 32MB. This is the memory usage for processing one image:
EDIT 2:
I created a minimal example (GUI with one picture Box and one Checkbox in buttonstyle). It turns out that the same thing is happening. The OutOfMemException occured at the same line (Image...). Especially for large BMPs the exception occours nearly always:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
namespace MinimalExampleTesting
{
public partial class Form1 : Form
{
private string imageFilepath;
private string autoModePath = #"C:\Users\Tim\Desktop\bmpordner";
//Define a filesystem watcher object
private FileSystemWatcher watcher;
public Form1()
{
InitializeComponent();
/*** Creating as FileSystemEventArgs watcher in order to monitor a specific folder ***/
watcher = new FileSystemWatcher();
Console.WriteLine(watcher.Path);
// set the path if already exists, otherwise we have to wait for it to be set
if (autoModePath != null)
watcher.Path = autoModePath;
// Watch for changes in LastAccess and LastWrite times and renaming of files or directories.
watcher.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite
| NotifyFilters.FileName | NotifyFilters.DirectoryName;
// Only watch for BMP files.
watcher.Filter = "*.bmp";
// Add event handler. Only on created, not for renamed, changed or something
// Get into the list of the watcher. Watcher fires event and "OnNewFileCreatedInDir" will be called
watcher.Created += new FileSystemEventHandler(OnNewFileInDir);
}
private void tb_AutoMode_CheckedChanged(object sender, EventArgs e)
{
//First of all test if the auto mode path is set and correctly exists currently:
if (!Directory.Exists(autoModePath) || autoModePath == null)
{
MessageBox.Show("Check if Auto Mode path is correctly set and if path exists",
"Error: Auto Mode Path not found");
return;
}
// Begin watching if the AutoModePath was at least set
if (autoModePath != null)
{
watcher.EnableRaisingEvents = tb_AutoMode.Checked; //Since we have a toogle butten, we can use the 'checked' state to enable or disable the automode
}
}
private void OnNewFileInDir(object source, FileSystemEventArgs evtArgs)
{
Console.WriteLine("New file in detected: " + evtArgs.FullPath);
//Force a garbage collection on every new event to free memory and also compact mem by removing fragmentation.
GC.Collect();
//Set the current filepath in the class with path of the file added to the folder:
imageFilepath = evtArgs.FullPath;
//Load the actual image:
Image currentImage = Image.FromFile(imageFilepath);
UpdatePictureBox(currentImage);
}
private void UpdatePictureBox(Image img)
{
if (pictureBox_Main.InvokeRequired)
{
MethodInvoker mi = delegate
{
pictureBox_Main.Image = img;
pictureBox_Main.Refresh();
};
pictureBox_Main.Invoke(mi);
}
else { //Otherwise (when the calculation is perfomed by the GUI-thread itself) no invoke necessary
pictureBox_Main.Image = img;
pictureBox_Main.Refresh();
}
img.Dispose();
}
}
}
Thanks in advance for further hints :)
SOLVED:
The issue seems to be, that event is fired immediately but the file is not yet finally copied. That means we have to wait until the file is free.
A Thread.Sleep(100) at the start of the event does the job. As I now know what to google for, I found two links:
This and this where you can find:
The OnCreated event is raised as soon as a file is created. If a file is being copied or transferred into a watched directory, the OnCreated event will be raised immediately, followed by one or more OnChanged events
So, what works best for my case, was to include a method to test if the file is still locked and than wait at the beginning of the event for an unlock of the file. No need for an additional thread or a BackgroundWorker.
See the code:
private void OnNewFileInDir(object source, FileSystemEventArgs evtArgs)
{
Console.WriteLine("New file detected: " + evtArgs.FullPath);
//Wait for the file to be free
FileInfo fInfo = new FileInfo(evtArgs.FullPath);
while (IsFileLocked(fInfo))
{
Console.WriteLine("File not ready to use yet (copy process ongoing)");
Thread.Sleep(5); //Wait for 5ms
}
//Set the current filepath in the class with path of the file added to the folder:
imageFilepath = evtArgs.FullPath;
//Load the actual image:
Image currentImage = Image.FromFile(imageFilepath);
UpdatePictureBox(currentImage);
}
private static bool IsFileLocked(FileInfo file)
{
FileStream stream = null;
try
{
//try to get a file lock
stream = file.Open(FileMode.Open, FileAccess.ReadWrite, FileShare.None);
}
catch (IOException)
{
//File isn't ready yet, so return true as it is still looked --> we need to keep on waiting
return true;
}
finally
{
if (stream != null){
stream.Close();
stream.Dispose();
}
}
// At the end, when stream is closed and disposed and no exception occured, return false --> File is not locked anymore
return false;
}
Nevertheless: Thanks for your help...it got me on the right track;)
As MSDN says about FileSystemWatcher:
Common file system operations might raise more than one event. For example, when a file is moved from one directory to another, several OnChanged and some OnCreated and OnDeleted events might be raised. Moving a file is a complex operation that consists of multiple simple operations, therefore raising multiple events. Likewise, some applications (for example, antivirus software) might cause additional file system events that are detected by FileSystemWatcher.
Maybe your Image is loaded severals times.
To test it, you can add this line after imageFilepath = evtArgs.FullPath;
imageFilepath = evtArgs.FullPath;
Task.Run(()=>{MessageBox.Show(imageFilepath);});
This will inform you about the fact that the Created event is fired, and will not hold up your program.
Edit
Put your line of code that give the OutOfMemory in a Try Catch.
Like this and this questions describes, you can get this error if your image is corrupt.
Problem is that application closes without any error, VS stays opened.
I have multiple dynamically created FileSystemWatchers, all of them have eventhandler on "Created" event. So this eventhandler method looks like this :
void watcher_FileCreated(object sender, FileSystemEventArgs e)
{
FileInfo f1 = new FileInfo(e.FullPath);
filesDataGrid.Rows.Add(f1.Name);
foreach (TLPclass table in parameterForm.getParameters)
{
//uses some funcion form another class
}
}
Line which causes program to close is the one where I'm adding File name to DataGridView - filesDataGrid.Rows.Add(f1.Name);
Also runs OK without that line.
Weird thing is that application runs normally, when launched from .exe file in projects folder. I can't see error in my code, but I guess theres something awfully wrong with it, if it doesn't even show error message.
And - what are the most common reasons why program could just shut down with no warnings?
The FileSystemWatcher will trigger the events in a separate thread. The logic inside the event handlers will need to take that fact in consideration and perform any synchronization needed. So you'll need something like this:
private void watcher_FileCreated(object sender, FileSystemEventArgs e)
{
if (filesDataGrid.InvokeRequired)
{
filesDataGrid.Invoke((MethodInvoker)delegate { watcher_FileCreated(sender, e); });
}
else
{
FileInfo f1 = new FileInfo(e.FullPath);
filesDataGrid.Rows.Add(f1.Name);
foreach (TLPclass table in parameterForm.getParameters)
{
//uses some funcion form another class
}
}
}
Wrap your function with a Try{}catch (Exception ex) { } block.
i Would think it is because DataGridRow needs to be created with the variables from the FileSystemWatcher first.
e.g
DataGridViewRow row = filesDataGrid.NewRow();
row["columnname"] = f1.name;
filesDataGrid.Rows.Add(row);
I've just finished making this recursive method:
/// <summary>
/// Recursively process a given directory and add its file to Library.xml
/// </summary>
/// <param name="sourceDir">Source directory</param>
public void ProcessDir(string sourceDir)
{
string[] fileEntries = Directory.GetFiles(sourceDir, "*.mp3");
foreach (string fileName in fileEntries)
{
Song newSong = new Song();
newSong.ArtistName = "test artist";
newSong.AlbumName = "test album";
newSong.Name = "test song title";
newSong.Length = 1234;
newSong.FileName = fileName;
songsCollection.Songs.Add(newSong);
}
string[] subdirEntries = Directory.GetDirectories(sourceDir);
foreach (string subdir in subdirEntries)
{
if ((File.GetAttributes(subdir) & FileAttributes.ReparsePoint) != FileAttributes.ReparsePoint)
{
ProcessDir(subdir);
}
}
}
Everything is working as expected, the only problem I'm having is: How do I know when this method finishes execution? Is there something made for that very purpose in .NET?
There's nothing special in .NET that tells you this... Basically, the first call to ProcessDir will return after the recursion has ended.
Well, you could always put a line of code indicating the end of execution after your initial ProcessDir call:
ProcessDir("MyDir");
Console.WriteLine("Done!");
You could try using a global variable to keep track of it.
private int _processDirTrack = 0;
public void ProcessDir(string sourceDir)
{
_processDirTrack++; // Increment at the start of each
string[] fileEntries = Directory.GetFiles(sourceDir, "*.mp3");
foreach (string fileName in fileEntries)
{
Song newSong = new Song();
newSong.ArtistName = "test artist";
newSong.AlbumName = "test album";
newSong.Name = "test song title";
newSong.Length = 1234;
newSong.FileName = fileName;
songsCollection.Songs.Add(newSong);
}
string[] subdirEntries = Directory.GetDirectories(sourceDir);
foreach (string subdir in subdirEntries)
{
if ((File.GetAttributes(subdir) & FileAttributes.ReparsePoint) != FileAttributes.ReparsePoint)
{
ProcessDir(subdir);
}
}
_processDirTrack--; // Decrement after the recursion. Fall through means it got to
// the end of a branch
if(_processDirTrack == 0)
{
Console.WriteLine("I've finished with all of them.");
}
}
I'm assuming the (albeit correct) answer RQDQ provided isn't the one you are looking for?
In case you have a long running task of which you want to check how far it is along you can use a BackgroundWorker.
Ofcourse this background worker doesn't magically know how much more files to process, so you have to call ReportProgress whenever you can give an estimate of how far along you are.
In order to try to estimate how much longer the processing will take, you could try the following:
Check for total disk space the folder occupies, and keep track of much you already processed.
Check for amount of files you have to process vs. how much you still have to process.
If you're wanting to know when it ends to signal/start another process in your application, you could raise an event. This may be over kill, but hey, it's another way to look at it if it suits your need. You just need to add an event to the class where ProcessDir() is a member.
private int _processDirTrack = 0;
public event EventHandler DirProcessingCompleted;
At the end of your method you would raise your event like so
DirProcessingCompleted(this, new EventArgs());
You subscribe to these events with an eventhandler somewhere else in your code
myClass.DirProcessingCompleted += new EventHandler(ProcessingComplete_Handler);
Note: you don't have to subscribe to an event in this manner; you could also subscribe with a delegate or lambda expression instead.
To wrap it all up you create your method that is called whenever an event is raised by the object.
private void ProcessingComplete_Handler(object sender, EventArgs e)
{
// perform other operations here that are desirable when the event is raised ...
}
I'm guessing what you're trying to figure out is how to know that you've reached the deepest level of recursion that is going to occur.
In this case, you'll know that you're in the last recursive call of the function when subdirEntries is empty, since the function will no longer recurse at that point. That's the best you can do, there's no universal answer as to how to know when a function will cease to recurse. It's entirely dependent upon what the conditions to recurse are.
Edit: Wanted to clarify. This will check each time you end a single chain of recursions. Considering your code can recurse multiple times per call, my solution will only signify the end of a single chain of recursions. In the case of recursively navigating a tree, this will occur at every leaf node, not at the deepest level of recursion.
Below is a method that I want to ship off into a background worker but I am struggling how to do it based on how created my method. As you can it doesn't return anything which is ok but it expects a directoryInfo object everytime it is recalled.
private void getSizeForTargetDirectory(DirectoryInfo dtar)
{
// generate a collection of objects. files comes first and then directories.
foreach (Object item in collection )
{
if (item == file)
{
track the size of the files as you encounter.
}
else if (item == directory)
{
// found a new directory, recall the method. !!!
}
}
}
This is my first time using a background worker so I'm a little stuck, I tried implementing something thanks to the help found here but got stuck when I realised my method was recursive.
How do I display progress during a busy loop?
I implemented a doWork event handler method but noticed that i needed to somehow recall the method if I had more files and folders to process on lower sub levels.
I have a simple button click event handler that calls my 'getSizeForTargetDirectory()' method when the current selected node is a directory.
private void retrieveInfoButton_Click(object sender, EventArgs e)
{
// check to see if the path is valid
// reset the labels and textfields.
string fullPath = treeDrives.SelectedNode.FullPath;
string sNodesName = treeDrives.SelectedNode.Text;
if (directory) // Enter here if its a directory.
{
string parentPath = treeDrives.SelectedNode.Parent.FullPath;
DirectoryInfo[] dirArray = populateFoldersArray(parentPath);
for (int i = 0; i < dirArray.Length; i++)
{
if (dirArray[i].Name == sNodesName)
{
getSizeForTargetDirectory(dirArray[i]);
// do work !
Hopefully that explains what I am trying to do and how I am doing it. Question is how can i use the report progress feature of the background worker class when the bulk of the work I am trying to ship is coming from a recursive method.
Through early testing I noticed that my getSize method was incredibly efficient after a few tweaks and reported size information for the current supplied folder very quickley but then again I use quite a powerful dev machine so this may not be true for all users.
Thanks For Reading, Hope someone can help !!!
I think it is much simpler to use the built-in methods on either Directory or DirectoryInfo to obtain all directories, or files, using the recursive search option:
public partial class Form1 : Form
{
private Action<float> updateProgMethod;
public Form1()
{
InitializeComponent();
}
private void Form1_Load(object sender, EventArgs e)
{
updateProgMethod = UpdateProgress;
}
private void GetDirectorySizeAsync(string path)
{
backgroundWorker.RunWorkerAsync(path);
}
private void backgroundWorker_DoWork(object sender, DoWorkEventArgs e)
{
DirectoryInfo di = new DirectoryInfo((string)e.Argument);
di.GetTotalSize(ProgressCallback);
}
// Takes callbacks from the GetTotalSize() method
private void ProgressCallback(float p)
{
// Invokes update progress bar on GUI thread:
this.BeginInvoke(updateProgMethod, new object[] { p });
}
// Actually updates the progress bar:
private void UpdateProgress(float p)
{
progressBar.Value = (int)(p * (progressBar.Maximum - progressBar.Minimum)) + progressBar.Minimum;
}
}
public static class IOExtensions
{
public static long GetTotalSize(this DirectoryInfo directory, Action<float> progressCallback)
{
FileInfo[] files = directory.GetFiles("*.*", SearchOption.AllDirectories);
long sum = 0;
int countDown = 0;
for (int i = 0; i < files.Length; i++)
{
sum += files[i].Length;
countDown--;
if (progressCallback != null && countDown <= 0)
{
countDown = 100;
progressCallback((float)i / files.Length);
}
}
return sum;
}
}
It's hard to guess progress without knowing the number of files or folders first!
EDIT: I've improved the code a little.
If, when you call a method, you don't know how long the method is going to take or how many discrete steps are going to be involved, then there is no way to display a progress bar while the method is executing.
In my opinion, the purpose of a progress bar is not to give reliable information about when a task is going to be completed. Rather, the purpose is to keep the user from freaking out and cancelling the whole operation because they think your program has locked up and isn't doing anything at all.
Since you're iterating through directories and sub-directories, a simpler approach here might be to just display the current directory in a Label. This would give the user a relaxing sense that things are happening, and if the directories are all ordered alphabetically, they can even gauge for themselves the overall progress of the operation.
I would report how far you have gotten since you don't know the goal until you get there. I would do it once per invocation. Perhaps # of files and # of directories seen so far.
If I'm reading a text file in shared access mode and another process truncates it, what is the easiest way to detect that? (I'm excluding the obvious choice of refreshing a FileInfo object periodically to check its size) Is there some convenient way to capture an event? (Filewatcher?)
There is, It's called FileSystemWatcher.
If you are developing a windows forms application, you can drag-and-drop it from the toolbox.
Here's some usage example:
private void myForm_Load(object sender, EventArgs e)
{
var fileWatcher = new System.IO.FileSystemWatcher();
// Monitor changes to PNG files in C:\temp and subdirectories
fileWatcher.Path = #"C:\temp";
fileWatcher.IncludeSubdirectories = true;
fileWatcher.Filter = #"*.png";
// Attach event handlers to handle each file system events
fileWatcher.Changed += fileChanged;
fileWatcher.Created += fileCreated;
fileWatcher.Renamed += fileRenamed;
// Start monitoring!
fileWatcher.EnableRaisingEvents = true;
}
void fileRenamed(object sender, System.IO.FileSystemEventArgs e)
{
// a file has been renamed!
}
void fileCreated(object sender, System.IO.FileSystemEventArgs e)
{
// a file has been created!
}
void fileChanged(object sender, System.IO.FileSystemEventArgs e)
{
// a file is modified!
}
It's in System.IO and System.dll so you should be able to use it in most type of projects.
FSW cannot work reliably, it is asynchronous. Assuming you don't get an exception, StreamReader.ReadLine() will return null when the file got truncated. Then check if the size changed. Beware of the unavoidable race condition, you'll need to verify assumptions about timing.
Just something to chew on; it may not apply to your situation:
chakrit's solution is correct for what you asked for, but I have to ask -- why are you reading a file while another process truncates it?
In particular, if you don't have some synchronization, reading/writing files concurrently is not particularly safe, and you may find you have other mysterious problems.