Develop Reference

User interaction in non-UI thread?

in my WPF - C# application, I have a time consuming function, which I execute with a BackgroundWorker. The job of this function is to add given data from a file into a database. Now and then, I need some user feedback, for example the data is already in the store and I want to ask the user, whether he wants to merge the data or create a new object or skip the data completely. Much like the dialog windows shows, if I try to copy a file to a location, where a file with the same name already exists.
The problem is, that I cannot call a GUI-window from a non GUI-thread. How could I implement this behavior?
Thanks in advance,
Frank

You could work with EventWaitHandle ou AutoResetEvent, then whenever you want to prompt the user, you could the signal UI, and then wait for the responde. The information about the file could be stored on a variable.

If possible... my suggestion is to architect your long running task into atomic operations. Then you can create a queue of items accessible by both your background thread and UI thread.
public class WorkItem<T>
{
public T Data { get; set; }
public Func<bool> Validate { get; set; }
public Func<T, bool> Action { get; set; }
}
You can use something like this class. It uses a queue to manage the execution of your work items, and an observable collection to signal the UI:
public class TaskRunner<T>
{
private readonly Queue<WorkItem<T>> _queue;
public ObservableCollection<WorkItem<T>> NeedsAttention { get; private set; }
public bool WorkRemaining
{
get { return NeedsAttention.Count > 0 && _queue.Count > 0; }
}
public TaskRunner(IEnumerable<WorkItem<T>> items)
{
_queue = new Queue<WorkItem<T>>(items);
NeedsAttention = new ObservableCollection<WorkItem<T>>();
}
public event EventHandler WorkCompleted;
public void LongRunningTask()
{
while (WorkRemaining)
{
if (_queue.Any())
{
var workItem = _queue.Dequeue();
if (workItem.Validate())
{
workItem.Action(workItem.Data);
}
else
{
NeedsAttention.Add(workItem);
}
}
else
{
Thread.Sleep(500); // check if the queue has items every 500ms
}
}
var completedEvent = WorkCompleted;
if (completedEvent != null)
{
completedEvent(this, EventArgs.Empty);
}
}
public void Queue(WorkItem<T> item)
{
// TODO remove the item from the NeedsAttention collection
_queue.Enqueue(item);
}
}
Your UI codebehind could look something like
public class TaskRunnerPage : Page
{
private TaskRunner<XElement> _taskrunner;
public void DoWork()
{
var work = Enumerable.Empty<WorkItem<XElement>>(); // TODO create your workItems
_taskrunner = new TaskRunner<XElement>(work);
_taskrunner.NeedsAttention.CollectionChanged += OnItemNeedsAttention;
Task.Run(() => _taskrunner.LongRunningTask()); // run this on a non-UI thread
}
private void OnItemNeedsAttention(object sender, NotifyCollectionChangedEventArgs e)
{
// e.NewItems contains items that need attention.
foreach (var item in e.NewItems)
{
var workItem = (WorkItem<XElement>) item;
// do something with workItem
PromptUser();
}
}
/// <summary>
/// TODO Use this callback from your UI
/// </summary>
private void OnUserAction()
{
// TODO create a new workItem with your changed parameters
var workItem = new WorkItem<XElement>();
_taskrunner.Queue(workItem);
}
}
This code is untested! But the basic principle should work for you.

Specifically to your case
private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
{
Thread.Sleep(1000);
var a = Test1("a");
Thread.Sleep(1000);
var b = (string)Invoke(new Func<string>(() => Test2("b")));
MessageBox.Show(a + b);
}
private string Test1(string text)
{
if (this.InvokeRequired)
return (string)this.Invoke(new Func<string>(() => Test1(text)));
else
{
MessageBox.Show(text);
return "test1";
}
}
private string Test2(string text)
{
MessageBox.Show(text);
return "test2";
}
Test2 is a normal method which you have to invoke from background worker. Test1 can be called directly and uses safe pattern to invoke itself.
MessageBox.Show is similar to yourForm.ShowDialog (both are modal), you pass parameters to it (text) and you return value (can be a value of property of yourForm which is set when form is closed). I am using string, but it can be any data type obviously.

From the input of the answers here, I came to the following solution:
(Mis)Using the ReportProgress-method of the Backgroundworker in Combination with a EventWaitHandle. If I want to interact with the user, I call the ReportProgress-method and setting the background process on wait. In the Handler for the ReportProgress event I do the interaction and when finished, I release the EventWaitHandle.
BackgroundWorker bgw;
public MainWindow()
{
InitializeComponent();
bgw = new BackgroundWorker();
bgw.DoWork += new DoWorkEventHandler(bgw_DoWork);
bgw.RunWorkerCompleted += new RunWorkerCompletedEventHandler(bgw_RunWorkerCompleted);
bgw.WorkerReportsProgress = true;
bgw.ProgressChanged += new ProgressChangedEventHandler(bgw_ProgressChanged);
}
// Starting the time consuming operation
private void Button_Click(object sender, RoutedEventArgs e)
{
bgw.RunWorkerAsync();
}
// using the ProgressChanged-Handler to execute the user interaction
void bgw_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
UserStateData usd = e.UserState as UserStateData;
// UserStateData.Message is used to see **who** called the method
if (usd.Message == "X")
{
// do the user interaction here
UserInteraction wnd = new UserInteraction();
wnd.ShowDialog();
// A global variable to carry the information and the EventWaitHandle
Controller.instance.TWS.Message = wnd.TextBox_Message.Text;
Controller.instance.TWS.Background.Set();
}
}
void bgw_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
MessageBox.Show(e.Result.ToString());
}
// our time consuming operation
void bgw_DoWork(object sender, DoWorkEventArgs e)
{
Thread.Sleep(2000);
// need 4 userinteraction: raise the ReportProgress event and Wait
bgw.ReportProgress(0, new UserStateData() { Message = "X", Data = "Test" });
Controller.instance.TWS.Background.WaitOne();
// The WaitHandle was released, the needed information should be written to global variable
string first = Controller.instance.TWS.Message.ToString();
// ... and again
Thread.Sleep(2000);
bgw.ReportProgress(0, new UserStateData() { Message = "X", Data = "Test" });
Controller.instance.TWS.Background.WaitOne();
e.Result = first + Controller.instance.TWS.Message;
}
I hope I did not overlooked some critical issues. I'm not so familar with multithreading - maybe there should be some lock(object) somewhere?

Blocking collections + Multiple Worker threads per blocking collection + Wait For Work Completion

I have to do action in batch of 1000 message say Action A, B, C. I can do these actions in parallel.
I created groups for them. To increase parallelism, I created subgroups with in each group. Task with in a subgroup needs to be executed serially. But two subgroups can execute in parallel.
After a batch of 1000 finishes, I have to do some processing ie save in db. But I am unable to understand , how to wait for all the task to finish (I am not interested in waiting in middle just at the end of 1000 taks). Any suggestions are welcome.
public class OrderlyThreadPool<t> : IDisposable
{
BlockingCollection<t> _workingqueue = null;
Action<t> _handler = null;
public OrderlyThreadPool(int wrkerCount, Action<t> handler)
{
_workingqueue = new BlockingCollection<t>();
_handler = handler;
Worker worker = new Worker(wrkerCount, Process); //WorkerCount is always 1
worker.Start();
}
public void AddItem(t item)
{
_workingqueue.Add(item);
}
private void Process()
{
foreach (t item in _workingqueue.GetConsumingEnumerable())
{
_handler(item);
}
}
public void Dispose()
{
_workingqueue.CompleteAdding();
_workingqueue = null;
}
}
public class Worker
{
int _wrkerCount = 0;
Action _action = null;
public Worker(int workerCount, Action action)
{
_wrkerCount = workerCount;
_action = action;
}
public void Start()
{
// Create and start a separate Task for each consumer:
for (int i = 0; i < _wrkerCount; i++)
{
Task.Factory.StartNew(_action);
}
}
}
So basically I will create OrderlyThreadPool for each subgroup.
I am recv messages from say source, which blocks if no message is available. So my code, looks like
while(true)
{
var message = GetMsg();
foreach(OrderlyThreadPool<Msg> a in myList)
{
a.AddMsg(message);
}
if(msgCount > 1000)
{
Wait for all threads to finish work;
}
else
{
msgCount =msgCount+1;
}
}

You start your tasks but you don't keep a reference. Simply store these tasks, expose them through the Worker and OrderlyThreadPool and use Task.WhenAll to wait for all of them to complete:
public class Worker
{
//...
List<Task> _tasks = new List<Task>();
public Task Completion { get { return Task.WhenAll(_tasks); } }
public void Start()
{
// Create and start a separate Task for each consumer:
for (int i = 0; i < _wrkerCount; i++)
{
Tasks.Add(Task.Factory.StartNew(_action));
}
}
}
public class OrderlyThreadPool<t> : IDisposable
{
//...
public Task Completion { get { return _worker.Completion; }}
}
await Task.WhenAll(myList.Select(orderlyThreadPool => orderlyThreadPool.Completion));
However, you should probably consider using TPL Dataflow instead. It's an actor-based framework that encapsulates completion, batching, concurrency levels and so forth...

How to know when all my Producer - Consumer jobs have finished

I have Winforms application that read several network folders and search for files inside this folders, this function receive List<stirng> folders:
private decimal _numberOfFiles;
private static List<string> _folders;
public delegate void OnFileAddDelegate(List<string> files);
public event OnFileAddDelegate OnFileAddEventHandler;
public delegate void OnFinishSearchDelegate();
public event OnFinishSearchDelegate OnFinishSearchEventHandler;
public void SearchFiles()
{
foreach (string folder in _folders)
{
if (Directory.Exists(folder))
{
var files = Directory.EnumerateFiles(folder, "*.doc", SearchOption.TopDirectoryOnly)
.OrderByDescending(x => new FileInfo(x).CreationTime).Take((int)_numberOfFiles).ToList<string>();
if (OnFileAddEventHandler != null)
OnFileAddEventHandler(files);
}
}
if (OnFinishSearchEventHandler != null)
OnFinishSearchEventHandler();
}
After OnFileAddEventHandler(files) event is fired my ProducerConsumer class start to check this List of files that found and do the work (if the file is OK fired up event to my main UI to add this files into my ListView):
public class ProducerConsumer
{
public delegate void OnFileAddDelegate(PcapFileDetails pcapFileDetails);
public event OnFileAddDelegate OnFileAddEventHandler;
public delegate void AllFilesProcessedDelegate();
public event AllFilesProcessedDelegate AllFilesProcessedEventHandler;
private readonly Queue<string> _queue;
private int counter;
public ProducerConsumer(int workerCount, IEnumerable<string> list)
{
_isSearchFinished = true;
_queue = new Queue<string>(list); // fill the queue
counter = _queue.Count; // set up counter
for (int i = 0; i < workerCount; i++)
Task.Factory.StartNew(Consumer);
}
private void Consumer()
{
FileChecker fileChecker = new FileChecker();
for (; ; )
{
string file;
lock (_queue)
{
// synchronize on the queue
if (_queue.Count == 0) return; // we are done
file = _queue.Dequeue(); // get file name to process
} // release the lock to allow other consumers to access the queue
// do the job
string result = fileChecker.Check(file); // Check my file
if (OnFileAddEventHandler != null && result ) // In case my file OK, fired up event to my main UI
OnFileAddEventHandler(file);
// decrement the counter
if (Interlocked.Decrement(ref counter) != 0)
continue; // not the last
// all done - we were the last
if (AllFilesProcessedEventHandler != null)
AllFilesProcessedEventHandler();
return;
}
}
}
Now while this search is in process my UI is locked to prevent unnecessary clicks and i want to know when all my folders finish to search to unlock.
But my problem is because i am search several folders the event AllFilesProcessedEventHandler() fired up several times and i want to know when all my searches finished.

Here is a recursive sample with QuickIO.Net
using System;
using System.Collections.Concurrent;
using System.Threading;
using System.Threading.Tasks;
using SchwabenCode.QuickIO;
namespace ConsoleApplication3
{
internal class Program
{
private static readonly BlockingCollection<QuickIOFileInfo> fileInfos = new BlockingCollection<QuickIOFileInfo>();
private static void Main(string[] args)
{
var task = Task.Factory.StartNew(() =>
{
Int32 totalSize = 0;
Parallel.ForEach(fileInfos.GetConsumingEnumerable(), fi =>
{
Interlocked.Add(ref totalSize, (int)fi.Bytes);
});
Console.WriteLine("All docs bytes amount to {0}", totalSize);
});
ProcessDirectory("C:\\");
fileInfos.CompleteAdding();
Task.WaitAll(task);
}
private static void ProcessDirectory(string path)
{
Parallel.ForEach(QuickIODirectory.EnumerateDirectories(path), dir =>
{
try
{
Parallel.ForEach(QuickIODirectory.EnumerateFiles(dir), file =>
{
if (file.AsFileInfo().Extension.Equals(".docx"))
fileInfos.Add(file);
});
ProcessDirectory(dir.FullName);
}
catch (Exception e)
{
Console.WriteLine("Unable to access directory {0}", dir.FullName);
}
});
}
}
}
Blocking collection will automatically signal to Parallel ForEach when all elements have been added, by calling CompleteAdding().
To scan a 256GB SSD, with 74GB left and total of 738k+ files took 16.8s.

Producer Consumer Pattern How do you notify completion?

Working on .net 2.0 I need to implement the some threading and I was looking at a dummy examples but cannot find anything which implement eventnotification.
Need to know when all is done and also some sort of progress bar if you like.
I am been playing with following code by cannot see to get the event notification correctly.
How do I detect that I have finished the processing and possible updating the ui with what I have been doing?
Example code
class Program
{
static void Main(string[] args)
{
using (PCQueue q = new PCQueue(2))
{
q.TaskCompleted += new EventHandler(OnTaskCompleted);
q.PercentageCompleted += new EventHandler(OnPercentageCompleted);
for (int i = 1; i < 100; i++)
{
string itemNumber = i.ToString(); // To avoid the captured variable trap
q.EnqueueItem(itemNumber);
}
Console.WriteLine("Waiting for items to complete...");
Console.Read();
}
}
private static void OnPercentageCompleted(object sender, EventArgs e)
{
}
static void OnTaskCompleted(object sender, EventArgs e)
{
}
}
public class PCQueue : IDisposable
{
readonly object locker = new object();
Thread[] _workers;
Queue<string> _itemQ = new Queue<string>();
public PCQueue(int workerCount)
{
_workers = new Thread[workerCount];
// Create and start a separate thread for each worker
for (int i = 0; i < workerCount; i++)
{
(_workers[i] = new Thread(Consume)).Start();
}
}
public void EnqueueItem(string item)
{
lock (locker)
{
_itemQ.Enqueue(item); // We must pulse because we're
Monitor.Pulse(locker); // changing a blocking condition.
}
}
void Consume()
{
while (true) // Keep consuming until
{ // told otherwise.
string item;
lock (locker)
{
while (_itemQ.Count == 0) Monitor.Wait(locker);
item = _itemQ.Dequeue();
}
if (item == null) return; // This signals our exit.
DoSomething(item); // Execute item.
}
}
private void DoSomething(string item)
{
Console.WriteLine(item);
}
public void Dispose()
{
// Enqueue one null item per worker to make each exit.
foreach (Thread worker in _workers)
{
EnqueueItem(null);
}
}
//where/how can I fire this event???
public event EventHandler TaskCompleted;
protected void OnCompleted(EventArgs e)
{
if (this.TaskCompleted != null)
{
this.TaskCompleted(this, e);
}
}
//where/how can I fire this event???
public event EventHandler PercentageCompleted;
protected void OnPercentageCompleted(EventArgs e)
{
if (this.PercentageCompleted != null)
{
this.PercentageCompleted(this, e);
}
}
}
Any suggestions?

You can't raise the progress event inside your queue for the simple reason that the queue does not know the total number items which are supposed to be processed. So it can't calculate a percentage. You just stick something in and it gets processed.
What you could do is to raise a ItemProcessed event and subscribe to that. Then in your main program you can do the logic of counting how many items were processed so far in relation to how many are supposed to be processed.
You can raise the complete event just before you are returning from your Consume function. However you need to keep track of how many threads are still active as Brian said in his answer. I modified the code to reflect that.
So something along these lines:
...
private int _ActiveThreads;
public PCQueue(int workerCount)
{
_ActiveThreads = workerCount;
_workers = new Thread[workerCount];
// Create and start a separate thread for each worker
for (int i = 0; i < workerCount; i++)
{
(_workers[i] = new Thread(Consume)).Start();
}
}
void Consume()
{
while (true) // Keep consuming until
{ // told otherwise.
string item;
lock (locker)
{
while (_itemQ.Count == 0) Monitor.Wait(locker);
item = _itemQ.Dequeue();
}
if (item == null) // This signals our exit.
{
if (Interlocked.Decrement(ref _ActiveThreads) == 0)
{
OnCompleted(EventArgs.Empty);
}
return;
}
DoSomething(item); // Execute item.
OnItemProcessed();
}
}
public event EventHandler ItemProcessed;
protected void OnItemProcessed()
{
var handler = ItemProcessed;
if (handler != null)
{
handler(this, EventArgs.Empty);
}
}
...
Of course you might want to create some meaningfull event args and actually pass the item which was processed to the event.
Then in main:
...
static void Main(string[] args)
{
using (PCQueue q = new PCQueue(2))
{
q.ItemProcessed += ItemProcessed;
q.TaskCompleted += OnTaskCompleted;
for (int i = 1; i <= totalNumberOfItems; i++)
{
string itemNumber = i.ToString(); // To avoid the captured variable trap
q.EnqueueItem(itemNumber);
}
Console.WriteLine("Waiting for items to complete...");
Console.Read();
}
}
private static int currentProcessCount = 0;
private static int totalNumberOfItems = 100;
private static void ItemProcessed(object sender, EventArgs e)
{
currentProcessCount++;
Console.WriteLine("Progress: {0}%", ((double)currentProcessCount / (double)totalNumberOfItems) * 100.0);
}
static void OnTaskCompleted(object sender, EventArgs e)
{
Console.WriteLine("Done");
}
...
Needless to say that all that static stuff should go. This is just based on your example.
One more remark:
Your PCQueue currently requires that you enqueue as many null values as you have worker threads otherwise only one thread will quit and the others will wait until your process quits. You can change that by looking at the first item and only removing it when it is not null - thus leaving the marker there for all threads. So Consume would change to this:
void Consume()
{
while (true) // Keep consuming until
{ // told otherwise.
string item;
lock (locker)
{
while (_itemQ.Count == 0) Monitor.Wait(locker);
item = _itemQ.Peek();
if (item != null) _itemQ.Dequeue();
else Monitor.PulseAll(); // if the head of the queue is null then make sure all other threads are also woken up so they can quit
}
if (item == null) // This signals our exit.
{
if (Interlocked.Decrement(ref _ActiveThreads) == 0)
{
OnCompleted(EventArgs.Empty);
}
return;
}
DoSomething(item); // Execute item.
OnItemProcessed();
}
}

In your PCQueue class you will need to keep track of how many worker threads are still active and raise TaskCompleted only after all threads have been instructed to end.
void Consume()
{
while (true)
{
string item;
lock (locker)
{
while (_itemQ.Count == 0) Monitor.Wait(locker);
item = _itemQ.Dequeue();
}
if (item == null)
{
// activeThreads is set to the number of workers in the constructor.
if (Interlocked.Decrement(ref activeThreads) == 0)
{
// Take a snapshot of the event so that a null check + invocation is safe.
// This works because delegates are immutable.
var copy = TaskCompleted;
if (copy != null)
{
copy(this, new EventArgs());
}
}
return;
}
DoSomething(item); // Execute item.
}
}
A couple of other points:
Kudos for getting the blocking queue implemented correctly. Most people get it wrong.
Remember to marshal the TaskCompleted event handler back onto the UI thread before touching any UI controls.
You could raise PercentCompleted from DoSomething, but without a clear indication of how many items the queue is suppose to hold the value will not make sense. I second Chris' recommendation on this point.

speeding up windows file search with C#

i made a program that search logical drives to find a specific file .if user type file name
an click search button , searching begins , but i don't know how to stop searching in the middle of process.can you help me ?

You should perform the search on a background thread so that it doesn't block the UI. This article has a good introduction and walkthrough of the changes that you'll need to make to your app.

You need to use Backgroundworker class in .net. It executes on separate thread and it has inbuilt methods/properties for cancellation, report progress and lot more...
Have a look at following article to get started with it:
http://msdn.microsoft.com/en-us/library/system.componentmodel.backgroundworker.aspx

You need to run the search in a background thread (Using BackgroundWorker is the most convenient way to do this) then you can still handle input to cancel it.
When you are handling the cancel request you may need to use Thread.Abort on the executing thread or BackgroundWorker.CancelAsync() on the BackgroundWorker.
Alternatively you can have the executing thread check a variable while in the processing loop or at the start of a recursive function - to cancel you simple need to set this variable when handling the cancel request.

As others have mentioned, your solution may be to use BackgroundWorker with CancelAsync method.
Here is some working code which you could use with minor modifications:
class Program
{
static void Main(string[] args)
{
var search = new FileSearcher().FindFile(#"d:\users", "autofac.dll", f => Console.WriteLine(f.FullName), () => Console.WriteLine("Finished"));
Console.WriteLine("C - cancel, else - finish");
for (; ; )
{
var command = Console.ReadLine();
switch (command)
{
case "C":
search.Cancel();
break;
default:
return;
}
}
}
}
public class FileSearcher
{
public FileSearch FindFile(string searchPath, string fileName, Action<FileInfo> onFileFound, Action onSearchFinished)
{
var search = new FileSearch(new DirectoryInfo(searchPath), fileName);
search.FileFound += onFileFound;
search.Finished += onSearchFinished;
search.Run();
return search;
}
}
public class FileSearch
{
readonly BackgroundWorker _worker = new BackgroundWorker();
readonly DirectoryInfo _searchPath;
readonly string _template;
public FileSearch(DirectoryInfo searchPath, string template)
{
_searchPath = searchPath;
_template = template;
_worker.DoWork += _worker_DoWork;
_worker.RunWorkerCompleted += _worker_RunWorkerCompleted;
_worker.WorkerSupportsCancellation = true;
}
void _worker_DoWork(object sender, DoWorkEventArgs e)
{
foreach (var directory in GetPartiallyFlatDirectories(_searchPath, 4))
{
if (_worker.CancellationPending)
break;
foreach (var file in directory.GetFiles(_template, SearchOption.AllDirectories))
FileFound.Raise(file);
}
}
static IEnumerable<DirectoryInfo> GetPartiallyFlatDirectories(DirectoryInfo directory, int flatDepth)
{
if (flatDepth == 0)
{
yield return directory;
yield break;
}
foreach (var subDir in directory.GetDirectories())
{
var flattened = GetPartiallyFlatDirectories(subDir, flatDepth - 1);
if (!flattened.Any())
yield return subDir;
else
foreach (var flatDirectory in flattened)
yield return flatDirectory;
}
}
void _worker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
Finished.Raise();
}
public void Cancel()
{
_worker.CancelAsync();
}
public event Action<FileInfo> FileFound;
public event Action Finished;
public void Run()
{
_worker.RunWorkerAsync();
}
}
public static class DelegateExtensions
{
public static void Raise<T>(this Action<T> action, T obj)
{
if (action != null)
action(obj);
}
public static void Raise(this Action action)
{
if (action != null)
action();
}
}