Develop Reference

Notify the ActionBlock is completed to another class

I have written a method to implement TPL ActionBlock, which will do a function to find the XPath of the element I am Posting to the block. I am sending the element from a real-time application (whenever the user finds an element Post it to the block). Now, I want to check the block is completed or not when I call a save button from another class. The logic is if the ActionBlok is completed when we click the save button element will save with some save logic, otherwise show a message box not yet ready. For the first time, this idea is working, but from the second element onwards the Actionblock is not accepting any. I am using block.Complete() and block.Completion.Wait() for the save check. Now I can show I am calling the code/logic.
Once create an element, post it ActionBlock
Class1
......
jobQ.Enqueue(familarizedElement);
......
Inside the ActionBlock
Class2
public class TPLCommonDataFlow
{
private ActionBlock<Element> _jobs;
public static bool TPLFlowStatus = false;
public static string JobStatus = string.Empty;
public TPLCommonDataFlow()
{
var executionDataflowBlockOptions = new ExecutionDataflowBlockOptions()
{
MaxDegreeOfParallelism = 2,
};
_jobs = new ActionBlock<Element>((job) =>
{
Thread.Sleep(5);
File.WriteAllText("C:\\Temp\\A.txt", "Started" + _jobs.InputCount.ToString());
JobStatus = "started";
Familiarization.FindAndUpdateXPath(job);
File.WriteAllText("C:\\Temp\\A.txt", "Finished");
}, executionDataflowBlockOptions);
_jobs.Complete();
//Wait for all messages to propagate through the network
_jobs.Completion.Wait();
//Checking all jobs are completed or not, if completed changing the boo, value.
if (_jobs.InputCount == 0)
{
TPLFlowStatus = true;
JobStatus = "stoped";
}
}
}
For the save I am checking the TPLFlowStatus boolean
Class3
if (class2.TPLFlowStatus == true)
{
//Save Logic
}
else
{
//Message Box showing Not Ready
}
Now what I want is to check each time the job is completed or not for each element in the save logic. If the block having two elements in Queue and one is finished, then the MessageBox needs to popup once the save button pressed. If all completed in the block, then need to go to the save logic.

The issue your seeing is that when a block is completed it will no longer accept new messages, you've told the block your done sending messages. To start sending a new batch of messages you can either keep the block alive by not calling complete and tracking the batch completion another way or you can just reset the block. A simple worker like this might be what you're looking for:
public class DataflowWorker
{
private ActionBlock<string> _jobs;
private void BuildJobsHandler()
{
_jobs = new ActionBlock<string>(x => Console.WriteLine(x) /*Do your work in the action block*/);
}
public async Task Enque(string element)
{
await _jobs.SendAsync(element);
}
public async Task CompleteJobsAndSave()
{
_jobs.Complete();
await _jobs.Completion;
//Save data when complete
BuildJobsHandler();
}
}

Run X number of Task<T> at any given time while keeping UI responsive

I have a C# WinForms (.NET 4.5.2) app utilizing the TPL. The tool has a synchronous function which is passed over to a task factory X amount of times (with different input parameters), where X is a number declared by the user before commencing the process. The tasks are started and stored in a List<Task>.
Assuming the user entered 5, we have this in an async button click handler:
for (int i = 0; i < X; i++)
{
var progress = Progress(); // returns a new IProgress<T>
var task = Task<int>.Factory.StartNew(() => MyFunction(progress), TaskCreationOptions.LongRunning);
TaskList.Add(task);
}
Each progress instance updates the UI.
Now, as soon as a task is finished, I want to fire up a new one. Essentially, the process should run indefinitely, having X tasks running at any given time, unless the user cancels via the UI (I'll use cancellation tokens for this). I try to achieve this using the following:
while (TaskList.Count > 0)
{
var completed = await Task.WhenAny(TaskList.ToArray());
if (completed.Exception == null)
{
// report success
}
else
{
// flatten AggregateException, print out, etc
}
// update some labels/textboxes in the UI, and then:
TaskList.Remove(completed);
var task = Task<int>.Factory.StartNew(() => MyFunction(progress), TaskCreationOptions.LongRunning);
TaskList.Add(task);
}
This is bogging down the UI. Is there a better way of achieving this functionality, while keeping the UI responsive?
A suggestion was made in the comments to use TPL Dataflow but due to time constraints and specs, alternative solutions are welcome
Update
I'm not sure whether the progress reporting might be the problem? Here's what it looks like:
private IProgress<string> Progress()
{
return new Progress<string>(msg =>
{
txtMsg.AppendText(msg);
});
}

Now, as soon as a task is finished, I want to fire up a new one. Essentially, the process should run indefinitely, having X tasks running at any given time
It sounds to me like you want an infinite loop inside your task:
for (int i = 0; i < X; i++)
{
var progress = Progress(); // returns a new IProgress<T>
var task = RunIndefinitelyAsync(progress);
TaskList.Add(task);
}
private async Task RunIndefinitelyAsync(IProgress<T> progress)
{
while (true)
{
try
{
await Task.Run(() => MyFunction(progress));
// handle success
}
catch (Exception ex)
{
// handle exceptions
}
// update some labels/textboxes in the UI
}
}
However, I suspect that the "bogging down the UI" is probably in the // handle success and/or // handle exceptions code. If my suspicion is correct, then push as much of the logic into the Task.Run as possible.

As I understand, you simply need a parallel execution with the defined degree of parallelization. There is a lot of ways to implement what you want. I suggest to use blocking collection and parallel class instead of tasks.
So when user clicks button, you need to create a new blocking collection which will be your data source:
BlockingCollection<IProgress> queue = new BlockingCollection<IProgress>();
CancellationTokenSource source = new CancellationTokenSource();
Now you need a runner that will execute your in parallel:
Task.Factory.StartNew(() =>
Parallel.For(0, X, i =>
{
foreach (IProgress p in queue.GetConsumingEnumerable(source.Token))
{
MyFunction(p);
}
}), source.Token);
Or you can choose more correct way with partitioner. So you'll need a partitioner class:
private class BlockingPartitioner<T> : Partitioner<T>
{
private readonly BlockingCollection<T> _Collection;
private readonly CancellationToken _Token;
public BlockingPartitioner(BlockingCollection<T> collection, CancellationToken token)
{
_Collection = collection;
_Token = token;
}
public override IList<IEnumerator<T>> GetPartitions(int partitionCount)
{
throw new NotImplementedException();
}
public override IEnumerable<T> GetDynamicPartitions()
{
return _Collection.GetConsumingEnumerable(_Token);
}
public override bool SupportsDynamicPartitions
{
get { return true; }
}
}
And runner will looks like this:
ParallelOptions Options = new ParallelOptions();
Options.MaxDegreeOfParallelism = X;
Task.Factory.StartNew(
() => Parallel.ForEach(
new BlockingPartitioner<IProgress>(queue, source.Token),
Options,
p => MyFunction(p)));
So all you need right now is to fill queue with necessary data. You can do it whenever you want.
And final touch, when the user cancels operation, you have two options:
first you can break execution with source.Cancel call,
or you can gracefully stop execution by marking collection complete (queue.CompleteAdding), in that case runner will execute all already queued data and finish.
Of course you need additional code to handle exceptions, progress, state and so on. But main idea is here.

Enumerating through background workers

Is there a way to enumerate through all background workers? Currently, I've written a small method that I add to as I create new workers. Only one worker should run at a time, so the check method is:
private bool CheckForWorkers() // returns true if any background workers are currently running
{
bool ret = false;
if (bgWorkerFoo.IsBusy || bgWorkerMeh.IsBusy || bgWorkerHmpf.IsBusy || bgWorkerWorkyWorky.IsBusy || bgWorkerOMGStahp.IsBusy)
{
ret = true;
}
return ret;
}
At the time a button is clicked that would start a worker, the click method does this:
if (CheckForWorkers())
{
MessageBox.Show("File generation already in progress. Please wait.", "Message");
return;
}
else
{
bgWorkerFoo.RunWorkerAsync();
}
I'd like to clean up my CheckForWorkers() method so that I don't need to add to it anytime a new worker is created for a different task, but I can't seem to find any way to run through each worker associated with the app. Maybe there isn't a way? Do all of the workers exist (are instantiated) prior to being used?

Why not do something similar to this?
Worker[] Workers => new[] { bgWorkerFoo, bgWorkerMeh, bgWorkerHmpf, bgWorkerWorkyWorky, bgWorkerOMGStahp };
private bool CheckForWorkers()
{
return Workers.Any(w => w != null && w.IsBusy);
}
It's likely you'll need to refer to the collection of workers in the future as well, so it makes sense to put them into a collection anyway
Or, for non-C#6 syntax, a bit uglier:
private Worker[] Workers { get { return new[] { bgWorkerFoo, bgWorkerMeh, bgWorkerHmpf, bgWorkerWorkyWorky, bgWorkerOMGStahp }; } }
private bool CheckForWorkers()
{
return Workers.Any(w => w != null && w.IsBusy);
}
To dynamically get all fields/properties in your class, you can do this:
private IEnumerable<Worker> GetWorkers()
{
var flags = BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public;
var fields = GetType().GetFields(flags);
var fieldValues = fields.Select(f => f.GetValue(this)).OfType<Worker>();
var properties = GetType().GetProperties(flags);
var propertyValues = properties.Select(f => f.GetValue(this)).OfType<Worker>();
return fieldValues.Concat(propertyValues).Where(w => w != null);
}
private bool CheckForWorkers()
{
return GetWorkers().Any(w => w.IsBusy);
}
Might be a good idea to cache GetWorkers(), but it depends on your use-case.

I'm going to suggest an alternative approach that may be appealing. The Microsoft Reactive Framework introduced a a lot of very useful functionality for dealing with concurrency and threads. Primarily the framework is used to deal with event sources in terms of IObservable<T>, but the framework also provides a lot of schedulers for dealing with processing on different threads.
One of the schedulers is called EventLoopScheduler and this allows you to create a scheduler that runs on a background thread and only allows one operation to occur at any one time. Any thread can schedule tasks and tasks can be scheduled immediately or in the future, or even recurringly.
The key point here is that you don't need to track multiple background workers as it doesn't matter how many operations you schedule they'll only run in series.
When using Windows Forms you can use a scheduler called ControlScheduler that allows you to set up a scheduler that will post operations to the UI thread.
Once you have these two set up they become very easy to use.
Try this code:
var form1 = new Form();
var label1 = new Label();
label1.AutoSize = true;
form1.Controls.Add(label1);
form1.Show();
var background = new EventLoopScheduler();
var ui = new ControlScheduler(form1);
var someValue = -1;
background.Schedule(() =>
{
var z = 42 * someValue;
var bgTid = System.Threading.Thread.CurrentThread.ManagedThreadId;
ui.Schedule(() =>
{
var uiTid = System.Threading.Thread.CurrentThread.ManagedThreadId;
label1.Text = $"{z} calc on {bgTid} updated on {uiTid}";
});
});
When I run this code I get this form showing on screen:
Clearly the calculation is correct and it can be seen that the thread ids are different.
You can even do more powerful things like this:
var booking =
background.SchedulePeriodic(0, TimeSpan.FromSeconds(1.0), state =>
{
var copy = state;
if (copy % 2 == 0)
{
ui.Schedule(() => label1.Text = copy.ToString());
}
else
{
background.Schedule(() => ui.Schedule(() => label1.Text = "odd"));
}
return ++state;
});
form1.FormClosing += (s, e) => booking.Dispose();
This code is creating a timer to run every second on the background thread. It uses the state variable to keep track of the number of times that it ran and updates the UI with the value of state when it is even, and otherwise, schedules on its own scheduler some code that will schedule on the UI to update label1.Text with the value "odd". It can get quite sophisticated, but everything is serialized and synchronized for you. Since this created a timer there is a mechanism to shut the timer down and that is calling booking.Dispose().
Of course, since this is using the Reactive Framework you could just use standard observables to do the above, like this:
var query =
from n in Observable.Interval(TimeSpan.FromSeconds(1.0), background)
select n % 2 == 0 ? n.ToString() : "odd";
var booking = query.ObserveOn(ui).Subscribe(x => label1.Text = x);
Notice that the same schedulers are used.

If you need to perform the same task for a (potentially large or unlimited) number of variables, this likely means that they are "homogenious" in some manner and thus should probably be combined into some kind of a "registry" container instead.
E.g. I used to do something like this:
var thread = new BgWorker();
pool.Add(thread);
<...>
foreach (var thread in pool) {
do_something();
}
The specific implementation of the registry van be anything, depending on what you need to do with your objects (e.g. a dictionary if you need to get a specific one in other scope than the one it was created in).

Hangfire Background Job with Return Value

I'm switching from Task.Run to Hangfire. In .NET 4.5+ Task.Run can return Task<TResult> which allows me to run tasks that return other than void. I can normally wait and get the result of my task by accessing the property MyReturnedTask.Result
Example of my old code:
public void MyMainCode()
{
List<string> listStr = new List<string>();
listStr.Add("Bob");
listStr.Add("Kate");
listStr.Add("Yaz");
List<Task<string>> listTasks = new List<Task<string>>();
foreach(string str in listStr)
{
Task<string> returnedTask = Task.Run(() => GetMyString(str));
listTasks.Add(returnedTask);
}
foreach(Task<string> task in listTasks)
{
// using task.Result will cause the code to wait for the task if not yet finished.
// Alternatively, you can use Task.WaitAll(listTasks.ToArray()) to wait for all tasks in the list to finish.
MyTextBox.Text += task.Result + Environment.NewLine;
}
}
private string GetMyString(string str)
{
// long execution in order to calculate the returned string
return str + "_finished";
}
As far as I can see from the Quick Start page of Hangfire, your main guy which is BackgroundJob.Enqueue(() => Console.WriteLine("Fire-and-forget"));
perfectly runs the code as a background job but apparently doesn't support jobs that have a return value (like the code I presented above). Is that right? if not, how can I tweak my code in order to use Hangfire?
P.S. I already looked at HostingEnvironment.QueueBackgroundWorkItem (here) but it apparently lacks the same functionality (background jobs have to be void)
EDIT
As #Dejan figured out, the main reason I want to switch to Hangfire is the same reason the .NET folks added QueueBackgroundWorkItem in .NET 4.5.2. And that reason is well described in Scott Hanselman's great article about Background Tasks in ASP.NET. So I'm gonna quote from the article:
QBWI (QueueBackgroundWorkItem) schedules a task which can run in the background, independent of
any request. This differs from a normal ThreadPool work item in that
ASP.NET automatically keeps track of how many work items registered
through this API are currently running, and the ASP.NET runtime will
try to delay AppDomain shutdown until these work items have finished
executing.

One simple solution would be to poll the monitoring API until the job is finished like this:
public static Task Enqueue(Expression<Action> methodCall)
{
string jobId = BackgroundJob.Enqueue(methodCall);
Task checkJobState = Task.Factory.StartNew(() =>
{
while (true)
{
IMonitoringApi monitoringApi = JobStorage.Current.GetMonitoringApi();
JobDetailsDto jobDetails = monitoringApi.JobDetails(jobId);
string currentState = jobDetails.History[0].StateName;
if (currentState != "Enqueued" && currentState != "Processing")
{
break;
}
Thread.Sleep(100); // adjust to a coarse enough value for your scenario
}
});
return checkJobState;
}
Attention: Of course, in a Web-hosted scenario you cannot rely on continuation of the task (task.ContinueWith()) to do more things after the job has finished as the AppDomain might be shut down - for the same reasons you probably want to use Hangfire in the first place.

asynchronous modification of ObservableCollections

I have problems, updating a WPF ListView Bound to a an ObservableCollection<T> within an Task Thread using (Task Parallel Library)
I have an Small Tool reading Edifact Files, and displaying an Count of each Segment (first three letters of a Line).
The contained Segments with their Counts are displayed in an Listbox.
Wenn I initially Load a File all works fine, and I see the GUI Counting up the Segments.
My Programm allowed switching to another File, If I do that (using exactly the same Code) it failes with the following Exception.
This type of CollectionView does not support changes to its SourceCollection from a thread different from the Dispatcher thread.
Here is the Code that fails
public class SegementLineCollection : ObservableCollection<SegmentLine>
{
public void IncrementCount(string Segment)
{
Segment = Segment.ToUpper();
SegmentLine line;
if (this.Select(x => x.SegmentName).Contains(Segment))
{
line = this.Where(x => x.SegmentName == Segment).FirstOrDefault();
}
else
{
line = new SegmentLine();
line.SegmentName = Segment;
this.Add(line); // <--- At this point comes the Exception
}
line.Count++;
}
}
Here is the TPL Call I use:
private string _FileName;
public string FileName
{
get
{
return _FileName;
}
set
{
_FileName = value;
OnPropertyChanged("FileName");
if (!String.IsNullOrEmpty(value))
new Task(() => StartFile()).Start();
}
}
Does anyone have any hit for me?
------------ E D I T ------------------
The provided Answers using TaskScheduler.FromCurrentSynchronizationContext() or Dispatcher did not do the Trick!
Is it possible that changing the Binding when loading the new does the trick.
Here is the Binding I use, the Reader Onject is Switched in the ViewModel, and the GUI is Notfied with INotifyPropertyChanged implementation

Use a dispatcher to access the collection:
if (Dispatcher.CurrentDispatcher.CheckAccess())
this.Add(...)
else
Dispatcher.CurrentDispatcher.Invoke(() => this.Add(...));

You need to call IncrementCount on the GUI thread.
With TPL you can use TaskScheduler.FromCurrentSynchroniztionContext() in your task or continuation.
var task = new Task<string>(() => { return "segment"; })
var task2 = task.ContinueWith(t => IncrementCount(t.Result),
TaskScheduler.FromCurrentSynchroniztionContext());
task.Start();

As you are acting on different thread you need to use Dispatcher.BeginInvoke to run an updates on a collection bound to UI

I have a solution for this kind of problems in the following blog..
http://bathinenivenkatesh.blogspot.co.uk/2011/07/wpf-build-more-responsive-ui.html
it got detailed explanation and code snippets...