I have written a method to implement TPL ActionBlock, which will do a function to find the XPath of the element I am Posting to the block. I am sending the element from a real-time application (whenever the user finds an element Post it to the block). Now, I want to check the block is completed or not when I call a save button from another class. The logic is if the ActionBlok is completed when we click the save button element will save with some save logic, otherwise show a message box not yet ready. For the first time, this idea is working, but from the second element onwards the Actionblock is not accepting any. I am using block.Complete() and block.Completion.Wait() for the save check. Now I can show I am calling the code/logic.
Once create an element, post it ActionBlock
Class1
......
jobQ.Enqueue(familarizedElement);
......
Inside the ActionBlock
Class2
public class TPLCommonDataFlow
{
private ActionBlock<Element> _jobs;
public static bool TPLFlowStatus = false;
public static string JobStatus = string.Empty;
public TPLCommonDataFlow()
{
var executionDataflowBlockOptions = new ExecutionDataflowBlockOptions()
{
MaxDegreeOfParallelism = 2,
};
_jobs = new ActionBlock<Element>((job) =>
{
Thread.Sleep(5);
File.WriteAllText("C:\\Temp\\A.txt", "Started" + _jobs.InputCount.ToString());
JobStatus = "started";
Familiarization.FindAndUpdateXPath(job);
File.WriteAllText("C:\\Temp\\A.txt", "Finished");
}, executionDataflowBlockOptions);
_jobs.Complete();
//Wait for all messages to propagate through the network
_jobs.Completion.Wait();
//Checking all jobs are completed or not, if completed changing the boo, value.
if (_jobs.InputCount == 0)
{
TPLFlowStatus = true;
JobStatus = "stoped";
}
}
}
For the save I am checking the TPLFlowStatus boolean
Class3
if (class2.TPLFlowStatus == true)
{
//Save Logic
}
else
{
//Message Box showing Not Ready
}
Now what I want is to check each time the job is completed or not for each element in the save logic. If the block having two elements in Queue and one is finished, then the MessageBox needs to popup once the save button pressed. If all completed in the block, then need to go to the save logic.
The issue your seeing is that when a block is completed it will no longer accept new messages, you've told the block your done sending messages. To start sending a new batch of messages you can either keep the block alive by not calling complete and tracking the batch completion another way or you can just reset the block. A simple worker like this might be what you're looking for:
public class DataflowWorker
{
private ActionBlock<string> _jobs;
private void BuildJobsHandler()
{
_jobs = new ActionBlock<string>(x => Console.WriteLine(x) /*Do your work in the action block*/);
}
public async Task Enque(string element)
{
await _jobs.SendAsync(element);
}
public async Task CompleteJobsAndSave()
{
_jobs.Complete();
await _jobs.Completion;
//Save data when complete
BuildJobsHandler();
}
}
Related
I have a background thread with a long running task.
The background thread searches for files according to given filters.
The task might be running very long so I do not want to wait for task completion in order to show some results.
Additionally, I do not want to lock up my UI thread in order to check if there are new results from the background task.
I would rather want to notify my main thread: "Hey, there is a new search result added".
Kind of like windows explorer, showing search results while the search is still ongoing:
foreach (FileInfo itemToFilter in unfilteredSearchResults)
{
if (extension == ".wav" || extension == ".flac" || extension == ".mp3")
{
// read audio information such as tags etc. Might take some time per file
WaveFileLib.WaveFile.WAVEFile audioItem = new WaveFileLib.WaveFile.WAVEFile(audioFile.FullName);
// Compare audio information such as tags, title, license, date, length etc to filter
if (Filter.AudioItemMatchesFilter(ref audioItem))
{
lock (threadLock)
{
// add search result to list of filtered audio files
this.AudioItemList.Add(audioItem);
}
// notify main thread in order to refresh ui (if applicable)
ParentView.AudioItemAdded();
}
}
}
the main thread can then generate a view from the newly added item.
Previously, this was quite easy with BeginInvoke but with .net core this possibility seems gone.
What are my alternatives / options to notify main thread about updated search results?
As supposed by Panagiotis Kanavos
public class AudioFileSearcher
{
// task caller
public AudioFileSearcher(string searchPath, bool includeSubFolders, Views.SoundListView.SoundList parentView)
{
this.progress1 = new Progress<int>(
{
backgroundWorker1_ProgressChanged();
});
Task.Run(async () => await FindAudioFiles(progress1));
}
// reference to be updated by task and read by event Handler
private Progress<int> progress1 { get; set; }
// do something on task progress
void backgroundWorker1_ProgressChanged()
{
// Update UI on main thread or whatever
}
// Task
public async Task FindAudioFiles(IProgress<int> progress)
{
foreach(AudioItem itemToProcess in allFoundaudioItems)
{
// do long processing in background task
// a new element has been generated, update UI
if (progress != null) progress.Report(0); // ~fire event
}
}
}
works like a charm, items are beeing loaded
You should use Microsoft's Reactive Framework (aka Rx) - NuGet System.Reactive and add using System.Reactive.Linq; - then you can do this:
IObservable<WaveFileLib.WaveFile.WAVEFile> query =
from itemToFilter in unfilteredSearchResults.ToObservable()
where extension == ".wav" || extension == ".flac" || extension == ".mp3"
from audioItem in Observable.Start(() => new WaveFileLib.WaveFile.WAVEFile(audioFile.FullName))
from x in Observable.Start(() =>
{
var a = audioItem;
var flag = Filter.AudioItemMatchesFilter(ref a);
return new { flag, audioItem = a };
})
where x.flag
select x.audioItem;
IDisposable subscription =
query
.ObserveOnDispatcher()
.Subscribe(audioItem =>
{
/* Your code to update the UI here */
});
It was unclear to me whether it's creating a new WAVEFile or the call to AudioItemMatchesFilter that is taking the time so I wrapped both in Observable.Start. You also had a ref call and that makes the code a bit messy.
Nonetheless, this code handles all of the calls in the background and automatically moves the results as they come in to the dispatcher thread.
I want to wait for Event when I get data in my C# Project.
When program read some data GetData other program creates an event at the end of read this data (call EventForGetData).
So, I need to wait for EventForGetData that finish read.
I wrote this code for this task but believe this code might write more optimal.
public static bool WaitEvent = true;
public void EventForGetData(string variable)
{
WaitEvent = false;
}
public static string ForWaitGetData()
{
WaitEvent = true;
while (WaitEvent)
{
System.Threading.Thread.Sleep(5);
Application.DoEvents();
}
return Variable;
}
public object GetData(){
// start read data
...
// Wait for finish to read data
ForWaitGetData();
// finish read data
...
return MyObject;
}
Try to use Task the first Task is to get your data, at the end do your processing or launch your event:
Example
Task task = Task.Factory.StartNew(() =>
{
//put you code here the first one
}).ContinueWith((rs) =>{
//Launch your event or anything you want
});
Note: the code that you will put inside the ContinueWith will be executed after the code that you write in the StartNew.
Is Looping inside a task really recommended?
example code:
public void doTask(){
Task.Factory.StartNew(() => {
do{
// do tasks here.... call webservice
}while(true till cancelled)
});
}
any answers would be great! :)
because it is a case for my webservice calling right now, and the memory consumption goes out of control.
So may I ask, is looping inside a task really good or not recommended at all?
As Requested by SLC, heres the code:
CancellationTokenSource tokenSrc;
Task myTask;
private void btnStart_Click(object sender, EventArgs e)
{
isPressed = !isPressed;
if(isPressed)
{
tokenSrc = new CancellationTokenSource();
myTask = Task.Factory.StartNew(() =>
{
do{
checkMatches(tokenSrc.Token);
}while(tokenSrc.IsCancellationRequested != true);
}, tokenSrc.Token);
}
else {
try{
tokenSrc.Cancel();
// Log to notepad
}
catch(Exception err){
// Log to notepad
}
finally {
if(myTask.IsCanceled || myTask.IsCompleted || myTask.isFaulted) {
myTask.Dispose();
}
}
}
}
private void checkMatches(CancellationTokenSource token)
{
try
{
if(!token.IsCancellationRequested)
{
//Create Endpoint...
//Bypass ServCertValidation for test purposes
ServicePointManager.ServerCertificateValidationCallback = new RemoteCertificateValidationCallback(delegate {return true;});
using(WebServiceAsmx.SoapClient client = new....)
{
client.CheckResp response = client.chkMatch();
// if's here for the response then put to logs
}
}
}
catch(Exception err)
{
// err.toLogs
}
}
It's perfectly fine to do this, especially if your task runs constantly, for example picking up a message queue.
while (not shutting down)
get next email to send
if exists next email to send
send
else
wait for 10 seconds
wend
Ensure that you have a way to get out if you need to cancel it, like you've done with a flag, and you should be fine.
Regarding webservices:
You should have no problem calling the webservice repeatedly, nor should it cause any memory spikes. However, you should make sure your initialisation code is not inside the loop:
BAD
while (notShuttingDown)
make a new connection
initialise
make a call to the service()
wend
GOOD
make a new connection
initialise
while (notShuttingDown)
make a call to the service
wend
Depending on your webservice it might be more optimal to create a batch operation, for example if your service is HTTP then hitting it repeatedly involves a lot of overhead. A persistent TCP connection might be better because it could be creating and destroying a lot of objects to make the calls.
For example
slow, lots of overhead:
myRecords = { cat, dog, mouse }
foreach record in myRecords
webservice check record
endforeach
faster:
myRecords = { cat, dog, mouse }
webservice check [myRecords] // array of records is passed instead of one by one
Debugging: The most likely risk is that somehow the task is not being disposed correctly - can you add this to your method to debug?
myTask = Task.Factory.StartNew(() =>
{
Console.Writeline("Task Started");
do{
checkMatches(tokenSrc.Token);
Thread.Sleep(10); // Some pause to stop your code from going as fast as it possibly can and putting your CPU usage to 100% (or 100/number of cores%)
}while(tokenSrc.IsCancellationRequested != true);
Console.Writeline("Task Stopped");
}
You might have to change that so it writes to a file or similar depending on if you have a console.
Then run it and make sure that only 1 task is being created.
The main idea here is to fetch some data from somewhere, when it's fetched start writing it, and then prepare the next batch of data to be written, while waiting for the previous write to be complete.
I know that a Task cannot be restarted or reused (nor should it be), although I am trying to find a way to do something like this :
//The "WriteTargetData" method should take the "data" variable
//created in the loop below as a parameter
//WriteData basically do a shedload of mongodb upserts in a separate thread,
//it takes approx. 20-30 secs to run
var task = new Task(() => WriteData(somedata));
//GetData also takes some time.
foreach (var data in queries.Select(GetData))
{
if (task.Status != TaskStatus.Running)
{
//start task with "data" as a parameter
//continue the loop to prepare the next batch of data to be written
}
else
{
//wait for task to be completed
//"restart" task
//continue the loop to prepare the next batch of data to be written
}
}
Any suggestion appreciated ! Thanks. I don't necessarily want to use Task, I just think it might be the way to go.
This may be over simplifying your requirements, but would simply "waiting" for the previous task to complete work for you? You can use Task.WaitAny and Task.WaitAll to wait for previous operations to complete.
pseudo code:
// Method that makes calls to fetch and write data.
public async Task DoStuff()
{
Task currTask = null;
object somedata = await FetchData();
while (somedata != null)
{
// Wait for previous task.
if (currTask != null)
Task.WaitAny(currTask);
currTask = WriteData(somedata);
somedata = await FetchData();
}
}
// Whatever method fetches data.
public Task<object> FetchData()
{
var data = new object();
return Task.FromResult(data);
}
// Whatever method writes data.
public Task WriteData(object somedata)
{
return Task.Factory.StartNew(() => { /* write data */});
}
The Task class is not designed to be restarted. so you Need to create a new task and run the body with the same Parameters. Next i do not see where you start the task with the WriteData function in its body. That will property Eliminate the call of if (task.Status != TaskStatus.Running) There are AFAIK only the class Task and Thread where task is only the abstraction of an action that will be scheduled with the TaskScheduler and executed in different threads ( when we talking about the Common task Scheduler, the one you get when you call TaskFactory.Scheduler ) and the Number of the Threads are equal to the number of Processor Cores.
To you Business App. Why do you wait for the execution of WriteData? Would it be not a lot more easy to gater all data and than submit them into one big Write?
something like ?
public void Do()
{
var task = StartTask(500);
var array = new[] {1000, 2000, 3000};
foreach (var data in array)
{
if (task.IsCompleted)
{
task = StartTask(data);
}
else
{
task.Wait();
task = StartTask(data);
}
}
}
private Task StartTask(int data)
{
var task = new Task(DoSmth, data);
task.Start();
return task;
}
private void DoSmth(object time)
{
Thread.Sleep((int) time);
}
You can use a thread and an AutoResetEvent. I have code like this for several different threads in my program:
These are variable declarations that belong to the main program.
public AutoResetEvent StartTask = new AutoResetEvent(false);
public bool IsStopping = false;
public Thread RepeatingTaskThread;
Somewhere in your initialization code:
RepeatingTaskThread = new Thread( new ThreadStart( RepeatingTaskProcessor ) ) { IsBackground = true; };
RepeatingTaskThread.Start();
Then the method that runs the repeating task would look something like this:
private void RepeatingTaskProcessor() {
// Keep looping until the program is going down.
while (!IsStopping) {
// Wait to receive notification that there's something to process.
StartTask.WaitOne();
// Exit if the program is stopping now.
if (IsStopping) return;
// Execute your task
PerformTask();
}
}
If there are several different tasks you want to run, you can add a variable that would indicate which one to process and modify the logic in PerformTask to pick which one to run.
I know that it doesn't use the Task class, but there's more than one way to skin a cat & this will work.
I am trying to achieve the following behaviour using the Task Parallel Library:
As messages arrive I would like to process them sequentially but in groups. So when the first message arrives it should be processed immediately. If 2 messages come in while the first is being processed then they should be processed in a group of 2.
I can almost get what I want using a BatchBlock linked to an ActionBlock
var batchBlock = new BatchBlock<int>(100);
var actionBlock = new ActionBlock<int[]>(list =>
{
// do work
// now trigger
batchBlock.TriggerBatch();
});
batchBlock.LinkTo(actionBlock);
The problem with the code above is that if an item arrives after the TriggerBatch() call then it needs to wait for the batch to fill up. If I trigger batch after each post instead then the ActionBlock always receives single messages.
Instead of BatchBlock, you could use BufferBlock with a Task the receives items from it and resends them in batches to the target, according to your logic. Because you need to try to send a message containing a batch, and cancel it if another item comes in, the target block (actionBlock in your sample) has to have BoundedCapacity set to 1.
So, what you do is that you first receive something. When you have that, you start sending asynchronously and you also try to receive more items. If sending completes first, you start over. If receiving completes first, you cancel sending, add the received items to the batch, and then start both asynchronous actions again.
The actual code is a bit more complicated, because it needs to deal with some corner cases (receiving and sending complete at the same time; sending couldn't be canceled; receiving completed, because the whole was completed; exceptions):
public static ITargetBlock<T> CreateBatchingWrapper<T>(
ITargetBlock<IReadOnlyList<T>> target)
{
// target should have BoundedCapacity == 1,
// but there is no way to check for that
var source = new BufferBlock<T>();
Task.Run(() => BatchItems(source, target));
return source;
}
private static async Task BatchItems<T>(
IReceivableSourceBlock<T> source, ITargetBlock<IReadOnlyList<T>> target)
{
try
{
while (true)
{
var messages = new List<T>();
// wait for first message in batch
if (!await source.OutputAvailableAsync())
{
// source was completed, complete target and return
target.Complete();
return;
}
// receive all there is right now
source.ReceiveAllInto(messages);
// try sending what we've got
var sendCancellation = new CancellationTokenSource();
var sendTask = target.SendAsync(messages, sendCancellation.Token);
var outputAvailableTask = source.OutputAvailableAsync();
while (true)
{
await Task.WhenAny(sendTask, outputAvailableTask);
// got another message, try cancelling send
if (outputAvailableTask.IsCompleted
&& outputAvailableTask.Result)
{
sendCancellation.Cancel();
// cancellation wasn't successful
// and the message was received, start another batch
if (!await sendTask.EnsureCancelled() && sendTask.Result)
break;
// send was cancelled, receive messages
source.ReceiveAllInto(messages);
// restart both Tasks
sendCancellation = new CancellationTokenSource();
sendTask = target.SendAsync(
messages, sendCancellation.Token);
outputAvailableTask = source.OutputAvailableAsync();
}
else
{
// we get here in three situations:
// 1. send was completed succesfully
// 2. send failed
// 3. input has completed
// in cases 2 and 3, this await is necessary
// in case 1, it's harmless
await sendTask;
break;
}
}
}
}
catch (Exception e)
{
source.Fault(e);
target.Fault(e);
}
}
/// <summary>
/// Returns a Task that completes when the given Task completes.
/// The Result is true if the Task was cancelled,
/// and false if it completed successfully.
/// If the Task was faulted, the returned Task is faulted too.
/// </summary>
public static Task<bool> EnsureCancelled(this Task task)
{
return task.ContinueWith(t =>
{
if (t.IsCanceled)
return true;
if (t.IsFaulted)
{
// rethrow the exception
ExceptionDispatchInfo.Capture(task.Exception.InnerException)
.Throw();
}
// completed successfully
return false;
});
}
public static void ReceiveAllInto<T>(
this IReceivableSourceBlock<T> source, List<T> targetCollection)
{
// TryReceiveAll would be best suited for this, except it's bugged
// (see http://connect.microsoft.com/VisualStudio/feedback/details/785185)
T item;
while (source.TryReceive(out item))
targetCollection.Add(item);
}
You can also use Timer; which will Trigger Batch on every 10 seconds