Develop Reference

ThreadPool behavior and TCPListener

I am starting TCPListener like this and when the job done I close the Socket.
I would like to know if the Thread which I start like this
ThreadPool.QueueUserWorkItem(ConnectClientsThredProc, args);
is going to destroy itself so I don't need any external control over it.
Would anyone so pleased to explain have I worry about it or don't.
Thank you!
class TCPListenerManager
{
TcpListener tcpListener;
HostListenerItem hostListener;
private bool _isServerWorking = false;
public TCPListenerManager(HostListenerItem hostListenerItem)
{
hostListener = hostListenerItem;
tcpListener = new TcpListener(IPAddress.Parse(hostListenerItem.IP4), hostListenerItem.Port);
var t = Task.Factory.StartNew(async () =>
{
await StartAsync(hostListenerItem.ClientsMax);
});
}
public async Task StartAsync(int clientsMax)
{
tcpListener.Start();
_isServerWorking = true;
for (int i = 0; i < clientsMax; i++)
{
if (_isServerWorking)
{
ServerConnectedEventArgs args = new ServerConnectedEventArgs();
args.TcpClient = await tcpListener.AcceptTcpClientAsync();
args.HostListener = hostListener;
OnServerConnected(args);
ThreadPool.QueueUserWorkItem(ConnectClientsThredProc, args);
}
}
}
private void ConnectClientsThredProc(object obj)
{
var args = (ServerConnectedEventArgs)obj;
if (args.TcpClient.Connected)
{
// Do some job and disconnect
args.TcpClient.Client.Close();
args.TcpClient.Client = null;
}
}
}

When ConnectClientsThredProc exits, thread is not "gone" but returned back to the pool (that's why thread pool exists in the first place). Anyway, you should not care about that indeed, unless you have long-running task performed in ConnectClientsThredProc. If it is long running - better not use thread pool thread, but start a new one (via Task.Factory.StartNew + TaskCreationOptions.LongRunning for example).
Also, you use Task.Factory, async\await, ThreadPool.QueueUserWorkItem, all that in a short piece of code, mixed together. Maybe you need to understand a bit better what are those tools and what are similarities and differences between them (especially async\await). For example, what is the reason for this:
var t = Task.Factory.StartNew(async () =>
{
await StartAsync(hostListenerItem.ClientsMax);
});
You start a task\thread, inside which you start yet another task and then wait for it to exit - makes little sense.
Instead of ThreadPool.QueueUserWorkItem you might use Task.Run with the same effect.

Thread safe with Linq and Tasks on a Collection

Given some code like so
public class CustomCollectionClass : Collection<CustomData> {}
public class CustomData
{
string name;
bool finished;
string result;
}
public async Task DoWorkInParallel(CustomCollectionClass collection)
{
// collection can be retrieved from a DB, may not exist.
if (collection == null)
{
collection = new CustomCollectionClass();
foreach (var data in myData)
{
collection.Add(new CustomData()
{
name = data.Name;
});
}
}
// This part doesn't feel safe. Not sure what to do here.
var processTasks = myData.Select(o =>
this.DoWorkOnItemInCollection(collection.Single(d => d.name = o.Name))).ToArray();
await Task.WhenAll(processTasks);
await SaveModifedCollection(collection);
}
public async Task DoWorkOnItemInCollection(CustomData data)
{
await DoABunchOfWorkElsewhere();
// This doesn't feel safe either. Lock here?
data.finished = true;
data.result = "Parallel";
}
As I noted in a couple comments inline, it doesn't feel safe for me to do the above, but I'm not sure. I do have a collection of elements that I'd like to assign a unique element to each parallel task and have those tasks be able to modify that single element of the collection based on what work is done. End result being, I wanted to save the collection after individual, different elements have been modified in parallel. If this isn't a safe way to do it, how best would I go about this?

Your code is the right way to do this, assuming starting DoABunchOfWorkElsewhere() multiple times is itself safe.
You don't need to worry about your LINQ query, because it doesn't actually run in parallel. All it does is to invoke DoWorkOnItemInCollection() multiple times. Those invocations may work in parallel (or not, depending on your synchronization context and the implementation of DoABunchOfWorkElsewhere()), but the code you showed is safe.

Your above code should work without issue. You are passing off one item to each worker thread. I'm not so sure about the async attribute. You might just return a Task, and then in your method do:
public Task DoWorkOnItemInCollection(CustomData data)
{
return Task.Run(() => {
DoABunchOfWorkElsewhere().Wait();
data.finished = true;
data.result = "Parallel";
});
}
You might want to be careful, with large amount of items, you could overflow your max thread count with background threads. In this case, c# just deletes your threads, which can be difficult to debug later.
I have done this before, It might be easier if instead of handing the whole collection to some magic linq, rather do a classic consumer problem:
class ParallelWorker<T>
{
private Action<T> Action;
private Queue<T> Queue = new Queue<T>();
private object QueueLock = new object();
private void DoWork()
{
while(true)
{
T item;
lock(this.QueueLock)
{
if(this.Queue.Count == 0) return; //exit thread
item = this.Queue.DeQueue();
}
try { this.Action(item); }
catch { /*...*/ }
}
}
public void DoParallelWork(IEnumerable<T> items, int maxDegreesOfParallelism, Action<T> action)
{
this.Action = action;
this.Queue.Clear();
this.Queue.AddRange(items);
List<Thread> threads = new List<Thread>();
for(int i = 0; i < items; i++)
{
ParameterizedThreadStart threadStart = new ParameterizedThreadStart(DoWork);
Thread thread = new Thread(threadStart);
thread.Start();
threads.Add(thread);
}
foreach(Thread thread in threads)
{
thread.Join();
}
}
}
This was done IDE free, so there may be typos.

I'm going to make the suggestion that you use Microsoft's Reactive Framework (NuGet "Rx-Main") to do this task.
Here's the code:
public void DoWorkInParallel(CustomCollectionClass collection)
{
var query =
from x in collection.ToObservable()
from r in Observable.FromAsync(() => DoWorkOnItemInCollection(x))
select x;
query.Subscribe(x => { }, ex => { }, async () =>
{
await SaveModifedCollection(collection);
});
}
Done. That's it. Nothing more.
I have to say though, that when I tried to get your code to run it was full of bugs and issues. I suspect that the code you posted isn't your production code, but an example you wrote specifically for this question. I suggest that you try to make a running compilable example before posting.
Nevertheless, my suggestion should work for you with a little tweaking.
It is multi-threaded and thread-safe. And it does do cleanly save the modified collection when done.

Task keeps looping when it should be finished

I'm trying to let my accounts do multiple logins with tasks. I am by no means an expert, I've only just started, but I'm stuck with the following problem I have.
For some reason the tasks keep looping, but they should not. I can't really figure out what the problem is, so I figured that someone on here might know the solution.
public async void Login()
{
for (int i = 0; i < objectsToCreate; i++)
{
try
{
this.Invoke((MethodInvoker)delegate()
{
MyEmail = listView1.Items[i].SubItems[0].Text;
MyPassword = listView1.Items[i].SubItems[1].Text;
MySecurity = listView1.Items[i].SubItems[2].Text;
});
var loginDetails = new LoginDetails(MyEmail, MyPassword, MySecurity, Platform.Ps3);
client[i] = new FutClient();
var loginResponse = await client[i].LoginAsync(loginDetails);
MessageBox.Show("Succesful login");
}
catch (Exception ex)
{
string foutMelding = ex.InnerException.ToString();
ListViewItem exception = new ListViewItem(time);
exception.SubItems.Add(foutMelding);
listView2.BeginInvoke(new MethodInvoker(() => listView2.Items.Add(exception)));
}
}
}
public void StartLogin()
{
Task[] tasks = new Task[objectsToCreate];
for (int i = 0; i < objectsToCreate; i++)
{
tasks[i] = new Task(() => Login());
}
foreach (Task task in tasks)
{
task.Start();
}
}
I call the StartLogin method with a button in my form. It then proceeds to login the accounts, but for some reason the tasks keep looping and I really can't figure out the reason, since I only just started with using Tasks. I would really appreciate it if someone could help me out.

It doesn't loop indefinitely - it executes exactly 16 times. This is because you essentially have a doubly nested for loop - one in your StartLogin method, and one in your Login method. Perhaps you aren't expecting the contents of your Login method to execute 4 times each time your Login method is called?

Based on the code you posted, I would expect the logic in the loop inside the Login() method to execute 16 times: 4 times for each time the Login() method itself is called, due to the for (int i = 0; i < objectsToCreate; i++) statement at the beginning of that method; and that, 4 times because you are creating 4 tasks to call the method.
It's hard to know what you meant to write, but I'm guessing you want to remove the loop from the Login() method itself. It looks like an artifact left over from a previous implementation attempt.
As an aside: it is a little odd to be mixing invocation of an async method with an explicit Task invocation of that method. It seems to me you could accomplish much the same result by simply calling the Login() method four times (minus the loop in the Login() method, of course).
Here's how I'd write the code:
public async void Login()
{
try
{
MyEmail = listView1.Items[i].SubItems[0].Text;
MyPassword = listView1.Items[i].SubItems[1].Text;
MySecurity = listView1.Items[i].SubItems[2].Text;
var loginDetails = new LoginDetails(MyEmail, MyPassword, MySecurity, Platform.Ps3);
client[i] = new FutClient();
var loginResponse = await client[i].LoginAsync(loginDetails);
MessageBox.Show("Succesful login");
}
catch (Exception ex)
{
string foutMelding = ex.InnerException.ToString();
ListViewItem exception = new ListViewItem(time);
exception.SubItems.Add(foutMelding);
listView2.Items.Add(exception);
}
}
public void StartLogin()
{
for (int i = 0; i < objectsToCreate; i++)
{
var _ = Login();
}
}
Note that, in this approach, since the Login() method is initially called from the UI thread (I presume...you didn't show the actual caller of the StartLogin() method, you don't need to use Invoke() or BeginInvoke() when accessing UI elements in that method.
(The var _ = is just to keep the compiler from complaining that I didn't await the async method. Here, that's on purpose :) ).

Threads problems with return value

I'm trying to start one thread, receive string from it and then start n-counts of threads with the received string. Code:
private void button2_Click(object sender, EventArgs e)
{
string post = null;
sync = new ManualResetEvent(false);
var thr = new Thread[1];
thr[0] = new Thread(delegate() { post = create_note(); });
thr[0].IsBackground = true;
thr[0].Start();
sync.WaitOne();
decimal value = Program.Data.numericUpDown1;
int i = 0;
int j = (int)(value);
thr = new Thread[j];
for (; i < j; i++)
{
thr[i] = new Thread(() => invite(post));
thr[i].IsBackground = true;
thr[i].Start();
}
}
public string create_note()
{
while (true)
{
string acc = "";
string proxy = "";
if (Program.Data.checkBox1 || Program.Data.checkBox2)
{
if (Program.Data.checkBox1)
Proxy.type = "http";
else if (Program.Data.checkBox2)
Proxy.type = "socks5";
lock (locker)
{
if (Proxy.proxies.Count == 0)
{
foreach (string prox in File.ReadAllLines(proxy_path))
{
if (prox.Contains(":"))
Proxy.proxies.Add(prox);
}
}
}
proxy = rand_proxy();
}
else if (!Program.Data.checkBox1 && !Program.Data.checkBox2)
Proxy.type = "none";
if (edit_accs.Count == 0)
{
break;
}
else
acc = edit_accs.Dequeue();
Od_post od_post = new Od_post(acc, proxy, Proxy.type);
string login = od_post.Auth();
if ()
{
string url = rand_url();
var text = new RandomString(Program.Data.textBox3).ToString();
string wall_post_text = od_post.wall_post_text(get_text(text), url);
if (wall_post_text == "Good")
{
string image_add = od_post.image_add(post_image_path);
if (image_add.Split('|')[0] == "Good")
{
if (Program.Data.checkBox5)
{
string change_name = od_post.change_name();
if (change_name == "Changed")
{
}
else
{
}
}
sync.Set();
return image_add.Split('|')[1];
}
else
{
}
}
else
{
}
}
else
{
lock (locker)
{
accs.Enqueue(acc);
Proxy.proxies.Remove(proxy);
}
}
}
return "Failed";
}
But it doesn't work. My app hangs, and the post doesn't receive the return value from create_note(). Why?

thr[0].Start();
sync.WaitOne();
This is a pretty common mistake when using a thread. Threading can give you two benefits. One is that code can run concurrently, allowing you to get more work done if you have a machine with multiple cores. And more commonly, it can run code asynchronously, a consideration when you have a user interface that you want to keep responsive.
You are getting neither. It does not run concurrently, your main thread isn't doing any work since it is waiting for the thread to complete. And it doesn't run asynchronously either, your main thread is frozen while the thread is doing its job.
You instead get all the disadvantages of a thread. A classic threading bug is a "race", your code has one. You hope that the "post" variable is assigned after the WaitOne() method completes. It might be, but the odds are just not that good. Since you call Set() before the assignment executes. It can only work correctly when you call Set() after the assignment. Deadlock is another classic threading bug, I don't see one but your debugger can easily show you. Deadlock is very likely here because you are freezing your main thread. Making calls like Invoke() on the thread is going to deadlock.
There's a very simple substitute for your code that does everything that you hope your current code does, minus the threading bugs:
string post = create_note();
Problem solved.
Minus addressing the reason that you considered writing this code in the first place. Which does require that you move all of the code that's now after the WaitOne() call into a callback that runs when the thread completes. Such code tends to be difficult to write, dealing with asynchronicity isn't that easy. But you can get help from .NET to get this right, like the BackgroundWorker.RunWorkerCompleted event, the Task class with its TaskScheduler.FromCurrentSynchronizationContext() method. And the async keyword added to C# version 5.

Implementing correct completion of a retryable block

Teaser: guys, this question is not about how to implement retry policy. It's about correct completion of a TPL Dataflow block.
This question is mostly a continuation of my previous question Retry policy within ITargetBlock. The answer to this question was #svick's smart solution that utilizes TransformBlock (source) and TransformManyBlock (target). The only problem left is to complete this block in a right way: wait for all the retries to be completed first, and then complete the target block. Here is what I ended up with (it's just a snippet, don't pay too many attention to a non-threadsafe retries set):
var retries = new HashSet<RetryingMessage<TInput>>();
TransformManyBlock<RetryableMessage<TInput>, TOutput> target = null;
target = new TransformManyBlock<RetryableMessage<TInput>, TOutput>(
async message =>
{
try
{
var result = new[] { await transform(message.Data) };
retries.Remove(message);
return result;
}
catch (Exception ex)
{
message.Exceptions.Add(ex);
if (message.RetriesRemaining == 0)
{
if (failureHandler != null)
failureHandler(message.Exceptions);
retries.Remove(message);
}
else
{
retries.Add(message);
message.RetriesRemaining--;
Task.Delay(retryDelay)
.ContinueWith(_ => target.Post(message));
}
return null;
}
}, dataflowBlockOptions);
source.LinkTo(target);
source.Completion.ContinueWith(async _ =>
{
while (target.InputCount > 0 || retries.Any())
await Task.Delay(100);
target.Complete();
});
The idea is to perform some kind of polling and verify whether there are still messages that waiting to be processed and there are no messages that require retrying. But in this solution I don't like the idea of polling.
Yes, I can encapsulate the logic of adding/removing retries into a separate class, and even e.g. perform some action when the set of retries becomes empty, but how to deal with target.InputCount > 0 condition? There is not such a callback that get called when there are no pending messages for the block, so it seems that verifying target.ItemCount in a loop with a small delay is an only option.
Does anybody knows a smarter way to achieve this?

Maybe a ManualResetEvent can do the trick for you.
Add a public property to TransformManyBlock
private ManualResetEvent _signal = new ManualResetEvent(false);
public ManualResetEvent Signal { get { return _signal; } }
And here you go:
var retries = new HashSet<RetryingMessage<TInput>>();
TransformManyBlock<RetryableMessage<TInput>, TOutput> target = null;
target = new TransformManyBlock<RetryableMessage<TInput>, TOutput>(
async message =>
{
try
{
var result = new[] { await transform(message.Data) };
retries.Remove(message);
// Sets the state of the event to signaled, allowing one or more waiting threads to proceed
if(!retries.Any()) Signal.Set();
return result;
}
catch (Exception ex)
{
message.Exceptions.Add(ex);
if (message.RetriesRemaining == 0)
{
if (failureHandler != null)
failureHandler(message.Exceptions);
retries.Remove(message);
// Sets the state of the event to signaled, allowing one or more waiting threads to proceed
if(!retries.Any()) Signal.Set();
}
else
{
retries.Add(message);
message.RetriesRemaining--;
Task.Delay(retryDelay)
.ContinueWith(_ => target.Post(message));
}
return null;
}
}, dataflowBlockOptions);
source.LinkTo(target);
source.Completion.ContinueWith(async _ =>
{
//Blocks the current thread until the current WaitHandle receives a signal.
target.Signal.WaitOne();
target.Complete();
});
I am not sure where your target.InputCount is set. So at the place you change target.InputCount you can add following code:
if(InputCount == 0) Signal.Set();

Combining hwcverwe answer and JamieSee comment could be the ideal solution.
First, you need to create more than one event:
var signal = new ManualResetEvent(false);
var completedEvent = new ManualResetEvent(false);
Then, you have to create an observer, and subscribe to the TransformManyBlock, so you are notified when a relevant event happens:
var observer = new RetryingBlockObserver<TOutput>(completedEvent);
var observable = target.AsObservable();
observable.Subscribe(observer);
The observable can be quite easy:
private class RetryingBlockObserver<T> : IObserver<T> {
private ManualResetEvent completedEvent;
public RetryingBlockObserver(ManualResetEvent completedEvent) {
this.completedEvent = completedEvent;
}
public void OnCompleted() {
completedEvent.Set();
}
public void OnError(Exception error) {
//TODO
}
public void OnNext(T value) {
//TODO
}
}
And you can wait for either the signal, or completion (exhaustion of all the source items), or both
source.Completion.ContinueWith(async _ => {
WaitHandle.WaitAll(completedEvent, signal);
// Or WaitHandle.WaitAny, depending on your needs!
target.Complete();
});
You can inspect the result value of WaitAll to understand which event was set, and react accordingly.
You can also add other events to the code, passing them to the observer, so that it can set them when needed. You can differentiate your behaviour and respond differently when an error is raised, for example