Develop Reference

C# Multithread dequeue List<string>

First of all, sorry for my english. I'm doing some C# exercices but I'm in a trouble with this.
Actually, I need to create 2 threads using Thread class. These threads needs to Dequeue items from a List (I have created a Dequeue method to do that) but I need to configure these 2 threads to Deqeueue this List at the same time. Aditionaly, this method needs to show the thread that executed him and the removed the object.
That's the method called by Thread:
private static object Lock = new object();
public string Consume()
{
string result = "";
while (StringList.Count > 0)
{
lock (Lock)
{
result = Dequeue();
}
}
return result;
}
And the thread call:
Thread t1 = new Thread(() => { Console.WriteLine("Thread 1: " + QC.Consume()); });
Thread t2 = new Thread(() => { Console.WriteLine("Thread 2: " + QC.Consume()); });
t1.Start();
t2.Start();
Actually, the program "is working" but it is only being executed by an unique thread. I don't know how to dequeue items from list at the same time with different threads + show which thread dequeued the item.
I'm not asking just to know if someone know the problem, I want to understand the problem and solve it.
Thanks to all!

I am trying to call a method in a loop .It should be called only 20 times in 10 seconds . I am using semaphore like the below code

By using the below code firstly some of the calls are not getting made lets say out of 250 , 238 calls are made and rest doesn't.Secondly I am not sure if the calls are made at the rate of 20 calls per 10 seconds.
public List<ShowData> GetAllShowAndTheirCast()
{
ShowResponse allShows = GetAllShows();
ShowCasts showCast = new ShowCasts();
showCast.showCastList = new List<ShowData>();
using (Semaphore pool = new Semaphore(20, 20))
{
for (int i = 0; i < allShows.Shows.Length; i++)
{
pool.WaitOne();
Thread t = new Thread(new ParameterizedThreadStart((taskId) =>
{
showCast.showCastList.Add(MapResponse(allShows.Shows[i]));
}));
pool.Release();
t.Start(i);
}
}
//for (int i = 0; i < allShows.Shows.Length; i++)
//{
// showCast.showCastList.Add(MapResponse(allShows.Shows[i]));
//}
return showCast.showCastList;
}
public ShowData MapResponse(Show s)
{
CastResponse castres = new CastResponse();
castres.CastlistResponse = (GetShowCast(s.id)).CastlistResponse;
ShowData sd = new ShowData();
sd.id = s.id;
sd.name = s.name;
if (castres.CastlistResponse != null && castres.CastlistResponse.Any())
{
sd.cast = new List<CastData>();
foreach (var item in castres.CastlistResponse)
{
CastData cd = new CastData();
cd.birthday = item.person.birthday;
cd.id = item.person.id;
cd.name = item.person.name;
sd.cast.Add(cd);
}
}
return sd;
}
public ShowResponse GetAllShows()
{
ShowResponse response = new ShowResponse();
string showUrl = ClientAPIUtils.apiUrl + "shows";
response.Shows = JsonConvert.DeserializeObject<Show[]>(ClientAPIUtils.GetDataFromUrl(showUrl));
return response;
}
public CastResponse GetShowCast(int showid)
{
CastResponse res = new CastResponse();
string castUrl = ClientAPIUtils.apiUrl + "shows/" + showid + "/cast";
res.CastlistResponse = JsonConvert.DeserializeObject<List<Cast>>(ClientAPIUtils.GetDataFromUrl(castUrl));
return res;
}
All the Calls should be made , but I am not sure where they are getting aborted and even please let me know how to check the rate of calls being made.

I'm assuming that your goal is to process all data about shows but no more than 20 at once.
For that kind of task you should probably use ThreadPool and limit maximum number of concurrent threads using SetMaxThreads.
https://learn.microsoft.com/en-us/dotnet/api/system.threading.threadpool?view=netframework-4.7.2
You have to make sure that collection that you are using to store your results is thread-safe.
showCast.showCastList = new List<ShowData>();
I don't think that standard List is thread-safe. Thread-safe collection is ConcurrentBag (there are others as well). You can make standard list thread-safe but it requires more code. After you are done processing and need to have results in list or array you can convert collection to desired type.
https://learn.microsoft.com/en-us/dotnet/api/system.collections.concurrent.concurrentbag-1?view=netframework-4.7.2
Now to usage of semaphore. What your semaphore is doing is ensuring that maximum 20 threads can be created at once. Assuming that this loop runs in your app main thread your semaphore has no purpose. To make it work you need to release semaphore once thread is completed; but you are calling thread Start() after calling Release(). That results in thread being executed outside "critical area".
using (Semaphore pool = new Semaphore(20, 20)) {
for (int i = 0; i < allShows.Shows.Length; i++) {
pool.WaitOne();
Thread t = new Thread(new ParameterizedThreadStart((taskId) =>
{
showCast.showCastList.Add(MapResponse(allShows.Shows[i]));
pool.Release();
}));
t.Start(i);
}
}
I did not test this solution; additional problems might arise.
Another issue with this program is that it does not wait for all threads to complete. Once all threads are started; program will end. It is possible (and in your case I'm sure) that not all threads completed its operation; this is why ~240 data packets are done when program finishes.
thread.Join();
But if called right after Start() it will stop main thread until it is completed so to keep program concurrent you need to create a list of threads and Join() them at the end of program. It is not the best solution. How to wait on all threads that program can add to ThreadPool
Wait until all threads finished their work in ThreadPool
As final note you cannot access loop counter like that. Final value of loop counter is evaluated later and with test I ran; code has tendency to process odd records twice and skip even. This is happening because loop increases counter before previous thread is executed and causes to access elements outside bounds of array.
Possible solution to that is to create method that will create thread. Having it in separate method will evaluate allShows.Shows[i] to show before next loop pass.
public void CreateAndStartThread(Show show, Semaphore pool, ShowCasts showCast)
{
pool.WaitOne();
Thread t = new Thread(new ParameterizedThreadStart((s) => {
showCast.showCastList.Add(MapResponse((Show)s));
pool.Release();
}));
t.Start(show);
}
Concurrent programming is tricky and I would highly recommend to do some exercises with examples on common pitfalls. Books on C# programming are sure to have a chapter or two on the topic. There are plenty of online courses and tutorials on this topic to learn from.
Edit:
Working solution. Still might have some issues.
public ShowCasts GetAllShowAndTheirCast()
{
ShowResponse allShows = GetAllShows();
ConcurrentBag<ShowData> result = new ConcurrentBag<ShowData>();
using (var countdownEvent = new CountdownEvent(allShows.Shows.Length))
{
using (Semaphore pool = new Semaphore(20, 20))
{
for (int i = 0; i < allShows.Shows.Length; i++)
{
CreateAndStartThread(allShows.Shows[i], pool, result, countdownEvent);
}
countdownEvent.Wait();
}
}
return new ShowCasts() { showCastList = result.ToList() };
}
public void CreateAndStartThread(Show show, Semaphore pool, ConcurrentBag<ShowData> result, CountdownEvent countdownEvent)
{
pool.WaitOne();
Thread t = new Thread(new ParameterizedThreadStart((s) =>
{
result.Add(MapResponse((Show)s));
pool.Release();
countdownEvent.Signal();
}));
t.Start(show);
}

Thread safe with Linq and Tasks on a Collection

Given some code like so
public class CustomCollectionClass : Collection<CustomData> {}
public class CustomData
{
string name;
bool finished;
string result;
}
public async Task DoWorkInParallel(CustomCollectionClass collection)
{
// collection can be retrieved from a DB, may not exist.
if (collection == null)
{
collection = new CustomCollectionClass();
foreach (var data in myData)
{
collection.Add(new CustomData()
{
name = data.Name;
});
}
}
// This part doesn't feel safe. Not sure what to do here.
var processTasks = myData.Select(o =>
this.DoWorkOnItemInCollection(collection.Single(d => d.name = o.Name))).ToArray();
await Task.WhenAll(processTasks);
await SaveModifedCollection(collection);
}
public async Task DoWorkOnItemInCollection(CustomData data)
{
await DoABunchOfWorkElsewhere();
// This doesn't feel safe either. Lock here?
data.finished = true;
data.result = "Parallel";
}
As I noted in a couple comments inline, it doesn't feel safe for me to do the above, but I'm not sure. I do have a collection of elements that I'd like to assign a unique element to each parallel task and have those tasks be able to modify that single element of the collection based on what work is done. End result being, I wanted to save the collection after individual, different elements have been modified in parallel. If this isn't a safe way to do it, how best would I go about this?

Your code is the right way to do this, assuming starting DoABunchOfWorkElsewhere() multiple times is itself safe.
You don't need to worry about your LINQ query, because it doesn't actually run in parallel. All it does is to invoke DoWorkOnItemInCollection() multiple times. Those invocations may work in parallel (or not, depending on your synchronization context and the implementation of DoABunchOfWorkElsewhere()), but the code you showed is safe.

Your above code should work without issue. You are passing off one item to each worker thread. I'm not so sure about the async attribute. You might just return a Task, and then in your method do:
public Task DoWorkOnItemInCollection(CustomData data)
{
return Task.Run(() => {
DoABunchOfWorkElsewhere().Wait();
data.finished = true;
data.result = "Parallel";
});
}
You might want to be careful, with large amount of items, you could overflow your max thread count with background threads. In this case, c# just deletes your threads, which can be difficult to debug later.
I have done this before, It might be easier if instead of handing the whole collection to some magic linq, rather do a classic consumer problem:
class ParallelWorker<T>
{
private Action<T> Action;
private Queue<T> Queue = new Queue<T>();
private object QueueLock = new object();
private void DoWork()
{
while(true)
{
T item;
lock(this.QueueLock)
{
if(this.Queue.Count == 0) return; //exit thread
item = this.Queue.DeQueue();
}
try { this.Action(item); }
catch { /*...*/ }
}
}
public void DoParallelWork(IEnumerable<T> items, int maxDegreesOfParallelism, Action<T> action)
{
this.Action = action;
this.Queue.Clear();
this.Queue.AddRange(items);
List<Thread> threads = new List<Thread>();
for(int i = 0; i < items; i++)
{
ParameterizedThreadStart threadStart = new ParameterizedThreadStart(DoWork);
Thread thread = new Thread(threadStart);
thread.Start();
threads.Add(thread);
}
foreach(Thread thread in threads)
{
thread.Join();
}
}
}
This was done IDE free, so there may be typos.

I'm going to make the suggestion that you use Microsoft's Reactive Framework (NuGet "Rx-Main") to do this task.
Here's the code:
public void DoWorkInParallel(CustomCollectionClass collection)
{
var query =
from x in collection.ToObservable()
from r in Observable.FromAsync(() => DoWorkOnItemInCollection(x))
select x;
query.Subscribe(x => { }, ex => { }, async () =>
{
await SaveModifedCollection(collection);
});
}
Done. That's it. Nothing more.
I have to say though, that when I tried to get your code to run it was full of bugs and issues. I suspect that the code you posted isn't your production code, but an example you wrote specifically for this question. I suggest that you try to make a running compilable example before posting.
Nevertheless, my suggestion should work for you with a little tweaking.
It is multi-threaded and thread-safe. And it does do cleanly save the modified collection when done.

Task.Factory.StartNew depend on parent thread?

I'm running this thread inside a method from a WCF service library.
The code below is executed at the end of the method. I do this because i don't want the user to wait for a background process to complete that does not affect the output from the WCF to the client.
The problem that i have now is that if i execute that thread and the client gets the response, the parent thread is killed; killing this thread as well. How do i make it so that the parent thread waits for this thread to finish, while performing the rest of the operations?
class Program
{
static void Main(string[] args)
{
Dictionary<string, string> sampleDict = getPopulatedDictionary();
var result = run(sampleDict);
}
public static int run(Dictionary<string, string> sampleDict_)
{
PerformCalculations(sampleDict_);
if (sampleDict_.Keys.Count > 10)
{
System.Threading.Tasks.Task.Factory.StartNew(() =>
{
backgroundprocess(sampleDict_);
});
}
//after returning i still want it to run
return sampleDict_.Keys.Count;
}
private static void backgroundprocess(Dictionary<string,string> dict)
{
foreach (var k in dict.Keys)
{
dict[k] = new Random().Next(2666).ToString();
}
}
}
In short, i want this method to kick off that thread and move onto return the value X but still wait for that thread to finish AFTER it returns the value.

Couldn't you do it as a continuation of the parent task. So execute
FameMappingEntry.SaveFameDBMap(toSaveIdentifiers); as a continuation of a successful completion of the parent task. And then you can wait on the continutation.
var childTask = parentTask.ContinueWith((pt) =>
{
FameMappingEntry.SaveFameDBMap(toSaveIdentifiers);
}, TaskContinuationOptions.OnlyOnRanToCompletion);
And then you can decide if you want to wait on the child task or use another continuation.

If you aren't going to do anything except wait for the background thread to complete, then you might as well just not create the new background thread in the first place and execute the code in-line.

Try this:
var task = System.Threading.Tasks.Task.Factory.StartNew(() =>
{
lock (toSaveIdentifiers)
{
FameMappingEntry.SaveFameDBMap(toSaveIdentifiers);
}
);
int x = dosomething();
task.Wait();
return x;
You should also lock objects in the thread that uses them, and not some other random thread.

Multithreading: When would I use a Join?

I see online that it says I use myThread.Join(); when I want to block my thread until another thread finishes. (One of the things I don't get about this is what if I have multiple threads).
But generally, I just don't get when I'd use .Join() or a condition that it's useful for. Can anyone please explain this to me like I'm a fourth grader? Very simple explanation to understand will get my answer vote.

Let's say you want to start some worker threads to perform some kind of calculation, and then do something afterwards with all the results.
List<Thread> workerThreads = new List<Thread>();
List<int> results = new List<int>();
for (int i = 0; i < 5; i++) {
Thread thread = new Thread(() => {
Thread.Sleep(new Random().Next(1000, 5000));
lock (results) {
results.Add(new Random().Next(1, 10));
}
});
workerThreads.Add(thread);
thread.Start();
}
// Wait for all the threads to finish so that the results list is populated.
// If a thread is already finished when Join is called, Join will return immediately.
foreach (Thread thread in workerThreads) {
thread.Join();
}
Debug.WriteLine("Sum of results: " + results.Sum());
Oh yeah, and don't use Random like that, I was just trying to write a minimal, easily understandable example. It ends up not really being random if you create new Random instances too close in time, since the seed is based on the clock.

In the following code snippet, the main thread calls Join() which causes it to wait for all spawned threads to finish:
static void Main()
{
Thread regularThread = new Thread(ThreadMethod);
regularThread.Start();
Thread regularThread2 = new Thread(ThreadMethod2);
regularThread2.Start();
// Wait for spawned threads to end.
regularThread.Join();
Console.WriteLine("regularThread returned.");
regularThread2.Join();
Console.WriteLine("regularThread2 returned.");
}
Note that if you also spun up a thread from the thread pool (using QueueUserWorkItem for instance), Join would not wait for that background thread. You would need to implement some other mechanism such as using an AutoResetEvent.
For an excellent introduction to threading, I recommend reading Joe Albahari's free Threading in C#

This is very simple program to demonstrate usage of Thread Join.Please follow my comments for better understanding.Write this program as it is.
using System;
using System.Threading;
namespace ThreadSample
{
class Program
{
static Thread thread1, thread2;
static int sum=0;
static void Main(string[] args)
{
start();
Console.ReadKey();
}
private static void Sample() { sum = sum + 1; }
private static void Sample2() { sum = sum + 10; }
private static void start()
{
thread1 = new Thread(new ThreadStart(Sample));
thread2 = new Thread(new ThreadStart(Sample2));
thread1.Start();
thread2.Start();
// thread1.Join();
// thread2.Join();
Console.WriteLine(sum);
Console.WriteLine();
}
}
}
1.First time run as it is (with comments) : Then result will be 0(initial value) or 1(when thread 1 finished) or 10 (Or thread finished)
2.Run with removing comment thread1.Join() : Result should be always more than 1.because thread1.Join() fired and thread 1 should be finished before get the sum.
3.Run with removing all coments :Result should be always 11

Join is used mainly when you need to wait that a thread (or a bunch of them) will terminate before proceding with your code.
For this reason is also particular useful when you need to collect result from a thread execution.
As per the Arafangion comment below, it's also important to join threads if you need to do some cleaning/housekeeping code after having created a thread.

Join will make sure that the treads above line is executed before executing lines below.

Another example, when your worker thread let's say reads from an input stream while the read method can run forever and you want to somehow avoid this - by applying timeout using another watchdog thread:
// worker thread
var worker = new Thread(() => {
Trace.WriteLine("Reading from stream");
// here is the critical area of thread, where the real stuff happens
// Sleep is just an example, simulating any real operation
Thread.Sleep(10000);
Trace.WriteLine("Reading finished");
}) { Name = "Worker" };
Trace.WriteLine("Starting worker thread...");
worker.Start();
// watchdog thread
ThreadPool.QueueUserWorkItem((o) => {
var timeOut = 5000;
if (!worker.Join(timeOut))
{
Trace.WriteLine("Killing worker thread after " + timeOut + " milliseconds!");
worker.Abort();
}
});

Adding a delay of 300ms in method "Sample" and a delay of 400ms in "Sample2" from devopsEMK's post would make it easier to understand.
By doing so you can observe that by removing the comment from "thread1.Join();" line, the main thread waits for the "thread1" to complete and only after moves on.