Develop Reference

How to prevent threads using the same variables

I have a multi-line textbox and I want to process each line with multi threads.
The textbox could have a lot of lines (1000+), but not as many threads. I want to use custom amount of threads to read all those 1000+ lines without any duplicates (as in each thread reading UNIQUE lines only, if a line has been read by other thread, not to read it again).
What I have right now:
private void button5_Click(object sender, EventArgs e)
{
for (int i = 0; i < threadCount; i++)
{
new Thread(new ThreadStart(threadJob)).Start();
}
}
private void threadJob()
{
for (int i = 0; i < txtSearchTerms.Lines.Length; i++)
{
lock (threadLock)
{
Console.WriteLine(txtSearchTerms.Lines[i]);
}
}
}
It does start the correct amount of threads, but they all read the same variable multiple times.

Separate data collection and data processing and next possible steps after calculation. You can safely collect results calculated in parallel by using ConcurrentBag<T>, which is simply thread-safe collection.
Then you don't need to worry about "locking" objects and all lines will be "processed" only once.
1. Collect data
2. Execute collected data in parallel
3. Handle calculated result
private string Process(string line)
{
// Your logic for given line
}
private void Button_Click(object sender, EventArgs e)
{
var results = new ConcurrentBag<string>();
Parallel.ForEach(txtSearchTerms.Lines,
line =>
{
var result = Process(line);
results.Add(result);
});
foreach (var result in results)
{
Console.WriteLine(result);
}
}
By default Parallel.ForEach will use as much threads as underlying scheduler provides.
You can control amount of used threads by passing instance of ParallelOptions to the Parallel.ForEach method.
var options = new ParallelOptions
{
MaxDegreeOfParallelism = Environment.ProcessorCount
};
var results = new ConcurrentBag<string>();
Parallel.ForEach(values,
options,
value =>
{
var result = Process(value);
results.Add(result);
});

Consider using Parallel.ForEach to iterate over the Lines array. It is just like a normal foreach loop (i.e. each value will be processed only once), but the work is done in parallel - with multiple Tasks (threads).
var data = txtSearchTerms.Lines;
var threadCount = 4; // or whatever you want
Parallel.ForEach(data,
new ParallelOptions() { MaxDegreeOfParallelism = threadCount },
(val) =>
{
//Your code here
Console.WriteLine(val);
});
The above code will need this line to be added at the top of your file:
using System.Threading.Tasks;
Alternatively if you want to not just execute something, but also return / project something then instead try:
var results = data.AsParallel(new ParallelLinqOptions()
{
MaxDegreeOfParallelism = threadCount
}).Select(val =>
{
// Your code here, I just return the value but you could return whatever you want
return val;
}).ToList();
which still executes the code in parallel, but also returns a List (in this case with the same values in the original TextBox). And most importantly, the List will be in the same order as your input.

There many ways to do it what you want.
Take an extra class field:
private int _counter;
Use it instead of loop index. Increment it inside the lock:
private void threadJob()
{
while (true)
{
lock (threadLock)
{
if (_counter >= txtSearchTerms.Lines.Length)
return;
Console.WriteLine(txtSearchTerms.Lines[_counter]);
_counter++;
}
}
}
It works, but it very inefficient.
Lets consider another way. Each thread will handle its part of the dataset independently from the others.
public void button5_Click(object sender, EventArgs e)
{
for (int i = 0; i < threadCount; i++)
{
new Thread(new ParameterizedThreadStart(threadJob)).Start(i);
}
}
private void threadJob(object o)
{
int threadNumber = (int)o;
int count = txtSearchTerms.Lines.Length / threadCount;
int start = threadNumber * count;
int end = threadNumber != threadCount - 1 ? start + count : txtSearchTerms.Lines.Length;
for (int i = start; i < end; i++)
{
Console.WriteLine(txtSearchTerms.Lines[i]);
}
}
This is more efficient because threads do not wait on the lock. However, the array elements are processed not in a general manner.

why for loop is not working properly ? like this code need to show name "mubeen" many time as mention is condition

Thread t;
string mystring = "";
private void button2_Click(object sender, EventArgs e)
{
t = new Thread(write);
t.Start();
while (t.IsAlive) ;
textBox1.Text = mystring;
}
void write()
{
int i;
for ( i=0; i<1000 ;i++ ) ;
mystring+= "mubeen" +i.ToString() + "\r\n";
}

The problem is that you included an erroneous semicolon ; at the end of your for loop. This means that the body of your for loop is an empty statement, rather than the string concatenation that you actually want it to be.
It's preferable for the loop variable to be declared in the loop itself, rather than being accessible outside of the loop. I suspect you did do this at first, but moved it when it wasn't accessible to your later usage. The fact that i wasn't usable by that code when i was declared in the loop should have been a clue to you that that line of code wasn't in the loop.
Next, it's a really bad idea to have a busyloop in general (while (t.IsAlive) ;), and even worse do to it on the UI thread, blocking the UI thread from doing anything while your work is being done. You should do your work asynchronously, leaving the UI thread to go and handle other UI related tasks in the meantime, and then update it when you're done. It's also preferable to avoid having a field that's being shared across threads, and that is living and accessible to code unrelated to this activity; it's best to keep state as narrowly scoped as possible:
private async void button2_Click(object sender, EventArgs e)
{
string myString = await Task.Run(() => Write());
textBox1.Text = myString;
}
private string Write()
{
StringBuilder output = new StringBuilder();
for (int i = 0; i < 1000; i++)
output.AppendLine("mubeen" + i);
return output.ToString();
}
Note that concatenating strings in a loop is also not going to scale up very well; it's better to use a StringBuilder or another similar tool designed for a large dynamic number of string concatenations.

your method of writing the for loop is wrong you need to put the code inside "{}" if there is more than one instruction you want to do and remove the semicolon you put next to the for
like this :
for ( i=0; i<1000 ;i++ )
{
mystring+= "mubeen" +i.ToString() + "\r\n";
}
and I just realized you did the same mistake with your while so you should change it like this for the same reason :
while (t.IsAlive)
{
textBox1.Text = mystring;
}

The reason that you have the output as mubeen1000 is because of your loop syntax
int i;
// since you don't specify the method to loop through,
// this is basically looping through doing nothing other than incrementing i, resulted in i = 1000, in which therefore exiting the loop
for (i=0; i<1000; i++);
// this is the next statement which is execute, which is setting mystring to mubeen1000
mystring+= "mubeen" +i.ToString() + "\r\n";
You should set the for loop as follow:
// since you expect it starts with mubeen1, start with i=1 and the condition i<=1000 (instead of i<1000)
int i;
for (i=1; i<=1000; i++)
// specify what you want to do in the body of the loop
{
mystring+= "mubeen" + i.ToString() + "\r\n";
}
Also, your while statement may seems a problem too, you should write it as follow:
while (t.IsAlive)
{
textBox1.Text = mystring;
}

Thread t;
string mystring = "";
private void button2_Click(object sender, EventArgs e)
{
t = new Thread(write);
t.Start();
while (t.IsAlive) ;
textBox1.Text = mystring;
}
void write()
{
int i;
for ( i=0; i<1000 ;i++ ) {
mystring+= "mubeen" +i.ToString() + "\r\n";
}
}

Infinite loop in task - cpu usage

I'm making service for watch on some controller data and if it changing then I write it to DB. Seems simple. previosly I realized the same with Delphi, but now I am on C# (.Net 4.5). Now service works good with 100 tasks, but eats about 7-8% of CPU time. My Delphi service eats about 0%.
How can I reduce time which service eat from CPU?
P.S.: each task has own nstance of class to connect and insert into DB and work with local copy of data.
int TagCnt = DataCtrl.TagList.Count;
stopExec = false;
if (TagCnt != 0)
{
tasks = new Task[TagCnt];
for (int i = 0; i <= TagCnt - 1; i++)
{
int TempID = i;
tasks[TempID] = Task.Run(async () => // make threads for parallel read-write tasks // async
{
Random rand = new Random();
TimeSpan delay = TimeSpan.FromMilliseconds(rand.Next(1000, 1500))
try
{
while (!stopExec)
{
cToken.ThrowIfCancellationRequested();
//do basic job here
await Task.Delay(delay, cToken);
}//while end
}
catch (...)
{
...
}
}, cToken);
}

Recently I've been facing a similar conundrum and managed to solve the erratic CPU usage by using a set of dedicated long-running tasks to carry out the asynchronous work in my app like so:
Dim NumThreads As Integer = 10
Dim CanTokSrc As New CancellationTokenSource
Dim LongRunningTasks As Task() = New Task(NumThreads) {}
Dim i As Integer
Do Until i = LongRunningTasks.Count
LongRunningTasks(i) = Task.Factory.StartNew(Sub()
Do Until CanTokSrc.IsCancellationRequested
'DO WORK HERE
Loop
End Sub, CanTokSrc.Token, TaskCreationOptions.LongRunning)
i = i + 1
Loop
This image shows the difference it made in CPU usage for the same workload (shown after 9am).
So I think bypassing the thread pool by using dedicated/ long running tasks like above could improve CPU utilization in some cases. It certainly did in mine :-)

I moved to timer instructions because it's a windows service. Every event on timer load is about 7-10% and between is 0%. I tried to apply tasks, ThreadSchedule - they seems more heavy.
private void OnReadTimer(object source, ElapsedEventArgs e) //check states on timer
{
int TagCnt = DataCtrl.TagList.Count;
po.MaxDegreeOfParallelism = DataCtrl.TagList.Count;
// string ss = "tags=" + TagCnt;
//int TempID;
Random rand = new Random();
try
{
if (TagCnt != 0)
{
ParallelLoopResult loopResult = Parallel.For(0, TagCnt - 1, po, (i, loopState) =>
{
po.CancellationToken.ThrowIfCancellationRequested();
int TempID = i;
Thread.Sleep(rand.Next(100, 200));
int ID = 0;
bool State = false;
long WT = 0;
int ParID = 0;
bool Save = false;
ReadStates(TempID, out ID, out State, out WT, out ParID, out Save);
lock (locker)
{
if (Save) WriteState(ID, State, WT, ParID);
}
});
}
}
catch (TaskCanceledException)
{
}
catch (System.NullReferenceException eNullRef)
{
AddLog("Error:" + eNullRef);
}
catch (System.ArgumentOutOfRangeException e0)
{
AddLog("Error:" + e0);
}
catch (Exception e1)
{
//AddLog("Error while processing data: " + e1);
}
}

I moved to basic threads with infinite loops inside. It gets endless threads for my needs. No heavy recreating/restarting and so on. Now it works nice like Delphi service, but more comfortable job with data and DB. I starts threads with this procedure from lambda new thread()=>:
void RWDeviceState(int i)
{
try
{
int TempID = i;
long StartTime;
long NextTime;
long Period = 3000;
int ID = 0;
bool State = false;
long WT = 0;
int ParID = 0;
bool Save = false;
while (ExecutionAllowed)
{
Save = false;
ReadStates(TempID, out ID, out State, out WT, out ParID, out Save);
lock (locker)
{
if (Save) WriteState(ID, State, WT, ParID);
}
StartTime = DateTime.Now.Ticks / TimeSpan.TicksPerMillisecond;
NextTime = StartTime + Period;
while (DateTime.Now.Ticks / TimeSpan.TicksPerMillisecond < NextTime && ExecutionAllowed)
{
Thread.Sleep(40);
}
}

There are two particular techniques that will help reduce CPU usage in long loop waits. One, is to use the threading sleep method. This is good for example in standalone applications, less in windows services.
In a service, for the second, you should be using timers. These fire at regular intervals, so in between the intervals the CPU is not solicited.

Create multiple threads and wait for them all to complete, calling again for the complete

I have a Web Service, to make the load of the database server for a local database, making 100 requests for records.
Since the process is slow, I want to create ten threads, not to use too much memory, making Web Service calls, and when one of the threads, finished, over 100 call records. How do part of the thread?
Example:
Create thread 1
Create thread 2
Create thread 3
Create thread 4
thread 1 complete change Web Service again
Edit
My code not working. Variable sendalways gets the value 10 and not 0,1,2,3,4 and etc.
Int32 page = 0;
do
{
for (int iterator=0; iterator < 10; iterator++)
{
listTask[iterator] = Task.Factory.StartNew(() =>
{
Int32 send = iterator + page * 10;
DoStatus("Page: " + send.ToString());
Processamento(parametros, filial, send);
});
}
Task.WaitAll(listTask);
page++;
}
while (true); // Test only

You're closing over the loop variable. You need to remember that lambdas close over variables not over values. Your tasks will each read the value of iterator at the time that the lambda executes iterator + page * 10. By the time that that happens the main thread has already incremented it to 10.
This is simple enough to resolve. Make a copy of the loop variable inside of your for loop so that the closure closes over that variable, which never changes.
for (int iterator=0; iterator < 10; iterator++)
{
int i = iterator;
listTask[iterator] = Task.Factory.StartNew(() =>
{
Int32 send = i + page * 10;
DoStatus("Page: " + send.ToString());
Processamento(parametros, filial, send);
});
}

If I understand your question, you want to create 10 threads, wait for all, then recreate 10 threads, etc. Each thread load 100 results.
In this answer, results are String but that can be changed.
private void Load()
{
Boolean loading = true;
List<String> listResult = new List<String>();
Int32 boucle = 0;
Task[] listTask = new Task[10];
do
{
// create 10 threads (=1000 results)
for (int iterator=0; iterator < 10; iterator++)
{
// [0-99] [100-199] [200-299] ...
Int32 start = 100 * iterator + 1000 * boucle;
Int32 end = start + 99;
listTask[iterator] = Task<List<String>>.Factory.StartNew(() =>
{
List<String> data = LoadData(start, end);
return data;
});
}
// wait for 10 threads to finish
Task.WaitAll(listTask);
// collapse results
for (int i=0; i < 10; i++)
{
listResult.AddRange((listTask[i] as Task<List<String>>).Result);
}
// check if there is 100 results in last thread
loading = (listTask[9] as Task<List<String>>).Result.Count == 100;
// ready for another iteration (next 1000 results)
boucle++;
}
while (loading);
}
private List<string> LoadData(int p1, int p2)
{
// TODO : load data from p1 to p2
throw new NotImplementedException();
}

Objects Passed to ParameterizedThreadStart being overwritten?

Background:
I'm working on a small app that will read Events from an Eventlog remotely via WMI. Basically I'm searching for when a workstation locks and unlocks.
Problem:
I create an array of threads. I loop through my dataset (computernames) and fire off multiple
ParameterizedThreadStart objects with a custom object (LockHunterArgs). The problem is that I know my dataset doesn't have duplicates in it. I added a console.writeline to the end of the thread function and it displays duplicates.
Also, before I tried using threads. If I ran the code synchronously it functioned fine. It just took a long time. So that's why I'm trying to introduce multithreading.
Example Output:
//...Snipped some unique lines above
Computer: COMP Time: 3/29/2012 8:05:11 AM Session: 3935dd76-6a10-41a9-bd96-86143c66482d
Computer: COMP Time: 3/29/2012 8:05:11 AM Session: 3935dd76-6a10-41a9-bd96-86143c66482d
//...Snipped some unique and duplicated lines below
My Hypothesis:
If I place a breakpoint in the first few lines of the get_lock_data function where it is casting and step to the next line. It is random. It will step forward once then hit the same line twice. I have even seen it go two lines down then go backwards. I assume that this is because I'm firing off threads and it is hitting the points at different times giving the illusion that it is going backwards. But it is almost like the object that is being passed in is being overwritten by later threads.
I tried creating another array of LockHunterArgs and creating and assigning them during the thread firing process but that also didn't work.
It is probably something dumb. Thanks in advance.
// lance
Code:
public class LockHunterArgs
{
public LockHunterArgs(string comp, DateTime limit, Guid session)
{
Computer = comp;
LimitTime = limit;
sessionID = session;
}
public string Computer;
public DateTime LimitTime;
public Guid sessionID;
}
public class LockHunter
{
private void get_lock_data(object args)
{
string computer = ((LockHunterArgs)args).Computer;
DateTime limitTime = ((LockHunterArgs)args).LimitTime;
Guid sessionID = ((LockHunterArgs)args).sessionID;
//....SNippet ... code connects to the box and pulls data...
Console.WriteLine("Computer: " + computer + " Time: " + limitTime.ToString() + " Session: " + sessionID.ToString());
}
public void HuntLocks()
{
//....Snippet... code connects to database and gets a list of objects (currentSessions)
Thread[] threadArray = new Thread[currentSessions.Count];
int cnt = 0;
foreach (LINQ.session sesson in currentSessions)
{
DateTime mostRecentTimestamp = (from q in db.actions
where q.session_id == sesson.uid
orderby q.timestamp descending
select q.timestamp).FirstOrDefault();
ParameterizedThreadStart start = new ParameterizedThreadStart(get_lock_data);
threadArray[cnt] = new Thread(start);
threadArray[cnt].Start(new LockHunterArgs(sesson.computername , mostRecentTimestamp, sesson.uid));
cnt++;
}
for (int i = 0; i < threadArray.Length; i++)
{
threadArray[i].Join();
}
Console.WriteLine(DateTime.Now.ToString() + " Threads have joined");
//....Snippet of saving the gathered data from the threads to the database
}
}
Solution:
I added a new class. Then looped through my LINQ-to-SQL results to create a list of that new class. Then I base my thread firing from that list instead of the LINQ-to-SQL generated one. All is well. Can anyone explain this?
public class TempSession
{
public TempSession(LINQ.session sess)
{
this.computername = sess.computername;
this.timestamp = sess.start_time;
this.uid = sess.uid;
}
public string computername;
public DateTime timestamp;
public Guid uid;
}
public void HuntLocks()
{
//select EventCode,TimeGenerated,Message from Win32_NTLogEvent WHERE logfile='Security' and (EventCode='4800' or EventCode='4801') and TimeGenerated > '20120327 08:08:08'
// 4800 = locked
// 4801 = unlocked
LINQ.Login_ActionsDataContext db = new LINQ.Login_ActionsDataContext();
List<LINQ.session> currentSessions = (from q in db.sessions
where q.end_time == null
orderby q.computername ascending
select q).ToList();
// START Solution Changes
List<TempSession> newCurrentSessions = new List<TempSession>();
foreach (LINQ.session session in currentSessions)
{
newCurrentSessions.Add(new TempSession(session));
}
Thread[] threadArray = new Thread[newCurrentSessions.Count];
// END solution changes
for (int i = 0; i < newCurrentSessions.Count; i++)
{
DateTime mostRecentTimestamp = (from q in db.actions
where q.session_id == newCurrentSessions[i].uid
orderby q.timestamp descending
select q.timestamp).FirstOrDefault();
ParameterizedThreadStart start = new ParameterizedThreadStart(get_lock_data);
threadArray[i] = new Thread(start);
threadArray[i].Start(new LockHunterArgs(newCurrentSessions[i].computername, mostRecentTimestamp, newCurrentSessions[i].uid));
}
for (int i = 0; i < threadArray.Length; i++)
{
threadArray[i].Join();
}
Console.WriteLine(DateTime.Now.ToString() + " Threads have joined");
db.actions.InsertAllOnSubmit(newActions);
Console.WriteLine(DateTime.Now.ToString() + " Found " + newActions.Count.ToString() + " locks");
db.SubmitChanges();
newActions = new List<LINQ.action>();
}

Use a temp variable to store the iterated value:
foreach (LINQ.session sesson in currentSessions)
{
var tempSession = session; // now use tempSession
....
This is a known side effect of closure of the iterated value.

I would say the problem is most likely in what you snipped out. I was unable to reproduce your issue with this faked data:
var guids = Enumerable.Range(1, 10)
.Select(i => Guid.NewGuid())
.ToArray();
var currentSessions = Enumerable.Range(1, 10)
.Select(i => new {computername = "pc" + i})
.Zip(guids,(a,g) => new {a.computername, uid = g});
var dbactions = Enumerable.Range(1, 10)
.Select(i => DateTime.Now.AddHours(-1*i))
.Zip(guids, (t,g) => new {session_id = g, timestamp = t});
Given this, can you provide a working example that isn't dependent on any of your local resources?