I have one problem. In my project i need for more then 100 threads, that's why i prefer to use ThreadPool. here is a part of code, but in this case i have got a lot of memory usage and my form is very laggy, cuz of A lot of BeginInvoke calls(i suppose).
Is there any solutions for this problem?
public void launch()
{
while (data.Count > 0)
{
string[] part;
if (data.Count> 1000)
{
part = data.Take(1000).ToArray();
data = data.Skip(1000).ToList();
}
else
{
part = data.Take(data.Count).ToArray(); data = data.Skip(1000).ToList();
}
foreach (var input in part)
{
try
{
char splitter = ':';
if (input.Contains(';')) splitter = ';';
string login = input.Split(splitter)[0];
string pass = input.Split(splitter)[1];
EncryptCore ec = new EncryptCore(new byte[15]);
PacketSend ps = new PacketSend(ec, "");
ps._login = login;
ps._password = pass;
ps.Brutted+=ps_Parsed;
ps.Failed+=ps_Failed;
ThreadPool.QueueUserWorkItem(ps.Parse);
}
catch { Interlocked.Increment(ref curr); }
}
Thread.Sleep(1000);
}
data.Clear();
}
private void ps_Brutted(User Account)
{
toGet.Add(Account);
Interlocked.Increment(ref good);
goodLabl.BeginInvoke(new Action(delegate()
{
goodLabl.Text = "Good: " + good;
}));
Update();
}
private void Update()
{
try
{
Interlocked.Increment(ref curr);
progLabel.BeginInvoke(new Action(delegate()
{
progLabel.Text = (double.Parse(curr.ToString()) / double.Parse(max.ToString())).ToString("#%");
}));
progressBar.BeginInvoke(new Action(delegate()
{
progressBar.Text = (double.Parse(curr.ToString()) / double.Parse(max.ToString()) * 100).ToString("#");
}));
checkedLabl.BeginInvoke(new Action(delegate()
{
checkedLabl.Text = "Checked: " + curr + " / " + max;
}));
}
catch { }
}
So you have thousands of tasks, and each time one of them completes you update the UI by scheduling 4 updates on the UI thread. What you might consider doing is having the UI update on a timer-based schedule (try 100ms or 200ms) based on the integers that are updating.
Aside from that, you've got some strange numeric operations going on. You can just cast an int to a double, you don't have to round-trip through a String.
Related
I am filling Richtextbox with adding lines from another thread that extract links from web. when the web urls links increase and go more than 9000 it hangs the UI and take long time don't know why ! , using button click event to fire this method ( thread )
Using AppendText(Environment.NewLine) method to fill the richtextbox
Here is the snippet code of my work :
if (URLLMemoRichTxt.Lines.Length == 0)
{
XtraMessageBox.Show("You have to get some links first");
return;
}
var thd = new Thread(() =>
{
if (this.InvokeRequired)
{
if (URLLMemoRichTxt.InvokeRequired)
{
URLLMemoRichTxt.Invoke((MethodInvoker)delegate ()
{
foreach (string line in URLLMemoRichTxt.Lines)
{
if (!GetEmailsListArraylist.Contains(line) && line.Trim() != string.Empty)
{
if (LinksToGetEmailsRichTxt.InvokeRequired)
{
LinksToGetEmailsRichTxt.Invoke((MethodInvoker)delegate ()
{
GetEmailsListArraylist.Add(line);
// LinksToGetEmailsRichTxt.Text += line + "\n";
LinksToGetEmailsRichTxt.AppendText(Environment.NewLine + line);
LinksToGetEmailsLabel.Text = LinksToGetEmailsRichTxt.Lines.Length.ToString();
});
}
else
{
GetEmailsListArraylist.Add(line);
// LinksToGetEmailsRichTxt.Text += line + "\n";
LinksToGetEmailsRichTxt.AppendText(Environment.NewLine + line);
LinksToGetEmailsLabel.Text = LinksToGetEmailsRichTxt.Lines.Length.ToString();
}
}
}
});
}
else
{
foreach (string line in URLLMemoRichTxt.Lines)
{
if (!GetEmailsListArraylist.Contains(line) && line.Trim() != string.Empty)
{
GetEmailsListArraylist.Add(line);
// LinksToGetEmailsRichTxt.Text += line + "\n";
LinksToGetEmailsRichTxt.AppendText(Environment.NewLine + line);
LinksToGetEmailsLabel.Text = LinksToGetEmailsRichTxt.Lines.Length.ToString();
}
}
if (MainTabcontrol.InvokeRequired)
{
MainTabcontrol.Invoke((MethodInvoker)delegate ()
{
MainTabcontrol.SelectedTabPageIndex = 1;
});
}
else
{
MainTabcontrol.SelectedTabPageIndex = 1;
}
}
}
else
{
}
if (MainTabcontrol.InvokeRequired)
{
MainTabcontrol.Invoke((MethodInvoker)delegate ()
{
MainTabcontrol.SelectedTabPageIndex = 1;
});
}
else
{
MainTabcontrol.SelectedTabPageIndex = 1;
}
});
thd.SetApartmentState(ApartmentState.MTA);
thd.Start();
using button click event to fire this method ( thread )
Here is your code, boiled down:
private void button1_Click(object sender, EventArgs e)
{
new Thread(() =>
{
if (URLLMemoRichTxt.InvokeRequired)
{
URLLMemoRichTxt.Invoke((MethodInvoker)delegate ()
{
// .. do some "work" in here ...
Thread.Sleep(9000);
});
}
}).Start();
}
Let's analyze what's happening here:
User clicks the button.
Running in the UI thread, the Click event creates a New Thread and starts it.
Inside that new thread, immediately ask if we are running in a different thread than the one that owns the RichTextBox with InvokeRequired (the answer is, of course, yes!).
Tell the application to run some code on the thread that owns the RichTextBox with Invoke().
Running in the UI thread again, do the actual "work".
End Result?
All your "work" is run in the main UI thread as if no threading ever occurred.
Why create a thread and then tell it to run its work back on the UI thread?
How should it be done correctly?
Here's a slightly more robust version of the boiled down code, with only the part that updates the UI within the Invoke() block:
private void button1_Click(object sender, EventArgs e)
{
new Thread(() =>
{
// do some "work" in our new thread
for(int i = 1; i <= 10; i++)
{
Thread.Sleep(1000); // the "work"
// when you need to UPDATE THE UI, now is the time for Invoke()
// note that only the part that is updating the UI is within this block
// the rest of the code/loop is still running in the new thread
URLLMemoRichTxt.Invoke((MethodInvoker)delegate ()
{
URLLMemoRichTxt.AppendText("Line " + i.ToString() + "\r\n");
});
// ... possibly more work in the new thread ...
}
}).Start();
}
With this approach, the UI will remain responsive while the for loop runs and the work is done. Additionally, the RichTextBox will happily update with the new entries without blocking.
I am trying to splice the work on multiple threads using ThreadPooling. I wanna use every available thread to assign (and calculate in a different part of the program down the line) the output of an array member to another array members.
It does work, but it is much slower than just adding them on a single thread. Is my usage wrong or is this operation too simple for multithreading?
arrayI and arrayX variables are in classScope, i couldnt pass them as QueueUserWorkItem argument without converting them in setNeuronInput.
if (layerType != 0)
{
for (arrayI = 0; arrayI < layerSize -1 ; arrayI++)
{
for (arrayX = 0; arrayX < network.Layers[layerIndex - 1].layerSize - 1; arrayX++)
{
ThreadPool.QueueUserWorkItem(new WaitCallback(setNeuronInput), null);
//Neurons[i].input[x] = _network.Layers[layerIndex - 1].Neurons[x].output;
}
}
}
//ThreadPool.GetAvailableThreads(out availableThreads, out placeHolder);
//while (availableThreads != maxThreads)
//{
// ThreadPool.GetAvailableThreads(out availableThreads, out placeHolder);
//}
//return;
}
public void setNeuronInput(object o)
{
try
{
//Console.WriteLine("Thread is working");
Neurons[arrayI].input[arrayX] = network.Layers[layerIndex - 1].Neurons[arrayX].output;
}
catch(Exception e)
{
Console.WriteLine(e);
Console.WriteLine("ArrayI is : " + arrayI);
Console.WriteLine("ArrayX is : " + arrayX);
Console.ReadLine();
}
}
Below is a function I have running in a while(true) loop in a thread running a Winforms GUI.
I have a button set to put text data into the inBuffer object. this always works, however when I place into the buffer object from a different thread, the data is detected, pulled, and printed out in the Console.out.WriteLine statement, however it never shows up in the Display (textBox) object
public void put()
{
string strDisplayMe = ModemKind.MainClass.inBuffer.MkRequest();
if (strDisplayMe != "")
{
Console.Out.WriteLine("FOUND SOMETHING IN BUFFER: " + strDisplayMe);
char[] DisplayMeArr = strDisplayMe.ToCharArray ();
for (int i = 0; i <= DisplayMeArr.Length -1; ++i)
{
this.display.Text += DisplayMeArr [i];
Thread.Sleep (100);
}
this.display.Text += '\n';
}
}
EDIT: this is a separate class from what is feeding it data through the static buffer objects
Only the main window thread can access/change controls... if you want update the UI from the thread that runs the loop, you need to sync the call with the main thread using the Control.Invoke method.
For instance...
public void put()
{
string strDisplayMe = ModemKind.MainClass.inBuffer.MkRequest();
if (strDisplayMe != string.Empty)
{
char[] DisplayMeArr = strDisplayMe.ToCharArray();
for (int i = 0; i <= DisplayMeArr.Length -1; ++i)
{
this.UpdateUI(() => { this.display.Text += DisplayMeArr[i]; });
Thread.Sleep(100);
}
this.UpdateUI(() => { this.display.Text += '\n'; });
}
}
private void UpdateUI(Action handler)
{
this.Invoke(handler);
}
You can use MethodInvoker to access TextBox from other thread then GUI thread. Make a method to access the TextBox using MethodInvoker. you are assigning text multiple times consider assigning it once for performance, and use StringBuilder for string concatenation.
public void put()
{
string strDisplayMe = ModemKind.MainClass.inBuffer.MkRequest();
if (strDisplayMe != "")
{
Console.Out.WriteLine("FOUND SOMETHING IN BUFFER: " + strDisplayMe);
char[] DisplayMeArr = strDisplayMe.ToCharArray ();
for (int i = 0; i <= DisplayMeArr.Length -1; ++i)
{
AssignToTextBox(this.display, this.display.Text + DisplayMeArr [i]);
Thread.Sleep (100);
}
AssignToTextBox(this.display, this.display.Text + '\n');
}
}
void AssignToTextBox(TextBox txtBox, string value)
{
if(txtBox.InvokeRequired)
{
txtBox.Invoke(new MethodInvoker(delegate { txtBox.text = value; }));
}
}
Edit You can use BeginInvoke instead of Invoke to call it asynchronously. Read more about the Invoke and BeginInvoke in this question.
void AssignToTextBox(TextBox txtBox, string value)
{
txtBox.BeginInvoke(new Action(()=>txtBox.Text = value ));
}
I'm writing a program, that should replace or remove some entries from a logfile.txt.
The code is working fine ( at least for small LogFiles). If i use a big file (like 27 MB) its getting very slow and the UI freeze. I cant click anything.
On Button click i execute this method:
private string delete_Lines(string[] lines, string searchString)
{
for (int i = 0; i < lines.Length; i++)
{
if (lines[i].Contains(searchString))
{
rtbLog.Text += "Deleting(row " + (i + 1) + "):\n" + lines[i] + "\n";
progressBar1.Value += 1;
if (cbDB == true)
{
while (is_next_line_block(lines, i) == true)
{
i++;
rtbLog.Text += lines[i] + "\n";
progressBar1.Value += 1;
}
}
}
else
{
res += lines[i]+"\n";
progressBar1.Value += 1;
}
}
tssLbl.Text = "Done!";
rtbLog.Text += "...Deleting finished\n";
return res;
}
Lines is the array of the logfile i am trying to clean up. every entry is a single row . tssLbl is a notification label and rtbLog is a richTextBox, where i'am tracking which row i am deleting.
is_next_line_block is just another method, which is checking of the next lines are part of the block i want to delete. The params of this method are the whole lines array and the line position.
private bool is_next_line_block(string[] lines, int curIndex)
{
if (curIndex < lines.Length-1)
{
if (lines[curIndex + 1].StartsWith(" "))
{
return true;
}
else
{
return false;
}
}
else
{
return false;
}
}
Have anybody any idea, what is causing that freezes and is slowing down the program? I know, that i could speed my code up by parallelizing it, but i cant imagine, that it takes so long to check up a 27 MB txt file without parallelism.
You have several issues here:
You are reading the whole file in buffer (array of string), I am guessing you are calling File.ReadAllLines(). Reading big files in buffer will slow you down, as well as in extreme case run you out of memory.
You are using += operation for your rich textbox Text property. That is time consuming operation as UI has to render the whole rich text box every time you update the text property that way. Better option is to use string builder to load these text, and update rich text box periodically.
To fix this you need to read the file as stream. Progress can be monitored based on bytes read instead of line position. You can run the read operation async and monitor progression on a timer, as shown in example below.
private void RunFileOperation(string inputFile, string search)
{
Timer t = new Timer();
int progress = 0;
StringBuilder sb = new StringBuilder();
// Filesize serves as max value to check progress
progressBar1.Maximum = (int)(new FileInfo(inputFile).Length);
t.Tick += (s, e) =>
{
rtbLog.Text = sb.ToString();
progressBar1.Value = progress;
if (progress == progressBar1.Maximum)
{
t.Enabled = false;
tssLbl.Text = "done";
}
};
//update every 0.5 second
t.Interval = 500;
t.Enabled = true;
// Start async file read operation
System.Threading.Tasks.Task.Factory.StartNew(() => delete_Lines(inputFile, search, ref progress, ref sb));
}
private void delete_Lines(string fileName, string searchString, ref int progress, ref StringBuilder sb)
{
using (var file = File.OpenText(fileName))
{
int i = 0;
while (!file.EndOfStream)
{
var line = file.ReadLine();
progress = (int)file.BaseStream.Position;
if (line.Contains(searchString))
{
sb.AppendFormat("Deleting(row {0}):\n{1}", (i + 1), line);
// Change this algorithm for nextline check
// Do this when it is next line, i.e. in this line.
// "If" check above can check if (line.startswith(" "))...
// instead of having to do it nextline next here.
/*if (cbDB == true)
{
while (is_next_line_block(lines, i) == true)
{
i++;
rtbLog.Text += lines[i] + "\n";
progressBar1.Value += 1;
}
}*/
}
}
}
sb.AppendLine("...Deleting finished\n");
}
As a follow up to your question on Task.Factory.Start() usage, it's done this way (generally):
// you might need to wrap this in a Dispatcher.BeginInvoke (see below)
// if you are not calling from the main UI thread
CallSomeMethodToSetVisualCuesIfYouHaveOne();
Task.Factory.StartNew(() =>
{
// code in this block will run in a background thread...
}
.ContinueWith(task =>
{
// if you called the task from the UI thread, you're probably
// ok if you decide not to wrap the optional method call below
// in a dispatcher begininvoke...
Application.Current.Dispatcher.BeginInvoke(new Action(()=>
{
CallSomeMethodToUnsetYourVisualCuesIfYouHaveAnyLOL();
}));
}
Hope this helps!
Thanks to everybody for the help, especially loopedcode, That's the working version (Took loopedcode's code and made some edit):
private void RunFileOperation(string inputFile, string search)
{
Timer t = new Timer();
StringBuilder sb = new StringBuilder();
{
rtbLog.Text = "Start Deleting...\n";
}
// Filesize serves as max value to check progress
progressBar1.Maximum = (int)(new FileInfo(inputFile).Length);
t.Tick += (s, e) =>
{
rtbLog.Text += sb.ToString();
progressBar1.Value = progress;
if (progress == progressBar1.Maximum)
{
t.Enabled = false;
tssLbl.Text = "done";
}
};
//update every 0.5 second
t.Interval = 500;
t.Enabled = true;
// Start async file read operation
if (rbtnDelete.Checked)
{
if (cbDelete.Checked)
{
System.Threading.Tasks.Task.Factory.StartNew(() => delete_Lines(inputFile, search, ref progress, ref sb, ref res1));
}
}
else
{
//..do something
}
private void delete_Lines(string fileName, string searchString, ref int progress, ref StringBuilder sb, ref StringBuilder res1)
{
bool checkNextLine=false;
using (var file = File.OpenText(fileName))
{
int i = 0;
while (!file.EndOfStream)
{
i++;
var line = file.ReadLine();
progress = (int)file.BaseStream.Position;
if (line.Contains(searchString))
{
sb.AppendFormat("Deleting(row {0}):\n{1}\n", (i), line);
checkNextLine = true;
}
else
{
if (cbDB && checkNextLine && line.StartsWith(" "))
{
sb.AppendFormat("{0}\n", line);
}
else
{
checkNextLine = false;
res1.AppendLine(line);
}
}
}
}
sb.AppendLine("\n...Deleting finished!);
}
I'm attempting to write a function which starts a thread for each "contact" and then queries (over the network) for results from that contact. I want my waiting function to wait at most 1.5 second for responses, and after that, simply terminate any remaining threads.
The issue I'm having is that the function is returning before all the threads are done, even though according to the logic, this should not be possible. The while loop in the main function should be waiting until all threads have fully completed, yet I get the following output:
FAIL: Storage test 1 exists 0 times in the DHT.
: Storage test 2 exists 0 times in the DHT.
Added storage test 1 to the entries.
Added storage test 2 to the entries.
(The FAIL lines comes from the main testing program seeing how many results were returned by Get().)
According to what I can see, this shouldn't be possible. Does anyone know where the race condition might be occurring (or any other assumptions I've made that are not correct)?
The function definition is as such:
public IList<Entry> Get(ID key)
{
ConcurrentBag<Entry> entries = new ConcurrentBag<Entry>();
List<Thread> threads = new List<Thread>();
foreach (Contact c in this.p_Contacts)
{
Thread t = new Thread(delegate()
{
try
{
FetchMessage fm = new FetchMessage(this, c, key);
fm.Send();
int ticks = 0;
// Wait until we receive data, or timeout.
while (!fm.Received && ticks < 1500)
{
Thread.Sleep(100);
ticks += 100;
}
if (fm.Received)
{
foreach (Entry e in fm.Values)
{
Console.WriteLine("Added " + e.Value + " to the entries.");
entries.Add(e);
}
if (entries.Count == 0)
Console.WriteLine("There were no entries to add.");
}
else
Console.WriteLine("The node did not return in time.");
}
catch (Exception e)
{
Console.WriteLine(e);
}
}
);
t.IsBackground = false;
t.Start();
}
while (true)
{
bool stopped = true;
foreach (Thread t in threads)
{
if (t.ThreadState != ThreadState.Stopped)
stopped = false;
}
if (stopped)
break;
Thread.Sleep(100);
}
return new List<Entry>(entries.ToArray());
}
Looks like you're never putting the Thread (t) in the List<Thread> (threads). Your foreach loop never executes.
The main thread just waits 100ms and continues.
#Toby has the correct answer, but if I may introduce some other things to improve the code. Essentially, you are manually managing your own ThreadPool and timeouts--and this is something that .Net provides for you out of the box. See: http://msdn.microsoft.com/en-us/library/system.threading.threadpool(v=VS.100).aspx
If you combine your ThreadPool with a .Net 4 Barrier, you can simplify the code a lot. Essentially, a Barrier will block all threads until they sync up. When you pass in the same barrier to your threads and sync up at the end, you can pause your main thread until all the worker threads are done. The refactored code would look like this:
// For the number of threads + 1 for the main thread
Barrier barrier = new Barrier(this.p_Contacts.count + 1);
ConcurrentBag<Entry> entries = new ConcurrentBag<Entry>();
foreach (Contact c in this.p_Contacts)
{
ThreadPool.RegisterWaitForSingleObject(
new EventWaitHandle(false, EventResetMode.AutoReset),
(stateInfo,timedOut) => {
try
{
FetchMessage fm = new FetchMessage(this, c, key);
fm.Send();
while(!fm.Received || !timedOut)
{
Thread.Sleep(100);
}
if(fm.Received)
{
foreach (Entry e in fm.Values)
{
entries.Add(e);
Console.WriteLine("Added " + e.Value + " to the entries.");
}
// avoid counting other thread's work
if (fm.Values.count == 0)
{
Console.WriteLine("There were no entries to add.");
}
}
else
{
Console.WriteLine("The node did not return in time.");
}
barrier.SignalAndWait();
}
catch(Exception e)
{
Console.WriteLine(e);
}
}, null, TimeSpan.FromSeconds(1.5), true);
);
}
// This limits total time waited to only 1.5 seconds
barrier.SignalAndWait(TimeSpan.FromSeconds(1.5));
return new List<Entry>(entries.ToArray());
Instead of manually managing the spin locks like you were doing, let .Net do it for you.
The Threads don't get added to your list so the while loop will break right away?
The solution to this problem was to use a ConcurrentDictionary to keep track of the contacts that had their threads finished:
public IList<Entry> Get(ID key)
{
ConcurrentBag<Entry> entries = new ConcurrentBag<Entry>();
ConcurrentDictionary<Contact, bool> done = new ConcurrentDictionary<Contact, bool>();
List<Thread> threads = new List<Thread>();
foreach (Contact c in this.p_Contacts)
{
Thread t;
ThreadStart ts = delegate()
{
try
{
FetchMessage fm = new FetchMessage(this, c, key);
fm.Send();
int ticks = 0;
// Wait until we receive data, or timeout.
while (!fm.Received && ticks < 1500)
{
Thread.Sleep(100);
ticks += 100;
}
if (fm.Received)
{
foreach (Entry e in fm.Values)
{
Console.WriteLine("Added " + e.Value + " to the entries.");
entries.Add(e);
}
if (entries.Count == 0)
Console.WriteLine("There were no entries to add.");
}
else
Console.WriteLine("The node did not return in time.");
Thread.MemoryBarrier();
done[c] = true;
}
catch (Exception e)
{
Console.WriteLine(e);
}
};
t = new Thread(ts);
done[c] = false;
t.IsBackground = true;
t.Start();
}
while (true)
{
bool stopped = true;
foreach (Contact c in this.p_Contacts)
{
if (!done[c])
stopped = false;
}
if (stopped)
break;
Thread.Sleep(100);
}
return new List<Entry>(entries.ToArray());
}