I need to do lots (over 100,000) of ADO writes to a MySQL database. If I do this one at a time it will take hours (not acceptable). So figured I could break them into batches of 20 writes at a time into BackgroundWorker. The problem is that while it writes all the records to the database it does not exit out of the method. This means the only way I am to know if the system was completed was by looking at the database.
The code that creates all the BackgroundWorker;
ObservableCollection<MyClass> mObj = new ObservableCollection<MyClass>();
int recordsPerWorker= 20;
int i = 0;
int loop = 1;
int nLoops = (int)Math.Ceiling((double)numItms / (double)recordsPerWorker);
int remain = numItms % recordsPerWorker;
foreach (var x in MyObject)
{
mObj.Add(x);
i++;
if (loop == nLoops)
recordsPerWorker = remain;
if (i == recordsPerWorker)
{
loop++;
BackgroundWorker worker = new BackgroundWorker();
worker.WorkerReportsProgress = true;
worker.DoWork += AddWoker_DoWork;
worker.RunWorkerCompleted += Worker_RunWorkerCompleted;
worker.RunWorkerAsync(mObj);
mObj = new ObservableCollection<MyClass>();
i = 0;
}
}
while (completeWorkers != nLoops)
{
}
Then Worker_RunWorkerCompleted being;
private void Worker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
completeWorkers++;
}
I have tried this method as well with the same results
I could be going around this all wrong, so please point me in a different direction.
Related
static void Main()
{
int timersLength = 4;
int interval = 1000;
int[] timerNum = new int[timersLength];
Timer[] timers = new Timer[timersLength];
for (int i = 0; i < timersLength; i++)
{
timerNum[i] = i;
timers[i] = new Timer(interval);
//Console.WriteLine($"Timer {timerNum[i]} is running");
timers[i].Elapsed += (o, e) =>
{
Console.WriteLine($"Timer {timerNum[i]} is running");
};
}
foreach (Timer timer in timers)
timer.Start();
Console.ReadKey();
}
whenever I'm trying to make a countdown to the Timer array it gives an
"System.IndexOutOfRangeException: 'Index was outside the bounds of the array."
on the Elapsed line:
timers[i].Elapsed += (o, e) =>
{
Console.WriteLine($"Timer {timerNum[i]} is running");
};
The code seems fine but for some reason it gives this error.
If you don't want to create a class to store the name of the Timer next to it then you can simply create a Dictionary<Timer, string> collection for mapping.
Dictionary<Timer, string> timers = new Dictionary<Timer, string>(timersLength);
for (int i = 0; i < timersLength; i++)
{
var timer = new Timer(interval);
timers[timer] = $"Timer {i}";
timer.Elapsed += (sender, _) => Console.WriteLine($"{timers[(Timer)sender]} is running");
}
foreach (KeyValuePair<Timer, string> mapping in timers)
mapping.Key.Start();
The Elapsed event handler will be called by passing the sender object
The sender's type is object so you need to cast it to Timer
That object can be used to do the look up in the timers collection
Since yo don't use the EventArgs of the Elapsed I suggest to use the discard operator there to express your intent more clearly
I work on console version of link tester.
I start the function on multiple threads but I can't cancel them by key pressing.
Have an idea how I can do that ?
try
{
Thread[] tr = new Thread[Variables.Threads];
int i = 0;
while (i < Variables.Threads && Variables.running)
{
tr[i] = new Thread(new ThreadStart(Program.Runner));
i++;
}
//Start each thread
foreach (Thread x in tr)
{
x.Start();
}
//Console.ReadKey();
Task.Factory.StartNew(() =>
{
while (Colorful.Console.ReadKey().Key != ConsoleKey.Escape);
Variables.running = false;
foreach (Thread x in tr)
{
x.Abort();
}
Program.Menu();
});
}
catch (Exception)
{
}
EDIT : When my threads are a near of end, all of my app don't move after
The console is a not a good environment to start learning multitasking, much less multithreading. My advise to learning Multitasking - and especially Multithreading - is the BackgroundWorker in a WindowsForms applciation. The event queue does the main thing of keeping your application alive, without blocking I/O. And while the BackgroundWorker is horribly dated and should be removed from production code, it is pretty good training wheels.
Also your current code is swallowing exceptions, including fatal ones. This is a cardinal sin of exception handling. Do not do that.
At the end of the day, you can only do cancelation checking and progress reporting between distinct lines of code. In this BGW example, I had the advantage that I had to write all the loops anyway - so deep cancelation checking and progress reporting was in the cards. But if you use "somebody elses code", chances are high you have to wait for one function call to return before you can report and check.
#region Primenumbers
private void btnPrimStart_Click(object sender, EventArgs e)
{
if (!bgwPrim.IsBusy)
{
//Prepare ProgressBar and Textbox
int temp = (int)nudPrim.Value;
pgbPrim.Maximum = temp;
tbPrim.Text = "";
//Start processing
bgwPrim.RunWorkerAsync(temp);
}
}
private void btnPrimCancel_Click(object sender, EventArgs e)
{
if (bgwPrim.IsBusy)
{
bgwPrim.CancelAsync();
}
}
private void bgwPrim_DoWork(object sender, DoWorkEventArgs e)
{
int highestToCheck = (int)e.Argument;
//Get a reference to the BackgroundWorker running this code
//for Progress Updates and Cancelation checking
BackgroundWorker thisWorker = (BackgroundWorker)sender;
//Create the list that stores the results and is returned by DoWork
List<int> Primes = new List<int>();
//Check all uneven numbers between 1 and whatever the user choose as upper limit
for(int PrimeCandidate=1; PrimeCandidate < highestToCheck; PrimeCandidate+=2)
{
//Report progress
thisWorker.ReportProgress(PrimeCandidate);
bool isNoPrime = false;
//Check if the Cancelation was requested during the last loop
if (thisWorker.CancellationPending)
{
//Tell the Backgroundworker you are canceling and exit the for-loop
e.Cancel = true;
break;
}
//Determin if this is a Prime Number
for (int j = 3; j < PrimeCandidate && !isNoPrime; j += 2)
{
if (PrimeCandidate % j == 0)
isNoPrime = true;
}
if (!isNoPrime)
Primes.Add(PrimeCandidate);
}
//Tell the progress bar you are finished
thisWorker.ReportProgress(highestToCheck);
//Save Return Value
e.Result = Primes.ToArray();
}
private void bgwPrim_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
pgbPrim.Value = e.ProgressPercentage;
}
private void bgwPrim_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
pgbPrim.Value = pgbPrim.Maximum;
this.Refresh();
if (!e.Cancelled && e.Error == null)
{
//Show the Result
int[] Primes = (int[])e.Result;
StringBuilder sbOutput = new StringBuilder();
foreach (int Prim in Primes)
{
sbOutput.Append(Prim.ToString() + Environment.NewLine);
}
tbPrim.Text = sbOutput.ToString();
}
else
{
tbPrim.Text = "Operation canceled by user or Exception";
}
}
#endregion
However in your case, Multithreading seems unesseary. Multitasking without Threads would propably be better. Threads only help if you got a CPU bound task. And "checking links" sounds like a Network bound task. Threads have some extra headaches that you are better off avoiding.
Hello I'm trying to update my chart(s) every second, all chart(s) should be always at the same time. For better understanding I'll include an image but first off I'm gonna explain what actually happens.
So I'm ping requests are sent, every time an result is there, it writes it down in an data point array called file. Everything fine, works as expected.
At the same time, two timers are running, one timer calls a method that prepares the data (let's say at a specific time no data is found in the array -> it should just set value 0). The prepared data is than in a buffer.
The second timer is updating the UI and reading from the tempData but this isn't quite working as expected or wished.
Timers:
myTimer.Interval = 1000;
myTimer.Tick += FileReadFunction;
aTimer.Elapsed += new System.Timers.ElapsedEventHandler(prepData);
aTimer.Interval = 1000;
Button Click which starts timers:
private void _tbStartAll_Click(object sender, EventArgs e)
{
lock (_hosts)
{
foreach (HostPinger hp in _hosts)
hp.Start();
myTimer.Start();
aTimer.Enabled = true;
}
}
Method for preparing Data in Form Class:
public void prepData(object objectInfo, EventArgs e)
{
foreach (NetPinger.source.AddGraph b in graphList)
{
b.prepareData();
}
}
Prep Data Method:
public void prepareData()
{
double unixTimestamp = (Int32)(DateTime.UtcNow.Subtract(new DateTime(1970, 1, 1))).TotalSeconds;
for (double i = unixTimestamp; unixTimestamp - graphSizing < i; i--)
{
bool exists;
try
{
exists = Array.Exists(file, element => element.XValue == i);
exists = true;
}
catch
{
exists = false;
}
try
{
if (exists == false)
{
TempBuffer = TempBuffer.Skip(1).Concat(new DataPoint[] { new DataPoint(i, 0) }).ToArray();
}
else
{
DataPoint point = Array.Find(file, element => element.XValue == i);
TempBuffer = TempBuffer.Skip(1).Concat(new DataPoint[] { (point) }).ToArray();
}
}
catch (Exception ex)
{
//just for debugging...
}
}
}
File Read Method in Form Class:
private void FileReadFunction(object objectInfo, EventArgs e)
{
foreach (NetPinger.source.AddGraph b in graphList)
{
b.fileRead();
}
}
Method FileRead / Update Chart:
public void fileRead()
{
//double unixTimestamp = (Int32)(DateTime.UtcNow.Subtract(new DateTime(1970, 1, 1))).TotalSeconds;
chart_holder.Series[0].Points.Clear();
foreach (DataPoint a in TempBuffer)
{
chart_holder.Series[0].Points.Add(a);
}
}
Image Example of what I mean with time synchronization:
I'm kinda out of ideas why it's not working out, is it because a thread is faster than another? Or what is the reason and how could I fix it? I'm very thankful for your help.
Greetings C.User
I solved the problem by changing the code a bit. To keep it synchronized I prepare the data first, before displaying it at all. After the data is prepared than all the data is getting displayed. Also I only use one timer now instead of two.
TLDR;
Non-trivial memory leak, can be seen easily in Resharper. See minimal example below.
I'm seeing a memory leak in the following program but failing to see why.
The program sends pings to a number of hosts asynchronously and determines if at least one is ok. To do that, a method (SendPing()) that runs these async operations is repeatedly called which runs them in a background thread (it doesn't have to, but in the actual application SendPing() will be called by the main UI thread which shouldn't be blocked).
The task seems pretty trivial but I think the leak occurs due to the way I create lambdas inside the SendPing() method. The program can be changed to not use lambdas but I'm more interested in understanding what causes the leak here.
public class Program
{
static string[] hosts = { "www.google.com", "www.facebook.com" };
static void SendPing()
{
int numSucceeded = 0;
ManualResetEvent alldone = new ManualResetEvent(false);
ManualResetEvent[] handles = new ManualResetEvent[hosts.Length];
for (int i = 0; i < hosts.Length; i++)
handles[i] = new ManualResetEvent(false);
BackgroundWorker worker = new BackgroundWorker();
worker.DoWork += (sender, args) =>
{
numSucceeded = 0;
Action<int, bool> onComplete = (hostIdx, succeeded) =>
{
if (succeeded) Interlocked.Increment(ref numSucceeded);
handles[hostIdx].Set();
};
for (int i = 0; i < hosts.Length; i++)
SendPing(i, onComplete);
ManualResetEvent.WaitAll(handles);
};
worker.RunWorkerCompleted += (sender, args) =>
{
Console.WriteLine("Succeeded " + numSucceeded);
BackgroundWorker bgw = sender as BackgroundWorker;
alldone.Set();
};
worker.RunWorkerAsync();
alldone.WaitOne();
worker.Dispose();
}
static void SendPing(int hostIdx, Action<int, bool> onComplete)
{
Ping pingSender = new Ping();
pingSender.PingCompleted += (sender, args) =>
{
bool succeeded = args.Error == null && !args.Cancelled && args.Reply != null && args.Reply.Status == IPStatus.Success;
onComplete(hostIdx, succeeded);
Ping p = sender as Ping;
p.Dispose();
};
string data = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
byte[] buffer = Encoding.ASCII.GetBytes(data);
PingOptions options = new PingOptions(64, true);
pingSender.SendAsync(hosts[hostIdx], 2000, buffer, options, hostIdx);
}
private static void Main(string[] args)
{
for (int i = 0; i < 1000; i++)
{
Console.WriteLine("Send ping " + i);
SendPing();
}
}
}
Resharper shows the leaks are due to uncollected closure objects (c__DisplayClass...).
From what I understand, there shouldn't be a leak because there are no circular references (as far as I see) and therefore GC should take of the leaks. I also call Dispose to release the thread (bgw) + sockets (Ping object) promptly. (Even if I didn't GC will clean them up eventually, won't it?)
Suggested changes from comments
Remove event handles before Disposing
Dispose ManualResetEvent
But the leak is still there!
Changed program:
public class Program
{
static string[] hosts = { "www.google.com", "www.facebook.com" };
static void SendPing()
{
int numSucceeded = 0;
ManualResetEvent alldone = new ManualResetEvent(false);
BackgroundWorker worker = new BackgroundWorker();
DoWorkEventHandler doWork = (sender, args) =>
{
ManualResetEvent[] handles = new ManualResetEvent[hosts.Length];
for (int i = 0; i < hosts.Length; i++)
handles[i] = new ManualResetEvent(false);
numSucceeded = 0;
Action<int, bool> onComplete = (hostIdx, succeeded) =>
{
if (succeeded) Interlocked.Increment(ref numSucceeded);
handles[hostIdx].Set();
};
for (int i = 0; i < hosts.Length; i++)
SendPing(i, onComplete);
ManualResetEvent.WaitAll(handles);
foreach (var handle in handles)
handle.Close();
};
RunWorkerCompletedEventHandler completed = (sender, args) =>
{
Console.WriteLine("Succeeded " + numSucceeded);
BackgroundWorker bgw = sender as BackgroundWorker;
alldone.Set();
};
worker.DoWork += doWork;
worker.RunWorkerCompleted += completed;
worker.RunWorkerAsync();
alldone.WaitOne();
worker.DoWork -= doWork;
worker.RunWorkerCompleted -= completed;
worker.Dispose();
}
static void SendPing(int hostIdx, Action<int, bool> onComplete)
{
Ping pingSender = new Ping();
PingCompletedEventHandler completed = null;
completed = (sender, args) =>
{
bool succeeded = args.Error == null && !args.Cancelled && args.Reply != null && args.Reply.Status == IPStatus.Success;
onComplete(hostIdx, succeeded);
Ping p = sender as Ping;
p.PingCompleted -= completed;
p.Dispose();
};
pingSender.PingCompleted += completed;
string data = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
byte[] buffer = Encoding.ASCII.GetBytes(data);
PingOptions options = new PingOptions(64, true);
pingSender.SendAsync(hosts[hostIdx], 2000, buffer, options, hostIdx);
}
private static void Main(string[] args)
{
for (int i = 0; i < 1000; i++)
{
Console.WriteLine("Send ping " + i);
SendPing();
}
}
}
There is no memory leak. dotMemory that you use analyzes the snapshots and indeed, in the context of one snapshot the auto-generated class created by the compiler for the completed event handler will still be in memory. Rewrite your main application like this:
private static void Main(string[] args)
{
for (int i = 0; i < 200; i++)
{
Console.WriteLine("Send ping " + i);
SendPing();
}
Console.WriteLine("All done");
Console.ReadLine();
}
Run the profiler, allow the application to reach the point where it outputs "All done", wait a few seconds and take a new snapshot. You will see there is no longer any memory leak.
It is worth mentioning that the class generated by the compiler for the PingCompleted event handler (that is c_DisplayClass6) will linger in memory after the method static void SendPing(int hostIdx, Action<int, bool> onComplete) exits. What happens there is that when pingSender.PingCompleted += (sender, args) =>... is executed the pingSender instance will take a reference to c_DisplayClass6. During the call to pingSender.SendAsync, the framework will retain a reference to pingSender in order to deal with running the async method and its completion. The async method you initiate by calling pingSender.SendAsync still runs when method SendPing exits. Because of that pingSender will survive a little while longer, hence c_DisplayClass6 will survive a little while longer too. However, after the pingSender.SendAsync operation completes, the framework will release its references to pingSender. At this point both pingSender and c_DisplayClass6 become garbage collectable and eventually the garbage collector will collect them. You can see this if you take a last snapshot like I was mentioning above. In that snapshot dotMemory will no longer detect a leak.
ManualResetEvent implements Dispose(). You are instantiating a number of ManualResetEvents and never calling dispose.
When an object implements dispose you need to call it. If you do not call it, there'll quite likely be memory leaks. You should use using statements, and try finally to dispose objects Simarly you should also have a using statement around Ping.
EDIT: This may be useful....
When should a ManualResetEvent be disposed?
EDIT: As stated here...
https://msdn.microsoft.com/en-us/library/498928w2(v=vs.110).aspx
When you create objects that include unmanaged resources, you must
explicitly release those resources when you finish using them in your
app.
EDIT: As stated here...
https://msdn.microsoft.com/en-us/library/system.threading.manualresetevent(v=vs.100).aspx
Dispose() Releases all resources used by the current instance of the
WaitHandle class. (Inherited from WaitHandle.)
The ManualResetEvent has unmanaged resources associated with it, which is fairly typical of most of the classes in the .NET Framework libraries which implements IDisposable.
EDIT: Try using this...
public class Program
{
static string[] hosts = { "www.google.com", "www.facebook.com" };
static void SendPing()
{
int numSucceeded = 0;
using (ManualResetEvent alldone = new ManualResetEvent(false))
{
BackgroundWorker worker = null;
ManualResetEvent[] handles = null;
try
{
worker = new BackgroundWorker();
DoWorkEventHandler doWork = (sender, args) =>
{
handles = new ManualResetEvent[hosts.Length];
for (int i = 0; i < hosts.Length; i++)
handles[i] = new ManualResetEvent(false);
numSucceeded = 0;
Action<int, bool> onComplete = (hostIdx, succeeded) =>
{
if (succeeded) Interlocked.Increment(ref numSucceeded);
handles[hostIdx].Set();
};
for (int i = 0; i < hosts.Length; i++)
SendPing(i, onComplete);
ManualResetEvent.WaitAll(handles);
foreach (var handle in handles)
handle.Close();
};
RunWorkerCompletedEventHandler completed = (sender, args) =>
{
Console.WriteLine("Succeeded " + numSucceeded);
BackgroundWorker bgw = sender as BackgroundWorker;
alldone.Set();
};
worker.DoWork += doWork;
worker.RunWorkerCompleted += completed;
worker.RunWorkerAsync();
alldone.WaitOne();
worker.DoWork -= doWork;
worker.RunWorkerCompleted -= completed;
}
finally
{
if (handles != null)
{
foreach (var handle in handles)
handle.Dispose();
}
if (worker != null)
worker.Dispose();
}
}
}
static void SendPing(int hostIdx, Action<int, bool> onComplete)
{
using (Ping pingSender = new Ping())
{
PingCompletedEventHandler completed = null;
completed = (sender, args) =>
{
bool succeeded = args.Error == null && !args.Cancelled && args.Reply != null && args.Reply.Status == IPStatus.Success;
onComplete(hostIdx, succeeded);
Ping p = sender as Ping;
p.PingCompleted -= completed;
p.Dispose();
};
pingSender.PingCompleted += completed;
string data = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
byte[] buffer = Encoding.ASCII.GetBytes(data);
PingOptions options = new PingOptions(64, true);
pingSender.SendAsync(hosts[hostIdx], 2000, buffer, options, hostIdx);
}
}
private static void Main(string[] args)
{
for (int i = 0; i < 1000; i++)
{
Console.WriteLine("Send ping " + i);
SendPing();
}
}
}
well i am new to C#, and implementing a code, in which i have two buttons, with one acting as starting of data acquisition and storing it in a csv file and other button to stop it.
well codes for all these are as follows:
//button for start DAQ
private void stdaq_Click(object sender, EventArgs e)
{
stopped = false;
process();
}
//button for stoping DAQ
private void spdaq_Click(object sender, EventArgs e)
{
stopped = true;
}
// process function
private process()
{
int iAvail = 0;
int iRead = 0;
string filename = #textBox3.Text;// taking csv file name from user
// jit:
//a function calculating the total number of values and storing it in iAvail
int[] iRawData = new Int32[iAvail];
double[] dScaledData = new Double[iAvail];
//a function transferring the data from buffer and storing it in dscaledData array
List<double> data = new List<double>();
for (int i = 0; i < iAvail; i++)
{
data.Add(dScaledData[i]);
}
Task myFirstTask = Task.Factory.StartNew(()
=>
{
while (stopped == false)
{
Write(data.ToArray(), filename);
// goto jit;
}
});
}
// csv creater and data writer
public static void Write(double[] data, string outputPath)
{
StringBuilder sb = new StringBuilder();
for (int i = 0; i < data.GetLength(0); i++)
{
if (stopped) break;
sb.AppendLine(string.Join(",", data[i]));
}
if (File.Exists(outputPath))
{
File.AppendAllText(outputPath, sb.ToString());
}
else
{
File.WriteAllText(outputPath, sb.ToString());
}
}
this is what i am implementing, and the problem with this code is that when the data is first transferred and written to the file, then again the same data is written again and again irrespective of new data and i tried implementing that Goto statement(can be seen in comments) but it is giving error - " Control cannot leave the body of an anonymous method or lambda expression ", and if i don't use the While loop the data is not written at all.
So i want to call my process function and to transfer data to csv starting on press of a start button, take fresh data everytime and write it to csv or can say call the process method again from it's start point and to stop it on click of the stop button, but i am unable to do it irrespective of various tries with different loops and some threading functions also.
please help with this.
Assuming you only need to Write once, you should remove this or change it from while to if:
while (stopped == false)
The loop will cause Write to be called infinitely until stopped becomes true.
Also, you might want to change Write to return rather than break if stopped is true, so that you don't write anything if you are supposed to be stopping:
if (stopped) break;
to
if (stopped) return;
If you want to generate data again and really do want to loop forever, just move that code into the loop:
Task myFirstTask = Task.Factory.StartNew(()
=>
{
while (stopped == false)
{
List<double> data = new List<double>();
// TODO: Generate data here - move all relevant code here
Write(data.ToArray(), filename);
}
});
I think this is a job for the BackgroundWorker.
This code will start you up:
public partial class Form1 : Form
{
int loopCounter = 0; // variable just used for illustration
private static BackgroundWorker bw = new BackgroundWorker(); // The worker object
// This function does your task
public void doSomeStuff(object sender, DoWorkEventArgs e)
{
for (int i = 0; i < 1000; i++)
{
loopCounter = i; // Pass the loop count to this variable just to report later how far the loop was when the worker got cancelled.
Thread.Sleep(100); // Slow down the loop
// During your loop check if the user wants to cancel
if (bw.CancellationPending)
{
e.Cancel = true;
return; // quit loop
}
}
}
// This button starts your task when pressed
private void button1_Click(object sender, EventArgs e)
{
bw.WorkerSupportsCancellation = true; // Set the worker to support cancellation
bw.DoWork += doSomeStuff; // initialize the event
if (!bw.IsBusy) // Only proceed to start the worker if it is not already running.
{
bw.RunWorkerAsync(); // Start the worker
}
}
// This button stops your task when pressed
private void button2_Click(object sender, EventArgs e)
{
// Request cancellation
bw.CancelAsync();
textBox1.Text = "The bw was cancelled when 'loopCounter' was at: " + loopCounter.ToString();
}
}