Develop Reference

Progress Bar Freezes part way

So in the snippet of below I very simply look in a specific folder and copy the images from the source to the destination.
The copy is VERY fast and it works great for the first bunch of folders (maybe 20 or so) which takes a few seconds. But then the progress bar stops moving and I get a spinning mouse cursor. I can look in the destination folder and it is still processing the folders.
When it's done I get the the "Process Complete" dialog box, the progress bar is 100% and everything ran fine.
Just want to make sure the end user doesn't think it's frozen.
private void readInvoices()
{
string InvoiceFile = txtInvoiceFile.Text;
//read in the text file and get all the invoices to copy
string[] Invoices = File.ReadAllLines(InvoiceFile);
//set the max val of the progress bar
progBar.Maximum = Invoices.Length;
try
{
//for every invoice
foreach (string invoice in Invoices)
{
//Set the source and destination directories
string sourceInvFolder = string.Format(#"{0}\{1}", txtSource.Text, invoice);
string destInvFolder = string.Format(#"{0}\{1}", txtDest.Text, invoice);
DirectoryInfo SourceDI = new DirectoryInfo(sourceInvFolder);
DirectoryInfo DestDI = new DirectoryInfo(destInvFolder);
//we know we have it in the CSV but does the directory actually exist?
//if so then let's process
if (Directory.Exists(SourceDI.FullName) == true)
{
//let's copy of the files
CopyAll(SourceDI, DestDI);
RenameFolder(sourceInvFolder);
}
//inc the progress bar
progBar.Increment(1);
}
}
catch (Exception ex)
{
MessageBox.Show("Error" + ex.Message);
}
finally
{
MessageBox.Show("Process Complete");
CleanUp();
}
}

The UI freezes because it's running in a single thread. A workaround to fix the freezing part is putting this line of code inside your loop.
Application.DoEvents();
This code checks if there are messages waiting to be processed, if there are, it processes them before proceeding to another loop. You can use a ProgressBar control to let the user see how much is processed already. If you don't want to stay with the single thread method, use a BackGroundWorker to prevent the form from appearing like it's frozen. This is multithreading, which means a separate thread is processing something while you do something else.
One thing to remember though, using the code above makes the whole looping process go slower since it has to do the checking for every loop, which means more work, in return, you get real-time progress report. The reason it looks frozen is that the loop hasn't finished yet, you'll have to let it finish first before you can do something else because it's running in a single thread.

if you want Just want to make sure the end user doesn't think it's frozen. you should use multihreading . the More suitable class for this task is BackgroundWorker
From MSDN :
The BackgroundWorker class allows you to run an operation on a separate, dedicated thread. Time-consuming operations like downloads and database transactions can cause your user interface (UI) to seem as though it has stopped responding while they are running. When you want a responsive UI and you are faced with long delays associated with such operations, the BackgroundWorker class provides a convenient solution.
try to follow the example provided by msdn and put the call readInvoices() of your method in DoWork event

Your code is running on the UI thread (at least I assume so as you have a MessageBox in your catch block).
So, it will not necessarily process UI updates.
Have a look into doing the work using the TPL.
http://msdn.microsoft.com/en-us/library/dd460717(v=vs.110).aspx

thread get 100% CPU very fast

I am implementing a very basic thread in C#:
private Thread listenThread;
public void startParser()
{
this.listenThread = new Thread(new ThreadStart(checkingData));
this.listenThread.IsBackground = true;
this.listenThread.Start();
}
private void checkingData()
{
while (true)
{
}
}
Then I immediately get 100% CPU. I want to check if sensor data is read inside the while(true) loop. Why it is like this?
Thanks in advance.

while (true) is what killing your CPU.
You can add Thread.Sleep(X) to you while to give CPU some rest before checking again.
Also, seems like you actually need a Timer.
Look at one of the Timer classes here http://msdn.microsoft.com/en-us/library/system.threading.timer.aspx.
Use Timer with as high pulling interval as you can afford, 1 sec, half a sec.
You need to tradeoff between CPU usage and the maximum delay you can afford between checks.

Let your loop sleep. It's running around and around and getting tired. At the very least, let it take a break eventually.

Because your function isn't doing anything inside the while block, it grabs the CPU, and, for all practical purposes, never lets go of it, so other threads can do their work
private void checkingData()
{
while (true)
{
// executes, immediately
}
}
If you change it to the following, you should see more reasonable CPU consumption:
private void checkingData()
{
while (true)
{
// read your sensor data
Thread.Sleep(1000);
}
}

you can use blocking queue. take a item from blocking queue will block the thread until there is a item put into the queue. that doesn't cost any cpu.
with .net4, you can use BlockingCollection http://msdn.microsoft.com/en-us/library/dd267312.aspx
under version 4, there is not blocking queue int .net framework.
you can find many implements of blocking queue if you google it.
here is a implementation
http://www.codeproject.com/KB/recipes/boundedblockingqueue.aspx
by the way. where does the data you wait come from?
EDIT
if you want to check file. you can use FileSystemWatcher to check it with thread block.
if your data comes from external API and the api doesn't block the thread, there is no way to block the thread except use Thread.Sleep

If you're polling for a condition, definitely do as others suggested and put in a sleep. I'd also add that if you need maximum performance, you can use a statistical trick to avoid sleeping when sensor data has been read. When you detect sensor data is idle, say, 10 times in a row, then start to sleep on each iteration again.

How would you change my Heartbeat process written in C#?

I'm looking at implementing a "Heartbeat" process to do a lot of repeated cleanup tasks throughout the day.
This seemed like a good chance to use the Command pattern, so I have an interface that looks like:
public interface ICommand
{
void Execute();
bool IsReady();
}
I've then created several tasks that I want to be run. Here is a basic example:
public class ProcessFilesCommand : ICommand
{
private int secondsDelay;
private DateTime? lastRunTime;
public ProcessFilesCommand(int secondsDelay)
{
this.secondsDelay = secondsDelay;
}
public void Execute()
{
Console.WriteLine("Processing Pending Files...");
Thread.Sleep(5000); // Simulate long running task
lastRunTime = DateTime.Now;
}
public bool IsReady()
{
if (lastRunTime == null) return true;
TimeSpan timeSinceLastRun = DateTime.Now.Subtract(lastRunTime.Value);
return (timeSinceLastRun.TotalSeconds > secondsDelay);
}
}
Finally, my console application runs in this loop looking for waiting tasks to add to the ThreadPool:
class Program
{
static void Main(string[] args)
{
bool running = true;
Queue<ICommand> taskList = new Queue<ICommand>();
taskList.Enqueue(new ProcessFilesCommand(60)); // 1 minute interval
taskList.Enqueue(new DeleteOrphanedFilesCommand(300)); // 5 minute interval
while (running)
{
ICommand currentTask = taskList.Dequeue();
if (currentTask.IsReady())
{
ThreadPool.QueueUserWorkItem(t => currentTask.Execute());
}
taskList.Enqueue(currentTask);
Thread.Sleep(100);
}
}
}
I don't have much experience with multi-threading beyond some work I did in Operating Systems class. However, as far as I can tell none of my threads are accessing any shared state so they should be fine.
Does this seem like an "OK" design for what I want to do? Is there anything you would change?

This is a great start. We've done a bunch of things like this recently so I can offer a few suggestions.
Don't use thread pool for long running tasks. The thread pool is designed to run lots of tiny little tasks. If you're doing long running tasks, use a separate thread. If you starve the thread pool (use up all the tasks), everything that gets queued up just waits for a threadpool thread to become available, significantly impacting the effective performance of the threadpool.
Have the Main() routine keep track of when things ran and how long till each runs next. Instead of each command saying "yes I'm ready" or "no I'm not" which will be the same for each command, just have LastRun and Interval fields which Main() can then use to determine when each command needs to run.
Don't use a Queue. While it may seem like a Queue type operation, since each command has it's own interval, it's really not a normal Queue. Instead put all the commands in a List and then sort the list by shortest time to next run. Sleep the thread until the first command is needed to run. Run that command. Resort the list by next command to run. Sleep. Repeat.
Don't use multiple threads. If each command's interval is a minute or few minutes, you probably don't need to use threads at all. You can simplify by doing everything on the same thread.
Error handling. This kind of thing needs extensive error handling to make sure a problem in one command doesn't make the whole loop fail, and so you can debug a problem when it occurs. You also may want to decide if a command should get immediately retried on error or wait until it's next scheduled run, or even delay it more than normal. You may also want to not log an error in a command if the error happens every time (an error in a command that runs often can easily create huge log files).

Instead of writing everything from scratch, you could choose to build your application using a framework that handles all of the scheduling and threading for you. The open-source library NCron is designed for exactly this purpose, and it is very easy to use.
Define your job like this:
class MyFirstJob : CronJob
{
public override void Execute()
{
// Put your logic here.
}
}
And create a main entry point for your application including scheduling setup like this:
class Program
{
static void Main(string[] args)
{
Bootstrap.Init(args, ServiceSetup);
}
static void ServiceSetup(SchedulingService service)
{
service.Hourly().Run<MyFirstJob>();
service.Daily().Run<MySecondJob>();
}
}
This is all the code you will need to write if you choose to go down this path. You also get the option to do more complex schedules or dependency injection if needed, and logging is included out-of-the-box.
Disclaimer: I am the lead programmer on NCron, so I might just be a tad biased! ;-)

I would make all your Command classes immutable to insure that you don't have to worry about changes to state.

Now a days 'Parallel Extensions' from microsoft should be the viable option to write concurrent code or doing any thread related tasks. It provides good abstraction on top of thread pool and system threads such that you need not to think in an imperative manner to get the task done.
In my opinion consider using it. By the way, your code is clean.
Thanks.

running variable will need to be marked as volatile if its state is going to be changed by another thread.
As to the suitability, why not just use a Timer?

Proper way to have an endless worker thread?

I have an object that requires a lot of initialization (1-2 seconds on a beefy machine). Though once it is initialized it only takes about 20 miliseconds to do a typical "job"
In order to prevent it from being re-initialized every time an app wants to use it (which could be 50 times a second or not at all for minutes in typical usage), I decided to give it a job que, and have it run on its own thread, checking to see if there is any work for it in the que. However I'm not entirely sure how to make a thread that runs indefinetly with or without work.
Here's what I have so far, any critique is welcomed
private void DoWork()
{
while (true)
{
if (JobQue.Count > 0)
{
// do work on JobQue.Dequeue()
}
else
{
System.Threading.Thread.Sleep(50);
}
}
}
After thought: I was thinking I may need to kill this thread gracefully insead of letting it run forever, so I think I will add a Job type that tells the thread to end. Any thoughts on how to end a thread like this also appreciated.

You need to lock anyway, so you can Wait and Pulse:
while(true) {
SomeType item;
lock(queue) {
while(queue.Count == 0) {
Monitor.Wait(queue); // releases lock, waits for a Pulse,
// and re-acquires the lock
}
item = queue.Dequeue(); // we have the lock, and there's data
}
// process item **outside** of the lock
}
with add like:
lock(queue) {
queue.Enqueue(item);
// if the queue was empty, the worker may be waiting - wake it up
if(queue.Count == 1) { Monitor.PulseAll(queue); }
}
You might also want to look at this question, which limits the size of the queue (blocking if it is too full).

You need a synchronization primitive, like a WaitHandle (look at the static methods) . This way you can 'signal' the worker thread that there is work. It checks the queue and keeps on working until the queue is empty, at which time it waits for the mutex to signal it again.
Make one of the job items be a quit command too, so that you can signal the worker thread when it's time to exit the thread

In most cases, I've done this quite similar to how you've set up -- but not in the same language. I had the advantage of working with a data structure (in Python) which will block the thread until an item is put into the queue, negating the need for the sleep call.
If .NET provides a class like that, I'd look into using it. A thread blocking is much better than a thread spinning on sleep calls.
The job you can pass could be as simple as a "null"; if the code receives a null, it knows it's time to break out of the while and go home.

If you don't really need to have the thread exit (and just want it to keep from keeping your application running) you can set Thread.IsBackground to true and it will end when all non background threads end. Will and Marc both have good solutions for handling the queue.

Grab the Parallel Framework. It has a BlockingCollection<T> which you can use as a job queue. How you'd use it is:
Create the BlockingCollection<T> that will hold your tasks/jobs.
Create some Threads which have a never-ending loop (while(true){ // get job off the queue)
Set the threads going
Add jobs to the collection when they come available
The threads will be blocked until an item appears in the collection. Whoever's turn it is will get it (depends on the CPU). I'm using this now and it works great.
It also has the advantage of relying on MS to write that particularly nasty bit of code where multiple threads access the same resource. And whenever you can get somebody else to write that you should go for it. Assuming, of course, they have more technical/testing resources and combined experience than you.

I've implemented a background-task queue without using any kind of while loop, or pulsing, or waiting, or, indeed, touching Thread objects at all. And it seems to work. (By which I mean it's been in production environments handling thousands of tasks a day for the last 18 months without any unexpected behavior.) It's a class with two significant properties, a Queue<Task> and a BackgroundWorker. There are three significant methods, abbreviated here:
private void BackgroundWorker_DoWork(object sender, DoWorkEventArgs e)
{
if (TaskQueue.Count > 0)
{
TaskQueue[0].Execute();
}
}
private void BackgroundWorker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
Task t = TaskQueue[0];
lock (TaskQueue)
{
TaskQueue.Remove(t);
}
if (TaskQueue.Count > 0 && !BackgroundWorker.IsBusy)
{
BackgroundWorker.RunWorkerAsync();
}
}
public void Enqueue(Task t)
{
lock (TaskQueue)
{
TaskQueue.Add(t);
}
if (!BackgroundWorker.IsBusy)
{
BackgroundWorker.RunWorkerAsync();
}
}
It's not that there's no waiting and pulsing. But that all happens inside the BackgroundWorker. This just wakes up whenever a task is dropped in the queue, runs until the queue is empty, and then goes back to sleep.
I am far from an expert on threading. Is there a reason to mess around with System.Threading for a problem like this if using a BackgroundWorker will do?

C# Timer or Thread.Sleep

I am running a windows service and using a loop and Thread.Sleep to repeat a task, would it be better to use a timer method?
If yes a code example would be great
I am currently using this code to repeat
int curMinute;
int lastMinute = DateTime.Now.AddMinutes(-1).Minute;
while (condition)
{
curMinute = DateTime.Now.Minute;
if (lastMinute < curMinute) {
// do your once-per-minute code here
lastMinute = curMinute;
}
Thread.Sleep(50000); // sleeps for 50 seconds
if (error condition that would break you out of this) {
break; // leaves looping structure
}
}

A timer is a better idea, IMO. That way, if your service is asked to stop, it can respond to that very quickly, and just not call the timer tick handler again... if you're sleeping, the service manager will either have to wait 50 seconds or kill your thread, neither of which is terribly nice.

class Program
{
static void Main(string[] args)
{
Timer timer = new Timer(new TimerCallback(TimeCallBack),null,1000,50000);
Console.Read();
timer.Dispose();
}
public static void TimeCallBack(object o)
{
curMinute = DateTime.Now.Minute;
if (lastMinute < curMinute) {
// do your once-per-minute code here
lastMinute = curMinute;
}
}
The code could resemble something like the one above

It's important to understand that your code will sleep for 50 seconds between ending one loop, and starting the next...
A timer will call your loop every 50 seconds, which isn't exactly the same.
They're both valid, but a timer is probably what you're looking for here.

Beware that calling Sleep() will freeze the service, so if the service is requested to stop, it won't react for the duration of the Sleep() call.

Yes, using a Timer will free up a Thread that is currently spending most of its time sleeping. A Timer will also more accurately fire every minute so you probably won't need to keep track of lastMinute anymore.

Not quite answering the question, but rather than having
if (error condition that would break you out of this) {
break; // leaves looping structure
}
You should probably have
while(condition && !error_condition)
Also, I'd go with a Timer.

I have used both timers and Thread.Sleep(x), or either, depending on the situation.
If I have a short piece of code that needs to run repeadedly, I probably use a timer.
If I have a piece of code that might take longer to run than the delay timer (such as retrieving files from a remote server via FTP, where I don't control or know the network delay or file sizes / count), I will wait for a fixed period of time between cycles.
Both are valid, but as pointed out earlier they do different things. The timer runs your code every x milliseconds, even if the previous instance hasn't finished. The Thread.Sleep(x) waits for a period of time after completing each iteration, so the total delay per loop will always be longer (perhaps not by much) than the sleep period.

I required a thread to fire once every minute (see question here) and I've now used a DispatchTimer based on the answers I received.
The answers provide some references which you might find useful.

I agree as well, using a timer is the best option. I have tried a solution similar to yours in the past and started having issues where the loop would misfire, and I would have to wait for another Thread.Sleep() before it would fire again. Also, it did cause all sorts of issues with stopping the service, I would get constant errors about how it wasn't responding and had to be closed.
#Prashanth's code should be exactly what you need.

You can use either one. But I think Sleep() is easy, clear and shorter to implement.

I would have to say a sleep is a better implementation with a state machine behind it. This would still keep you in control of the application at all times, but allowing any response needed at any specific time. This also will handle timer callbacks that are shorter than the "Processing execution time in the loop"
For example..
<!-- language: c# -->
public enum State
{
Idle = 0,
Processing = 1,
Stop = 100,
}
public void Run()
{
State state = State.Idle; // could be a member variable, so a service could stop this too
double intervalInSeconds = 60;
System.DateTime nextExecution = System.DateTime.Now.AddSeconds(intervalInSeconds);
while (state != State.Stop)
{
switch (state)
{
case State.Idle:
{
if (nextExecution > System.DateTime.Now)
{
state = State.Processing;
}
}
break;
case State.Processing:
{
// do your once-per-minute code here
// if you want it to stop, just set it state to stop.
// if this was a service, you could stop execution by setting state to stop, also
// only time it would not stop is if it was waiting for the process to finish, which you can handle in other ways
state = State.Idle;
nextExecution = System.DateTime.Now.AddSeconds(intervalInSeconds);
}
break;
default:
break;
}
System.Threading.Thread.Sleep(1);
}
}