Develop Reference

What options do I have for long running processes in my C# Console Application?

I'm working on a console app that kicks off a fairly long running process (2-3 minutes). What options are available for displaying progress, or even just writing a "." to screen every few seconds so the user knows the app hasn't stopped responding?

You have (at least) 3 ways to do this:
Simplest: On program start, set a global variable to false, then start a thread that writes a dot to the console, Thread.Sleep()s a second, repeats until the variable is set to true. On your main thread do your work, when finished set the variable to true and then Join() the other thread.
Still simple: Add another global variable to the mix, type int. In your worker thread increase it, whenever some progress is made, on the console writer thread reduce interval to say 250ms, but draw a dot only if progress counter has changed. This gives an idea of how fast your app progresses
A bit advanced: Create a boolean as in first step plus an AutoResetEvent, let the console writer thread repeatedly wait on the event, when app makes progress Set() the event. When finished set boolean to true and Set() again before Join()ing. The advanced part in this is to know, in what portions to report progress.

The simplest approach is if your application has an incremental loop then you could put a simple full stop on the the console. Not nice, but it does give an indication of "I'm alive".
But that approach can result in too much info (too many characters going to console) or too few as your hooking into a bit of the code that does not really have that responsibility.
So, perhaps a nicer way that you can reuse is to create a console user class that is run in a thread with a 1 second tick. It can use backspacing and the good old | / | / - sequence to give the impression of something rotating. It can also check for key presses to allow the user to exit.
Another option, depending on the nature of your users, is to use a logging framework like NLog. That way your implementation is UI independent and you can see what is happening in more or less detail at run time in the console, from another machine, whatever.

How to always running a control and all its child on a separate thread?

I have written an application which does lots of calculation on huge floating point numbers that makes the UI not responsive at all most of the time.
I am thinking of adding a status bar to this application and show some info about cpu load, used memory and a progress bar. Consider adding labels and progress bar to the statusbar as childs, how can I run this status bar on a separate thread which can be reliable to be responsive as much as possible?
I can already use progress bars and system diagnosting stuff normally. What I am looking for is your ideas and tips, possibly with some codes!
Update
I want the status bar shows real time cpu and memory details. How to workaround this?

You've got this the wrong way round. You should run all the UI in the same thread, and run the long calculation in a background worker. Trying to run UI in different threads within the same app just leads to pain.

Best thing is to just do a google search on BackgroundWorker. Here is one particular result on how to use it.
http://midnightprogrammer.net/post/Using-Background-Worker-in-C.aspx

Opinion on User Experience - C# Winforms

I’ve got a process which will take a little under 5 seconds to complete. The user will most likely notice the program flicker for a few seconds after pushing the “go” button.
My question is:
Is this something that would normally be dumped onto a background worker, or is there another .NET method for handling small tasks, or is this something that shouldn’t be a concern?
FYI:
The process opens a user specified excel file, processes an unknown number of lines (max 1.5 million due to excel I believe), and queries a database (very quick query). So at the worst case scenario the user uploads a 1.5 million row excel file and is running on a very slow internet connection.

If you don't want the user to be able to do anything while the file is being uploaded, then you don't need to put it on a different thread.
If you want the user to be able to go on to other tasks while the file is uploading, put it on a different thread.
As a general rule of thumb, if I have a situation where I absolutely don't want the user to do anything while a long-running process is going, I disable the controls on the form until the task is complete, and usually use a status indicator to show that progress is happening.
My personal guideline for whether or not to allow user interaction is if the results of a process could be altered by a user action in mid-stream.
For example, one program that we have parses a bunch of queries on a highly normalized database (normalized to the point where reporting is sloooow) into "reportable" tables, and I don't want the user altering data in one of the source tables while the query is running, because it will give goofy results.
If there is no harm in allowing user interaction while the process is occuring, then put it in another thread.
Edit
Actually, on reading #UrbanEsc and #archer's comments, I agree with them. Still put it on a different thread and freeze the controls (and include a progress indicator where possible).

I would push this to a background worker. Doing so will keep the UI responsive. If the process ever does lag for more than a few seconds, users start getting nervous ...especially when the lagging process causes the UI to be 'frozen'.

From a user experience point of view it might be best to hand the job over to a different thread or an asynchronous worker and tell the user that his request is being processed in the background. Once the worker finishes, a success/failure message can be handled and shown to the user as required.

The cheapest way to handle the problem is to turn the cursor into an hourglass during the processing. That tells user please wait, I'm busy.
According to the budget (time and/or effort) you're willing to throw in it, using a backgroundworker and some reporting GUI is certainly a plus. But it's up to you according to your app.
For example, I'm currently modifying an in-house app that has 3 users. In that case, the hourglass is OK: All 3 of them will quickly learn they just have to wait. Don't get me wrong: this app is damn important. Without it, the small company that uses it would just die. But if I ask them for 2 hours of extra budget for a nice and tested little GUI, background thread, blah vs an hourglass, what do you think they'll say?
On the other hand, if it's an important operation in your flagship product, of course be nice to your users! Don't hesitate: background thread. Especially if the operation may actually take much longer than those 5 seconds.
Conclusion: Be pragmatic!

I would put it into a background worker or fire of a task if you are in .NET 4.0, for example:
void OnButtonClick(...)
{
new TaskFactory().StartNew(() => { /* your excel and query code */ });
}

I'll vote for the background worker process, since a frozen UI is like a frozen application, and most of users will think your application isn't doing anything at all.
UI thread for a progress bar or some animation, info text noticing what's going on + background worker thread = win

I think every process not related with the UI itself should be started as a separate thred or, in this case, as a bg worker. This will help to maintain the app healthy and easy to improve/fix in the future.
Also, as a user or tester, I really hate flicking and freezing windows...
Regards.

A general rule of thumb is any operation that takes a second or longer to complete requires some form of feedback to the user. This can be a progress bar, message, etc. Anything longer then that then the user becomes frustrated (not sure if they did something wrong, hate waiting, etc).
For operations like this that can take longer based on the environment (number of apps, available memory, data size, hard drive speed, etc) they should ALWAYS be put on a background thread and pipe messages back to the UI. I love the BackGroundWorker for this.

Is there an 'Application.DoEvents' at the form level?

I currently have an application that spawns multiple instances of a single win form. Each instance of the form has a timer which updates a number of controls and some of the properties of the form itself at rate of approx 1 tick/sec. Once the application reaches a certain number of forms it stops painting the 'updates' that occur within the timer ticks.
I want to hold the UI thread at the bottom of the timer and allow the form to repaint without calling Application.DoEvents (my understanding is that Application.DoEvents() handles all pending window messages across the application (which could include other timers 'ticks'? - not sure) when all I want it to process is the redrawing of the form now that a number of the controls have been updated).

It's not clear to me why you want to call DoEvents at all. Why not just let the window repaint itself appropriately? What's wrong with the way it behaves at the moment?
Calls to DoEvents usually indicate that something should be in a background thread - in your case it sounds like it might actually just be that you've got too many events going on. If your GUI is so complex that it can't redraw itself in a second, it sounds like that's probably a design issue with the UI itself.

If you just want the form or a specific control to redraw itself, just call Control.Refresh on the form/control in question.
That being said, this is usually (like Application.DoEvents) a sign that a redesign is in order. If you have long running processes like this, it's a much better idea to push the work into a BackgroundWorker, which will make the actual computational work run in a background thread, and prevent the UI from becoming unresponsive.

There are three categories of Windows messages that are dispatched by the message loop and a call to Application.DoEvents(). First are messages that are sent with SendMessage(). They are important, they need to be dispatched right away because there is some other program waiting on the result of the SendMessage() call. They are not put on the message queue, Windows calls the window procedure directly.
Then there are messages that are put on the message queue with PostMessage(). They can wait, they are merely notifications. All of the keyboard and mouse messages fit that category.
Then there are the low priority messages, WM_TIMER and WM_PAINT. They only get dispatched when there's nothing else to do, no SendMessage is pending and the message queue is empty.
Sounds like you got yourself into a situation where you starving Windows so badly that it can't get around to dispatching those low priority message anymore. Probably neither getting timer Ticks nor Paint events anymore. Calling DoEvents only once a second will certainly do that. This is bad, your program is now affecting the operation of other programs as well. You'll have to fix that. Do so by re-architecting your app so you no longer depend on DoEvents.

My 2c:
Having a timer in each form is going to slow down performance to a crawl. Each window at each second interval is going to request the application to switch to that window's "context" and execute some code. That whole switch is likely longer than your code actually takes to run. Couple that with the likely fact that all of your forms are in the same "context".
Instead consider something like this if possible:
List<MyForm> listofmyform;
Timer t = new Timer()
t.tick += delegate
{
foreach(MyForm thisform in listofmyform)
thisform.DoUpdate();
}
This only uses one timer to update all of your forms, only requires one "context switch" and will likely cure some performance problems you're having.

C# Winforms How to update toolStrip in function

I'm building a UI for a program, and I can't figure out why my progress bar won't become visible after the convert button is clicked.
private void convertButton_Click(object sender, EventArgs e)
{
toolStripProgressBar.Visible = true;
...
toolStripProgressBar.Visible = false;
}
I ran into a similar problem with tkinter in Python, and I had to call a function to update the idle tasks. Is there a way to do this with windows forms without using threads?
Edit: On a side note, this is a progress bar in a toolStrip that also contains a label that gets updated with status bar text. Is there any way to get the label on the left side and the progress bar on the other instead of right next to each other on the left?

Well, there is a way to do this without using threads (Application.DoEvents) but I strongly recommend against you using it. Re-entrancy is nasty, and you really don't want the UI thread tied up at all.
Use BackgroundWorker instead - it's easy, and it's pretty much designed for progress bars. It takes the hassle out of using a separate thread and reporting progress back to the UI thread. No need for Control.Invoke etc - it takes care of that for you.
There are lots of tutorials for BackgroundWorker - it shouldn't take you too long to get going with it.

Per the question you asked for the way to do this WITHOUT threads, that is to do it with Application.DoEvents();. (Just add that call right after setting the progress bar as visible.)
Now I do agree with Jon Skeet though that BackgroundWorker is a better way of doing this, but it does use a separate thread.

You need to execute your process in a thread separate from the UI thread, and then have it periodically report back to the UI thread with it's progress. If your convert operation is working inside the UI thread, it will simply go unresponsive until the operation is complete.

The progress bar can only become visible when it is allowed to paint which occurs during the processing of messages. Message processing cannot normally happen while you are in the middle of an event handler. If you want the progress bar to show up you will have to set the visiblitity to true, start a background thread to complete the work and return from the handler.

I'm guessing the problem is that the "..." in your code is a long-running process. UI updates are not instantaneous, but must run through the message queue in windows and then be painted to the screen. The queue is pumped and painting takes place in the same thread as your events.
As a result, any long-running tasks need to be moved to a different thread. More than that, your line line of code needs to called after that thread terminates. Otherwise you set the progress bar and then immediately turn it off again.
One way to do that is with a BackgroundWorker control.

Here go two links trying to explain you how things work:
(1) (2)
Now, I will try to explain it as shortly as I can. Most of what happens inside a windows forms application happens in a single thread, usually the same thread Main() runs in. If you open Program.cs, you will see that Main() has a line that looks like the following:
Application.Run(new Form1());
If you debug the application at any moment and examine the call stack, you will see it will trace back to that Run method. This means that a Windows Forms application is in fact a continuous run of the Run method. So, what is Run doing? Run is eating a message queue through which Windows sends messages to it. Run then dispatches those messages to the correct controls, which themselves do things like add text which corresponds to the key being pressed, redraw themselves, etc. Notice that all this happens during and endless loop running alongside a single thread, so weather you are typing or simply moving the window around, loads of those messages are being passed onto the application, which in turn is processing them and reacting accordingly, all in that single thread. Controls can also send messages to themselves through the queue and even you can place messages in the pump via Control.BeginInvoke. One of the things those controls do is to raise events according to what happens. So, if you click a button, the code you've written to handle that click will ultimately and indirectly be run by the Application.Run method.
Now, what is happening with your code is that even though you are changing the visible status of your progress bar to visible and then updating its Value, you are then changing its visibility to false, all in the same method. This means that only after you leave the method, will Application.Run() be able to continue iterating and consuming the message queue, effectively asking the progress bar to update its display. When that happens, you've already left the progress bar's visibility to false, the last thing you did before exiting the method. DoEvents() is a quick and dirty workaround to your problem as it reads the messages in the queue and processes them. I don't really feel comfortable using it as it can bring reentrancy problems.
Using threads is a good solution, but I would recommend using a ThreadPool thread instead of a custom thread in this kind of situation, as I tend to use custom threads only in cases where I have a limited number of long lived threads and I need to control their life cycles. The easiest and most practical way to use threads is to use the BackgroundWorker component, even though I would recommend going through the pains of understanding how to do Windows Forms multithreading with delegates if you want to really understand what is going on.

My solution is to call refresh on the status strip.
I believe this causes the UI thread to repaint the status strip.
toolStripStatusBar1.PerformStep();
statusStrip1.Refresh();
This is for .NET 4.0. Even though this question is old it was the first I found on googling this issue.