Develop Reference

Are there alternatives to invoke in WPF when passing data from a different thread?

I know how to pass data from a worker thread to the main thread via Invoke/BeginInvoke.
I can also pull from a thread safe collection with a timer from the main thread.
I prefer using tasks (Task.Factory.StartNewTask()) and using Backgroundworker with them seams a bit clumsy.
Sometimes the gui is a bit laggy which is due to (Begin)Invoke I assume.
Pulling with a timer also doesn't feel like the right way.
Creating a new Backgroundworker for each new task seams also strange.
Are there any other possibilities?

In .NET, memory is shared within an AppDomain, which means all threads can access all data. So, what you're actually doing is controlling access to particular bits of data from particular threads so they don't interfere with each other.
Invoke and BeginInvoke allow you to run code on the UI thread, which is useful because UI controls can only be accessed from the UI thread. BackgroundWorker is another solution, as is SynchronizationContext.
However, they all work by sending known windows messages to the UI message loop. If you call Invoke too often, you send too many messages and the UI thread is swamped which makes the UI "a bit laggy".
If this happens, you must slow down the rate that messages are sent. There are a couple of ways to do this:
1) Call Invoke less frequently: this means waiting for bigger "chunks" of state changes in your background thread before it calls Invoke to update the UI.
2) Use a UI Timer: there is no point in trying to update the UI faster than the human eye can detect. A UI Timer also sends windows messages to the UI message loop, but at a known rate. The Tick handler can then pull the necessary data from shared memory to update the UI.
Both approaches have their strengths and weaknesses. The choice really depends on how easy it is to group state changes in the background thread into bigger chunks, while making sure the UI doesn't miss any state changes.

How can I use threads to run database queries in XNA?

I'm currently developing a project with XNA that is pulling information (ID, name, file location, etc) about each of my objects (each object will be displayed on screen) from a local SQL database.
I'd like to run my database queries on a separate thread so the rendered screen doesn't freeze if the database hangs or some other unforeseen event occurs. I'm using XNA 4.0 and the application will only be running on windows. Is this possible, and if so, how?

There are a number of options available. Generally speaking you need the query to run in a separate thread. You can use
Thread pool
QueueUserWorkItem
Tasks
Background worker
Async calls to the database
Parallel invoke
Manually created threads here and here
I would start with thread pooling and see how that works, dedicated manual threads are not that robust in terms of memory management and reuse.

Not to do it at all. Seriously. There are good reasons for using threads, but your reasons are bogus:
the rendered screen doesn't freeze if the database hangs or some other unforeseen event occur
Databases dont hang and unforseen events are unforseen events. How you can cope with the database not answering for 3 minutes, for example? Show a screen with objects that are unknown?

How do you mean "best"? There are a lot of ways to use threads and they all have strengths and weaknesses.
Declaring a new thread explicitly and starting it gives you the most direct control over the execution state of that thread:
var myDbThread = new Thread(()=>myDbRepo.GetRecordById<MyEntity>(idString));
myDbThread.Start();
Now, as long as you have a reference to myDbThread, you can abort it, pause it, join on it, etc. BUT, with control comes responsibility; you have to manage the threads you create yourself.
For most parallel tasks, using the ThreadPool is recommended. However, you lose some of the control:
Action myDbLambda = () => myEntityProperty = myDbRepo.GetRecordById<MyEntity>(idString);
var asyncResult = myDbLambda.BeginInvoke();
Once asyncResult.IsComplete returns true, myEntityProperty has the value. You can also architect it as a Func, and use a callback to set the value (this is recommended). The Asynchronous Model is built in to the BeginInvoke()/EndInvoke() method pair, and many exceptions like timeouts are expected by the ThreadPool, which will simply restart the timed-out thread. However, you can't "give up" and terminate a ThreadPool thread, "joining" on a ThreadPool thread is a little trickier, and if you're launching a lot of threads, the ThreadPool will start them in 250ms intervals which may not be the best use of processor.
There are many ways to use the ThreadPool; before delegates became even more important to .NET programming in v3.5, ThreadPool.QueueUserWorkItem was the main method. Now, as I said, delegates have BeginInvoke and EndInvoke methods allowing you to kick off background processes with the asynchronous model built in behind the scenes. In WinForms/WPF, you can also create BackgroundWorker components which are event-driven, allowing you to monitor progress and completion in a GUI element.
One thing to be aware of; it is virtually never a good idea to use background threads in ASP.NET. Unless you really know what you're doing, best-case you won't get the results of the behavior you sent to the worker thread, and worst-case you can crash your site trying.

BackgroundWorker component in services

I know the BackgroundWorker should not be used in Windows Services but would anyone have a good online reference explaining why?

BackgroundWorker relies on a current SynchronizationContext being set in order to function. It's really intended and designed specifically for working with UI code.
It's typically better in a service to self-manage your threads, since there are no UI synchronization issues. Using the threading API (or .NET 4 Task API) is a much better option here.

Well, it's okayish to use a BGW in a service, it just doesn't do anything especially useful. Its reason for being is its ability to raise the ProgressChanged and RunWorkerCompleted events on a specific thread. Getting code to run on a specific thread is a very non-trivial thing to do. You cannot simply inject a call into the thread while it is executing code. That causes horrible re-entrancy problems. The thread has to be 'idle', in a state where inject code doesn't cause trouble.
Having a thread in an idle state is a fairly unnatural condition. You use threads to run code, not for them to be idly spinning its heels. This is however the way a UI thread works. It spends 99% of its time in the message loop, waiting for Windows to tell it to do something. A button click, a paint request, a keyboard press, that sort of thing. While it is inside the message loop, it is in fact idle. A very good time to execute injected code.
Which is what Winforms' Control.Begin/Invoke and WPF's Dispatcher.Begin/Invoke do. They put a delegate in a queue, the queue is emptied and the delegate targets executed by the message loop. The WindowsFormsSynchronizationContext and DispatcherSynchronizationContext classes are the synchronization providers that uses them. Winforms and WPF replace SynchronizationContext.Current with an instance of them. Which in turn gets used by BGW to raise the events. Which makes them run on the UI thread. Which allows you to update the non thread-safe user interface components from a worker thread.
You can probably see where this is heading, a service uses neither. The default synchronization provider doesn't synchronize anything. It simply uses a threadpool thread to call the Send or Post callback. Which is what will happen when you use BGW in a service. Now there is actually no point at all in having these events. You might as well let the DoWork handler call the event handling methods directly. After all, the thread on which DoWork runs is just another threadpool thread as well.
Well, no real harm done, other than making it quite a bit slower.

I've used BackgroundWorker in windows services many times without any ill effect. While its use of SynchronizationContext may be unnecessary, I haven't observed it causing problems or poor performance.

Find which module is blocking the UI thread

This is a two part question:
I am working on a big project where multiple plugins developed by different teams are loaded inside one common container shell. At times I can see that my UI updates are blocked as there are multiple parallel UI updates, i want to know if there is a way to find which component is blocking the ui thread
In .net how can i create a separate UI thread which requires dedicated UI intensive work?
Much Appreciate your help. Thanks.

Use the debugger. Debug + Break All when you notice it blocking. Then Debug + Windows + Threads and select the main thread. The call stack window shows you what it is doing.
A corner case is where these plugins are using a lot of calls to Control.Begin/Invoke or Dispatcher.Begin/Invoke. Your UI thread is not blocked in this case, it is just being overwhelmed by requests to dispatch the delegate targets. And doesn't get around to doing its normal duties anymore, like repainting the windows and responding mouse and keyboard events. There's little you can do about this beyond working with the plugin authors to get them to mend their ways.
You've already got an UI thread, the thread that created the first window. Creating additional threads that have their own windows is possible but causes unsolvable problems with window Z-order (a window will disappear underneath the window of another app) and generous helpings of window interop threading misery.

Visual Studio 2010's (in the higher SKUs) include features to check for this. If you run your program under the Concurrency Profiler, you can see exactly which threads are waiting on which locks when the deadlock occurs. In addition, it will highlight the deadlock (I believe in bright red) to make it easy to track down.

One approach you can take (though it may require a bit of redesign) is to disallow all plugin logic from running in the UI thread. All operations that require updates to the UI must be routed through well-defined service interfaces that can interpret, dispatch and perhaps even throttle the UI updates. This is only practical if your plugins are not deeply UI-centric and you have a service model that allows you to isolate the data being manipulated by the plugins from the visualization of that data. Without knowing more about your application, I can't give more concrete recommendations.

Here are two possible solutions to the problem that I came up with quickly. I am sure there are other equally valid solutions though.
Option 1: Instead of using the push model (via the ISynchronizeInvoke methods) switch to a pull (or poll) model in which the UI queries the plugin for updates. This has the following advantages.
It breaks the tight coupling between the UI and worker/plugin threads that Control.Invoke imposes.
It puts the responsibility of updating the UI thread on the UI thread where it should belong anyway.
The UI thread gets to dictate when and how often the update should take place.
There is no risk of the UI message pump being overrun as would be the case with the marshaling techniques initiated by the worker/plugin thread.
The worker/plugin thread does not have to wait for an acknowledgement that the update was performed before proceeding with its next steps (ie. you get more throughput on both the UI and worker/plugin threads).
Option 2: Have the plugin accept an ISynchronizeInvoke instance instead of an actual Form or Control. This special synchronizing object will be implemented using a dedicated thread and a queue that acts as buffer between the plugin and the UI. It will accept update messages via the normal Invoke or BeginInvoke methods, which means you can keep the plugin architecture and interfaces mostly intact, and then forward those messages on to the UI after some type of filtering, merging, and throttling operations have occurred. The number of update messages existing in the synchronizing object will ebb and flow as the UI and plugin threads work load changes. It could be smart enough to change its forwarding strategy as the rate of messages increase.

How do I Yield to the UI thread to update the UI while doing batch processing in a WinForm app?

I have a WinForms app written in C# with .NET 3.5. It runs a lengthy batch process. I want the app to update status of what the batch process is doing. What is the best way to update the UI?

The BackgroundWorker sounds like the object you want.

The quick and dirty way is using Application.DoEvents() But this can cause problems with the order events are handled. So it's not recommended
The problem is probably not that you have to yield to the ui thread but that you do the processing on the ui thread blocking it from handling messages. You can use the backgroundworker component to do the batch processing on a different thread without blocking the UI thread.

Run the lengthy process on a background thread. The background worker class is an easy way of doing this - it provides simple support for sending progress updates and completion events for which the event handlers are called on the correct thread for you. This keeps the code clean and concise.
To display the updates, progress bars or status bar text are two of the most common approaches.
The key thing to remember is if you are doing things on a background thread, you must switch to the UI thread in order to update windows controls etc.

To beef out what people are saying about DoEvents, here's a description of what can happen.
Say you have some form with data on it and your long running event is saving it to the database or generating a report based on it. You start saving or generating the report, and then periodically you call DoEvents so that the screen keeps painting.
Unfortunately the screen isn't just painting, it will also react to user actions. This is because DoEvents stops what you're doing now to process all the windows messages waiting to be processed by your Winforms app. These messages include requests to redraw, as well as any user typing, clicking, etc.
So for example, while you're saving the data, the user can do things like making the app show a modal dialog box that's completely unrelated to the long running task (eg Help->About). Now you're reacting to new user actions inside the already running long running task. DoEvents will return when all the events that were waiting when you called it are finished, and then your long running task will continue.
What if the user doesn't close the modal dialog? Your long running task waits forever until this dialog is closed. If you're committing to a database and holding a transaction, now you're holding a transaction open while the user is having a coffee. Either your transaction times out and you lose your persistence work, or the transaction doesn't time out and you potentially deadlock other users of the DB.
What's happening here is that Application.DoEvents makes your code reentrant. See the wikipedia definition here. Note some points from the top of the article, that for code to be reentrant, it:
Must hold no static (or global) non-constant data.
Must work only on the data provided to it by the caller.
Must not rely on locks to singleton resources.
Must not call non-reentrant computer programs or routines.
It's very unlikely that long running code in a WinForms app is working only on data passed to the method by the caller, doesn't hold static data, holds no locks, and calls only other reentrant methods.
As many people here are saying, DoEvents can lead to some very weird scenarios in code. The bugs it can lead to can be very hard to diagnose, and your user is not likely to tell you "Oh, this might have happened because I clicked this unrelated button while I was waiting for it to save".

Use Backgroundworker, and if you are also trying to update the GUI thread by handling the ProgressChanged event(like, for a ProgressBar), be sure to also set WorkerReportsProgress=true, or the thread that is reporting progress will die the first time it tries to call ReportProgress...
an exception is thrown, but you might not see it unless you have 'when thrown' enabled, and the output will just show that the thread exited.

Use the backgroundworker component to run your batch processing in a seperate thread, this will then not impact on the UI thread.

I want to restate what my previous commenters noted: please avoid DoEvents() whenever possible, as this is almost always a form of "hack" and causes maintenance nightmares.
If you go the BackgroundWorker road (which I suggest), you'll have to deal with cross-threading calls to the UI if you want to call any methods or properties of Controls, as these are thread-affine and must be called only from the thread they were created on. Use Control.Invoke() and/or Control.BeginInvoke() as appropriate.

If you are running in a background/worker thread, you can call Control.Invoke on one of your UI controls to run a delegate in the UI thread.
Control.Invoke is synchronous (Waits until the delegate returns). If you don't want to wait you use .BeginInvoke() to only queue the command.
The returnvalue of .BeginInvoke() allows you to check if the method completed or to wait until it completed.

Application.DoEvents() or possibly run the batch on a separate thread?

DoEvents() was what I was looking for but I've also voted up the backgroundworker answers because that looks like a good solution that I will investigate some more.