Ive checked already some answers, but still Im not convinced what is this right approach of acquiring and releasing COM objects in parallel.
In particular I use a Parallel.ForEach to increase performance, and inside it, it makes calls to MS.Outlook (2010 ExchangeServer). However, by releasing the COM objects I get occasionally COMExceptions.
What is the right approach of working with COM objects with the Parallel library ?
System.Threading.Tasks.Parallel.ForEach(myList, myItem =>
{
String freeBusySlots = "";
Outlook.Recipient myReceipient = null;
try
{
myReceipient = namespaceMAPI.CreateRecipient(myItem.ToString());
}
catch (Exception ex)
{
...
}
finally
{
if (myReceipient == null)
{
...
}
Marshal.ReleaseComObject(myReceipient ); // -> I get an exception here sometimes ... how to avoid this
myReceipient = null;
}
}); // Parllel.forEach
Outlook Object Model cannot be used from secondary threads. Sometimes it works, but it tends to bomb out at the most inappropriate moment.
As of Outlook 2013, Outlook will immediately raise an error if an OOM object is accessed from a secondary thread.
If your code is running from another application, keep in mind that all calls will be serialized to the main Outlook thread anyway so there is really no point using multiple threads.
Also note that Extended MAPI (C++ or Delphi only) or Redemption (which wraps Extended MAPI and can be accessed from any language - I am its author) can be used from multiple threads, but your mileage will vary depending on the particular MAPI provider (IMAP4 store provider is the worst).
As a general rule you should never be using ReleaseComObject. Instead just wait for the RCW object to be collected and let the GC do the work for you. Using this API correctly is extremely hard.
Additionally it's very likely that all of Outlooks COM objects live in the STA. If that is the case there is nothing gained by operating over them in parallel. Every call made from a background thread will simply be marashalled back to the foreground thread for processing. Hence the background thread will add no value, just confusion.
I'm not 100% certain these are STA objects but I'd be very surprised if they weren't.
Related
I need to get EntryIDs for all contacts in Outlook folder. If I have 6000 contacts in a folder, it takes about 100 seconds to execute (it doesn't matter much if if's background thread or main thread, I tried both). The code is like this:
List<Outlook.ContactItem> contactItemsList = null;
Outlook.Items folderItems = null;
Outlook.MAPIFolder folderSuggestedContacts = null;
Outlook.NameSpace ns = null;
Outlook.MAPIFolder folderContacts = null;
object itemObj = null;
try
{
contactItemsList = new List<Outlook.ContactItem>();
ns = Application.GetNamespace("MAPI");
// getting items from the Contacts folder in Outlook
folderContacts = ns.GetDefaultFolder(Outlook.OlDefaultFolders.olFolderContacts);
folderItems = folderContacts.Items;
for (int i = 1; folderItems.Count >= i; i++)
{
itemObj = folderItems[i];
if (itemObj is Outlook.ContactItem)
contactItemsList.Add(itemObj as Outlook.ContactItem);
else
Marshal.ReleaseComObject(itemObj);
}
Marshal.ReleaseComObject(folderItems);
folderItems = null;
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show(ex.Message);
}
finally
{
if (folderItems != null)
Marshal.ReleaseComObject(folderItems);
if (folderContacts != null)
Marshal.ReleaseComObject(folderContacts);
if (folderSuggestedContacts != null)
Marshal.ReleaseComObject(folderSuggestedContacts);
if (ns != null)
Marshal.ReleaseComObject(ns);
}
var ids = contactItemsList.Select(c => c.EntryID).ToArray();
The part to collect items takes about 5-8 seconds while the last line takes about 80-90 seconds.
Is there a faster way? I thought of Items.SetColumns initially but it turns out it doesn't work for EntryID (which seems strange decision of Outlook developers as EntryID is just a string and SetColumns is intended for quick retrieval of string properties).
Moreover, Outlook almost freezes for all this time (~100 seconds) even with BackgroundWorker. You can accidentally click something but it's like FPS rate is 1-2 FPS or so. It seems background execution doesn't help much. I suspect it's because background execution is great when the task being executed is not CPU-intensive but getting EntryIDs is heavy operation and thus severely interferes with UI.
I also have some legacy C++/COM code which does the same and it's slow as well. Looks like .NET interop is not the root cause of the issue. Maybe there is another API call which I should use for that?
I'm currently testing with Outlook 2010 64-bit.
Use MAPITable.GetTable to retrieve properties from multiple items in a single call without opening them at all (which is really expensive).
Secondly, OOM cannot be used on a secondary thread - it's never been supported, and Outlook 2016 will raises an exception as soon as it detects that it is being used from anything but the main UI thread. Only Extended MAPI (C++ or Delphi) is thread-safe. You can also use Redemption (any language - I am its author) and its RDO family of objects - it is 100% Extended MAPI based and can be used from a secondary thread. In particular, you can use
RDOFolder.Items.MAPITable.ExecSQL("SELECT EntryID FROM FOLDER WHERE MessageClass = 'IPM.Contact' ")
to retrieve entry ids of all contacts as a recordset.
OK, got it. Splitting "for" in chunks and keeping the number of simultaneously existing Outlook.ContactItem objects (like in my comment above) was one part of the optimization but more important is another thing.
The docs say you cannot use Items.SetColumns("EntryID") but never explain why. Actually, you cannot pass it there because EntryID is always returned! So, passing any other light-weight property (I used "Initials") will do the trick. It returns the object with just EntryID and Initials fields set and this improves performance by 4-5 times.
Combined with chunks optimization, I now have everything completed in 6 seconds in the background thread (with the main thread it's even faster).
I'm having a hard time with some code I have that apparently struggles when called from the second window created by the ShareTarget contract (when you share something to the app, it opens in a small standalone window).
This is my code so far:
// Blur and resize the image to get the average HSL color
// Assume that stream is an IRandomAccessStream pointing to valid image data
HslColor hslMean;
using (RandomAccessStreamImageSource imageProvider = new RandomAccessStreamImageSource(stream))
using (BlurEffect blurEffect = new BlurEffect(imageProvider) { KernelSize = 256 })
{
Color mean = await DispatcherHelper.GetFromUIThreadAsync(async () =>
{
WriteableBitmap
blurred = new WriteableBitmap((int)decoder.PixelWidth, (int)decoder.PixelHeight),
result = await blurEffect.GetBitmapAsync(blurred, OutputOption.Stretch),
resized = result.Resize(1, 1, WriteableBitmapExtensions.Interpolation.Bilinear);
return resized.GetPixel(0, 0);
});
hslMean = mean.ToHsl();
}
Note: that DispatcherHelper.GetFromUIThreadAsync method just checks the thread access to the UI thread, and if needed it schedules the code to a CoreDispatcher object that was obtained with CoreApplication.MainView.CoreWindow.Dispatcher.
Problem: this code works 100% fine if my app is already open, as at that point that CoreDispatcher object has already been created by previous calls to that DispatcherHelper class, so the method just uses the stored dispatcher to schedule the work and it works fine. But, if the app is closed when the ShareTarget window is opened (so that DispatcherHelper has to create the dispatcher for the first time) the CoreApplication.MainView.CoreWindow line throws an exception. A very weird one:
COMException:
A COM call to an ASTA was blocked because the call chain originated in or passed through another ASTA. This call pattern is deadlock-prone and disallowed by apartment call control.
A COM call (IID: {638BB2DB-451D-4661-B099-414F34FFB9F1}, method index: 6) to an ASTA (thread 10276) was blocked because the call chain originated in or passed through another ASTA (thread 4112). This call pattern is deadlock-prone and disallowed by apartment call control.
So, I needed a way to make that method reliable even when being called from different windows. I've tried different options:
#1: Just invoking that code without dispatching to a different thread, as in theory I should be on the UI thread at this point ---> FAIL (The application called an interface that was marshalled for a different thread. (Exception from HRESULT: 0x8001010E (RPC_E_WRONG_THREAD)))
#2: Manually calling CoreApplication.MainView.CoreWindow.Dispatcher to dispatch that code block ---> FAIL (I get that weird COMException mentioned above)
#3: Manually using CoreApplication.MainView.Dispatcher to dispatch the code block (as it was the .CoreWindow part that spawned the exception) ---> FAIL (COMException: item not found)
#4: Using CoreApplication.GetCurrentView().CoreWindow.Dispatcher, CoreApplication.GetCurrentView().Dispatcher, Window.Current.CoreWindow.Dispatcher and Window.Current.Content.Dispatcher to schedule that code ---> FAIL (wrong thread again, I get the usual marshalling exception)
All these marshalling exception are thrown at the line result = await blurEffect.GetBitmapAsync(blurred, OutputOption.Stretch), so I suspect it might be something related to the Lumia Imaging SDK.
I mean, I'm quite sure that I am in fact on the UI thread, or otherwise I wouldn't have managed to create an instance of the WriteableBitmap class, right?
Why is it that I can create WriteableBitmap objects (and they need to be created on the UI thread as far as I know), but that GetBitmapAsync method from the Lumia SDK always throws that marshalling exception? I'm using it everywhere in my app without any problems, why is it that it just won't work from the ShareTarget window? Is there anything I need to do?
Thanks for your help!
Looks like this is a bug in the Lumia Imaging SDK (that was originally written for WP8.1, which didn't have multiple windows/dispatchers), so unless the call to the library is made from the dispatcher associated with the main app window (which of course can only be retrieved if the app is open in the background when the ShareTarget window pops up), it will just fail.
The only solution at this point is to replace that call to the Lumia SDK with some other code that doesn't rely on that particular library (in this case for example, it is possible to just get the ARGB array from the WriteableBitmap object and calculate the mean color manually).
I have a webcam control using DirectShow.NET. I've created a custom control to display video and capture an image from a webcam. I'm using that custom control in another WPF window. I have a function public Bitmap CaptureImage() in the custom control to abstract away a little bit of the DirectShow programming and simply return a Bitmap. Since the images are relatively large (1920x1080), the GetCurrentImage() function of the IVMRWindowlessControl9 takes quite a while to process (2-3 seconds). I've stepped through my code and can confirm that this call is the only one that takes a long time to process.
Because of this, the GUI thread in my main WPF window hangs, causing it to be unresponsive for a few seconds, so if I want to display a progress spinner while the image is being captured, it will just remain frozen.
Here is the code for CaptureImage():
public Bitmap CaptureImage()
{
if (!IsCapturing)
return null;
this.mediaControl.Stop();
IntPtr currentImage = IntPtr.Zero;
Bitmap bmp = null;
try
{
int hr = this.windowlessControl.GetCurrentImage(out currentImage);
DsError.ThrowExceptionForHR(hr);
if (currentImage != IntPtr.Zero)
{
BitmapInfoHeader bih = new BitmapInfoHeader();
Marshal.PtrToStructure(currentImage, bih);
...
// Irrelevant code removed
...
bmp = new Bitmap(bih.Width, bih.Height, stride, pixelFormat, new IntPtr(currentImage.ToInt64() + Marshal.SizeOf(bih)));
bmp.RotateFlip(RotateFlipType.RotateNoneFlipY);
}
}
catch (Exception ex)
{
MessageBox.Show("Failed to capture image:" + ex.Message);
}
finally
{
Marshal.FreeCoTaskMem(currentImage);
}
return bmp;
}
In order to fix this, I've tried to run this as a background task as follows:
public async void CaptureImageAsync()
{
try
{
await Task.Run(() =>
{
CaptureImage();
});
}
catch(Exception ex)
{
MessageBox.Show(ex.Message);
}
}
I've tried multiple ways to do this including using BackgroundWorkers, but it seems that any time I make this call asynchronously, it creates this error:
Unable to cast COM object of type 'DirectShowLib.VideoMixingRenderer9'
to interface type 'DirectShowLib.IVMRWindowlessControl9'. This
operation failed because the QueryInterface call on the COM component
for the interface with IID '{8F537D09-F85E-4414-B23B-502E54C79927}'
failed due to the following error: No such interface supported
(Exception from HRESULT: 0x80004002 (E_NOINTERFACE)).
The error always happens when on this line:
int hr = this.windowlessControl.GetCurrentImage(out currentImage);
Calling CaptureImage() synchronously yields normal results. The image is captured and everything works as intended. However, switching to using any kind of asynchronous functionality gives that error.
You are having two issues here. First of all the original slowliness of the API is behavior by design. MSDN mentions this as:
However, frequent calls to this method will degrade video playback performance.
Video memory might be pretty slow for read-backs - this is the 2-3 second processing problem, not the resolution of the image per se. Bad news is that it is highly likely that even polling snapshots from background thread is going to affect the visual stream.
This method was and is intended for taking sporadic snapshots, esp. those initiated by user interactively, not automated. Applications needing more intensive and automated, and those not affecting visual feed snapshots are supposed to intercept the feed before it is sent to video memory (there are options to do it, and most popular yet clumsy is use of Sample Grabber).
Second, you are likely to be hitting .NET threading issue described in this question, which triggers the mentioned exception. It is easy to use the same interface pointer in native code development by sneaky violating COM threading rules and passing the interface pointer between apartments. Since CLR is adding a middle layer between your code and COM object doing additional safety checks, you can no longer operate with the COM object/interfaces from background thread because COM threading rules are enforced.
I suppose you either have to keep suffering from long freezes associated with direct API calls, or add native code development that helps to bypass the freezes (especially, for example, a helper filter which captures frames before sending to video memory and in the same time implementing helper functionality for .NET caller to support background thread calls). Presumably you can also do all DirectShow related stuff in a pool of helper MTA threads which solve you the background thread caller problem, but in this case you will most likely need to move this code from your UI thread which is STA - I don't think it something people often, if at all, doing.
as I am new in multithreaded application I would like to have some advice from more experienced people before starting to write the code...
I need to queue data received on serial port in serial port event for further processing.
So I have the following event handler:
void jmPort_ReceivedEvent(object source, SerialEventArgs e)
{
SetStatusLabel("Iddle...", lbStatus);
SetPicVisibility(ledNotReceiving, true);
SetPicVisibility(ledReceiving, false);
String st = jmPort.ReadLine();
if (st != null)
{
lines.Enqueue(st); //"lines" is the ConcurrentQueue<string> object
StartDataProcessing(lines); //???
SetStatusLabel("Receiving data...", lbStatus);
SetPicVisibility(ledNotReceiving, false);
SetPicVisibility(ledReceiving, true);
}
else
{
jmPort.Close();
jmPort.Open();
}
}
Within the StartDataProcessing I need to dequeue strings and update MANY UI controlls (using the InvokeRequired...this I already know :-)).
What is the best approach and colision free (without deadlock) approach to achieve this?
How to call StartDataProcessing method in more threads and safely dequeue (TryDequeue) the lines queue, make all needed computations and update UI controlls?
I have to appoint that the communication is very fast and that I am not using the standard SerialPort class. If I simply write all received strings without further processing to console window it works just well.
I am working in .NET 4.5.
Thank you for any advice...
Updated question: Ok, so what will be the best way to run the task from the datareceived event using TPL? Is it necessary to create another class (object) that will process data and use callbacks to update UI or it is possible to load some form method from the event? I'll could be very happy if someone can give me the direction what exactly to do within the datareceived event. What to do as the first step because studying all possible ways is not the solution I have time for. I need to begin with some particular way... There is so many different possible multithreading approaches and after reading about them I am still more confused and I don't know what will be the best a fastest solution... Usual Thread(s), BackgroundWorker, TPL, async-await...? :-( Because my application uses .NET 4.5 I would like to use some state-of-the-art solution :-) Thank you for any advice...
So after a lot of trying it is working to my satisfaction now.
Finally I've used the standard .NET SerialPort class as the third-party Serial class causes somae problems with higher baudrates (115200). It uses WinAPI directly so the finall code was mixed - managed and unmanaged. Now, even the standard .NET 4.5 SerialPort class works well (I've let my application successfully running through a whole night).
So, for everyone that need to deal with C#, SerialPort and higher rates (only for clarification - the device sending messages to PC is the STM32F407 /using USART 2/. I've tried it also with Arduino Due and it works as well) my datareceived event is in the following form now:
private void serialPort1_DataReceived(object sender, System.IO.Ports.SerialDataReceivedEventArgs e)
{
//the SetXXXXX functions are using the .InvokeRequired approach
//because the UI components are updated from another thread than
//the thread they were created in
SetStatusLabel("Iddle...", lbStatus);
SetPicVisibility(Form1.frm.ledNotReceiving, true);
SetPicVisibility(Form1.frm.ledReceiving, false);
String st = serialPort1.ReadLine();
if (st != null)
{
lines.Enqueue(st);
Task.Factory.StartNew(() => StartDataProcessing(lines)); // lines is global ConcurrentQueue object so in fact there is no need to pass it as parameter
SetStatusLabel("Receiving data...", lbStatus);
SetPicVisibility(Form1.frm.ledNotReceiving, false);
SetPicVisibility(Form1.frm.ledReceiving, true);
}
}
Within the StartDataProcessing function:
1. TryDequeue(lines, out str)
2. Use the ThreadPool.QueueUserWorkItem(lCallBack1, tmp); where tmp is needed part of the str (without EOF, without the message number etc.)
lCallBack1 = new WaitCallback(DisplayData);
Within the DisplayData function all the UI controls are updated
This approach mixes the ThreadPool and TPL ways but it is not a problem because the ThreadPool is used by TPL in background operation anyway.
Another working method I've tried was the following:
ThreadPool.QueueUserWorkItem(lCallBack, lines);
instead of :
Task.Factory.StartNew(() => StartDataProcessing(lines));
This method was working well but I've not tested it in over night run.
By my subjective perception the Task.... method updated the controls more smoothly but it can be only my personal feeling :-)
So, I hope this answer will help someone as I know from forums that many people are dealing with with unreliable communication based on the micocontroller <--> PC
My (surprising :-) ) conclusion is that the standard .NET SerialPort is able to handle messages even at higher baudrates. If you still run into troubles with buffer overrun then try to play with the SerialPort buffer size and SerialPort threshold. For me the settings 1024/500 are satisfactory (max size of the message send by microcontroller is 255 bytes so 500 bytes means that 2 messages are in buffer before the event is fired.)
You can also remove all SetXXXX calls from the datareceived event as they are not really needed and they can slow down the communication a little...
I am very close to real-time data capturing now and it is exactly what I've needed.
Good luck to everyone :-)
Within the StartDataProcessing I need to dequeue strings and update MANY UI controlls
No, you do not. You need to dequeue strings and then enqueue them again into the multiple queues for the different segments of the UI.
If you want to be fast, you scatter all operations and definitely the UI into separate windows that run their own separate message pumps and thus can update independently in separate UI threads.
The general process would be:
1 thread handles the serial port and takes the data and queues it.
Another one dequeues it and distributes it to separate processing threads from which
the data goes to multiple output queues all responsible for one part of the UI (depending on whether the UI Will turn a bottleneck).
There is no need to be thread safe in dequeuing. How serial is the data? Can you skip data when another update for the same piece arrives?
Read up on TPL and tasks - there are base libraries for parallel processing which come with a ton of documentation.
In my c# application multiple clients will access the same server, to process one client ata a time below code is written.In the code i used Moniter class and also the queue class.will this code affect the performance.if i use Monitor class, then shall i remove queue class from the code.
Sometimes my remote server machine where my application running as service is totally down.is the below code is the reasond behind, coz all the clients go in a queue, when i check the netstatus -an command using command prompt, for 8 clients it shows 50 connections are holding in Time-wait...
Below is my code where client acces the server ...
if (Id == "")
{
System.Threading.Monitor.Enter(this);
try
{
if (Request.AcceptTypes == null)
{
queue.Enqueue(Request.QueryString["sessionid"].Value);
string que = "";
que = queue.Dequeue();
TypeController.session_id = que;
langStr = SessionDatabase.Language;
filter = new AllThingzFilter(SessionDatabase, parameters, langStr);
TypeController.session_id = "";
filter.Execute();
Request.Clear();
return filter.XML;
}
else
{
TypeController.session_id = "";
filter = new AllThingzFilter(SessionDatabase, parameters, langStr);
filter.Execute();
}
}
finally
{
System.Threading.Monitor.Exit(this);
}
}
Locking this is pretty wrong, it won't work at all if every thread uses a different instance of whatever class this code lives in. It isn't clear from the snippet if that's the case but fix that first. Create a separate object just to store the lock and make it static or give it the same scope as the shared object you are trying to protect (also not clear).
You might still have trouble since this sounds like a deadlock rather than a race. Deadlocks are pretty easy to troubleshoot with the debugger since the code got stuck and is not executing at all. Debug + Break All, then Debug + Windows + Threads. Locate the worker threads in the thread list. Double click one to select it and use Debug + Call Stack to see where it got stuck. Repeat for other threads. Look back through the stack trace to see where one of them acquired a lock and compare to other threads to see what lock they are blocking on.
That could still be tricky if the deadlock is intricate and involves multiple interleaved locks. In which case logging might help. Really hard to diagnose mandelbugs might require a rewrite that cuts back on the amount of threading.