Develop Reference

How do I make my application responsive during Entity Framework operations?

I have a fairly simple WPF application that uses Entity Framework. The main page of the application has a list of records that I am getting from a database on startup.
Each record has a picture, so the operation can be a little slow when the wireless signal is poor. I'd like this (and many of my SQL operations) to perform in the background if possible. I have async/await setup and at first it seemed to be working exactly as I wanted but now I'm seeing that my application is becoming unresponsive when accessing the DB.
Eventually I'm thinking I'm going to load up the text in one query and the images in another background operation and load them as they come in. This way I get the important stuff right away and the pictures can come in in the background, but the way things are going it's still looking like it will lock up if I do this.
On top of that, I'm trying to implement something to handle connectivity issues (in case the wifi cuts out momentarily) so that the application notifies the user of a connection issue, automatically retries a few times, etc. I put a try catch for SQL exception which seems to be working for me, but the whole application locks up for about a minute while it is trying to connect to the DB.
I tried testing my async/await using await Task.Delay() and everything is very responsive as expected while awaiting the delay, but everything locks up when awaiting the .ToListAsync(). Is this normal and expected? My understanding of async/await is pretty limited.
My code is kind of messy (I'm new) but it does what I need it to do for the most part. I understand there's probably plenty of improvements I can make and better ways to do things, but one step at a time here. My main goal right now is to keep the application from crashing during database accessing exceptions and to keep the user notified of what the application is doing (searching, trying to access db, unable to reach DB and retrying, etc) as opposed to being frozen, which is what they're going to think when they see it being unresponsive for over a minute.
Some of my code:
In my main view model
DataHelper data = new DataHelper();
private async void GetQualityRegisterQueueAsync()
{
try
{
var task = data.GetQualityRegisterAsync();
IsSearching = true;
await task;
IsSearching = false;
QualityRegisterItems = new ObservableCollection<QualityRegisterQueue>(task.Result);
OrderQualityRegisterItems();
}
catch (M1Exception ex)
{
Debug.WriteLine(ex.Message);
Debug.WriteLine("QualityRegisterLogViewModel.GetQualityRegisterQueue() Operation Failed");
}
}
My Data Helper Class
public class DataHelper
{
private bool debugging = false;
private const int MAX_RETRY = 2;
private const double LONG_WAIT_SECONDS = 5;
private const double SHORT_WAIT_SECONDS = 0.5;
private static readonly TimeSpan longWait = TimeSpan.FromSeconds(LONG_WAIT_SECONDS);
private static readonly TimeSpan shortWait = TimeSpan.FromSeconds(SHORT_WAIT_SECONDS);
private enum RetryableSqlErrors
{
ServerNotFound = -1,
Timeout = -2,
NoLock = 1204,
Deadlock = 1205,
}
public async Task<List<QualityRegisterQueue>> GetQualityRegisterAsync()
{
if(debugging) await Task.Delay(5000);
var retryCount = 0;
using (M1Context m1 = new M1Context())
{
for (; ; )
{
try
{
return await (from a in m1.QualityRegisters
where (a.qanClosed == 0)
//orderby a.qanAssignedDate descending, a.qanOpenedDate
orderby a.qanAssignedDate.HasValue descending, a.qanAssignedDate, a.qanOpenedDate
select new QualityRegisterQueue
{
QualityRegisterID = a.qanQualityRegisterID,
JobID = a.qanJobID.Trim(),
JobAssemblyID = a.qanJobAssemblyID,
JobOperationID = a.qanJobOperationID,
PartID = a.qanPartID.Trim(),
PartRevisionID = a.qanPartRevisionID.Trim(),
PartShortDescription = a.qanPartShortDescription.Trim(),
OpenedByEmployeeID = a.qanOpenedByEmployeeID.Trim(),
OpenedByEmployeeName = a.OpenedEmployee.lmeEmployeeName.Trim(),
OpenedDate = a.qanOpenedDate,
PartImage = a.JobAssembly.ujmaPartImage,
AssignedDate = a.qanAssignedDate,
AssignedToEmployeeID = a.qanAssignedToEmployeeID.Trim(),
AssignedToEmployeeName = a.AssignedEmployee.lmeEmployeeName.Trim()
}).ToListAsync();
}
catch (SqlException ex)
{
Debug.WriteLine("SQL Exception number = " + ex.Number);
if (!Enum.IsDefined(typeof(RetryableSqlErrors), ex.Number))
throw new M1Exception(ex.Message, ex);
retryCount++;
if (retryCount > MAX_RETRY) throw new M1Exception(ex.Message, ex); ;
Debug.WriteLine("Retrying. Count = " + retryCount);
Thread.Sleep(ex.Number == (int)RetryableSqlErrors.Timeout ?
longWait : shortWait);
}
}
}
}
}
Edit: Mostly looking for general guidance here, though a specific example of what to do would be great. For these types of operations where I am downloading data, is it just a given that if I need the application to be responsive I need to be making multiple threads? Is that a common solution to this type of problem? Is this not something I should be expecting async/await to solve?

If you call this method from your UI thread, you will overload the capture of UI thread context and back on itself. Also, your service will not be necessarily "Performant" because it must wait until the UI thread is free before it can continue.
The solution is simple: just call the method passing the ConfigureAwait "false" parameter when you made the call.
.ToListAsync().ConfigureAwaiter(false);
I hope it helps

Define Next Start Point When Number of Items Unknown

I have a web service I need to query and it takes a value that supports pagination for its data. Due to the amount of data I need to fetch and how that service is implemented I intended to do a series of concurrent http web requests to accumulate this data.
Say I have number of threads and page size how could I assign each thread to pick its starting point that doesn't overlap with the other thread? Its been a long time since I took parallel programming and I'm floundering a bit. I know I could find my start point with something like start = N/numThreads * threadNum however I don't know N. Right now I just spin up X threads and each loop until they get no more data. Problem is they tend to overlap and I end up with duplicate data. I need unique data and not to waste requests.
Right now I have code that looks something like this. This is one of many attempts and I see why this is wrong but its better to show something. The goal is to in parallel collect pages of data from a webservice:
int limit = pageSize;
data = new List<RequestStuff>();
List<Task> tasks = new List<Task>();
for (int i = 0; i < numThreads; i++)
{
tasks.Add(Task.Factory.StartNew(() =>
{
try
{
List<RequestStuff> someData;
do
{
int start;
lock(myLock)
{
start = data.Count;
}
someKeys = GetDataFromService(start, limit);
lock (myLock)
{
if (someData != null && someData.Count > 0)
{
data.AddRange(someData);
}
}
} while (hasData);
}
catch (AggregateException ex)
{
//Exception things
}
}));
}
Task.WaitAll(tasks.ToArray());
Any inspiration to solve this without race conditions? I need to stick to .NET 4 if that matters.

I'm not sure there's a way to do this without wasting some requests unless you know the actual limit. The code below might help eliminate the duplicate data as you will only query on each index once:
private int _index = -1; // -1 so first request starts at 0
private bool _shouldContinue = true;
public IEnumerable<RequestStuff> GetAllData()
{
var tasks = new List<Task<RequestStuff>>();
while (_shouldContinue)
{
tasks.Add(new Task<RequestStuff>(() => GetDataFromService(GetNextIndex())));
}
Task.WaitAll(tasks.ToArray());
return tasks.Select(t => t.Result).ToList();
}
private RequestStuff GetDataFromService(int id)
{
// Get the data
// If there's no data returned set _shouldContinue to false
// return the RequestStuff;
}
private int GetNextIndex()
{
return Interlocked.Increment(ref _index);
}
It could also be improved by adding cancellation tokens to cancel any indexes you know to be wasteful, i.e, if index 4 returns nothing you can cancel all queries on indexes above 4 that are still active.
Or if you could make a reasonable guess at the max index you might be able to implement an algorithm to pinpoint the exact limit before retrieving any data. This would probably only be more efficient if your guess was fairly accurate though.

Are you attempting to force parallelism on the part of the remote service by issuing multiple concurrent requests? Paging is generally used to limit the amount of data returned to only that which is needed, but if you need all of the data, then attempting to first page and then reconstruct it later seems like a poor design. Your code becomes needlessly complex, difficult to maintain, you'll likely just move the bottleneck from code you control to somewhere else, and now you've introduced data integrity issues (what happens if all of these threads access different versions of the data you are trying to query?). By increasing the complexity and number of calls, you are also increasing the likelihood of problems occurring (eg. one of the connections gets dropped).
Can you state the problem you are attempting to solve so perhaps instead we can help architect a better solution?

Reverse geocoding in my background agent causes my live tile not to update when processing times for rest of code called by agent is fast

I'm having a problem with my Windows Phone 8 weather app's background agent.
Whenever the background agent is run, a new http weather request is made when certain conditions (that are not relevant to the problem I'm having) are met. When these conditions are unmet, cached weather data is used instead.
Furthermore, if you have set your live tile's location to your "Current Location", the background agent will use reverse geocoding to determine the name of the area for the location you're currently at. This is done whether new or cached data is used i.e. every time my app's background agent is run.
The problem I'm having is that whenever cached data is used, the live tile is not updating. But it doesn't appear to cause an exception to occur because the app's background agent never gets blocked, even if there's been more than two times where the live tile fails to update.
This is the relevant excerpt from the background agent's view model's "public async Task getWeatherForTileLocation()" method that's called from the scheduled agent:
Scheduled agent excerpt:
protected async override void OnInvoke(ScheduledTask task)
{
LiveTileViewModel viewModel = new LiveTileViewModel();
await viewModel.getWeatherForTileLocation();
// Etc.
}
getWeatherForTileLocation() excerpt:
// If the default location is 'Current Location', then update its coordinates.
if ((int)IsolatedStorageSettings.ApplicationSettings["LocationDefaultId"] == 1)
{
try
{
// Get new coordinates for current location.
await this.setCoordinates();;
}
catch (Exception e)
{
}
}
// Depending on the time now, since last update (and many other factors),
// must decide whether to use cached data or fresh data
if (this.useCachedData(timeNow, timeLastUpdated))
{
this.ExtractCachedData(); // This method works absolutely fine, trust me. But the live tile never updates when it's run outside debugging.
// Not because of what it does, but because of how fast it executes.
}
else
{
// a httpClient.GetAsync() call is made here that also works fine.
}
The setCoordinates method, as well the reverse geocoding related methods that are called from it:
public async Task<string> setCoordinates()
{
// Need to initialise the tracking mechanism.
Geolocator geolocator = new Geolocator();
// Location services are off.
// Get out - don't do anything.
if (geolocator.LocationStatus == PositionStatus.Disabled)
{
return "gps off";
}
// Location services are on.
// Proceed with obtaining longitude + latitude.
else
{
// Setup the desired accuracy in meters for data returned from the location service.
geolocator.DesiredAccuracyInMeters = 50;
try
{
// Taken from: http://bernhardelbl.wordpress.com/2013/11/26/geolocator-getgeopositionasync-with-correct-timeout/
// Because sometimes GetGeopositionAsync does not return. So you need to add a timeout procedure by your self.
// get the async task
var asyncResult = geolocator.GetGeopositionAsync();
var task = asyncResult.AsTask();
// add a race condition - task vs timeout task
var readyTask = await Task.WhenAny(task, Task.Delay(10000));
if (readyTask != task) // timeout wins
{
return "error";
}
// position found within timeout
Geoposition geoposition = await task;
// Retrieve latitude and longitude.
this._currentLocationLatitude = Convert.ToDouble(geoposition.Coordinate.Latitude.ToString("0.0000000000000"));
this._currentLocationLongitude = Convert.ToDouble(geoposition.Coordinate.Longitude.ToString("0.0000000000000"));
// Reverse geocoding to get your current location's name.
Deployment.Current.Dispatcher.BeginInvoke(() =>
{
this.setCurrentLocationName();
});
return "success";
}
// If there's an error, may be because the ID_CAP_LOCATION in the app manifest wasn't include.
// Alternatively, may be because the user hasn't turned on the Location Services.
catch (Exception ex)
{
if ((uint)ex.HResult == 0x80004004)
{
return "gps off";
}
else
{
// Something else happened during the acquisition of the location.
// Return generic error message.
return "error";
}
}
}
}
/**
* Gets the name of the current location through reverse geocoding.
**/
public void setCurrentLocationName()
{
// Must perform reverse geocoding i.e. get location from latitude/longitude.
ReverseGeocodeQuery query = new ReverseGeocodeQuery()
{
GeoCoordinate = new GeoCoordinate(this._currentLocationLatitude, this._currentLocationLongitude)
};
query.QueryCompleted += query_QueryCompleted;
query.QueryAsync();
}
/**
* Event called when the reverse geocode call returns a location result.
**/
void query_QueryCompleted(object sender, QueryCompletedEventArgs<IList<MapLocation>> e)
{
foreach (var item in e.Result)
{
if (!item.Information.Address.District.Equals(""))
this._currentLocation = item.Information.Address.District;
else
this._currentLocation = item.Information.Address.City;
try
{
IsolatedStorageSettings.ApplicationSettings["LiveTileLocation"] = this._currentLocation;
IsolatedStorageSettings.ApplicationSettings.Save();
break;
}
catch (Exception ee)
{
//Console.WriteLine(ee);
}
}
}
I've debugged the code many times, and have found no problems when I have. The http request when called is good, cached data extraction is good, reverse geocoding does always return a location (eventually).
But I did notice that when I'm using cached data, the name of the current location is retrieved AFTER the scheduled task has created the updated live tile but before the scheduled task has finished.
That is, the name of the location is retrieved after this code in the scheduled agent is run:
extendedData.WideVisualElement = new LiveTileWideFront_Alternative()
{
Icon = viewModel.Location.Hourly.Data[0].Icon,
Temperature = viewModel.Location.Hourly.Data[0].Temperature,
Time = viewModel.Location.Hourly.Data[0].TimeFull.ToUpper(),
Summary = viewModel.Location.Hourly.Data[0].Summary + ". Feels like " + viewModel.Location.Hourly.Data[0].ApparentTemperature + ".",
Location = IsolatedStorageSettings.ApplicationSettings["LiveTileLocation"].ToString().ToUpper(),
PrecipProbability = viewModel.Location.Hourly.Data[0].PrecipProbabilityInt
};
But before:
foreach (ShellTile tile in ShellTile.ActiveTiles)
{
LiveTileHelper.UpdateTile(tile, extendedData);
break;
}
NotifyComplete();
Obviously due to memory constraints I can't create an updated visual element at this point.
For comparison, when I'm not using cached data, the reverse geocoding query always manages to return a location before the http request code has finished.
So as the view model's getWeatherForTileLocation() method is using "await" in the scheduled agent, I decided to make sure that the method doesn't return anything until the current location's name has been retrieved. I added a simple while loop to the method's footer that would only terminate after the _currentLocation field has received a value i.e. the reverse geocoding has completed:
// Keep looping until the reverse geocoding has given your current location a name.
while( this._currentLocation == null )
{
}
// You can exit the method now, as you can create an updated live tile with your current location's name now.
return true;
When I debugged, I think this loop ran around 3 million iterations (a very big number anyway). But this hack (I don't know how else to describe it) seemed to work when I'm debugging. That is, when the target of my build was my Lumia 1020, and when I created a live tile fresh from it, which calls:
ScheduledActionService.Add(periodicTask);
ScheduledActionService.LaunchForTest(periodicTaskName, TimeSpan.FromSeconds(1));
To ensure I don't have to wait for the first scheduled task. When I debugged this first scheduled task, everything works fine: 1) a reverse geocoding request is made, 2) cached data extracted correctly, 3) hacky while loop keeps iterating, 4) stops when the reverse geocoding has returned a location name, 5) tile gets updated successfully.
But subsequent background agent calls that do use cached data don't appear to update the tile. It's only when non-cached data is used that the live tile updates. I should remind you at this point the reverse geocoding query always manages to return a location before the http request code has finished i.e. the hacky loop iterates only once.
Any ideas on what I need to do in order to ensure that the live tile updates correctly when cached data is used (read: when the processing of data, after the reverse geocoding query is made, is much faster than a http request)? Also, is there a more elegant way to stop the getWeatherForTileLocation() from exiting than my while loop? I'm sure there is!
Sorry for the long post but wanted to be as thorough as possible!
This has been giving me sleepless nights (literally) for the last 72 hours, so your help and guidance would be most appreciated.
Many thanks.
Bardi

You have done a great job of providing lots of detail, but it is very disconnected so it is a litle hard to follow. I think the root of your problem is the following:
// Reverse geocoding to get your current location's name.
Deployment.Current.Dispatcher.BeginInvoke(() =>
{
this.setCurrentLocationName();
});
You are attempting to get the location name, but your setCoordinates method will have already completed by the time the setCurrentLocationName method gets around to executing.
Now because you need to be in a UI thread to do any tile updating anyways, I would suggest just dispatching from the begining:
protected async override void OnInvoke(ScheduledTask task)
{
Deployment.Current.Dispatcher.BeginInvoke(() =>
{
LiveTileViewModel viewModel = new LiveTileViewModel();
await viewModel.getWeatherForTileLocation();
}
}
This would remove the need to do any other dispatching in the future.
Two more things:
Generally weather data includes the name of the location you are getting data for. If this is the case, just use that data rather than doing the reverse geocode. This will save you some memory and save time.
If you do need to get the location, I might suggest pulling out a "LocationService" that can get data for you. In this class you can use a TaskCompltionSource to await the event rather than having code follow many different paths.
public class LocationService
{
public static Task<Location> ReverseGeocode(double lat, double lon)
{
TaskCompletionSource<Location> completionSource = new TaskCompletionSource<Location>();
var geocodeQuery = new ReverseGeocodeQuery();
geocodeQuery.GeoCoordinate = new GeoCoordinate(lat, lon);
EventHandler<QueryCompletedEventArgs<IList<MapLocation>>> query = null;
query = (sender, args) =>
{
geocodeQuery.QueryCompleted -= query;
MapLocation mapLocation = args.Result.FirstOrDefault();
var location = Location.FromMapLocation(mapLocation);
completionSource.SetResult(location);
};
geocodeQuery.QueryCompleted += query;
geocodeQuery.QueryAsync();
}
return completionSource.Task;
}
Using a TaskCometionSource allows you to await the method rather than using the event.
var location = await locationService.ReverseGeocode(lat, lon);
This example uses another Location class that I created do just hold things like City and State.
The key thing with background agents is to ensure that code always flows "synchronously". This doesn't mean code cannot be asynchronous, but does mean that code needs to be called one after the other. So if you have something that has events, you could continue all other code after the event.
Hope that helps!

I don't see your deferral call. When you use async you have to tell the task that you're deferring completion till later. Can't remember the method off the top of my head but it's either on the base class of your background task or on the parameter you get.
The reason it probably works with cache data is that it probably isn't actually an async operation.

I think this is sorted now! Thanks so much Shawn for the help. The setLocationName() method call is now awaited, and it looks like this now:
public Task<string> setLocationName()
{
var reverseGeocode = new ReverseGeocodeQuery();
reverseGeocode.GeoCoordinate = new System.Device.Location.GeoCoordinate(this._currentLocationLatitude, this._currentLocationLongitude );
var tcs = new TaskCompletionSource<string>();
EventHandler<QueryCompletedEventArgs<System.Collections.Generic.IList<MapLocation>>> handler = null;
handler = (sender, args) =>
{
MapLocation mapLocation = args.Result.FirstOrDefault();
string l;
if (!mapLocation.Information.Address.District.Equals(""))
l = mapLocation.Information.Address.District;
else
l = mapLocation.Information.Address.City;
try
{
System.DateTime t = System.DateTime.UtcNow.AddHours(1.0);
if (t.Minute < 10)
IsolatedStorageSettings.ApplicationSettings["LiveTileLocation"] = l + " " + t.Hour + ":0" + t.Minute;
else
IsolatedStorageSettings.ApplicationSettings["LiveTileLocation"] = l + " " + t.Hour + ":" + t.Minute;
IsolatedStorageSettings.ApplicationSettings.Save();
this._currentLocationName = IsolatedStorageSettings.ApplicationSettings["LiveTileLocation"].ToString();
}
catch (Exception ee)
{
//Console.WriteLine(ee);
}
reverseGeocode.QueryCompleted -= handler;
tcs.SetResult(l);
};
reverseGeocode.QueryCompleted += handler;
reverseGeocode.QueryAsync();
return tcs.Task;
}

Stack overflow error in console application

I currently have a console application written in C# for production server environment. It's quite simple, but I've been testing it for +- 3 months now on my own server (semi-production, but not a disaster if the program fails). I've been getting stack overflow errors every few weeks or so, though.
Of course, pasting my entire source code here would be quite a long piece, so I will try to explain it the best I can: the program is in an infinite while loop (this is probably the cause of the issue), and it checks every second a few small things and prints to the console every so often (if no activity, every 15min, otherwise up-to every second).
Now why and how can I fix the stack overflow errors? Being able to run it for a couple weeks without issue may seem like a lot, but it is obviously not in a server production environment. My guess is that it's the fact I'm using a while loop, but what alternatives do I have?
Edit: here's a portion of the code:
int timeLoop = 0;
while (true)
{
// If it has been +-10min since last read of messages file, read again
if (timeLoop > 599)
{
Console.WriteLine(DateTime.Now.ToString("HH:mm:ss") + " Reading messages...");
messagesFile = File.ReadAllText(#"messages.cfg");
messages = messagesFile.Split(Environment.NewLine.ToCharArray(), StringSplitOptions.RemoveEmptyEntries);
timeLoop = 0;
}
// For each message, check if time specified equals current time, if so say message globally.
foreach (string x in messages)
{
string[] messageThis = x.Split('~');
int typeLoop = 0;
if (messageThis[2] != "no-loop")
typeLoop = Convert.ToInt16(messageThis[2].Remove(0, 5));
DateTime checkDateTime = DateTime.ParseExact(messageThis[0], "HH:mm:ss", null);
if (typeLoop == 0)
{
if (checkDateTime.ToString("HH:mm:ss") == DateTime.Now.ToString("HH:mm:ss"))
publicMethods.sayGlobal(messageThis[1]);
}
else
{
DateTime originalDateTime = checkDateTime;
do
{
checkDateTime = checkDateTime.AddHours(typeLoop);
if (checkDateTime.ToString("HH:mm:ss") == DateTime.Now.ToString("HH:mm:ss"))
publicMethods.sayGlobal(messageThis[1]);
} while (checkDateTime.ToString("HH:mm:ss") != originalDateTime.ToString("HH:mm:ss"));
}
}
timeLoop++;
Thread.Sleep(1000);
}
Also, I forgot that I actually have this code on Github, which probably helps a lot. I know you shouldn't be using any links to code and have them here, so that's why I included the snippet above - but if you are interested in helping me out the repository is located here. Another note - I know that doing it like this is not very accurate on timing - but this is not much of an issue at the moment.

BattleEyeClient.Connect can call itself in some (failing) circumstances, and this method can be called by sayGlobal - so you probably want to change this code block (from line 98):
catch
{
if (disconnectionType == BattlEyeDisconnectionType.ConnectionLost)
{
Disconnect(BattlEyeDisconnectionType.ConnectionLost);
Connect();
return BattlEyeConnectionResult.ConnectionFailed;
}
else
{
OnConnect(loginCredentials, BattlEyeConnectionResult.ConnectionFailed);
return BattlEyeConnectionResult.ConnectionFailed;
}
}
Maybe keep track of how many reconnection attempts you make or transform this section of code so that it is a while loop rather than a recursive call during this failure mode.
Even worse, of course, is if that recursive Connect call succeeds, this catch block then returns BattlEyeConnectionResult.ConnectionFailed which it probably shouldn't.

Why does parallel this code work sometimes?

I wanted to parallelize a piece of code, but the code actually got slower probably because of overhead of Barrier and BlockCollection. There would be 2 threads, where the first would find pieces of work wich the second one would operate on. Both operations are not much work so the overhead of switching safely would quickly outweigh the two threads.
So I thought I would try to write some code myself to be as lean as possible, without using Barrier etc. It does not behave consistent however. Sometimes it works, sometimes it does not and I can't figure out why.
This code is just the mechanism I use to try to synchronize the two threads. It doesn't do anything useful, just the minimum amount of code you need to reproduce the bug.
So here's the code:
// node in linkedlist of work elements
class WorkItem {
public int Value;
public WorkItem Next;
}
static void Test() {
WorkItem fst = null; // first element
Action create = () => {
WorkItem cur=null;
for (int i = 0; i < 1000; i++) {
WorkItem tmp = new WorkItem { Value = i }; // create new comm class
if (fst == null) fst = tmp; // if it's the first add it there
else cur.Next = tmp; // else add to back of list
cur = tmp; // this is the current one
}
cur.Next = new WorkItem { Value = -1 }; // -1 means stop element
#if VERBOSE
Console.WriteLine("Create is done");
#endif
};
Action consume = () => {
//Thread.Sleep(1); // this also seems to cure it
#if VERBOSE
Console.WriteLine("Consume starts"); // especially this one seems to matter
#endif
WorkItem cur = null;
int tot = 0;
while (fst == null) { } // busy wait for first one
cur = fst;
#if VERBOSE
Console.WriteLine("Consume found first");
#endif
while (true) {
if (cur.Value == -1) break; // if stop element break;
tot += cur.Value;
while (cur.Next == null) { } // busy wait for next to be set
cur = cur.Next; // move to next
}
Console.WriteLine(tot);
};
try { Parallel.Invoke(create, consume); }
catch (AggregateException e) {
Console.WriteLine(e.Message);
foreach (var ie in e.InnerExceptions) Console.WriteLine(ie.Message);
}
Console.WriteLine("Consume done..");
Console.ReadKey();
}
The idea is to have a Linkedlist of workitems. One thread adds items to the back of that list, and another thread reads them, does something, and polls the Next field to see if it is set. As soon as it is set it will move to the new one and process it. It polls the Next field in a tight busy loop because it should be set very quickly. Going to sleep, context switching etc would kill the benefit of parallizing the code.
The time it takes to create a workitem would be quite comparable to executing it, so the cycles wasted should be quite small.
When I run the code in release mode, sometimes it works, sometimes it does nothing. The problem seems to be in the 'Consumer' thread, the 'Create' thread always seems to finish. (You can check by fiddling with the Console.WriteLines).
It has always worked in debug mode. In release it about 50% hit and miss. Adding a few Console.Writelines helps the succes ratio, but even then it's not 100%. (the #define VERBOSE stuff).
When I add the Thread.Sleep(1) in the 'Consumer' thread it also seems to fix it. But not being able to reproduce a bug is not the same thing as knowing for sure it's fixed.
Does anyone here have a clue as to what goes wrong here? Is it some optimization that creates a local copy or something that does not get updated? Something like that?
There's no such thing as a partial update right? like a datarace, but then that one thread is half doen writing and the other thread reads the partially written memory? Just checking..
Looking at it I think it should just work.. I guess once every few times the threads arrive in different order and that makes it fail, but I don't get how. And how I could fix this without adding slowing it down?
Thanks in advance for any tips,
Gert-Jan

I do my damn best to avoid the utter minefield of closure/stack interaction at all costs.
This is PROBABLY a (language-level) race condition, but without reflecting Parallel.Invoke i can't be sure. Basically, sometimes fst is being changed by create() and sometimes not. Ideally, it should NEVER be changed (if c# had good closure behaviour). It could be due to which thread Parallel.Invoke chooses to run create() and consume() on. If create() runs on the main thread, it might change fst before consume() takes a copy of it. Or create() might be running on a separate thread and taking a copy of fst. Basically, as much as i love c#, it is an utter pain in this regard, so just work around it and treat all variables involved in a closure as immutable.
To get it working:
//Replace
WorkItem fst = null
//with
WorkItem fst = WorkItem.GetSpecialBlankFirstItem();
//And
if (fst == null) fst = tmp;
//with
if (fst.Next == null) fst.Next = tmp;

A thread is allowed by the spec to cache a value indefinitely.
see Can a C# thread really cache a value and ignore changes to that value on other threads? and also http://www.yoda.arachsys.com/csharp/threads/volatility.shtml