Develop Reference

Complex LINQ query vs Complex for loops

Right now I have this complex function, I have a list of MainObjects(call them MO) and for each of these objects I have to loop over a list of objects(call them C) with a title, a status and a list of sub-objects (call them E). The function loops over these sub-objects(E) and uses it's title and quantity properties.
The goal of the function is to create a dictionary(D1), where the Key is a C(title) and the Values is another dictionary(D2), where the Key is E(title) and the Values yet another dictionary(D3), where the Key is C(status) and the value E(quantity).
So in the end I will have all (unique) C(title)'s wherein I can see all (unique) E(title)'s wherein I can see all different C(status)'s and the E(quantity) of these statuses (with as extra challenge if 2 E(quantity)'s would have the same status with the same title on the same course they should be added to each other and then put in as value).
I made this all work fine.
However. The function is big and hard to understand, so I'm looking for a more approachable way of dealing with this problem.
One of these ways was supposed to be LINQ. However, I have little to no knowledge about this and for a massively complex function as this I can hardly understand how to deal with this in LINQ.
I'm also concerned about performance since this WPF project is heavily dependable on user-experience. So I'm not sure if LINQ would actually make things faster, slower or same.
Here is where you guys come in.
Is LINQ a better way to deal with this problem?
Is the performance similar to the one of my function?
Is the LINQ query more understandable?
Is there an alternative way of dealing with this complex function
rather then the 2 methods I'm describing?
Underneath you will find the function I used to deal with this function my way.
It is done in 3 steps:
Step1: Loop the MO's, C's, E's and create a list of dictionaries.
Step2: Join the duplicate key's of the result of step1 and create a
first stage dictionary.
Step3: Split the deeper dictionaries so that
we can use the E object as intended.
Result: has been put in the 'final' object. A list of dictionaries with as keys C(title) and values a list of dictionaries. This list of dictionaries with as keys E(title) and values a Dictionary. This Dictionary has as keys C(status) and values E(quantity). This E(quantity) is a combined value of each quantity of each E of the same C(status) for a same C.
//DateTime start = DateTime.Now; //start performance test
//start -> step 1
List<Dictionary<string/*C(title)*/, Dictionary<int/*C(status)*/, List<E>>>> firstResultList = new List<Dictionary<string, Dictionary<int, List<E>>>>();
foreach(MO mo in listOfMOs)
{
foreach (C c in mo.listOfCs)
{
Dictionary<string, Dictionary<int, List<E>>> D1 = new Dictionary<string, Dictionary<int, List<E>>>();
int cStatus = c.status;
Dictionary<int, List<E>> D2 = new Dictionary<int, List<E>>();
List<E> eList = new List<E>();
foreach (E e in c.listOfEs)
{
eList.Add(e);
}
D2.Add(cStatus, eList);
D1.Add(c.Title, D2);
firstResultList.Add(D1);
}
}
//firstResultList = step1 results
//Console.WriteLine(firstResultList.ToString());
//
//step1 -> step2
Dictionary<string/*C(title)*/, List<Dictionary<int/*C(status)*/, List<E>>>> groupedDict = new Dictionary<string, List<Dictionary<int, List<E>>>>();
foreach (Dictionary<string, Dictionary<int, List<E>>> dict in firstResultList)
{
List<Dictionary<int, List<E>>> listje;
if(groupedDict.ContainsKey(dict.Keys.ElementAt(0)))
{
listje = groupedDict[dict.Keys.ElementAt(0)];
}
else
{
listje = new List<Dictionary<int, List<E>>>();
}
listje.Add(dict[dict.Keys.ElementAt(0)]);
groupedDict[dict.Keys.ElementAt(0)] = listje;
}
//groupedDict = step2 results
//Console.WriteLine(groupedDict.ToString());
//
//step2 -> step3
Dictionary<string/*C(title)*/, List<Dictionary<string/*E(title)*/, Dictionary<int/*C(status)*/, int/*E(quantity)*/>>>> final = new Dictionary<string, List<Dictionary<string, Dictionary<int, int>>>>();
int index = 0;
foreach (List<Dictionary<int, List<E>>> list in groupedDict.Values)
{
//Within one unique C
List<Dictionary<string, Dictionary<int, int>>> eStatusQuantityList = new List<Dictionary<string, Dictionary<int, int>>>();
foreach (Dictionary<int, List<E>> dict in list)
{
foreach (List<E> eList in dict.Values)
{
foreach(E e in eList)
{
if (eStatusQuantityList.Count > 0)
{
foreach (Dictionary<string, Dictionary<int, int>> dict2 in eStatusQuantityList)
{
Dictionary<int, int> statusQuantityDict;
if (dict2.ContainsKey(e.Title))
{
statusQuantityDict = dict2[e.Title];
//int quantity = statusQuantityDict.value//statusQuantityDict[dict.Keys.ElementAt(0)];
int quantity = 0;
int value;
bool hasValue = statusQuantityDict.TryGetValue(dict.Keys.ElementAt(0), out value);
if (hasValue) {
quantity = value;
} else {
// do something when the value is not there
}
statusQuantityDict[dict.Keys.ElementAt(0)] = quantity + e.Quantity;
dict2[e.Title] = statusQuantityDict;
}
else
{
statusQuantityDict = new Dictionary<int, int>();
statusQuantityDict.Add(dict.Keys.ElementAt(0), e.Quantity);
dict2.Add(e.Title, statusQuantityDict);
}
}
}
else
{
Dictionary<string, Dictionary<int, int>> test = new Dictionary<string, Dictionary<int, int>>();
Dictionary<int, int> test2 = new Dictionary<int, int>();
test2.Add(dict.Keys.ElementAt(0), e.Quantity);
test.Add(e.Title, test2);
eStatusQuantityList.Add(test);
}
}
}
}
//ending
string key = groupedDict.Keys.ElementAt(index);
final[key] = eStatusQuantityList;
index++;
//
}
//final contains step3 results
//Console.WriteLine(final.ToString());
/*
for (int i = 0; i<final.Keys.Count; i++)
{
Console.WriteLine(final.Keys.ElementAt(i));
}
for (int i = 0; i < final.Values.Count; i++)
{
Console.WriteLine(final.Values.ElementAt(i));
}
*/
//
//TimeSpan duration = DateTime.Now - start; //end performance test
//Console.WriteLine("That took " + duration.TotalMilliseconds + " ms"); //performance test results //60.006 is fine, 600.006 is OOM. //Our range of objects is max. 300 MO's though
As you can see this is a hell of a function. But it works fine (2-5ms (avg. 2.5) for our max target of MO's). But I can see people (other then myself) messing up when they have to readjust this function for some reason. So any improvement in maintainability or readability would be cool.

Is LINQ a better way to deal with this problem?
Better is subjective. Better looking? Better performance? Better (as in easier) understanding?
Is the performance similar to the one of my function?
LINQ performance is usually not quite as good as doing it manually, however there is always a trade off because LINQ can be (not always) easier to understand.
Is the LINQ query more understandable?
It can be. But if you've ever used reSharper where it looks at your code and says it can turn it into a LINQ query then you'll know that sometimes it makes it less understandable.
Is there an alternative way of dealing with this complex function
rather then the 2 methods I'm describing?
Mix-n-match? You can hand-code performance critical parts and leave the rest in LINQ. But to find the performance critical parts you should use a profiler rather than just guessing.

Using the Concurrent Dictionary - Thread Safe Collection Modification

Recently I was running into the following exception when using a generic dictionary
An InvalidOperationException has occurred. A collection was modified
I realized that this error was primarily because of thread safety issues on the static dictionary I was using.
A little background: I currently have an application which has 3 different methods that are related to this issue.
Method A iterates through the dictionary using foreach and returns a value.
Method B adds data to the dictionary.
Method C changes the value of the key in the dictionary.
Sometimes while iterating through the dictionary, data is also being added, which is the cause of this issue. I keep getting this exception in the foreach part of my code where I iterate over the contents of the dictionary. In order to resolve this issue, I replaced the generic dictionary with the ConcurrentDictionary and here are the details of what I did.
Aim : My main objective is to completely remove the exception
For method B (which adds a new key to the dictionary) I replaced .Add with TryAdd
For method C (which updates the value of the dictionary) I did not make any changes. A rough sketch of the code is as follows :
static public int ChangeContent(int para)
{
foreach (KeyValuePair<string, CustObject> pair in static_container)
{
if (pair.Value.propA != para ) //Pending cancel
{
pair.Value.data_id = prim_id; //I am updating the content
return 0;
}
}
return -2;
}
For method A - I am simply iterating over the dictionary and this is where the running code stops (in debug mode) and Visual Studio informs me that this is where the error occured.The code I am using is similar to the following
static public CustObject RetrieveOrderDetails(int para)
{
foreach (KeyValuePair<string, CustObject> pair in static_container)
{
if (pair.Value.cust_id.Equals(symbol))
{
if (pair.Value.OrderStatus != para)
{
return pair.Value; //Found
}
}
}
return null; //Not found
}
Are these changes going to resolve the exception that I am getting.
Edit:
It states on this page that the method GetEnumerator allows you to traverse through the elements in parallel with writes (although it may be outdated). Isnt that the same as using foreach ?

For modification of elements, one option is to manually iterate the dictionary using a for loop, e.g.:
Dictionary<string, string> test = new Dictionary<string, string>();
int dictionaryLength = test.Count();
for (int i = 0; i < dictionaryLength; i++)
{
test[test.ElementAt(i).Key] = "Some new content";
}
Be weary though, that if you're also adding to the Dictionary, you must increment dictionaryLength (or decrement it if you move elements) appropriately.
Depending on what exactly you're doing, and if order matters, you may wish to use a SortedDictionary instead.
You could extend this by updating dictionaryLength explicitly by recalling test.Count() at each iteration, and also use an additional list containing a list of keys you've already modified and so on and so forth if there's a danger of missing any, it really depends what you're doing as much as anything and what your needs are.
You can further get a list of keys using test.Keys.ToList(), that option would work as follows:
Dictionary<string, string> test = new Dictionary<string, string>();
List<string> keys = test.Keys.ToList();
foreach (string key in keys)
{
test[key] = "Some new content";
}
IEnumerable<string> newKeys = test.Keys.ToList().Except(keys);
if(newKeys.Count() > 0)
// Do it again or whatever.
Note that I've also shown an example of how to find out whether any new keys were added between you getting the initial list of keys, and completing iteration such that you could then loop round and handle the new keys.
Hopefully one of these options will suit (or you may even want to mix and match- for loop on the keys for example updating that as you go instead of the length) - as I say, it's as much about what precisely you're trying to do as much as anything.

Before doing foreach() try out copying container to a new instance
var unboundContainer = static_container.ToList();
foreach (KeyValuePair<string, CustObject> pair in unboundContainer)
Also I think updating Value property is not right from thread safety perspectives, refactor your code to use TryUpdate() instead.

Most efficient way to compare lots variables? C#

I am getting a datatable with customer data from a MySql databaes and a customer object from a web service.
I want to compare every value in the datatable with the values in the object and if there is one field that differs I want to perfrom some tasks.
I know I can get the values from the datatable with:
string mCompanyName = row["Company Name"].ToString();
string mCreatedDate = row["Created Date"].Tostring();
//etc..
Then I get the values from the web service
string wsCompanyName = customer.companyName;
string wsCreatedDate = customer.createdDate;
There are about 50 fields and doing
if( mCompanyName != wsCompanyName & mCreatedDate != wsCreatedDate and so on..) (or similar)
{
//Do something
}
seems to be a bit tedious and not very nice so how should I perform this? Is there a much better way to chuck it into a list and use some fancy LINQ?
Thanks in advance.

For cases like this I sometimes put them ("the objects") in something IEnumerable (make sure to "line them up") and use the SequenceEqual extension method. It performs standard Equals()'ity and is "cheap enough for my usage".
For instance:
var equal = (new object[] { row["A"], row["B"] })
.SequenceEqual(new object[] { x.A, x.B });
This requires LINQ, of course.

I'd put them in a Dictionary and search that way:
Dictionary<string, string> mData = new Dictionary<string, string>();
mData.Add("Company Name", row["Company Name"].ToString());
Dictionary<string, string> wsData = new Dictionary<string, string>();
wsData.Add("Company Name", customer.CompanyName);
Then loop through:
foreach (KeyValuePair<string, string> pair in mData)
{
if (wsData[pair.Key] == pair.Value)
{
// Do something
}
}
This way, for every entry in mData (the data from your database), it will look for an entry in wsData with the same name.
I wouldn't create individual variables for each piece of data. It would be difficult to maintain, and would not scale well (lots of copy and pastes).

I think this might help you, but it needs to modify this to use in your scenerio stackoverflow link

C# - Removing Items from Dictionary in while loop

I have this and all seems to work fine but not sure why and if its valid.
Dictionary<string, List<string>> test = new Dictionary<string, List<string>>();
while (test.Count > 0)
{
var obj = test.Last();
MyMethod(obj);
test.Remove(obj.Key);
}
Update: Thanks for the answers, I have updated my code to explain why I don't do Dictionary.Clear();

I don't understand why you are trying to process all Dictonary entries in reverse order - but your code is OK.
It might be a bit faster to get a list of all Keys and process the entries by key instead of counting again and again...
E.G.:
var keys = test.Keys.OrderByDescending(o => o).ToList();
foreach (var key in keys)
{
var obj = test[key];
MyMethod(obj);
test.Remove(key);
}
Dictonarys are fast when they are accessed by their key value. Last() is slower and counting is not necessary - you can get a list of all (unique) keys.

There is nothing wrong with mutating a collection type in a while loop in this manner. Where you get into trouble is when you mutate a collection during a foreach block. Or more generally use a IEnumerator<T> after the underlying collection is mutated.
Although in this sample it would be a lot simpler to just call test.Clear() :)

That works, fine, since you're not iterating over the dictionary while removing items. Each time you check test.Count, it's like it's checking it from scratch.
That being said, the above code could be written much simpler and more effectively:
test.Clear();

It works because Count will be updated every time you remove an object. So say count is 3, test.Remove will decriment the count to 2, and so on, until the count is 0, then you will break out of the loop

Yes, this should be valid, but why not just call Dictionary.Clear()?

All you're doing is taking the last item in the collection and removing it until there are no more items left in the Dictionary.
Nothing out of the ordinary and there's no reason it shouldn't work (as long as emptying the collection is what you want to do).

So, you're just trying to clear the Dictionary, correct? Couldn't you just do the following?
Dictionary<string, List<string>> test = new Dictionary<string, List<string>>();
test.Clear();

This seems like it will work, but it looks extremely expensive. This would be a problem if you were iterating over it with a foreach loop (you can't edit collections while your iterating).
Dictionary.Clear() should do the trick (but you probably already knew that).

Despite your update, you can probably still use clear...
foreach(var item in test) {
MyMethod(item);
}
test.Clear()
Your call to .Last() is going to be extremely inefficient on a large dictionary, and won't guarantee any particular ordering of the processing regardless (the Dictionary is an unordered collection)

I used this code to remove items conditionally.
var dict = new Dictionary<String, float>
var keys = new String[dict.Count];
dict.Keys.CopyTo(keys, 0);
foreach (var key in keys) {
var v = dict[key];
if (condition) {
dict.Remove(key);
}

loop one sequence and check againt other list

i have a namespace string like "Company.Product.Sub1.Sub2.IService".
The Sub1/Sub2 can differ in their count, but normally their is one part which matches to
a Dictionary with AssemblyFullname as key and path to it as value.
Now ive written this code
string fullName = interfaceCodeElement.FullName;
var fullNameParts = interfaceCodeElement.FullName.Split('.').Reverse();
KeyValuePair<string, string> type = new KeyValuePair<string,string>();
foreach (var item in fullNameParts)
{
var match = references.Where(x => x.Key.Contains(item)).ToList();
if (match.Count > 0)
{
type = match[0];
break;
}
}
Works but doesnt look nice in my opinion.
I tried it with linq but i dont know how ive to write it.
var matches = from reference in refs
where reference.Key.Contains(fullNameParts.Reverse().
Thanks for help.

To first put it into English, you're trying to go through the parts (backwards) of the Fullname in interfaceCodeElement and find the first that matches (as a substring, case-sensitive) any of the keys in references (which is a Dictionary<string, string> from fullname to path). Your result, type, is a KeyValuePair<string, string> although it's not clear if you actually need that (both the key and value) or just one or the other.
One, it seems a little odd to have a Dictionary in such a case, since you're not able to lookup as a key, but I guess it still works for the purpose :) Switching to something like List<Tuple<string, string>> or List<KeyValuePair<string, string>> might make sense, as the order of the pairs that comes from iteration over references will potentially affect which pair is selected into type.
In order to try to make it easier to understand, I'll add a let here:
var bestMatchPerPart = from part in fullNameParts
let firstMatchPair = references.FirstOrDefault(pair => pair.Key.Contains(part))
where firstMatchPair != null // ignore parts that have no match
// since we want the pair, not the part, select that
select firstMatchPair;
var type = bestMatchPerPart.FirstOrDefault()
// to match original behavior, empty pair in result instead of null if no match
?? new KeyValuePair<string, string>();

This should give you a list of the matches:
var listOfMatches = fullNameParts.Where(fp => references.Where(r => r.Key.Contains(fp))).ToList();
Edit: So based on your comments I think I kind of understand. Assuming you have some list of these fullNames somewhere:
// Making this up because I am nor sure what you have to start with
List<string> yourListOfAllYourFullNames = GetThisList();
var listOfMatches = yourListOfAllYourFullNames.Where(
fnl => fnl.Split('.').Reverse().Where(
fnp => references.Where(r => r.Key.Contains(fnp))).Count() > 0).ToList();