I have a dictionary which has an integer Key that represents a year, and a Value which is a list of object Channel. I need to flatten the data and create a new object from it.
Currently, my code looks like this:
Dictionary<int, List<Channel>> myDictionary;
foreach(var x in myDictionary)
{
var result = (from a in x.Value
from b in anotherList
where a.ChannelId == b.ChannelId
select new NewObject
{
NewObjectYear = x.Key,
NewObjectName = a.First().ChannelName,
}).ToList();
list.AddRange(result);
}
Notice that I am using the Key to be the value of property NewObjectYear.
I want to get rid of foreach since the dictionary contains a lot of data and doing some joins inside the iteration makes it very slow. So I decided to refactor and came up with this:
var flatten = myDictionary.SelectMany(x => x.Value.Select(y =>
new KeyValuePair<int, Channel>(x.Key, y))).ToList();
But with this, I couldn't get the Key directly. Using something like flatten.Select(x => x.Key) is definitely not the correct way. So I tried finding other ways to flatten that would be favorable for my scenario but failed. I also thought about creating a class which will contain the year and the list from the flattened but I don't know how.
Please help me with this.
Also, is there also another way that doesn't have the need to create a new class?
It seems to me you are trying to do only filtering, you do not need join for that:
var anotherListIDs = new HashSet<int>(anotherList.Select(c => c.ChannelId));
foreach (var x in myDictionary)
{
list.AddRange(x.Value
.Where(c => anotherListIDs.Contains(c.ChannelId))
.Select(c => new NewObject
{
NewObjectYear = x.Key,
NewObjectName = c.First().ChannelName,
}));
}
You do realise, that if the second element of the list in a specific dictionary element has a matching channelId, that you return the first element of this list, don't you?
var otherList = new OtherItem[]
{
new OtherItem() {ChannelId = 1, ...}
}
var dictionary = new Dictionary<int, List<Channel>[]
{
{ 10, // Key
new List<Channel>() // Value
{
new Channel() {ChannelId = 100, Name = "100"},
new Channel() {ChannelId = 1, Name = "1"},
},
};
Although the 2nd element has a matching ChannelId, you return the Name of the first element.
Anyway, let's assume this is what you really want. You are right, your function isn't very efficient.
Your dictionary implements IEnumerable<KeyValuePair<int, List<Channel>>. Therefore every x in your foreach is a KeyValuePair<int, List<Channel>. Every x.Value is a List<Channel>.
So for every element in your dictionary (which is a KeyValuePair<int, List<Channel>), you take the complete list, and perform a full inner join of the complete list with otherList, and for the result you take the key of the KeyValuePair and the first element of the List in the KeyValuePair.
And even though you might not use the complete result, but only the first or the first few, because of FirstOrDefault(), or Take(3), you do this for every element of every list in your Dictionary.
Indeed your query could be much more efficient.
As you use the ChannelIds in your OtherList only to find out if it is present, one of the major improvements would be to convert the ChannelIds of OtherList to a HashSet<int> where you have superior fast lookup to check if the ChannelId of one of the values in your Dictionary is in the HashSet.
So for every element in your dictionary, you only have to check every ChannelId in the list to see if one of them is in the HashSet. As soon as you've found one, you can stop and return only the first element of the List and the Key.
My solution is an extension function of Dictionary>. See Extension Methods Demystified
public static IEnumerable<NewObject> ExtractNewObjects(this Dictionary<int, List<Channel>> dictionary,
IEnumerable<OtherItem> otherList)
{
// I'll only use the ChannelIds of the otherList, so extract them
IEnumerable<int> otherChannelIds = otherList
.Select(otherItem => otherItem.ChannelId);
return dictionary.ExtractNewObjects(otherChannelIds);
}
This calls the other ExtractNewobjects:
public static IEnumerable<NewObject> ExtractNewObjects(this Dictionary<int, List<Channel>> dictionary,
IEnumerable<int> otherChannelIds)
{
var channelIdsSet = new HashSet<int>(otherChannelIds));
// duplicate channelIds will be removed automatically
foreach (KeyValuePair<int, List<Channel>> keyValuePair in dictionary)
{
// is any ChannelId in the list also in otherChannelIdsSet?
// every keyValuePair.Value is a List<Channel>
// every Channel has a ChannelId
// channelId found if any of these ChannelIds in in the HashSet
bool channelIdFound = keyValuePair.Value
.Any(channel => otherChannelIdsSet.Contains(channel.ChannelId);
if (channelIdFound)
{
yield return new NewObject()
{
NewObjectYear = keyValuePair.Key,
NewObjectName = keyValuePair.Value
.Select(channel => channel.ChannelName)
.FirstOrDefault(),
};
}
}
}
usage:
IEnumerable<OtherItem> otherList = ...
Dictionary<int, List<Channel>> dictionary = ...
IEnumerable<Newobject> extractedNewObjects = dictionary.ExtractNewObjects(otherList);
var someNewObjects = extractedNewObjects
.Take(5) // here we see the benefit from the yield return
.ToList();
We can see four efficiency improvements:
the use of HashSet<int> enables a very fast lookup to see if the ChannelId is in OtherList
the use of Any() stops enumerating the List<Channel> as soon as we've found a matching Channelid in the HashSet
the use of yield return makes that you don't enumerate over more elements in your Dictionary than you'll actually use.
The use of Select and FirstOrDefault when creating NewObjectName prevents exceptions if List<Channel> is empty
Related
I have a List<Map> and I wanted to update the Map.Target property based from a matching value from another List<Map>.
Basically, the logic is:
If mapsList1.Name is equal to mapsList2.Name
Then mapsList1.Target = mapsList2.Name
The structure of the Map class looks like this:
public class Map {
public Guid Id { get; set; }
public string Name { get; set; }
public string Target { get; set; }
}
I tried the following but obviously it's not working:
List<Map> mapsList1 = new List<Map>();
List<Map> mapsList2 = new List<Map>();
// populate the 2 lists here
mapsList1.Where(m1 => mapsList2.Where(m2 => m1.Name == m2.Name) ) // don't know what to do next
The count of items in list 1 will be always greater than or equal to the count of items in list 2. No duplicates in both lists.
Assuming there are a small number of items in the lists and only one item in list 1 that matches:
list2.ForEach(l2m => list1.First(l1m => l1m.Name == l2m.Name).Target = l2m.Target);
If there are more than one item in List1 that must be updated, enumerate the entire list1 doing a First on list2.
list1.ForEach(l1m => l1m.Target = list2.FirstOrDefault(l2m => l1.Name == l2m.Name)?.Target ?? l1m.Target);
If there are a large number of items in list2, turn it into a dictionary
var d = list2.ToDictionary(m => m.Name);
list1.ForEach(m => m.Target = d.ContainsKey(m.Name) ? d[m.Name].Target : m.Target);
(Presumably list2 doesn't contain any repeated names)
If list1's names are unique and everything in list2 is in list1, you could even turn list1 into a dictionary and enumerate list2:
var d=list1.ToDictionary(m => m.Name);
list2.ForEach(m => d[m.Name].Target = m.Target);
If List 2 has entries that are not in list1 or list1 has duplicate names, you could use a Lookup instead, you'd just have to do something to avoid a "collection was modified; enumeration may not execute" you'd get if you were trying to modify the list it returns in response to a name
mapsList1.Where(m1 => mapsList2.Where(m2 => m1.Name == m2.Name) ) // don't know what to do next
LINQ Where doesn't really work like that / that's not a statement in itself. The m1 is the entry from list1, and the inner Where would produce an enumerable of list 2 items, but it doesn't result in the Boolean the outer Where is expecting, nor can you do anything to either of the sequences because LINQ operations are not supposed to have side effects. The only thing you can do with a Where is capture or use the sequence it returns in some other operation (like enumerating it), so Where isn't really something you'd use for this operation unless you use it to find all the objects you need to alter. It's probably worth pointing out that ForEach is a list thing, not a LINQ thing, and is basically just another way of writing foreach(var item in someList)
If collections are big enough better approach would be to create a dictionary to lookup the targets:
List<Map> mapsList1 = new List<Map>();
List<Map> mapsList2 = new List<Map>();
var dict = mapsList2
.GroupBy(map => map.Name)
.ToDictionary(maps => maps.Key, maps => maps.First().Target);
foreach (var map in mapsList1)
{
if (dict.TryGetValue(map.Name, out var target))
{
map.Target = target;
}
}
Note, that this will discard any possible name duplicates from mapsList2.
So I have a collection of objects who have multiple properties, two of these are groupname and personname. Now I need to count in the collection how many of each object belong to a certain group and person. So in other words, I need to group by groupname, then personname and then count how many objects have this combination. First I created this
public MultiKeyDictionary<string, string, int> GetPersonsPerGroup(IEnumerable<Home> homes ,List<string> gr, List<string> na)
{
List<string> groups = gr;
groups.Add("");
List<string> names = na;
names.Add("");
List<Home> Filtered = homes.ToList();
Filtered.ForEach(h => h.RemoveNull());
var result = new MultiKeyDictionary<string, string, int>();
int counter1 = 0;
foreach (var g in groups)
{
int counter2 = 0;
foreach (var n in names)
{
int counter3 = 0;
foreach (Home h in Filtered)
{
if (h.GroupName == g && h.PersonName == n)
{
counter3++;
if (counter3 > 100)
break;
}
}
if (counter3 > 0)
{
result.Add(g,n,counter3);
}
counter2++;
}
counter1++;
}
Which may look good, but the problem is that the "home" parameter can contain more than 10000 objects, with more than 1500 unique names and around 200 unique groups. Which causes this to iterate like a billion times really slowing my program down. So I need an other way of handling this. Which made me decide to try using linq. Which led to this creation:
var newList = Filtered.GroupBy(x => new { x.GroupName, x.PersonName })
.Select(y => (MultiKeyDictionary<string, string, int>)result.Add(y.Key.GroupName, y.Key.PersonName, y.ToList().Count));
Which gives an error "Cannot convert type 'void' to 'MultiKeyDictionary<string,string,int>' and I have no idea how to solve it. How can I make it so that the result of this query gets stored all in one MultikeyDictionary without having to iterate over each possible combination and counting all of them.
Some information:
MultiKeyDictionary is a class I defined (something I found on here actually), it's just a normal dictionary but with two keys assosiated to one value.
The RemoveNull() method on the Home object makes sure that all the properties of the Home object are not null. If it is the case the value gets sets to something not null ("null", basic date, 0, ...).
The parameters are:
homes = a list of Home objects received from an other class
gr = a list of all the unique groups in the list of homes
na = a list of all the unique names in the list of homes
The same name can occur on different groups
Hopefully someone can help me get further!
Thanks in advance!
Select must return something. You are not returning but only adding to an existing list. Do this instead:
var newList = Filtered.GroupBy(x => new { x.GroupName, x.PersonName }):
var result = new MultiKeyDictionary<string, string, int>);
foreach(var y in newList)
{
result.Add(y.Key.GroupName, y.Key.PersonName, y.ToList().Count));
}
The reason you are getting error below:
"Cannot convert type 'void' to 'MultiKeyDictionary'
is because you are trying to cast the returned value from Add which is void to MultiKeyDictionary<string,string,int> which clearly cannot be done.
If MultiKeyDictionary requires the two keys to match in order to find a result, then you might want to just use a regular Dictionary with a Tuple as a composite type. C# 7 has features that make this pretty easy:
public Dictionary<(string, string), int> GetPersonsPerGroup(IEnumerable<Home> homes ,List<string> gr, List<string> na)
{
return Filtered.GroupBy(x => (x.GroupName, x.PersonName))
.ToDictionary(g => g.Key, g => g.Count);
}
You can even associate optional compile-time names with your tuple's values, by declaring it like this: Dictionary<(string groupName, string personName), int>.
Your grouping key anonymous object should work fine as a standard Dictionary key, so no reason to create a new type of Dictionary unless it offers special access via single keys, so just convert the grouping to a standard Dictionary:
var result = Filtered.GroupBy(f => new { f.GroupName, f.PersonName })
.ToDictionary(fg => fg.Key, fg => fg.Count());
In C#, I have an object type 'A' that contains a list of key value pairs.
The key value pairs is a category string and a value string.
To instantiate object type A, I would have to do the following:
List<KeyValuePair> keyValuePairs = new List<KeyValuePair>();
keyValuePairs.Add(new KeyValuePair<"Country", "U.S.A">());
keyValuePairs.Add(new KeyValuePair<"Name", "Mo">());
keyValuePairs.Add(new KeyValuePair<"Age", "33">());
A a = new A(keyValuePairs);
Eventually, I will have a List of A object types and I want to manipulate the list so that i only get unique values and I base it only on the country name. Therefore, I want the list to be reduced to only have ONE "Country", "U.S.A", even if it appears more than once.
I was looking into the linq Distinct, but it does not do what I want because it I can't define any parameters and because it doesn't seem to be able to catch two equivalent objects of type A. I know that I can override the "Equals" method, but it still doesn't solve the my problem, which is to render the list distinct based on ONE of the key value pairs.
To expand upon Karl Anderson's suggestion of using morelinq, if you're unable to (or don't want to) link to another dll for your project, I implemented this myself awhile ago:
public static IEnumerable<T> DistinctBy<T, U>(this IEnumerable<T> source, Func<T, U>selector)
{
var contained = new Dictionary<U, bool>();
foreach (var elem in source)
{
U selected = selector(elem);
bool has;
if (!contained.TryGetValue(selected, out has))
{
contained[selected] = true;
yield return elem;
}
}
}
Used as follows:
collection.DistinctBy(elem => elem.Property);
In versions of .NET that support it, you can use a HashSet<T> instead of a Dictionary<T, Bool>, since we don't really care what the value is so much as that it has already been hashed.
Check out the DistinctBy syntax in the morelinq project.
A a = new A(keyValuePairs);
a = a.DistinctBy(k => new { k.Key, k.Value }).ToList();
You need to select the distinct property first:
Because it's a list inside a list, you can use the SelectMany. The SelectMany will concat the results of subselections.
List<A> listOfA = new List<A>();
listOfA.SelectMany(a => a.KeyValuePairs
.Where(keyValue => keyValue.Key == "Country")
.Select(keyValue => keyValue.Value))
.Distinct();
This should be it. It will select all values where the key is "Country" and concat the lists. Final it will distinct the country's. Given that the property KeyValuePairs of the class A is at least a IEnumerable< KeyValuePair< string, string>>
var result = keyValuePairs.GroupBy(x => x.Key)
.SelectMany(g => g.Key == "Country" ? g.Distinct() : g);
You can use the groupby statement. From here you can do all kind off cool stuf
listOfA.GroupBy(i=>i.Value)
You can groupby the value and then sum all the keys or something other usefull
I have a list that I want to put in a dictionary, for simplicity the values being inserted will all be the same.
I can use a foreach loop.
List<string> list = new List<string>();
list.Add("Earth");
list.Add("Wind");
list.Add("Fire");
list.Add("Water");
list.Add("Water"); // Will NOT BE INSERTED using the foreach loop
var myDictionary= new Dictionary<string, int>();
foreach (string value in list)
{
if (!myDictionary.ContainsKey(value))
{
myDictionary.Add(value, 1);
}
}
The above works.
But I want to use ToDictionary do the same in the following way -
Dictionary<string, int> myDictionary2 = list.ToDictionary(i => i, i => 1);
Of course this fails because I'm adding "Water" twice.
What is the correct way of checking for duplicate entries when using ToDictionary?
You could use Distinct() to filter out duplicates:
Dictionary<string, int> myDictionary2 = list.Distinct().ToDictionary(i => i, i => 1);
The same approach would make your traditional loop much clearer too, since you don't have to check "manually" for duplicates:
foreach (string value in list.Distinct())
{
myDictionary.Add(value, 1);
}
Distinct is one option that avoids the duplicate key issue. If you need a count of duplicates, you might try something more like this GroupBy as follows:
var dict = list.GroupBy(i => i).ToDictionary(g => g.Key, g => g.Count());
If your application is not just a simple string-list/duplicate-count structure, you might get some mileage from choosing a different structure like a Lookup that you can get from calling the ToLookup extension -or possibly going with a Grouping like the GroupBy I used above.
So I a collection of dictionary items in a list:
List<Dictionary<string, string>> inputData = new List<Dictionary<string, string>>(inputs);
List<Dictionary<string, string>> itemStack = new List<Dictionary<string, string>>();
Now what I want to do is for each inputData dictionary item I want to check if itemStack has the same value (Dictionary Item) already.
I was thinking it would be like?
foreach (var item in inputData)
{
if(!itemStack.Contains(item){ itemStack.Add(item)}
else{ //Duplicate found}
}
It doesn't really check the items values inside? It just assumes that it doesn't have it...
All i want is if itemStack contains and item that is already in the stack don't include it.
I know I'm missing something obvious.
Thanks,
Dictionary is reference type, so it doesn't check the "deep" value like you expected.
You will have to write your own "Contains" method, either as totally separate method or extension of the Dictionary itself then use it instead, for example:
if(!MyContains(itemStack, item)){ itemStack.Add(item)}
True that a HashSet would be better, but if you want to do it here, try this (assuming you are filtering duplicate keys only):
foreach (var item in inputData.Keys)
{
if (itemStack.Where(x => x.Key == item.Key).Count() > 0)
// There was a duplicate
}
Or, if you only care when the data is coming out you can call:
itemStack.Distinct()
I think, your way is right. On my mind, HashSet is good, but when you add a new element, it performs the same test on the contents of the same items.
Regards.
Based on your initial problem statement, you might do something like this:
var aggregateKnownKeys = itemStack.SelectMany(d => d.Keys);
itemStack.AddRange(
inputData.Select(d=> d.Where(p => !aggregateKnownKeys.Contains(p.Key))
.ToDictionary(p => p.Key, p => p.Value)));
If you only need to combine two dictionaries then you could do this to skip keys that exist in itemStack:
var inputData = new Dictionary<string, string>();
var itemStack = new Dictionary<string, string>();
var oldStack = itemStack;
itemStack = new[] { inputData.SkipWhile(d => oldStack.Keys.Contains(d.Key)), itemStack }
.SelectMany(d => d)
.ToDictionary(d => d.Key, d => d.Value);
Okay so this isn't quite a full answer but it's what I did.
So I have a List of items and instead of doing a full compare to whats in an List(Hence the other considered) I just did a single item check:
if(!String.IsNullOrEmpty(item["itemId"]))
{
alert.DaleksApproaching(item["itemId"]);
}
So when it does see it has a value it just does another event to get rid of it.
The idea of using LINQ and the method approaches about(Contains and Distinct)I like. I have yet to try that, but I plan on doing that. For this it doesn't use LINQ :(
Thanks everyone!