I have a list (or can be array) of strings that I want to dynamically create an anonymous object from. How do I do this?
var dataSet = new DataSet();
dataSet.ReadXml(#"");
var dataTable = dataSet.Tables[0];
var dataRow = dataTable.Rows[0];
var keys = new List<string> {"Column1","Column2"};
var result = new {keys[0] = dataRow[keys[0]], keys[1] = dataRow[keys[1]]}
So that list named "keys" is going to be created outside this method and can contain 1 to many values. I tried creating a dictionary and looping through the list and adding key/value pairs to the dictionary but then I couldnt figure out how to convert the dictionary back to an anonymous type. I also experimented with the expando objects but that didn't seem to get me any farther.
I must be able to return an anonymous type as the result of this method will be using with the GroupBy clause of a LINQ query.
Here is the method I had to dynamically create the dictionary:
public object Key(DataRow dataRow, List<String> keys)
{
var dictionary = new IDictionary<string, object>;
foreach (string key in keys)
{
dictionary.Add(key, dataRow[key]);
}
return dictionary;
}
Here is my LINQ query:
var duplicates = dataTable.AsEnumerable().GroupBy(r => Key(r, keys)).Where(c => c.Count() > 1).ToList();
The GroupBy clause works if I hardcode in an anonymous type from the Key() method. Basically I just need the GroupBy clause to be dynamically set based upon the values in the keys list.
Stripping down your question, what you want is to be able to group a list of items based on a runtime property which could be composed of one or more properties of that item. In essence, it means you need a selector function (which is your Key method) that transforms an item into a key.
In order for GroupBy to work, it needs to be able to compare any two instances of the key to see if they're equal. This means the key needs to implement a meaningful Equals() method, or you need an IEqualityComparer implementation that does the work for you. In this case I wouldn't bother with creating a new Key, just write an Equality Comparer that can compare two DataRows directly:
var duplicates = dataTable
.AsEnumerable()
.GroupBy(r => r, new MyDataRowComparer(keys))
.Where(c => c.Count() > 1)
.ToList();
internal class MyDataRowComparer : IEqualityComparer<DataRow>
{
private readonly string[] _keys;
public MyDataRowComparer(string[] keys)
{
_keys = keys; // keep the keys to compare by.
}
public bool Equals(DataRow x, DataRow y)
{
// a simple implementation that checks if all the required fields
// match. This might need more work.
bool areEqual = true;
foreach (var key in _keys)
{
areEqual &= (x[key] == y[key]);
}
return areEqual;
}
public int GetHashCode(DataRow obj)
{
// Add implementation here to create an aggregate hashcode.
}
}
Related
I have a dictionary which has an integer Key that represents a year, and a Value which is a list of object Channel. I need to flatten the data and create a new object from it.
Currently, my code looks like this:
Dictionary<int, List<Channel>> myDictionary;
foreach(var x in myDictionary)
{
var result = (from a in x.Value
from b in anotherList
where a.ChannelId == b.ChannelId
select new NewObject
{
NewObjectYear = x.Key,
NewObjectName = a.First().ChannelName,
}).ToList();
list.AddRange(result);
}
Notice that I am using the Key to be the value of property NewObjectYear.
I want to get rid of foreach since the dictionary contains a lot of data and doing some joins inside the iteration makes it very slow. So I decided to refactor and came up with this:
var flatten = myDictionary.SelectMany(x => x.Value.Select(y =>
new KeyValuePair<int, Channel>(x.Key, y))).ToList();
But with this, I couldn't get the Key directly. Using something like flatten.Select(x => x.Key) is definitely not the correct way. So I tried finding other ways to flatten that would be favorable for my scenario but failed. I also thought about creating a class which will contain the year and the list from the flattened but I don't know how.
Please help me with this.
Also, is there also another way that doesn't have the need to create a new class?
It seems to me you are trying to do only filtering, you do not need join for that:
var anotherListIDs = new HashSet<int>(anotherList.Select(c => c.ChannelId));
foreach (var x in myDictionary)
{
list.AddRange(x.Value
.Where(c => anotherListIDs.Contains(c.ChannelId))
.Select(c => new NewObject
{
NewObjectYear = x.Key,
NewObjectName = c.First().ChannelName,
}));
}
You do realise, that if the second element of the list in a specific dictionary element has a matching channelId, that you return the first element of this list, don't you?
var otherList = new OtherItem[]
{
new OtherItem() {ChannelId = 1, ...}
}
var dictionary = new Dictionary<int, List<Channel>[]
{
{ 10, // Key
new List<Channel>() // Value
{
new Channel() {ChannelId = 100, Name = "100"},
new Channel() {ChannelId = 1, Name = "1"},
},
};
Although the 2nd element has a matching ChannelId, you return the Name of the first element.
Anyway, let's assume this is what you really want. You are right, your function isn't very efficient.
Your dictionary implements IEnumerable<KeyValuePair<int, List<Channel>>. Therefore every x in your foreach is a KeyValuePair<int, List<Channel>. Every x.Value is a List<Channel>.
So for every element in your dictionary (which is a KeyValuePair<int, List<Channel>), you take the complete list, and perform a full inner join of the complete list with otherList, and for the result you take the key of the KeyValuePair and the first element of the List in the KeyValuePair.
And even though you might not use the complete result, but only the first or the first few, because of FirstOrDefault(), or Take(3), you do this for every element of every list in your Dictionary.
Indeed your query could be much more efficient.
As you use the ChannelIds in your OtherList only to find out if it is present, one of the major improvements would be to convert the ChannelIds of OtherList to a HashSet<int> where you have superior fast lookup to check if the ChannelId of one of the values in your Dictionary is in the HashSet.
So for every element in your dictionary, you only have to check every ChannelId in the list to see if one of them is in the HashSet. As soon as you've found one, you can stop and return only the first element of the List and the Key.
My solution is an extension function of Dictionary>. See Extension Methods Demystified
public static IEnumerable<NewObject> ExtractNewObjects(this Dictionary<int, List<Channel>> dictionary,
IEnumerable<OtherItem> otherList)
{
// I'll only use the ChannelIds of the otherList, so extract them
IEnumerable<int> otherChannelIds = otherList
.Select(otherItem => otherItem.ChannelId);
return dictionary.ExtractNewObjects(otherChannelIds);
}
This calls the other ExtractNewobjects:
public static IEnumerable<NewObject> ExtractNewObjects(this Dictionary<int, List<Channel>> dictionary,
IEnumerable<int> otherChannelIds)
{
var channelIdsSet = new HashSet<int>(otherChannelIds));
// duplicate channelIds will be removed automatically
foreach (KeyValuePair<int, List<Channel>> keyValuePair in dictionary)
{
// is any ChannelId in the list also in otherChannelIdsSet?
// every keyValuePair.Value is a List<Channel>
// every Channel has a ChannelId
// channelId found if any of these ChannelIds in in the HashSet
bool channelIdFound = keyValuePair.Value
.Any(channel => otherChannelIdsSet.Contains(channel.ChannelId);
if (channelIdFound)
{
yield return new NewObject()
{
NewObjectYear = keyValuePair.Key,
NewObjectName = keyValuePair.Value
.Select(channel => channel.ChannelName)
.FirstOrDefault(),
};
}
}
}
usage:
IEnumerable<OtherItem> otherList = ...
Dictionary<int, List<Channel>> dictionary = ...
IEnumerable<Newobject> extractedNewObjects = dictionary.ExtractNewObjects(otherList);
var someNewObjects = extractedNewObjects
.Take(5) // here we see the benefit from the yield return
.ToList();
We can see four efficiency improvements:
the use of HashSet<int> enables a very fast lookup to see if the ChannelId is in OtherList
the use of Any() stops enumerating the List<Channel> as soon as we've found a matching Channelid in the HashSet
the use of yield return makes that you don't enumerate over more elements in your Dictionary than you'll actually use.
The use of Select and FirstOrDefault when creating NewObjectName prevents exceptions if List<Channel> is empty
So I have a collection of objects who have multiple properties, two of these are groupname and personname. Now I need to count in the collection how many of each object belong to a certain group and person. So in other words, I need to group by groupname, then personname and then count how many objects have this combination. First I created this
public MultiKeyDictionary<string, string, int> GetPersonsPerGroup(IEnumerable<Home> homes ,List<string> gr, List<string> na)
{
List<string> groups = gr;
groups.Add("");
List<string> names = na;
names.Add("");
List<Home> Filtered = homes.ToList();
Filtered.ForEach(h => h.RemoveNull());
var result = new MultiKeyDictionary<string, string, int>();
int counter1 = 0;
foreach (var g in groups)
{
int counter2 = 0;
foreach (var n in names)
{
int counter3 = 0;
foreach (Home h in Filtered)
{
if (h.GroupName == g && h.PersonName == n)
{
counter3++;
if (counter3 > 100)
break;
}
}
if (counter3 > 0)
{
result.Add(g,n,counter3);
}
counter2++;
}
counter1++;
}
Which may look good, but the problem is that the "home" parameter can contain more than 10000 objects, with more than 1500 unique names and around 200 unique groups. Which causes this to iterate like a billion times really slowing my program down. So I need an other way of handling this. Which made me decide to try using linq. Which led to this creation:
var newList = Filtered.GroupBy(x => new { x.GroupName, x.PersonName })
.Select(y => (MultiKeyDictionary<string, string, int>)result.Add(y.Key.GroupName, y.Key.PersonName, y.ToList().Count));
Which gives an error "Cannot convert type 'void' to 'MultiKeyDictionary<string,string,int>' and I have no idea how to solve it. How can I make it so that the result of this query gets stored all in one MultikeyDictionary without having to iterate over each possible combination and counting all of them.
Some information:
MultiKeyDictionary is a class I defined (something I found on here actually), it's just a normal dictionary but with two keys assosiated to one value.
The RemoveNull() method on the Home object makes sure that all the properties of the Home object are not null. If it is the case the value gets sets to something not null ("null", basic date, 0, ...).
The parameters are:
homes = a list of Home objects received from an other class
gr = a list of all the unique groups in the list of homes
na = a list of all the unique names in the list of homes
The same name can occur on different groups
Hopefully someone can help me get further!
Thanks in advance!
Select must return something. You are not returning but only adding to an existing list. Do this instead:
var newList = Filtered.GroupBy(x => new { x.GroupName, x.PersonName }):
var result = new MultiKeyDictionary<string, string, int>);
foreach(var y in newList)
{
result.Add(y.Key.GroupName, y.Key.PersonName, y.ToList().Count));
}
The reason you are getting error below:
"Cannot convert type 'void' to 'MultiKeyDictionary'
is because you are trying to cast the returned value from Add which is void to MultiKeyDictionary<string,string,int> which clearly cannot be done.
If MultiKeyDictionary requires the two keys to match in order to find a result, then you might want to just use a regular Dictionary with a Tuple as a composite type. C# 7 has features that make this pretty easy:
public Dictionary<(string, string), int> GetPersonsPerGroup(IEnumerable<Home> homes ,List<string> gr, List<string> na)
{
return Filtered.GroupBy(x => (x.GroupName, x.PersonName))
.ToDictionary(g => g.Key, g => g.Count);
}
You can even associate optional compile-time names with your tuple's values, by declaring it like this: Dictionary<(string groupName, string personName), int>.
Your grouping key anonymous object should work fine as a standard Dictionary key, so no reason to create a new type of Dictionary unless it offers special access via single keys, so just convert the grouping to a standard Dictionary:
var result = Filtered.GroupBy(f => new { f.GroupName, f.PersonName })
.ToDictionary(fg => fg.Key, fg => fg.Count());
I have a Table in my Entities with 370 columns ! Furthermore i have a string Array which is not known before runtime (comes from a website).
e.g.:
string [] columns = {"column1", "column2", "column3"}
How can i fire a linq to my entities which gives me only the result with the giving columns?
I searched for hours, but don`t suceed till now - any suggestions?
This is not something that you can do with Linq-to-Entities. You need to be able to declare the columns in your code.
A better approach would be to create the query in Sql using the column names in your array, and use something like Dapper to map the results to your objects.
You could instantiate an ExpandoObject in your Delegate and use Reflection to get the columns specified in the incoming array.
The following:
List<IDictionary<String, Object>> List = Context.Row_Type.Select(delegate(Row_Type Row) {
IDictionary<String, Object> Out = new ExpandoObject() as IDictionary<String, Object>;
PropertyInfo[] PIs = Row.GetType().GetProperties();
foreach(PropertyInfo PI in PIs) {
if(PI.GetIndexParameters().Length == 0) {
Out.Add(PI.Name, PI.GetValue(Row));
}
}
return Out;
}).ToList();
Will return a List < IDictionary< String, Object>> with all the properties, to return the desired columns just discriminate by PI.Name:
if(PI.Name == "Desired column"){ // Or Array index
// Add to Out:
Out.Add( PI.Name, PI.GetValue(Row) )
}
I'm guessing you don't really want to return a class with unknown column names. Would a dictionary of column names and values work for you? Your query would still have to retrieve all of the columns, but you can only return the ones you care about.
string [] columns = {"column1", "column2", "column3"}
var entity = GetEntity();
var dictionary = columns.ToDictionary(c => c, c => entity.GetType().GetProperty(c).GetValue(entity));
Or, if you have a collection...
var entities = GetEntities();
var results = entities
.Select(e => columns.ToDictionary(c => c, c => e.GetType().GetProperty(c).GetValue(e)));
In C#, I have an object type 'A' that contains a list of key value pairs.
The key value pairs is a category string and a value string.
To instantiate object type A, I would have to do the following:
List<KeyValuePair> keyValuePairs = new List<KeyValuePair>();
keyValuePairs.Add(new KeyValuePair<"Country", "U.S.A">());
keyValuePairs.Add(new KeyValuePair<"Name", "Mo">());
keyValuePairs.Add(new KeyValuePair<"Age", "33">());
A a = new A(keyValuePairs);
Eventually, I will have a List of A object types and I want to manipulate the list so that i only get unique values and I base it only on the country name. Therefore, I want the list to be reduced to only have ONE "Country", "U.S.A", even if it appears more than once.
I was looking into the linq Distinct, but it does not do what I want because it I can't define any parameters and because it doesn't seem to be able to catch two equivalent objects of type A. I know that I can override the "Equals" method, but it still doesn't solve the my problem, which is to render the list distinct based on ONE of the key value pairs.
To expand upon Karl Anderson's suggestion of using morelinq, if you're unable to (or don't want to) link to another dll for your project, I implemented this myself awhile ago:
public static IEnumerable<T> DistinctBy<T, U>(this IEnumerable<T> source, Func<T, U>selector)
{
var contained = new Dictionary<U, bool>();
foreach (var elem in source)
{
U selected = selector(elem);
bool has;
if (!contained.TryGetValue(selected, out has))
{
contained[selected] = true;
yield return elem;
}
}
}
Used as follows:
collection.DistinctBy(elem => elem.Property);
In versions of .NET that support it, you can use a HashSet<T> instead of a Dictionary<T, Bool>, since we don't really care what the value is so much as that it has already been hashed.
Check out the DistinctBy syntax in the morelinq project.
A a = new A(keyValuePairs);
a = a.DistinctBy(k => new { k.Key, k.Value }).ToList();
You need to select the distinct property first:
Because it's a list inside a list, you can use the SelectMany. The SelectMany will concat the results of subselections.
List<A> listOfA = new List<A>();
listOfA.SelectMany(a => a.KeyValuePairs
.Where(keyValue => keyValue.Key == "Country")
.Select(keyValue => keyValue.Value))
.Distinct();
This should be it. It will select all values where the key is "Country" and concat the lists. Final it will distinct the country's. Given that the property KeyValuePairs of the class A is at least a IEnumerable< KeyValuePair< string, string>>
var result = keyValuePairs.GroupBy(x => x.Key)
.SelectMany(g => g.Key == "Country" ? g.Distinct() : g);
You can use the groupby statement. From here you can do all kind off cool stuf
listOfA.GroupBy(i=>i.Value)
You can groupby the value and then sum all the keys or something other usefull
I have a array of string say:
String[] Fields=new String[]{RowField,RowField1}
In which I can use the below query to get the values by specifying the values is query i.e RowField and RowField1:
var Result = (
from x in _dataTable.AsEnumerable()
select new
{
Name = x.Field<object>(RowField),
Name1 = x.Field<object>(RowField1)
})
.Distinct();
But if suppose I have many values in the Array like:
String[] Fields= new String[]
{
RowField,
RowField1,
RowField2,
.......
RowField1000
};
How can I use the query here without specifying each of the rowfield in the query?
How can i iterate through the array items inside the LINQ?
var Result = (
from x in _dataTable.AsEnumerable()
select (
from y in Fields
select new KeyValuePair<string, object>(y, x))
.ToDictionary())
.Distinct(DictionariesComparer);
You'll also need to write your own .ToDictionary() extension method and DictionariesComparer method (as Dictionary doesn't implement IEquatable).
Essentially, you want to retrieve specific fields from a DataTable without hardcoding the field names.
The following code will return a single dictionary object per row with the fields you specify in your array. There is no need to create additional extension methods or comparers:
var result = (from row in _dataTable.AsEnumerable()
let projection = from fieldName in fields
select new {Name = fieldName, Value = row[fieldName]}
select projection.ToDictionary(p=>p.Name,p=>p.Value));
The inner select picks the field values you need from each table row and stores them in the projection variable. The outer select converts this variable in a Dictionary
You can iterate over the result to get specific fields like this:
foreach (var row in result)
{
Console.WriteLine(row["field1"]);
}
EDIT:
The above code doesn't return distinct values. It is possible to return distinct values without writing a special comparer using group by but the code is not very pretty:
var result = (from row in table.AsEnumerable()
let projection = from fieldName in fields
select new { Name = fieldName, Value = row[fieldName] }
group projection by projection.Aggregate((v, p) =>
new {
Name = v.Name + p.Name,
Value = (object)String.Format("{0}{1}", v.Value, p.Value)
}) into g
select g.FirstOrDefault().ToDictionary(p=>p.Name,p=>p.Value));
The Aggregate creates a new projection whose Name and Value properties are the concatenation of all name and value fields. The result of the aggregate is used to group all rows and return the first row of each group. It works but it is definitely ugly.
It would be better to create a simple DictionaryComparer like the following code:
public class DictionaryComparer<TKey,TValue>: EqualityComparer<Dictionary<TKey,TValue>>
{
public override bool Equals(Dictionary<TKey, TValue> x, Dictionary<TKey, TValue> y)
{
//True if both sequences of KeyValuePair items are equal
var sequenceEqual = x.SequenceEqual(y);
return sequenceEqual;
}
public override int GetHashCode(Dictionary<TKey, TValue> obj)
{
//Quickly detect differences in size, defer to Equals for dictionaries
//with matching sizes
return obj.Count;
}
}
This allows you to write:
var result = (from row in table.AsEnumerable()
let projection = from fieldName in fields
select new {Name = fieldName, Value = row[fieldName]}
select projection.ToDictionary(p=>p.Name,p=>p.Value))
.Distinct(new DictionaryComparer<string, object>());
There is no foreach linq expression. I typically create my own extension method
Something along the lines of:
public static void Foreach<T>(this IEnumerable<T> items, Action<T> action)
{
foreach(T t in items)
{
action(t);
}
}
However beware if you're planning on using this with Linq2SQL as it could create a lot of db hits!