Best practise in splitting up query result by a column - c#

I have a query which returns data for a number of dataseries in one lump.
I would like to split this data up into the data series id, by the dataseries id. The query cannot be changed.
Its not safe to assume that the data is ordered by the series id, so what would be the best way to do this? LINQ?

I suggest grouping the result by the DataSeriesId.
var groupedResult = QueryDatabase().GroupBy(item => item.DataSeriesId);
Now you can access the grouped data as follows. The example will just print all groups with all items.
foreach(var group in groupedResult)
{
Console.WriteLine("Group: " + group.Key);
foreach(var item in group)
{
Console.WriteLine(" Item: " + item);
}
}
Or you can group the database query result into a list of lists.
IList<IList<DataItem>> = QueryDatabase()
.GroupBy(item => item.DataSeriesId)
.Select(group => group.ToList())
.ToList();
Or you can build a dictionary from DataSeriesId to a list of data items from the database query result.
IDictionary<Int32, IList<DataItem>> = QueryDatabase()
.GroupBy(item => item.DataSeriesId)
.ToDictionary(group => group.DataSeriesId, group => group.ToList());
Or use Enumerable.ToLookUp() if you don't want to alter the dictionary later.
UPDATE
Just noticed the "Best practice" in the question an the "DataReader" tag. Well LINQ is easy to write and easy to get right but it is probably not the fastest solution. So depending on your requirements using LINQ might be a good or bad choice. I would prefer LINQ if performance is not (yet) an issue. Else I would consider building a dictionary from DataSeriesId to a list of data items while reading the data.
IDictionary<Int32, IList<DataItem>> result =
new Dictionary<Int32, IList<DataItem>>();
while (dataSource.DataAvailiable)
{
DataItem item = dataSource.ReadItem();
IList<DataItem> items;
if (!result.TryGetValue(item.DataSeriesId))
{
items = new List<DataItem>();
result.Add(item.DataSeriesId, items);
}
items.Add(item);
}

Related

How can I find the first items in a list with a different value for a property using Linq?

I have an ordered list of objects, and I would like to find the index of each item where a property changes, and get a dictionary/list of pairs matching index to property. For example, finding the index of each new first letter in a list of words ordered alphabetically.
I can do this with a foreach loop:
Initials = new Dictionary<char, int>();
int i = 0;
foreach (var word in alphabeticallyOrderedList))
{
if (!Initials.ContainsKey(word.First()))
{
Initials[word.First()] = i;
}
i++;
}
But I feel like there should be an elegant way of doing this with Linq.
You could have the same functionality with LINQ by using the overload of Select that exposes the index and by using GroupBy + ToDictionary:
Initials = alphabeticallyOrderedList
.Select((word, index) => new { Word = word, WordIndex = index })
.GroupBy(x => x.Word[0])
.ToDictionary(charGroup => charGroup.Key, charGroup => charGroup.First().WordIndex);
But to quote myself:
LINQ is not always more readable, especially when indexes are important. You also lose some debugging, exception handling and logging capabilities if you use a large LINQ query

Performance Improvement Tips for ForEach loop in C#?

I need to optimize the below foreach loop. The foreach loop is taken more time to get the unique items.
Instead can the FilterItems be converted into a list collection. If so how to do it. Then i will take unique items easily from it.
The problem arises when i have 5,00,000 items in FilterItems.
Please suggest some ways to optimize the below code:
int i = 0;
List<object> order = new List<object>();
List<object> unique = new List<object>();
// FilterItems IS A COLLECTION OF RECORDS. CAN THIS BE CONVERTED TO A LIST COLLECTION DIRECTLY, SO THAT I CAN TAKE THE UNIQUE ITEMS FROM IT.
foreach (Record rec in FilterItems)
{
string text = rec.GetValue(“Column Name”);
int position = order.BinarySearch(text);
if (position < 0)
{
order.Insert(-position - 1, text);
unique.Add(text);
}
i++;
}
It's unclear what you mean by "converting FilterItems into a list" when we don't know anything about it, but you could definitely consider sorting after you've got all the items, rather than as you go:
var strings = FilterItems.Select(record => record.GetValue("Column Name"))
.Distinct()
.OrderBy(x => x)
.ToList();
The use of Distinct() here will avoid sorting lots of equal items - it looks like you only want distinct items anyway.
If you want unique to be in the original order but order to be the same items, just sorted, you could use:
var unique = FilterItems.Select(record => record.GetValue("Column Name"))
.Distinct()
.ToList();
var order = unique.OrderBy(x => x).ToList();
Now Distinct() isn't guaranteed to preserve order - but it does so in the current implementation, and that's the most natural implementation, too.

Simplest way to check db values against values in an array

I'm looping through the items in my database using C# .NET and I'm attempting to display different data dependant on if a column value matches any of the values in an array. Because my array could potentially have hundreds of values, I'd rather not create hundreds of different IF statements, if possible. Is there a simpler way to achieve this?
Here's some example code, where "Items" is my db data and "Categories" is a column of said data:
var categoryList = new List<int> { 1, 2, 3, 4 };
foreach(var item in Items){
if(item.Categories.Any(x => #categoryList.Equals(x))){
<p>Yes</p>
}else{
<p>No</p>
}
}
The answer I give is based on the answer of this question. I modified the code to your situation.
foreach(var item in Items)
{
bool hasCategory = categoryList.Any(x => item.Categories.Any(c => c.Id == x));
}
or for larger collections (performance-wise):
bool hasCategory = item.Categories.Select(c => c.Id)
.Intersect(categoryList)
.Any();
Edit:
At first I thought item.Categories was a collection of IDs or something but then I started doubting. If item.Categories is just a single integer, following code will work:
foreach(var item in Items)
{
if(categoryList.Any(x => x == item.Categories))
<p>Yes</p>
else
<p>No</p>
}

Simplifying linq when filtering data

I wanted to ask for suggestions how I can simplify the foreach block below. I tried to make it all in one linq statement, but I couldn't figure out how to manipulate "count" values inside the query.
More details about what I'm trying to achieve:
- I have a huge list with potential duplicates, where Id's are repeated, but property "Count" is different numbers
- I want to get rid of duplicates, but still not to loose those "Count" values
- so for the items with the same Id I summ up the "Count" properties
Still, the current code doesn't look pretty:
var grouped = bigList.GroupBy(c => c.Id).ToList();
foreach (var items in grouped)
{
var count = 0;
items.Each(c=> count += c.Count);
items.First().Count = count;
}
var filtered = grouped.Select(y => y.First());
I don't expect the whole solution, pieces of ideas will be also highly appreciated :)
Given that you're mutating the collection, I would personally just make a new "item" with the count:
var results = bigList.GroupBy(c => c.Id)
.Select(g => new Item(g.Key, g.Sum(i => i.Count)))
.ToList();
This performs a simple mapping from the original to a new collection of Item instances, with the proper Id and Count values.
var filtered = bigList.GroupBy(c=>c.Id)
.Select(g=> {
var f = g.First();
f.Count = g.Sum(c=>c.Count);
return f;
});

Matching the order of one array to another

I have an int array of ID's that are ordered properly. Then I have an an array of unordered objects that have ID properties.
I would like to order the objects by ID that match the order of the int array.
Something along the lines of
newObjectArray = oldObjectArray.MatchOrderBy(IdArray)
Would be most desirable
I feel like I should be able to accomplish this using LINQ but I have yet to find a way.
My current method doesn't seem very efficient since it has to query on every iteration of the collection. I suspect that performance can suffer for sufficiently large collections. Which eventually will happen.
Here is my current implementation:
//this is just dummy data to show you whats going on
int[] orderedIDs = new int[5] {5534, 5632, 2334, 6622, 2344};
MemberObject[] searchResults = MyMethodToGetSearchResults();
MemberObject[] orderedSearchResults = new MemberObject[orderedIDs.Count()];
for(int i = 0; i < orderedIDs.Count(); i++)
{
orderedSearchResults[i] = searchResults
.Select(memberObject => memberObject)
.Where(memberObject => memberObject.id == orderedIDs[i])
.FirstOrDefault();
}
A brute force implementation:
MemberObject[] sortedResults =
IdArray.Select(id => searchResults
.FirstOrDefault( item => item.id == id ))
However, this requires reiterating searchResults for every item in IdArray and doesn't deal too neatly with items that have duplicate ids.
Things improve if you make an ILookup of your search results, so that grabbing the correct search result for each item in IdArray is now O(1) time.
ILookup<int, MemberObject> resultLookup = searchResults.ToLookup(x => x.id);
Now:
MemberObject[] sortedResults =
IdArray.SelectMany(id => resultLookup[id])

Categories

Resources