I do have a string of Empids separated by comma like:
EMpID:"2007,2008,2002,1992,1000,2108,1085
and I need to retrieve the records of all those specified employees using LINQ query.
I tried it with looping but I need to get that in efficient and faster way.
Here goes what i did using looping.
string[] EMpID_str = LeaveDictionary["EMpID"].ToString().Split(',');
for (int i = 0; i < EMpID_str.Length; i++)
{
EMpID = Convert.ToInt32(EMpID_str[i]);
//Linq to get data for each Empid goes here
}
But What I need is to use single LINQ or Lambda query to retrieve the same.Without looping
First convert your ,(comma) separated empId to string array like below:
var empArr = EmpId.split(',');
var employeesResult = emplyeeList.Where(x => empArr.contains(x.EmpId.ToString()));
I hope, it will help someone.
If the Ids that you want to fetch are numbers, not strings, then you should not convert the string to an array of strings, but to a sequence of numbers:
IEnumerable<int> employeeIdsToFetch = LeaveDictionary["EMpID"].ToString()
.Split(',')
.Select(splitText => Int32.Parse(splitText));
To fetch all employees with thees Ids:
var fetchedEmployees = dbContext.Employees
.Where(employee => employeeIdsToFetch.Contains(employee.Id))
.Select(employee => new
{
// Select only the employee properties that you plan to use:
Id = employee.Id,
Name = employee.Name,
...
});
You can use the Expression class to build a Func<int, bool> from your string and use it with the Where methode:
var str = "2,5,8,9,4,6,7";
var para = Expression.Parameter(typeof(int));
var body = str.Split(",")
.Select(s => int.Parse(s))
.Select(i => Expression.Constant(i))
.Select(c => Expression.Equal(para, c))
.Aggregate((a, b) => Expression.Or(a, b));
Func<int, bool> func = Expression.Lambda<Func<int, bool>>(body, para).Compile();
and if you this solution to work with linq to SQL just dont compile the expression at the end and let the linq to SQL engine compile it to an efficent SQL expression.
Instead of the Aggregate Method (which will produce an expression with linear complexity) one could use an divide and conquer approach to fold the values into one value.
For example with this class:
public static class Helper
{
public static T EfficientFold<T>(this List<T> list, Func<T, T, T> func)
{
return EfficientFold(list, 0, list.Count, func);
}
private static T EfficientFold<T>(List<T> list, int lowerbound, int upperbound, Func<T, T, T> func)
{
int diff = upperbound - lowerbound;
var mid = lowerbound + diff / 2;
if (diff < 1)
{
throw new Exception();
}
else if (diff == 1)
{
return list[lowerbound];
}
else
{
var left = EfficientFold(list, lowerbound, mid, func);
var right = EfficientFold(list, mid, upperbound, func);
return func(left, right);
}
}
}
and then we can do
var body = str.Split(",")
.Select(s => int.Parse(s))
.Select(i => Expression.Constant(i))
.Select(c => Expression.Equal(para, c))
.ToList()
.EfficientFold((a, b) => Expression.Or(a, b));
which gives the evaluation a complexity of log(n).
Related
I have this string:
string input = "1,2,3,4,s,6";
Pay attention to the s character.
I just want to convert this string in a List<int> using LINQ. I initially tried in this way:
var myList = new List<int>();
input.Split(',').ToList().ForEach(n =>
myList.Add(int.TryParse(n, out int num) ? num : -1)
);
lista.RemoveAll(e => e == -1);
But I prefer not have any -1 instead of a no-number characters.
So now I try with this:
var myList = new List<int>();
input.Split(',').ToList()
.FindAll(n => int.TryParse(n, out int _))
.ForEach(num => myList.Add(int.Parse(num)));
I prefer this, but is really a shame that the parsing happening two times (TryParse at first and then Parse). But, from what I understand, the out variable in TryParse is useless (or not?).
Have you others suggests (using LINQ)?
public class ParsesStringsToIntsWithLinq
{
public IEnumerable<int> Parse(string input)
{
var i = 0;
return (from segment in input.Split(',')
where int.TryParse(segment, out i)
select i);
}
}
[TestClass]
public class Tests
{
[TestMethod]
public void IgnoresNonIntegers()
{
var input = "1,2,3,4,s,6";
var output = new ParsesStringsToIntsWithLinq().Parse(input);
Assert.IsTrue(output.SequenceEqual(new []{1,2,3,4,6}));
}
}
It doesn't return a List<int> but I have to draw the line somewhere. You can make a list out of it.
Using a nice extension method
public static IEnumerable<T> AsSingleton<T>(this T source) {
yield return source;
}
(which you can replace with new[] { n } if preferred)
input.Split(',').SelectMany(s => Int32.TryParse(s, out var n) ? n.AsSingleton() : Enumerable.Empty<int>()).ToList()
I prefer to make a nice helper function:
Func<string, int?> tryParse = s => int.TryParse(s, out int n) ? (int?)n : null;
Then it's a simple matter to parse:
string input = "1,2,3,4,s,6";
List<int> myList =
input
.Split(',')
.Select(s => tryParse(s))
.Where(n => n.HasValue)
.Select(n => n.Value)
.ToList();
That gives:
1
2
3
4
6
int i = 0;
var myList = (from s in input.Split(',') where int.TryParse(s, out i) select i).ToList();
If the numbers are always single ASCII digits:
var myList = "1,2,3,4,s,6".Select(c => c ^ 48).Where(i => i < 10).ToList();
Few slower RegEx alternatives for fun:
var myList2 = Regex.Split("1,2,3,4,s,6", "[^0-9]+").Select(int.Parse).ToList(); // if the string starts and ends with digits
var myList3 = Regex.Replace("1,2,3,4,s,6", "[^0-9]+", " ").Trim().Split(' ').Select(int.Parse).ToList();
var myList4 = Regex.Matches("1,2,3,4,s,6", "[0-9]+").Cast<Match>().Select(m => int.Parse(m.Value)).ToList();
Why does it have to be LINQ?
Try:
//Come up a better name...
public static List<int> ConvertToIntListNoLinq(string input)
{
List<int> output = new List<int>();
foreach(string s in input.Split(','))
{
if(int.TryParse(s, out int result))
{
output.Add(result);
}
}
return output;
}
Fiddle
Here's a generic LINQ extension, which utilizes a delegate. This will allow you to pass in a function returning a bool, while "retaining" the result of the out variable (like int.TryParse).
Usage:
string input = "1,2,3,4,s,6";
List<int> myList = input.Split(',').SelectTry<string, int>(int.TryParse).ToList();
Code:
using System.Collections.Generic;
public static class LINQExtensions
{
public delegate bool TryFunc<TSource, TResult>(TSource source, out TResult result);
public static IEnumerable<TResult> SelectTry<TSource, TResult>(
this IEnumerable<TSource> source, TryFunc<TSource, TResult> selector)
{
foreach (TSource item in source)
{
TResult result;
if (selector(item, out result))
{
yield return result;
}
}
}
}
I think this is a clean way too. Even though it uses that extra variable, the benefit we get is it is clean and understandable.
string ids = "2,4,2,4,5,s"
const int inValidInt = -99;
var ids = ids.Split(',')
.Select(id =>
{
int parsedId = int.TryParse(id, out parsedId) ? parsedId : inValidInt;
return parsedId;
})
.Where(x => x != inValidInt).ToList();
You can do it like this:
List<int> numbers = input
.Split(',')
.Where(t => int.TryParse(t, out int a))
.Select(int.Parse)
.ToList();
You don't need to call .Split(...).ToList() as String[] is already enumerable.
You can use multiple statements in a lambda with braces.
The FindAll, ForEach and RemoveAll methods are not Linq methods, they're members of List<T>. Their Linq equivalent is Where.
Like so:
List<Int32> numbers = "1,2,3,4,s,6"
.Split(',')
.Select( s => { Int32 val; return Int32.TryParse( s, NumberStyles.Integer, CultureInfo.InvariantCulture, out val ) ? val : -1 } )
.Where( n => n != -1 )
.ToList();
You can make it more concise with a helper method:
static Int32 Parse(String s) {
Int32 ret;
if( Int32.TryParse( s, NumberStyles.Integer, CultureInfo.InvariantCulture, out ret ) ) {
return ret;
}
return -1;
}
Becomes:
List<Int32> numbers = "1,2,3,4,s,6"
.Split(',')
.Select( s => Parse( s ) )
.Where( n => n != -1 )
.ToList();
If you don't want to reserve -1 then you can use nullable ints:
static Int32? Parse(String s) {
Int32 ret;
if( Int32.TryParse( s, NumberStyles.Integer, CultureInfo.InvariantCulture, out ret ) ) {
return ret;
}
return null;
}
List<Int32> numbers = "1,2,3,4,s,6"
.Split(',') // String to String[]
.Select( s => Parse( s ) ) // String[] to IEnumerable<Int32?>
.Where( n => n != null ) // filter out nulls
.Select( n => n.Value ) // IEnumerable<Int32?> to IEnumerable<Int32>
.ToList(); // IEnumerable<Int32> to List<Int32>
I've been trying for a long time to find a "clean" pattern to handle a .SelectMany with anonymous types when you don't always want to return a result. My most common use case looks like this:
We have a list of customers that I want to do reporting on.
Each customer's data resides in a separate database, so I do a parallel .SelectMany
In each lambda expression, I gather results for the customer toward the final report.
If a particular customer should be skipped, I need to return a empty list.
I whip these up often for quick reporting, so I'd prefer an anonymous type.
For example, the logic may looks something like this:
//c is a customer
var context = GetContextForCustomer(c);
// look up some data, myData using the context connection
if (someCondition)
return myData.Select(x => new { CustomerID = c, X1 = x.x1, X2 = x.x2 });
else
return null;
This could be implemented as a foreach statement:
var results = new List<WhatType?>();
foreach (var c in customers) {
var context = GetContextForCustomer(c);
if (someCondition)
results.AddRange(myData.Select(x => new { CustomerID = c, X1 = x.x1, X2 = x.x2 }));
}
Or it could be implemented with a .SelectMany that is pre-filtered with a .Where:
customers
.Where(c => someCondition)
.AsParallel()
.SelectMany(c => {
var context = GetContextForCustomer(c);
return myData.Select(x => new { CustomerID = c, X1 = x.x1, X2 = x.x2 });
})
.ToList();
There are problems with both of these approaches. The foreach solution requires initializing a List to store the results, and you have to define the type. The .SelectMany with .Where is often impractical because the logic for someCondition is fairly complex and depends on some data lookups. So my ideal solution would look something like this:
customers
.AsParallel()
.SelectMany(c => {
var context = GetContextForCustomer(c);
if (someCondition)
return myData.Select(x => new { CustomerID = c, X1 = x.x1, X2 = x.x2 });
else
continue? return null? return empty list?
})
.ToList();
What do I put in the else line to skip a return value? None of the solutions I can come up with work or are ideal:
continue doesn't compile because it's not an active foreach loop
return null causes an NRE
return empty list requires me to initialize a list of anonymous type again.
Is there a way to accomplish the above that is clean, simple, and neat, and satisfies all my (picky) requirements?
You could return an empty Enumerable<dynamic>. Here's an example (though without your customers and someCondition, because I don't know what they are, but of the same general form of your example):
new int[] { 1, 2, 3, 4 }
.AsParallel()
.SelectMany(i => {
if (i % 2 == 0)
return Enumerable.Repeat(new { i, squared = i * i }, i);
else
return Enumerable.Empty<dynamic>();
})
.ToList();
So, with your objects and someCondition, it would look like
customers
.AsParallel()
.SelectMany(c => {
var context = GetContextForCustomer(c);
if (someCondition)
return myData.Select(x => new { CustomerID = c, X1 = x.x1, X2 = x.x2 });
else
return Enumerable.Empty<dynamic>();
})
.ToList();
Without knowing what someCondition and myData look like...
Why don't you just Select and Where the contexts as well:
customers
.Select(c => GetContextForCustomer(c))
.Where(ctx => someCondition)
.SelectMany(ctx =>
myData.Select(x => new { CustomerID = c, X1 = x.x1, X2 = x.x2 });
EDIT: I just realized you need to carry both the customer and context further, so you can do this:
customers
.Select(c => new { Customer = c, Context = GetContextForCustomer(c) })
.Where(x => someCondition(x.Context))
.SelectMany(x =>
myData.Select(d => new { CustomerID = x.Customer, X1 = d.x1, X2 = d.x2 });
You can try following:
customers
.AsParallel()
.SelectMany(c => {
var context = GetContextForCustomer(c);
if (someCondition)
return myData.Select(x => new { CustomerID = c, X1 = x.x1, X2 = x.x2 });
else
return Enumerable.Empty<int>().Select(x => new { CustomerID = 0, X1 = "defValue", X2 = "defValue" });
})
.ToList();
All anonymous types with the same set of properties (the same names and types) are combined into one one anonymous class by compiler. That's why both your Select and the one on Enumerable.Empty will return the same T.
You can create your own variarion of SelectMany LINQ method which supports nulls:
public static class EnumerableExtensions
{
public static IEnumerable<TResult> NullableSelectMany<TSource, TResult> (
this IEnumerable<TSource> source,
Func<TSource, IEnumerable<TResult>> selector)
{
if (source == null)
throw new ArgumentNullException("source");
if (selector == null)
throw new ArgumentNullException("selector");
foreach (TSource item in source) {
IEnumerable<TResult> results = selector(item);
if (results != null) {
foreach (TResult result in results)
yield return result;
}
}
}
}
Now you can return null in the selector lambda.
The accepted answer returns dynamic. The cleanest would be to move the filtering logic into a Where which makes the whole thing look better in linq context. Since you specifically rule that out in the question and I'm not a fan of delegates written over multiple lines in a linq call I will try this, but one can argue its more hacky.
var results = new
{
customerID = default(int), //notice the casing of property names
x1 = default(U), //whatever types they are
x2 = default(V)
}.GetEmptyListOfThisType();
foreach (var customerID in customers) {
var context = GetContextForCustomer(customerID);
if (someCondition)
results.AddRange(myData.Select(x => new { customerID, x.x1, x.x2 }));
}
public static List<T> GetEmptyListOfThisType<T>(this T item)
{
return new List<T>();
}
Notice the appropriate use of property names which is in accordance with other variable names, hence you dont have to write the property names a second time in the Select call.
Currently, this is just something I am curious about, I don't have any code I am working on but I am wondering how this could be achieved...
Lets say for example that I have an application that tracks the results of all the football teams in the world. What I want to be able to do is to identify the longest "win" streak for any given team.
I imagine I would most likely have some sort of data table like so:
MatchDate datetime
TeamA string
TeamB string
TeamAGoals int
TeamBGoals int
So what I would want to do for example is find the longest win streak where TeamA = "My Team" and obviously this would mean TeamAGoals must be greater than TeamBGoals.
As I have said, this is all just for example. It may be better for a different DB design for something like this. But the root question is how to calculate the longest streak/run of matching results.
This is an old question now, but I just had to solve the same problem myself, and thought people might be interested in a fully LINQ implementation of Rawling's LongestStreak extension method. This uses Aggregate with a seed and result selector to run through the list.
public static int LongestStreak<TSource>(
this IEnumerable<TSource> source,
Func<TSource, bool> predicate)
{
return source.Aggregate(
new {Longest = 0, Current = 0},
(agg, element) => predicate(element) ?
new {Longest = Math.Max(agg.Longest, agg.Current + 1), Current = agg.Current + 1} :
new {agg.Longest, Current = 0},
agg => agg.Longest);
}
There's no out-of-the-box LINQ method to count streaks, so you'll need a custom LINQy method such as
public static int LongestStreak<TSource>(
this IEnumerable<TSource> source,
Func<TSource, bool> predicate)
{
int longestStreak = 0;
int currentStreak = 0;
foreach (TSource s in source)
{
if (predicate(s))
currentStreak++;
else
{
if (currentStreak > longestStreak) longestStreak = currentStreak;
currentStreak = 0;
}
}
if (currentStreak > longestStreak) longestStreak = currentStreak;
return longestStreak;
}
Then, to use this, first turn each "match result" into a pair of "team results".
var teamResults = matches.SelectMany(m => new[] {
new {
MatchDate = m.MatchDate,
Team = m.TeamA,
Won = m.TeamAGoals > m.TeamBGoals },
new {
MatchDate = m.MatchDate,
Team = m.TeamB,
Won = m.TeamBGoals > m.TeamAGoals }
});
Group these by team.
var groupedResults = teamResults.GroupBy(r => r.Team);
Then calculate the streaks.
var streaks = groupedResults.Select(g => new
{
Team = g.Key,
StreakLength = g
// unnecessary if the matches were ordered originally
.OrderBy(r => r.MatchDate)
.LongestStreak(r => r.Won)
});
If you want the longest streak only, use MoreLinq's MaxBy; if you want them all ordered, you can use OrderByDescending(s => s.StreakLength).
Alternatively, if you want to do this in one pass, and assuming matches is already ordered, using the following class
class StreakAggregator<TKey>
{
public Dictionary<TKey, int> Best = new Dictionary<TKey, int>();
public Dictionary<TKey, int> Current = new Dictionary<TKey, int>();
public StreakAggregator<TKey> UpdateWith(TKey key, bool success)
{
int c = 0;
Current.TryGetValue(key, out c);
if (success)
{
Current[key] = c + 1;
}
else
{
int b = 0;
Best.TryGetValue(key, out b);
if (c > b)
{
Best[key] = c;
}
Current[key] = 0;
}
return this;
}
public StreakAggregator<TKey> Finalise()
{
foreach (TKey k in Current.Keys.ToArray())
{
UpdateWith(k, false);
}
return this;
}
}
you can then do
var streaks = teamResults.Aggregate(
new StreakAggregator<string>(),
(a, r) => a.UpdateWith(r.Team, r.Won),
(a) => a.Finalise().Best.Select(kvp =>
new { Team = kvp.Key, StreakLength = kvp.Value }));
and OrderBy or whatever as before.
You can get all results of team with single query:
var results = from m in Matches
let homeMatch = m.TeamA == teamName
let awayMatch = m.TeamB == teamName
let hasWon = (homeMatch && m.TeamAGoals > m.TeamBGoals) ||
(awayMatch && m.TeamBGoals > m.TeamAGoals)
where homeMatch || awayMatch
orderby m.MatchDate
select hasWon;
Then just do simple calculation of longest streak:
int longestStreak = 0;
int currentStreak = 0;
foreach (var hasWon in results)
{
if (hasWon)
{
currentStreak++;
if (currentStreak > longestStreak)
longestStreak = currentStreak;
continue;
}
currentStreak = 0;
}
You can use it as is, extract to method, or create IEnumerable extension for calculating longest sequence in results.
You could make use of string.Split. Something like this:
int longestStreak =
string.Concat(results.Select(r => (r.ours > r.theirs) ? "1" : "0"))
.Split(new[] { '0' })
.Max(s => s.Length);
Or, better, create a Split extension method for IEnumerable<T> to avoid the need to go via a string, like this:
public static IEnumerable<IEnumerable<T>> Split<T>(this IEnumerable<T> items, Predicate<T> p)
{
while (true)
{
items = items.SkipWhile(i => !p(i));
var trueItems = items.TakeWhile (i => p(i)).ToList();
if (trueItems.Count > 0)
{
yield return trueItems;
items = items.Skip(trueItems.Count);
}
else
{
break;
}
}
}
You can then simply do this:
int longestStreak = results.Split(r => r.ours > r.theirs).Max(g => g.Count());
I've an object that is include property ID with values between 101 and 199. How to order it like 199,101,102 ... 198?
In result I want to put last item to first.
The desired ordering makes no sense (some reasoning would be helpful), but this should do the trick:
int maxID = items.Max(x => x.ID); // If you want the Last item instead of the one
// with the greatest ID, you can use
// items.Last().ID instead.
var strangelyOrderedItems = items
.OrderBy(x => x.ID == maxID ? 0 : 1)
.ThenBy(x => x.ID);
Depending whether you are interested in the largest item in the list, or the last item in the list:
internal sealed class Object : IComparable<Object>
{
private readonly int mID;
public int ID { get { return mID; } }
public Object(int pID) { mID = pID; }
public static implicit operator int(Object pObject) { return pObject.mID; }
public static implicit operator Object(int pInt) { return new Object(pInt); }
public int CompareTo(Object pOther) { return mID - pOther.mID; }
public override string ToString() { return string.Format("{0}", mID); }
}
List<Object> myList = new List<Object> { 1, 2, 6, 5, 4, 3 };
// the last item first
List<Object> last = new List<Object> { myList.Last() };
List<Object> lastFirst =
last.Concat(myList.Except(last).OrderBy(x => x)).ToList();
lastFirst.ForEach(Console.Write);
Console.WriteLine();
// outputs: 312456
// or
// the largest item first
List<Object> max = new List<Object> { myList.Max() };
List<Object> maxFirst =
max.Concat(myList.Except(max).OrderBy(x => x)).ToList();
maxFirst.ForEach(Console.Write);
Console.WriteLine();
// outputs: 612345
Edit: missed the part about you wanting the last item first. You could do it like this :
var objectList = new List<DataObject>();
var lastob = objectList.Last();
objectList.Remove(lastob);
var newList = new List<DataObject>();
newList.Add(lastob);
newList.AddRange(objectList.OrderBy(o => o.Id).ToList());
If you are talking about a normal sorting you could use linq's order by method like this :
objectList = objectList.OrderBy(ob => ob.ID).ToList();
In result I want to put last item to first
first sort the list
List<int> values = new List<int>{100, 56, 89..};
var result = values.OrderBy(x=>x);
add an extension method for swaping an elements in the List<T>
static void Swap<T>(this List<T> list, int index1, int index2)
{
T temp = list[index1];
list[index1] = list[index2];
list[index2] = temp;
}
after use it
result .Swap(0, result.Count -1);
You can acheive this using a single Linq statment.
var ordering = testData
.OrderByDescending(t => t.Id)
.Take(1)
.Union(testData.OrderBy(t => t.Id).Take(testData.Count() - 1));
Order it in reverse direction and take the top 1, then order it the "right way round" and take all but the last and union these together. There are quite a few variants of this approach, but the above should work.
This approach should work for arbitrary lists too, without the need to know the max number.
How about
var orderedItems = items.OrderBy(x => x.Id)
var orderedItemsLastFirst =
orderedItems.Reverse().Take(1).Concat(orderedItems.Skip(1));
This will iterate the list several times so perhaps could be more efficient but doesn't use much code.
If more speed is important you could write a specialised IEnumerable extension that would allow you to sort and return without converting to an intermediate IEnumerable.
var myList = new List<MyObject>();
//initialize the list
var ordered = myList.OrderBy(c => c.Id); //or use OrderByDescending if you want reverse order
I'm trying to create a generic function to help me select thousands of records using LINQ to SQL from a local list. SQL Server (2005 at least) limits queries to 2100 parameters and I'd like to select more records than that.
Here would be a good example usage:
var some_product_numbers = new int[] { 1,2,3 ... 9999 };
Products.SelectByParameterList(some_product_numbers, p => p.ProductNumber);
Here is my (non-working) implementation:
public static IEnumerable<T> SelectByParameterList<T, PropertyType>(Table<T> items,
IEnumerable<PropertyType> parameterList, Expression<Func<T, PropertyType>> property) where T : class
{
var groups = parameterList
.Select((Parameter, index) =>
new
{
GroupID = index / 2000, //2000 parameters per request
Parameter
}
)
.GroupBy(x => x.GroupID)
.AsEnumerable();
var results = groups
.Select(g => new { Group = g, Parameters = g.Select(x => x.Parameter) } )
.SelectMany(g =>
/* THIS PART FAILS MISERABLY */
items.Where(item => g.Parameters.Contains(property.Compile()(item)))
);
return results;
}
I have seen plenty of examples of building predicates using expressions. In this case I only want to execute the delegate to return the value of the current ProductNumber. Or rather, I want to translate this into the SQL query (it works fine in non-generic form).
I know that compiling the Expression just takes me back to square one (passing in the delegate as Func) but I'm unsure of how to pass a parameter to an "uncompiled" expression.
Thanks for your help!
**** EDIT:** Let me clarify further:
Here is a working example of what I want to generalize:
var local_refill_ids = Refills.Select(r => r.Id).Take(20).ToArray();
var groups = local_refill_ids
.Select((Parameter, index) =>
new
{
GroupID = index / 5, //5 parameters per request
Parameter
}
)
.GroupBy(x => x.GroupID)
.AsEnumerable();
var results = groups
.Select(g => new { Group = g, Parameters = g.Select(x => x.Parameter) } )
.SelectMany(g =>
Refills.Where(r => g.Parameters.Contains(r.Id))
)
.ToArray()
;
Results in this SQL code:
SELECT [t0].[Id], ... [t0].[Version]
FROM [Refill] AS [t0]
WHERE [t0].[Id] IN (#p0, #p1, #p2, #p3, #p4)
... That query 4 more times (20 / 5 = 4)
I've come up with a way to chunk the query into pieces - i.e. you give it 4000 values, so it might do 4 requests of 1000 each; with full Northwind example. Note that this might not work on Entity Framework, due to Expression.Invoke - but is fine on LINQ to SQL:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
namespace ConsoleApplication5 {
/// SAMPLE USAGE
class Program {
static void Main(string[] args) {
// get some ids to play with...
string[] ids;
using(var ctx = new DataClasses1DataContext()) {
ids = ctx.Customers.Select(x => x.CustomerID)
.Take(100).ToArray();
}
// now do our fun select - using a deliberately small
// batch size to prove it...
using (var ctx = new DataClasses1DataContext()) {
ctx.Log = Console.Out;
foreach(var cust in ctx.Customers
.InRange(x => x.CustomerID, 5, ids)) {
Console.WriteLine(cust.CompanyName);
}
}
}
}
/// THIS IS THE INTERESTING BIT
public static class QueryableChunked {
public static IEnumerable<T> InRange<T, TValue>(
this IQueryable<T> source,
Expression<Func<T, TValue>> selector,
int blockSize,
IEnumerable<TValue> values) {
MethodInfo method = null;
foreach(MethodInfo tmp in typeof(Enumerable).GetMethods(
BindingFlags.Public | BindingFlags.Static)) {
if(tmp.Name == "Contains" && tmp.IsGenericMethodDefinition
&& tmp.GetParameters().Length == 2) {
method = tmp.MakeGenericMethod(typeof (TValue));
break;
}
}
if(method==null) throw new InvalidOperationException(
"Unable to locate Contains");
foreach(TValue[] block in values.GetBlocks(blockSize)) {
var row = Expression.Parameter(typeof (T), "row");
var member = Expression.Invoke(selector, row);
var keys = Expression.Constant(block, typeof (TValue[]));
var predicate = Expression.Call(method, keys, member);
var lambda = Expression.Lambda<Func<T,bool>>(
predicate, row);
foreach(T record in source.Where(lambda)) {
yield return record;
}
}
}
public static IEnumerable<T[]> GetBlocks<T>(
this IEnumerable<T> source, int blockSize) {
List<T> list = new List<T>(blockSize);
foreach(T item in source) {
list.Add(item);
if(list.Count == blockSize) {
yield return list.ToArray();
list.Clear();
}
}
if(list.Count > 0) {
yield return list.ToArray();
}
}
}
}
Easiest way to do this: Use LINQKit (Free, non-restrictive license)
Working version of code:
public static IEnumerable<T> SelectByParameterList<T, PropertyType>(this Table<T> items, IEnumerable<PropertyType> parameterList, Expression<Func<T, PropertyType>> propertySelector, int blockSize) where T : class
{
var groups = parameterList
.Select((Parameter, index) =>
new
{
GroupID = index / blockSize, //# of parameters per request
Parameter
}
)
.GroupBy(x => x.GroupID)
.AsEnumerable();
var selector = LinqKit.Linq.Expr(propertySelector);
var results = groups
.Select(g => new { Group = g, Parameters = g.Select(x => x.Parameter) } )
.SelectMany(g =>
/* AsExpandable() extension method requires LinqKit DLL */
items.AsExpandable().Where(item => g.Parameters.Contains(selector.Invoke(item)))
);
return results;
}
Example usage:
Guid[] local_refill_ids = Refills.Select(r => r.Id).Take(20).ToArray();
IEnumerable<Refill> results = Refills.SelectByParameterList(local_refill_ids, r => r.Id, 10); //runs 2 SQL queries with 10 parameters each
Thanks again for all your help!
LINQ-to-SQL still works via standard SQL parameters, so writing a fancy expression isn't going to help. There are 3 common options here:
pack the ids into (for example) csv/tsv; pass down as a varchar(max) and use a udf to split it (at the server) into a table variable; join to the table variable
use a table-valued-parameter in SQL Server 2008
have a table on the server that you could push the ids into (perhaps via SqlBulkCopy) (perhaps with a "session guid" or similar); join to this table
The first is the simplest; getting a "split csv udf" is trivial (just search for it). Drag the udf onto the data-context and consume from there.
Pass IQuerable to the Contains function instead of list or array. please see the below example
var df_handsets = db.DataFeed_Handsets.Where(m => m.LaunchDate != null).
Select(m => m.Name);
var Make = (from m in db.MobilePhones
where (m.IsDeleted != true || m.IsDeleted == null)
&& df_handsets.Contains(m.Name)
orderby m.Make
select new { Value = m.Make, Text = m.Make }).Distinct();
when you pass list or array it is passed in form of parameters and its exceed the counts when the list items count is greater than 2100.
You can create your own QueryProvider
public class QueryProvider : IQueryProvider
{
// Translates LINQ query to SQL.
private readonly Func<IQueryable, DbCommand> _translator;
// Executes the translated SQL and retrieves results.
private readonly Func<Type, string, object[], IEnumerable> _executor;
public QueryProvider(
Func<IQueryable, DbCommand> translator,
Func<Type, string, object[], IEnumerable> executor)
{
this._translator = translator;
this._executor = executor;
}
#region IQueryProvider Members
public IQueryable<TElement> CreateQuery<TElement>(Expression expression)
{
return new Queryable<TElement>(this, expression);
}
public IQueryable CreateQuery(Expression expression)
{
throw new NotImplementedException();
}
public TResult Execute<TResult>(Expression expression)
{
bool isCollection = typeof(TResult).IsGenericType &&
typeof(TResult).GetGenericTypeDefinition() == typeof(IEnumerable<>);
var itemType = isCollection
// TResult is an IEnumerable`1 collection.
? typeof(TResult).GetGenericArguments().Single()
// TResult is not an IEnumerable`1 collection, but a single item.
: typeof(TResult);
var queryable = Activator.CreateInstance(
typeof(Queryable<>).MakeGenericType(itemType), this, expression) as IQueryable;
IEnumerable queryResult;
// Translates LINQ query to SQL.
using (var command = this._translator(queryable))
{
var parameters = command.Parameters.OfType<DbParameter>()
.Select(parameter => parameter)
.ToList();
var query = command.CommandText;
var newParameters = GetNewParameterList(ref query, parameters);
queryResult = _executor(itemType,query,newParameters);
}
return isCollection
? (TResult)queryResult // Returns an IEnumerable`1 collection.
: queryResult.OfType<TResult>()
.SingleOrDefault(); // Returns a single item.
}
public object Execute(Expression expression)
{
throw new NotImplementedException();
}
#endregion
private static object[] GetNewParameterList(ref string query, List<DbParameter> parameters)
{
var newParameters = new List<DbParameter>(parameters);
foreach (var dbParameter in parameters.Where(p => p.DbType == System.Data.DbType.Int32))
{
var name = dbParameter.ParameterName;
var value = dbParameter.Value != null ? dbParameter.Value.ToString() : "NULL";
var pattern = String.Format("{0}[^0-9]", dbParameter.ParameterName);
query = Regex.Replace(query, pattern, match => value + match.Value.Replace(name, ""));
newParameters.Remove(dbParameter);
}
for (var i = 0; i < newParameters.Count; i++)
{
var parameter = newParameters[i];
var oldName = parameter.ParameterName;
var pattern = String.Format("{0}[^0-9]", oldName);
var newName = "#p" + i;
query = Regex.Replace(query, pattern, match => newName + match.Value.Replace(oldName, ""));
}
return newParameters.Select(x => x.Value).ToArray();
}
}
static void Main(string[] args)
{
using (var dc=new DataContext())
{
var provider = new QueryProvider(dc.GetCommand, dc.ExecuteQuery);
var serviceIds = Enumerable.Range(1, 2200).ToArray();
var tasks = new Queryable<Task>(provider, dc.Tasks).Where(x => serviceIds.Contains(x.ServiceId) && x.CreatorId==37 && x.Creator.Name=="12312").ToArray();
}
}