Develop Reference

Getting all Data from SQL using LINQ with comma-separated IDs

I do have a string of Empids separated by comma like:
EMpID:"2007,2008,2002,1992,1000,2108,1085
and I need to retrieve the records of all those specified employees using LINQ query.
I tried it with looping but I need to get that in efficient and faster way.
Here goes what i did using looping.
string[] EMpID_str = LeaveDictionary["EMpID"].ToString().Split(',');
for (int i = 0; i < EMpID_str.Length; i++)
{
EMpID = Convert.ToInt32(EMpID_str[i]);
//Linq to get data for each Empid goes here
}
But What I need is to use single LINQ or Lambda query to retrieve the same.Without looping

First convert your ,(comma) separated empId to string array like below:
var empArr = EmpId.split(',');
var employeesResult = emplyeeList.Where(x => empArr.contains(x.EmpId.ToString()));
I hope, it will help someone.

If the Ids that you want to fetch are numbers, not strings, then you should not convert the string to an array of strings, but to a sequence of numbers:
IEnumerable<int> employeeIdsToFetch = LeaveDictionary["EMpID"].ToString()
.Split(',')
.Select(splitText => Int32.Parse(splitText));
To fetch all employees with thees Ids:
var fetchedEmployees = dbContext.Employees
.Where(employee => employeeIdsToFetch.Contains(employee.Id))
.Select(employee => new
{
// Select only the employee properties that you plan to use:
Id = employee.Id,
Name = employee.Name,
...
});

You can use the Expression class to build a Func<int, bool> from your string and use it with the Where methode:
var str = "2,5,8,9,4,6,7";
var para = Expression.Parameter(typeof(int));
var body = str.Split(",")
.Select(s => int.Parse(s))
.Select(i => Expression.Constant(i))
.Select(c => Expression.Equal(para, c))
.Aggregate((a, b) => Expression.Or(a, b));
Func<int, bool> func = Expression.Lambda<Func<int, bool>>(body, para).Compile();
and if you this solution to work with linq to SQL just dont compile the expression at the end and let the linq to SQL engine compile it to an efficent SQL expression.
Instead of the Aggregate Method (which will produce an expression with linear complexity) one could use an divide and conquer approach to fold the values into one value.
For example with this class:
public static class Helper
{
public static T EfficientFold<T>(this List<T> list, Func<T, T, T> func)
{
return EfficientFold(list, 0, list.Count, func);
}
private static T EfficientFold<T>(List<T> list, int lowerbound, int upperbound, Func<T, T, T> func)
{
int diff = upperbound - lowerbound;
var mid = lowerbound + diff / 2;
if (diff < 1)
{
throw new Exception();
}
else if (diff == 1)
{
return list[lowerbound];
}
else
{
var left = EfficientFold(list, lowerbound, mid, func);
var right = EfficientFold(list, mid, upperbound, func);
return func(left, right);
}
}
}
and then we can do
var body = str.Split(",")
.Select(s => int.Parse(s))
.Select(i => Expression.Constant(i))
.Select(c => Expression.Equal(para, c))
.ToList()
.EfficientFold((a, b) => Expression.Or(a, b));
which gives the evaluation a complexity of log(n).

Check if the item in the List is incremental and not redundant

I need to retain all the list that are redundant and not incremental. But my code so far is for the items that are redundant only
_lst.Add(new MSheetValue
{
Column = 1,
Line = "1",
Pdf = "PDF1"
});
_lst.Add(new MSheetValue
{
Column = 1,
Line = "1",
Pdf = "PDF1"
});
_lst.Add(new MSheetValue
{
Column = 1,
Line = "1",
Pdf = "PDF2"
});
_lst.Add(new MSheetValue
{
Column = 1,
Line = "2",
Pdf = "PDF2"
});
_lst.Add(new MSheetValue
{
Column = 1,
Line = "3",
Pdf = "PDF2"
});
_lst.Add(new MSheetValue
{
Column = 1,
Line = "1",
Pdf = "PDF3"
});
_lst.Add(new MSheetValue
{
Column = 1,
Line = "3",
Pdf = "PDF3"
});
Here is my code
var result = _lst.GroupBy(x => new { x.Line, x.Pdf })
.Where(x => x.Skip(1).Any()).ToList();
and the result is
Column = 1,
Line = "1",
Pdf = "PDF1"
But i also need the list that are not incremental
so i also need this
Column = 1,
Line = "1",
Pdf = "PDF3"
Column = 1,
Line = "3",
Pdf = "PDF3"
How can i solve it. I tried searching for a solution and test what i've found but i can't solve it. it doesn't return what i expected

var distinctItems = _lst.Distinct();
To match on only some of the properties, create a custom equality comparer, e.g.:
class DistinctItemComparer : IEqualityComparer<Item> {
public bool Equals(Item x, Item y) {
return x.Column == y.Column &&
x.Line == y.Line &&
x.Pdf == y.Pdf;
}
public int GetHashCode(Item obj) {
return obj.Column.GetHashCode() ^
obj.Line.GetHashCode() ^
obj.Pdf.GetHashCode();
}
}
Then use it like this:
var distinctItems = _lst.Distinct(new DistinctItemComparer());
Or try it:
var distinctItems = _lst.GroupBy(x => x.Id).Select(y => y.First());

using zip to get the adjacent items and then comparing the adjacent items and selecting the items that are not adjacent may do the trick. This example is a little oversimplified as you may want to compare the field with Pdfs as well. The Union adds the duplicates to the non-adjacents.
return _lst.Zip(_lst.Skip(1), (a, b) => new { a, b})
.Where(w => w.b.Line != w.a.Line + 1)
.Select(w => w.b)
.Union(_lst.GroupBy(x => new { x.Line, x.Pdf })
.Where(x => x.Skip(1).Any()).ToList()
.SelectMany(s => s));

Using some handy extension methods:
public static class Ext {
public static IEnumerable<(TKey Key, T Value)> ScanPair<T, TKey>(this IEnumerable<T> src, TKey seedKey, Func<(TKey Key, T Value), T, TKey> combine) {
using (var srce = src.GetEnumerator()) {
if (srce.MoveNext()) {
var prevkv = (seedKey, srce.Current);
while (srce.MoveNext()) {
yield return prevkv;
prevkv = (combine(prevkv, srce.Current), srce.Current);
}
yield return prevkv;
}
}
}
public static IEnumerable<IGrouping<int, TRes>> GroupByWhile<T, TRes>(this IEnumerable<T> src, Func<T, T, bool> test, Func<T, TRes> result) =>
src.ScanPair(1, (kvp, cur) => test(kvp.Value, cur) ? kvp.Key : kvp.Key + 1).GroupBy(kvp => kvp.Key, kvp => result(kvp.Value));
public static IEnumerable<IGrouping<int, TRes>> GroupBySequential<T, TRes>(this IEnumerable<T> src, Func<T, int> SeqNum, Func<T, TRes> result) =>
src.GroupByWhile((prev, cur) => SeqNum(prev) + 1 == SeqNum(cur), result);
public static IEnumerable<IGrouping<int, T>> GroupBySequential<T>(this IEnumerable<T> src, Func<T, int> SeqNum) => src.GroupBySequential(SeqNum, e => e);
public static IEnumerable<T> DistinctBy<T, TKey>(this IEnumerable<T> src, Func<T, TKey> keyFun, IEqualityComparer<TKey> comparer = null) {
var seenKeys = new HashSet<TKey>(comparer);
foreach (var e in src)
if (seenKeys.Add(keyFun(e)))
yield return e;
}
public static int ToInteger(this string s) => Convert.ToInt32(s);
}
ScanPair is a variation of my APL inspired Scan operator (which is like Aggregate only returns the intermediate results). I discovered I was doing a lot of Scan with tuples to carry the original information, so ScanPair combines the intermediate results with the original values.
Using ScanPair, GroupByWhile runs a test on each element and groups while the test is true.
Using GroupByWhile, GroupBySequential groups when each elements sequence number is sequential.
DistinctBy returns the distinct objects based on a key selection function. I cheat and use this rather than create an IEqualityComparer for MSheetValue.
Finally, ToInteger is just a handy extension for reading flow.
With these extension methods, processing the _lst is relatively straightforward:
var nonSeq = _lst.GroupBy(m => m.Pdf) // need to test each Pdf
.Select(mg => mg.GroupBySequential(m => m.Line.ToInteger())) // get the sequential groups
.Where(mg => mg.Count() > 1) // keep the ones with non-sequential lines
// parse each non-sequential group into just the unique entries and flatten
.Select(mg => mg.SelectMany(m => m).DistinctBy(m => new { m.Column, m.Line, m.Pdf }));

Convert a string in a List<int> using LINQ (cleaner way)

I have this string:
string input = "1,2,3,4,s,6";
Pay attention to the s character.
I just want to convert this string in a List<int> using LINQ. I initially tried in this way:
var myList = new List<int>();
input.Split(',').ToList().ForEach(n =>
myList.Add(int.TryParse(n, out int num) ? num : -1)
);
lista.RemoveAll(e => e == -1);
But I prefer not have any -1 instead of a no-number characters.
So now I try with this:
var myList = new List<int>();
input.Split(',').ToList()
.FindAll(n => int.TryParse(n, out int _))
.ForEach(num => myList.Add(int.Parse(num)));
I prefer this, but is really a shame that the parsing happening two times (TryParse at first and then Parse). But, from what I understand, the out variable in TryParse is useless (or not?).
Have you others suggests (using LINQ)?

public class ParsesStringsToIntsWithLinq
{
public IEnumerable<int> Parse(string input)
{
var i = 0;
return (from segment in input.Split(',')
where int.TryParse(segment, out i)
select i);
}
}
[TestClass]
public class Tests
{
[TestMethod]
public void IgnoresNonIntegers()
{
var input = "1,2,3,4,s,6";
var output = new ParsesStringsToIntsWithLinq().Parse(input);
Assert.IsTrue(output.SequenceEqual(new []{1,2,3,4,6}));
}
}
It doesn't return a List<int> but I have to draw the line somewhere. You can make a list out of it.

Using a nice extension method
public static IEnumerable<T> AsSingleton<T>(this T source) {
yield return source;
}
(which you can replace with new[] { n } if preferred)
input.Split(',').SelectMany(s => Int32.TryParse(s, out var n) ? n.AsSingleton() : Enumerable.Empty<int>()).ToList()

I prefer to make a nice helper function:
Func<string, int?> tryParse = s => int.TryParse(s, out int n) ? (int?)n : null;
Then it's a simple matter to parse:
string input = "1,2,3,4,s,6";
List<int> myList =
input
.Split(',')
.Select(s => tryParse(s))
.Where(n => n.HasValue)
.Select(n => n.Value)
.ToList();
That gives:
1
2
3
4
6

int i = 0;
var myList = (from s in input.Split(',') where int.TryParse(s, out i) select i).ToList();
If the numbers are always single ASCII digits:
var myList = "1,2,3,4,s,6".Select(c => c ^ 48).Where(i => i < 10).ToList();
Few slower RegEx alternatives for fun:
var myList2 = Regex.Split("1,2,3,4,s,6", "[^0-9]+").Select(int.Parse).ToList(); // if the string starts and ends with digits
var myList3 = Regex.Replace("1,2,3,4,s,6", "[^0-9]+", " ").Trim().Split(' ').Select(int.Parse).ToList();
var myList4 = Regex.Matches("1,2,3,4,s,6", "[0-9]+").Cast<Match>().Select(m => int.Parse(m.Value)).ToList();

Why does it have to be LINQ?
Try:
//Come up a better name...
public static List<int> ConvertToIntListNoLinq(string input)
{
List<int> output = new List<int>();
foreach(string s in input.Split(','))
{
if(int.TryParse(s, out int result))
{
output.Add(result);
}
}
return output;
}
Fiddle

Here's a generic LINQ extension, which utilizes a delegate. This will allow you to pass in a function returning a bool, while "retaining" the result of the out variable (like int.TryParse).
Usage:
string input = "1,2,3,4,s,6";
List<int> myList = input.Split(',').SelectTry<string, int>(int.TryParse).ToList();
Code:
using System.Collections.Generic;
public static class LINQExtensions
{
public delegate bool TryFunc<TSource, TResult>(TSource source, out TResult result);
public static IEnumerable<TResult> SelectTry<TSource, TResult>(
this IEnumerable<TSource> source, TryFunc<TSource, TResult> selector)
{
foreach (TSource item in source)
{
TResult result;
if (selector(item, out result))
{
yield return result;
}
}
}
}

I think this is a clean way too. Even though it uses that extra variable, the benefit we get is it is clean and understandable.
string ids = "2,4,2,4,5,s"
const int inValidInt = -99;
var ids = ids.Split(',')
.Select(id =>
{
int parsedId = int.TryParse(id, out parsedId) ? parsedId : inValidInt;
return parsedId;
})
.Where(x => x != inValidInt).ToList();

You can do it like this:
List<int> numbers = input
.Split(',')
.Where(t => int.TryParse(t, out int a))
.Select(int.Parse)
.ToList();

You don't need to call .Split(...).ToList() as String[] is already enumerable.
You can use multiple statements in a lambda with braces.
The FindAll, ForEach and RemoveAll methods are not Linq methods, they're members of List<T>. Their Linq equivalent is Where.
Like so:
List<Int32> numbers = "1,2,3,4,s,6"
.Split(',')
.Select( s => { Int32 val; return Int32.TryParse( s, NumberStyles.Integer, CultureInfo.InvariantCulture, out val ) ? val : -1 } )
.Where( n => n != -1 )
.ToList();
You can make it more concise with a helper method:
static Int32 Parse(String s) {
Int32 ret;
if( Int32.TryParse( s, NumberStyles.Integer, CultureInfo.InvariantCulture, out ret ) ) {
return ret;
}
return -1;
}
Becomes:
List<Int32> numbers = "1,2,3,4,s,6"
.Split(',')
.Select( s => Parse( s ) )
.Where( n => n != -1 )
.ToList();
If you don't want to reserve -1 then you can use nullable ints:
static Int32? Parse(String s) {
Int32 ret;
if( Int32.TryParse( s, NumberStyles.Integer, CultureInfo.InvariantCulture, out ret ) ) {
return ret;
}
return null;
}
List<Int32> numbers = "1,2,3,4,s,6"
.Split(',') // String to String[]
.Select( s => Parse( s ) ) // String[] to IEnumerable<Int32?>
.Where( n => n != null ) // filter out nulls
.Select( n => n.Value ) // IEnumerable<Int32?> to IEnumerable<Int32>
.ToList(); // IEnumerable<Int32> to List<Int32>

How to get distinct values with corresponding data from IEnumerable

I need to be able to return back only the records that have a unique AccessionNumber with it's corresponding LoginId. So that at the end, the data looks something like:
A1,L1
A2,L1
A3,L2
However, my issue is with this line of code because Distinct() returns a IEnumerable of string and not IEnumerable of string[]. Therefore, compiler complains about string not containing a definition for AccessionNumber and LoginId.
yield return new[] { record.AccessionNumber, record.LoginId };
This is the code that I am trying to execute:
internal static IEnumerable<string[]> GetTestDataForSpecificItemType(ItemTypes itemTypeCode)
{
IEnumerable<StudentAssessmentTestData> data = DataGetter.GetTestData("MyTestData");
data = data.Where(x => x.ItemTypeCode.Trim() == itemTypeCode.ToString());
var z = data.Select(x => x.AccessionNumber).Distinct();
foreach (var record in z)
{
yield return new[] { record.AccessionNumber, record.LoginId };
}
}

That's cause you are selecting only that property AccessionNumber by saying the below
var z = data.Select(x => x.AccessionNumber).Distinct();
You probably want to select entire StudentAssessmentTestData record
data = data.Where(x => x.ItemTypeCode.Trim() == itemTypeCode.ToString()).Distinct();
foreach (var record in data)
{
yield return new[] { record.AccessionNumber, record.LoginId };
}

Instead of using Distinct, use GroupBy. This:
var z = data.Select(x => x.AccessionNumber).Distinct();
foreach (var record in z)
{
yield return new[] { record.AccessionNumber, record.LoginId };
}
should be something like this:
return data.GroupBy(x => x.AccessionNumber)
.Select(r => new { AccessionNumber = r.Key, r.First().LoginId});
The GroupBy() call ensures only unique entries for AccessionNumber and the First() ensures that only the first one LoginId with that AccessionNumber is returned.
This assumes that your data is sorted in a way that if there are multiple logins with the same AccessionNumber, the first login is correct.

If you want to choose distinct values based on a certain property you can do it in several ways.
If it is always the same property you wish to use for comparision, you can override Equals and GetHashCode methods in the StudentAssessmentTestData class, thus allowing the Distinct method to recognize how the classes differ from each other, an example can be found in this question
However, you can also implement a custom IEqualityComparer<T> for your implementation, for example the following version
// Custom comparer taking generic input parameter and a delegate function to do matching
public class CustomComparer<T> : IEqualityComparer<T> {
private readonly Func<T, object> _match;
public CustomComparer(Func<T, object> match) {
_match = match;
}
// tries to match both argument its return values against eachother
public bool Equals(T data1, T data2) {
return object.Equals(_match(data1), _match(data2));
}
// overly simplistic implementation
public int GetHashCode(T data) {
var matchValue = _match(data);
if (matchValue == null) {
return 42.GetHashCode();
}
return matchValue.GetHashCode();
}
}
This class can then be used as an argument for the Distinct function, for example in this way
// compare by access number
var accessComparer = new CustomComparer<StudentTestData>(d => d.AccessionNumber );
// compare by login id
var loginComparer = new CustomComparer<StudentTestData>(d => d.LoginId );
foreach (var d in data.Distinct( accessComparer )) {
Console.WriteLine( "{0}, {1}", d.AccessionNumber, d.LoginId);
}
foreach (var d in data.Distinct( loginComparer )) {
Console.WriteLine( "{0}, {1}", d.AccessionNumber, d.LoginId);
}
A full example you can find in this dotnetfiddle

Add a LinqExtension method DistinctBy as below.
public static class LinqExtensions
{
public static IEnumerable<TSource> DistinctBy<TSource, TKey>(this IEnumerable<TSource> source, Func<TSource, TKey> keySelector)
{
HashSet<TKey> seenKeys = new HashSet<TKey>();
foreach (TSource element in source)
{
if (seenKeys.Add(keySelector(element)))
{
yield return element;
}
}
}
}
Use it in your code like this:
var z = data.DistinctBy(x => x.AccessionNumber);
internal static IEnumerable<string[]> GetTestDataForSpecificItemType(ItemTypes itemTypeCode)
{
IEnumerable<StudentAssessmentTestData> data = DataGetter.GetTestData("MyTestData");
data = data.Where(x => x.ItemTypeCode.Trim() == itemTypeCode.ToString());
var z = data.DistinctBy(x => x.AccessionNumber);
foreach (var record in z)
{
yield return new[] { record.AccessionNumber, record.LoginId };
}
}

This is the code that finally worked:
internal static IEnumerable<string[]> GetTestDataForSpecificItemType(ItemTypes itemTypeCode)
{
var data = DataGetter.GetTestData("MyTestData");
data = data.Where(x => x.ItemTypeCode.Trim() == itemTypeCode.ToString());
var z = data.GroupBy(x => new{x.AccessionNumber})
.Select(x => new StudentAssessmentTestData(){ AccessionNumber = x.Key.AccessionNumber, LoginId = x.FirstOrDefault().LoginId});
foreach (var record in z)
{
yield return new[] { record.AccessionNumber, record.LoginId };
}
}
Returns a sequence that looks like similar to this:
Acc1, Login1
Acc2, Login1
Acc3, Login2
Acc4, Login1
Acc5, Login3

You can try this. It works for me.
IEnumerable<StudentAssessmentTestData> data = DataGetter.GetTestData("MyTestData");
data = data.Where(x => x.ItemTypeCode.Trim() == itemTypeCode.ToString());
var z = data.GroupBy(x => x.AccessionNumber).SelectMany(y => y.Take(1));
foreach (var record in z)
{
yield return new[] { record.AccessionNumber, record.LoginId };
}

I'm not 100% sure what you're asking. You either want (1) only records with a unique AccessionNumber , if two or more records had the same AccessionNumber then don't return them, or (2) only the first record for each AccessionNumber.
Here's both options:
(1)
internal static IEnumerable<string[]> GetTestDataForSpecificItemType(ItemTypes itemTypeCode)
{
return
DataGetter
.GetTestData("MyTestData");
.Where(x => x.ItemTypeCode.Trim() == itemTypeCode.ToString())
.GroupBy(x => x.AccessionNumber)
.Where(x => !x.Skip(1).Any())
.SelectMany(x => x)
.Select(x => new [] { x.AccessionNumber, x.LoginId });
}
(2)
internal static IEnumerable<string[]> GetTestDataForSpecificItemType(ItemTypes itemTypeCode)
{
return
DataGetter
.GetTestData("MyTestData");
.Where(x => x.ItemTypeCode.Trim() == itemTypeCode.ToString())
.GroupBy(x => x.AccessionNumber)
.SelectMany(x => x.Take(1))
.Select(x => new [] { x.AccessionNumber, x.LoginId });
}

How to create a For_Each_With_Condition_With_Index Extension method (EM)

I have a ForEachWithIndex EM
static void ForEachWithIndex<T>(this IEnumerable<T> enu, Action<T, int> action)
{
int i = 0;
foreach(T item in enu)
action(item, i++);
}
I call it like this
my_int_array.ForEachWithIndex((x, i) => x += i);
Now i want to create one which checks for condition and then perform that action.
Usually i use above as
my_int_array.ForEachWithIndex((x,i) =>
{
if (x != 0)
x += i;
});
I want a EM that takes that condition as parameter also. How to do that?

I would try to avoid building one big extension method which does it all. Break it out, just like LINQ does.
Personally I wouldn't actually do any of this though - I'd build a query with LINQ, then use a foreach statement for the action:
// Assuming you want the *original* indexes
var query = array.Select((Value, Index) => new { value, Index })
.Where(pair => pair.Index != 0);
foreach (var pair in query)
{
// Do something
}
It's hard to know exactly what you're trying to do, given that incrementing the lambda parameter won't really achieve anything. I would strongly encourage you to think of composing blocks though... and you may find Eric Lippert's views on foreach vs ForEach interesting.

Just add the condition delegate to parameters list:
static void ForEachWithIndexWithCondition<T>(this IEnumerable<T> enu,
Func<T, int, bool> condition, Action<T, int> action)
{
int i = 0;
foreach (T item in enu)
{
if (condition(item, i))
action(item, i);
i++;
}
}
Usage:
var list = new List<string> { "Jonh", "Mary", "Alice", "Peter" };
list.ForEachWithIndexWithCondition(
(s, i) => i % 2 == 0,
(s, i) => Console.WriteLine(s));

You need to pass an additional Func parameter, something like this:
public static void ForEachWithIndex<T>(this IEnumerable<T> enu,
Action<T, int> action, Func<T, int, bool> condition)
{
int i = 0;
foreach (T item in enu)
{
if (condition(item, i))
{
action(item, i);
}
++i;
}
}
And this is what the code for your example look like:
my_int_array.ForEachWithIndex((x, i) => x += i, (x, i) => x != 0);