Develop Reference

Can't use .ToList() with IQueryable<T> [duplicate]

I'm migrating some stuff from one mysql server to a sql server but i can't figure out how to make this code work:
using (var context = new Context())
{
...
foreach (var item in collection)
{
IQueryable<entity> pages = from p in context.pages
where p.Serial == item.Key.ToString()
select p;
foreach (var page in pages)
{
DataManager.AddPageToDocument(page, item.Value);
}
}
Console.WriteLine("Done!");
Console.Read();
}
When it enters into the second foreach (var page in pages) it throws an exception saying:
LINQ to Entities does not recognize the method 'System.String
ToString()' method, and this method cannot be translated into a store
expression.
Anyone know why this happens?

Just save the string to a temp variable and then use that in your expression:
var strItem = item.Key.ToString();
IQueryable<entity> pages = from p in context.pages
where p.Serial == strItem
select p;
The problem arises because ToString() isn't really executed, it is turned into a MethodGroup and then parsed and translated to SQL. Since there is no ToString() equivalent, the expression fails.
Note:
Make sure you also check out Alex's answer regarding the SqlFunctions helper class that was added later. In many cases it can eliminate the need for the temporary variable.

As others have answered, this breaks because .ToString fails to translate to relevant SQL on the way into the database.
However, Microsoft provides the SqlFunctions class that is a collection of methods that can be used in situations like this.
For this case, what you are looking for here is SqlFunctions.StringConvert:
from p in context.pages
where p.Serial == SqlFunctions.StringConvert((double)item.Key.Id)
select p;
Good when the solution with temporary variables is not desirable for whatever reasons.
Similar to SqlFunctions you also have the EntityFunctions (with EF6 obsoleted by DbFunctions) that provides a different set of functions that also are data source agnostic (not limited to e.g. SQL).

The problem is that you are calling ToString in a LINQ to Entities query. That means the parser is trying to convert the ToString call into its equivalent SQL (which isn't possible...hence the exception).
All you have to do is move the ToString call to a separate line:
var keyString = item.Key.ToString();
var pages = from p in context.entities
where p.Serial == keyString
select p;

Cast table to Enumerable, then you call LINQ methods with using ToString() method inside:
var example = contex.table_name.AsEnumerable()
.Select(x => new {Date = x.date.ToString("M/d/yyyy")...)
But be careful, when you calling AsEnumerable or ToList methods because you will request all data from all entity before this method. In my case above I read all table_name rows by one request.

Had a similar problem.
Solved it by calling ToList() on the entity collection and querying the list.
If the collection is small this is an option.
IQueryable<entity> pages = context.pages.ToList().Where(p=>p.serial == item.Key.ToString())
Hope this helps.

Upgrading to Entity Framework Version 6.2.0 worked for me.
I was previously on Version 6.0.0.
Hope this helps,

Change it like this and it should work:
var key = item.Key.ToString();
IQueryable<entity> pages = from p in context.pages
where p.Serial == key
select p;
The reason why the exception is not thrown in the line the LINQ query is declared but in the line of the foreach is the deferred execution feature, i.e. the LINQ query is not executed until you try to access the result. And this happens in the foreach and not earlier.

If you really want to type ToString inside your query, you could write an expression tree visitor that rewrites the call to ToString with a call to the appropriate StringConvert function:
using System.Linq;
using System.Data.Entity.SqlServer;
using System.Linq.Expressions;
using static System.Linq.Expressions.Expression;
using System;
namespace ToStringRewriting {
class ToStringRewriter : ExpressionVisitor {
static MethodInfo stringConvertMethodInfo = typeof(SqlFunctions).GetMethods()
.Single(x => x.Name == "StringConvert" && x.GetParameters()[0].ParameterType == typeof(decimal?));
protected override Expression VisitMethodCall(MethodCallExpression node) {
var method = node.Method;
if (method.Name=="ToString") {
if (node.Object.GetType() == typeof(string)) { return node.Object; }
node = Call(stringConvertMethodInfo, Convert(node.Object, typeof(decimal?));
}
return base.VisitMethodCall(node);
}
}
class Person {
string Name { get; set; }
long SocialSecurityNumber { get; set; }
}
class Program {
void Main() {
Expression<Func<Person, Boolean>> expr = x => x.ToString().Length > 1;
var rewriter = new ToStringRewriter();
var finalExpression = rewriter.Visit(expr);
var dcx = new MyDataContext();
var query = dcx.Persons.Where(finalExpression);
}
}
}

In MVC, assume you are searching record(s) based on your requirement or information.
It is working properly.
[HttpPost]
[ActionName("Index")]
public ActionResult SearchRecord(FormCollection formcollection)
{
EmployeeContext employeeContext = new EmployeeContext();
string searchby=formcollection["SearchBy"];
string value=formcollection["Value"];
if (formcollection["SearchBy"] == "Gender")
{
List<MvcApplication1.Models.Employee> emplist = employeeContext.Employees.Where(x => x.Gender == value).ToList();
return View("Index", emplist);
}
else
{
List<MvcApplication1.Models.Employee> emplist = employeeContext.Employees.Where(x => x.Name == value).ToList();
return View("Index", emplist);
}
}

I got the same error in this case:
var result = Db.SystemLog
.Where(log =>
eventTypeValues.Contains(log.EventType)
&& (
search.Contains(log.Id.ToString())
|| log.Message.Contains(search)
|| log.PayLoad.Contains(search)
|| log.Timestamp.ToString(CultureInfo.CurrentUICulture).Contains(search)
)
)
.OrderByDescending(log => log.Id)
.Select(r => r);
After spending way too much time debugging, I figured out that error appeared in the logic expression.
The first line search.Contains(log.Id.ToString()) does work fine, but the last line that deals with a DateTime object made it fail miserably:
|| log.Timestamp.ToString(CultureInfo.CurrentUICulture).Contains(search)
Remove the problematic line and problem solved.
I do not fully understand why, but it seems as ToString() is a LINQ expression for strings, but not for Entities. LINQ for Entities deals with database queries like SQL, and SQL has no notion of ToString(). As such, we can not throw ToString() into a .Where() clause.
But how then does the first line work? Instead of ToString(), SQL have CAST and CONVERT, so my best guess so far is that linq for entities uses that in some simple cases. DateTime objects are not always found to be so simple...

My problem was that I had a 'text' data type for this column (due to a migration from sqlite).
Solution: just change the data type to 'nvarchar()' and regenerate the table.
Then Linq accepts the string comparison.

I am working on retiring Telerik Open Access and replacing it with Entity Framework 4.0. I came across same issue that telerik:GridBoundColumn filtering stopped working.
I find out that its not working only on System.String DataTypes. So I found this thread and solved it by just using .List() at the end of my Linq query as follows:
var x = (from y in db.Tables
orderby y.ColumnId descending
select new
{
y.FileName,
y.FileSource,
y.FileType,
FileDepartment = "Claims"
}).ToList();

Just turn the LINQ to Entity query into a LINQ to Objects query (e.g. call ToArray) anytime you need to use a method call in your LINQ query.

Not able to Select after using Expression Tree in Group By

I have been referring this post to group by using expression tree. Here is my code:
String[] fields = { "DepartmentID", "SkillID" };
var groupLambda = GroupByExpression<Person>(fields);
var query = dbContext.People.GroupBy(groupLambda.Compile());
var queryResult = query.ToList();
Here is the method GroupByExpression which uses solution given in aforesaid post (Thanks Daniel!):
public static Expression<Func<TItem, object>> GroupByExpression<TItem>(string[] propertyNames)
{
var properties = propertyNames.Select(name => typeof(TItem).GetProperty(name)).ToArray();
var propertyTypes = properties.Select(p => p.PropertyType).ToArray();
var tupleTypeDefinition = typeof(Tuple).Assembly.GetType("System.Tuple`" + properties.Length);
var tupleType = tupleTypeDefinition.MakeGenericType(propertyTypes);
var constructor = tupleType.GetConstructor(propertyTypes);
var param = Expression.Parameter(typeof(TItem), "x");
var body = Expression.New(constructor, properties.Select(p => Expression.Property(param, p)));
var expr = Expression.Lambda<Func<TItem, object>>(body, param);
return expr;
}
I want to be able to identify fields in the group by keys with strong names in select part like query.Select(x => new { x.Key.DepartmentID, x.Key.SkillID });
How do I do this?

Now... I won't give you the solution to the question you asked, but I'll try to help you :-)
If you want to do dynamic queries, you should probably use DynamicLinq
With DynamicLinq you can do things like:
IQueryable query = context.YourTable;
var groups = query.GroupBy("new (Field1, Field2)");
I'm rereading your question...
I want to be able to identify fields in the group by keys with strong names in select part like query.Select(x => new { x.Key.DepartmentID, x.Key.SkillID });
You can't. GroupBy in general will return a IGrouping<TKey, TSource>. TKey is dynamic (because you build it based on strings), so you can't "extract" it and pass it to the compiler, so you can't do the select with strong names.
There is a single exception: if you know the types and numbers of the GroupBy TKey then something can be done. So, you gave us:
String[] fields = { "DepartmentID", "SkillID" };
If you always have two int then you can cast your query with:
.Cast<IGrouping<Tuple<int, int>, Person>>()
.Select(x => new { x.Key.DepartmentID, x.Key.SkillID });
Note that, as I've written in a comment, your GroupBy will be executed client-side, and everything after the GroupBy will be executed client-side (where client-side == where your program is vs sql-side == where your sql server is)!
DynamicLinq will solve the problem of executing the query sql-side instead of client-side, but won't solve the problem of strong vs weak naming (after a DynamicLinq you can: A) use .Cast<>() method or B) return a dynamic object/IEnumerable<dynamic>)

The syntax you're using new { x.Key.DepartmentID, x.Key.SkillID } constructs an anonymous class at compile time. If you want to create an anonymous class at runtime, see here. However, that won't allow you to "identify fields in the group by keys with strong names". If you want to construct an anonymous class at runtime, but be able to use those names at compile time, I'm afraid that's impossible.

Return an IQueryable<dynamic> to filter in the parent method

For example, I have this code:
IQueryable<MyModel> q = new List<MyModel>().AsQueryable(); // this is just an example, this is obviously not a list
var query = from item in q select new { item.Property };
var oneItem = query.FirstOrDefault(x => x.SomeProperty == somevalue);
var allItems = query.ToArray();
Now in a bit more complex situation, I need to get oneItem and allItems in two different methods. So to follow DRY, i'd like to move my query to a private method and then in the consuming ones just call this.GetQuery().FirstOrDefault() or .ToArray() as required.
However, when I try to have the method as IQueryable<dynamic> I get the 'An expression tree may not contain a dynamic operation' error. If I change it to IQueryable<object> then my filtering in the oneItem doesn't work.

You need to return
IQueryable<MyObject>
You can make your methods/classes dry by using genrics eg
IQuerable<T> GetQueryable()
Then the consumer can specify what T should be and your away.
You can't use dynamic with linq. See here to understand why.

For two methods to communicate they must understand the same type so you really want to project into a named type.
If you insist on using dynamic programming it can be done but you will need a lot of casting because dynamic is not a type but just a way of treating object:
IQueryable<MyModel> q = new List<MyModel>().AsQueryable(); // this is just an example, this is obviously not a list
IQueryable<object> query = from item in q select (object)new { item.Property };
var oneItem = query.FirstOrDefault(x => ((dynamic)x).SomeProperty == somevalue);
object[] allItems = query.ToArray();

Make a Search Method generic using LINQ

I have a method in my project that repeats over and over:
public PAC PAC_GetByCodiPac(string codiPac)
{
var sel = _gam.PAC.Where(pac => pac.CODI_PAC == codiPac);
if (sel.Count() > 0)
return sel.First();
return null;
}
The table PAC means (patient), so I have these methods for all the tables I have.
How can I make a generic method for this?
Thanks in advance.

Here is your generic method. Note, that as others pointed out FirstOrDefault is better than count and then first, so I'm using it here. But it's also possible to write the expression so that it mimics what your original code does. Please let me know if you need additional help with this.
public static T GetByCodi<T>(IQueryable<T> table, string codi, string fieldName) where T : class
{
// x
ParameterExpression parameter = Expression.Parameter(typeof(T), "x");
Expression currentExpression = parameter;
Type currentType = typeof(T);
PropertyInfo property = currentType.GetProperty(fieldName);
// x.CODI_xxx
currentExpression = Expression.Property(currentExpression, property);
// x.CODI_xxx == codi
currentExpression = Expression.Equal(currentExpression, Expression.Constant(codi));
// x => x.CODI_xxx == codi
LambdaExpression lambdaExpression = Expression.Lambda(currentExpression, parameter);
return table.FirstOrDefault((Func<T, bool>)lambdaExpression.Compile());
}
You use it like this:
PAC xxx = GetByCodi<PAC>(_gam.PAC, codiPac, "CODI_PAC");
Edit 1:
I changed the code according to the comment so that you can pass arbitrary ID field name in.

I see that what you asked is a very straight forward where query even doesn't require to have have it on a separate method.
Also you can simply enhance your query link the following:
public PAC PAC_GetByCodiPac(string codiPac)
{
return _gam.PAC.FirstOrDefault(pac => pac.CODI_PAC == codiPac);
}
FirstOrDefault will return the first item on the array, if not it will return null.

If you want a generic method that lets you specify any table and any predicate for records from that table then you can't really get any better than the built-in Where<T>(...) and (as others have already pointed out) the FirstOrDefault<T>(...) extension methods.
Your code would then look like so:
var result = _gam.PAC.Where(pac => pac.CODI_PAC == codiPac).FirstOrDefault();
// OR
var result = _gam.PAC.FirstOrDefault(pac => pac.CODI_PAC == codiPac);
The best you could get then, writing your own generic method, would be this:
public T FirstOrDefault<T>(IQueryable<T> source,
Expression<Func<T, bool>> predicate)
{
return source.Where(predicate).FirstOrDefault();
// OR
// return source.FirstOrDefault(predicate);
}
And that is really just redundant. Especially when your calling code would be actually longer using the helper method:
var result = FirstOrDefault(_gam.PAC, pac => pac.CODI_PAC == codiPac);
// versus
var result = _gam.PAC.FirstOrDefault(pac => pac.CODI_PAC == codiPac);
And even worse, your code is no longer using a fluent, composable syntax. This just makes readability and maintenance more difficult.
If you stick with using the IQueryable<T> extension methods then you can do composition like this:
var result = _gam.PAC
.Where(pac => pac.CODI_PAC == codiPac)
.Where(pac => pac.SomeOtherProperty == someOtherValue)
.FirstOrDefault();
// OR
var result = (from pac in _gam.PAC
where pac.CODI_PAC == codiPac
where pac.SomeOtherProperty == someOtherValue
select pac).FirstOrDefault();
One very important thing to note here is that the predicate parameter in the IQueryable<T>.Where<T>(...) extension method is of type Expression<Func<T, bool>>. This allows the IQueryable<T> provider to construct the native SQL (or other native provider query) at the very last moment before returning a result.
Not using Expression<Func<T, bool>> means that your query would be the equivalent of this:
var result =
_gam.PAC
.ToArray()
.Where(pac => pac.CODI_PAC == codiPac)
.FirstOrDefault();
And that would mean the query will load every record from the "PAC" table into memory before selecting the first filtered result and throwing out the rest of the results.
The bottom-line is that by making a generic helper method you are rewriting existing framework code and you open yourself to performance and maintenance issues while also reducing code readability.
I hope this helps.

I'm not sure if you are asking for this, but this method could be in a static class and method and so you'd be able to call it from everywhere.

An easy solution will be:
//a generic method
private PAC PAC_GetPAC(Func<PAC, bool> predicate)
{
return _gam.PAC.Where(predicate).FirstOrDefault();
}
public PAC PAC_GetPACById(long id)
{
return PAC_GetPAC(p => p.ID == id);
}
public PAC PAC_GetByCodiPac(string codiPac)
{
return PAC_GetPAC(p => pac.CODI_PAC == codiPac);
}

Dynamic WHERE clause in LINQ

What is the best way to assemble a dynamic WHERE clause to a LINQ statement?
I have several dozen checkboxes on a form and am passing them back as: Dictionary<string, List<string>> (Dictionary<fieldName,List<values>>) to my LINQ query.
public IOrderedQueryable<ProductDetail> GetProductList(string productGroupName, string productTypeName, Dictionary<string,List<string>> filterDictionary)
{
var q = from c in db.ProductDetail
where c.ProductGroupName == productGroupName && c.ProductTypeName == productTypeName
// insert dynamic filter here
orderby c.ProductTypeName
select c;
return q;
}

(source: scottgu.com)
You need something like this? Use the Linq Dynamic Query Library (download includes examples).
Check out ScottGu's blog for more examples.

I have similar scenario where I need to add filters based on the user input and I chain the where clause.
Here is the sample code.
var votes = db.Votes.Where(r => r.SurveyID == surveyId);
if (fromDate != null)
{
votes = votes.Where(r => r.VoteDate.Value >= fromDate);
}
if (toDate != null)
{
votes = votes.Where(r => r.VoteDate.Value <= toDate);
}
votes = votes.Take(LimitRows).OrderByDescending(r => r.VoteDate);

You can also use the PredicateBuilder from LinqKit to chain multiple typesafe lambda expressions using Or or And.
http://www.albahari.com/nutshell/predicatebuilder.aspx

A simple Approach can be if your Columns are of Simple Type like String
public static IEnumerable<MyObject> WhereQuery(IEnumerable<MyObject> source, string columnName, string propertyValue)
{
return source.Where(m => { return m.GetType().GetProperty(columnName).GetValue(m, null).ToString().StartsWith(propertyValue); });
}

It seems much simpler and simpler to use the ternary operator to decide dynamically if a condition is included
List productList = new List();
productList =
db.ProductDetail.Where(p => p.ProductDetailID > 0 //Example prop
&& (String.IsNullOrEmpty(iproductGroupName) ? (true):(p.iproductGroupName.Equals(iproductGroupName)) ) //use ternary operator to make the condition dynamic
&& (ID == 0 ? (true) : (p.ID == IDParam))
).ToList();

I came up with a solution that even I can understand... by using the 'Contains' method you can chain as many WHERE's as you like. If the WHERE is an empty string, it's ignored (or evaluated as a select all). Here is my example of joining 2 tables in LINQ, applying multiple where clauses and populating a model class to be returned to the view. (this is a select all).
public ActionResult Index()
{
string AssetGroupCode = "";
string StatusCode = "";
string SearchString = "";
var mdl = from a in _db.Assets
join t in _db.Tags on a.ASSETID equals t.ASSETID
where a.ASSETGROUPCODE.Contains(AssetGroupCode)
&& a.STATUSCODE.Contains(StatusCode)
&& (
a.PO.Contains(SearchString)
|| a.MODEL.Contains(SearchString)
|| a.USERNAME.Contains(SearchString)
|| a.LOCATION.Contains(SearchString)
|| t.TAGNUMBER.Contains(SearchString)
|| t.SERIALNUMBER.Contains(SearchString)
)
select new AssetListView
{
AssetId = a.ASSETID,
TagId = t.TAGID,
PO = a.PO,
Model = a.MODEL,
UserName = a.USERNAME,
Location = a.LOCATION,
Tag = t.TAGNUMBER,
SerialNum = t.SERIALNUMBER
};
return View(mdl);
}

Just to share my idea for this case.
Another approach by solution is:
public IOrderedQueryable GetProductList(string productGroupName, string productTypeName, Dictionary> filterDictionary)
{
return db.ProductDetail
.where
(
p =>
(
(String.IsNullOrEmpty(productGroupName) || c.ProductGroupName.Contains(productGroupName))
&& (String.IsNullOrEmpty(productTypeName) || c.ProductTypeName.Contains(productTypeName))
// Apply similar logic to filterDictionary parameter here !!!
)
);
}
This approach is very flexible and allow with any parameter to be nullable.

You could use the Any() extension method. The following seems to work for me.
XStreamingElement root = new XStreamingElement("Results",
from el in StreamProductItem(file)
where fieldsToSearch.Any(s => el.Element(s) != null && el.Element(s).Value.Contains(searchTerm))
select fieldsToReturn.Select(r => (r == "product") ? el : el.Element(r))
);
Console.WriteLine(root.ToString());
Where 'fieldsToSearch' and 'fieldsToReturn' are both List objects.

This is the solution I came up with if anyone is interested.
https://kellyschronicles.wordpress.com/2017/12/16/dynamic-predicate-for-a-linq-query/
First we identify the single element type we need to use ( Of TRow As DataRow) and then identify the “source” we are using and tie the identifier to that source ((source As TypedTableBase(Of TRow)). Then we must specify the predicate, or the WHERE clause that is going to be passed (predicate As Func(Of TRow, Boolean)) which will either be returned as true or false. Then we identify how we want the returned information ordered (OrderByField As String). Our function will then return a EnumerableRowCollection(Of TRow), our collection of datarows that have met the conditions of our predicate(EnumerableRowCollection(Of TRow)). This is a basic example. Of course you must make sure your order field doesn’t contain nulls, or have handled that situation properly and make sure your column names (if you are using a strongly typed datasource never mind this, it will rename the columns for you) are standard.

System.Linq.Dynamic might help you build LINQ expressions at runtime.
The dynamic query library relies on a simple expression language for formulating expressions and queries in strings.
It provides you with string-based extension methods that you can pass any string expression into instead of using language operators or type-safe lambda extension methods.
It is simple and easy to use and is particularly useful in scenarios where queries are entirely dynamic, and you want to provide an end-user UI to help build them.
Source: Overview in Dynamic LINQ
The library lets you create LINQ expressions from plain strings, therefore, giving you the possibility to dynamically build a LINQ expression concatenating strings as you require.
Here's an example of what can be achieved:
var resultDynamic = context.Customers
.Where("City == #0 and Age > #1", "Paris", 50)
.ToList();