Say I have a table Table1 with a string field [ProductString] with values:
Alpha, alphanumeric or numeric: eg ABC, B4, U2, C 5, 100, U1, U5, U6, U11
I want to be able to take a where clause like "ProductString >= U5", and pass this to a LINQ statement as a string so it evaluates
Table1.Where(t=> t.ProductString >= 'U5');
Normally this would return results U5 and U6.
However, this I want to be able to use a NaturalSortComparer somehow so that the results returned are U5, U6 and U11.
I know how to use the comparer in an OrderBy, by I wanted to be able to use it at the Where stage.
Using natural sort comparer:
var comparer = new NaturalComparer();
Table1.Where(t=>
comparer.Compare(t.ProductString, "U5") >= 0);
Presuming all your product strings is on the format U%number% then why not abuse that fact?
Table1.Where(t=> int.Parse(t.ProductString.Replace("U","")) >= 5);
If you're using LINQ to Entities I'm not certain this will compile to a store expression (i.e that SQL knows what to do with this - I guess it should).
I'm a little confused, given the accepted answer, about whether this question relates to LINQ to Entities or not. The accepted answer doesn't appear to be a solution that would work in the LINQ to Entities context, but the comments on the question by the OP seem to confirm that this is being executed in the database context. Anyway, this answer is specifically targeted toward LINQ to Entities.
I think doing this in SQL Server would be hard, but not impossible. The problem is that .NET knows what NaturalSortComparer is, but SQL Server (where you want the query to ultimately take place) has no such concept. The best idea I can think of would consist of 2 parts:
Create a UDF (User Defined Function) in SQL server that will give a product that is orderable via natural sort: CREATE FUNCTION Naturalize(#val as nvarchar(max)) RETURNS nvarchar(1000). There's a pretty cool answer here that creates a UDF wrapper around a CLR function to accomplish just that.
Next create a function mapping for your DbContext that maps the UDF above to a function that can be called inside an EF query against the DbContext. Something like this:
[DbFunction("MyContext", "Naturalize")]
public static string Naturalize(this string value)
{
throw new NotSupportedException("This function can only be invoked from LINQ to Entities.");
}
Once you've got these two pieces in place, you can readily use this new function inside an entity query to compare strings using the Naturalized value in the comparison:
Table1.Where(t=> t.ProductString.Naturalize() >= "U5".Naturalize());
Bear in mind that the UDF will be executed against every row contained in the query, which is the whole table in the above example. You'll want to make sure to pare down your query to something manageable before applying the function as a sub-query. Or you may want to try applying some type of UDF-based index on the table in question.
If you are going to be doing searches like this a lot, then what will be the best thing to do is add two new fields to your table, [ProductCode] & [ProductNumber] which separate the two parts of the [ProductString].
Then you comparison becomes:
Table1.Where(t=> t.ProductCode == "U" && t.ProductNumer > 5);
Related
I have a Linq query that reads from a SQL table and 1 of the fields it returns are from a custom function (in C#).
Something like:
var q = from my in MyTable
select new
{
ID = my.ID,
Amount = GetAmount(ID)
};
If I do a q.Dump() in LinqPad, it shows the results, which tells me that it runs the custom function without trying to send it to SQL.
Now I want to union this to another query, with:
var q1 = (from p in AnotherQuery.Union(q)...
and the I get the error that Method has no supported translation to SQL.
So, my logic tells me that I need to dump q in memory and then try to union to that. I've tried doing that with ToList() and creating a secondary query that populates itself from the List, but that leads to a long list of different errors. Am I on the right track, by trying to get q in memory and union on that, or are there better ways of doing this?
You can't use any custom functions in a LINQ query that gets translated - only the functions supported by the given LINQ provider. If you want your query to happen on the server, you need to stick with the supported functions (even if it sometimes means having to inline code that would otherwise be reused).
The difference between your two queries boils down to when (and where) the projection happens. In your first case, the data from MyTable is returned from the DB - in your sample, just the ID. Then, the projection happens on top of this - the GetAmount method is called in your application for each of ID.
On the other hand, there's no such way for this to happen in your second query, since you're not using GetAmount in the final projection.
You either need to replace the custom function with inlined query the provider understands, or refactor all your queries to use the supported functions in addition with whatever you need to do in-memory. There's no point in giving you any sample code, since it depends entirely on your actual query, and what you're really trying to query for.
I have a many-to-many relationship between tables of Games and Genres. During an analysis, i need to get items from Games that match specific criteria.
The problem is, to check for this criteria, i need to analyse genres of this specific game. And linq won't let me do it.
My request now looks like this:
var result = GDB.Games.Where((g)=>
g.GamesToGenres.Select((gtg)=>
(weights.ContainsKey(gtg.Genre.Name) ? weights[gtg.Genre.Name]:0.0)
).Sum() > Threshhold
).ToArray();
When I execute it, I receive SQL exception
Only one expression can be specified in the select list when the
subquery is not introduced with EXISTS.
Is there a workaround? How can i perform such Select inside of Where?
EDIT: weights is a Dictionary<string, double>.
EDIT: I was playing with lambdas, and found out a strange thing in their behaviour:
this code won't work, throwing nvarchar to float conversion exception:
Func<string, double> getW = (name) => 1;
var t = GDB.Games.Where((g)=>
g.GamesToGenres.Select((gtg)=>
getW(gtg.Genre.Name)
).Sum() > Threshhold
).ToArray();
but this one will work nicely:
var t = GDB.Games.Where((g)=>
g.GamesToGenres.Select((gtg)=>
1
).Sum() > Threshhold
).ToArray();
This leads me to conclusion that linq lambdas are not usual lambdas. What's wrong with them, then? What are their limitations? What i can and what i can't do inside of them? Why is it ok for me to place a .select call inside of lambda, but not my own call of getW?
RESOLVED. See the answer below. Long story short, C# can't into clojures unless explicitly told so. If anyone knows better answer, i am still confused.
Your problem is you're trying to select something form the dictionary weights that exists in your application and not in your DB. If it was the result of a query to your DB, use the query.Single(...) in its place
Well, i am confused beyond imagination. The following code works perfectly:
Func<Game, bool> predicate = (g) =>
g.GamesToGenres.Select((gtg) =>
(weights.ContainsKey(gtg.Genre.Name) ? weights[gtg.Genre.Name] : 0.0)
).Sum() > Threshhold;
var t = GDB.Games.Where(predicate).ToArray();
careful reader might want to say "Hey! Isn't that the very same code you wrote in the question? You just explicitly assigned it to a variable!", and he will be right. Right now it seems like C# lambda processor is a piece of something, and it creates clojure only when you explicitly declare a lambda. If someone can describe this phenomena to me, i will be gratefull, for right now i am more confused than a newborn baby.
LINQ allows you to combine SQL data with local data like (Dictionary, etc.) with one restriction. You need to select data from SQL first. This means your code will work if you replace GDB.Games.Where with GDB.Games.ToList().Where. You can ask about performance, but you able to select a slice of data like GameId, Genre Name, etc. Then filter out games. Then return end list of full game's info by game ID list.
Suppose I have a collection (of arbitrary size) of IQueryable<MyEntity> (all for the same MyEntity type). Each individual query has successfully been dynamically built to encapsulate various pieces of business logic into a form that can be evaluated in a single database trip. Is there any way I can now have all these IQueryables executed in a single round-trip to the database?
For example (simplified; my actual queries are more complex!), if I had
ObjectContext context = ...;
var myQueries = new[] {
context.Widgets.Where(w => w.Price > 500),
context.Widgets.Where(w => w.Colour == 5),
context.Widgets.Where(w => w.Supplier.Name.StartsWith("Foo"))
};
I would like to have EF perform the translation of each query (which it can do indivudually), then in one database visit, execute
SELECT * FROM Widget WHERE Price > 500
SELECT * FROM Widget WHERE Colour = 5
SELECT W.* FROM Widget
INNER JOIN SUpplier ON Widget.SupplierId = Supplier.Id
WHERE Supplier.Name LIKE 'Foo%'
then convert each result set into an IEnumerable<Widget>, updating the ObjectContext in the usual way.
I've seen various posts about dealing with multiple result sets from a stored procedure, but this is slightly different (not least because I don't know at compile time how many results sets there are going to be). Is there an easy way, or do I have to use something along the lines of Does the Entity Framework support the ability to have a single stored procedure that returns multiple result sets??
No. EF deosn't have query batching (future queries). One queryable is one database roundtrip. As a workaround you can try to play with it and for example use:
string sql = ((ObjectQuery<Widget>)context.Widgets.Where(...)).ToTraceString();
to get SQL of the query and build your own custom command from all SQLs to be executed. After that you can use similar approach as with stored procedures to translate results.
Unless you really need to have each query executed separately you can also union them to single query:
context.Widgets.Where(...).Union(context.Widgets.Where(...));
This will result in UNION. If you need just UNION ALL you can use Concat method instead.
It might be late answer, hopefully it would help some one else with the same issue.
There is Entity Framework Extended Library on NuGet which provides the future queries feature (among others). I played a bit with it and it looks promising.
You can find more information here.
I have an IQueryable that has a list of pages.
I want to do: Pages.OrderByDescending(o => CalculateSort(o.page));
the method calculate sort is similar to that here is a plain english version:
public int calculatesort(page p)
{
int rating = (from r in db.rating select r). sum();
int comments = //query database for comments;
float timedecayfactor = math.exp(-page.totalhoursago);
return sortscore = (rating +comments)* timedecayfactor;
}
when I run a code similar to the one above an error is thrown that the mothode calculatesort cannot be converted to sql.
How can I do a conver the function above to be understood by sql so that I can use it to sort the pages?
Is this not a good approach for large data? Is there another method used to sort sets of results other than dynamically at the database?
I havent slept for days trying to fix this one :(
your code is nowhere near compiling so I'm guessing a lot here but I hope this gives an idea none the less.
As several have posted you need to give Linq-2-Sql an expression tree. Using query syntax that's what happens (by compiler magic)
from p in pages
let rating = (from r in db.rating
where r.PageId == p.PageId
select r.Value).Sum()
let comments = (from c in db.Comments
where c.PageId == p.PageId
select 1).Count()
let timedecayfactor = Math.Exp(-(p.totalhoursago))
orderby (rating + comments)*timedecayfactor descending
select p;
I haven't actually tried this against a database, there's simply too many unknown based on your code, so there might still be stuff that can't be translated.
The error occurs because LINQ cannot convert custom code/methods into SQL. It can convert only Expression<Func<>> objects into SQL.
In your case, you have a complex logic to do while sorting, so it might make sense to do it using a Stored Procedure, if you want to do it in the DB Layer.
Or load all the objects into main memory, and run the calculate sort method on the objects in memory
EDIT :
I don't have the code, so Describing in english is the best I can do :
Have table with structure capable of temporarily storing all the current users data.
Have a calculated field in the Pages table that holds the value calculated from all the non-user specific fields
Write a stored procedure that uses values from these two sources (temp table and calc field) to actually do the sort.
Delete the temp table as the last part in the stored proc
You can read about stored procs here and here
var comments = db.comments.Where(...);
Pages.OrderByDescending(p=>(db.rating.Sum(r=>r.rate) + comments.Count()) * Math.Exp(-p.totalhoursago))
Linq is expecting Calculatesort to return a "queryable" expression in order to generate its own SQL.
In can embed your 'calculatesort' method in this lambda expression. (I replaced your variables with constants in order to compile in my environment)
public static void ComplexSort(IQueryable<string> Pages)
{
Pages.OrderByDescending(p =>
{
int rating = 99;//(from r in db.rating select r). sum();
int comments = 33;//query database for comments;
double timedecayfactor = Math.Exp(88);
return (rating + comments) * timedecayfactor;
});
}
Also, you can even try to run that in parallel (since .net 4.0) replacing the first line with
Pages.AsParallel().OrderByDescending(p =>
Yes, counting previous answers: the LINQ to SQL doesn't know how to translate CalculateSort method. You should convert LINQ to SQL to ordinary LINQ to Object before using custom method.
Try to use this in the way you call the CalculateSort by adding AsEnumerable:
Pages.AsEnumerable().OrderByDescending(o => CalculateSort(o.page));
Then you're fine to use the OrderByDescending extension method.
UPDATE:
LINQ to SQL will always translate the query in the code into Expression tree. It's quite almost the same concept as AST of any programming language. These expression trees are further translated into SQL expression specific to SQL Server's SQL, because currently LINQ to SQL only supports SQL Server 2005 and 2008.
So, in my last post I was asking how to build a dynamic search filter using LINQ and EF4 (See Here) and finally came up with the solution of building the expression as a string and parse it to an expression using the Dynamic LINQ library.
I that solved the problem. I was able to generate a Expression<Func<TSource, out bool>> and pass it to the Where() method of the DbSet. I am also trying to do this using MySql as a database behind EF4.
The problem came when I tried to apply string operations to integers, like searching a database record which consecutive number starts with 1234.
My initial expression was something like: record.ConsecutiveNumber.ToString().StartsWith("1234"). Sadly, as expected, things were not that easy as EF4 fails to query the DbSet with exception:
"LINQ to Entities does not recognize
the method 'System.String ToString()'
method, and this method cannot be
translated into a store expression."
After some Google search I found that this is a common problem. But C'MON! Is there a way to perform a search function that can search records with a consecutive number starting by "1234"?
How pros implement search features with EF4? This is with a single property filter. What if I wanna add multiple filters? God, my head hurts... :/
Thanks!
EDIT:
Thought #1: What about a stored procedure? What about calling a MySql stored procedure from Linq? Am I aiming way too high?
You can use the SqlFunctions.StringConvert method. It requires a double (or decimal) so you'll have to cast your int ConsecutiveNumber.
Replace:
record.ConsecutiveNumber.ToString().StartsWith("1234")
With:
SqlFunctions.StringConvert((double)record.ConsecutiveNumber).StartsWith("1234")
Have you looked at the Dynamic LinQ Library:
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx
And for your question
How to use "contains" or "like" in a dynamic linq query?
Previously I have gotten the code for this lib and just taken a look inside, it is pretty easy to follow.
This would be my thought process on getting it to work. Hopefully it points you in the right direction.
According to other posts SqlFunctions.StringConvert((double)record.ConsecutiveNumber) works for Sql Server.
Problem with converting int to string in Linq to entities
And here is relevant information on linq conversions.
Linq int to string
And here is an answer hinting at writing your own sql function for stringconvert
Using a SQL Function in Entity Framework Select
If SqlFunctions.StringConvert doesn't work for you I'd suggest looking at figuring out how to do it in Sql and then writing your own [EdmFunction()] attribute based method.
I haven't got a clue if this will work over Linq to EF or not but presuming that they mapped the Math operations, this might solve your need:
record.ConsecutiveNumber / Math.Pow(10, Math.Truncate(Math.Log10(record.ConsecutiveNumber) - 3)) == 1234
This is basically dividing the number by a power of 10 just big enough to leave the first 4 digits.
I know this is very hacky and inefficient even if it works, but there you go. :)
Any method calls in a LINQ to Entities query that are not explicitly mapped to a canonical function will result in a runtime NotSupportedException exception being thrown.
Check mapping canonical function here:
http://msdn.microsoft.com/en-us/library/bb738681.aspx
In this case, you can use Math function. (I don't think code first can use in product project at that time)