I have a many-to-many relationship between tables of Games and Genres. During an analysis, i need to get items from Games that match specific criteria.
The problem is, to check for this criteria, i need to analyse genres of this specific game. And linq won't let me do it.
My request now looks like this:
var result = GDB.Games.Where((g)=>
g.GamesToGenres.Select((gtg)=>
(weights.ContainsKey(gtg.Genre.Name) ? weights[gtg.Genre.Name]:0.0)
).Sum() > Threshhold
).ToArray();
When I execute it, I receive SQL exception
Only one expression can be specified in the select list when the
subquery is not introduced with EXISTS.
Is there a workaround? How can i perform such Select inside of Where?
EDIT: weights is a Dictionary<string, double>.
EDIT: I was playing with lambdas, and found out a strange thing in their behaviour:
this code won't work, throwing nvarchar to float conversion exception:
Func<string, double> getW = (name) => 1;
var t = GDB.Games.Where((g)=>
g.GamesToGenres.Select((gtg)=>
getW(gtg.Genre.Name)
).Sum() > Threshhold
).ToArray();
but this one will work nicely:
var t = GDB.Games.Where((g)=>
g.GamesToGenres.Select((gtg)=>
1
).Sum() > Threshhold
).ToArray();
This leads me to conclusion that linq lambdas are not usual lambdas. What's wrong with them, then? What are their limitations? What i can and what i can't do inside of them? Why is it ok for me to place a .select call inside of lambda, but not my own call of getW?
RESOLVED. See the answer below. Long story short, C# can't into clojures unless explicitly told so. If anyone knows better answer, i am still confused.
Your problem is you're trying to select something form the dictionary weights that exists in your application and not in your DB. If it was the result of a query to your DB, use the query.Single(...) in its place
Well, i am confused beyond imagination. The following code works perfectly:
Func<Game, bool> predicate = (g) =>
g.GamesToGenres.Select((gtg) =>
(weights.ContainsKey(gtg.Genre.Name) ? weights[gtg.Genre.Name] : 0.0)
).Sum() > Threshhold;
var t = GDB.Games.Where(predicate).ToArray();
careful reader might want to say "Hey! Isn't that the very same code you wrote in the question? You just explicitly assigned it to a variable!", and he will be right. Right now it seems like C# lambda processor is a piece of something, and it creates clojure only when you explicitly declare a lambda. If someone can describe this phenomena to me, i will be gratefull, for right now i am more confused than a newborn baby.
LINQ allows you to combine SQL data with local data like (Dictionary, etc.) with one restriction. You need to select data from SQL first. This means your code will work if you replace GDB.Games.Where with GDB.Games.ToList().Where. You can ask about performance, but you able to select a slice of data like GameId, Genre Name, etc. Then filter out games. Then return end list of full game's info by game ID list.
Related
I am very new to Mongo (this is actually day 1) and using the C# driver that is available for it. One thing that I want to know (as I am not sure how to word it in Google) is how does mongo handle executing queries when I want to grab a part of the collection.
What I mean by this is that I know that with NHibernate and EF Core, the query is first built and it will only fire when you cast it. So say like an IQueryable to IEnnumerable, .ToList(), etc.
Ex:
//Query is fired when I call .ToList, until that point it is just building it
context.GetLinqQuery<MyObject>().Where(x => x.a == 'blah').ToList();
However, with Mongo's examples it appears to me that if I want to grab a filtered result I will first need to get the collection, and then filter it down.
Ex:
var collection = _database.GetCollection<MyObject>("MyObject");
//Empty filter for ease of typing for example purposes
var filter = Builders<MyObject>.Filter.Empty;
var collection.Find(filter).ToList();
Am I missing something here, I do not think I saw any overload in the GetCollection method that will accept a filter. Does this mean that it will first load the whole collection into memory, then filter it? Or will it still be building the query and only execute it once I call either .Find or .ToList on it?
I ask this because at work we have had situations where improper positioning of .ToList() would result is seriously weak performance. Apologies if this is not the right place to ask.
References:
https://docs.mongodb.com/guides/server/read_queries/
The equivalent to your context.GetLinqQuery<MyObject>() would be to use AsQueryable:
collection.AsQueryable().Where(x => x.a == "blah").ToList();
The above query will be executed server side* and is equivalent to:
collection.Find(Builders<MyObject>.Filter.Eq(x => x.a, "blah")).ToEnumerable().ToList();
* The docs state that:
Only LINQ queries that can be translated to an equivalent MongoDB query are supported. If you write a LINQ query that can’t be translated you will get a runtime exception and the error message will indicate which part of the query wasn’t supported.
Say I have a table Table1 with a string field [ProductString] with values:
Alpha, alphanumeric or numeric: eg ABC, B4, U2, C 5, 100, U1, U5, U6, U11
I want to be able to take a where clause like "ProductString >= U5", and pass this to a LINQ statement as a string so it evaluates
Table1.Where(t=> t.ProductString >= 'U5');
Normally this would return results U5 and U6.
However, this I want to be able to use a NaturalSortComparer somehow so that the results returned are U5, U6 and U11.
I know how to use the comparer in an OrderBy, by I wanted to be able to use it at the Where stage.
Using natural sort comparer:
var comparer = new NaturalComparer();
Table1.Where(t=>
comparer.Compare(t.ProductString, "U5") >= 0);
Presuming all your product strings is on the format U%number% then why not abuse that fact?
Table1.Where(t=> int.Parse(t.ProductString.Replace("U","")) >= 5);
If you're using LINQ to Entities I'm not certain this will compile to a store expression (i.e that SQL knows what to do with this - I guess it should).
I'm a little confused, given the accepted answer, about whether this question relates to LINQ to Entities or not. The accepted answer doesn't appear to be a solution that would work in the LINQ to Entities context, but the comments on the question by the OP seem to confirm that this is being executed in the database context. Anyway, this answer is specifically targeted toward LINQ to Entities.
I think doing this in SQL Server would be hard, but not impossible. The problem is that .NET knows what NaturalSortComparer is, but SQL Server (where you want the query to ultimately take place) has no such concept. The best idea I can think of would consist of 2 parts:
Create a UDF (User Defined Function) in SQL server that will give a product that is orderable via natural sort: CREATE FUNCTION Naturalize(#val as nvarchar(max)) RETURNS nvarchar(1000). There's a pretty cool answer here that creates a UDF wrapper around a CLR function to accomplish just that.
Next create a function mapping for your DbContext that maps the UDF above to a function that can be called inside an EF query against the DbContext. Something like this:
[DbFunction("MyContext", "Naturalize")]
public static string Naturalize(this string value)
{
throw new NotSupportedException("This function can only be invoked from LINQ to Entities.");
}
Once you've got these two pieces in place, you can readily use this new function inside an entity query to compare strings using the Naturalized value in the comparison:
Table1.Where(t=> t.ProductString.Naturalize() >= "U5".Naturalize());
Bear in mind that the UDF will be executed against every row contained in the query, which is the whole table in the above example. You'll want to make sure to pare down your query to something manageable before applying the function as a sub-query. Or you may want to try applying some type of UDF-based index on the table in question.
If you are going to be doing searches like this a lot, then what will be the best thing to do is add two new fields to your table, [ProductCode] & [ProductNumber] which separate the two parts of the [ProductString].
Then you comparison becomes:
Table1.Where(t=> t.ProductCode == "U" && t.ProductNumer > 5);
I am trying to implement some search functionality within our app and have a situation where a User can select multiple Topics from a list and we want to return all activities that match at least one of the selected Topics. Each Activity can have 0-to-many Topics.
I can write a straight SQL query that gives me the results I want like so:
SELECT *
FROM dbo.ACTIVITY_VERSION av
WHERE ACTIVITY_VERSION_ID IN (
SELECT ACTIVITY_VERSION_ID
FROM dbo.ACTIVITY_TOPIC at
WHERE at.TOPIC_ID IN (3,4)
)
What I can't figure out is how to write a LINQ query (we are using Linq to Sql) that returns the same results.
I've tried:
activities.Where(x => criteria.TopicIds.Intersect(x.TopicIds).Any());
this works if activities is a list of in memory objects (i.e. a Linq to Objects query), but I get an error if I try to use the same code in a query that hits the database. The error I receive is:
Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator.
I believe that this means that Linq to Sql doesn't know how to translate either Intersect or Any (or possibly both). If that is the case, I understand why it isn't working, but I still don't know how to make it do what I want it to and my Google-fu has not provided me with anything that works.
Haven't tested it. But this is how you ll go about it.
List<int> IDs = new List<int>();
IDs.Add(3);
IDs.Add(4);
var ACTIVITY_VERSION_IDs = ACTIVITY_TOPIC
.Where(AT => IDs.Contains(AT.TOPIC_ID))
.Select(AT=>AT.ACTIVITY_VERSION_ID)
var results = ACTIVITY_VERSION
.Where(AV => ACTIVITY_VERSION_IDs.Contains(AV.ACTIVITY_VERSION_ID))
I've been searching here and Google, but I'm at a loss. I need to let users search a database for reports using a form. If a field on the form has a value, the app will get any reports with that field set to that value. If a field on a form is left blank, the app will ignore it. How can I do this? Ideally, I'd like to just write Where clauses as Strings and add together those that are not empty.
.Where("Id=1")
I've heard this is supposed to work, but I keep getting an error: "could not be resolved in the current scope of context Make sure all referenced variables are in scope...".
Another approach is to pull all the reports then filter it one where clause at a time. I'm hesitant to do this because 1. that's a huge chunk of data over the network and 2. that's a lot of processing on the user side. I'd like to take advantage of the server's processing capabilities. I've heard that it won't query until it's actually requested. So doing something like this
var qry = ctx.Reports
.Select(r => r);
does not actually run the query until I do:
qry.First()
But if I start doing:
qry = qry.Where(r => r.Id = 1).Select(r => r);
qry = qry.Where(r => r.reportDate = '2010/02/02').Select(r => r);
Would that run the query? Since I'm adding a where clause to it. I'd like a simple solution...in the worst case I'd use the Query Builder things...but I'd rather avoid that (seems complex).
Any advice? :)
Linq delays record fetching until a record must be fetched.
That means stacking Where clauses is only adding AND/OR clauses to the query, but still not executing.
Execution of the generated query will be done in the precise moment you try to get a record (First, Any etc), a list of records(ToList()), or enumerate them (foreach).
.Take(N) is not considered fetching records - but adding a (SELECT TOP N / LIMIT N) to the query
No, this will not run the query, you can structure your query this way, and it is actually preferable if it helps readability. You are taking advantage of lazy evaluation in this case.
The query will only run if you enumerate results from it by using i.e. foreach or you force eager evaluation of the query results, i.e. using .ToList() or otherwise force evaluation, i.e evaluate to a single result using i.e First() or Single().
Try checking out this dynamic Linq dll that was released a few years back - it still works just fine and looks to be exactly what you are looking for.
Could someone help me with this exception? I don't understand what it means or how to fix it... It is an SqlException with the following message:
All queries combined using a UNION, INTERSECT or EXCEPT operator must have an equal number of expressions in their target lists.
I get it when running a query in pseudo code looking like this:
// Some filtering of data
var query = data.Subjects
.Where(has value)
.Where(has other value among some set of values);
// More filtering, where I need to have two different options
var a = query
.Where(some foreign key is null);
var b = query
.Where(some foreign key is not null)
.Where(and that foreign key has a property which is what I want);
query = a.Union(b);
// Final filter and then get result as a list
var list = query
.Where(last requirement)
.ToList();
If I remove the a.Union(b) parts, it runs without the exception. So I know the error is there. But why do I get it? And how can I fix it? Am I doing something too crazy here? Have I misunderstood how to use the Union thing?
Basically what I have is some entities which have a foreign key to some other entity. And I need to get all the entities which either have that foreign key set to null or where that foreign entity fulfills some requirements.
Judging from the SQL error you listed you may be experiencing the same issue I was. Basically when Linq to SQL queries that use the Concat or Union extension method on two different queries it appears that there is a bug in Linq to SQL which optimizes each projection separately without regard to the fact that the projection must stay the same in order to accomplish the SQL Union.
References:
LINQ to SQL produces incorrect TSQL when using UNION or CONCAT
Linq to SQL Union Same Fieldname generating Error
If this happens to be your problem as well I've found a solution that is working for me as shown below.
var queryA =
from a in context.TableA
select new
{
id,
name,
onlyInTableA,
}
var queryB =
from b in context.TableB
let onlyInTableA = default(string)
select new
{
id,
name,
onlyInTableA,
}
var results = queryA.Union(queryB).ToList();
Since this looks like a problem with the generated SQL, you should try to use either an SQL Profiler, or use this code for DebuggerWritter class to write the SQL to your Output Window in Visual Studio.
The SQL error is normally caused by the fields retrieved for UNION is not the same for the 2 queries. For example, if the first query might have 3 fields, but the second query has 4 fields, this error will occur. So, seeing the generated SQL will definitely help in this case.
Can you perhaps write it in a single query?
.Where(row => row.ForeignKey == null || row.ForeignKey.SomeCondition);
There are also ways of merging expressions (OrElse), but that isn't trivial.
Not sure where the error comes from, though!
edit: haven't tested it, but this should be logically equivalent to a UNION:
public static IQueryable<T> WhereAnyOf<T>(
this IQueryable<T> source,
params Expression<Func<T, bool>>[] predicates)
{
if (source == null) throw new ArgumentNullException("source");
if (predicates == null) throw new ArgumentNullException("predicates");
if (predicates.Length == 0) return source.Where(row => false);
if (predicates.Length == 1) return source.Where(predicates[0]);
var param = Expression.Parameter(typeof(T), "row");
Expression body = Expression.Invoke(predicates[0], param);
for (int i = 1; i < predicates.Length; i++)
{
body = Expression.OrElse(body,
Expression.Invoke(predicates[i], param));
}
return source.Where(Expression.Lambda<Func<T, bool>>(body, param));
}
query = a.Union(b);
Not a good idea to mutate captured variables... Likely the cause of the error.
UPDATE: ok not
Here is another idea. The hint is in the error message.
var a = query
.Where(some foreign key is null)
.Select(x => x);
Or play by adding another 'fake' Where till they do become equal :)
I would call data.GetCommand(query) and analyze the resulting DbCommand (especially the generated SQL string). That should give you a clue to what goes wrong.
There is no projection going on anywhere so I would expect both target lists to be the same.
You could try to reduce your query to a smaller one that still doesn't work. Start with query.Union(query) (this should at least work). Than add your Where calls one by one to see when it stops working.
It must be one of your Where calls that adds extra columns to your select list.
Are you by any chance passing in a value to the 'select' side in a variable, or are you returning the same field more than once? SP1 introduced a bug where it tries to 'optimize' out such things and that can cause union queries to break (due to the query parts 'optimizing' out different passed-in params).
If you post your actual query rather than pseudo code it makes it easier to identify if this is the case.
(And a workaround if this is the case is to materialize the individual parts first and then do a client-side (L2O) union).
jpierson has the problem summarised correctly.
I also had the problem, this time caused by some literals in the select statement:
Dim results = (From t in TestDataContext.Table1 _
Where t.ID = WantedID _
Select t.name, SpecialField = 0, AnotherSpecialField = 0, t.Address).Union _
From t in TestDataContext.Table1 _
Where t.SecondID = WantedSecondID _
Select t.name, SpecialField = 1, AnotherSpecialField = 0, t.Address)
The first sub-query of "SpecialField = 0" and the "AnotherSpecialField = 0" were optimised, resulting in one field instead of two being used in the union, which will obviously fail.
I had to change the first query so that the SpecialField & AnotherSpecialField had different values, much like in the second sub-query.
Well I had an issue with this. Using Sql 08 i had two table functions that returned an int and a string in both cases. I created a complex object and used linq to attempt a UNION. Had an IEqualityComparer to do the comparision. All compiled fine, but crashed with a unsupported overload. Ok, i realised the problem discussed seemed to smack of defered execution. So i get the collections, and place ToList(), then do the UNION and it is all good. Not sure if this is helpful, but it works for me