In my application there is an issue part that allows questions and responses. Something like (Pseudo code, as this is actually generated from Entity Framework):
class Response
{
string Author;
string Comment;
DateTime Date;
}
class Issue
{
IEnumerable<Response> Responses;
}
We have a summary page where we just want to show the last two responses. I tried a linq query like this:
from issue in db.Issue
let responses = from response in issue.Responses orderby response.Date
select new
{
Issue = issue,
Question = responses.FirstOrDefault(),
Answer = responses.Skip(1).FirstOrDefault()
}
But this gives me the error that Skip can only be used on ordered collections. I checked responses and it was an IOrderedEnummerable. I thought maybe the problem was that it was Enumerable instead of IOrderedQueryable, and saw that this happened because issue.Response is a collection so I switched the let statement to be:
let response = from response in db.Responses where response.IssueId = issue.ID // etc.
but this did not resolve the issue (but response did become an IOrderedQueryable) so I'm not really sure why entity won't accept the skip here. If I put the skip in the let statement, it works without problem (but then I can't get the first response). The issue seams to only occur by trying to put a portion of this statement in a variable before using it.
The problem here is: how will/should EF translate your query into a SQL statement? There is no straightforward SQL equivalent of Skip(1). Just try to write your query in SQL and you should see what I mean.
If you want an "easy" solution, then just get all responses from the DB and identify the ones you need in code.
If you want to minimize the data being read from the DB, the solutions might range from creating a view to writing a stored procedure to changing your tables so that your tables better reflect the data model in the application.
I'm not quite sure what's going on here, but wouldn't this be maybe a little simpler:
var theResponse = db.Issue.Select(i => new {
Issue = i,
Question = i.Responses.FirstOrDefault(),
Answer = i.Responses.OrderBy(r => r.Date).Skip(1).FirstOrDefault()
});
But this is also weird, because you are getting a full Issue object with all of its properties and whatnot, including the Reponse objects and stuffing it back into an Issue property of your dynamic type beside all of the Response objects...
Related
So what I am trying to do is write code that will pick an appropriate personnel based on the data that I have to submit a license to a district board. This code is obfuscated from my actual codebase so please bear with the student/class example.
The logic should say, if there was no teacher present, or teacher has no license then use the state school directors license. if we cant find that, use the national.
The fall back works fine when I exclude the state director from the query, it runs without problem. However, when I try to include this, it throws a 'System.InvalidOperationException'. I have verified that all students have a current address and an appropriate state exists in the dictionary.
It looks like my current syntax of using a dictionary in an iqueryable is invalid and EF doesnt know how to work with a dictionary!
var qry = from student in FilteredContext // type is IQueryable<Student>
let studentsAddress = student.Addresses.FirstOrDefault(x=>x.IsCurrent)
let instructor = lastClass == null ? : lastClass.Instructor
let stateSchoolDirector = _personnelContext.Context.FirstOrDefault(x=>x.ID == SchoolDirectors.ByState[currentAddress.State])
let nationalSchoolDirector = _personnelContext.Context.FirstOrDefault(x => x.ID == SchoolDirectors.NationalSchoolDirector)
let personnelForLicense = instructor ?? stateSchoolDirector ?? national
select personnelForLicense.License;
var results = qry.ToList();
So my question is, is there any way to represent a dictionary or something like it within iqueryable? If not, are there any reasonable work arounds? I am totally stuck here and despite seraching could not easily find anything like it on StackOverflow or google results. The closest suggestion I could find was telling me to convert to .AsEnumerable() which I fail to see how it will work with my let statements
Just for going full circle, the only option that I could think of was to convert the dictionary into an actual SQL table/entity on the context
I'm trying to loop through a large table and write the entries to a csv file. If I load all Objects into memory I get an OutOfMemoryException. My Employer class is mapped with fluent nhibernate.
Here's what I've tried:
This Loads all object on first iteration and crashes.
var myQuerable = DataProvider.GetEmployer(); // returns IQuerably
foreach (var emp in myQuerable)
{
// stuff...
}
No luck here:
var myEnumerator = myQuerable.GetEnumerator();
I thought this would work:
for (int i = 0; i <= myQuerable.Count(); i++)
{
Employer e = myQuerable.ElementAt(i);
}
but am getting this exception:
Could not parse expression
'value(NHibernate.Linq.NhQueryable`1[MyProject.Model.Employer]).ElementAt(0)':
This overload of the method 'System.Linq.Queryable.ElementAt' is currently not supported
Am I missing something here? Is this even possible with nHibernate?
Thanks!
I don't think loading your entries one by one could resolve your problem fully, as this is gonna go to another bad direction - huge loading on database side and longer response time for your C# method. I can't imagine how long it will take, as you've already god OutOfMemoryException exception that indicate you have huge number of records. I think the mechanism you really should take is pagination. There're various materials on the Internet about this topic, such as NHibernate 3 paging and determining the total number of rows.
Cheers!
Looks like I'm going to have to follow this artical:
http://ayende.com/blog/4548/nhibernate-streaming-large-result-sets
or use straight ado for performance.
Thanks for the help!
I have a many-to-many relationship between tables of Games and Genres. During an analysis, i need to get items from Games that match specific criteria.
The problem is, to check for this criteria, i need to analyse genres of this specific game. And linq won't let me do it.
My request now looks like this:
var result = GDB.Games.Where((g)=>
g.GamesToGenres.Select((gtg)=>
(weights.ContainsKey(gtg.Genre.Name) ? weights[gtg.Genre.Name]:0.0)
).Sum() > Threshhold
).ToArray();
When I execute it, I receive SQL exception
Only one expression can be specified in the select list when the
subquery is not introduced with EXISTS.
Is there a workaround? How can i perform such Select inside of Where?
EDIT: weights is a Dictionary<string, double>.
EDIT: I was playing with lambdas, and found out a strange thing in their behaviour:
this code won't work, throwing nvarchar to float conversion exception:
Func<string, double> getW = (name) => 1;
var t = GDB.Games.Where((g)=>
g.GamesToGenres.Select((gtg)=>
getW(gtg.Genre.Name)
).Sum() > Threshhold
).ToArray();
but this one will work nicely:
var t = GDB.Games.Where((g)=>
g.GamesToGenres.Select((gtg)=>
1
).Sum() > Threshhold
).ToArray();
This leads me to conclusion that linq lambdas are not usual lambdas. What's wrong with them, then? What are their limitations? What i can and what i can't do inside of them? Why is it ok for me to place a .select call inside of lambda, but not my own call of getW?
RESOLVED. See the answer below. Long story short, C# can't into clojures unless explicitly told so. If anyone knows better answer, i am still confused.
Your problem is you're trying to select something form the dictionary weights that exists in your application and not in your DB. If it was the result of a query to your DB, use the query.Single(...) in its place
Well, i am confused beyond imagination. The following code works perfectly:
Func<Game, bool> predicate = (g) =>
g.GamesToGenres.Select((gtg) =>
(weights.ContainsKey(gtg.Genre.Name) ? weights[gtg.Genre.Name] : 0.0)
).Sum() > Threshhold;
var t = GDB.Games.Where(predicate).ToArray();
careful reader might want to say "Hey! Isn't that the very same code you wrote in the question? You just explicitly assigned it to a variable!", and he will be right. Right now it seems like C# lambda processor is a piece of something, and it creates clojure only when you explicitly declare a lambda. If someone can describe this phenomena to me, i will be gratefull, for right now i am more confused than a newborn baby.
LINQ allows you to combine SQL data with local data like (Dictionary, etc.) with one restriction. You need to select data from SQL first. This means your code will work if you replace GDB.Games.Where with GDB.Games.ToList().Where. You can ask about performance, but you able to select a slice of data like GameId, Genre Name, etc. Then filter out games. Then return end list of full game's info by game ID list.
This seems to be like a common use case... but somehow I cannot get it working.
I'm attempting to use MongoDB as an enumeration store with unique items. I've created a collection with a byte[] Id (the unique ID) and a timestamp (a long, used for enumeration). The store is quite big (terabytes) and distributed among different servers. I am able to re-build the store from scratch currently, since I'm still in the testing phase.
What I want to do is two things:
Create a unique id for each item that I insert. This basically means that if I insert the same ID twice, MongoDB will detect this and give an error. This approach seems to work fine.
Continuously enumerate the store for new items by other processes. The approach I took was to add a second index to InsertID and used a high precision timestamp on this along with the server id and a counter (just to make it unique and ascending).
In the best scenario this would mean that the enumerator would keep track of an index cursor for every server. From what I've learned from mongodb query processing I expected this behavior. However, when I try to execute the code (below) it seems to take forever to get anything.
long lastid = 0;
while (true)
{
DateTime first = DateTime.UtcNow;
foreach (var item in collection.FindAllAs<ContentItem>().OrderBy((a)=>(a.InsertId)).Take(100))
{
lastid = item.InsertId;
}
Console.WriteLine("Took {0:0.00} for 100", (DateTime.UtcNow - first).TotalSeconds);
}
I've read about cursors, but am unsure if they fulfill the requirements when new items are inserted into the store.
As I said, I'm not bound to any table structure or something like that... the only things that are important is that I can get new items over time and without getting duplicate items.
-Stefan.
Somehow I figured it out... more or less...
I created the query manually and ended up with something like this:
db.documents.find({ "InsertId" : { "$gt" : NumberLong("2020374866209304106") } }).limit(10).sort({ "InsertId" : 1 });
The LINQ query I put in the question doesn't generate this query. After some digging in the code I found that it should be this LINQ query:
foreach (var item in collection.AsQueryable().Where((a)=>(a.InsertId > lastid)).OrderBy((a) => (a.InsertId)).Take(100))
The AsQueryable() seems to be the key to execute the rewriting of LINQ to MongoDB queries.
This gives results, but still they appeared to be slow (4 secs for 10 results, 30 for 100). However, when I added 'explain()' I noticed '0 millis' in the query execution.
I stopped the process doing bulk inserts and tada, it works, and fast. In other words: the issues I was having were due to the locking behavior of MongoDB, and due to the way I interpreted the linq implementation. Since the former is the result of initial bulk-filling the data store, this means that the problem is solved.
On the 'negative' part of the solution: I would have preferred a solution that involved serializable cursors or something like that... this 'take' solution has to iterate the b-tree over and over again. If someone has an answer for this, please let me know.
-Stefan.
I have a very strange problem with a LINQ to Entities query with EF1.
I have a method with a simple query:
public DateTime GetLastSuccessfulRun(string job)
{
var entities = GetEntities();
var query = from jr in entities.JOBRUNS
where jr.JOB_NAME == job && jr.JOB_INFO == "SUCCESS"
orderby jr.JOB_END descending
select jr.JOB_END;
var result = query.ToList().FirstOrDefault();
return result.HasValue ? result.Value : default(DateTime);
}
The method GetEntities returns an instance of a class that is derived from System.Data.Objects.ObjectContext and has automatically been created by the EF designer when I imported the schema of the database.
The query worked just fine for the last 15 or 16 months. And it still runs fine on our test system. In the live system however, there is a strange problem: Depending on the value of the parameter job, it returns the correct results or an empty result set, although there is data it should return.
Anyone ever had a strange case like that? Any ideas what could be the problem?
Some more info:
The database we query against is a Oracle 10g, we are using an enhanced version of the OracleEFProvider v0.2a.
The SQl statement that is returned by ToTraceString works just fine when executed directly via SQL Developer, even with the same parameter that is causing the problem in the LINQ query.
The following also returns the correct result:
entities.JOBRUNS.ToList().Where(x => x.JOB_NAME == job && x.JOB_INFO == "SUCCESS").Count();
The difference here is the call to ToList on the table before applying the where clause. This means two things:
The data is in the database and it is correct.
The problem seems to be the query including the where clause when executed by the EF Provider.
What really stuns me is, that this is a live system and the problem occurred without any changes to the database or the program. One call to that method returned the correct result and the next call five minutes later returned the wrong result. And since then, it only returns the wrong results.
Any hints, suggestions, ideas etc. are welcome, never mind, how far-fetched they seem! Please post them as answers, so I can vote on them, just for the fact for reading my lengthy question and bothering thinking about that strange problem... ;-)
First of all remove ObjectContext caching. Object context internally uses UnitOfWork and IdentityMap patterns. This can have big impact on queries.