For some odd reason, I am unable to access the properties of this object EVEN if I cast it as its model type. Does anyone have an idea as to why? (It may be obvious, but I'm pretty new to C# so please be patient! :o) )
Users currentUser = new Users();
currentUser = (from x in db_tc.Users where x.Id == Convert.ToInt32(User.Identity.Name) select x);
When I call currentUser, I am only able to access the CRUD methods and a List<Users> property called usrList. I didn't create the list definition, so I imagine this is some piece of the Entity framework that is automagically created.
I did try casting currentUser with (Users) prior to the entity query, it didn't help at all.
That's because you've only created the query, you haven't actually executed it. Add Single() (or First() etc.) to get the result:
var currentUser = (from x in db_tc.Users where x.Id == Convert.ToInt32(User.Identity.Name) select x).SingleOrDefault();
Single(): Gets the first element of the sequence, but will throw an exception if no element is found or if the sequence has more than one element.
First(): Gets the first element of the sequence, but will throw an exception if no element is found.
SingleOrDefault() and FirstOrDefault(): Same as above, but will return default(T) instead of throwing on the empty sequence.
This "deferred execution" aspect of LINQ is probably the most difficult part to understand. The basic idea is that you can build up a query using the query operations (Where(), Select(), etc.), and then you can execute that query to actually get its results, using one of the non-deferred execution operations (Single(), ToList(), etc.)
The operation you have here will return a list of matches the DB will return, it is a collection.
If you intend on it only returning a single record append .First(); to the end of your linq query.
Also remove this line Users currentUser = new Users();
and add this var currentUser =...
Some more tips from the "good LINQ practices" wagon...
LINQ should "usually" return var, then you convert to the data type you are expecting. Another good practice I have found is to immediately validate the return from LINQ, as any usage without validation is highly exception prone. For instance:
var qUser = (from x in db.Users select x);
if (qUser != null && currentUser.Count() > 0)
{
List<User> users = (List<User>)qUser.ToList();
... process users result ...
}
The not null and count greater than 0 check should be required after each LINQ query. ;)
And don't forget to wrap in try-catch for SqlException!
Related
I have an endpoint which should list all Candidatures filtered by search term, so what I do is to create a method which accepts candidature entity and searchTerm as params. Then I pass that method to Where clause, but the problem is that I got NullReferenceException because navigation properties are nulls. If I put statement inside Where clause instead of the method then it doesn't throw Exception. The question is how to fix this, but I want to keep the external method because there will be a lot more logic, but I need to have access to all navigation properties i.e. they should be populated.
if (!string.IsNullOrEmpty(searchTerm))
{
query = query.Where(c => FilterBySearchTerm(c, searchTerm));
}
var result = await query.Select(c => new CandidaturesResponseModel()
{
Id = c.Id,
Name = c.PrimaryMember.FullName, // that's filled
}).ToListAsync();
private bool FilterBySearchTerm(Candidature c, string searchTerm)
{
return c.PrimaryMember.FirstName.Contains(searchTerm); // here is the exception because PrimaryMember navigation property is null. So I want this to be filled.
}
The issue is that you're materializing the query by using your FilterBySearchTerm method. EF cannot translate random methods to SQL, so it has to go ahead and run the query, get the results back and then apply your Where. EF would actually throw an exception in the past, but EF Core handles this silently.
Anyways, once the query is run, you're done. Your filtering is happening in-memory, and at that point, without an Include, your related entities are not there to work with. Long and short, you'll need to build your filter in place (rather than using a separate method) in order for EF to be able to translate that to SQL.
An alternative approach which may serve you better is pass a queryable to your FilterBySearchTerm method. For example, instead of doing:
query = query.Where(c => FilterBySearchTerm(c, searchTerm));
Do
query = FilterBySearchTerm(query, searchTerm);
Then, inside FilterBySearchTerm, you can directly apply Where clauses to the passed in query. That allows you to build an actual query that EF can understand, while also encapsulating the logic.
Just use Include method to add PrimaryMember
query = query.Include(x=> x.PrimaryMember).Where(c => FilterBySearchTerm(c, searchTerm));
I'm using for a GetAppRolesForUser function (and have tried variations of based on answers here):
private AuthContext db = new AuthContext();
...
var userRoles = Mapper.Map<List<RoleApi>>(
db.Users.SingleOrDefault(u => u.InternetId == username)
.Groups.SelectMany(g => g.Roles.Where(r => r.Asset.AssetName == application)));
I end up with this in SQL Profiler for every single RolesId each time:
exec sp_executesql N'SELECT
[Extent2].[GroupId] AS [GroupId],
[Extent2].[GroupName] AS [GroupName]
FROM [Auth].[Permissions] AS [Extent1]
INNER JOIN [Auth].[Groups] AS [Extent2] ON [Extent1].[GroupId] = [Extent2].[GroupId]
WHERE [Extent1].[RolesId] = #EntityKeyValue1',N'#EntityKeyValue1 int',#EntityKeyValue1=6786
How do I refactor so EF produces a single query for userRoles and doesn't take 18 seconds to run?
I think the problem is you're lazy loading the groups and roles.
One solution is eager load them before you call SingleOrDefault
var user = db.Users.Include(x => x.Groups.Select(y => y.Roles))
.SingleOrDefault(u => u.InternetId == username);
var groups = user.Groups.SelectMany(
g => g.Roles.Where(r => r.Asset.AssetName == application));
var userRoles = Mapper.Map<List<RoleApi>>(groups);
Also note : there is no sanity checking for null here.
TheGeneral's answer covers why you are getting caught out with lazy loading. You may also need to include Asset to get AssetName.
With AutoMapper you can avoid the need to Eager Load the entities by employing .ProjectTo<T>() to the IQueryable, provided there is a User accessible in Group.
For instance:
var roles = db.Groups.Where(g => g.User.Internetid == username)
.SelectMany(g => g.Roles.Where(r => r.Asset.AssetName == application))
.ProjectTo<RoleApi>()
.ToList();
This should leverage the deferred execution where AutoMapper will effectively project in the .Select() needed to populate the RoleApi instance based on your mapping/inspection.
Here is another way of avoiding lazy loading. You can also look at projection and have only those fields which you need rather than loading the entire columns.
var userRoles = Mapper.Map<List<RoleApi>>(
db.Users.Where(u => u.InternetId == username).Select(../* projection */ )
.Groups.SelectMany(g => g.Roles.Where(r => r.Asset.AssetName == application)));
EF also comes with Include:
var userRoles = Mapper.Map<List<RoleApi>>(
db.Users.Where(u => u.InternetId == username).Select(../* projection */ )
.Include(g => g.Roles.Where(r => r.Asset.AssetName == application)));
Then can iterate the collection using multiple for loops.
You have to be aware of two differences:
The difference between IEnumerable and IQueryable
The difference between functions that return IQueryable<TResult> (lazy) and functions that return TResult (executing)
Difference between Enumerable and Queryable
. A LINQ statement that is AsEnumerable is meant to be processed in your local process. It contains all code and all calls to execute the statement. This statement is executed as soon as GetEnumerator and MoveNext are called, either explicitly, or implicitly using foreach or LINQ statements that don't return IEnumerable<...>, like ToList, FirstOrDefault, and Any.
In contrast, an IQueryable is not meant to be processed in your process (however it can be done if you want). It is usually meant to be processed by a different process, usually a database management system.
For this an IQueryable holds an Expression and a Provider. The Expression represents the query that must be executed. The Provider knows who must execute the query (the DBMS), and which language this executor uses (usually SQL). When GetEnumerator and MoveNext are called, the Provider takes the Expression and translates it into the language of the Executor. The query is sen't to the executor. The returned data is presented AsEnumerable, where GetEnumerator and MoveNext are called.
Because of this translation into SQL, an IQueryable can't do all the things that an IEnumerable can do. The main thing is that it can't call your local functions. It can't even execute all LINQ functions. The better the quality of the Provider the more it can do. See supported and unsupported LINQ methods
Lazy LINQ methods and executing LINQ methods
There are two groups of LINQ methods. Those that return `IQueryable<...>/IEnumerable<...> and those that do not.
The first group use lazy loading. This means that at the end of the LINQ statement the query has been created, but it is not executed yet. Only 'GetEnumeratorandMoveNextwill make that theProviderwill translate theExpression` and order the DBMS to execute the query.
Concatenating IQueryables will only change the Expression. This is a fairly fast procedure. Hence there is no performance gain if you make one big LINQ expression instead of concatenate them before you execute the query.
Usually the DBMS is smarter and better prepared to do selections than your process. The transfer of selected data to your local process is one of the slower parts of your query.
Advice: Try to create your LINQ statements such, that the executing
statement is the last one that can be executed by the DBMS. Make sure
that you only select the properties that you actually plan to use.
So for example, don't transfer foreign keys if you don't use them.
Back to your question
Leaving the mapper out of the question you start with:
db.Users.SingleOrDefault(...)
SingleOrDefault is a non-lazy function. It doesn't return IQueryable<...>. It will execute the query. It will transport one complete User to your local process, including its Roles.
Advice postpone the SingleOrDefault to the last statement:
var result = myDbcontext.Users
.Where(user => user.InternetId == username)
.SelectMany(user => user.Groups.Roles.Where(role => role.Asset.AssetName == application))
// until here, the query is not executed yet, execute it now:
.SingleOrDefault();
In words: From the sequence of Users, keep only those Users with an InternetId that equals userName. From all remaining Users (which you hope to be only one), select the sequence of Roles of the Groups of each User. However, we don't want to select all Roles, we only keep the Roles with an AssetName equal to application. Now put all remaining Roles into one collection (the many part in SelectMany), and select the zero or one remaining Role that you expect.
I have a query that looks something like this
return (from item in context.MERCHANDISE_ITEMS
where item.ORGID == orgId
select new MerchandiseModel()
{
Description = item.DESCRIPTION,
...,
Images = context.MERCHANDISE_IMAGES.Where(i => i.ITEMID == item.ID).ToDictionary(i => i.ID, i => i.IMAGENAME)
}).FirstOrDefault();
Images is a Dictionary and MERCHANDISE_IMAGES.ID is int and MERCHANDISE_IMAGES.IMAGENAME is string.
This is throwing System.NotSupportedException, I've tried doing linq (no lambda), same exception.
Here is the exception:
LINQ to Entities does not recognize the method
'System.Collections.Generic.Dictionary`2[System.Int32,System.String] ToDictionary[MERCHANDISE_IMAGES,Int32,String](System.Collections.Generic.IEnumerable`1[Hylton.Infrastructure.EntityFrameworkORM.MERCHANDISE_IMAGES],
System.Func`2[Hylton.Infrastructure.EntityFrameworkORM.MERCHANDISE_IMAGES,
System.Int32], System.Func`2[Hylton.Infrastructure.EntityFrameworkORM.MERCHANDISE_IMAGES,System.String])' method,
and this method cannot be translated into a store expression.
I have no idea what I'm doing wrong, I've looked through every post on this issue and everything I've seen is doing it the same way. Can anyone point out what I'm missing?
Short version: EF can't convert ToDictionary to SQL.
In my experience, the best way to go is to simplify the Select to an anonymous type with all the stuff from the DBOs you'll need, call ToList (or something else to force the evaluation) and then do the ToDictionary and other massaging of the data.
The added benefit of this is by forcing it to evaluate, you'll avoid leaving the using your context is in and closing it, which will cause an error when you try and use the return.
I am using LINQ-to-Entities (EF 6.1.3) to perform the following query:
var users = msgList.Select(m => m.From)
.Union(msgList.Select(m => m.To))
.Distinct()
.Where(u => u.ID != userId) //userId is an assigned local var.
.ToList();
msgList is a List (already fetched, not a queryable and lazy loading is off) of Messages which consists of some fields like From and To which are guaranteed to be non-null. Both From and To were Included in the original query, so they are guaranteed to be non-null.
My User object is also guaranteed to be non-null, so there's nothing that can actually be null.
However, this line is sometimes throwing a null pointer exception, and sometimes executing perfectly with the exact same user, exact same database, exactly same data (nothing altered). Load is not an issue as it's a code not yet in production and I'm the only one testing it.
The exception seems to be thrown at the Where call:
at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
How can this happen?
UPDATE: This is of course not a duplicate of What is a NullReferenceException, and how do I fix it?. Any sane developer with even a little knowledge in .NET/C#/OOP knows what that error is and that this question has nothing to do with it, even though it involves that exception as a part of it.
UPDATE 2: I've switched it to assigning to a list each line, as suggested below:
var msgListSelection = msgList.Select(m => m.From).ToList();
var union = msgListSelection.Union(msgList.Select(m => m.To)).ToList();
var distinct = union.Distinct().ToList();
var where = distinct.Where(u => u.ID != userId).ToList();
var users = where;
The exception occurs at the where line:
var where = distinct.Where(u => u.ID != User.ID).ToList();
If distinct returned null, it would have been thrown on ToList call of var distinct = union.Distinct().ToList(); on the line above.
Am I missing something?
UPDATE 2: My User class is a POCO C# type mapped to an Entity type in my database which has an ID property of long, and my Message class is again a POCO type mapped in Entity Framework, with navigation properties From and To to some User instances guaranteed to be non-null. They are annotated as Required and I've also checked them at the database level just to be sure.
UPDATE 3: My EF context lives from the beginning of the request (set at a delegating handler in the beginning of the request) to the end. I don't think the problem is related to the lifespan of the DbContext as there are many controllers with the same mechanism with tens of methods that access the context, and I'm only having such problem with this particular method.
UPDATE 4: I've added a null check on distincts:
var distinct = union.Distinct().ToList();
if(distinct == null)
{
throw new Exception("distinct was null");
}
var where = distinct.Where(u => u.ID != userId).ToList();
It seems to pass that point with no problem, but throw the null pointer exception at the last line var where = distinct.Where(u => u.ID != userId).ToList(); which sorts out the possibility that distinct may be null.
UPDATE 5: I've wrote an API testing tool and sent about 250 requests to the same endpoint with the same user. The first one failed with this error, and all the rest succeeded successfully. There seems to be a problem with the first request.
You may be experiencing what is caused by the closure principle. You reference the User property in your LINQ query. Because the LINQ query in itself is executed as an (anonymous) method delegate, the closure principle applies.
Quoting the above link:
In essence, a closure is a block of code which can be executed at a
later time, but which maintains the environment in which it was first
created - i.e. it can still use the local variables etc of the method
which created it, even after that method has finished executing.
The usage of the User property is subject to this principle. Its value can have changed upon the execution of the LINQ query. To protect against this, the User property should be copied to a local variable and that referenced in the LINQ query. Like so:
var user = User;
var users = msgList.Select(m => m.From)
.Union(msgList.Select(m => m.To))
.Distinct()
.Where(u => u.ID != user.ID)
.ToList();
Update
When using a local reference copy to the user property, another possibility for the NullReferenceException may lie with the Select-Union-Distinct methods. When calling ToList, the Where clause is executed on all items in the union of the two Select clauses. By default, Distinct executes the Equals method from the IQuality interface, which would be called on the elements from Select(m => m.From) . If this element is null, it would cause the NullReferenceException.
I query a database and push the result out to the console and a file using two methods like this:
var result = from p in _db.Pages
join r in rank on p.PkId equals r.Key
orderby r.NumPages descending
select new KeyNameCount
{
PageID = p.PkId,
PageName = p.Name,
Count = r.NumPages
};
WriteFindingsToFile(result, parentId);
WriteFindingsToConsole(result, parentId);
IEnumerable<T>, not IQuerable<T> is used as parametertype for result when used as a parameter in the two method calls.
In both calls the result is iterated in a foreach. This leads to two identical calls against the database, one for each method.
I could refactor the two methods into one and only use one foreach, but that would fast become very hard to maintain (adding write to html, write to xml, etc.)
I am pretty sure this is a farly common question, but using google has not made me any wiser, so I turn to you guys :-)
Every time you access a LINQ query it will requery the database to refresh the data. To stop this happening use .ToArray() or .ToList().
e.g.
var result = (from p in _db.Pages
select p).ToList(); //will query now
Write(result); //will not requery
Write(result2); //will not requery
It's important to understand that a raw LINQ query is run when it is used, not when it is written in the code (e.g. don't dispose of your _db before then).
It can be surprising when you realise how it really works. This allows method chaining and later modification of the query to be reflected in the final query run on the DB. It is important to always keep in mind as it can cause run-time bugs that will not be caught at compile time, usually because the DB connection is closed before the list is used, as you are passing around what appear to be a simple IEnumerable.
EDIT: Changed to remove my opinion and reflect the discussion in the comments. Personally I think the compiler should assume that the end result of chained queries is immediately run unless you explicitly say that it'll be further modified later. Just to avoid the run-time bugs it inevitably causes.
If you look at the function definition for IQuerayble, you will see that it also implements IEnumerable. So you can pass IQueryable as parameter to an IEnumerable function without actually enumerating it.
But because of Linqs deffered execution pattern, the IQueryable will only be executed against the database when you iterate over it (with a for loop for example as in your case, or with ToList() or functions like First/Single).
Here is a blog post that explains how this works.
If you change your code to the following you will hit the database only once and then pass the result in memory to your functions:
var result = (from p in _db.Pages
join r in rank on p.PkId equals r.Key
orderby r.NumPages descending
select new KeyNameCount
{
PageID = p.PkId,
PageName = p.Name,
Count = r.NumPages
}).ToList();
WriteFindingsToFile(result, parentId);
WriteFindingsToConsole(result, parentId);
Any time you iterate over your LINQ result the databased will be queried too.
This is called Deferred query execution. You may have a deeper look into at one (out of many) corresponding MSDN articles!
Execution of the query is deferred until the query variable is iterated over in a foreach or For Each loop