Develop Reference

EF Core query from Include but no need to return in result

Okay, so this is probably from not knowing how to use EF core correctly as this is my 2nd day using but I appear to have to run .Include() in order to query against the "inner join" this method creates.
I've got this bit working and filtering down my results, but I don't want to return the include in the model as it's making a 256kb JSON response turn into a 2.8Mb JSON response.
public IEnumerable<Property> GetFeaturedProperties()
{
var properties = _db.Properties
.Include(p => p.GeoInfo)
.Include(p => p.FeaturedUntil)
.ToArray();
var featuredProperties = new List<Property>();
var DateTimeNow = DateTime.Now;
var midnightToday = DateTimeNow.AddHours(-DateTimeNow.Hour)
.AddMinutes(-DateTimeNow.Minute)
.AddSeconds(-DateTimeNow.Second-1);
for (var i = 0; i < properties.Count(); i++)
{
var property = properties[i];
if(property.FeaturedUntil.Any(p => p.FeaturedDate >= midnightToday))
featuredProperties.Add(property);
}
return featuredProperties;
}
So the offending line is .Include(p => p.FeaturedUntil). As this is an Array of dates that can be up anything from 10-1000 rows per joined row. It includes ALL data, even historical so this is really racking up data.
Can I run my query and then run something to .RemoveInclude(p => p.FeaturedUntil)?

You don't need to load the navigation properties in order to apply filtering. When you access a navigation property inside LINQ to Entities query, it's translated to the corresponding SQL construct, including JOINs. No real object/collections are involved. The whole query (with some exceptions) executes at server (database) side.
In your case, the following simple query will do the job:
public IEnumerable<Property> GetFeaturedProperties()
{
var DateTimeNow = DateTime.Now;
var midnightToday = DateTimeNow.AddHours(-DateTimeNow.Hour)
.AddMinutes(-DateTimeNow.Minute)
.AddSeconds(-DateTimeNow.Second-1);
return _db.Properties
.Include(p => p.GeoInfo) // assuming you want to return this data
.Where(p => p.FeaturedUntil.Any(p => p.FeaturedDate >= midnightToday))
.ToList();
}
For more info, see How Queries Work documentation topic.

How to use First Method in querying data with Entity Datasource?

I have the following code:
var orders = context.Orders
.Include("Clients")
.GroupBy(i => i.Clients.ClientName)
.Select(i => i.OrderByDescending(it => it.OrderDate).FirstOrDefault());
I want to get only the last order made by each client. The basis for the code I got from here Remove duplicates in the list using linq, from Freddy's answer. (I'm including "Clients" because Orders has ClientId but not client name and the results are to be displayed in a grid including the client name which I'm getting from Clients).
This works properly.
MY QUESTION:
Is it possible to do this using an asp.net Entity Datasource control?
Is it possible to use FirstOrDefault in some way in the asp.net Entity Datasource control?

If you move the Include to the end of the query ...
var orders = context.Orders
.GroupBy(i => i.Clients.ClientName)
.Select(i => i.OrderByDescending(it => it.OrderDate).FirstOrDefault())
.Include("Clients");
... you'll get Orders with their clients included.
With the original Include the query shape changes after the Include was applied. This always makes Include ineffective. You can move it to the end of the query, because even after the grouping, the query still return Orders, so the Include is applicable.
Note however that this is a tremendous overkill. Entire Client records are queried from the database, entire Client objects are materialized and in the end you only display their names. It's much better to project the required data to a DTO that exactly contains the data you want to display. For example:
var orders = context.Orders
.GroupBy(i => i.Clients.ClientName)
.Select(i => i.OrderByDescending(it => it.OrderDate).FirstOrDefault())
.Select(o => new OrderDto
{
o.OrderNumber,
o. ... // More Order properties
Client = o.Clients.Name
});
The DTO should be a class containing these properties.
I don't know the Entity Datasource control. From what I see in the MSDN documentation it seems too restricted to even shape the query sufficiently to get the last orders of each client. And it expects an entity set, no DTOs.

Instead of calling OrderbyDescending try using the max operated as explained here

I found here that with the EntityDataSource you can use:
Select="top(1) it.[OrderDate]"
However if you want to Order by DESC the top will be executed before the Order by DESC.
If you want the Order by executed before the top, in other words to get the last Item, instead of top do this in Code behind:
protected void entityDataSource_Selecting(object sender,EntityDataSourceSelectingEventArgs e)
{
e.SelectArguments.MaximumRows = 1;
}
All of this I got from that link in the Qustion and Answer.

I found that I can use the EntityDataSource's QueryCreated event as demonstrated in Filter with EntityDatasource in ASP.NET webforms in the question and answers.
In the case of this question I wrote
protected void EntityDataSource1_QueryCreated(object sender, QueryCreatedEventArgs e)
{
var ordersQuery = e.Query.OfType<Orders>();
e.Query = ordersQuery.Include("Clients")
.GroupBy(i => i.Clients.ClientName)
.Select(i => i.OrderByDescending(it => it.OrderDate).FirstOrDefault());
}

Using EF Include on TPH

I have implemented code first db architecture with simple inheritance using THP:
And I need to query all notifications of all type.
TargetUser property in NotificationUser table is a association.
I'm trying to execute next code:
var notifications = _context.Notifications;
foreach (var notification in notifications)
{
Debug.WriteLine((notification is NotificationUser)? ((NotificationUser) notification).TargetUser?.Name : "-");
}
In database propety TargetUser is set to correct foreign key, but in code I don't get any result. Lazy loading is enabled.
Is it possible to user eager loading? I had already tried to write _context.Notifications.Include('TargetUser') byt it throws an exception.
Upd. The exception is:
A specified Include path is not valid. The EntityType 'Core.Concrete.NotificationBase' does not declare a navigation property with the name 'TargetUser'.
Tried to modify this answer to:
var notifications = _context.Notifications.OfType<NotificationUser>()
.Include(n => n.TargetUser)
.Cast<NotificationBase>()
.Union(_context.Notifications.OfType<NotificationPlace>()
but still the same exception is thrown.

I know this is an old thread, but I'd still like to post some improvements for somebody looking for the same solution.
1. Network Redundancy
Selecting Ids and then running a query, that loads items with the Ids is redundant and the same effect can be achieved by simply running this
Solution:
var userNotifications = _context.Notifications
.OrderByDescending(n => n.DateTime)
.Skip(offset)
.Take(limit)
.OfType<NotificationUser>()
.Include(n => n.TargetUser)
.Include(n => n.TargetUser.Images)
.ToList();
That way, you aren't waiting for 2 DB connections, but just one. Also you save some traffic.
2. Paging on ignored entities?
One would assume, that this specific method is used for only viewing Entities of an inherited type so I would expect Skip and Take to work directly on only entities of said type. e.g. I want to skip 10 NotificationUsers, not 10 Users (of which only 4 are NotificationUsers for example).
Solution: Move ofType higher up the query
var userNotifications = _context.Notifications
.OfType<NotificationUser>()
.OrderByDescending(n => n.DateTime)
.Skip(offset)
.Take(limit)
.Include(n => n.TargetUser)
.Include(n => n.TargetUser.Images)
.ToList();
3. Async/Await
When writing an API, you should think about using async/await as that doesn't block the thread thus wastes less resources (this will probably require you to rewrite a lot of your existing code if you don't use it already though).
Please study the advantages of async/await and use them in scenarios like waiting for a result.
Solution: Change this
private List<NotificationUser> GetNotificationUsers(int offset, int limit)
{
return _context.Notifications
.OfType<NotificationUser>()
.OrderByDescending(n => n.DateTime)
.Skip(offset)
.Take(limit)
.Include(n => n.TargetUser)
.Include(n => n.TargetUser.Images)
.ToList();
}
into this
private async Task<List<NotificationUser>> GetNotificationUsersAsync(int offset, int limit)
{
return await _context.Notifications
.OfType<NotificationUser>()
.OrderByDescending(n => n.DateTime)
.Skip(offset)
.Take(limit)
.Include(n => n.TargetUser)
.Include(n => n.TargetUser.Images)
.ToListAsync();
}
NOTE:
You also then have to change any place that uses this method from
var x = GetNotificationUsers(skip, take);
to
var x = await GetNotificationUsersAsync(skip, take);
And make that method async and return a task as well

I don't know about the amount of entities you will work with. If possible, I would try to do the union not on the DB server:
var userNotifications = _context.Notifications.OfType<NotificationUser>()
.Include(n => n.TargetUser).ToList();
var placeNotifications = _context.Notifications.OfType<NotificationPlace>().ToList();
var notifications = userNotifications.Union(placeNotifications);
See https://stackoverflow.com/a/27643393/2342504

Already tried a lot of different solutions, and none fit my requirements, as I'm working on an API, and the query must support pagination and make constant requests to the database (and not loading all entities in memory).
Finally found out a solution, perhaps not the best, but enough for now.
Firstly, I request a portion of ordered data (pagination logic):
var notifications = _context.Notifications
.OrderByDescending(n => n.DateTime)
.Skip(offset)
.Take(limit);
(At this point I'm not interested in any properties) Next, I'm getting the Id's of loaded items for each entity type:
var ids = notifications.OfType<NotificationUser>().Select(n => n.Id).ToList();
and lastly load specific entities including all properties:
var userNotifications = _context.Notifications.OfType<NotificationUser>()
.Include(n => n.TargetUser)
.Include(n => n.TargetUser.Images)
.Where(n => ids.Contains(n.Id))
.ToList();
all entities goes to list and sorted one more time.
A lot of bad stuff here, hope someone can provide better solution.

If I understood well, you want to get all the entities from the table + the property TargetUser when relevant (for entities of type NotificationUser). You said that "Lazy loading is enabled", which is a good thing.
You tried something like (updated to latest version of C#):
var notifications = _context.Notifications;
foreach (var notification in notifications)
{
Debug.WriteLine((notification is NotificationUser notificationUser)
? notificationUser.TargetUser?.Name
: "-");
}
If you don't get any result, it probably means that your entities are badly configured to work with lazy loading:
Are your classes public?
Are your navigation properties defined as public, virtual?
See: https://learn.microsoft.com/en-us/ef/ef6/querying/related-data#lazy-loading

How to do mongodb queries faster?

I have lots of queries like sample1,sample2 and sample3. There are more than 13 million records in mongodb collection. So this query getting long time. Is there any way to faster this query?
I think using IMongoQuery object to resolve this problem. Is there any better way?
Sample 1:
var collection = new MongoDbRepo().DbCollection<Model>("tblmodel");
decimal total1 = collection.FindAll()
.SelectMany(x => x.MMB.MVD)
.Where(x => x.M01.ToLower() == "try")
.Sum(x => x.M06);
Sample 2:
var collection = new MongoDbRepo().DbCollection<Model>("tblmodel");
decimal total2 = collection.FindAll().Sum(x => x.MMB.MVO.O01);
Sample 3:
var list1= collection.FindAll()
.SelectMany(x => x.MHB.VLH)
.Where(x => x.V15 > 1).ToList();
var list2= list1.GroupBy(x => new { x.H03, x.H09 })
.Select(lg =>
new
{
Prop1= lg.Key.H03,
Prop2= lg.Count(),
Prop3= lg.Sum(w => w.H09),
});

The function FindAll returns a MongoCursor. When you add LINQ extension methods on to the FindAll, all of the processing happens on the client, not the Database server. Every document is returned to the client. Ideally, you'll need to pass in a query to limit the results by using Find.
Or, you could use the AsQueryable function to better utilize LINQ expressions and the extension methods:
var results = collection.AsQueryable().Where(....);
I don't understand your data model, so I can't offer any specific suggestions as to how to add a query that would filter more of the data on the server.
You can use the SetFields chainable method after FindAll to limit the fields that are returned if you really do need to return every document to the client for processing.
You also might find that writing some of the queries using the MongoDB aggregation framework might produce similar results, without sending any data to the client (except the results). Or, possibly a Map-Reduce depending on the nature of the data.

using nhibernate, how would i construct a query that wants the most recent 50 rows

i have a table called orders and i have a column called Last Update (and an order object with a LastUpdate property). I want to construct a query using nhibernate to get the last 50 rows so i don't go to the database and get everything and then have to filter results in my application.
is this possible in nhibernate. I am trying to use the LINQ api

Here's the LINQ version of this query.
var orders = session.Query<Order>()
.OrderByDescending(x => x.LastUpdate)
.Take(50);
Here's the screen shot of the code sample...
Here's the screen shot from NHibernate Profiler...

If you are using a Criteria then use SetMaxResults(50) and do a descending sort on the date time.

You can use SetMaxResults(50), although depending on which 50 rows you want (latest? first? last?) you'll probably also need to do a SortBy expression as well.

var orders = session.Query<Linq>()
.OrderByDescending(x => x.LastUpdate)
.Take(50);

In general case suggesing LastUdate can be nullable using Linq2SQL you may write extension method to your IQueriable:
public static partial class FooTable
{
public static IQueryable<FooTable> LastUpdated(this IQueryable<FooTable> queryable, int count)
{
return queryable.Where(x => (x.LastUdate != null))
.OrderByDescending(x => x.LastUdate)
.Take(count);
}
}