I would like to be able to reuse fragments of my select lambda expressions in my Entity Framework Core 2.0 queries.
For example:
var result = await ctx.Customers
.Select(cust => new CustomerDto {
CustomerId = cust.Id,
CustomerName = cust.Name,
CurrentValue = cust.Orders
.Where(order => order.OrderDate >= DateTime.Now.AddDays(-30)
.Sum(order => order.TotalValue)
})
.ToListAsync();
Since I might want to calculate the CurrentValue property in other queries (in practice the sub-query is more complex than this), I would ideally like to refactor the above code to something like:
var result = await ctx.Customers
.Select(cust => new CustomerDto {
CustomerId = cust.Id,
CustomerName = cust.Name,
CurrentValue = CalculateCustomerCurrentValueExpr(cust)
})
.ToListAsync();
I have created Linq predicates using a Linq.Expression, but I have been unable to find a way to use an Expression as an element of the select statement.
Any help would be much appreciated.
Update - Performance with .AsExpandable()/.Invoke()
For anyone interested, I ran some test code ten times which produced the following result:
Standard Inline Code: 17ms (58,609 ticks)
With .AsExpandable() and inline code 16ms (58,029 ticks)
With .AsExpandable() and .Invoke() 16ms (58,224 ticks)
I suspect that if more test cycles had been run, the average processing time for all three scenarios would have been the same - at least with the level of accuracy I could measure at (simple StopWatch()).
Thanks to all contributors, particularly SergeyA for the solution and Ivan Stoev for the simple explanation of .AsExpandable()
You can reuse expressions with AsExpandable extension from LinqKit liblary (http://www.albahari.com/nutshell/linqkit.aspx).
Example:
Expression<Func<Customer,long>> func = c => c.Orders
.Where(order => order.OrderDate >= DateTime.Now.AddDays(-30)
.Sum(order => order.TotalValue);
var result = await ctx.Customers
.AsExpandable() // this allow to unwrap injected expression
.Select(cust => new CustomerDto {
CustomerId = cust.Id,
CustomerName = cust.Name,
CurrentValue = func.Invoke(cust) // this inject predefined expression
})
.ToListAsync();
I store my expressions in a static file and reuse the expressions where I need them to ensure to include all related data. Maybe this can work for you as well
In GetStore() I reuse an expression called ClientAccess and passes that to the expression at ShopExpressions.
GetPage() uses a simple straight forward implementation.
ShopExpressions.cs:
public static IQueryable<IStore> StoreLite(IQueryable<IStore> dbSet)
{
var result = dbSet
.Include(str => str.VATs)
.ThenInclude(vat => vat.VAT)
.ThenInclude(vat => vat.Culture)
.ThenInclude(cult => cult.Items)
.ThenInclude(itm => itm.Culture)
.Include(str => str.Options)
.ThenInclude(opt => opt.Items)
.ThenInclude(itm => itm.Option)
.Include(str => str.Cultures)
.ThenInclude(cult => cult.Items)
.ThenInclude(itm => itm.Culture)
.ThenInclude(cult => cult.Items)
.ThenInclude(itm => itm.Culture)
.Include(str => str.Pages)
.ThenInclude(page => page.Sections)
.ThenInclude(section => section.Elements);
return result;
}
public static IQueryable<IStore> Store(IQueryable<IStore> dbSet)
{
var result = StoreLite(dbSet)
.Include(str => str.Categorys)
.ThenInclude(cat => cat.Products)
.ThenInclude(prd => prd.InfoItems)
.ThenInclude(itm => itm.Culture)
.ThenInclude(cult => cult.Items)
.ThenInclude(itm => itm.Culture);
return result;
}
public static IQueryable<IPage> Page(IQueryable<IPage> dbSet)
{
var result = dbSet
.Include(page => page.Sections)
.ThenInclude(sec => sec.Elements)
.Include(page => page.CSS)
.Include(page => page.Script)
.Include(page => page.Meta);
return result;
}
Controller.cs:
[HttpGet]
public async Task<IStore> GetStore(int id)
{
IStore result = await ShopExpressions.Store(GenericExpressions.ClientAccess(this.Worker.GetRepo<Store>().DbSet))
.SingleAsync(str => str.Id.Equals(id));
this.Worker.ValidateClientAccess(result);
return result;
}
[HttpGet]
public async Task<IStore> GetStoreLite(int id)
{
IStore result = await ShopExpressions.StoreLite(GenericExpressions.ClientAccess(this.Worker.GetRepo<Store>().DbSet))
.SingleAsync(str => str.Id.Equals(id));
this.Worker.ValidateClientAccess(result);
return result;
}
[HttpGet]
public async Task<IPage> GetPage(int id)
{
IPage result = await ShopExpressions.Page(this.Worker.GetRepo<Page>().DbSet)
.SingleAsync(page => page.Id.Equals(id));
return result;
}
Related
I recently converted my code to more LINQ based code. But ReSharper warns me that the FindAsync function may return null. I don't want to check for null as I'm a bit obsessed with order. Does not checking for null cause problems in the future?
One of the old methods:
public async Task<IList<Post>> GetPostsByTagIdAsync(int tagId)
{
var posts = new List<Post>();
var postIds = context.PostsTags
.Where(pt => pt.TagId == tagId)
.Select(p => p.PostId);
foreach (var id in postIds)
posts.Add(await context.Posts.FindAsync(id));
return posts;
}
New version:
public async Task<IList<Post>> GetPostsByTagIdAsync(int tagId) =>
await context.PostsTags
.Where(pt => pt.TagId == tagId)
.Select(p => context.Posts.FindAsync(p.PostId).Result)
.ToListAsync();
Both versions are ineffective. Better to ask for Posts in one database roundtrip.
public Task<IList<Post>> GetPostsByTagIdAsync(int tagId)
{
var query =
from pt in context.PostsTags
join p in context.Posts on pt.PostId equals p.Id
where pt.TagId == tagId
select p;
return query.ToListAsync();
}
Also query can be simplified if you have right navigation properties
public Task<IList<Post>> GetPostsByTagIdAsync(int tagId) =>
context.PostsTags
.Where(pt => pt.TagId == tagId)
.Select(pt => pt.Post)
.ToListAsync();
Note that, I have removed async - it will also speedup execution, because compiler do not create additional state machine for handling asynchronous operations.
return await result.Select(student => new MarkSheetsStudentByIdDto
{
Id = student.RegId,
FullName = student.FullName,
AnnualMarkSheets = student.TermOne
.Select(x => new MarkSheetDto
{
Rank = x.Rank
...
Comments = student.Comments.Where(x => x.StudentId.Equals(student.RegId)).Select(x => x.CommentText)
}).Union(student.TermTwo
.Select(x => new MarkSheetDto
{
Rank = x.Rank
...
Comments = student.Comments.Where(x => x.StudentId.Equals(student.RegId)).Select(x => x.CommentText)
})).OrderBy(c => c.Rank).ToList()
}).ToList();
For the above example code snippet, I am getting the following error at runtime.
42601: syntax error at or near \"SELECT\"\r\n\r\nPOSITION: 5680
I used ToList() method otherwise I am getting the following error.
Collections in the final projection must be an 'IEnumerable' type
such as 'List'. Consider using 'ToList' or some other mechanism to
convert the 'IQueryable' or 'IOrderedEnumerable' into an
'IEnumerable'.
Can anyone please guide me on how to address this scenario?
Try to load data via Include and then do projection on the client-side:
var rawResult = await result
.Include(x => x.Comments)
.Include(x => x.TermOne)
.Include(x => x.TermTwo)
.Take(1)
.ToListAsync(cancellationToken);
return rawResult.Select(student => new MarkSheetsStudentByIdDto
{
Id = student.RegId,
FullName = student.FullName,
AnnualMarkSheets = student.TermOne
.Select(x => new MarkSheetDto
{
Rank = x.Rank
...
Comments = student.Comments.Where(x => x.StudentId.Equals(student.RegId)).Select(x => x.CommentText)
}).Union(student.TermTwo
.Select(x => new MarkSheetDto
{
Rank = x.Rank
...
Comments = student.Comments.Where(x => x.StudentId.Equals(student.RegId)).Select(x => x.CommentText)
})).OrderBy(c => c.Rank).ToList()
}).ToList();
Includes can be replaced with Select later when you experienced that not needed data is requested from database.
So I have an scenario where I have an Opportunity table that has customer reference and customer Id.
And Customer table that has reference to ProjectManager's table.
Now, I have opportunityId and using that id I need to get Project Manager's information.
Below is my working code,
public async Task<Object> GetProjectManagerIdAgainstCustomer(int? opportunityId)
{
var customerId = await _context.Opportunities
.Where(Opportunity => Opportunity.Id == opportunityId)
.Include(Opp => Opp.Customer)
.Select(Opp => Opp.CustomerId)
.FirstOrDefaultAsync();
var ProjectManager = await _context.Customers
.Where(Customer => Customer.Id == customerId)
.Include(Customer => Customer.ProjectManager)
.Select(x => new {
email = x.ProjectManager.ProjectManagerEmail,
fullname = x.ProjectManager.ProjectManagerName,
})
.FirstOrDefaultAsync();
return (ProjectManager);
}
Now, one of my issues is that these two queries make two trips to the database, which takes me to the main aim of the question, how to optimise it so that all of this is done within one trip and if you guys can find any other issues with this code, that would be great?
Additionaly having any related documentation would help alot.
Secondly, is there any way I could have gotten the ProjectManager object within the attached Customer object, for eg. by using include?
It seems like you are looking for a join operation.
Try the one documented here:
https://learn.microsoft.com/en-us/dotnet/csharp/linq/perform-inner-joins
Using this you can create a join query that will be executed on your database and you will only receive the final output.
I hope you have navigation property Opportunities for Customer.
public async Task<Object> GetProjectManagerIdAgainstCustomer(int? opportunityId)
{
var query =
from c in _context.Customers
from o in c.Opportunities
where o.Id == opportunityId
select new
{
email = c.ProjectManager.ProjectManagerEmail,
fullname = c.ProjectManager.ProjectManagerName,
}
return await query.FirstOrDefaultAsync();
}
Method chain syntax
public async Task<Object> GetProjectManagerIdAgainstCustomer(int? opportunityId)
{
var query = _context.Customers
.SelectMany(c => c.Opportunities.Where(o => o.Id == opportunityId, (c, o) => c)
.Select(c => new
{
email = c.ProjectManager.ProjectManagerEmail,
fullname = c.ProjectManager.ProjectManagerName,
});
return await query.FirstOrDefaultAsync();
}
I have method which return List and I wanna try to populate ValueTuple from another standard list I get error:
Cannot implicitly convert type 'System.Collections.Generic.List<(long PaymentId, long TransactionId)>' to 'System.Collections.Generic.List>'
The code looks like below:
public async Task<List<ValueTuple<(long, long)>>> CreditTransactionAsync(CancellationToken cancellationToken)
{
List<(long PaymentId, long TransactionId)> paymentTransactionList = new List<ValueTuple<long, long>>();
var paymentTransactions = _dbContext.PaymentTransactions
.AsEnumerable()
.Where(x => transactionIdsList.Any(a => a.TransactionId == x.TransactionId))
.Select(x => new
{
PaymentId = x.PaymentId,
TransactionId = x.TransactionId
})
.ToList();
// This line shows error..
paymentTransactionList = paymentTransactions.Select(x => (PaymentId: x.PaymentId, TransactionId: x.TransactionId));
return paymentTransactionList;
}
You can rewrite your Select statement a little bit and map transactions to list of tuple (long, long) directly, without intermediate anonymous type
.Select(x => (x.PaymentId, x.TransactionId))
The full code
var paymentTransactions = _dbContext.PaymentTransactions
.AsEnumerable()
.Where(x => transactionIdsList.Any(a => a.TransactionId == x.TransactionId))
.Select(x => (x.PaymentId, x.TransactionId))
.ToList();
return paymentTransactions;
You also should properly declare the return type of your method, like Task<List<(long paymentId, long transactionId)>>
ValueTuple<(long, long)> is a value tuple that contains a single element that is a value tuple that contains two elements.
You probably meant List<(long, long)>, not List<ValueTuple<(long, long)>>, however personally I'd say "don't use ValueTuple<...> on public APIs", but if you do: at least name them.
Also: AsEnumerable() on a db-context is usually a terrible mistake - especially before a Where:
public async Task<List<(long PaymentId, long TransactionId)>> CreditTransactionAsync(CancellationToken cancellationToken = default)
{
var localList = transactionIdsList.Select(a => a.TransactionId).ToList();
return _dbContext.PaymentTransactions
.Where(x => localList.Contains(x.TransactionId))
.Select(x => new { x.PaymentId, x.TransactionId })
.AsEnumerable()
.Select(x => (x.PaymentId, x.TransactionId))
.ToList();
}
I want to pass dynamic lambda expressions to the function below, but I'm not sure how to define the .Take() or .OrderByDescending() on the expression object.
If I want to call the function below, then I want to be able to do this:
dbprovider.Query = (x => x.ConfigurationReference == "172.16.59.175")
.Take(100)
.OrderByDescending(x.Date)
FindEntities(db, dbprovider.Query)
But I can't (this syntax is invalid). Any ideas?
public static List<T> FindEntities<T>(TrackingDataContext dataContext, System.Linq.Expressions.Expression<Func<T, bool>> find) where T : class
{
try
{
var val = dataContext.GetTable<T>().Where(find).ToList<T>();
return val;
}
catch (Exception ex)
{
throw ex;
}
}
The parameter is of type:
System.Linq.Expressions.Expression<Func<T, bool>> find
That means it can take a predicate (the "where" clause), and only a predicate. Thus the only bit you can pass in there is the filter:
x => x.ConfigurationReference == "172.16.59.175"
To do what you want, you would need to add the rest of the code in FindEntities, so that it becomes:
var val = dataContext.GetTable<T>().Where(find)
.OrderByDescending(x => x.Date).Take(100).ToList<T>();
(note also that the Take should really be after the OrderByDescending)
One way you could do that would be:
public static List<T> FindEntities<T>(TrackingDataContext dataContext,
System.Linq.Expressions.Expression<Func<T, bool>> find,
Func<IQueryable<T>, IQueryable<T>> additonalProcessing = null
) where T : class
{
var query = dataContext.GetTable<T>().Where(find);
if(additonalProcessing != null) query = additonalProcessing(query);
return query.ToList<T>();
}
and call:
var data = FindEntities(db, x => x.ConfigurationReference == "172.16.58.175",
q => q.OrderByDescending(x => x.Date).Take(100));
However, frankly I'm not sure what the point of this would be... the caller could do all of that themselves locally more conveniently, without using FindEntities at all. Just:
var data = db.GetTable<T>()
.Where(x => x.ConfigurationReference == "172.16.58.175")
.OrderByDescending(x => x.Date).Take(100).ToList();
or even:
var data = db.SomeTable
.Where(x => x.ConfigurationReference == "172.16.58.175")
.OrderByDescending(x => x.Date).Take(100).ToList();
or just:
var data = (from row in db.SomeTable
where row.ConfigurationReference == "172.16.58.175"
orderby row.Date descending
select row).Take(100).ToList();