I'm tring to make a database query inside a LINQ statement asynchronous, but I'm running into an error. The code below runs fine with out async/await
var newEntities = _repositoryMapping.Mapper.Map<List<Entry>>(entries);
newEntities = newEntities.Where(async e => await !_context.Entries.AnyAsync(c => c.Id == e.Id)).ToList();
Severity Code Description Project File Line Suppression State
Error CS4010 Cannot convert async lambda expression to delegate type
'Func<Entry, bool>'. An async lambda expression may return
void, Task or Task, none of which are convertible to
'Func<Entry,
bool>'
Other than breaking this up into a foreach loop, how can I make this work with async/await?
If you care about performance, code should be smarter. You just need to send one query and check what is already present in database.
Prepared extension which can do that in generic way:
newEntities = (await newEntities.FilterExistentAsync(_context.Entries, e => e.Id)).ToList();
Implementation is not so complex
public static class QueryableExtensions
{
public static async Task<IEnumerable<T>> FilterExistentAsync<T, TProp>(this ICollection<T> items,
IQueryable<T> dbQuery, Expression<Func<T, TProp>> prop, CancellationToken cancellationToken = default)
{
var propGetter = prop.Compile();
var ids = items.Select(propGetter).ToList();
var parameter = prop.Parameters[0];
var predicate = Expression.Call(typeof(Enumerable), "Contains", new[] { typeof(TProp) }, Expression.Constant(ids), prop.Body);
var predicateLambda = Expression.Lambda(predicate, parameter);
var filtered = Expression.Call(typeof(Queryable), "Where", new[] {typeof(T)}, dbQuery.Expression,
predicateLambda);
var selectExpr = Expression.Call(typeof(Queryable), "Select", new[] {typeof(T), typeof(TProp)}, filtered, prop);
var selectQuery = dbQuery.Provider.CreateQuery<TProp>(selectExpr);
var existingIds = await selectQuery.ToListAsync(cancellationToken);
return items.Where(i => !existingIds.Contains(propGetter(i)));
}
}
For the Exception, you can add a extension for IEnumerable to support async
public static class MyExtensions
{
public static async Task<IEnumerable<T>> Where<T>(this IEnumerable<T> source,
Func<T, Task<bool>> func)
{
var tasks = new List<Task<bool>>();
foreach (var element in source)
{
tasks.Add(func(element));
}
var results = await Task.WhenAll<bool>(tasks.ToArray());
var trueIndex = results.Select((x, index) => new { x, index })
.Where(x => x.x)
.Select(x => x.index).ToList();
var filterSource = source.Where((x, index) => trueIndex.Contains(index));
return filterSource;
}
}
Then you can use someting like below
var result = await users.Where(async x => await TestAsync(x));
Full code here https://dotnetfiddle.net/lE2swz
Related
I am trying to modify the a generic IQueryable extension method to perform a GroupBy operation. The method I have performs a generic Where on a dynamically defined column:
public static IQueryable<TEntity> WhereById<TEntity, TKey>(
this IQueryable<TEntity> query, TKey value, string colName)
where TEntity : class
{
var param = Expression.Parameter(typeof(TEntity), "e");
var propAccess = Expression.PropertyOrField(param, colName);
var valExpr = Expression.Constant(value);
BinaryExpression predicate;
predicate = Expression.Equal(propAccess, valExpr);
var predicateLambda = Expression.Lambda<Func<TEntity, bool>>(predicate, param);
return query.Where(predicateLambda);
}
This works perfectly as in:
IQueryable<TEntity> entities = _crudApiDbContext.Set<TEntity>()
.WhereById<TEntity, int>(id, selectField);
Now I need a generic GroupBy(). I am trying the following:
public static IQueryable<TEntity> GroupBy<TEntity>(this IQueryable<TEntity> query, string colName)
where TEntity : class
{
var param = Expression.Parameter(typeof(TEntity), "e");
var propAccess = Expression.PropertyOrField(param, colName);
BinaryExpression predicate;
predicate = Expression.XXX(propAccess); <=what should XXX be?
var predicateLambda = Expression.Lambda<Func<TEntity, int>>(propAccess);
return (IQueryable<TEntity>)query.GroupBy(predicateLambda);
}
and I guess my question is what should XXX be? Or maybe because GroupBy() is an extension method the approach needs to be different?
Update
There is a lot of interest as to why I need this - I am building a generic Blazor form for faceted browsing. So I have a generic method for filtered and sorted search, but I also need to know for certain columns with options, how many options remain after applying the search conditions. To that end I will perform a GroupBy on each of those columns with the IQueryable which has the search conditions applied to it. Example of faceted search:
This is solution how to do grouping by dynamic field. Since we don't know type of grouping key, I've decided to make it as object.
public static class QueryableExtensions
{
static Expression MakePropPath(Expression objExpression, string path)
{
return path.Split('.').Aggregate(objExpression, Expression.PropertyOrField);
}
public static IQueryable<IGrouping<object, TEntity>> GroupBy<TEntity>(this IQueryable<TEntity> query, string colName)
where TEntity : class
{
var param = Expression.Parameter(typeof(TEntity), "e");
var propAccess = MakePropPath(param, colName);
var keyLambda = Expression.Lambda(Expression.Convert(propAccess, typeof(object)), param);
var groupCall = Expression.Call(typeof(Queryable), nameof(Queryable.GroupBy),
new[] { typeof(TEntity), typeof(object) }, query.Expression,
keyLambda);
return query.Provider.CreateQuery<IGrouping<object, TEntity>>(groupCall);
}
}
Usage is simple in your case:
var result = query.GroupBy("Genre")
.Select(g => new
{
g.Key,
Count = g.Count()
})
.ToList();
I am writing NUnit test cases. I am writing test cases to get some values. Below is my setup.
[Test]
public async Task GetGeographyList_StateUnderTest_ExpectedBehavior()
{
// Arrange
var geographyBusinessLogic = this.CreateGeographyBusinessLogic();
Geography geographyObj = new Geography() { Id = 101, Country = "India", Region = "APAC", SubRegion = "Asia South" };
IEnumerable<Geography> geographyObjList = new List<Geography>() { geographyObj };
//this.geographyRepository.setupGetAsync<Geography>(geographyObjList);
this.geographyRepository.Setup(
x => x.GetAsync()).ReturnsAsync(geographyObjList);
// Act
var result = await geographyBusinessLogic.GetGeographyList();
// Assert
Assert.IsNotNull(result);
}
In the above code, x => x.GetAsync() throws an error:
An expression tree may not contain a call or invocation that uses optional arguments
Below is my implementation of geographyBusinessLogic.GetGeographyList():
public async Task<IEnumerable<GeographyEntity>> GetGeographyList()
{
var listOfGeographies = await this.GeographyRepository.GetAsync().ConfigureAwait(false);
IEnumerable<GeographyEntity> result = from o in listOfGeographies
select new GeographyEntity
{
Id = o.Id,
Country = o.Country,
Region = o.Region,
SubRegion = o.SubRegion
};
return result;
}
Below is the implementation of the GetAsync method:
public async Task<IEnumerable<T>> GetAsync(Expression<Func<T, bool>> filter = null, Func<IQueryable<T>, IOrderedQueryable<T>> orderBy = null, params Expression<Func<T, object>>[] includes)
{
IQueryable<T> query = this.dbSet;
foreach (Expression<Func<T, object>> include in includes)
{
query = query.Include(include);
}
if (filter != null)
{
query = query.Where(filter);
}
if (orderBy != null)
{
query = orderBy(query);
}
return await query.ToListAsync().ConfigureAwait(false);
}
Can someone help me understand this error and can someone tell me what I am missing here? Any help would be greatly appreciated.
You need to fill in all the optional arguments when you setup the mock. For example:
this.geographyRepository.Setup(x => x.GetAsync(
It.IsAny<Expression<Func<Geography, bool>>>(),
It.IsAny<Func<IQueryable<Geography>, IOrderedQueryable<Geography>>>(),
It.IsAny<Expression<Func<Geography, object>>[]>())).ReturnsAsync(geographyObjList);
I have a method which searchs Items in the Database. Because I use it multiple times I made it generic:
public async Task Search(Expression<Func<T, bool>> criteria, Func<T, object> order)
{
return Task.Run(async () => Searchlist.AddRange((await Service.SearchAsync(criteria, false)).AsEnumerable().OrderBy(order)));
}
Because it is generic, I implement the parameter order, that they are ordered correctly. I call the method like this:
await Search(GetCriteria(), p => p.Description);
But I have some objects which are ordered by multiple (between 2 and 4) properties. So they are ordered like this:
SearchAsync(criteria, false)).AsEnumerable().OrderBy(x => x.Date).ThenBy(y => y.Nr).ThenBy(z => z.Type))
Can I create a parameter where I can put the Methods to call. Like .OrderBy(x => x.Date).ThenBy(y => y.Nr).ThenBy(z => z.Type) or only .OrderBy(x => x.Date).
Thanks
If you are able to break the current interface of your function, go with:
public async Task Search(Expression<Func<T, bool>> criteria, Func<T, object>[] order)
{
var elems = await Service.SearchAsync(criteria, false);
var sorted = elems.AsEnumerable().OrderBy(order.First());
foreach(var subOrder in order.Skip(1)){
sorted = sorted.ThenBy(subOrder);
}
Searchlist.AddRange(sorted)
}
await Search(GetCriteria(), new []{p => p.Date, p => p.Description});
If you dont want to break the current interface, go with:
public async Task Search(
Expression<Func<T, bool>> criteria,
Func<T, object> mainOrder,
params Func<T, object>[] subOrders)
{
var elems = await Service.SearchAsync(criteria, false);
var sorted = elems.AsEnumerable().OrderBy(mainOrder);
foreach(var subOrder in subOrders){
sorted = sorted.ThenBy(subOrder);
}
Searchlist.AddRange(sorted)
}
await Search(GetCriteria(), p => p.Date, p => p.Description);
I have the below method (it's an extension method but not relevant to this question) and I would like to use GroupBy on the results of the method.
class MyClass
{
public async Task<string> GetRank()
{
return "X";
}
public async static Task Test()
{
List<MyClass> items = new List<MyClass>() { new MyClass() };
var grouped = items.GroupBy(async _ => (await _.GetRank()));
}
}
The type of grouped is IGrouping<Task<string>, MyClass>, however I need to group by the actual awaited result of the async method (string). Despite using await and making the lambda async, I still get IGrouping<Task<string>, ..> instead of IGrouping<string, ...>
How to use GroupBy and group by a result of async Task<string> method and get a grouping by string?
You probably are looking to await all your tasks first, then group
// projection to task
var tasks = items.Select(y => AsyncMethod(y);
// Await them all
var results = await Task.WhenAll(tasks)
// group stuff
var groups = results.GroupBy(x => ...);
Full Demo here
Note : You didnt really have any testable code so i just plumbed up something similar
Update
the reason why you example isn't working
items.GroupBy(async _ => (await _.GetRank()))
is because and async lambda is really just a method that returns a task, this is why you are getting IGrouping<Task<string>, MyClass>
You need to wait for all you tasks to finish first before you can think about doing anything with the results from the task
To further explain what is happening take a look at this SharpLab example
Your async lambda basically resolves to this
new Func<int, Task<string>>(<>c__DisplayClass1_.<M>b__0)
Here is an asynchronous version of GroupBy. It expects a task as the result of keySelector, and returns a task that can be awaited:
public static async Task<IEnumerable<IGrouping<TKey, TSource>>>
GroupByAsync<TSource, TKey>(this IEnumerable<TSource> source,
Func<TSource, Task<TKey>> keySelector)
{
var tasks = source.Select(async item => (Key: await keySelector(item), Item: item));
var entries = await Task.WhenAll(tasks);
return entries.GroupBy(entry => entry.Key, entry => entry.Item);
}
It can be used like this:
class MyClass
{
public async Task<string> GetRank()
{
await Task.Delay(100);
return "X";
}
public async static Task Test()
{
var items = new List<MyClass>() { new MyClass(), new MyClass() };
var grouped = items.GroupByAsync(async _ => (await _.GetRank()));
foreach (var grouping in await grouped)
{
Console.WriteLine($"Key: {grouping.Key}, Count: {grouping.Count()}");
}
}
}
Output:
Key: X, Count: 2
I have a function like this:
public async Task<SomeViewModel> SampleFunction()
{
var data = service.GetData();
var myList = new List<SomeViewModel>();
myList.AddRange(data.select(x => new SomeViewModel
{
Id = x.Id,
DateCreated = x.DateCreated,
Data = await service.GetSomeDataById(x.Id)
}
return myList;
}
My await isn't working as it can only be used in a method or lambda marked with the async modifier. Where do I place the async with this function?
You can only use await inside an async method/delegate. In this case you must mark that lambda expression as async.
But wait, there's more...
Select is from the pre-async era and so it doesn't handle async lambdas (in your case it would return IEnumerable<Task<SomeViewModel>> instead of IEnumerable<SomeViewModel> which is what you actually need).
You can however add that functionality yourself (preferably as an extension method), but you need to consider whether you wish to await each item before moving on to the next (sequentialy) or await all items together at the end (concurrently).
Sequential async
static async Task<TResult[]> SelectAsync<TItem, TResult>(this IEnumerable<TItem> enumerable, Func<TItem, Task<TResult>> selector)
{
var results = new List<TResult>();
foreach (var item in enumerable)
{
results.Add(await selector(item));
}
return results.ToArray();
}
Concurrent async
static Task<TResult[]> SelectAsync<TItem, TResult>(this IEnumerable<TItem> enumerable, Func<TItem, Task<TResult>> selector)
{
return Task.WhenAll(enumerable.Select(selector));
}
Usage
public Task<SomeViewModel[]> SampleFunction()
{
return service.GetData().SelectAsync(async x => new SomeViewModel
{
Id = x.Id,
DateCreated = x.DateCreated,
Data = await service.GetSomeDataById(x.Id)
}
}
You're using await inside of a lambda, and that lambda is going to be transformed into its own separate named method by the compiler. To use await it must itself be async, and not just be defined in an async method. When you make the lambda async you now have a sequence of tasks that you want to translate into a sequence of their results, asynchronously. Task.WhenAll does exactly this, so we can pass our new query to WhenAll to get a task representing our results, which is exactly what this method wants to return:
public Task<SomeViewModel[]> SampleFunction()
{
return Task.WhenAll(service.GetData().Select(
async x => new SomeViewModel
{
Id = x.Id,
DateCreated = x.DateCreated,
Data = await service.GetSomeDataById(x.Id)
}));
}
Though maybe too heavyweight for your use case, using TPL Dataflow will give you finer control over your async processing.
public async Task<List<SomeViewModel>> SampleFunction()
{
var data = service.GetData();
var transformBlock = new TransformBlock<X, SomeViewModel>(
async x => new SomeViewModel
{
Id = x.Id,
DateCreated = x.DateCreated,
Data = await service.GetSomeDataById(x.Id)
},
new ExecutionDataflowBlockOptions
{
// Let 8 "service.GetSomeDataById" calls run at once.
MaxDegreeOfParallelism = 8
});
var result = new List<SomeViewModel>();
var actionBlock = new ActionBlock<SomeViewModel>(
vm => result.Add(vm));
transformBlock.LinkTo(actionBlock,
new DataflowLinkOptions { PropagateCompletion = true });
foreach (var x in data)
{
transformBlock.Post(x);
}
transformBlock.Complete();
await actionBlock.Completion;
return result;
}
This could be substantially less long-winded if service.GetData() returned an IObservable<X> and this method returned an IObservable<SomeViewModel>.