I've got an OData WebAPI method as follows:
// GET: odata/Employees
[EnableQuery]
public IQueryable<DTOs.Employee> GetEmployees()
{
return this.AttemptOperation(context =>
{
IQueryable<DTOs.Employee> employees
= context.Employees.Project().To<DTOs.Employee>();
return employees;
});
}
It returns data to to the service if I don't specify a filter.
But as soon as I add $filter=EmployeeID eq '1' to the URL I get an exception.
The exception is from an AWS DynamoDB context library I use for performing LINQ queries against DynamoDB. However, what it is indicating is that the context doesn't have a table for DTOs.Employee.
This of course is obvious, the context has the entities, not the DTOs.
How can I get the IQueryable where clause specified from the client to translate back to the proper entity type?
For example, the client needs to query against DTOs.Employee.EmployeeID and it needs to translate into a where clause against Entities.Employee.EmployeeID.
Easy:
public IQueryable<DTOs.Employee> GetEmployees()
{
return this.AttemptOperation(context =>
{
// I commented here your old code:
// IQueryable<DTOs.Employee> employees = context.Employees.Project().To<DTOs.Employee>();
// This is our new code:
var employees = context.Employees.Project()
return employees.select(m=> new EmployeeDto {
property1 = m.property1,
property2 = m.property2
}
});
}
Important: You cannot use an EmployeeDto constructor. It is forbidden to do so under Linq.
Related
I am using MongoDB.Driver 2.10.4
I want to get all documents that have an id in a list of ids that I get from my controller
I am using this code :
var pending_processes = mongo_context.collectionName
.AsQueryable()
.Where(x => request.ids.Contains(x.Id))
.Select(x => new ViewModel()
{
process_id = x.Id,
// date = not important just accessing the x object
state = new States()
{
timeline = x.states.timeline,
part = x.states.part,
}
})
.ToList();
It works fine but if I make my function async and do an await and replace ToList() with ToListAsync() I get the following error:
The source IQueryable doesn't implement IAsyncEnumerable<Application.Process.Query.GetPendingProcesses.ViewModel>. Only sources that implement IAsyncEnumerable can be used for Entity Framework asynchronous operations.
Clearly there is something I am not getting here my concern is I don't want my code to run synchronously this would be really bad. usually when dealing with postgresql context I always use the ToListAsync() but here in order to use linq with mongo I had to use AsQueryable() and AsQueryable() as I understand it does not get the data it's a normal query that I need to execute afterwards but when I use with it ToList() everything works but when I use ToListAsync() I get the error.
I just want to know what is behind all of this and is the code above synchronous or asynchronous?
I just changed my query using Find and ForEachAsync() and now all works well. I just did not use AsQueryable() because the mongodb driver as I understand it use these other functions but provides a way to use linq so I used the default methods without linq-
var result = new GetPendingProcessesViewModel();
var filter = Builders<Domain.MongoDocuments.Process>
.Filter.Where(x => own_processes.Contains(x.Id));
await _elba_mongo_context.process
.Find(filter)
.ForEachAsync(x =>
result.pending_processes_own.Add(
new GetPendingProcessesViewModelItem()
{
process_id = x.Id,
state = new States()
{
timeline = x.states.timeline,
part = x.states.part,
supplier = x.states.supplier
}
}
)
);
You can get the documentation and references for your MongoDB driver version from GitHub.
Sure you can keep it asynchronous, but first you have switch out AsQueryable to some other method which returns back and IQueryable.
In a nutshell ToListAsync() works on a IQueryable<T> only, when you turned it in to a IEnumerable via AsEnumerable() you lost the ability to call it. Its explained well here
You have a couple of choices, either implement IDbAsyncEnumerable see here or change the result list you have into an async list with Task.FromResult()
Option 1:
// try this in your controller
public async Task<List<PeopleStatesType>> GetAsyncStatesList()
{
//for e.g.
List<PeopleType> peopleList = new List<PeopleType>()
{
new PeopleType(){ Name = "Frank", Gender = "M" },
new PeopleType(){ Name = "Rose", Gender = "F" } //..
};
var result = from e in peopleList
where e.Gender == "M"
select e;
return await Task.FromResult(result.ToList());
}
Option 2:
Use this class
public class AsyncEnumerableQuery<T> : EnumerableQuery<T>, IDbAsyncEnumerable<T> {
public AsyncEnumerableQuery(IEnumerable<T> enumerable) : base(enumerable) {
}
public AsyncEnumerableQuery(Expression expression) : base(expression) {
}
public IDbAsyncEnumerator<T> GetAsyncEnumerator() {
return new InMemoryDbAsyncEnumerator<T>(((IEnumerable<T>) this).GetEnumerator());
}
IDbAsyncEnumerator IDbAsyncEnumerable.GetAsyncEnumerator() {
return GetAsyncEnumerator();
}
private class InMemoryDbAsyncEnumerator<T> : IDbAsyncEnumerator<T> {
private readonly IEnumerator<T> _enumerator;
public InMemoryDbAsyncEnumerator(IEnumerator<T> enumerator) {
_enumerator = enumerator;
}
public void Dispose() {
}
public Task<bool> MoveNextAsync(CancellationToken cancellationToken) {
return Task.FromResult(_enumerator.MoveNext());
}
public T Current => _enumerator.Current;
object IDbAsyncEnumerator.Current => Current;
}
}
// FindAll: with a condition like .Find(x => x.user == "Jone Doe")
// see [here][3]
var task = collection.Find(p => true).ToListAsync();
Update Option 3: Simple Async Get
public async Task<IEnumerable<MyMongoEntity>> Where(Expression<Func<MyMongoEntity, bool>> expression = null)
{
return await context.GetCollection<MyMongoEntity>(typeof(MyMongoEntity).Name, expression).Result.ToListAsync();
}
Based on your comment, for a simple get documents collection, this helper should work.
From your error, it seems like the mongo_context.collectionName is returning something from Entity Framework?
Only sources that implement IAsyncEnumerable can be used for Entity Framework asynchronous operations.
Make sure you are calling the AsQueryable extension method directly on the Mongo collection. (Your code just shows mongo_context.collectionName.AsQueryable() so I'm not sure you're doing that)
Hooking into the LINQ provider requires getting access to an IQueryable instance. The driver provides an AsQueryable extension method on IMongoCollection.
var collection = db.GetCollection<Person>("people");
var queryable = collection.AsQueryable();
Reference: https://mongodb.github.io/mongo-csharp-driver/2.10/reference/driver/crud/linq/#queryable
The AsQueryable extension above actually returns an IQueryable instance that implements IMongoQueryable and has all the same async extensions that other ORMs (Entity Framework, NHibernate, etc.) have - including ToListAsync.
I am trying to use an Async method with ThenInclude filter and Where.
Its throwing this error. Think has it has something to do with this line:
LkDocumentTypeProduct = new Collection<LkDocumentTypeProduct>(dept.LkDocumentTypeProduct.Where(c => c.LkDocumentTypeId == lkDocumentTypeId).ToList())
and also skipping these lines below in my App Service: How would I make this ThenInclude Filter Async and fix the skipping of lines?
//ITS SKIPPING THESE THREE LINES BELOW
IEnumerable<Product> ProductModel = mapper.Map<IEnumerable<Product>>(Products);
var ProductDto = mapper.Map<IEnumerable<Product>, IEnumerable<ProductDto>>(ProductModel);
return ProductDto;
Error:
ArgumentException: Expression of type 'System.Collections.Generic.IAsyncEnumerable`1[Data.Entities.LkDocumentTypeProduct]' cannot be used for parameter of type
'System.Collections.Generic.IEnumerable`1[Data.Entities.LkDocumentTypeProduct]' of method 'System.Collections.Generic.List`1[Data.Entities.LkDocumentTypeProduct] ToList[LkDocumentTypeProduct]
(System.Collections.Generic.IEnumerable`1[Data.Entities.LkDocumentTypeProduct])'
Parameter name: arg0
Repository:
public async Task<IEnumerable<Product>> GetProductsByDocumentType(int lkDocumentTypeId)
{
var ProductsByDocumentType = All
.Include(dept => dept.LkDocumentTypeProduct)
.Select(dept => new Product
{
ProductId = dept.ProductId,
ProductName = dept.ProductName,
ProductCode = dept.ProductCode,
LkDocumentTypeProduct = new Collection<LkDocumentTypeProduct>(dept.LkDocumentTypeProduct.Where(c => c.LkDocumentTypeId == lkDocumentTypeId).ToList())
}).Where(dept=>dept.LkDocumentTypeProduct.Any());
return await ProductsByDocumentType.ToListAsync();
}
AppService:
public async Task<IEnumerable<ProductDto>> GetProductsByDocumentTypeId(int id)
{
var Products = await ProductRepository.GetProductsByDocumentType(id);
//ITS SKIPPING THESE THREE LINES BELOW !
IEnumerable<Product> ProductModel = mapper.Map<IEnumerable<Product>>(Products);
var ProductDto = mapper.Map<IEnumerable<Product>, IEnumerable<ProductDto>>(ProductModel);
return ProductDto;
}
Controller:
[HttpGet("[Action]/{id}")]
[ProducesResponseType(typeof(ProductDto), StatusCodes.Status200OK)]
[ProducesResponseType(StatusCodes.Status404NotFound)]
public async Task<ActionResult<IEnumerable<ProductDto>>> GetByDocumentType(int id)
{
IEnumerable<ProductDto> Product = await ProductAppService.GetProductsByDocumentTypeId(id);
if (Product == null)
{
return NotFound();
}
How to add where clause to ThenInclude
It's not "skipping" lines, it's erroring on this line:
IEnumerable ProductModel = mapper.Map>(Products);
Automapper is a helper, not a magician. :)
From what I'm guessing you had a method that was returning Product entities that you've been told to change over to a DTO. Automapper can help you, in an Async way:
Firstly, ignore the fact that you're working with async collections. Automapper maps objects to one another really well, not collections, it works within collections.
In your case given your repository is returning IEnumerable<Product>, to map this to an IEnumerable<ProductDto> use this in your Service:
public async Task<IEnumerable<ProductDto>> GetProductsByDocumentTypeId(int id)
{
var products = await ProductRepository.GetProductsByDocumentType(id);
var dtos = await products.Select(x => Mapper.Map<ProductDto>(x)).ToListAsync();
return dtos;
}
That should get you working, but isn't ideal. The reason is that the repository is returning back a collection of Entities which means that we'd be materializing all fields in each of those entities returned, whether we need them or not. This also opens the door if Automapper "touches" any related entities that haven't been eager loaded as this will trigger lazy-load calls to the database. You may cater for this with Include, but as code matures these lazy load hits can sneak in or you have eager load costs for fields you no longer need.
Automapper offers a brilliant method to address this, ProjectTo but it requires code to leverage EF's IQueryable implementation rather than returning IEnumerable.
For instance if we change the repository method to:
public IQueryable<Product> GetProductsByDocumentType(int lkDocumentTypeId)
{
var query = _dbContext.Products
.Where(p => p.LkDocumentTypeProduct.Any(c => c.LkDocumentTypeId == lkDocumentTypeId));
return query;
}
Then in the service:
public async Task<IEnumerable<ProductDto>> GetProductsByDocumentTypeId(int id)
{
var dtos = await ProductRepository.GetProductsByDocumentType(id)
.ProjectTo<ProductDto>().ToListAsync();
return dtos;
}
What this does is change the repository to return an IQueryable, basically return a promise of retrieving a known set of entities that our consumer can query against. I would be a bit wary of the "All" property as to what that property returns. I've seen a few cases where methods like that do something like return _context.Products.ToList(); which is a real performance / resource trap.
We feed that Queryable to Automapper's provided extension method ProjectTo which then queries just the columns needed to satisfy the ProductDto. The advantages of this approach are considerable. We don't need to explicitly Include related tables or worry about tripping lazy loaded references, and the built query only pulls back the fields our DTO cares about.
I have that Action:
[EnableQuery]
public IHttpActionResult Get()
{
var ordWeb = orderCtx.ORDER.AsQueryable();
var ordWebDTO =ordWeb.ProjectTo<ORDER>(mapper.ConfigurationProvider);
return Ok(ordWebDTO.toList);
}
This an action inside a controller.
orderWebDTO is a result of a mapping with some fields coming from different tables of a Database.
In that case Odata query coming from Url should be processed AFTER "return" call.
when I use Odata Query in the URL (ex. localhost/Controller?%24top=30) EntityFramework load all data from database WITHOUT filter them (in the example: last 30 records).
It's very expensive: I have more than 35k records, and it load all of them and AFTER get last 30...
How can resolve it?
UPDATE 09.13.18
I have that kind of mapping with one value calculated while mapping work.
var c = new MapperConfiguration(
cfg => cfg.CreateMap<ORDER, ORDER_WEB>()
.ForMember(....)
.ReverseMap()
);
mapper = c.CreateMapper();
In the ORDER_WEB model I have that:
public class ORDER_WEB
{
...
...
public string ValueFromEntityFrameworkModel {get; set;}
public string Set_ORDER
{
get
{
ORDER_TYPE tipo = new ORDER_TYPE();
return tipo.GetData(ValueFromEntityFrameworkModel);
}
set { }
}
without toList() It cannot work...
For this reason OData work on ALL records and AFTER assign the values mapping including Set_ORDER.
The point is that : is it possible to do an OData query (with attributes/parameters) with few records and AFTER assign values mapping?
I hope to be clear...
There are errors in your code sample, but if this accurately reflects what you are doing in your actual code sample, then
ordWebDTO.ToList()
Will go to the database and retrieve all 35k records AND THEN apply the OData filters you were looking to apply. Compare that to:
[EnableQuery]
public IQueryable<ORDER> Get()
{
var ordWeb = orderCtx.ORDER.AsQueryable();
var ordWebDTOs =ordWeb.ProjectTo<ORDER>(mapper.ConfigurationProvider);
return ordWebDTOs;
}
This will return an IQueryable against which the OData filters will be applied so that when the list is materialized, it is an efficient query to the database.
I'm using Web API + AbpOData + EF and need to calculate some properties of the objects returned from the database on the server.
The basic code looks something like this:
[AbpApiAuthorize(AppPermissions.OData_Permission_Consume)]
public class ActivityLogsController : AbpODataEntityController<ActivityLogs>
{
[EnableQuery(PageSize = 50000)]
public override IQueryable<ActivityLogs> Get()
{
var objectContext = new MyObjectContext(); //EF
return objectContext.ActivityLogs.GetAll();
}
}
I'm just returning values from database, all's fine.
However what I need is to Convert two datetime value to local time. Like below
[AbpApiAuthorize(AppPermissions.OData_Permission_Consume)]
public class ActivityLogsController : AbpODataEntityController<ActivityLogs>
{
[EnableQuery(PageSize = 50000)]
public override IQueryable<ActivityLogs> Get()
{
var objectContext = new MyObjectContext(); //EF
return objectContext.ActivityLogs.Select(d => new ActivityLogs()
{
Id = d.ID,
Activity = d.Activity,
StartTime = d.StartTime.Value.AddHours(5),
EndTime = d.EndTime.Value.AddHours(5),
Duration = d.Duration
});
}
}
I getting below error
The entity or complex type 'ActivityLogs' cannot be constructed in a LINQ to Entities query.
how i can impliment this using abp odata framework(.net zero). keeping in mind that we need to return the same IQueryable that's returned from EF call.
The error is caused by impossibility to transform AddHours method to SQL.
You have 2 options:
Create a view in DB where you will keep your additional logic.
Add your business for DateTime properties in your client side.
I have a problem in architecting my application.
I have the following structure with only important aspects shown.
namespace Domain
{
public class Invoice
{
//properties
}
public class InvoiceRepository
{
public Linq2SqlContext context = new Linq2SqlContext();
public IQueryable<Invoice> GetInvoices()
{
var query = from inv in _dbctx.Invoices orderby inv.invInvoiceDate descending select GetInvoice(inv) ;
return query;
}
}
public class InvoiceService()
{
public InvoiceRepository _repository = new InvoiceRepositroy();
public IQueryable<Invoice> GetInvoices()
{
return _repository.GetInvoices();
}
}
}
namespace MyApp
{
public class UI
{
public InvoiceService _service = new InvoiceService();
public void FilterInvoices()
{
var query =
(
from i in _service.GetInvoices()
from s in _service.GetStatuses()
where i.ProjectID == _projectid &&
s.ID == i.Status
select new
{
InvoiceID = i.ID,
DocumentTotal = i.TotalDue.ToString(),
Created = i.Created,
WeekEnding = i.WeekEnding,
Status = s.staStatus
}
).Skip(_pageIndex * _pageSize).Take(_pageSize);
}
}
{
So I want to return IQueryable from my service so I can
filter from client code. But the problem I'm coming up with
is the FilterInvoices method errors with "No supported
translation to sql" because of the GetInvoice method
which is iused to return an Invoice entity (this is
a layer on top op the LInq2 sql layer) and not an Linq2sql Invoice entity.
So how do I return a IQueryable from my service with this structure??
Also how do I sort and return a IQureyable in repository GetInvoices.
Hope this makes sense.
Malcolm
linq2sql thinks GetInvoice (within GetInvoices) is a stored procedure. One way around it
var query = from inv in _dbctx.Invoices orderby inv.invInvoiceDate descending select inv ;
though that would pass back the objects generated by your datacontext. If you wanted to populated custom objects you could iterated over the collection creating your custom Invoice objects and populating them.
foreach(var inv in query) { somelist.Add(new MyCustomInvoince() { id = inv.id ... }
EDIT: The above will return a list. Use the following to return IQueryable
return from item in query
select GetInvoice(item);
The difference is at this stage your are using Linq2Objects, and that provider will know how to call GetInvoice
You cannot query with LTS (Linq to SQL) something built 'on top' of the LTS layer.
The reason is that the LTS entities layer is a mapping of the content of you database, and the query you perform is 'just' translated into SQL.
I personnaly use another approach to keep an independance between my layers...
I create interfaces that match my LTS entites, and I use the Cast<>() method to have my repository return the interface instead of the concrete implementation.
It works perfectly.
You need to extend the base entity (no pb as it is a partial class) :
partial class Employee : IEmployee
And you need this property in your repository :
public IQueryable<IEmployee> Query
{
get
{
return this._context.Employees.Cast<IEmployee>();
}
}
Based on this, you could code a generic repository, but it's another story (more complicated)
This is just off the top of my head, but you could try:
from i in _service.GetInvoices().AsEnumerable()