I have that Action:
[EnableQuery]
public IHttpActionResult Get()
{
var ordWeb = orderCtx.ORDER.AsQueryable();
var ordWebDTO =ordWeb.ProjectTo<ORDER>(mapper.ConfigurationProvider);
return Ok(ordWebDTO.toList);
}
This an action inside a controller.
orderWebDTO is a result of a mapping with some fields coming from different tables of a Database.
In that case Odata query coming from Url should be processed AFTER "return" call.
when I use Odata Query in the URL (ex. localhost/Controller?%24top=30) EntityFramework load all data from database WITHOUT filter them (in the example: last 30 records).
It's very expensive: I have more than 35k records, and it load all of them and AFTER get last 30...
How can resolve it?
UPDATE 09.13.18
I have that kind of mapping with one value calculated while mapping work.
var c = new MapperConfiguration(
cfg => cfg.CreateMap<ORDER, ORDER_WEB>()
.ForMember(....)
.ReverseMap()
);
mapper = c.CreateMapper();
In the ORDER_WEB model I have that:
public class ORDER_WEB
{
...
...
public string ValueFromEntityFrameworkModel {get; set;}
public string Set_ORDER
{
get
{
ORDER_TYPE tipo = new ORDER_TYPE();
return tipo.GetData(ValueFromEntityFrameworkModel);
}
set { }
}
without toList() It cannot work...
For this reason OData work on ALL records and AFTER assign the values mapping including Set_ORDER.
The point is that : is it possible to do an OData query (with attributes/parameters) with few records and AFTER assign values mapping?
I hope to be clear...
There are errors in your code sample, but if this accurately reflects what you are doing in your actual code sample, then
ordWebDTO.ToList()
Will go to the database and retrieve all 35k records AND THEN apply the OData filters you were looking to apply. Compare that to:
[EnableQuery]
public IQueryable<ORDER> Get()
{
var ordWeb = orderCtx.ORDER.AsQueryable();
var ordWebDTOs =ordWeb.ProjectTo<ORDER>(mapper.ConfigurationProvider);
return ordWebDTOs;
}
This will return an IQueryable against which the OData filters will be applied so that when the list is materialized, it is an efficient query to the database.
Related
I am trying to use an Async method with ThenInclude filter and Where.
Its throwing this error. Think has it has something to do with this line:
LkDocumentTypeProduct = new Collection<LkDocumentTypeProduct>(dept.LkDocumentTypeProduct.Where(c => c.LkDocumentTypeId == lkDocumentTypeId).ToList())
and also skipping these lines below in my App Service: How would I make this ThenInclude Filter Async and fix the skipping of lines?
//ITS SKIPPING THESE THREE LINES BELOW
IEnumerable<Product> ProductModel = mapper.Map<IEnumerable<Product>>(Products);
var ProductDto = mapper.Map<IEnumerable<Product>, IEnumerable<ProductDto>>(ProductModel);
return ProductDto;
Error:
ArgumentException: Expression of type 'System.Collections.Generic.IAsyncEnumerable`1[Data.Entities.LkDocumentTypeProduct]' cannot be used for parameter of type
'System.Collections.Generic.IEnumerable`1[Data.Entities.LkDocumentTypeProduct]' of method 'System.Collections.Generic.List`1[Data.Entities.LkDocumentTypeProduct] ToList[LkDocumentTypeProduct]
(System.Collections.Generic.IEnumerable`1[Data.Entities.LkDocumentTypeProduct])'
Parameter name: arg0
Repository:
public async Task<IEnumerable<Product>> GetProductsByDocumentType(int lkDocumentTypeId)
{
var ProductsByDocumentType = All
.Include(dept => dept.LkDocumentTypeProduct)
.Select(dept => new Product
{
ProductId = dept.ProductId,
ProductName = dept.ProductName,
ProductCode = dept.ProductCode,
LkDocumentTypeProduct = new Collection<LkDocumentTypeProduct>(dept.LkDocumentTypeProduct.Where(c => c.LkDocumentTypeId == lkDocumentTypeId).ToList())
}).Where(dept=>dept.LkDocumentTypeProduct.Any());
return await ProductsByDocumentType.ToListAsync();
}
AppService:
public async Task<IEnumerable<ProductDto>> GetProductsByDocumentTypeId(int id)
{
var Products = await ProductRepository.GetProductsByDocumentType(id);
//ITS SKIPPING THESE THREE LINES BELOW !
IEnumerable<Product> ProductModel = mapper.Map<IEnumerable<Product>>(Products);
var ProductDto = mapper.Map<IEnumerable<Product>, IEnumerable<ProductDto>>(ProductModel);
return ProductDto;
}
Controller:
[HttpGet("[Action]/{id}")]
[ProducesResponseType(typeof(ProductDto), StatusCodes.Status200OK)]
[ProducesResponseType(StatusCodes.Status404NotFound)]
public async Task<ActionResult<IEnumerable<ProductDto>>> GetByDocumentType(int id)
{
IEnumerable<ProductDto> Product = await ProductAppService.GetProductsByDocumentTypeId(id);
if (Product == null)
{
return NotFound();
}
How to add where clause to ThenInclude
It's not "skipping" lines, it's erroring on this line:
IEnumerable ProductModel = mapper.Map>(Products);
Automapper is a helper, not a magician. :)
From what I'm guessing you had a method that was returning Product entities that you've been told to change over to a DTO. Automapper can help you, in an Async way:
Firstly, ignore the fact that you're working with async collections. Automapper maps objects to one another really well, not collections, it works within collections.
In your case given your repository is returning IEnumerable<Product>, to map this to an IEnumerable<ProductDto> use this in your Service:
public async Task<IEnumerable<ProductDto>> GetProductsByDocumentTypeId(int id)
{
var products = await ProductRepository.GetProductsByDocumentType(id);
var dtos = await products.Select(x => Mapper.Map<ProductDto>(x)).ToListAsync();
return dtos;
}
That should get you working, but isn't ideal. The reason is that the repository is returning back a collection of Entities which means that we'd be materializing all fields in each of those entities returned, whether we need them or not. This also opens the door if Automapper "touches" any related entities that haven't been eager loaded as this will trigger lazy-load calls to the database. You may cater for this with Include, but as code matures these lazy load hits can sneak in or you have eager load costs for fields you no longer need.
Automapper offers a brilliant method to address this, ProjectTo but it requires code to leverage EF's IQueryable implementation rather than returning IEnumerable.
For instance if we change the repository method to:
public IQueryable<Product> GetProductsByDocumentType(int lkDocumentTypeId)
{
var query = _dbContext.Products
.Where(p => p.LkDocumentTypeProduct.Any(c => c.LkDocumentTypeId == lkDocumentTypeId));
return query;
}
Then in the service:
public async Task<IEnumerable<ProductDto>> GetProductsByDocumentTypeId(int id)
{
var dtos = await ProductRepository.GetProductsByDocumentType(id)
.ProjectTo<ProductDto>().ToListAsync();
return dtos;
}
What this does is change the repository to return an IQueryable, basically return a promise of retrieving a known set of entities that our consumer can query against. I would be a bit wary of the "All" property as to what that property returns. I've seen a few cases where methods like that do something like return _context.Products.ToList(); which is a real performance / resource trap.
We feed that Queryable to Automapper's provided extension method ProjectTo which then queries just the columns needed to satisfy the ProductDto. The advantages of this approach are considerable. We don't need to explicitly Include related tables or worry about tripping lazy loaded references, and the built query only pulls back the fields our DTO cares about.
I'm using Web API + AbpOData + EF and need to calculate some properties of the objects returned from the database on the server.
The basic code looks something like this:
[AbpApiAuthorize(AppPermissions.OData_Permission_Consume)]
public class ActivityLogsController : AbpODataEntityController<ActivityLogs>
{
[EnableQuery(PageSize = 50000)]
public override IQueryable<ActivityLogs> Get()
{
var objectContext = new MyObjectContext(); //EF
return objectContext.ActivityLogs.GetAll();
}
}
I'm just returning values from database, all's fine.
However what I need is to Convert two datetime value to local time. Like below
[AbpApiAuthorize(AppPermissions.OData_Permission_Consume)]
public class ActivityLogsController : AbpODataEntityController<ActivityLogs>
{
[EnableQuery(PageSize = 50000)]
public override IQueryable<ActivityLogs> Get()
{
var objectContext = new MyObjectContext(); //EF
return objectContext.ActivityLogs.Select(d => new ActivityLogs()
{
Id = d.ID,
Activity = d.Activity,
StartTime = d.StartTime.Value.AddHours(5),
EndTime = d.EndTime.Value.AddHours(5),
Duration = d.Duration
});
}
}
I getting below error
The entity or complex type 'ActivityLogs' cannot be constructed in a LINQ to Entities query.
how i can impliment this using abp odata framework(.net zero). keeping in mind that we need to return the same IQueryable that's returned from EF call.
The error is caused by impossibility to transform AddHours method to SQL.
You have 2 options:
Create a view in DB where you will keep your additional logic.
Add your business for DateTime properties in your client side.
I have a function that runs a somewhat complex LINQ query, but I've verified that the simplified code below also has the problem. I specifically tell the query to order by RequiredDate, which is a DateTime. This is completely ignored, however--the sorting actually occurs by another property, PONumber. The database is all random test data, so nothing is ordered except the Id column. I'm not sure why the other property is being used instead of the column I'm trying to sort by. I use Kendo UI, so the IEnumerable is converted to a Kendo type in the controller, but the LINQ to Entities query returns the incorrect order. What is causing this problem?
(simplified versions are below)
Class:
public partial class PurchaseOrder : BaseEntity
{
public virtual int PONumber { get; set; }
public virtual DateTime RequiredDate { get; set; }
}
Mapping:
public PurchaseOrderMap()
{
ToTable("PurchaseOrder");
HasKey(c => c.Id);
Property(u => u.PONumber).IsRequired();
Property(u => u.RequiredDate).IsRequired();
}
Service (this fetches the data):
public virtual IEnumerable<PurchaseOrder> GetAllPOs()
{
var query = _poRepository.Table;
query = query.Where(p => p.Shipment == null);
query = query.OrderBy(p => p.RequiredDate);
return query;
}
Function is called in the controller by this code. DataSourceRequest and DataSourceResult are functions in Kendo UI.
public ActionResult POList([DataSourceRequest]DataSourceRequest request)
{
var pos = _poService.GetAllPOs();
DataSourceResult result = pos.ToDataSourceResult(request, o => PreparePOModelForList(o));
return Json(result);
}
The actual query against the DB (courtesy of SQL Profiler) is:
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[PONumber] AS [PONumber],
[Extent1].[RequiredDate] AS [RequiredDate],
[Extent1].[LastUpdateDate] AS [LastUpdateDate],
FROM [dbo].[PurchaseOrder] AS [Extent1]
ORDER BY [Extent1].[PONumber] ASC
OFFSET 0 ROWS FETCH NEXT 10 ROWS ONLY
Based on the OFFSET 0 ROWS FETCH NEXT 10 ROWS ONLY I'm guessing you have some additional logic somewhere which is attempting to apply pagination via the Skip() and Take() methods. My guess is you do some additional sorting there that you are missing. I can't prove that based on the code you have given, but try to figure out what is generating your OFFSET ... FETCH NEXT ... and I suspect you'll find your answer.
I've got an OData WebAPI method as follows:
// GET: odata/Employees
[EnableQuery]
public IQueryable<DTOs.Employee> GetEmployees()
{
return this.AttemptOperation(context =>
{
IQueryable<DTOs.Employee> employees
= context.Employees.Project().To<DTOs.Employee>();
return employees;
});
}
It returns data to to the service if I don't specify a filter.
But as soon as I add $filter=EmployeeID eq '1' to the URL I get an exception.
The exception is from an AWS DynamoDB context library I use for performing LINQ queries against DynamoDB. However, what it is indicating is that the context doesn't have a table for DTOs.Employee.
This of course is obvious, the context has the entities, not the DTOs.
How can I get the IQueryable where clause specified from the client to translate back to the proper entity type?
For example, the client needs to query against DTOs.Employee.EmployeeID and it needs to translate into a where clause against Entities.Employee.EmployeeID.
Easy:
public IQueryable<DTOs.Employee> GetEmployees()
{
return this.AttemptOperation(context =>
{
// I commented here your old code:
// IQueryable<DTOs.Employee> employees = context.Employees.Project().To<DTOs.Employee>();
// This is our new code:
var employees = context.Employees.Project()
return employees.select(m=> new EmployeeDto {
property1 = m.property1,
property2 = m.property2
}
});
}
Important: You cannot use an EmployeeDto constructor. It is forbidden to do so under Linq.
I was reading about query interceptors. I was disappointed because thats more like a filter instead of an interceptor. In other words you can eather include records or not include them. You are not able to modify records for instance.
If I want to create a query interceptor for my entity Users I could then do something like:
[QueryInterceptor("Users")] // apply to table users
public Expression<Func<User, bool>> UsersOnRead()
{
return cust => cust.IsDeleted == false;
}
What if I instead create the operation: NOTE IS VERY IMPORTANT TO HAVE THE OPERATION NAME JUST LIKE THE ENTITY NAME OTHERWISE IT WILL NOT WORK
[WebGet]
public IEnumerable<User> Users()
{
return this.CurrentDataSource.Users.Where(x=>x.IsDeleted==false);
}
Placing this method instead of the query interceptor makes my service behave exactly the same. Plus I have more power! Is taking this approach a better solution?
I played around a little more with this and one of the issues is navigation properties won't be filtered. Let's say you have an entity called SalesPeople that has a link into IEnumberable of Customers
If you do
[QueryInterceptor("Customers")] // only show active customers
public Expression<Func<Customers, bool>> ActiveCustomers()
{
return cust => cust.IsDeleted == false;
}
when you query your OData feed like WCFDataService.svc/SalesPeople?$expand=Customers the results set for Customers will still have the filter applied.
But this
[WebGet]
public IQueryable<Customers> Customers()
{
return this.CurrentDataSource.Customers.Where(x=>x.IsDeleted==false);
}
When running OData query like WCFDataService.svc/Customers you will have the filtered list on active customers, but when running this WCFDataService.svc/SalesPeople?$expand=Customers the results set for the Customers will include deleted customers.