i am making rest api in ASP.NET CORE using MySQL db. In one of my repositories i am getting, filtering, paging and sorting data. Although i get a WARN message: __
queryObj__Categorie.. uses a row limitating operation (Skip/Take) which may lead to unpredictable results. So my question is, what kind of unpredictable results may i get with the following code???
public async Task<QueryResult<User>> GetAll(UserQuery queryObj)
var query = context.users
.Include(users=> users.Category)
.Include(users=> users.Tags)
.Include(users=> users.Localization)
.AsQueryable();
if (queryObj.Categories.Length > 0)
query = query.Where(v => queryObj.Categories.Contains(v.Category.Name));
if (queryObj.Localizations.Length > 0)
query = query.Where(v => queryObj.Localizations.Contains(v.Localization.Id));
int usersCount = query.Count();
//only orders when there is specific value set in queryObj
query = query.ApplyOrdering(queryObj, COLUMNS_MAP);
query = query.ApplyPaging(queryObj);
var users = await query.ToListAsync();
var queryResult = new QueryResult<Users>();
queryResult.items = users;
queryResult.itemsCount = usersCount ;
return queryResult;
}
My paging extension method:
public static IQueryable<T> ApplyPaging<T>(this IQueryable<T> query, IQueryObject queryObj)
{
return query.Skip((queryObj.Page - 1) * queryObj.PageSize).Take(queryObj.PageSize);
}
Related
I am trying to remove duplicate code throughout my project and I am at a standstill trying to figure this out. What I am trying to do is create a base linq query that will be reused to add things like Where, Take...etc in multiple different methods.
public IQueryable<Object> FooLinq(int id)
{
using (var ctx = new dbEntities())
{
var results =
(from account in ctx.account
join memberProducts in ctx.tblMemberProducts on account.Id equals memberProducts.AccountId
orderby account.date descending
select new{account,memberProducts}).ToList();
return results;
}
}
So that would be by base query above and I would have a seperate method that would reuse VioLinq but this time would use a where clause in it.
public List<IncomingViolations> Foo1(int id)
{
//Linq query FooLinq() where Name == "Bob"
}
You'll need to do two things:
Return the query prior to materializing it.
Make sure the context is still in scope when the final query is materialized.
These two requirements will play off each other somewhat, and there are a number of approaches you can take to meet them.
For example, you could make your method take the context as a parameter, forcing the caller to provide it and manage its lifecycle.
public IQueryable<AccountInfo> FooLinq(DbEntities ctx, int id)
{
return
from account in ctx.account
orderby account.date descending
select new AccountInfo()
{
Name = account.Name,
Mid = account.MemberID,
Date = account.Date,
Address = account.Address,
};
}
public List<IncomingViolations> Foo1(int id)
{
using(var ctx = new dbEntities())
{
//Linq query FooLinq() where Name == "Bob"
return FooLinq(ctx).Where(v => v.Name == "Bob").ToList();
}
}
You could alternatively inject the context as a constructor-injected dependency, and use a DI framework to manage the context's lifecycle.
You can do it as Queryable then add conditions to it.
For example:
public List<account> GetAccountsByName(string name, bool usePaging, int offset = 0, int take = 0) {
var query = GetMyQuery();
query = query.Where(x => x.Name == name);
query = query.OrderBy(x => x.Name);
if(usePaging) {
query = query.Take(take).Skip(offset);
}
query = PrepareSelectForAccount(query);
return query.ToList(); .
}
public IQueryable<account> GetMyQuery(){
return ctx.account.AsQueryable();
}
public IQueryable<account> PrepareSelectForAccount(IQueryAble<account> query){
return query.Select(select new AccountInfo()
{
Name = account.Name,
Mid = account.MemberID,
Date = account.Date,
Address = account.Address,
}
);
}
Sure, but don't call .ToList(), and return IQueryable<T> instead of List<T>. LINQ is based on the concept of deferred execution which means the query is not actually performed until the enumerable is iterated over. Until then, all you have done is built an object which knows how to do the query when the time comes.
By returning an IQueryable<T> from a function that sets up the "basic query," you are then free to tack on additional LINQ methods (such as .Where() or .Take()) to produce a modified query. At this point you are still simply setting up the query; it is actually performed only when you iterate over the enumerable, or call something like .ToList() which does that for you.
I have the following code, it stops executing with "StackOverFlow unhandled error".
var Requests= GetList();// return List of Request objects
IQueryable<Order> pos= Enumerable.Empty<Order>().AsQueryable();
if (Requests != null)
{
if (Requests.Count > 0)
{
var GeneralReq = Requests.Select(loc => loc.Id).ToList();
pos = db.Order.Where(loc => loc.Deleted == false && GeneralReq.Any(a => a == loc.Id));
//HERE, stop executing with StackOverFlow Error.
}
}
the problem exactly in here:
pos = db.Order.Where(loc => loc.Deleted == false && GeneralReq.Any(a => a == loc.Id));
Without knowing what GetList() does the likely problem is that it is returning far too many records to safely inject IDs into your .Any() expression. EF will want to turn that .Any() into a query like:
WHERE orders.OrderId IN (22, 25, 45, 46, 52, 66, ...)
This generally isn't efficient and there are limits to the # of IDs you can pass in. It's better to resolve these criteria as a join. I'm not certain if this can result in a stack overflow so I don't believe the code sample you have is complete because without a .ToList() or similar expression, the IQueryable<Order> pos expression would not have materialized yet.
I would look at the relationships that you can resolve between Requests and Orders. If the Request entity as a reference to an Order, then you can alter GetList() to return an IQueryable<Request> then to get the orders:
IQueryable<Request> requests = GetList();
IQueryable<Order> orders = requests.Select(x => x.Order);
From there you can .Select() the details of the orders that you want, and materialize the resulting data.
IQueryable<Request> requests = GetList();
List<OrderViewModel> orderVMs = requests.Select(x => new OrderViewModel
{
OrderId = x.Order.OrderId,
RequestId = x.RequestId,
CustomerName = x.Customer.Name,
OrderNumber = x.Order.OrderNumber,
// ...
}).ToList();
If GetList() can return a significant # or results then use .Take() and .Skip() rather than .ToList() to paginate your results.
We are using ICriteria and now we would like to switch to more readable QueryOver in Nhibernate
Can someone give me a hint how to convert this generic pagination logic for Icriteria to QueryOver?
public static PagedList<T> PagedList<T>(this ICriteria criteria,
ISession session, int pageIndex, int pageSize) where T : class
{
if (pageIndex < 0)
pageIndex = 0;
var countCrit = (ICriteria)criteria.Clone();
countCrit.ClearOrders(); // so we don’t have missing group by exceptions
var results = session.CreateMultiCriteria()
.Add<long>(countCrit.SetProjection(Projections.RowCountInt64()))
.Add<T>(criteria.SetFirstResult(pageIndex * pageSize).SetMaxResults(pageSize))
.List();
var totalCount = ((IList<long>)results[0])[0];
return new PagedList<T>((IList<T>)results[1], totalCount, pageIndex, pageSize);
}
The way I am using it:
var session = ... // get a ISession
// the QueryOver
var query = session.QueryOver<MyEntity>();
// apply all filtering, sorting...
query...
// GET A ROW COUNT query (ICriteria)
var rowCount = CriteriaTransformer.TransformToRowCount(query.UnderlyingCriteria);
// ask for a list, but with a Future, to combine both in one SQL statement
var list = query
.Future<MyEntity>()
.ToList();
// execute the main and count query at once
var count = rowCount
.FutureValue<int>()
.Value;
// list is now in memory, ready to be used
var list = futureList
.ToList();
So, we are using QueryOver, and profiting from underlying criteria and transformer. With a Future, we also execute that all at one command.
I am new to mvc i had problem with following code it is throwing an exception:
LINQ to Entities does not recognize the method 'System.String ToString(System.DateTime)' method, and this method cannot be translated into a store expression.
public ActionResult JobSearchList(PostJobModel model)
{
try
{
if (Session["USER_ID"] != null)
{
var subscriber_Id = RL_Constants.RES_ID;
var company_Id = RL_Constants.COMP_ID;
var resource_Typa = RL_Constants.RES_TYPE;
var jobPostDetails = (from jobPost in reslandentity.JOB_POSTING
where jobPost.COMP_ID == RL_Constants.COMP_ID
select new PostJobModel
{
POST_DT=Convert.ToString(jobPost.POST_DT),
POST_END_DT=Convert.ToString(jobPost.POST_END_DT),
POSITIONS_CNTS=Convert.ToString(jobPost.POSITIONS_CNT),
JOB_TYPE = jobPost.JOB_TYPE,
SKILLS = jobPost.SKILLS,
DURATION = jobPost.DURATION,
CATEGORY=jobPost.CATEGORY,
PREREQUISITES = jobPost.PREREQUISITES,
LOCATION=jobPost.LOCATION,
RATE=jobPost.RATE,
PERKS = jobPost.PERKS,
CONTACT_PERSON=jobPost.CONTACT_NAME,
CONTACT_EMAIL=jobPost.CONTACT_INFO,
CONTACT_PHONE=jobPost.CONTACT_INFO,
POST_TITLE=jobPost.TITLE,
DESCRIPTION=jobPost.DESCR
}).ToList();
model.GetJobPostDetails = jobPostDetails;
}
return View(model);
}
catch (Exception ex)
{
throw ex;
}
}
The error is pretty self explantory, ToString is not supported by the LINQ provider therefore you can't use it in a query.
Pull down the raw data and perform your conversions in memory.
var jobPostDetails = reslandentity.JOB_POSTING
.Where(x => x.COMP_ID == RL_Constants.COMP_ID)
.AsEnumerable() // materialize query, Select will be performed in memory
.Select(x => new {
POST_DT = x.POST_DT.ToString(),
POST_END_DT = x.POST_END_DT.ToString(),
POSITIONS_CNTS = x.POSITIONS_CNT.ToString(),
...
})
.ToList();
AsEnumerable will switch the context from IQueryable<T> to IEnumerable<T> therefore allows for CLI-specific methods to be called as part of the query. The query will still only be materialized after ToList is called as AsEnumerable retains delayed execution.
Take a look at SqlFunctions when trying to convert something inside a LINQ Query.
SqlFunctions.DateName("dd", jobPost.POST_DT) +
SqlFunctions.DateName("mm", jobPost.POST_DT) +
SqlFunctions.DateName("yyyy", jobPost.POST_DT) +
This functions can also be used inside where clauses etc.
I'm using jquery datatables plugin in my mvc application. filtering works perfectly against the fields with string datatype. but i cant use filtering against non string fields without enumerating the results first. here is my code.
public List<MerchAgreementMain> getAgreementbyAccountNo(jQueryDataTableParamModel model)
{
var entity = new CCMSEntities();
var query = (from _first in entity.First
from _second in entity.Second
where _first.No == _second.No
select new MerchAgreementMain
{
AcctNo = _Account.AcctNo, // datatype long
BusnName = _Account.BusnName,
CreatedBy = _Account.CreatedBy,
CreationDate = _Account.CreationDate ?? DateTime.MinValue,
Status = _Reflib.Descp
});
if (model.sSearch != null)
{
var x = query.AsEnumerable().Where(p => Convert.ToString(p.AcctNo).Contains(model.sSearch) || p.BusnName.Contains(model.sSearch) || p.CreatedBy.Contains(model.sSearch)||
p.Status.Contains(model.sSearch));
this.displayRecods = x.Count();
return x.ToList();
}
//var result = query.ToPagedList(Convert.ToInt16(model.sEcho),Convert.ToInt16(model.iDisplayLength));
return query.ToList();
}
this works fine. but the issue is that the query is enumerated first before applying the filters and the database contains thousands of records; that seems to be a bad practice and can take some delay before showing the filtered result. how do i apply the filter in an Iqueryable for long?
Instead of
query.AsEnumerable().Where(....
apply filters directly.
query.Where(
Please see: IQueryable vs. IEnumerable in terms of LINQ to SQL queries
Since you are doing AsEnumerable your query is iterating first and then applying the filter. If you apply the filter without AsEnumerable then the query is IQueryable and it will apply the filters only when iterating over the resultset.