Develop Reference

Using where clause with continuation in oData client using C#

I'm using the Microsoft oData client to retrieve user information from a third-party API. I have the following code.
var query = Context.Users.Where(x => x.username.EndsWith("10"));
var result = query.ToList().Select(x => x.username);
However, this would only return 100 records. I can use the following code to retrieve all the records without the condition;
DataServiceCollection<User> users = new DataServiceCollection<User>(
Context.Users
);
while (users.Continuation != null)
{
//use the token to query for more users
//and load the results back into the collection
users.Load(
Context.Execute<User>(users.Continuation)
);
//print the current count of users retrieved
Console.WriteLine(users.Count);
}
How can I combine the two? i.e. to retrieve all the records with condition (x.username.EndsWith("10")).

I would say that it is a good practice to get only 100 rows at a time. But you can use the oDatas $top to get more than 100 rows. Microsoft oData client uses the the Take function for that: https://learn.microsoft.com/en-us/odata/client/query-options
I should be able to combine the two simply by:
var result= Context.Users.Where(x => x.username.EndsWith("10")).Take(1000).Select(x => x.username);

I'm using queries like this in oData Client version 7.6.2, but I think it will work in newer versions too.
var users = new List<User>();
DataServiceQueryContinuation<User> nextLink = null;
var query = Context.Users.Where(x => x.username.EndsWith("10")) as DataServiceQuery<User>;
var response = await query.ExecuteAsync() as QueryOperationResponse<User>;
do
{
if (nextLink != null)
{
response = await Context.ExecuteAsync<User>(nextLink) as QueryOperationResponse<User>;
}
users.AddRange(response);
}
while ((nextLink = response.GetContinuation()) != null);

Entity framework, Row Limiting Operation Without OrderBy Warning

i am making rest api in ASP.NET CORE using MySQL db. In one of my repositories i am getting, filtering, paging and sorting data. Although i get a WARN message: __
queryObj__Categorie.. uses a row limitating operation (Skip/Take) which may lead to unpredictable results. So my question is, what kind of unpredictable results may i get with the following code???
public async Task<QueryResult<User>> GetAll(UserQuery queryObj)
var query = context.users
.Include(users=> users.Category)
.Include(users=> users.Tags)
.Include(users=> users.Localization)
.AsQueryable();
if (queryObj.Categories.Length > 0)
query = query.Where(v => queryObj.Categories.Contains(v.Category.Name));
if (queryObj.Localizations.Length > 0)
query = query.Where(v => queryObj.Localizations.Contains(v.Localization.Id));
int usersCount = query.Count();
//only orders when there is specific value set in queryObj
query = query.ApplyOrdering(queryObj, COLUMNS_MAP);
query = query.ApplyPaging(queryObj);
var users = await query.ToListAsync();
var queryResult = new QueryResult<Users>();
queryResult.items = users;
queryResult.itemsCount = usersCount ;
return queryResult;
}
My paging extension method:
public static IQueryable<T> ApplyPaging<T>(this IQueryable<T> query, IQueryObject queryObj)
{
return query.Skip((queryObj.Page - 1) * queryObj.PageSize).Take(queryObj.PageSize);
}

Iterate through dbset to get next n rows

I'm trying to find out what's a good way to continually iterate through a dbset over various function calls, looping once at the end.
Essentially I've got a bunch of Ads in the database, and I want to getNext(count) on the dbset.Ads
Here's an example
ID Text Other Columns...
1 Ad1 ...
2 Ad2 ...
3 Ad3 ...
4 Ad4 ...
5 Ad5 ...
6 Ad6 ...
Let's say that in my View, I determine I need 2 ads to display for User 1. I want to return Ads 1-2. User 2 then requests 3 ads. I want it to return 3-5. User 3 needs 2 ads, and the function should return ads 6 and 1, looping back to the beginning.
Here's my code that I've been working with (it's in class AdsManager):
Ad NextAd = db.Ads.First();
public IEnumerable<Ad> getAds(count)
{
var output = new List<Ad>();
IEnumerable<Ad> Ads = db.Ads.OrderBy(x=>x.Id).SkipWhile(x=>x.Id != NextAd.Id);
output.AddRange(Ads);
//If we're at the end, handle that case
if(output.Count != count)
{
NextAd = db.Ads.First();
output.AddRange(getAds(count - output.Count));
}
NextAd = output[count-1];
return output;
}
The problem is that the function call IEnumerable<Ad> Ads = db.Ads.OrderBy(x=>x.Id).SkipWhile(x=>x.Id != NextAd.Id); throws an error on AddRange(Ads):
LINQ to Entities does not recognize the method 'System.Linq.IQueryable'1[Domain.Entities.Ad] SkipWhile[Ad](System.Linq.IQueryable'1[Domain.Entities.Ad], System.Linq.Expressions.Expression'1[System.Func`2[Domain.Entities.Ad,System.Boolean]])' method, and this method cannot be translated into a store expression.
I originally had loaded the entire dbset into a Queue, and did enqueue/dequeue, but that would not updat when a change was made to the database. I got the idea for this algorithm based on Get the next and previous sql row by Id and Name, EF?
What call should I be making to the database to get what I want?
UPDATE: Here's the working Code:
public IEnumerable<Ad> getAds(int count)
{
List<Ad> output = new List<Ad>();
output.AddRange(db.Ads.OrderBy(x => x.Id).Where(x => x.Id >= NextAd.Id).Take(count + 1));
if(output.Count != count+1)
{
NextAd = db.Ads.First();
output.AddRange(db.Ads.OrderBy(x => x.Id).Where(x => x.Id >= NextAd.Id).Take(count - output.Count+1));
}
NextAd = output[count];
output.RemoveAt(count);
return output;
}

SkipWhile is not supported in the EF; it can't be translated into SQL code. EF basically works on sets and not sequences (I hope that sentence makes sense).
A workaround is to simply use Where, e.g.:
IEnumerable<Ad> Ads = db.Ads.OrderBy(x=>x.Id).Where(x=>x.Id >= NextAd.Id);

Maybe you can simplify this into something like this:
Ad NextAd = db.Ads.First();
public IQueryable<Ad> getAds(count)
{
var firstTake = db.Ads
.OrderBy(x => x.Id)
.Where(x => x.Id >= NextAd.Id);
var secondTake = db.Ads
.OrderBy(x => x.Id)
.Take(count - result.Count());
var result = firstTake.Concat(secondTake);
NextAd = result.LastOrDefault()
return result;
}
Sorry, haven't tested this, but it should work.

Optimize query using HQL / Antlr.Runtime.NoViableAltException

I have code:
if (request.OrderBy != "PricesCount")
query = query.ApplyOrder(request);
else
{
if (request.OrderDirection == "ASC")
{
query = query.ToList().OrderBy(p => p.Prices.Count).AsQueryable(); //must be optimize!
}
else
query = query.ToList().OrderByDescending(p => p.Prices.Count).AsQueryable(); //must be optimize!
}
query = query.Page(pageNumber, pageSize);
var result = query.ToList();
query has type NHibernate.Linq.NhQueryable<Book>
I must remove the ToList() which causes loading all Books from DB.
If I try to use some code:
query = query.OrderBy(p => p.Prices.Count);
...
var result = query.ToList();//I have Antlr.Runtime.NoViableAltException
Exception of type 'Antlr.Runtime.NoViableAltException' was thrown. [.Take[Book](.Skip[Book](.OrderBy[Book,System.Int32](NHibernate.Linq.NhQueryable`1[Book], Quote((p, ) => (p.Prices.Count)), ), p1, ), p2, )]

result = query.Where(p=>p.Price > 5).ToList(); //put whatever filter you want
You don't need to do .ToList().AsQueryable().ToList() like you are in your first segment.
If you can't filter results, then you should implement some sort of paging:
result = query.Skip(x).Take(y).ToList(); //You will need to send in X and Y based on what page you are on and how many items per page you use
result = query.Where(p=>p.Price>5).OrderBy(p=>p.Prices.Count).ToList()
Like I said in comments, if you don't provide a where clause, or if you don't do .Take(y) then this will return all items in the database. You will pass in X and Y yourself if you do .Skip(x).Take(y), you need to determine what is appropriate paging for your application.

Index was out of range exception in query

each Route contains Locations in specific order.
For example:
NY -> LA is different from LA -> NY.
I would like to write a method that gets locations array and return true or false whether route with the same locations and order exists.
I need to do it using linq to entities and entity framework (Route and Location are entities).
Here is what I wrote:
public bool IsRouteExists(IList<LocationInRoute> locationsInRoute)
{
Route route = null;
if (locationsInRoute.Count > 0)
{
var query = GetRoutesQuery().
Where(x => x.Locations.Count() == locationsInRoute.Count);
for (int i = 0; i < locationsInRoute.Count; i++)
{
long locationId = locationsInRoute[i].LocationId;
query = query.Where(x =>
x.Locations.ElementAt(i).LocationId == locationId); //THROWS EXCEPTION
}
route = query.SingleOrDefault();
}
return route!=null;
}
I get the following exception in the marked line:
Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
What is the reason for this exception?
EDIT
The exception accurs when executing route = query.SingleOrDefault(); and the exception complains about Where(x => x.Locations.ElementAt(i).LocationId == locationId);.

I believe this query is completely wrong. First of all it is not linq-to-entities query and it will never be because linq to entities is not able to work with indexes. I think comparing ordered sequences will have to be executed in memory = linq-to-objects.
Another problem is this:
for (int i = 0; i < locationsInRoute.Count; i++)
{
long locationId = locationsInRoute[i].LocationId;
query = query.Where(x => x.Locations.ElementAt(i).LocationId == locationId);
}
route = query.SingleOrDefault();
I think this is known gotcha when using Linq, query built in loop and deferred execution - I believe this always compares locationId with the last element.
In my opinion the most efficient way to do this is stored procedure with table valued parameter passing your expected sequence and using SQL cursor to compare sequences in the stored procedure.

Looks like perhaps your x.Locations.Count() might be less than your locationsInRoute.Count. are you sure it's not? I'm saying that b/c you're calling x.Locations.ElementAt(i), which will throw if i > Count().
As a sidenote, a better solution to what you're doing is to override equality or implement an IComparer on your class that you want unique and then you can use things like Any() and Contains() for your test.

If you getting an index out of range exception it must mean that number of elements in the locationsRoute collection exceeds the number of elements in IQueryable. If you attempting to test that each location in the provided list is contained in route you should be able to do something like:
var locationIds = locationsInRoute.Select(l => l.LocationId);
query = query.Where(r => r.Locations.All(l => locationIds.Contains(l.LocationId)))

I'm guessing it has to do with your use of ElementAt, which can't be translated to SQL (see the Operators with No Translation section), to operate on your IQueryable. This materializes the IQueryable's result set on the first iteration and so subsequent iterations Route items will be unable to access their related Locations sets. This should probably only happen on the second iteration, but the myriad implications of the deferred execution nature of LINQ is not entirely clear to me in ever case ;-) HTH
You could put a breakpoint at SingleOrDefault and inspect the SQL statement it is executing there, to see why there is no record returned for the SingleOrDefault to find. Though the SQL may be pretty ugly depending on how many routes you have.

Thanks to #Ladislav Mrnka's advice, here is the solution:
public class LocationSequenceEqual : IEqualityComparer<Location>
{
public bool Equals(Location x, Location y)
{
return x.Id == y.Id;
}
public int GetHashCode(Location obj)
{
return obj.Id.GetHashCode();
}
}
public bool IsRouteExists(IList<LocationInRoute> locationsInRoute)
{
Route route = null;
if (locationsInRoute.Count > 0)
{
var query = GetRoutesQuery().
Where(x => x.Locations.Count() == locationsInRoute.Count);
query = query.Where(x => x.Locations.OrderBy(l => l.Order).
Select(l => l.Location).SequenceEqual(locations, new LocationSequenceEqual()));
route = query.FirstOrDefault();
}
return route!=null;
}

If Location has Order as you indicate above, this can be done entirely in (Linq to) SQL:
public bool IsRouteExists(IList<LocationInRoute> locationsInRoute)
{
Route route = null;
if (locationsInRoute.Count == 0)
return;
var possibleRoutes = GetRoutesQuery().
Where(x => x.Locations.Count() == locationsInRoute.Count);
var db = GetDataContext(); //get a ref to the DataContext or pass it in to this function
for (var i = 0; i < locationsInRoute.Length; i++)
{
var lcoationInRoute = locationsInRoute[i];
possibleRoutes = possibleRoutes.Where(x => x.Locations.Any(l => l.Id == locationInRoute.Id && l.Order == locationInRoute.Order));
}
route = possibleRoutes.FirstOrDefault();
return route!=null;
}