Develop Reference

C# IEnumerable Return Changes after Function Parameter Changes

I was writing unit tests to compare an original response to a filtered response using a request object as a parameter. In doing so I noticed that if I change the request object after getting a response the IEnumerable list will change - As I type this, my thinking is that because it is an IEnumerable with LINQ, the request.Filter property is a reference in the LINQ query, which is what causes this behavior. If I converted this to a list instead of an IEnumerable, I suspect the behavior would go away because the .ToList() will evaluate the LINQ expressions instead of deferring. Is that the case?
public class VendorResponse {
public IEnumerable<string> Vendors { get; set; }
}
var request = new VendorRequest() {
Filter = ""
};
var response = await _service.GetVendors(request);
int vendorCount = response.Vendors.Count(); // 20
request.Filter = "at&t";
int newCount = response.Vendors.Count(); // 17
public async Task<VendorResponse> GetVendors(VendorRequest request)
{
var vendors = await _dataService.GetVendors();
return new VendorResponse {
Vendors = vendors.Where(v => v.IndexOf(request.Filter) >= 0)
}
}

If deferred execution is preferable, you can capture the current state of request.Filter with a local variable and use that in the Where predicate
public async Task<VendorResponse> GetVendors(VendorRequest request)
{
var filter = request.Filter;
var vendors = await _dataService.GetVendors();
return new VendorResponse {
Vendors = vendors.Where(v => v.IndexOf(filter) >= 0)
}
}

Yes!
This is an example of deferred execution of an IEnumerable, which just encapsulates a query on some data without encapsulating the result of that query.
An IEnumerable can be enumerated (via its IEnumerator), and "knows" how to enumerate the query it encapsulates, but this will not actually happen until something executes the enumeration.
In your case the enumeration is executed by the call to .Count() which needs to know how many items are in the result of the query. The enumeration occurs every time you call .Count(), so changing the filter between the two invocations leads to you getting two different results.
As you have correctly deduced, calling .ToList() and capturing the result in a variable before performing any further operations would lead to you capturing the resulting data rather than the query, and so lead to both counts having the same value.
Try this out yourself. In future, be sure to force the evaluation of the enumerable before passing to other queries, or returning out to unknown code, otherwise you or your users will encounter unexpected behaviour and possible performance issues.
Hope this helps :)
Edit 1:
As Moho has pointed out, and you have also alluded to in your original post, this is also a result of the request.Filter being captured by the IEnumerable as a reference type. If you can capture the value and pass this in instead, the result of the IEnumerable will no longer be modified by changing the filter.

How can I add information to a C# oData result set after the filters have been applied

I am using c# MVC and have an oData-powered view to display data from a large table (~1 million rows). Much of the data can be queried and displayed straight from the database, but one column needs to be reformatted to include data looked up from another database.
It would be too slow to run these lookups across all 1 million rows in the database, but it would be fine to run them against the 20 rows that are being returned to the user. I do not need to sort or filter on these columns.
How can I run these lookups after the rows have already been filtered?
One potential pattern could be:
public class LogController : System.Web.Http.OData.OdataController
{
public IQueryable<LogResult> GetLogs(ODataQueryOptions opts)
{
ODataV2(Request);
var logs = LogRepository.All();
/* Do something here to filter out unwanted results */
// I'll need to convert to a list if I'm going to run a foreach over it
var logsList = logs.ToList()
foreach (var log in logsList)
{
log.CalculatedProperty = ExpensiveFunction(log);
}
return logsList.AsQueryable();
}
}
}
Alternatively maybe there is a way to post-process the output of GetLogs before it is returned to the user? Or I can use a calculated property of Log?
The key thing is I want to run ExpensiveFunction() 20 times, instead of 1 million.

You can use some little trick to prevent converting to list and vise versa:
/* Do something here to filter out unwanted results */
return logs.Select(l => {
l.CalculatedProperty = ExpensiveFunction(log);
return l; // or even yield return
});
I can't say that this solution is the best, but in some cases it might be a bit better.

Can you apply the QueryOption and then update the result?
Change return type to IHttpActionResult
var logsList = ops.ApplyTo(logs, new ODataQuerySettings()).toList();
// reform logsList
return MakeGenericResult(logsList.AsQueryable(), logsList.AsQueryable().GetType());
private IHttpActionResult Ok(object content, Type type)
{
var resultType = typeof(OkNegotiatedContentResult<>).MakeGenericType(type);
return Activator.CreateInstance(resultType, content, this) as IHttpActionResult;
}

Filter Generic Collection Down to Records Containing Any Item in Collection of String

string rep = "Joe Shmoe"
ObjectSet<StoreData> storeData = edmContext.StoreData;
ObjectSet<CallData> callData = edmContext.CallData;
IEnumerable<string> repStoreData = storeData.Where(r => r.RepName == rep).Select(s => s.Location);
IEnumerable<CallData> repCallData = Here is where I want to filter down the callData collection down to just the records that have a location that is contained in the repStoreData collection
I've tried using some form of Join and Any but don't really understand the arguments those are asking for.
This was my best attempt and it is a no go.
... = callData.Join(d => d.LOCATION.Any(repStoreData));

Well you don't have to use a join. You could just use:
callData.Where(d => repStoreData.Contains(d.LOCATION))
That's assuming d.LOCATION is a single string.
However, you probably don't want to do that with your current declaration of repStoreData as IEnumerable<string> - LINQ won't be able to turn that into a query to be executed at the database.
If you're able to declare repStoreData as IQueryable<string>, however, that would be more likely to work well. I don't know whether that will work with ObjectSet<T>, but I'd hope so.

Dapper Multiple Results From single query

Hi i am trying to get to grips with Dapper.
My situation is i want to pull two values from a query into two separate strings. Im not sure if i am going about this in the correct way, but this is what i am doing:
string sql = #"Select type, name
FROM ZipData
WHERE Zip = #zip";
using (var multi = conn.QueryMultiple(sql, new { zip = zip }))
{
string result = multi.Read<string>().SingleOrDefault();
}
And i am getting Cannot access a disposed object. Object name: 'GridReader'. when trying to read the second string.The thing is it gets the first value correctly and has both the fields in in the reader i am trying to get. Im sure im misusing the api.
What am i doing wrong here? Ive googled but can find a specific example.

You are mis-using QueryMultiple. That is defined for compound SQL statements that return multiple result sets. Something like:
SELECT Foo FROM MyTable;
SELECT Bar FROM MyOtherTable;
On the other hand, you are trying to get two different columns from a single result set, so you should just use the normal Query method:
var result = conn.Query(sql, new { zip = zip }).Single();
var type = result.type;
var name = result.name;
Query returns an enumerable (because generally a query can return multiple rows). It appears that you only want one row, however, so we invoke .Single at the end to just get that row. From there, the return type is dynamic so you can simply refer to the properies implied by the columns in your SELECT statement: type and name.

Castle ActiveRecord - 2nd level cache - need explanation of this behaviour

I am just doing some experiments on Castle AR and 2nd level cache of NH. In the following two methods, I can see caching working fine but only for the repetition of the call of each. In other words if I call RetrieveByPrimaryKey twice for same PK, the object is found in cache. And if I call RetrieveAll twice, I see SQL issued only once.
But if I call RetrieveAll and then RetrieveByPrimaryKey with some PK, I see two SQL statements getting issued. My question is, Why AR does not look for that entity in cache first? Sure it would have found it there as a result of previous call to RetrieveAll.
public static T RetrieveByPrimaryKey(Guid id)
{
var res = default(T);
var findCriteria = DetachedCriteria.For<T>().SetCacheable(true);
var eqExpression = NHibernate.Criterion.Expression.Eq("Id", id);
findCriteria.Add(eqExpression);
var items = FindAll(findCriteria);
if (items != null && items.Length > 0)
res = items[0];
return res;
}
public static T[] RetrieveAll()
{
var findCriteria = DetachedCriteria.For<T>().SetCacheable(true);
var res = FindAll(findCriteria);
return res;
}

You're using caching on specific queries. that means that cache lookup is done in the following way:
search the cahce for results of a query with identical syntax AND the same parameters. If found- use cached results.
nHibernate (this has nothing to do with AR, by the way) doesn't know that logically, one query 'contains' the other. so this is why you're getting 2 db trips.
I would suggest using ISession.Get to retreive items by ID (it's the recommended method). I think (not tested it though) that Get can use items cached by other queries.
here's a nice blog post from ayende about it.