I'm busy with a LINQ to SQL project that basically creates multiple threads for each entity type in my database, which constantly queries information from the DB in a thread.
Here's a pseudo example:
streamer.DefineDataExpression<Contacts>(x => x.FirstName == "Bob");
while(true)
{
List<Contacts> MyContactsResult = streamer.ResultList;
// do whatever with MyContactsResult
}
The above code doesn't exist, but this is what I have so far for the 'streamer' class (it obviously doesn't work, but you can see what I'm trying to achieve above):
public void DefineExpression(System.Linq.Expressions.Expression<System.Func<T, bool>> expression)
{
using (var db = new LINQDataContext())
{
ResultList = db.GetTable<T>().Where(expression);
}
}
How do I go about creating a method like 'DefineExpression' that will allow me to query a LINQ type dynamically?
Why not use the Dynamic LINQ provider, as mentioned by Scott Guthrie. I think that would give you everything you are looking for, because you can define the query as a string. Therefore, you can more easily build a string representation of your query, and execute on the fly.
Related
I have been trying to pass in a dynamic list of Expressions to a MongoDB C# Driver query using Linq ... This method works for me with regular Linq queries against an ORM, for example, but results in an error when applied to a MongoDB query ... (FYI: I am also using LinqKit's PredicateBuilder)
//
// I create a List of Expressions which I can then add individual predicates to on an
// "as-needed" basis.
var filters = new List<Expression<Func<Session, Boolean>>>();
//
// If the Region DropDownList returns a value then add an expression to match it.
// (the WebFormsService is a home built service for extracting data from the various
// WebForms Server Controls... in case you're wondering how it fits in)
if (!String.IsNullOrEmpty(WebFormsService.GetControlValueAsString(this.ddlRegion)))
{
String region = WebFormsService.GetControlValueAsString(this.ddlRegion).ToLower();
filters.Add(e => e.Region.ToLower() == region);
}
//
// If the StartDate has been specified then add an expression to match it.
if (this.StartDate.HasValue)
{
Int64 startTicks = this.StartDate.Value.Ticks;
filters.Add(e => e.StartTimestampTicks >= startTicks);
}
//
// If the EndDate has been specified then add an expression to match it.
if (this.EndDate.HasValue)
{
Int64 endTicks = this.EndDate.Value.Ticks;
filters.Add(e => e.StartTimestampTicks <= endTicks);
}
//
// Pass the Expression list to the method that executes the query
var data = SessionMsgsDbSvc.GetSessionMsgs(filters);
The GetSessionMsgs() method is defined in a Data services class ...
public class SessionMsgsDbSvc
{
public static List<LocationOwnerSessions> GetSessionMsgs(List<Expression<Func<Session, Boolean>>> values)
{
//
// Using the LinqKit PredicateBuilder I simply add the provided expressions
// into a single "AND" expression ...
var predicate = PredicateBuilder.True<Session>();
foreach (var value in values)
{
predicate = predicate.And(value);
}
//
// ... and apply it as I would to any Linq query, in the Where clause.
// Additionally, using the Select clause I project the results into a
// pre-defined data transfer object (DTO) and only the DISTINCT DTOs are returned
var query = ApplCoreMsgDbCtx.Sessions.AsQueryable()
.Where(predicate)
.Select(e => new LocationOwnerSessions
{
AssetNumber = e.AssetNumber,
Owner = e.LocationOwner,
Region = e.Region
})
.Distinct();
var data = query.ToList();
return data;
}
}
Using the LinqKit PredicateBuilder I simply add the provided expressions into a single "AND" expression ... and apply it as I would to any Linq query, in the Where() clause. Additionally, using the Select() clause I project the results into a pre-defined data transfer object (DTO) and only the DISTINCT DTOs are returned.
This technique typically works when I an going against my Telerik ORM Context Entity collections ... but when I run this against the Mongo Document Collection I get the following error ...
Unsupported filter: Invoke(e => (e.Region.ToLower() == "central"),
{document})
There is certainly something going on beneath the covers that I am unclear on. In the C# Driver for MongoDB documentation I found the following NOTE ...
"When projecting scalars, the driver will wrap the scalar into a
document with a generated field name because MongoDB requires that
output from an aggregation pipeline be documents"
But honestly I am not sure what that neccessarily means or if it's related to this problem or not. The appearence of "{document}" in the error suggests that it might be relevant though.
Any additional thoughts or insight would be greatly appreciated though. Been stuck on this for the better part of 2 days now ...
I did find this post but so far am not sure how the accepted solution is much different than what I have done.
I'm coming back to revisit this after 4 years because while my original supposition did work it worked the wrong way which was it was pulling back all the records from Mongo and then filtering them in memory and to compound matters it was making a synchronous call into the database which is always a bad idea.
The magic happens in LinqKit's expand extension method
That flattens the invocation expression tree into something the Mongo driver can understand and thus act upon.
.Where(predicate.Expand())
I tried to create a generic CreateOrUpdate function for EntityFramework where my classes/tables always have an ID field. Since that data is sent to a webpage and posted back to the server I am totally disconnected from the context and have to use the standard (set.Any / set.FirstOrDefault) way of checking to see if my object already exists.
The method works perfectly in terms of doing its job and creating/updating as required. But what I have found is that the db.Set<T>().FirstOrDefault(whereFunction); brings back all data from the DB and then performs the FirstOrDefault in memory. I would much prefer this to happen in SQL, but no matter what I have tried I can't get it to work.
Do you have any suggestions on how I might make the FirstOrDefault function translate to proper SQL so that I don't retrieve too much from the DB?
Also, I have tried First, Any & Count, all of which get all of the rows from the DB.
public void CreateOrUpdateEntity<T>(T entity) where T : class
{
using (var db = new ProjectContext())
{
Func<T, bool> whereFunction = m => m.As<dynamic>().ID == entity.As<dynamic>().ID;
var firstValue = db.Set<T>().FirstOrDefault(whereFunction);
if (firstValue == null)
{
db.Set<T>().Attach(entity);
db.ChangeTracker.Entries<T>().First(e => e.Entity == entity).State = EntityState.Added;
}
else
{
db.ChangeTracker.Entries<T>().First(e => e.Entity == entity).State = EntityState.Modified;
}
}
}
You should use expression instead of Func<T, bool> delegate. Usage of delegate invokes Enumerable.FirstOrDefault(Func<T,TResult>) method which is executed in memory (and requires all data loaded to client), instead of Queryable.FirstOrDefault(Expression<Func<T, TResult>>) which is translated into SQL and executed on server side.
NOTE: I have doubts about whether your delegate can be translated into SQL.
BTW: In order to get entity by id you can use method Find:
var firstValue = db.Set<T>().Find(entity.As<dynamic>().ID);
I have two tables Studies and Series. Series are FK'd back to Studies so one Study contains a variable number of Series.
Each Series item has a Deleted column indicating it has been logically deleted from the database.
I am trying to implement a Deleted property in the Study class that returns true only if all the contained Series are deleted.
I am using O/R Designer generated classes, so I added the following to the user modifiable partial class for the Study type:
public bool Deleted
{
get
{
var nonDeletedSeries = from s in Series
where !s.Deleted
select s;
return nonDeletedSeries.Count() == 0;
}
set
{
foreach (var series in Series)
{
series.Deleted = value;
}
}
}
This gives an exception "The member 'PiccoloDatabase.Study.Deleted' has no supported translation to SQL." when this simple query is executed that invokes get:
IQueryable<Study> dataQuery = dbCtxt.Studies;
dataQuery = dataQuery.Where((s) => !s.Deleted);
foreach (var study in dataQuery)
{
...
}
Based on this http://www.foliotek.com/devblog/using-custom-properties-inside-linq-to-sql-queries/, I tried the following approach:
static Expression<Func<Study, bool>> DeletedExpr = t => false;
public bool Deleted
{
get
{
var nameFunc = DeletedExpr.Compile();
return nameFunc(this);
}
set
{ ... same as before
}
}
I get the same exception when a query is run that there is no supported translation to SQL. (
The logic of the lambda expression is irrelevant yet - just trying to get past the exception.)
Am I missing some fundamental property or something to allow translation to SQL? I've read most of the posts on SO about this exception, but nothing seems to fit my case exactly.
I believe the point of LINQ-to-SQL is that your entities are mapped for you and must have correlations in the database. It appears that you are trying to mix the LINQ-to-Objects and LINQ-to-SQL.
If the Series table has a Deleted field in the database, and the Study table does not but you would like to translate logical Study.Deleted into SQL, then extension would be a way to go.
public static class StudyExtensions
{
public static IQueryable<study> AllDeleted(this IQueryable<study> studies)
{
return studies.Where(study => !study.series.Any(series => !series.deleted));
}
}
class Program
{
public static void Main()
{
DBDataContext db = new DBDataContext();
db.Log = Console.Out;
var deletedStudies =
from study in db.studies.AllDeleted()
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}
}
}
This maps your "deleted study" expression into SQL:
SELECT t0.study_id, t0.name
FROM study AS t0
WHERE NOT EXISTS(
SELECT NULL AS EMPTY
FROM series AS t1
WHERE (NOT (t1.deleted = 1)) AND (t1.fk_study_id = t0.study_id)
)
Alternatively you could build actual expressions and inject them into your query, but that is an overkill.
If however, neither Series nor Study has the Deleted field in the database, but only in memory, then you need to first convert your query to IEnumerable and only then access the Deleted property. However doing so would transfer records into memory before applying the predicate and could potentially be expensive. I.e.
var deletedStudies =
from study in db.studies.ToList()
where study.Deleted
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}
When you make your query, you will want to use the statically defined Expression, not the property.
Effectively, instead of:
dataQuery = dataQuery.Where((s) => !s.Deleted);
Whenever you are making a Linq to SQL query, you will instead want to use:
dataQuery = dataQuery.Where(DeletedExpr);
Note that this will require that you can see DeletedExpr from dataQuery, so you will either need to move it out of your class, or expose it (i.e. make it public, in which case you would access it via the class definition: Series.DeletedExpr).
Also, an Expression is limited in that it cannot have a function body. So, DeletedExpr might look something like:
public static Expression<Func<Study, bool>> DeletedExpr = s => s.Series.Any(se => se.Deleted);
The property is added simply for convenience, so that you can also use it as a part of your code objects without needing to duplicate the code, i.e.
var s = new Study();
if (s.Deleted)
...
Say I have a LINQ-to-XML query that generates an anonymous type like this:
var aQuery =
(from a in document.Root.Elements("items")
select new {
id = a.Attribute("id").Value,
type = a.Attribute("type").Value,
modified = a.Attribute("modified").Value
});
if there a way to store that query expression in a variable or constant and then execute at runtime? The basic idea is that I have a bunch of these expressions and it would be handy if they could all be defined in one place and then invoked dynamically thru a single method where I just need to pass in the XML document and which expression to use. Thanks.
You could define them as methods quite easily, though you'd forfit the right to use anonymous types.
public static IQueryable<Item> GetItemsFromXml(XDocument document)
{
return (from a in document.Root.Elements("items")
select new Item
{
Id = a.Attribute("id").Value,
Type = a.Attribute("type").Value,
Modified = a.Attribute("modified").Value
});
}
Having said that, patterns like the repository pattern are used to wrap the whole process of accessing data.
I'm trying to implement encrypted columns in EF4 and using the CTP5 features to allow simple use of POCO's to query the database. Sorry that this is a lot of words, but I hope the below gives enough to explain the need and the problem!
So, bit of background, and my progress so far:
The intention is that if you query the tables without using our DAL then the data is rubbish, but I don't want the developers to worry about if/when/how the data is encrypted.
For simplicity, at this stage I'm working on the assumption any string column will be encrypted.
Now, I have successfully implemented this for returning the data using the Objectmaterialized event, and for data commits using the SavingChanges event.
So given the following class:
public class Thing
{
public int ID { get; set; }
[Required]
public string Name { get; set; }
public DateTime Date { get; set; }
public string OtherString { get; set; }
}
The below query returns all the required values and the POCO materialized has clear data in it.
var things = from t in myDbContext.Things
select t;
where myDbContext.Things is a DbSet<Thing>
Likewise, passing an instance of Thing to Things.Add()
(with clear string data in the Name and/or OtherString values)
and then calling myDbContext.SaveChanges() encrypts the strings before it gets to the data store.
Now, the problem I have is in this query:
var things = from t in myDbContext.Things
where t.Name == "Hairbrush"
select t;
This results in the unencrypted value being compared to the encrypted value in the DB. Obviously I don't want to get all the records from the database, materialize them, and then filter the results based on any supplied Where clause... so what I need to do is: intercept that query and rewrite it by encrypting any strings in the Where clause.
So I've looked at:
writing a query provider, but that doesn't seem like the right solution... (is it?)
writing my own IQueryable wrapper for the DbSet which will capture the expression, run over it using an expression tree visitor and then forward the new expression to the DbSet...
Attempts at both have left me somewhat lost! I prefer the second solution i think since it feels a bit neater, and is probably clearer to other developers in future. But I'm happy to go with either or another better option!!
The main thing I am struggling with is when/how the LINQ expression is applied to the object... I think i've got myself a bit confused as to where the expression executes in the IQueryable object thus I'm not sure which method I need to implement in my wrapper to then grab and manipulate the expression being passed in...
I'm sure I'm missing something fairly obvious here and I'm waiting for that light bulb moment... but its not coming!!
Any help will be very gratefully received!
Thought I'd let you know what my final solution was.
In the end I have gone a wrapper class which implements a Where method, but without going to the extent of implementing IQueryable entirely. LINQ will still execute against the class (at least to the extent that I want/need it to) and will call the Where method with the expression from the LINQ.
I then traverse this ExpressionTree and replace my strings with encrypted values before forwarding the new expressiontree to the internal DbSet. and then returning the result.
Its pretty crude, and has its limitation, but works for our particular circumstance without problem.
Thanks,
Ben
you should use the QueryInterceptor attribute, search here in SO or in google and you find examples on how to use it.
a snippet:
[QueryInterceptor("Orders")]
public Expression<Func<Order, bool>> FilterOrders()
{
return o => o.Customer.Name == /* Current principal name. */;
}
// Insures that the user accessing the customer(s) has the appropriate
// rights as defined in the QueryRules object to access the customer
// resource(s).
[QueryInterceptor ("Customers")]
public Expression<Func<Customer, bool>> FilterCustomers()
{
return c => c.Name == /* Current principal name. */ &&
this.CurrentDataSource.QueryRules.Contains(
rule => rule.Name == c.Name &&
rule.CustomerAllowedToQuery == true
);
}
You can use David Fowler's Query Interceptor:
https://github.com/davidfowl/QueryInterceptor
One example of its use:
IQueryable q = ...;
IQueryable modifed = q.InterceptWith(new MyInterceptor());
And on class MyInterceptor:
protected override Expression VisitBinary(BinaryExpression node) {
if (node.NodeType == ExpressionType.Equal) {
// Change == to !=
return Expression.NotEqual(node.Left, node.Right);
}
return base.VisitBinary(node);
}