This seems like it should be straightforward, but I can't figure out how to make it work. Imagine the following models:
public class ModelA
{
public string Code {get;set;}
}
public class ModelB
{
public string Code {get;set;}
}
I am reading a set of ModelA out of a database, and I also need to grab ModelB based on the Code. The kicker is I need to match only a part of the code.
For example, ModelA the code might be "1234.00" and ModelB the code might be "001234.20" -- I want to link the two models based on both codes containing the 1234 part. I would prefer it to be a join, but I don't see how that's possible, so iterating through the first data set and matching them is fine too, but I still can't make that work.
I created a string extension method called TruncateCode, which will give me the part that I want, but LINQ doesn't seem to support calling extensions within a comparison or join (throws an exception complaining the method is unrecognized and can't be used). This is basically what I was attempting:
var query = a in db.ModelASet
(where clause)
select a;
foreach(ModelA item in query)
{
var query2 = b in db.ModelBSet
where b.Code.TruncatedCode() == item.Code.TruncatedCode() // this doesn't work :(
select b;
}
Dear reader, how would you approach this?
(INB4 "Why is your data laid out like this?" - This is a legacy database from over 10 years ago man, I'm just trying to get this to work.)
You need to materialize the results using ToList() or something similar in order to use your extension method.
var list = (a in db.ModelASet
select a).ToList();
foreach(var item in list)
{
var query2 = b in list
where b.Code.TruncatedCode() == item.Code.TruncatedCode() // this doesn't work :(
select b;
}
Related
consider the following scenario:
public class DBEntry {
public string Id;
}
public class ComputedEntry {
public string Id;
public int ComputedIndex;
}
IQueryable<DBEntry> databaseQueryable; // Somewhere hidden behind the API
IQueryable<ComputedEntry> entryQueryable; // Usable with the API
Let's assume each DBEntry has a unique Id and not much else. A ComputedEntry has a 1:n relationship with DBEntry, meaning that a DBEntrycan be expanded into more than a single ComputedEntryduring execution of the query.
Now, I am trying to query entryQueryable to get a range of computed indices, e.g:
entryQueryable.Where(dto => dto.ComputedIndex < 10 && dto.Id == "some-id");
What I'm looking for is a way of separating the given query expression to only push down the relevant parts of the query to the databaseQueryable. In the example above something like this should happen (probably in the implementation of IQueryableProvider.Execute when using the entryQueryable):
var results = databaseQueryable.Where(e => e.Id == "some-id").ToList();
int i = 0;
return results.Select(e => new ComputedEntry(e.Id, i++));
So basically I'd like the query to be separated and the relevant/compatible parts should be pushed down to the databaseQueryable.
The obvious question would be: How should I approach this? I tried to figure out a way of separating the expression with an ExpressionVisitor, but haven't been very successful here and it seems like this is a rather complex task.
Any ideas? Maybe there is an already existing method of optimizing/translating the query I am not aware of? I have looked through the documentation but couldn't find anything useful here.
Many thanks for your suggestions!
I create a simple stored procedure with some joins with the customer table and other related tables, which takes in two parameters. I can execute this SP in SQL and works.
I drag and drop this SP to my DBML file and recompile.
I add the below code in order to call the SP and return it in a List
public IQueryable<Entities.Customer> AllCustomerRanges(int CId, int ItemID)
{
List<Entities.Customer> c = myDataContext.spCustomerRanges(CId, ItemID).ToList();
}
This gives me the error:
Cannot implicitly convert type 'System.Collections.Generic.List< spCustomerRangesResult>' to 'System.Collections.Generic.List< Entities.Customer>'
Now i dont have a class spCustomerRangesResult but after some research I'm puzzled if i have done something wrong or if i need to implement a class with all the properties that the Customer class has (which sounds a little long winded) or if i've just made an error.
Any idea of how i can call a SP which shows the data in a List?
new class spCustomerRangesResult automatically generated based on sp result, you should convert it to Entities.Customer like this:
public IQueryable<Entities.Customer> AllCustomerRanges(int CId, int ItemID)
{
var c = myDataContext.spCustomerRanges(CId, ItemID).ToList();
if (c == null)
return null;
var customers = c.Select(a => new Entities.Customer
{
FirstName=a.spResultFirstName,
LastName = a.spResultLastName
//this just example conversion, change it as needed.
});
return customers;
}
please note, that I return IQueryable even though the approach that you take when using ToList() but yet returning IQuerybale may not be needed. I dont know all details so this only to show how to convert but the whole method may need re-factoring.
I have two tables Studies and Series. Series are FK'd back to Studies so one Study contains a variable number of Series.
Each Series item has a Deleted column indicating it has been logically deleted from the database.
I am trying to implement a Deleted property in the Study class that returns true only if all the contained Series are deleted.
I am using O/R Designer generated classes, so I added the following to the user modifiable partial class for the Study type:
public bool Deleted
{
get
{
var nonDeletedSeries = from s in Series
where !s.Deleted
select s;
return nonDeletedSeries.Count() == 0;
}
set
{
foreach (var series in Series)
{
series.Deleted = value;
}
}
}
This gives an exception "The member 'PiccoloDatabase.Study.Deleted' has no supported translation to SQL." when this simple query is executed that invokes get:
IQueryable<Study> dataQuery = dbCtxt.Studies;
dataQuery = dataQuery.Where((s) => !s.Deleted);
foreach (var study in dataQuery)
{
...
}
Based on this http://www.foliotek.com/devblog/using-custom-properties-inside-linq-to-sql-queries/, I tried the following approach:
static Expression<Func<Study, bool>> DeletedExpr = t => false;
public bool Deleted
{
get
{
var nameFunc = DeletedExpr.Compile();
return nameFunc(this);
}
set
{ ... same as before
}
}
I get the same exception when a query is run that there is no supported translation to SQL. (
The logic of the lambda expression is irrelevant yet - just trying to get past the exception.)
Am I missing some fundamental property or something to allow translation to SQL? I've read most of the posts on SO about this exception, but nothing seems to fit my case exactly.
I believe the point of LINQ-to-SQL is that your entities are mapped for you and must have correlations in the database. It appears that you are trying to mix the LINQ-to-Objects and LINQ-to-SQL.
If the Series table has a Deleted field in the database, and the Study table does not but you would like to translate logical Study.Deleted into SQL, then extension would be a way to go.
public static class StudyExtensions
{
public static IQueryable<study> AllDeleted(this IQueryable<study> studies)
{
return studies.Where(study => !study.series.Any(series => !series.deleted));
}
}
class Program
{
public static void Main()
{
DBDataContext db = new DBDataContext();
db.Log = Console.Out;
var deletedStudies =
from study in db.studies.AllDeleted()
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}
}
}
This maps your "deleted study" expression into SQL:
SELECT t0.study_id, t0.name
FROM study AS t0
WHERE NOT EXISTS(
SELECT NULL AS EMPTY
FROM series AS t1
WHERE (NOT (t1.deleted = 1)) AND (t1.fk_study_id = t0.study_id)
)
Alternatively you could build actual expressions and inject them into your query, but that is an overkill.
If however, neither Series nor Study has the Deleted field in the database, but only in memory, then you need to first convert your query to IEnumerable and only then access the Deleted property. However doing so would transfer records into memory before applying the predicate and could potentially be expensive. I.e.
var deletedStudies =
from study in db.studies.ToList()
where study.Deleted
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}
When you make your query, you will want to use the statically defined Expression, not the property.
Effectively, instead of:
dataQuery = dataQuery.Where((s) => !s.Deleted);
Whenever you are making a Linq to SQL query, you will instead want to use:
dataQuery = dataQuery.Where(DeletedExpr);
Note that this will require that you can see DeletedExpr from dataQuery, so you will either need to move it out of your class, or expose it (i.e. make it public, in which case you would access it via the class definition: Series.DeletedExpr).
Also, an Expression is limited in that it cannot have a function body. So, DeletedExpr might look something like:
public static Expression<Func<Study, bool>> DeletedExpr = s => s.Series.Any(se => se.Deleted);
The property is added simply for convenience, so that you can also use it as a part of your code objects without needing to duplicate the code, i.e.
var s = new Study();
if (s.Deleted)
...
I'm trying to implement encrypted columns in EF4 and using the CTP5 features to allow simple use of POCO's to query the database. Sorry that this is a lot of words, but I hope the below gives enough to explain the need and the problem!
So, bit of background, and my progress so far:
The intention is that if you query the tables without using our DAL then the data is rubbish, but I don't want the developers to worry about if/when/how the data is encrypted.
For simplicity, at this stage I'm working on the assumption any string column will be encrypted.
Now, I have successfully implemented this for returning the data using the Objectmaterialized event, and for data commits using the SavingChanges event.
So given the following class:
public class Thing
{
public int ID { get; set; }
[Required]
public string Name { get; set; }
public DateTime Date { get; set; }
public string OtherString { get; set; }
}
The below query returns all the required values and the POCO materialized has clear data in it.
var things = from t in myDbContext.Things
select t;
where myDbContext.Things is a DbSet<Thing>
Likewise, passing an instance of Thing to Things.Add()
(with clear string data in the Name and/or OtherString values)
and then calling myDbContext.SaveChanges() encrypts the strings before it gets to the data store.
Now, the problem I have is in this query:
var things = from t in myDbContext.Things
where t.Name == "Hairbrush"
select t;
This results in the unencrypted value being compared to the encrypted value in the DB. Obviously I don't want to get all the records from the database, materialize them, and then filter the results based on any supplied Where clause... so what I need to do is: intercept that query and rewrite it by encrypting any strings in the Where clause.
So I've looked at:
writing a query provider, but that doesn't seem like the right solution... (is it?)
writing my own IQueryable wrapper for the DbSet which will capture the expression, run over it using an expression tree visitor and then forward the new expression to the DbSet...
Attempts at both have left me somewhat lost! I prefer the second solution i think since it feels a bit neater, and is probably clearer to other developers in future. But I'm happy to go with either or another better option!!
The main thing I am struggling with is when/how the LINQ expression is applied to the object... I think i've got myself a bit confused as to where the expression executes in the IQueryable object thus I'm not sure which method I need to implement in my wrapper to then grab and manipulate the expression being passed in...
I'm sure I'm missing something fairly obvious here and I'm waiting for that light bulb moment... but its not coming!!
Any help will be very gratefully received!
Thought I'd let you know what my final solution was.
In the end I have gone a wrapper class which implements a Where method, but without going to the extent of implementing IQueryable entirely. LINQ will still execute against the class (at least to the extent that I want/need it to) and will call the Where method with the expression from the LINQ.
I then traverse this ExpressionTree and replace my strings with encrypted values before forwarding the new expressiontree to the internal DbSet. and then returning the result.
Its pretty crude, and has its limitation, but works for our particular circumstance without problem.
Thanks,
Ben
you should use the QueryInterceptor attribute, search here in SO or in google and you find examples on how to use it.
a snippet:
[QueryInterceptor("Orders")]
public Expression<Func<Order, bool>> FilterOrders()
{
return o => o.Customer.Name == /* Current principal name. */;
}
// Insures that the user accessing the customer(s) has the appropriate
// rights as defined in the QueryRules object to access the customer
// resource(s).
[QueryInterceptor ("Customers")]
public Expression<Func<Customer, bool>> FilterCustomers()
{
return c => c.Name == /* Current principal name. */ &&
this.CurrentDataSource.QueryRules.Contains(
rule => rule.Name == c.Name &&
rule.CustomerAllowedToQuery == true
);
}
You can use David Fowler's Query Interceptor:
https://github.com/davidfowl/QueryInterceptor
One example of its use:
IQueryable q = ...;
IQueryable modifed = q.InterceptWith(new MyInterceptor());
And on class MyInterceptor:
protected override Expression VisitBinary(BinaryExpression node) {
if (node.NodeType == ExpressionType.Equal) {
// Change == to !=
return Expression.NotEqual(node.Left, node.Right);
}
return base.VisitBinary(node);
}
I have a View Model that has some serious nesting. I need to populate it from Entity Framework 4. I tried creating one big linq statement to populate it, but it says it doesn't recognize the .ToList() methods. It compiles fine. Runtime error is
LINQ to Entities does not recognize the method
'System.Collections.Generic.List`1[ProductDepartment] ToList[ProductDepartment]
(System.Collections.Generic.IEnumerable`1[ProductDepartment])' method,
and this method cannot be translated into a store expression.
What is a more efficient way to populate something like this without doing several thousand database calls?
List<Product> Products {
int ID
string Name
...
List<Department> Departments {
int ID
string Name
}
List<Image> Images {
int ID
string Name
}
List<Price> Prices {
int ID
string Name
List<Version> Versions {
int ID
string Name
List<Pages> Pages {
int ID
string Name
} } } }
The horrible Linq code looks something like this
var myProducts = (from myProduct in DC.MyProducts
where p => p.productGroup == 1
select new Product {
ID = myProduct.ID,
Name = myProduct.Name,
Departments = (from myDept in DC.MyDepartments
where q => q.fkey = myProduct.pkey
select new Department {
ID = myDept.ID,
Name = myDept.Name
}).ToList(),
...
//Same field assignment with each nesting
}).ToList();
Update:
The fix was to remove all the .ToLists(), which worked better anyway.
Now I have to do filtering and sorting on the end product.
Well for starters, that is one crazy model, but i'm assuming you already know this.
Do you really need all that info at once?
I'll play devil's advocate here and assume you do, in which case you have a couple of logical choices:
1) As #xandy mentioned - use .Include to eager load your associations in the one call. This is assuming you have setup navigational properties for your entites in your EDMX.
2) Use a View. Put all that crazy joining logic inside the database, making your EF work a very simple select from the view. The downside of this is your queries to the view basically become read only, as i don't believe you can perform updates to an entity which is mapped to a view.
So it's your choice - if this is a readonly collection for displaying data, use a View, otherwise eager-load your associations in the one hit.
Also, be careful when writing your LINQ queries - i see you have multiple .ToList statements, which will cause the query to be executed.
Build up your query, then perform the .ToList once at the end.
why do you require all this informataion at one go? You can use lazy loading when a nested property is accessed?