I'm using generics because I need a lot of reusability with different types of data. my main problem is querying data. I'm looking for a way to query something like this:
public void test<T>(int id, T type) where T : class
{
using (var ctx = myDbContext())
{
var myTbl = ctx.Set<T>();
//this line gets the primary key of the table
string key = myTbl.GetPrimaryKey(ctx);
//this is the query I want:
var myResult = myTbl.FirstOrDefault(x => x.key == id);
//let's say if key = "UserId", then (x => x.UserId == id) or something that translates to this.
}
}
also I have implemented following method:
public object GetPropertyValue(object src, string propertyName)
that I can use to get value of a specific property.
but my problem is that I can't use it inside the .FirstOrDefault() call because of the LINQ to query issues with methods.
I currently use this code instead:
var myResult = myTbl.ToList().FirstOrDefault(x => (int)GetPropertyValue(x, key) == id);
which is fine with a few number of rows in database, but when data grows in future it will have a lot of performance impact.
P.S: I'm using EF power tools for reverse engineering code first
Umm, your code sample is completely unclear. Please, atleast provide correct variable names, because now I don't even know where you use 'key' variable.
Also, if you want to store and query objects and it's properties of various inheritance and nestings, consider to use NoSQL databases instead of relation based SQL engines.
First -> Dont use myTbl.ToList().FirstOrDefault(x => (int)GetPropertyValue(x, key) == id) because it generates a select that brings all rows from that table, then filter by id in memory. You should translate your filter to an Expression> that will generare a select filtered by the Id column.
Build your linq expression like this:
var x = Expression.Parameter(typeof(T), "x");
string keyPropName = type.GetPrimaryKey(ctx);
var equalExp = Expression.Equal(
Expression.Property(x, keyPropName),
Expression.Constant(id)
);
var lambda = Expression.Lambda<Func<T, bool>>(equalExp, x); //x => x.Id == idValue
var myResult = myTbl.FirstOrDefault(lambda);
I used your GetPropertyValue and GetPrimaryKey methods.
Related
I have a table that keeps track of Agreements that I wish to build a Web API around. I would like to be able to use a GET method to be able to search the table. However, I want the parameters to be optional. For example, if I use a GET call and specify Date == 1/23/2018 as well as Status == Active I would like it to return all agreements that meet that criteria even though the other possible parameters may equal null.
I am using binding on the controller to pull from the Uri that expects an Agreement like object. This is simply the data context for the Table Agreement. I've been able to get this to work. The issue comes with the query. If the user just specifies two of the variables the rest of the parameters are set to null.
How would I write the linq to SQL query so that only the non null parameters are used to query? Something to the effect of parsing through possible parameters and then building the query off of the non null values.
Note: I do not want the client to have to specify a value for all parameters. Just the ones relevant to them and then have the controller/ service class sort it out.
Controller
namespace API.Controllers
{
public class AgreementsController : ApiController
{
private AgreementRepository agreementRepository;
public AgreementsController()
{
this.agreementRepository = new AgreementRepository();
}
public AGREEMENT[] Get([FromUri]AGREEMENT Agreement)
{
return agreementRepository.GetAgreement(Agreement);
}
}
}
Service
namespace API.Services
{
public class AgreementRepository
{
private DatabaseEntities db = new DatabaseEntities();
public AGREEMENT[] GetAgreement(AGREEMENT agmt)
{
var query = from a in db.AGREEMENTS select a;
query.Where(a => ??????);
AGREEMENT[] agreements = query.ToArray<AGREEMENT>();
return agreements;
}
}
}
EDIT
I would like to find a way to avoid hard coding parameter values and would rather have the query be built dynamically based upon available parameters. This way changes to the table and/or the data context will directly be reflected and handled appropriately by this code.
EDIT 2
Ive taken a crack at this using reflection to build a dictionary and the attempt to build queries dynamically by looping through the dictionary. The query below appears to not be valid as every agreement is being returned.
public AGREEMENT[] GetAgreement(AGREEMENT agmt)
{
Type type = agmt.GetType();
PropertyInfo[] properties = type.GetProperties();
Dictionary<string, object> propDictionary = new Dictionary<string, object>();
foreach (PropertyInfo property in properties)
{
var propertyValue = property.GetValue(agmt, null);
if (propertyValue != null)
{
propDictionary.Add(property.Name, propertyValue);
}
}
var query = from a in db.AGREEMENTS select a;
foreach (var prop in propDictionary)
{
query.Where(A => A.GetType().GetProperty(prop.Key).GetValue(A,null) == prop.Value);
}
AGREEMENT[] agreements = query.ToArray<AGREEMENT>();
return agreements;
}
You can do a null check for each optional parameter e.g:
var query = from a in db.AGREEMENTS select a;
query = query.Where(a => a.something == mandatoryParamter);
if(optionalParameter1 != null)
{
query = query.Where(a => a.something == optionalParameter1);
}
if(optionalParameter2 != null)
{
query = query.Where(a => a.something == optionalParameter2);
}
etc.
You can add a where clause like below if nothing has been supplied then optional paremater will be null.
var query = from a in db.AGREEMENTS select a;
query = query.Where(a => a.something == mandatoryParamter)
.Where(optionalParameter1 == null ||
a => a.something == optionalParameter1)
.Where(optionalParameter2 == null ||
a => a.something == optionalParameter2);
From my point of view the easiest and most powerful approach would be to use OData.
You can get started at https://learn.microsoft.com/en-us/aspnet/web-api/overview/odata-support-in-aspnet-web-api/supporting-odata-query-options
If you want to only apply filter and then display and format results in custom way, I would recommend "Invoking Query Options Directly" section from reference above.
You can use Nullable Type operator and run query with the optional parameter:
public AGREEMENT[] GetAgreement(AGREEMENT agmt)
{
var query = from a in db.AGREEMENTS select a;
query.Where(a => a.col1 ?? default(col1 Type) && a.col2 ?? default(col2 Type));
AGREEMENT[] agreements = query.ToArray<AGREEMENT>();
return agreements;
}
Here you can add as many parameters as you want to use. I hope this helps.
You can create the query at run-time using methods of Expression class. The core logic of the answer would be:
Create the IQueryable<Model> result = dbContext.Models.AsQueryable();
Detect which properties should participate in query, using keys which exists in query string or route values if you use them.
For each property which should be included in criteria:
result = result.Where(x=> create a criteria for the property);
At last, return result or result.ToList() or whatever type which you need as result.
Important Note - Before jumping to the example
Before reading the example, please keep in mind:
In a real world application, the criteria for each field can be different, for example for a string property you may need StartsWith, for another one you may need Contains, for an int property you may need range using >= and/or <=, for another int property you may want to check for exact value and so on.
Instead of trying to write a general purpose logic, consider creating a specific search logic. You can see good examples around, for example:
Filter/Search using Multiple Fields - ASP.NET MVC
Filtering the data at the controller before it is rendered in a view
If the number of properties and number of entity classes are too much, what you can do to facilitate the development, is generating the search logic using code generation (for example using T4 templates) at design/coding time. Then you can change them simply to what you need later.
Example
In the following example, I've implemented a search method which allows you to search between products, based on the passed values using query string. You may want to include route values as well.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using System.Net.Http;
using System.Web.Http;
using WebApiTest.Models;
public class ProductsController : ApiController
{
[HttpGet]
public IEnumerable<Product> Search([FromUri] Product product)
{
var parameters = Request.GetQueryNameValuePairs().Select(x => x.Key.ToLower());
using (var db = new TestDBEntities())
{
var result = db.Products.AsQueryable();
foreach (var property in typeof(Product).GetProperties())
{
if (parameters.Contains(property.Name.ToLower()))
{
var x = Expression.Parameter(typeof(Product), "x");
var propertyExpression = Expression.Property(x, property.Name);
var valueExpression = Expression.Convert(
Expression.Constant(property.GetValue(product)),
property.PropertyType);
var criteria = Expression.Equal(propertyExpression, valueExpression);
var lambda = Expression.Lambda<Func<Product, bool>>(criteria, x);
result = result.Where(lambda);
}
}
return result.ToList();
}
}
}
I have got the following expression that works with mockup data - hereby not using Entity Framework:
public static Expression<Func<Resource, bool>> FilterResourcesByUserCriteria(IEnumerable<FilterValue> filterValuesForUser)
{
Expression<Func<Resource, bool>> filter = (resource) =>
// Get filter values for the current resource in the loop
resource.ResourceFilterValues
// Group filter values for user
.GroupBy(filterValue => filterValue.FilterValue.FilterGroup.Id)
// Each group must fulfill the following logic
.All(filterGroup =>
// For each filter group, only select the user values from the same group
filterValuesForUser
.Where(filterValueForUser => filterValueForUser.FilterGroup.Id == filterGroup.Key)
.Select(filterValueForUser => filterValueForUser.FilterValue1)
// Each group must at least one value in the sublist of filter values of the current user
.Any(filterValueForUser => filterGroup
.Select(resourceFilterValue => resourceFilterValue.FilterValue.FilterValue1)
.Any(x => x == filterValueForUser))
);
}
However, I get this famous exception when I try to insert this expression in the where clause of my repository method (using Entity Framework):
Unable to create a constant value of type. Only primitive types or enumeration types are supported in this context.
I suspect this has something to do with a parameter called filterValuesForUser, which is a collection of a complex (i.e. custom) type.
Is this behavior even possible in Entity Framework where I do a subquery that is not directly related to Entity Framework? What I want to achieve here is to query on a subset of a custom list for each group in the query.
Any solutions for this or other workarounds? I'd like to minimize the amount of database calls, preferrably limit it to just one.
The exact query you are asking for is impossible with LinqToEF (due to limitation of SQL). But fear not. It is possible to salvage your problem with a slight tweaking.
public static Expression<Func<Resource, bool>> FilterResourcesByUserCriteria(FilterValue filterValueForUser)
{
//I assume you can write this part yourself.
}
public IQueryable<Resource> GetResources()
{
IQueryable<Resource> resources = _context.Resources;
IEnumerable<FilterValue> filterValuesForUser = GetFilterValues();
IEnumerable<IQueryable<Resource>> queries = from filter in filterValuesForUser
let filterExp = FilterResourcesByUserCriteria(filter)
select resources.Where(filterExp);
return Enumerable.Aggregate(queries, (l, r) => Queryable.Concat(l, r));
}
Types and Extension methods expanded for clarity.
In addition to Aron's answer, I used the PredicateBuilder utility in the LinqKit assembly to generate 1 expression rather than multiple and separate expresssions. This also avoids doing multiple database calls.
Here is how you can achieve this (pseudo-code):
public IQueryable<Resource> GetResources()
{
MyContext ctx = new MyContext ();
IEnumerable<Expression<Func<Resource, bool>>> queries =
filterValuesForUser.GroupBy(x => x.FilterGroup)
.Select(filter => SecurityFilters.FilterResourcesByUserCriteriaEF(filter.Select(y => y.FilterValue1)))
.Select(filterExpression => { return filterExpression; });
Expression<Func<Resource, bool>> query = PredicateBuilder.True<Resource>();
foreach (Expression<Func<Resource, bool>> filter in queries)
{
query = query.And(filter);
}
return ctx.Resources.AsExpandable().Where(query);
}
public static Expression<Func<Resource, bool>> FilterResourcesByUserCriteriaEF(IEnumerable<string> filterValuesForUser)
{
// From the resource's filter values, check if there are any present in the user's filter values
return (x) => x.ResourceFilterValues.Any(y => filterValuesForUser.Contains(y.FilterValue.FilterValue1));
}
I'm still having issues with getting this working in my repository but that has something do with something blocking AsExpandable() from working properly.
I have been referring this post to group by using expression tree. Here is my code:
String[] fields = { "DepartmentID", "SkillID" };
var groupLambda = GroupByExpression<Person>(fields);
var query = dbContext.People.GroupBy(groupLambda.Compile());
var queryResult = query.ToList();
Here is the method GroupByExpression which uses solution given in aforesaid post (Thanks Daniel!):
public static Expression<Func<TItem, object>> GroupByExpression<TItem>(string[] propertyNames)
{
var properties = propertyNames.Select(name => typeof(TItem).GetProperty(name)).ToArray();
var propertyTypes = properties.Select(p => p.PropertyType).ToArray();
var tupleTypeDefinition = typeof(Tuple).Assembly.GetType("System.Tuple`" + properties.Length);
var tupleType = tupleTypeDefinition.MakeGenericType(propertyTypes);
var constructor = tupleType.GetConstructor(propertyTypes);
var param = Expression.Parameter(typeof(TItem), "x");
var body = Expression.New(constructor, properties.Select(p => Expression.Property(param, p)));
var expr = Expression.Lambda<Func<TItem, object>>(body, param);
return expr;
}
I want to be able to identify fields in the group by keys with strong names in select part like query.Select(x => new { x.Key.DepartmentID, x.Key.SkillID });
How do I do this?
Now... I won't give you the solution to the question you asked, but I'll try to help you :-)
If you want to do dynamic queries, you should probably use DynamicLinq
With DynamicLinq you can do things like:
IQueryable query = context.YourTable;
var groups = query.GroupBy("new (Field1, Field2)");
I'm rereading your question...
I want to be able to identify fields in the group by keys with strong names in select part like query.Select(x => new { x.Key.DepartmentID, x.Key.SkillID });
You can't. GroupBy in general will return a IGrouping<TKey, TSource>. TKey is dynamic (because you build it based on strings), so you can't "extract" it and pass it to the compiler, so you can't do the select with strong names.
There is a single exception: if you know the types and numbers of the GroupBy TKey then something can be done. So, you gave us:
String[] fields = { "DepartmentID", "SkillID" };
If you always have two int then you can cast your query with:
.Cast<IGrouping<Tuple<int, int>, Person>>()
.Select(x => new { x.Key.DepartmentID, x.Key.SkillID });
Note that, as I've written in a comment, your GroupBy will be executed client-side, and everything after the GroupBy will be executed client-side (where client-side == where your program is vs sql-side == where your sql server is)!
DynamicLinq will solve the problem of executing the query sql-side instead of client-side, but won't solve the problem of strong vs weak naming (after a DynamicLinq you can: A) use .Cast<>() method or B) return a dynamic object/IEnumerable<dynamic>)
The syntax you're using new { x.Key.DepartmentID, x.Key.SkillID } constructs an anonymous class at compile time. If you want to create an anonymous class at runtime, see here. However, that won't allow you to "identify fields in the group by keys with strong names". If you want to construct an anonymous class at runtime, but be able to use those names at compile time, I'm afraid that's impossible.
I have an existing function like this
public int sFunc(string sCol , int iId)
{
string sSqlQuery = " select " + sCol + " from TableName where ID = " + iId ;
// Executes query and returns value in column sCol
}
The table has four columns to store integer values and I am reading them separately using above function.
Now I am converting it to Entity Framework .
public int sFunc(string sCol , int iId)
{
return Convert.ToInt32(TableRepository.Entities.Where(x => x.ID == iId).Select(x => sCol ).FirstOrDefault());
}
but the above function returns an error
input string not in correct format
because it returns the column name itself.
I don't know how to solve this as I am very new to EF.
Any help would be appreciated
Thank you
Not going to be useful for the OP 8 years later, but this question has plenty of views, so I thought it could be helpful for others to have a proper answer.
If you use Entity Framework, you should do Linq projection (Select()), because that leads to the correct, efficient query on the db side, instead of pulling in the entire entity.
With Linq Select() you normally have to provide a lambda, though, so having your your column/property name in a string poses the main difficulty here.
The easiest solution is to use Dynamic LINQ (EntityFramework.DynamicLinq Nuget package). This package provides alternatives to the original Linq methods, which take strings as parameters, and it translates those strings into the appropriate expressions.
Example:
async Task<int> GetIntColumn(int entityId, string intColumnName)
{
return await TableRepository.Entities
.Where(x => x.Id == entityId)
.Select(intColumnName) // Dynamic Linq projection
.Cast<int>()
.SingleAsync();
}
I also made this into an async call, because these days all database calls should be executed asynchronously. When you call this method, you have to await it to get the result (i.e.: var res = await GetIntColumn(...);).
Generic variation
Probably it's more useful to change it into an extension method on IQueryable, and make the column/property type into a generic type parameter, so you could use it with any column/property:
(Provided you have a common interface for all your entities that specifies an Id property.)
public static async Task<TColumn> GetColumn<TEntity, TColumn>(this IQueryable<TEntity> queryable, int entityId, string columnName)
where TEntity : IEntity
{
return await queryable
.Where(x => x.Id == entityId)
.Select(columnName) // Dynamic Linq projection
.Cast<TColumn>()
.SingleAsync();
}
This is called like this: var result = await TableRepository.Entities.GetColumn<Entity, int>(id, columnName);
Generic variation that accepts a list of columns
You can extend it further to support selecting multiple columns dynamically:
public static async Task<dynamic> GetColumns<TEntity>(this IQueryable<TEntity> queryable, int entityId, params string[] columnNames)
where TEntity : IEntity
{
return await queryable
.Where(x => x.Id == entityId)
.Select($"new({string.Join(", ", columnNames)})")
.Cast<dynamic>()
.SingleAsync();
}
This is called like this: var result = await TableRepository.Entities.GetColumns(id, columnName1, columnName2, ...);.
Since the return type and its members are not known compile-time, we have to return dynamic here. Which makes it difficult to work with the result, but if all you want is to serialize it and send it back to the client, it's fine for that purpose.
This might help to solve your problem:
public int sFunc(string sCol, int iId)
{
var _tableRepository = TableRepository.Entities.Where(x => x.ID == iId).Select(e => e).FirstOrDefault();
if (_tableRepository == null) return 0;
var _value = _tableRepository.GetType().GetProperties().Where(a => a.Name == sCol).Select(p => p.GetValue(_tableRepository, null)).FirstOrDefault();
return _value != null ? Convert.ToInt32(_value.ToString()) : 0;
}
This method now work for dynamically input method parameter sCol.
Update:
This is not in context of current question but in general how we can select dynamic column using expression:
var parameter = Expression.Parameter(typeof(EntityTable));
var property = Expression.Property(parameter, "ColumnName");
//Replace string with type of ColumnName and entity table name.
var selector = Expression.Lambda<Func<EntityTable, string>>(property, parameter);
//Before using queryable you can include where clause with it. ToList can be avoided if need to build further query.
var result = queryable.Select(selector).ToList();
You have to try with dynamic LINQ. Details are HERE
Instead of passing the string column name as a parameter, try passing in a lambda expression, like:
sFunc(x => x.FirstColumnName, rowId);
sFunc(x => x.SecondColumnName, rowId);
...
This will in the end give you intellisense for column names, so you avoid possible errors when column name is mistyped.
More about this here: C# Pass Lambda Expression as Method Parameter
However, if you must keep the same method signature, i.e. to support other/legacy code, then you can try this:
public string sFunc(string sCol , int iId)
{
return TableRepository.Entities.Where(x => x.ID == iId).Select(x => (string) x.GetType().GetProperty(sCol).GetValue(x)});
}
You might need to adjust this a bit, I didn't have a quick way of testing this.
You can do this:
var entity = _dbContext.Find(YourEntity, entityKey);
// Finds column to select or update
PropertyInfo propertyInfo = entity.GetType().GetProperty("TheColumnVariable");
I'll need to expose a bit of background before going further: I have a project that uses nHibernate and some generic repository to forward predicates and return entities as follows:
public abstract class GenericRepository< T >
{
...
public virtual T Single( Expression< Func< T, bool > > predicates )
{
// Get object from nHibernate session object
var retObj = Session
.Query< T >()
.Where( predicates ).SingleOrDefault();
return retObj;
}
...
}
I can then for example get an entity this way:
var entity = Context.MyGenericEntityRepository.Single( e => e.Id == id );
// or
var entity = Context.MyGenericEntityRepository.Single( e => e.Name == name );
But, I also have some entities that because of the nature of the project aren't saved to the database but to the file system (bunch of files). So I use a derived repository that uses some sort of DataAccess class to get the entities from the file system as follows:
public class NotGenericRepository
{
...
// for example
public IList<Entity> All()
{
return _entityDataAccess.All();
}
...
}
As said, the 2nd type of entity isn't stored in the database, however, to facilitate my journey I created a sort of in-memory database system using DataSets and DataTables.
So when the solution is first started, I have a singleton called CustomDatabase that gets initialised, creates the DataTables in memory, adds relations between the DataTalbes, and adds them to a general DataSet before scanning the file system to populate the tables.
Using this, I can now query my DataTables instead of scanning through the file system tree every time. I then set up some events in my CustomDatabase so whenever a row is added/deleted/updated, the changes are reflected onto the file system.
So... This is it for the background, sorry for the length...
My question is now fairly simple, I'm looking for a way to somehow translate the lambda expression forwarded by the repository to my DataAccess class, so I can then analyse it and select from my DataTables accordingly...
For example, a call to the repository such as:
var entity = Context.MyNotGenericEntityRepository.Single( e => e.Id == id );
// or
var entity = Context.MyNotGenericEntityRepository.Single( e => e.Name == name );
Should be translated within the DataAccess as:
DataRow entityRow = CustomDatabase.Tables[ "EntityName" ].AsEnumerable().Where( e => e.Field< Guid >( "Id" ) == id);
// or
DataRow entityRow = CustomDatabase.Tables[ "EntityName" ].AsEnumerable().Where( e => e.Field< string >( "Name" ) == name);
Or:
var entity = from myRow in CustomDatabase.Tables[ "EntityName" ].AsEnumerable()
where myRow.Field<Guid>( "Id" ) == id
select myRow;
// or
var entity = from myRow in CustomDatabase.Tables[ "EntityName" ].AsEnumerable()
where myRow.Field<string>( "Name" ) == name
select myRow;
I have absolutely no idea how to do this, I've been looking all over the net, but the problem is that I don't really know how to name this problem so I haven't found much so far... :(
Any help is appreciated since I'm expecting there'll be several ways to tackle this problem :)
Thanks!!
It's difficult to tell what's been skipped in the derived repository class example, but it could be as simple as overriding (or overloading) Single to return All().Single(..).
Assuming your predicate expressions are of the form in your question, you can convert the expression to one for a DataRow, and then use that to filter the table corresponding to the entity:
public static DataRow GetSingleRow<T>(Expression<Func<T, bool>> expr)
{
var bodyExpr = ((LambdaExpression)expr).Body;
var binExpr = (BinaryExpression)bodyExpr;
var propExpr = (MemberExpression)binExpr.Left;
PropertyInfo targetProperty = (PropertyInfo)propExpr.Member;
Type targetType = targetProperty.DeclaringType;
var valueExpr = binExpr.Right;
string entityName = targetType.Name;
var fieldMethod = typeof(DataRowExtensions).GetMethod("Field", new[] { typeof(DataRow), typeof(string) });
var methodInfo = fieldMethod.MakeGenericMethod(targetProperty.PropertyType);
var propNameExpr = Expression.Constant(targetProperty.Name);
var rowParamExpr = Expression.Parameter(typeof(DataRow), "dr");
var fieldInvocationExpr = Expression.Call(methodInfo, rowParamExpr, propNameExpr);
var eqExpr = Expression.Equal(fieldInvocationExpr, valueExpr);
Func<DataRow, bool> predicate = Expression.Lambda<Func<DataRow, bool>>(eqExpr, rowParamExpr).Compile();
return CustomDatabase.Tables[entityName].AsEnumerable().Single(predicate);
}