I am trying to build a dynamic query in linq to SQL. In short, the problem I am trying to solve is as follows:
We have an existing SQL Server database with a number of tables that we want to be able to search in generically. This is a legacy database, so we are using Entity framework data-first (not code first).
In our database, we also have a metadata table that holds the table and column names of the other tables, and which columns are searchable.
For example, we might have a table named 'Vehicles' with a PK column called 'Vehicle_Id', and searchable columns named 'Type' and 'Colour'. Our metadata table would contain entries for the columns 'Type' and 'Colour' marking them as searchable.
The EF model and mapping for the Vehicle entity would look something like this:
public class Vehicle
{
public int Id { get; set; }
public string Type { get; set; }
public string Colour { get; set; }
}
public class VehicleMap : EntityTypeConfiguration<Vehicle>
{
public VehicleMap()
{
This.ToTable("VEHICLES");
This.HasKey(k => k.Id);
This.Property(p => p.Id).HasColumnName("VEHICLE_ID");
This.Property(p => p.Type).HasColumnName("TYPE");
This.Property(p => p.Colour).HasColumnName("COLOUR");
}
}
A user then defines the search requirements (for instance, 'Type' of 'Car' or 'Colour' of 'Red' or 'Blue'). These search requirements are based on the column names.
I would like to be able to dynamically build the query, based on the field names defined in our metadata table, something like this:
var query = from v in context.Vehicles select v.Id;
foreach (var criterion in criteria)
{
var fieldName = criterion.Field; // Database field name, e.g. "TYPE"
var value = criterion.Value; // Search value, e.g. "CAR"
var filter = [Filter based on field/value]
results = query.Where(filter).ToList();
// do something with results...
}
At the moment, I think I am about half-way there with the following extension method which is getting the entity mapping for a given table from the EF context. What I'm getting lost with is then finding the field based on field name (e.g. 'COLOUR') and building the filter based on this.
public static EntityType GetTableMapping(this DbContext context, string tableName)
{
var objectContext = ((IObjectContextAdapter)context).ObjectContext;
var storageMetadata = ((EntityConnection)objectContext.Connection).GetMetadataWorkspace().GetItems(DataSpace.SSpace);
var entityProps = storageMetadata.Where(s => s.BuiltInTypeKind == BuiltInTypeKind.EntityType).Select(s => s as EntityType);
return (from m in entityProps where string.Compare(m.Name, tableName, StringComparison.OrdinalIgnoreCase) == 0 select m).SingleOrDefault();
}
Related
I'm writing a ASP.NET Core Web API project. As a data source It will be using existing (and pretty big) database. But not entire database. The API will use only some of the tables and even in these tables it will not use all the columns.
Using Reverse engineering and scaffolding I was able to generate DbContext and Entity classes... and it got me thinking. There is a table with 30 columns (or more). I'm using this table, but I only need 5 columns.
My question is:
Is there any advantage of removing 25 unused columns from C# entity object? Does it really matter?
The advantage of leaving them there unused is that in case of someone wants to add new functionality that will need one of them, he will not need to go to the db and reverse engineer needed columns (there are there already).
The advantage of removing unused is... ?
EDIT: Here is the sample code:
public class FooContext : DbContext
{
public FooContext(DbContextOptions<FooContext> options)
: base(options)
{
}
public DbSet<Item> Items { get; set; }
}
[Table("item")]
public class Item
{
[Key]
[Column("itemID", TypeName = "int")]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
[Column("name", TypeName = "varchar(255)")]
public string Name { get; set; }
}
Sample usage:
public ItemDto GetItem(int id)
{
var item = _fooContext.Items.Where(i => i.Id == id).FirstOrDefault();
// Here I have item with two fields: Id and Name.
var itemDto = _mapper.Map<ItemDto>(item);
return itemDto;
}
Obviously I'm curious about more complex operations. Like... when item entity is being included by other entity. For example:
_foo.Warehouse.Include(i => i.Items)
or other more complex functions on Item entity
Your entity needs to match what's in the database, i.e. you need a property to match each column (neglecting any shadow properties). There's no choice here, as EF will complain otherwise.
However, when you actually query, you can select only the columns you actually need via something like:
var foos = await _context.Foos
.Select(x => new
{
Bar = x.Bar,
Baz = z.Baz
})
.ToListAsync();
Alternatively, if you don't need to be able to insert/update the table, you can instead opt to use DbQuery<T> instead of DbSet<T>. With DbQuery<T>, you can use anything class you want, and project the values however you like, via FromSql.
I'm trying to perform a simple query and the result data is almost all null.
I have this table structure
Table Registros
ID | Autonumeric
TareaM_Id | Numeric
Fecha | Date/Time
and Macro_tareas table
ID | Autonumeric
Nombre | Short Text
I have mapped the classes in C# like this:
[Table("Registros")]
public class Registro
{
[Column("ID")]
public virtual int ID { get; set; }
[Column("Fecha")]
public virtual DateTime Fecha { get; set; }
[Column("TareaM_Id")]
public virtual int TareaM_Id { get; set; }
public virtual MacroTarea MacroT { get; set; }
}
[Table("Macro_tarea")]
public class MacroTarea
{
[Column("ID")]
public virtual int ID { get; set; }
[Column("Nombre")]
public virtual string Nombre{ get; set; }
public virtual ICollection<Registro> Registros { get; set; }
}
This is the query i'm trying to use
string sql = #"SELECT reg.ID, mac.ID
FROM Registros as reg INNER JOIN Macro_tarea as mac on reg.TareaM_Id = mac.ID
WHERE Fecha = #Fecha";
using (IDbConnection db = new OleDbConnection(ConnectionString))
{
var result = db.Query<Registro,MacroTarea, Registro>(sql,
(reg,mac) =>
{
reg.MacroTarea = mac;
return reg;
}
,new { #Fecha = new DateTime(2019, 1, 4).Date }
, splitOn: "mac.ID")
.AsList();
}
I'm trying to only retrieve ids, but both id become null why is this happening?
The thing is, if I add Registros.Fecha and Macro_tarea.Nombre to the query, it got the value correctly. But id keep coming null.
Apparently the issue is happening only with ids. I suspect this issue is due to duplicate column names.
I'm working with Microsoft Access just in cast that matters.
My question is not similar to the possible duplicate because I have the classes defined as they should be mapped.
Renaming your database columns because your code cannot cope with the data is not a good idea. In the world of separation of concerns, why should your database care? There are good database reasons to name ID columns "Id", and you may not even have the option to change them.
There's another issue with Dapper mapping that renaming columns does not get around; repeated types. If you are trying to map to more than one instance of a class Dapper gets confused, and renaming columns won't work because you will rename both instances.
Here is the solution I have come up with. It's similar to a lot of examples that use a dictionary, except:
it can nest to as many levels as you like
can cope with Dappers 7 item limit
can cope with duplicates of the same class
can be reused e.g., for Get, GetCurrent and GetAll
In this example there is an Auction that has many Lots. Each Lot may have 1 or many Items. Items might be packs of Items. The Items are from a limited catalogue and we like relational data, so a Things table contains the details on each Item, like colour, size, etc. Here we are only getting a single Lot, but getting an Auction is the same with another level on top for Auction.
Parameter 1 - The SQL to get everything in one go
Parameter 2 - A Type array of each object we'll get back. For this reason it's best to order your SELECT to group the fields into the classes
Parameter 3 - Call the method we're about to write with the SQL result
Parameter 4 - Standard parameter array for the SQL. SQL Injection is bad, m'kay?
public async Task<List<Lot>> GetAll(int auctionId)
{
using (var connection = new SqlConnection(_appSettings.ConnectionString))
{
await connection.OpenAsync();
var result = new List<Lot>();
await connection.QueryAsync($#"
SELECT [Lot].*,
[Item].[Id],
[Item].[LotId],
[Item].[Notes],
itemDetails.[Id],
itemDetails.[ThingId],
itemDetails.[Colour],
itemDetails.[Size],
[SubItem].[Id],
[SubItem].[ItemId],
[SubItem].[Notes],
subItemDetails.[Id],
subItemDetails.[ThinId],
subItemDetails.[Colour],
subItemDetails.[Size]
FROM [Lot]
INNER JOIN [Item] ON [Item].[LotId] = [Lot].[Id]
LEFT JOIN [Thing] AS itemDetails ON itemDetails.[Id] = [Item].[ThingId]
LEFT JOIN [SubItem] ON [SubItem].[ItemId] = [Item].[Id]
LEFT JOIN [Thing] AS subItemDetails ON subItemDetails.[Id] = [SubItem].[ThingId]
WHERE [AuctionId] = #{nameof(auctionId)}
ORDER BY [Lot].[Id], [Item].[Id], [Expansion].[Id];",
new Type[] {
typeof(Lot),
typeof(Item),
typeof(Thing),
typeof(Expansion),
typeof(Thing)
},
MapResult(result),
new
{
AuctionId = auctionId
}
);
return result.ToList();
}
}
private Func<object[], Lot> MapResult(List<Lot> result)
{
return (obj) =>
{
Lot lot = (Lot)obj[0];
Item item = (Item)obj[1];
Thing itemDetails = (Thing)obj[2];
SubItem subItem = (SubItem)obj[3];
Thing subItemDetails = (Thing)obj[4];
if (lot != null)
{
if (result.Any(a => a.Id == lot.Id))
{
lot = result.First(a => a.Id == lot.Id);
}
else
{
result.Add(lot);
}
}
if (item != null)
{
if (lot.Items.Any(i => i.Id == item.Id))
{
item = lot.Items.First(i => i.Id == item.Id);
}
else
{
lot.Items.Add(item.FromThing(itemDetails));
}
}
if (expansion != null)
{
if (item.SubItems.Any(e => e.Id == subItem.Id) == false)
{
item.SubItems.Add(subItem.FromThing(subItemDetails));
}
}
return null;
};
}
MapResult is the meat of the code. It returns a Func with two types, the Type array we defined above and the return Type, and takes a List of the top level object.
I then map each item from the object array to another of it's actual type. This keeps the code easier to read, and enables properties and methods of the object to be accessed without issue.
Then it's a case of stepping down the hierarchy, checking at each step if one already exists with a matching id, and swapping the iterator to a reference to it if it does. This means that following code will add to the existing item.
In the particular case I've also added a FromThing function to allow easier combining of object properties.
As we discussed in comments, this is an issue due to duplicate column names in two tables. This is where the similar issue and solution could be found. But, it does not include "mapping by code" as you said. So it is not exact duplicate.
I suggest you change the names of ID fields in your tables to avoid colliding them. Of-course, you should also change the name of your POCO properties and mappings accordingly.
If you cannot change the column names in table, change the POCO property name, and use the column alias in SQL query to match those new property names.
I hope this helps you.
The problem was effectively the name of the properties.
I solved it using Custom Column Mapping to do it i got two possible solutions:
Without extensions
First, we define a Dictionary with the name of the column as key, and the name of the property as value
IDictionary<string, string> columnMaps = new Dictionary<string, string>()
{
{ "Macro_tarea.ID", "ID" },
{ "Registros.ID", "ID" }
};
Then, we define a delegate to obtain the PropertyInfo object of the property to which we intend to assign the alias of the previous dictionary
var mapper = new Func<Type, string, PropertyInfo>((type, columnName) =>
{
if (columnMaps.ContainsKey(columnName))
return type.GetProperty(columnMaps[columnName]);
else
return type.GetProperty(columnName);
});
Now, we define an object that implements the ITypeMap interface using CustomPropertyTypeMap implementation
ITypeMap MacroTareaMapper = new CustomPropertyTypeMap(typeof(Macro_tarea),
(type, columnName) => mapper(type, columnName));
ITypeMap RegistrosMapper = new CustomPropertyTypeMap(typeof(Registros),
(type, columnName) => mapper(type, columnName));
Then we register them
SqlMapper.SetTypeMap(typeof(Macro_tarea), MacroTareaMapper);
SqlMapper.SetTypeMap(typeof(Registros), RegistrosMapper);
Simpler solution with Dapper.FluentMap
It is implemented as follows:
We create a class that inherits from EntityMap<T> and using the Map method we define which column corresponds to each property. For example,
internal class Macro_tareaMap : EntityMap<Macro_tarea>
{
internal Macro_tareaMap()
{
//Mi propiedad ID esta asociada a la columna Macro_tarea.ID
Map(x => x.ID).ToColumn("Macro_tarea.ID");
}
}
Then just register it
FluentMapper.Initialize((config) =>
{
config.AddMap(new Macro_tareaMap());
});
Hope it helps another people!
Source: https://medium.com/dapper-net/custom-columns-mapping-1cd45dfd51d6
I have a DbContext class and I'm using code first apporach in my application. I have few common standard tables that contains "Id" and "Value" columns and i want to query them passing the table name and column name but there is no option in entity framework to pass.
Example:
Common tables:
Client_ClientDepartment (Id, Value)
Client_ClientDesignation (Id, Value)
Client_ClientCompany (Id, Value)
What I want to do is to pass table name and Id to get the value. I have created a common method as
public string GetClientValue(string id, string tableName)
{
DatabaseContext dbContext = new DatabaseContext();
//Query the database and get value based on table and id.
string value = dbContent. ("query here")
return value ;
}
Can I do it in entity framework? Is it possible?
using ( DatabaseContext dbContext = new DatabaseContext())
{
var blogs = dbContext.Database.SqlQuery<>("query here").ToList();
}
I believe you can run a custom query like this
using (var context = new BloggingContext())
{
var blogNames = context.Database.SqlQuery<string>(
"SELECT Name FROM dbo.Blogs").ToList();
}
Source: https://msdn.microsoft.com/en-us/library/jj592907(v=vs.113).aspx
Sorry I had to answer instead of comment, but don't got the badge yet.
Actually, you normally don't pass table and column names in EF.
You have classes and properties, which become tables and columns in the resulting database.
Your context should look something like this:
public class DatabaseContext : DbContext
{
public DatabaseContext(): base(YourWebConfigConnectionStringName){}
public DbSet<Client_ClientDepartment> ClientDepartment { get; set; }
public DbSet<Client_ClientDesignation> ClientDesignation { get; set; }
With this you are basically registering your "table" classes.
You then address them in code like this:
using (var context=new DatabaseContext())
{
var department = context.ClientDepartment.First(d => d.Id == someIdVariable);
Which is the analogy to a SQL query SELECT TOP 1 department WHERE ID=someId
You can also pass SQL statements as described in the other answers, but that too will only work if you properly registered your classes as DBSets in your DatabaseContext class.
P.S: I left out the Database initializer in the DBContext class, which is something you also need in code first.
Before I get started, note that I've simplified the data structure for what I'm trying to do and in the real world it isn't as awful of an implementation as you might think. Or maybe it is. Regardless, I can't change the way the data is structured so I'm not looking for suggestions on how to better structure the data. I'm just hoping to see if there is a way I can do what I'm asking in Entity Framework 6.
Consider I have the following tables:
Person:
ID FirstName LastName
1 John Smith
2 Jane Doe
Job:
ID Name
1 Developer
ExtendedData:
ID TableName RowID FieldName FieldValue
1 Person 1 MiddleInitial A
2 Person 1 Gender M
3 Person 2 MiddleInitial B
4 Person 2 Gender F
5 Job 1 Description Develop Stuff
The purpose of this ExtendedData table is to allow for additional data to be stored when there isn't a column for the data.
For example, here "MiddleInitial" is not a column in the Person table, but in the ExtendedData table we can add a row to store that data.
In my "Person" class I can add the following code to add an ExtendedData property:
public virtual ICollection<ExtendedData> ExtendedData { get; set; }
Then I can create a relationship in Entity Framework with this:
modelBuilder.Entity<Person>()
.HasMany(e => e.ExtendedData)
.WithRequired(e => e.Person)
.HasForeignKey(e => e.RowID);
The concern I have, is if I call...
john = Persons.Where(a => a.ID == 1);
john.ExtendedData...
... I'll get back all Extended Data rows where RowID = 1, including the row for the "Job" table.
Obviously, I could do something like...
john.ExtendedData.Where(a => a.TableName == "Person")...
... but this is a little dangerous because what if I (or some other developer) forget to specify that extra filter in the code?
I tried doing something like this...
modelBuilder.Entity<Person>()
.HasMany(e => (ICollection<ExtendedData>))e.ExtendedData.Where(a => a.TableName == "Person"))
.WithRequired(e => e.Person)
.HasForeignKey(e => e.RowID);
... but received an error at run time stating...
The expression 'e => Convert(e.ExtendedData.Where(a => (a.TableName ==
"Person")))' is not a valid property expression. The expression should
represent a property: C#: 't => t.MyProperty' VB.Net: 'Function(t)
t.MyProperty'.
The sense I make of it is it wants me to specify a property from "e" and not try to do any further wizardry.
Is there anywhere I can modify the Entity Framework model such that when I call
person.ExtendedData it will only return to me ExtendedData records where the TableName = "Person"? Or must I remember to always include that extra filter when trying to pull data from the ExtendedData table?
What you really want here is Table Per Hierarchy. Define a base entity for your table, then define an inherited entity for each variation, and customize the discriminator.
Public abstract class ExtendedData
{
public int Id {get;set;}
public string FieldName {get;set;}
public string FieldValue {get;set;}
}
public class ExtendedPersonData : ExtendedData
{}
public class ExtendedJobData : ExtendedData
{}
public class Person
{
....
public virtual ICollection<ExtendedPersonData> ExtendedData {get;set}
}
public class Job
{
....
public virtual ICollection<ExtendedJobData> ExtendedData {get;set;}
}
public class Context : DbContext
{
....
public DbSet<ExtendedData> ExtendedData {get;set;}
}
modelBuilder.Entity<ExtendedData>()
.Map<ExtendedPersonData>(m => m.Requires("TableName").HasValue("Person"))
.Map<ExtendedJobData>(m => m.Requires("TableName").HasValue("Job"));
with the inherited classes, you can now do Non-Polymorphic Queries against the table data, i.e.
IQueryable<ExtendedPersonData> query = from e in context.ExtendedData
.OfType<ExtendedPersonData>()
select e;
This will generate a SQL query which only returns records where the discriminator column matches ("TableName" == "Person" in this case). Returning a Person and querying it's ExtendedData would automatically create a Non-Polymorphic Query, due to the collection type defined on the entity.
Note that in this scenario, you can mix shared columns and columns unique to each variation. Any columns specific to a unique variation would automatically become nullable in the table, and would only be filled by the entities that provide it. In your case, however, you have no unique columns, so the Inherited classes are just implementation stubs.
http://weblogs.asp.net/manavi/inheritance-mapping-strategies-with-entity-framework-code-first-ctp5-part-1-table-per-hierarchy-tph
Can I make my EF objects retrieve only specific columns in the sql executed?
If I have a column that contains a large amount of data that really slows down the query, how can I have my objects exclude that column from the sql generated?
If my table has Id(int), Name(int), Data(blob), how can I make my query be
select Id, Name from TableName
instead of
select Id, Name, Data from TableName
From the suggestion below, my method is
public List<T> GetBy<T>(DbContext context,Expression<Func<T, bool>> exp, Expression<Func<T,T>> columns) where T : class
{
return dbContext.Set<T>().Where(exp).Select<T,T>(columns).ToList();
}
And I'm calling it like so
List<CampaignWorkType> list = GetBy<CampaignWorkType>(dbContext, c => c.Active == true, n => new { n.Id, n.Name });
i got an error like below.
Cannot implicitly convert type 'AnonymousType#1' to 'Domain.Campaign.CampaignWorkType'
how i can solve this?
The solution is:
First, define a surrogate type:
public class CampaignWorkTypesSimpleList
{
public int Id { get; set; }
public string Name { get; set; }
}
Then change generic method like this:
public List<U> GetBy<T,U>(DbContext context,Expression<Func<T, bool>> exp, Expression<Func<T,U>> columns)
where T : class
where U : class
{
return dbContext.Set<T>().Where(exp).Select<T, U>(columns).ToList();
}
Finally, execute it.
List<CampaignWorkTypesSimpleList> list = this.GetBy<CampaignWorkType, CampaignWorkTypesSimpleList>(dbContext, c => c.Active == true, n => new CampaignWorkTypesSimpleList { Id = n.Id, Name = n.Name });
Assuming you are using the EF designer, remove the [Data] column from the Entity Model using the Entity Model design surface.