I have an application that is using MVC framework that currently reads from a model object of a SQL table and puts the results in a list format. The table has its own .cs class with all the necessary fields and the code filters the results based on other factors.
The problem I am running into, is that I need to find a way to add new tables to this list without making changes to the code itself. Ideally, I would like to add a list of tables I need to read from into the web.config and create a class for them in the project structure and the code will dynamically read from that.
Currently with one table the code looks like this:
var assets = _model.FMIF.ToList();
var results = BuildFormatedResult(assets);
And I have been able to add another table to this structure like this:
var assets = _model.FMIF.ToList();
var assets2 = _model.FMHZ.ToList();
assets = assets.Concat(assets2).ToList();
var results = BuildFormatedResult(assets);
I have tried to make this dynamic using a method like this:
var test = "FMIF";
var assets = _model.+test+.ToList();
var results = BuildFormatedResult(assets);
but it does not appear to be able to read the variable as a model.
Is there a best practice way to do something like this? Even if it is way different than what I have tried I am kind of at a loss here. Not super familiar with MVC structure so any help or advice is greatly appreciated.
Thanks!
This is an approach I've used to dynamically get a handle to a DbSet. Firstly I define a couple of methods (This does assume that you have the classes already generated for each of the tables.)
public static DbSet GetDbSet(MyDbContext db, string tableName)
{
// Find the EF entity that corresponds to this table
ObjectContext objectContext = (db as IObjectContextAdapter).ObjectContext;
var mappings = GetEntityMappings(objectContext);
var entityName = mappings[tableName];
// Now get the corresponding DbSet
DbSet dbSet = (DbSet)db.Set(Type.GetType("schema_name." + entityName));
return dbSet;
}
public static Dictionary<string, string> GetEntityMappings(ObjectContext objectContext)
{
var EntityMappings = new Dictionary<string, string>();
// Build a list of database table names to EF entity names
// Get a list of entities
MetadataWorkspace workspace = objectContext.MetadataWorkspace;
var entities = workspace.GetItems<EntityType>(DataSpace.CSpace);
foreach (EntityType et in entities)
{
// Get the entity set that uses this entity type
var entitySet = workspace
.GetItems<EntityContainer>(DataSpace.CSpace)
.Single()
.EntitySets
.Single(s => s.ElementType.Name == et.Name);
// Find the mapping between conceptual and storage model for this entity set
var mapping = workspace.GetItems<EntityContainerMapping>(DataSpace.CSSpace)
.Single()
.EntitySetMappings
.Single(s => s.EntitySet == entitySet);
// Find the storage entity set (table) that the entity is mapped to
var table = mapping
.EntityTypeMappings.Single()
.Fragments.Single()
.StoreEntitySet;
string tableName = (string)table.MetadataProperties["Table"].Value ?? table.Name;
EntityMappings.Add(tableName, et.Name);
}
return EntityMappings;
}
Basically it is poking around in the EntityFramework metadata in order to map a string of a table name to the entity name (which aren't necessarily the same). This is slightly simplified as ideally you would cache the mappings.
Then finally call the GetDbSet() method, I've used AsQueryable() as I potentially need to add where clauses etc.
var dbSet = GetDbSet(db, "table_name").AsQueryable();
in my database i have many of tables one of Tables is nambed Company". and its columns names are as follows ("ComName","ComAddress","ComType" ..etc)
I have a function which get the columns name from Company table :
public List<string> GetColumnNames(string tablename)
{
var names = typeof(Company).GetProperties().Select(p => p.Name).ToArray();
return names.ToList();
}
Until now everything is good but what if i want to get the columns of other tables ? so i have to put the name of the new table (that is given in the function as "tablename") instead of "Company" in this sentence :
var names = typeof(Company).GetProperties().Select(p => p.Name).ToArray();
but it is string so its not working. any idea ?
typeof(Company).GetProperties() isn't fetching the names of the columns of your table at all though - it's fetching the names of the properties of the C# class/entity that you're using to represent that table. Do you already have a process to guarantee you have C# entities with property names that exactly match the column names of your tables? If so, you can do the same principle (but instead of using typeof(Company) you'd need to find the type by name, which there are various ways of doing). If not, you'll need to to actually query the database to find the column names.
As far as i know, you can use the Type parameter.
public List<string> GetColumnNames(Type tablename)
{
var names = tablename.GetProperties().Select(p => p.Name).ToArray();
return names.ToList();
}
How can I get the type of each column in a SQL Server table or view using Entity Framework?
I need to do this BEFORE I get all the data from the table, because I want to allow users to filter, for example, from a date before the data is actually retrieved from the SQL Server table/view.
So if I had a table of Books with a publish date, I would like to return each column (Name, Publisher, Publish Date) type in order to allow filtering beforehand. I need this code to do this because I will not necessarily know the columns, since the user may use several different tables.
The .NET Framework type is fine, I don't need the SQL Server type...
Here's some example code:
using (var ArgoEntities = new ARGOEntities())
{
//find the types here before the user performs the query so i can build the below code
var query = from b in ArgoEntities.tbl_Books
where b.PublishDate>[user specified date] //some date here the user enters
select b;
var book = query.First();
}
EDIT: I can do this so far only by getting the first record in the table, like this...
using (ARGOEntities ArgoEntities = new ARGOEntities())
{
//var fred = typeof(ARGOEntities).GetProperties();
//var fred = ArgoEntities.GetType().GetProperties();
var a=ArgoEntities.tbl_Books.FirstOrDefault();
var b = ObjectContext.GetObjectType(a.GetType());
var c=b.GetProperties();
}
but i repeat, I DON'T want to get any records first.
you can use the GetProperty and then PropertyType:
using (var ArgoEntities = new ARGOEntities())
{
//find the types here before the user performs the query so i can build the below code
//Like this you can retrieve the types:
foreach (string propertyName in ArgoEntities.CurrentValues.PropertyNames)
{
var propertyInfo = ArgoEntities.Entity.GetType().GetProperty(propertyName);
var propertyType = propertyInfo.PropertyType;
}
//
var query = from b in ArgoEntities.tbl_Books
where b.PublishDate>[user specified date] //some date here the user enters
select b;
var book = query.First();
}
Ryios' comment lead me to the extremely simple answer I knew it had to be, which will give me an array of PropertyInfo for each field in the table.
var columns=typeof(tbl_Books).GetProperties();
I'm using entity framework to connect with the database. I've one little problem:
I've one table which have one varbinary(MAX) column(with filestream).
I'm using SQL request to manage the "Data" part, but EF for the rest(metadata of the file).
I've one code which has to get all files id, filename, guid, modification date, ... of a file. This doesn't need at all the "Data" field.
Is there a way to retrieve a List but without this column filled?
Something like
context.Files.Where(f=>f.xyz).Exclude(f=>f.Data).ToList();
??
I know I can create anonymous objects, but I need to transmit the result to a method, so no anonymous methods. And I don't want to put this in a list of anonymous type, and then create a list of my non-anonymous type(File).
The goal is to avoid this:
using(RsSolutionsEntities context = new RsSolutionsEntities())
{
var file = context.Files
.Where(f => f.Id == idFile)
.Select(f => new {
f.Id, f.MimeType, f.Size, f.FileName, f.DataType,
f.DateModification, f.FileId
}).FirstOrDefault();
return new File() {
DataType = file.DataType, DateModification = file.DateModification,
FileId = file.FileId, FileName = file.FileName, Id = file.Id,
MimeType = file.MimeType, Size = file.Size
};
}
(I'm using here the anonymous type because otherwise you will get a NotSupportedException: The entity or complex type 'ProjectName.File' cannot be constructed in a LINQ to Entities query.)
(e.g. this code throw the previous exception:
File file2 = context.Files.Where(f => f.Id == idFile)
.Select(f => new File() {Id = f.Id, DataType = f.DataType}).FirstOrDefault();
and "File" is the type I get with a context.Files.ToList(). This is the good class:
using File = MyProjectNamespace.Common.Data.DataModel.File;
File is a known class of my EF datacontext:
public ObjectSet<File> Files
{
get { return _files ?? (_files = CreateObjectSet<File>("Files")); }
}
private ObjectSet<File> _files;
Is there a way to retrieve a List but without this column filled?
Not without projection which you want to avoid. If the column is mapped it is natural part of your entity. Entity without this column is not complete - it is different data set = projection.
I'm using here the anonymous type because otherwise you will get a
NotSupportedException: The entity or complex type 'ProjectName.File'
cannot be constructed in a LINQ to Entities query.
As exception says you cannot project to mapped entity. I mentioned reason above - projection make different data set and EF don't like "partial entities".
Error 16 Error 3023: Problem in mapping fragments starting at line
2717:Column Files.Data in table Files must be mapped: It has no
default value and is not nullable.
It is not enough to delete property from designer. You must open EDMX as XML and delete column from SSDL as well which will make your model very fragile (each update from database will put your column back). If you don't want to map the column you should use database view without the column and map the view instead of the table but you will not be able to insert data.
As a workaround to all your problems use table splitting and separate the problematic binary column to another entity with 1 : 1 relation to your main File entity.
I'd do something like this:
var result = from thing in dbContext.Things
select new Thing {
PropertyA = thing.PropertyA,
Another = thing.Another
// and so on, skipping the VarBinary(MAX) property
};
Where Thing is your entity that EF knows how to materialize. The resulting SQL statement shouldn't include the large column in its result set, since it's not needed in the query.
EDIT: From your edits, you get the error NotSupportedException: The entity or complex type 'ProjectName.File' cannot be constructed in a LINQ to Entities query. because you haven't mapped that class as an entity. You can't include objects in LINQ to Entities queries that EF doesn't know about and expect it to generate appropriate SQL statements.
You can map another type that excludes the VarBinary(MAX) column in its definition or use the code above.
you can do this:
var files = dbContext.Database.SqlQuery<File>("select FileId, DataType, MimeType from Files");
or this:
var files = objectContext.ExecuteStoreQuery<File>("select FileId, DataType, MimeType from Files");
depending on your version of EF
I had this requirement because I have a Document entity which has a Content field with the content of the file, i.e. some 100MB in size, and I have a Search function that I wanted to return the rest of the columns.
I chose to use projection:
IQueryable<Document> results = dbContext.Documents.Include(o => o.UploadedBy).Select(o => new {
Content = (string)null,
ContentType = o.ContentType,
DocumentTypeId = o.DocumentTypeId,
FileName = o.FileName,
Id = o.Id,
// etc. even with related entities here like:
UploadedBy = o.UploadedBy
});
Then my WebApi controller passes this results object to a common Pagination function, which applies a .Skip, .Take and a .ToList.
This means that when the query gets executed, it doesn't access the Content column, so the 100MB data is not being touched, and the query is as fast as you'd want/expect it to be.
Next, I cast it back to my DTO class, which in this case is pretty much exactly the same as the entity class, so this might not be a step you need to implement, but it's follows my typical WebApi coding pattern, so:
var dtos = paginated.Select(o => new DocumentDTO
{
Content = o.Content,
ContentType = o.ContentType,
DocumentTypeId = o.DocumentTypeId,
FileName = o.FileName,
Id = o.Id,
UploadedBy = o.UploadedBy == null ? null : ModelFactory.Create(o.UploadedBy)
});
Then I return the DTO list:
return Ok(dtos);
So it uses projection, which might not fit the original poster's requirements, but if you're using DTO classes, you're converting anyway. You could just as easily do the following to return them as your actual entities:
var dtos = paginated.Select(o => new Document
{
Content = o.Content,
ContentType = o.ContentType,
DocumentTypeId = o.DocumentTypeId,
//...
Just a few extra steps but this is working nicely for me.
For EF Core 2
I implemented a solution like this:
var files = context.Files.AsNoTracking()
.IgnoreProperty(f => f.Report)
.ToList();
The base idea is to turn for example this query:
SELECT [f].[Id], [f].[Report], [f].[CreationDate]
FROM [File] AS [f]
into this:
SELECT [f].[Id], '' as [Report], [f].[CreationDate]
FROM [File] AS [f]
you can see the full source code in here:
https://github.com/aspnet/EntityFrameworkCore/issues/1387#issuecomment-495630292
I'd like to share my attempts to workaround this problem in case somebody else is in the same situation.
I started with what Jeremy Danyow suggested, which to me is the less painful option.
// You need to include all fields in the query, just make null the ones you don't want.
var results = context.Database.SqlQuery<myEntity>("SELECT Field1, Field2, Field3, HugeField4 = NULL, Field5 FROM TableName");
In my case, I needed a IQueryable<> result object so I added AsQueryable() at the end. This of course let me add calls to .Where, .Take, and the other commands we all know, and they worked fine. But there's a caveat:
The normal code (basically context.myEntity.AsQueryable()) returned a System.Data.Entity.DbSet<Data.DataModel.myEntity>, while this approach returned System.Linq.EnumerableQuery<Data.DataModel.myEntity>.
Apparently this means that my custom query gets executed "as is" as soon as needed and the filtering I added later is done afterwards and not in the database.
Therefore I tried to mimic Entity Framework's object by using the exact query EF creates, even with those [Extent1] aliases, but it didn't work. When analyzing the resulting object, its query ended like
FROM [dbo].[TableName] AS [Extent1].Where(c => ...
instead of the expected
FROM [dbo].[TableName] AS [Extent1] WHERE ([Extent1]...
Anyway, this works, and as long as the table is not huge, this method will be fast enough. Otherwise you have no option than to manually add the conditions by concatenating strings, like classic dynamic SQL. A very basic example in case you don't know what I'm talking about:
string query = "SELECT Field1, Field2, Field3, HugeField4 = NULL, Field5 FROM TableName";
if (parameterId.HasValue)
query += " WHERE Field1 = " + parameterId.Value.ToString();
var results = context.Database.SqlQuery<myEntity>(query);
In case your method sometimes needs this field, you can add a bool parameter and then do something like this:
IQueryable<myEntity> results;
if (excludeBigData)
results = context.Database.SqlQuery<myEntity>("SELECT Field1, Field2, Field3, HugeField4 = NULL, Field5 FROM TableName").AsQueryable();
else
results = context.myEntity.AsQueryable();
If anyone manages to make the Linq extensions work properly like if it was the original EF object, please comment so I can update the answer.
I'm using here the anonymous type because otherwise you will get a
NotSupportedException: The entity or complex type 'ProjectName.File'
cannot be constructed in a LINQ to Entities query.
var file = context.Files
.Where(f => f.Id == idFile)
.FirstOrDefault() // You need to exeucte the query if you want to reuse the type
.Select(f => new {
f.Id, f.MimeType, f.Size, f.FileName, f.DataType,
f.DateModification, f.FileId
}).FirstOrDefault();
And also its not a bad practice to de-normalize the table into further, i.e one with metadata and one with payload to avoid projection. Projection would work, the only issue is, need to edit any time a new column is added to the table.
I tried this:
From the edmx diagram (EF 6), I clicked the column I wanted to hide from EF and on their properties you can set their getter and setter to private. That way, for me it works.
I return some data which includes a User reference, so I wanted to hide the Password field even though it's encrypted and salted, I just didn't want it on my json, and I didn't want to do a:
Select(col => new {})
because that's a pain to create and maintain, especially for big tables with a lot of relationships.
The downside with this method is that if you ever regenerate your model, you would need to modify their getter and setter again.
Using Entity Framework Power Tools you can do the following in efpt.config.json:
"Tables": [
{
"ExcludedColumns": [
"FileData"
],
"Name": "[dbo].[Attachment]",
"ObjectType": 0
}
]
I've a table like this :
ID Name Key Value
1 Mydata1 Mail myval1
1 Mydata1 Name myval2
1 Mydata1 Nick myval3
1 Yourdata key2 myval4
and a class like this :
[Serializable]
public class Mydata
{
public string Mail{get;set;}
public string Name{get;set;}
public string Nick{get;set;}
}
I extract from my Table data
where Name = 'MyData1' in a list.
How To serialize this data in my class.
Thanks!
You should look into LINQ to SQL, LINQ to Entities, NHibernate, etc. These are OR Mappers, or Object/Relational Mappers. They are designed to query your database for you, and return serialized objects from your domain. They can greatly reduce the amount of effort it takes to build an application with objects that are persisted to and retrieved from a database server.
Note that Linq to SQL required SQL Server, while the other two can be used with any database.
If you have your class already, you are looking for "POCO" support in an ORM.
E.g. Entities Framework v4 does this. See here for details. I understand nHibernate has this support today.
Alternatively in your data access layer, you can do the conversion in your data access layer (DTO == Data Transfer Object) using a data reader:
var rs = dbCmd.ExecuteReader();
var results = new List<MyDto>();
while (rs.Read()) {
results.Add(new MyDto {
Id = rs.ReadInt("Id"),
Name = rs.ReadString("Name"),
...
};
}
Using LINQ to SQL:
using (var dc = MyDataContext()) {
var results = from d in dc.MyDataType
select new MyDto {
Id = d.Id,
Name = d.Name,
...
};
}
However I tend to add a method to customise the generated entity with a helper method to perform the mapping (which helps avoid repeating the property copy code for each query).