Develop Reference

Convert EntityModel to DTO Using "Basic" Custom Method

First, I apologise if this is a dupe, finding the right search terms seemed impossible...
We are trying to adopt some best practice and looking at refactoring duplicate code in our projects. On a number of occasions we have something like;
public List<EventModel> GetEvents(bool showInactive, bool showPastEvents)
{
return eventRepository
.GetEvents(_customerId, showInactive, showPastEvents)
.Select(e => New EventModel() { Id = e.EventId, Name = e.EventName, Capacity = e.EventCapacity, Active = e.EventActive })
.ToList();
}
So we tried doing something like this instead;
public List<EventModel> GetEvents(bool showInactive, bool showPastEvents)
{
return eventRepository
.GetEvents(_customerId, showInactive, showPastEvents)
.Select(e => ConvertPocoToModel(e))
.ToList();
}
private EventModel ConvertPocoToModel(TsrEvent tsrEvent)
{
EventModel eventModel = new EventModel()
{
Id = tsrEvent.EventId,
Name = tsrEvent.EventName,
Capacity = tsrEvent.EventCapacity,
Active = tsrEvent.EventActive
};
return eventModel;
}
Sometimes this works, but intermittently we get;
System.NotSupportedException: 'LINQ to Entities does not recognize the
method 'Bll.Models.EventModel ConvertPocoToModel(Dal.Pocos.TsrEvent)'
method, and this method cannot be translated into a store expression.'
I am aware we could add .ToList() or similar to force the conversion to happen in C# but I believe that means SQL will execute SELECT * instead of SELECT EVentId, EventName, EventCapacity, EventActive
Can anyone explain;
Why EF is having issues trying to understand how to handle this simple mapping?
why it work intermittently?
How we should be doing it?

Entity framework doesnt know how to translate your method. You have to use method which returns Expression<Func<TsrEvent,EventModel>> or an property which stores it.
public List<EventModel> GetEvents(bool showInactive, bool showPastEvents)
{
return eventRepository
.GetEvents(_customerId, showInactive, showPastEvents)
.Select(ConvertPocoToModelExpr)
.ToList();
}
private static Expression<Func<TsrEvent,EventModel>> ConvertPocoToModelExpr => (x)=>new EventModel()
{
Id = x.EventId,
Name = x.EventName,
Capacity = x.EventCapacity,
Active = x.EventActive
};

You have to be aware about the differences between an IEnumerable and an IQueryable.
An IEnumerable object holds everything to enumerate over the sequence. You can ask for the first element, and once you've got an element you can ask for the next one, as long as there is a next one. The IEnumerable is meant to be processes locally by your process.
Enumeration at its lowest level is done by asking for the Enumerator and repeatedly calling MoveNext, until you don't need anymore elements. Like this:
IEnumerable<Student> students = ...
IEnumerator<Student> studentEnumerator = students.GetEnumerator();
while (studentEnumerator.MoveNext())
{
// there is still a Student to process:
Student student = studentEnumerator.Current;
ProcessStudent(student);
}
You can do this explicitly, or call it implicitly using foreach or one of the LINQ functions.
On the other hand, an IQueryable is meant to be processed by a different process, usually a database management system. The IQueryable holds an Expression and a Provider. The Expression expresses the query that must be performed in some generic format. The Provider knows who must execute the query (usually a database management system), and the language that this process uses (usually something SQL like).
As soon as you start enumerating by calling GetEnumerator, the Expression is sent to the Provider, who tries to translate the Expression into SQL and executes the query. The fetched data is put into an enumerable sequence, and the enumerator is returned.
Back to your question
The problem is, that SQL does not know ConvertPocoToModel. Hence your provider can't convert the Expression. The compiler can't detect this, because it does not know how smart your Provider is. That is why you don't get this error until you call GetEnumerator, in your case by calling ToList.
Solution
The solution is to make a function that changes the expression. The easiest method would be an extension function. See extension methods demystified. This way you can use it like any other LINQ method:
public static IQueryable<EventModel> ToEventModels(this IQueryable<TsrEvent> tsrEvents)
{
return tsrEvent.Select(tsrEvent => new EventModel
{
Id = tsrEvent.EventId,
Name = tsrEvent.EventName,
Capacity = tsrEvent.EventCapacity,
Active = tsrEvent.EventActive
};
}
Note that I omit the () in the constructor: SQL can't call constructors!
Usage:
var result = dbContext.TsrEvents
.Where(tsrEvent => tsrEvent.Active && tsrEvent.Date == Today)
.ToEventModels()
.GroupBy(...)
... etc
Or, if your GetEvents returns an IQueryable<TsrEvents>
return eventRepository.GetEvents(_customerId, showInactive, showPastEvents)
.ToEventModels();
Final Remark
It is better to let your data-fetch-functions return IQueryable<...> and IEnumerable<...> as long as possible. Let only the end-user materialize the query. It would be a waste of processing power if you do the ToList() and your caller only wants to do FirstOrDefault()

Entity Framework Core Linq query returning ids which do not exist in database

I wonder if there is an easy way using Linq to SQL with Entity Framework Core to query check if a given list of ids exist in the database and which returns the list of ids that do not exist.
The use case I come across this is if the user can do something with a list of object (represented through the list of their ids) I want to check if these ids exist or not.
Of course I could query all objects/object ids that exist in the database and cross check in a second step.
Just wondering if it would be possible in one step.
What I mean in code:
public class MyDbObject
{
public int Id { get; set; }
public string Name { get; set; }
}
public IActionResult DoSomethingWithObjects([FromQuery]List<int> ids}
{
List<int> idsThatDoNotExistInTheDb = DbContext.MyDbObject.Where(???)
return NotFound("those ids do not exist: " + string.Join(", ", idsThatDoNotExist));
}

You can obtain the list of IDs that match, then remove them from the original list, like this:
var validIds = DbContext
.MyDbObject
.Where(obj => ids.Contains(obj.Id))
.Select(obj => obj.Id);
var idsThatDoNotExistInTheDb = ids.Except(validIds);
This approach may be slow, though, so you may be better off doing it in a stored procedure that takes a table-valued parameter (how?)
Note: Pre-checks of this kind are not bullet-proof, because a change may happen between the moment when you validate IDs and the moment when you start the operation. It is better to structure your APIs in a way that it validates and then does whatever it needs to do right away. If validation fails, the API returns a list of errors.

Using extension method in Linq to Entities

Code:
public static IEnumerable<TableRowModel> GetRows()
{
var to_ret = db.TableRows.select(x=> new TableRowModel(){
TableRowId = x.TableRowId,
Type = x.Type,
Name = x.Name,
CreatedAt = x.CreatedAt,
ModifiedAt = x.ModifiedAt,
Enums = x.Enums.Select(y => y.ToEnumModel()),
});
return to_ret;
}
public static EnumModel ToEnumModel(this Enum x)
{
var to_ret = new EnumModel()
{
CFPId = x.CFPId,
CreatedAt = x.CreatedAt,
ModifiedAt = x.ModifiedAt,
};
return to_ret;
}
I get the following error when using the GetRows method:
LINQ to Entities does not recognize the method
Given the error, it's understood that LINQ To Entities is not able to recognize the extension method ToEnumModel.
I would like to know if there is a way around this?
So that I would not be repeating ToEnumModel code again in GetRows extension.

Under normal circumstances, when performing an operation like a Where in EntityFramework, it isn't actually executed in memory like when you operate on an enumerable (like a List). Instead, it converted into SQL which is then executed by the Db provider. This means that doing certain things, such as using extension methods on objects, is not an option, as the converter cannot turn such things into SQL.
The quickest and easiest solution would be to load your set in-memory, where it can be operated on like any other enumerable. You can do this like so:
var result = myDbSet.AsEnumerable().Etc()
Etc() here represents all other operations you want to run. Be advised however, that this will load all data into memory, which may be prohibitively slow and expensive in some scenarios. One way to alleviate this is to put AsEnumerable() right before you need to use the extension method, thus offloading as many operations as possible to the provider. For example this:
var result = myDbSet.Where([condition]).AsEnumerable().Select(item => item.ExtensionMethod());
Might be much quicker and lighter than this:
var result = myDbSet.AsEnumerable().Where([condition]).Select(item => item.ExtensionMethod());

Your ability to use extension methods in EF is limited, but you can still extend your IQueryables
public static class Extensions
{
public static IQueryable<MyModelVM> SelectVMs(this IQueryable<MyModel> models)
{
return models.Select(x => new MyModelVM { MyModelId = x.MyModelId });
}
}

I don't understand Dapper's mapping, multimapping and QueryMultiple

Title says it all, I'm trying to use it but I don't understand it. It's possible that the problem is a lack of knowledge due to that I'm an amateur, but I've read a dozen questions about this thing and googled for three days, and I still don't understand it.
I have SO many questions that I'm not sure that I should write it all in only one Question, or even if someone would read it all. If someone have other solution or think I should split it in different questions... well, I'm open to suggestions.
I was going to write an example, but again I read dozen of examples for days and didn't help me.
I just can't make my mind to understand how work something like the example at github:
var sql =
#"select * from #Posts p
left join #Users u on u.Id = p.OwnerId
Order by p.Id";
var data = connection.Query<Post, User, Post>(sql, (post, user) => { post.Owner = user; return post;});
So, Post have a property of type User and that property is called Owner, right? Something like:
public class Post
{
...
public User Owner { get; set;}
}
Therefore Query<Post, User, Post> will return a Post instance with all the properties and what not, AND will create a User instance and assign it to Post.Owner property? How would simple parameters be added to that query, for example if someone wanted to pass the id as a int parameter like ...WHERE Id = #Id", new {Id = id}, where should the parameter be added given that the parameter right now is (post, user) => { post.Owner = user; return post;}? The parameter always refer to the types given, you can only use the simple typical parameters for the dynamic query, both can be used simultaneously? How?
Also, how does it to differentiate what DB field goes to what object? It makes something like class name=DB table name? What happens if the classes don't have the same name as the DB table and I want to use the [Table] attribte, will it work or the attribute is only for Dapper.Contrib.Extensionsmethods? Would it work with objects that share the same DB table?
Regarding same table for different objects question, f.i. lets say I have a Person object that have a BankAccount object:
public class Person
{
...
public BankAccount Account {get; set;}
...
}
public class BankAccount
{
private string _Account;
public string Account
{
get { return _Account; }
set
{
if(!CheckIfIBANIsCorrect(value))
throw new Exception();
_Account = value;
}
}
private bool CheckIfIBANIsCorrect(string IBAN)
{
//...
//Check it
}
}
I could store the string account at the same table than Person, since every person would have a single account referred by the person's Id. How should I map something like that? Is there even a way, should I simply load the result in a dynamic object and then create all the objects, will Query create the rest of the Person object and I should bother to create the nested object myself?
And by the way, how is splitOnsupposedly be used in all this? I understand that it should split the result into various "groups" so you can split the results by Ids f.i. and take what you need, but I don't understand how should I retrieve the info from the different "groups", and how it return the different "groups", lists, enumerables, what?.
QueryMultiple is other thing that is FAR beyond my understanding regardles how much questions and answers I read.
You know... how the * does that .Read thing work? All I read here or googling assumes that Read is some sort of automagic thing that can miracly discern between objects. Again, do it divide results by class names so I just have to be sure every object have the correct table name? And again what happens with [Table] attribute in this case?
I think the problem I'm having is that I can't find(I suppose it doesn't exist) a single web page that describes it all(the examples at GitHub are very scarce), and I only still finding answers to concrete cases that doesn't answer exactly what I'm trying to understand but only that concrete cases, which are confusing me more and more while I read them, since everyone seems to use a bunch of different methods without explaining WHY or HOW.

I think that your main problem with the Dapper querying of joined table queries is thinking that the second argument in the list is always the "param" argument. Consider the following code:
var productsWithoutCategories = conn.Query<Product>(
"SELECT * FROM Products WHERE ProductName LIKE #nameStartsWith + '%'",
new { nameStartsWith = "a" }
);
Here, there are two arguments "sql" and "param" - if we used named arguments then the code would look like this:
var productsWithoutCategories = conn.Query<Product>(
sql: "SELECT * FROM Products WHERE ProductName LIKE #nameStartsWith + '%'",
param: new { nameStartsWith = "a" }
);
In your example, you have
var data = connection.Query<Post, User, Post>(sql, (post, user) => { post.Owner = user; return post;});
The second argument there is actually an argument called "map" which tells Dapper how to combine entities for cases where you've joined two tables in your SQL query. If we used named arguments then it would look like this:
var data = connection.Query<Post, User, Post>(
sql: sql,
map: (post, user) => { post.Owner = user; return post;}
);
I'm going to use the class NORTHWND database in a complete example. Say we have the classes
public class Product
{
public int ProductId { get; set; }
public string ProductName { get; set; }
public bool Discontinued { get; set; }
public Category Category { get; set; }
}
public class Category
{
public int CategoryID { get; set; }
public string CategoryName { get; set; }
}
and we want to build a list of Products, with the nested Category type populated, we'd do the following:
using (var conn = new SqlConnection("Server=.;Database=NORTHWND;Trusted_Connection=True;"))
{
var productsWithCategories = conn.Query<Product, Category, Product>(
"SELECT * FROM Products INNER JOIN Categories ON Categories.CategoryID = Products.CategoryID,
map: (product, category) =>
{
product.Category = category;
return product;
},
splitOn: "CategoryID"
);
}
This goes through all the rows of JOIN'd Product and Category data and generates a list of unique Products but can't be sure how to combine the Category data with it, so it requires a "map" function which takes a Product instance and a Category instance and which must return a Product instance which has the Category data combined with it. In this example, it's easy - we just need to set the Category property on the Product instance to the Category instance.
Note that I've had to specify a "splitOn" value. Dapper presumes that the key columns of tables will simply be called "Id" and, if they are, then it can deal with joins on those columns automatically. However, in this case, we're joining on a column called "CategoryID" and so we have to tell Dapper to split the data back up (into Products and into Categories) according to that column name.
If we also wanted to specify "param" object to filter down the results, then we could do something like the following:
var productsWithCategories = conn.Query<Product, Category, Product>(
"SELECT * FROM Products INNER JOIN Categories ON Categories.CategoryID = Products.CategoryID WHERE ProductName LIKE #nameStartsWith + '%'",
map: (product, category) =>
{
product.Category = category;
return product;
},
param: new { nameStartsWith = "a" },
splitOn: "CategoryID"
);
To answer your final question, QueryMultiple simply executes multiple queries in one go and then allows you to read them back separately. For example, instead of doing this (with two separate queries):
using (var conn = new SqlConnection("Server=.;Database=NORTHWND;Trusted_Connection=True;"))
{
var categories = conn.Query("SELECT * FROM Categories");
var products = conn.Query("SELECT * FROM Products");
}
You could specify a single SQL statement that includes both queries in one batch, but you would then need to read them separately out of the combined result set that is returned from QueryMultiple:
using (var conn = new SqlConnection("Server=.;Database=NORTHWND;Trusted_Connection=True;"))
{
var combinedResults = conn.QueryMultiple("SELECT * FROM Categories; SELECT * FROM Products");
var categories = combinedResults.Read<Category>();
var products = combinedResults.Read<Product>();
}
I think that the other examples I've seen of QueryMultiple are a little confusing as they are often returning single values from each query, rather than full sets of rows (which is what is more often seen in simple Query calls). So hopefully the above clears that up for you.
Note: I haven't covered your question about the [Table] attribute - if you're still having problems after you've tried this out then I would suggest creating a new question for it. Dapper uses the "splitOn" value to decide when the columns for one entity end and the next start (in the JOIN example above there were fields for Product and then fields for Category). If you renamed the Category class to something else then the query will still work, Dapper doesn't rely upon the table name in this case - so hopefully you won't need the [Table] at all.

How to create a custom property in a Linq-to-SQL entity class?

I have two tables Studies and Series. Series are FK'd back to Studies so one Study contains a variable number of Series.
Each Series item has a Deleted column indicating it has been logically deleted from the database.
I am trying to implement a Deleted property in the Study class that returns true only if all the contained Series are deleted.
I am using O/R Designer generated classes, so I added the following to the user modifiable partial class for the Study type:
public bool Deleted
{
get
{
var nonDeletedSeries = from s in Series
where !s.Deleted
select s;
return nonDeletedSeries.Count() == 0;
}
set
{
foreach (var series in Series)
{
series.Deleted = value;
}
}
}
This gives an exception "The member 'PiccoloDatabase.Study.Deleted' has no supported translation to SQL." when this simple query is executed that invokes get:
IQueryable<Study> dataQuery = dbCtxt.Studies;
dataQuery = dataQuery.Where((s) => !s.Deleted);
foreach (var study in dataQuery)
{
...
}
Based on this http://www.foliotek.com/devblog/using-custom-properties-inside-linq-to-sql-queries/, I tried the following approach:
static Expression<Func<Study, bool>> DeletedExpr = t => false;
public bool Deleted
{
get
{
var nameFunc = DeletedExpr.Compile();
return nameFunc(this);
}
set
{ ... same as before
}
}
I get the same exception when a query is run that there is no supported translation to SQL. (
The logic of the lambda expression is irrelevant yet - just trying to get past the exception.)
Am I missing some fundamental property or something to allow translation to SQL? I've read most of the posts on SO about this exception, but nothing seems to fit my case exactly.

I believe the point of LINQ-to-SQL is that your entities are mapped for you and must have correlations in the database. It appears that you are trying to mix the LINQ-to-Objects and LINQ-to-SQL.
If the Series table has a Deleted field in the database, and the Study table does not but you would like to translate logical Study.Deleted into SQL, then extension would be a way to go.
public static class StudyExtensions
{
public static IQueryable<study> AllDeleted(this IQueryable<study> studies)
{
return studies.Where(study => !study.series.Any(series => !series.deleted));
}
}
class Program
{
public static void Main()
{
DBDataContext db = new DBDataContext();
db.Log = Console.Out;
var deletedStudies =
from study in db.studies.AllDeleted()
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}
}
}
This maps your "deleted study" expression into SQL:
SELECT t0.study_id, t0.name
FROM study AS t0
WHERE NOT EXISTS(
SELECT NULL AS EMPTY
FROM series AS t1
WHERE (NOT (t1.deleted = 1)) AND (t1.fk_study_id = t0.study_id)
)
Alternatively you could build actual expressions and inject them into your query, but that is an overkill.
If however, neither Series nor Study has the Deleted field in the database, but only in memory, then you need to first convert your query to IEnumerable and only then access the Deleted property. However doing so would transfer records into memory before applying the predicate and could potentially be expensive. I.e.
var deletedStudies =
from study in db.studies.ToList()
where study.Deleted
select study;
foreach (var study in deletedStudies)
{
Console.WriteLine(study.name);
}

When you make your query, you will want to use the statically defined Expression, not the property.
Effectively, instead of:
dataQuery = dataQuery.Where((s) => !s.Deleted);
Whenever you are making a Linq to SQL query, you will instead want to use:
dataQuery = dataQuery.Where(DeletedExpr);
Note that this will require that you can see DeletedExpr from dataQuery, so you will either need to move it out of your class, or expose it (i.e. make it public, in which case you would access it via the class definition: Series.DeletedExpr).
Also, an Expression is limited in that it cannot have a function body. So, DeletedExpr might look something like:
public static Expression<Func<Study, bool>> DeletedExpr = s => s.Series.Any(se => se.Deleted);
The property is added simply for convenience, so that you can also use it as a part of your code objects without needing to duplicate the code, i.e.
var s = new Study();
if (s.Deleted)
...