I want to write and read JSON data into/from a PostgreSQL database and (de-)serialize them lazily. Please see the following C# types:
// project: Business
public class Person // shall be persisted with EF Core in the database
{
public int Id { get; private set; }
public string Name { get; set; }
public PersonDto? RawData { get; set; }
}
// vendor assembly (via NuGet)
public class PersonDto // input from another system, coming as JSON over the wire
{
public string Name { get; set; }
public int Age { get; set; }
// ...a lot of other properties
}
// project: Persistence
public class PersonContext : DbContext
{
public DbSet<Person> Persons { get; private set; }
}
// project: Service
public class Handler
{
private readonly PersonContext _context;
public Handler(PersonContext context) => _context = context;
public async Task HandleInputAsync(PersonDto input)
{
var person = new Person { Name = input.Name, RawData = input };
await _context.Persons.AddAsync(person);
await _context.SaveChangesAsync();
}
}
On the database (PostgreSQL), the data should be persisted like this:
Id (int)
Name (text)
RawData (json)
1
Mike
{ "Name": "Mike", "Age": 33 }
So I want to store the raw input in a JSON column. I've written a custom converter to do the JSON (de-)serialization. This implementation works, but it has the downside in that the (de-)serialization happens whenever materializing an object from the DB into an EF Core entity. But I only need to access RawData on certain occasions, therefore the (de-)serialization is often not necessary (in terms of computation). Furthermore, I want to avoid the unnecessary allocation of PersonDto if not necessary.
Now comes my sixty-four-thousand-dollar question: is it possible to do the (de-)serialization only on demand/lazily? Some usage examples from inside my business layer:
var name = person.Name; // person.RawData has not yet been used and is therefore uninitialized
var rawData = person.RawData; // now the deserialization takes place.
I tried the following approach:
// project: Business
public class Person
{
public int Id { get; private set; }
public string Name { get; set; }
public string? RawData { get; set; }
public PersonDto GetRawData() => JsonSerializer.Deserialize<PersonDto>(RawData);
}
That works, but I wouldn't say I like the following facts:
The string? RawData gets loaded from the DB no matter whether GetRawData() is called or not → large unneeded string in memory
My business layer has to work with JSON (de-)serialization APIs. From a Clean Architecture perspective, I don't like that a high-level component (Person) needs to access low-level details like JSON.
So is there any alternative to solve this in EF Core 7?
Thanks in advance!
Related
I have a MySql database with columns Id int and Name:json
Places Table Sample
Id Name
1 {"en":"Sphinx","ar":"أبو الهول","fr":"Le sphinx"}
C# Place class
public class Place
{
[Key, Column("id")]
public int Id { get; set; }
[Column("name")]
public string Name { get; set; }
}
I'm connecting with EntityFramework 6 and connection success and retrieve data like this
{Id = 1, Name = "{\"en\":\"Sphinx\", \"ar\":\"أبو الهول\", \"fr\":\"Le sphinx\"}" }
What I want how to Map Name to new Object not JSON string
something like this
Place class
public class Place
{
[Key, Column("id")]
public int Id { get; set; }
[Column("name")]
public Localized<string> Name { get; set; }
}
Localized class
public class Localized<T>
{
public T en { get; set; } // english localization
public T ar { get; set; } // arabic localization
public T fr { get; set; } // french localization
}
when I do this Name property come with NULL value
Code in Repository
using (var context = new PlacesEntityModel())
{
return context.Places.Take(5).ToList();
}
I don't want to use AutoMapper,
I want something in EntityFramework to select only one language in Database Level without fetching all other data and then map it
how to fix this?
You can try extension method to map from your entity type.
public class Place
{
[Key, Column("id")]
public int Id { get; set; }
[Column("name")]
public string Name { get; set; }
}
public class PlaceDTO
{
[Key, Column("id")]
public int Id { get; set; }
[Column("name")]
public Localized<string> Name { get; set; }
}
public class Localized<T>
{
public T en { get; set; } // english localization
public T ar { get; set; } // arabic localization
public T fr { get; set; } // french localization
}
Extenstion Method ToDto
public static class Extensions
{
public static PlaceDTO ToDto(this Place place)
{
if (place != null)
{
return new PlaceDTO
{
Id = place.Id,
Name = JsonConvert.DeserializeObject<Localized<string>>(place.Name)
};
}
return null;
}
}
Usage
var place = new Place() { Id = 1, Name = "{\"en\":\"Sphinx\", \"ar\":\"أبو الهول\", \"fr\":\"Le sphinx\"}" };
var placeDTO = place.ToDto();
Console.WriteLine($"{placeDTO.Id}-{placeDTO.Name.ar}-{placeDTO.Name.en}-{placeDTO.Name.fr}");
First of all, by using a class with a property per language, you restrict yourself. You'd always have to add new properties if you add new languages, which would of course be feasible, but unnecessary complicated. Furthermore you'd usually have the language as a string-ish object (or be able to convert), hence this would lead to code like this
Localized<string> name = ...;
switch(language)
{
case "en":
return name.en;
case "ar":
return name.ar;
case "fr":
return name.fr;
default:
throw new LocalizationException();
}
which is error-prone and overly complicated. For your problem, I think I'd opt to use some kind of dictionary
IDictionary<string, string> names = ...;
if(names.ContainsKey(language))
{
return names[language];
}
else
{
throw new LocalizationException();
}
which is easily extensible by just adding more translations to the dictionary.
To convert your JSON string to an IDcitionary<string, string>, you could use the following code
localizedNames = JObject.Parse(Name)
.Children()
.OfType<JProperty>()
.ToDictionary(property => property.Name,
property => property.Value.ToString());
From within your class this would effectively be
public class Place
{
[Key, Column("id")]
public int Id { get; set; }
[Column("name")]
public string Name { get; set; }
public Dictionary<string, string> LocalizedNames
{
get
{
return JObject.Parse(Name)
.Children()
.OfType<JProperty>()
.ToDictionary(property => property.Name,
property => property.Value.ToString());
}
}
}
The localized values can be accessed like
var localizedPlaceName = place.LocalizedNames[language];
Please note: Depending on your needs and use cases, you should consider the following issues:
Caching
In my snippet, the JSON string is parsed every time the localized names are accessed. Depending on how often you access it, this might be detrimental to performance, which could be mitigated by caching the result (don't forget to delete the cache when Name is set).
Separation of concerns
The class as is is supposed to be a pure model class. You might want to introduce domain classes that encapsulate the presented logic, rather than adding the logic to the model class. Having a factory that creates readily localized objects based on the localizable object and the language could be an option, too.
Error handling
In my code there is no error handling. Depending on the reliability of input you should consider additional error handling.
devart.com/dotconnect/mysql/docs/EF-JSON-Support.html
Like what #Nkosi said
In that case then, take a look at this article devart.com/dotconnect/mysql/docs/EF-JSON-Support.html
It probably can given that the library was able to build that feature in. You would need to figure out what they they did (reverse engineer)
I usually just use JSON.Net, I notice that another answer referenced JObject, but without going into whether your data-model is the right model, I generally find that you can do:
var MyObjectInstance = JObject.Parse(myJsonString).ToObject<MyObjectType>();
I notice that you have ComponentModel attributes on your class. I don't know off hand how many of these JSon.Net supports, and you'd have to research that. It definitely supports some attributes from XML serialization, and also has some of it's own.
Note that you can also convert a JSOn array into a list:
var MyObjectList = JArray.Parse(myJsonString).ToObject<IEnumerable<MyObjectType>();
I want something in EntityFramework to select only one language in
Database Level without fetching all other data and then map it
if you want it to be from database level, you can always create a view and then include this view in your project.
Example :
CREATE VIEW `PlacesLocalized` AS
SELECT
Id
, TRIM(REPLACE(name->'$.en', '"','')) AS en
, TRIM(REPLACE(name->'$.ar', '"','')) AS ar
, TRIM(REPLACE(name->'$.fr', '"','')) AS fr
FROM
places
This would create a model class Like :
public class PlacesLocalized
{
public int Id { get; set; }
public string en {get; set;}
public string ar {get; set;}
public string fr {get; set;}
}
Then, you can do :
var places = context.PlacesLocalized.Where(x=> x.en == "Sphinx");
But if you don't have enough permissions to do this in the database level, then you would need to specify the query in your EF. There is no easy way to change the execution logic of Entity Framework just for specific classes. That's why Entity Framework included SqlQuery method, which would give more flexibility to have custom queries when needed (like yours).
So, if you need to specify the localization from Entity Framework, then you would do a repository class to specify all custom queries you need including creating any DTO needed.
The basic way would be something like this :
public enum Localized
{
English,
Arabic,
French
}
public class PlaceRepo : IDisposable
{
private readonly PlacesEntityModel _context = new PlacesEntityModel();
public List<Place> GetPlacesLocalized(Localized localized = Localized.English)
{
string local = localized == Localized.Arabic ? "$.ar"
: localized == Localized.French ? "$.fr"
: "$.en";
return _context.Places.SqlQuery("SELECT Id, name-> #p0 as Name FROM places", new[] { local })
.Select(x=> new Place { Id = x.Id, Name = x.Name.Replace("\"", string.Empty).Trim() })
.ToList();
}
private bool _disposed = false;
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
protected virtual void Dispose(bool disposing)
{
if (!_disposed)
{
if (disposing)
{
_context.Dispose();
}
_disposed = true;
}
}
~PlaceRepo()
{
Dispose(false);
}
}
now, you can do this :
using(var repo = new PlaceRepo())
{
var places = repo.GetPlacesLocalized(Localized.Arabic);
}
public class Place
{
[Key, Column("id")]
public int Id { get; set; }
[Column("name")]
public string Name { get; set; }
public static explicit operator Place(PlaceDTO dto)
{
return new Place()
{
Id = dto.Id,
Name = dto.Name
};
}
}
public class PlaceDTO
{
[Key, Column("id")]
public int Id { get; set; }
[Column("name")]
public Localized<string> Name { get; set; }
public static explicit operator PlaceDTO(Place pls)
{
return new PlaceDTO()
{
Id = pls.Id,
Name = pls.Name
};
}
}
var placeDTO = (placeDto)place;
we can achieve this using explicit operator without using auto mapper
I have two entity classes that have a one-to-many relationship.
public class Call : IEntity
{
public int Id { get; set; }
public int UserId { get; set; }
public virtual User User { get; set; }
}
public class User : IEntity
{
public int Id { get; set; }
public string Username { get; set; }
public virtual ICollection<Call> Calls { get; set; }
}
And I have a view model for 'Call' operations on the web layer.
public class CallVm : IViewModel
{
public string Id { get; set; }
public string UserFullname { get; set; }
}
And I use a method to convert my 'Call' object to 'CallVm' object.
This method is briefly as follows.
public CallVm MapCallVm(Call call)
{
return call == null ? null : new CallVm { Id = call.Id, UserFullname = call.User?.Fullname };
}
When I read the 'Call' entity from the database, I sometimes include 'User' and sometimes I don't. When I do not include it, there is no User property definition in the Call object because it is lazy loading. Therefore, I get the following error in MapCallVm method.
The ObjectContext instance has been disposed and can no longer be used for operations that require a connection.
Is there a way to check this? I just want to assign UserFullname = call.User?.Fullname when there is a eager load.
The only solution I can think of is controlling with try-catch. Is there a different solution?
You can use DbReferenceEntry.IsLoaded Property.
Gets or sets a value indicating whether the entity has been loaded
from the database.
if (_dbContext.Entry(Call).Reference(e => e.User).IsLoaded)
Updated
If you are getting value without dbContext, you should force the query to Eager loading instead.
Read the following post to have a better understanding.
Should we disable lazy loading of Entity Framework in web apps?
As #Phong's answer - avoid passing DbContext. Normally, your repository class should map DB entities to simple POCO/DTO objects.
I suggest to introduce mapper class. This will help you to unit test your logic
// Interface to inject to repository
public interface ICallMapper
{
CallVm Map(Call call);
}
public class CallMapper : ICallMapper
{
public CallVm Map(Call call)
{
return call == null ? null : new CallVm { Id = call.Id, UserFullname = call.User?.Username };
}
}
Pass mapper to repository and ensure that your objects are not connected with DB anymore
public class CallRepository : ICallRepository
{
private readonly ICallMapper _callMapper;
public CallRepository(ICallMapper callMapper)
{
_callMapper = callMapper;
}
public IList<CallVm> GetList()
{
// Call DB and get entities
var calls = GetCalls();
// Map DB entities to plain model
return calls.Select(_callMapper.Map).ToList();
}
}
This lets you to get rid of your error. And makes your program more structable and testable.
I am migrating /re-developing a web app from JavaScript to the ASP.NET MVC Framework using C#/ JS (with Handlebars.NET) for my Bachelor thesis.
So far I have created a Web.API and the actual app with a form.
In the app I enter details to create a new Employee, which is then Posted to the API, which receives that Json-Object as a "Business Object" BOEmployee.
Said BOEmployee looks like this (simplified):
public class BOEmployee
{
public int ID_Employee { get; set; }
public int ID_Company { get; set; }
public string lastName { get; set; }
public string firstName { get; set; }
}
I want to map this object to two other objects, representing tables of the underlying database, to then save them to the database. The two target tables are auto generated with Entity Framework.
Here are the table objects:
1. Employee:
public partial class Employee
{
public int ID_Employee { get; set; }
public int ID_Company { get; set; }
}
2. Employee_Details:
public partial class Employee_Detail
{
public int ID_Employee_Detail { get; set; }
public int ID_Employee { get; set; }
public string lastName { get; set; }
public string firstName { get; set; }
}
Now I could map them manually by assigning every attribute but clearly that is a horribly unsustainable idea. So I was looking for a way to automate that mapping process automatically using Json.Net like this:
[HttpPost]
public BOEmployee SaveEmployee([FromBody] string employee)
{
using (var context = new myDBEntities())
{
JavaScriptSerializer serializer = new JavaScriptSerializer();
Employee_Detail dbEmployeeDetails = serializer.Deserialize<Employee_Detail>(BOEmployee);
Employee dbEmployee = serializer.Deserialize<Employee>(BOemployee);
}
}
Now what happens when I run that code is, that the serializer-function complains that the input values cannot be null, which to my understanding is because the target Objects (e.g. Employee) do not have all attributes that are given in the serialized Json-Object.
The Error Message is this:
Value cannot be null.\r\nParameter name: input",
"ExceptionType":"System.ArgumentNullException"
Now my question would be, how can I map my object to the different Database tables? Or am I completely on the wrong path now?
Fundamental changes to the program structure cannot be made any more due to available time (and I am basically a complete beginner in programming).
I recommend AutoMapper than what you are using there.
I am building an ASP Web API application and this time I thought I will go with the MVC pattern. I got along with most of the stuff, but there is one thing of which I am unsure. First of all my project consists of the following:
Data Layer
Business Layer
Model Layer (just the model with the properties)
Service Application (here are my controllers)
every one of them in a separate project
Lets say I have the following controller
public class TestController : ApiController
{
ISomeService _someBusiness;
public TestController(ISomeService someBusiness)
{
_someBusiness = someBusiness;
}
public **SomeModelObject** GetModelObject(ind id)
{
return _someBusiness .GetSomeModelObject(id);
}
}
Now my problem is the return value of GetModelObject(int id). Here it says SomeModelObject. That implies that my Service application (or my controller) has to know everything about the model which is being used (so I dont see the point in defining it in a separate .dll). One way would be to define the model (precisely the get/set mothods) as an interface, but I think that it would be too much that every model class has an interface (mostly because, as I said, just the properties are being stored inside the model), and despite that I just does not feel right to build an interface for a class which only stores data. So, is there any generic response type which is being used in this case (even some completely different approach), or do I have to use my model classes (or may i just always use string and it is being converted to the appropriate format by the client) ?
There's a good reason to use an interface to hide the complexity of the model object. It holds data, sure. But it holds unnecessary data that is only meaningful to the data layer. Take this EF model:
public class Employee
{
public int Id { get; set; }
public string EmployeeNumber { get; set; }
public string Name { get; set; }
public virtual Collection<TimeCard> TimeCards { get; set; }
public int DepartmentId { get; set; }
public virtual Department Department { get; set; }
}
This is a fairy common EF model. It contains a surrogate key Id, and a foreign key DepartmentId. Those values are meaningless except for the database and, by extension, for entity framework. EmployeeNumber is the natural key which uniquely identifies the entity in the user's domain.
Outside of database access, you should really only deal with natural data values. You could do this by declaring yet another data-carrying class in the Business layer and perform mapping, or a better idea is to use an interface to hide all of the members that are not useful.
public interface IEmployee
{
string EmployeeNumber { get; }
string Name { get; set; }
ICollection<ITimeCard> TimeCards { get; }
IDepartment Department { get; set; }
}
Notice the lack of some setters in the interface. You'll never want to change the EmployeeNumber because that is the natural key for the entity. Likewise, you'll never assign a collection object to the TimeCards property. You'll only ever iterate over, add, or remove them.
Now your Employee class becomes
public class Employee : IEmployee
{
public int Id { get; set; }
public string EmployeeNumber { get; set; }
public string Name { get; set; }
public virtual Collection<TimeCard> TimeCards { get; set; }
ICollection<ITimeCard> IEmployee.TimeCards { get { return TimeCards; } }
public int DepartmentId { get; set; }
public virtual Department Department { get; set; }
IDepartment IEmployee.Department { get { return Department; } set { Department = value; } }
}
In your business layer and above, you'll only use variable of IEmployee, IDepartment, and ITimeCard. So you are exposing a tighter API to the higher layers, which is a good thing.
You could try to use a generic approach at controller level:
public class BusinessController<T> : ApiController
{
ISomeService _someBusiness;
public TestController(ISomeService someBusiness)
{
_someBusiness = someBusiness;
}
public T GetModelObject(ind id)
{
return _someBusiness.GetSomeModelObject(id);
}
}
Finally your controlers inherit from BusinessController instead of ApiController:
public class TestController : BusinessController<SomeModelObject>
{
}
You could also take advance of the templating to inject the right "ISomeService" by using an IoC container and a bootstrapper.
I just started using Entity Framework and it created a Context class which I can use to get all the data i need from it. But I am facing an issue on how I should organize my code, by watching the demos, the person just uses the framework and codes everything on a console application. What is the best way to use Entity Framework and that it looks clean?, what I mean by this is...right now using aspx pages, I could just use the aspx.cs to get the data or save the data. But I do not want this, I would like it to be more organized although the Entity Framework did almost everything by creating the objects etc.. but still, I need to use things like
using(var myobject = new MyContextData())
{
blah blah..
}
would you say that it would be nicer to write classes that would wrap these calls?. I would really appreciate any inputs as it would really make me a better programmer using the entity framework.
Regards
This question should everyone, who provides some tutorial about EF, ask. It is hard to say what is the best way, but put all code in the codebehind classes (aspx.cs) does not help extensibility and testability. Please, try to read this article:
http://www.asp.net/mvc/tutorials/getting-started-with-ef-using-mvc/implementing-the-repository-and-unit-of-work-patterns-in-an-asp-net-mvc-application
Not only it is official tutorial on asp.net, but it mostly shows, that Framework EF could be used correctly in currently fancy Repository pattern
Edit:
I think that Generic Repository is Anti Pattern. But I do not understand #TomTom comment.
Original Answer:
As Radim Köhler mentioned you need to implement Repository and Unit of Work patterns
But the article he provided in my opinion is not fully correct.
At my current job I use following implementation of these patterns.
For example, we have three types of entities: Person, Good and Order. I created repository for Persons. In common case Repository must not be generic. It must contain methods which represent specific queries for this entity. So by looking at the interface of repository you can tell what kinds of queries executed for entity (Person, e.g.). As you will see I created DTO for Person called PersonWrap. For creating PersonWrap from Person and updating Person from PersonWrap you can use AutoMapper instead of PersonWrap() constructor and Update() method. Because EntityFramework DbContext implements Unit of Work pattern, you just need to provide created DbContext to repository methods. If repository method is a separate action and you do not need DbContext outside of this method you can create and dispose it inside this method.
public class Person {
public int Id { get; set; }
public string FirstName { get; set; }
public string SecondName { get; set; }
public DateTime RegistrationDate { get; set; }
public List<Order> Orders { get; set; }
}
public class Good {
public int Id { get; set; }
public string Name { get; set; }
public decimal Price { get; set; }
}
public class Order {
public int Id { get; set; }
public Person Person { get; set; }
public Good Good { get; set; }
public int Count { get; set; }
}
public class MyDbContext: DbContext
{
public IDbSet<Person> Persons { get { return Set<Person>(); }}
public IDbSet<Good> Goods { get { return Set<Good>(); }}
public IDbSet<Order> Orders { get { return Set<Order>(); }}
}
public class PersonRepository {
public IEnumerable<Person> GetAll() {
using (var context = new MyDbContext()) {
return context.Persons.ToList();
}
}
public IEnumerable<Person> GetLastWeekPersons() {
using (var context = new MyDbContext()) {
return context.Persons.Where(p => p.RegistrationDate > new DateTime().AddDays(-7)).ToList();
}
}
public Person GetById(int id, MyDbContext context) {
return context.Persons.Include(p => p.Orders).FirstOrDefault(p => p.Id == id);
}
public Person GetById(int id) {
using (var context = new MyDbContext()) {
return GetById(id, context);
}
}
}
public class PersonWrap {
public int Id { get; set; }
public string FirstName { get; set; }
public string SecondName { get; set; }
public int OrderCount { get; set; }
public PersonWrap(Person person) {
Id = person.Id;
FirstName = person.FirstName;
SecondName = person.SecondName;
OrderCount = person.Orders.Count;
}
public void Update(Person person) {
person.FirstName = FirstName;
person.SecondName = SecondName;
}
}
public class PersonDetailsViewController {
public PersonWrap Person { get; protected set; }
public PersonDetailsViewController(int personId) {
var person = new PersonRepository().GetById(personId);
if (person != null) {
Person = new PersonWrap(person);
}
}
public void Save() {
using (var context = new MyDbContext()) {
var person = new PersonRepository().GetById(Person.Id, context);
Person.Update(person);
context.SaveChanges();
}
}
}
You are on the right track for creating classes to handle your EF.
The biggest benefit for doing it this way is able to unit test easily.
Test early and test often is always a good idea.
I suggest putting your EF related classes in a separate project.