Question:
Why does passing a list from a protected internal class to an API method and then back to my UI take 4 seconds when direct data access in Entity takes .4? Is it due to the actual passing of the list through these methods?
Background:
I wish to create an "API" layer that sits between my data access layer using Entity Framework and my UI layer. The idea is to limit access to the CRUD operations and force everything through the API however I'm noticing terrible performance.
When I use my class structure here, the get method in this case takes 4 seconds to run:
public class API
{
DataAccessClass _dataAccess = new DataAccessClass();
public List<Items> GetById(int id)
{
return _dataAccess.Get(id);
}
protected internal class DataAccessClass
{
protected internal List<Items> GET(int id)
{
using (var context = dbcontext)
{
return context.GetItems();
}
}
protected internal List<Items> GET(long id)
{
using (var context = dbcontext)
{
return context.GetItems();
}
}
}
}
However when I use my dbcontext directly in code (which I want to prevent), it runs in .4 seconds using the same code found in the protected class above:
using (var context = dbcontext)
{
return context.GetItems();
}
EDIT:
When I exclude the data access portion of the API which is the protected internal portion, and run the using statement in the API directly (just cutting out the protected internal portion) then I get the acceptable .4 seconds.
To solve my performance issue, I removed overloads from the GET class above.
My performance issues from the original question where resolved by changing method names on two of my get methods (one that took an int and one long). Not sure why these overloads causes issues but by removing the overloads and specifying names directly, my issue with performance was resolved.
My working class looks like:
public class API
{
DataAccessClass _dataAccess = new DataAccessClass();
public List<Items> GetById(int id)
{
return _dataAccess.Get(id);
}
protected internal class DataAccessClass
{
protected internal List<Items> GetByInt(int id)
{
using (var context = dbcontext)
{
return context.GetItems();
}
}
protected internal List<Items> GetByLong(long id)
{
using (var context = dbcontext)
{
return context.GetItems();
}
}
}
}
Related
I have a requirement to add a project and translate its description with an external API translation service into several languages (so they exist in the database and it's possible to fetch a project in different languages later). Since translation takes quite some time - I first need to return the API response and then translate description and add additional locale rows to the database.
The db schema looks like this:
DB Context setup:
public class ProjectsContext : DbContext
{
public ProjectsContext(DbContextOptions<ProjectsContext> options)
: base(options)
{
}
protected override void OnModelCreating(ModelBuilder mb)
{
mb.Entity<Project>().ToTable("Project");
mb.Entity<ProjectLocale>().ToTable("ProjectLocales")
.HasKey(pl => new {pl.ProjectId, pl.Locale});
}
public DbSet<Project> Projects { get; set; }
}
In the Startup class, ConfigureServices method:
services.AddDbContext<ProjectsContext>(options =>
options.UseMySql(envConfig.PROJECTS_DB_CONNECTION_STRING_MASTER));
The models:
public class Project
{
public int Id { get; private set; }
public List<ProjectLocale> ProjectLocales { get; private set; } = new List<ProjectLocale>();
public async Task TranslateDescription(Translator translator, LanguageEnum currentLanguage)
{
ProjectLocales = await translator.Translate(ProjectLocales, currentLanguage);
}
}
public class ProjectLocale
{
public int ProjectId { get; set; }
public string Locale { get; set; }
public string Description { get; set; }
}
In the repository I have the following Add and AddProjectDescriptionTranslation methods:
public void Add(Project project)
{
projectsContext.Projects.Add(project);
projectsContext.SaveChanges();
AddProjectDescriptionTranslations(project);
}
private async Task AddProjectDescriptionTranslations(Project project)
{
await project.TranslateDescription(translator, headers.LanguageEnum);
projectsContext.Projects.Update(project);
projectsContext.SaveChanges();
}
I use Add method in the controller's POST method, and the next row is already returning the added project with the description in the current language. Translator and Headers are injected into my repository class.
The current solution gives me an error (that I can view only if I follow the async method execution with a breakpoint):
Cannot access a disposed object. A common cause of this error is disposing a context that was
resolved from dependency injection and then later trying to use the same context instance
elsewhere in your application. This may occur if you are calling Dispose() on the context,
or wrapping the context in a using statement. If you are using dependency injection, you should
let the dependency injection container take care of disposing context instances.
Object name: 'ProjectsContext'.
Is there any solution to this problem? Or maybe a better approach to achieve the same goal? I need the translations to be saved in the database after the response is given to the user in the current language (I receive the current language project's description from the frontend) to optimize the response time.
Thanks in advance for your time!
It hard to say what you trying to achieve and how you are doing it, so this answer might be useless. I am not sure how you are disposing the context, but you have to make sure you dispose it after all the calls are made.
the most straightforward approach it to have
public async Task Add(Project project)
{
using (var projectsContext= new ProjectsContext())
{
// Perform data access using the context
projectsContext.Projects.Add(project);
projectsContext.SaveChanges();
await AddProjectDescriptionTranslations(project, projectsContext);
}
}
}
private async Task AddProjectDescriptionTranslations(Project project, ProjectsContext projectsContext)
{
await project.TranslateDescription(translator, headers.LanguageEnum);
projectsContext.Projects.Update(project);
projectsContext.SaveChanges();
}
So, not sure if it's the best approach but I was able to solve it the following way:
Added another constructor for ProjectContext and OnConfiguring method:
public ProjectsContext(string connectionString)
{
this.connectionString = connectionString;
}
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
if (connectionString != null)
{
optionsBuilder.UseMySql(connectionString);
}
}
Added ProjectContextFactory:
public class ProjectsContextFactory
{
private readonly EnvConfig envConfig;
public ProjectsContextFactory(IOptions<EnvConfig> envConfig)
{
this.envConfig = envConfig.Value;
}
public ProjectsContext CreateProjectsContext()
{
return new ProjectsContext(envConfig.PROJECTS_DB_CONNECTION_STRING_MASTER);
}
}
Made the following changes in Add and AddTranslations methods:
public void Add(Project project)
{
projectsContext.Projects.Add(project);
projectsContext.SaveChanges();
AddProjectDescriptionTranslations(project);
}
private async Task AddProjectDescriptionTranslations(Project project)
{
using (var context = projectsContextFactory.CreateProjectsContext())
{
context.Attach(project);
await project.TranslateDescription(translator, headers.LanguageEnum);
context.Projects.Update(project);
context.SaveChanges();
}
}
It allows me to save the project with the current language description and return API response before the description gets translated into other languages and gets saved to the database.
The way I am utilising the MVC pattern at the moment in my ASP.NET application (using Entity Framework) is as follows:
1) My Models folder contains all EF entities, as well as my ViewModels
2) I have a Helpers folders where I store classes created for the purposes of the particular application.
3) In my Helpers folder, I have a static class named MyHelper which contains methods that access the DB using EF.
namespace myApp.Helpers
{
public static class MyHelper
{
public static async Task<ProductVM> GetProductAsync(int productId)
{
using (var context = new myEntities())
{
return await context.vwxProducts.Where(x => x.ProductId == productId).Select(x => new ProductVM { A = x.A, B = x.B }).FirstOrDefaultAsync();
}
}
}
}
4) My controllers then call these functions where necessary:
namespace myApp.Controllers
{
public class ProductController : Controller
{
[HttpGet]
public async Task<ActionResult> Index(int productId)
{
var productVM = await MyHelper.GetProductAsync(productId);
return View(productVM);
}
}
}
I usually encounter comments in SO of the type "don't use a static class, static classes are evil, etc". Would this apply in such a scenario? If yes, why? Is there a better 'structure' my app should follow for best practices and for avoiding such pitfalls?
You can't really use a static class for this. Your Entity Framework context should have one and only one instance per request. Your methods here instantiate a new context for each method, which is going to cause a ton of problems with Entity Framework.
The general concept is fine, but your MyHelper class should be a normal class. Add a constructor that takes an instance of your context, and then use a DI container to inject the context into the helper class and the helper class into your controller.
UPDATE
Helper
namespace myApp.Helpers
{
public class MyHelper
{
private readonly DbContext context;
public MyHelper(DbContext context)
{
this.context = context;
}
public async Task<ProductVM> GetProductAsync(int productId)
{
return await context.vwxProducts.Where(x => x.ProductId == productId).Select(x => new ProductVM { A = x.A, B = x.B }).FirstOrDefaultAsync();
}
}
}
Controller
namespace myApp.Controllers
{
public class ProductController : Controller
{
private readonly MyHelper myHelper;
public ProductController(MyHelper myHelper)
{
this.myHelper = myHelper;
}
[HttpGet]
public async Task<ActionResult> Index(int productId)
{
var productVM = await myHelper.GetProductAsync(productId);
return View(productVM);
}
}
}
Then, you just need to set up a DI container to inject everything. The code for that is entirely dependent on which container you end up going with, so I can't really help you further. It's usually pretty straight-forward, though. Just read the docs for the container. You'll want to set the life-time scope of your objects to the request. Again, it's different for different containers, but they'll all have some sort of request-scope.
I was thinking to add comment to ChrisPratt's answer, but it ended being too long, so let me add separate answer.
Basically, this is not a life/death choice. Sure, static methods are not as flexible as classes for db access. But they are not bad per-se. One DbContext per request is a something to aim for. It is not an absolute must. It is kinda like dependency injection - you get more flexibility and in turn increase code complexity.
Look at these three questions and their answers, by taking into account everything they say, I'm sure you'll be able to answer your question yourself:
Why would I use static methods for database access
When to use static classes in C#
One DbContext per web request... why?
EDIT: Chris left good comment on my answer and I've changed answer a bit to take into account what he said.
Your idea is correct and I use it always. But the style is like this:
1) For each entity (i.e User) we have a static class inside Providers folder. In this class we can do general methods (i.e create, Get, GetAll , ..)
public static class Users
{
public static IEnumerable<kernel_Users> GetAll()
{
Kernel_Context db = new Kernel_Context();
return db.kernel_Users;
}
public static kernel_Users Get(int userId)
{
Kernel_Context db = new Kernel_Context();
return db.kernel_Users.Where(c => c.UserId == userId).FirstOrDefault();
}
...
}
2) We have another class that is not static.It is inside Models folder. This is the place that we can access to an instance of the entity :
public partial class kernel_Users
{
[Key]
public int UserId { get; set; }
public string Username { get; set; }
public string Password { get; set; }
[NotMapped]
public string FullName
{
get
{
return FirstName + " " + LastName;
}
}
public bool Delete(out string msg)
{
...
}
...
}
I use a static class that has the context injected into a static constructor for the purposes of loading a cache of data that rarely changes. And it (should) be thread safe. I hope this helps you, it's very handy in my experience:
public static class StaticCache<T> where T: class
{
private static List<T> dbSet;
public static Dictionary<string, List<T>> cache = new Dictionary<string, List<T>>();
private static readonly object Lock = new object();
public static void Load(DbContext db, string connStr, string tableName)
{
lock (Lock)
{
try
{
if (connStr != null)
{
using (db)
{
dbSet = db.Set<T>().ToList();
cache.Add(tableName, dbSet);
}
}
}
catch { }
}
}
}
void Testit()
{
var context = new YourContextSubClass(connStr);
StaticCache<TableEntity>.Load(context, connstr, "tableEntityNameString");
}
I have read all sorts of posts on this subject but couldn't find the answer to my question.
The general consensus is that I should be creating a context for 1 unit of work (say 1 web page). I have enclosed every method in my Database.cs (see below) with 'using' hence to me - that implies that each time a method from this class is called - a context is created. So if I was to call 2 Methods from Database.cs from the same Action in the HomeController.cs - would that mean that 2 contexts are created?
Would it not be better to declare a private field inside Database.cs like so:
private Entities db = new Entities()
And have each method within the Database.cs class access it? Which approach is the best?
My current implementation (I'm only going to include the method Verify but there are many methods in the Database class):
HomeController.cs
[AllowAnonymous]
public class HomeController : Controller
{
private IDatabase Database;
public HomeController()
{
this.Database = new Database();
}
[HttpGet]
public ActionResult Verify(string id)
{
if (Database.VerifyUser(id))
{
return View();
}
else
{
ViewBag.Error = "There was an error with the verification process";
return View();
}
}
}
Database.cs
public class Database : IDatabase
{
... some other methods ...
public bool VerifyUser(string verificationHash)
{
using (Entities db = new Entities())
{
var userToVerify = db.VerifyUser(verificationHash);
int count = userToVerify.Count();
if (count == 1)
{
return true;
}
else
{
return false;
}
}
}
}
db.VerifyUser(..) - this is a call to a stored procedure
Yes that means there are two instances of DbContext.
The better is to have one instance of DbContext in your Database class and use this instance in all your methods.
public class Database : IDatabase, IDisposeable
{
private Entities db;
public Database()
{
db = new Entities()
}
... some other methods ...
public bool VerifyUser(string verificationHash)
{
var userToVerify = db.VerifyUser(verificationHash);
int count = userToVerify.Count();
if (count == 1)
{
return true;
}
else
{
return false;
}
}
public void Dispose()
{
db.Dispose()
}
}
Then when you finish from Database instance you dispose it and it will dispose the DbContext
public class HomeController : Controller
{
private IDatabase Database;
public HomeController()
{
this.Database = new Database();
}
[HttpGet]
public ActionResult Verify(string id)
{
using(this.Database)
{
if (Database.VerifyUser(id))
{
return View();
}
else
{
ViewBag.Error = "There was an error with the verification process";
return View();
}
}
}
}
BTW: you may prefer to dispose your resources at the controller level. In that case, you don't need to add using statement in your actions
e.g. add this to your controller:
protected override void Dispose(bool disposing)
{
this.Database.Dispose();
base.Dispose(disposing);
}
Yes in your design DbContext created and disposed in every method calls.
Actually, it is not a good solution to put all database operations to a class and create DbContext over and over again. You probably have a problem in future with that class. It might have hundred methods in time and so it is hard to maintain and all entities are not related with each other semantically so it may cause confusion. I think it is a better solution to seperate entity types into classes. For example, you have an users, projects, departments. If we apply my solution to these entities then the uml class diagram will be like this.
All repositories takes a reference to DbContext. It is called Dependency Injection. It means that dbcontext is instantiated once and passes its reference through necessary repositories so there are no context re-creation. Also there is a generic repository which you can put standard procedures.
So you can use repositories like this.
[HttpGet]
public ActionResult Verify(string id){
using(var context = new DbContext())
{
var userRepo = new UserRepository(context);
//Department repository can be used over the same context.
var departmentRepo = new DepartmentRepository(context);
if(userRepo.verifyUser(id)){
return View();
}
}
}
I have a .Net Solution with multiple projects for the layers which are as follows.
Business (solution folder)
Core (project)
Interfaces (project)
Data (solution folder)
Data (project)
Presentation (solution folder)
AdminPanel (project)
Shared (solution folder)
Common (project)
I have implemented the generic repository pattern with IoC layer using Unity i.e. Microsoft.Practices.Unity. Everything works great except when I want to do multi table simple or complex joins. I have tried many different ways. I have gone through every single existing thread here on SO regarding joins in repository but none of them are helpful in what I need.
Here's my repository class.
public class Repository<TEntity> : IRepository<TEntity> where TEntity : class, IEntity
{
protected BuyatimeshareModel Context { get; private set; }
public Repository(IContextFactory contextFactory)
: this(contextFactory.Get())
{
}
protected Repository(BuyatimeshareModel context)
{
context.Database.Log = message => { Common.Logging.Log(message); };
Context = context;
}
IDbSet<TEntity> DbSet
{
get
{
return Context.Set<TEntity>();
}
}
public TEntity Add(TEntity instance)
{
DbSet.Add(instance);
Context.SaveChanges();
return instance;
}
public void Remove(TEntity instance)
{
DbSet.Remove(instance);
Context.SaveChanges();
}
public TEntity FindOne(Expression<Func<TEntity, bool>> predicate)
{
return DbSet.AsQueryable().FirstOrDefault(predicate);
}
public IEnumerable<TEntity> All()
{
return DbSet.AsQueryable();
}
public IEnumerable<TEntity> Query() { IQueryable<TEntity> query = DbSet; return query.ToList(); }
public IEnumerable<TEntity> FindAll(Expression<Func<TEntity, bool>> predicate)
{
return DbSet.AsQueryable().Where(predicate);
}
public int Count()
{
return DbSet.AsQueryable().Count();
}
public int Count(Expression<Func<TEntity, bool>> predicate)
{
return DbSet.AsQueryable().Count(predicate);
}
public bool Exists(Expression<Func<TEntity, bool>> predicate)
{
return DbSet.AsQueryable().Any(predicate);
}
}
Here's my IoC layer.
public class UnityControllerFactory : DefaultControllerFactory
{
IUnityContainer container;
public UnityControllerFactory(IUnityContainer container)
{
this.container = container;
}
protected override IController GetControllerInstance(RequestContext requestContext, Type controllerType)
{
try
{
if (controllerType == null)
throw new ArgumentNullException("controllerType");
if (!typeof(IController).IsAssignableFrom(controllerType))
throw new ArgumentException(string.Format(
"Type requested is not a controller: {0}", controllerType.Name),
"controllerType");
return container.Resolve(controllerType) as IController;
}
catch { return null; }
}
public static void Configure()
{
IUnityContainer container = new UnityContainer();
/*string connectionString = ConfigurationManager.ConnectionStrings["Connection"].ConnectionString;*/
container.RegisterType<IContextFactory, ContextFactory>(new ContainerControlledLifetimeManager())//, new InjectionConstructor(connectionString))
.RegisterType<IUnitOfWork, UnitOfWork>(new ContainerControlledLifetimeManager())
.RegisterType<IAdminService, AdminService>()
.RegisterType(typeof(IRepository<>), typeof(Repository<>));
ControllerBuilder.Current.SetControllerFactory(new UnityControllerFactory(container));
}
}
And finally here's my service layer where I'm trying to query some data from the repository layer.
public class AdminService : IAdminService
{
private readonly IRepository<press_releases> pressReleasesRepo;
private readonly IRepository<tblads> adsRepo;
private readonly IRepository<tblresorts> resortsRepo;
public AdminService(IRepository<press_releases> _pressReleasesRepo, IRepository<tblads> _adsRepo, IRepository<tblresorts> _resortsRepo)
{
pressReleasesRepo = _pressReleasesRepo;
adsRepo = _adsRepo;
resortsRepo = _resortsRepo;
}
public List<press_releases> Test()
{
var data = pressReleasesRepo.FindAll(p => p.pr_id > 0);
var data1 =
(from a in adsRepo.Query()
join r in resortsRepo.Query() on a.resort_id equals r.resort_id
where a.ad_id == 413
select new
{
OwnerId = a.owner_id,
ResortName = r.name,
AdId = a.ad_id,
AskingPrice = a.askingPriceInt
}).ToList();
var model = data.ToList();
return model;
}
}
Now here we have two queries. data and data1
data is simply querying one single table with a criteria and its returning results as expected. All good here.
But, in data1, it is also technically returning exactly the end result that I want BUT I also have a raw SQL logger in place to see what's happening behind the scenes and what it's doing is to query both ads table and resorts table separately using a select * equivalent statement and then when the results return from both tables, it's applying the joins within memory and returning the result after filtering through the where clause. So basically its scanning through around 100000 rows even though I have join and where clause in place and in the end, it returns one single row with ad id 413.
One thing to note down is that I'm calling Query method of the repository class while doing the join for data1 which is returning an IEnumerable. I was not able to change it to IQueryable because then an exception was being thrown having some message like the linq query has references to different contexts.
Can anyone guide me to modify this code so that I can apply real sql joins between the repositories or if there's a way to modify this code by adding a middle layer to act as a bridge. I need something to go forward from this point. I've been looking everywhere but no avail. I'm sure that I'm not the first one ever in this world who could have this problem. There definitely would be others having similar issue and may be someone who have found a real way.
My tools and technologies are as follows.
Visual Studio 2013
.Net framework 4.0
MySQL Database
MySQL for Visual Studio 1.2.4
MySQL Connector 6.9.6
Entity Framework 6
Windows 10 Pro x64
Please let me know if there's anything that I might have missed.
Any help would be greatly appreciated.
Edit:
Here's the ContextFactory class
public class ContextFactory : IContextFactory
{
private bool _isDisposed;
private SomeModel _context;
public ContextFactory()
{ }
public SomeModel Get()
{
_context = new SomeModel();
return _context;
}
~ContextFactory()
{
Dispose(false);
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
protected virtual void Dispose(bool disposing)
{
if (!_isDisposed && disposing && (_context != null))
_context.Dispose();
_isDisposed = true;
}
}
Here is the method you have in your repository which you are using in your data1.
public IEnumerable<TEntity> Query() { IQueryable<TEntity> query = DbSet; return query.ToList(); }
Whenever you hit '.ToList()' it will hit the database.
If you want to create the query first and then execute then use 'DbSet' property of your repository.
I'm using the repository pattern with a context and ninject as the IOC. I have a service which handles getting and setting page properties in the database.
public class MyContext : DbContext
{
public MyContext() : base ("DefaultConnection")
{
}
public DbSet<PageProperty> PageProperties { get; set; }
public DbSet<Contact> Contacts { get; set; }
}
public class DefaultRepository : IRepository
{
MyContext _context;
public DefaultRepository(MyContext context)
{
_context = context;
}
public IQueryable<PageProperty> PageProperties { get { return _context.PageProperties; } }
public IQueryable<Contact> Contacts { get { return _context.Contacts; } }
}
public class ModuleLoader : NinjectModule
{
public ModuleLoader()
{
}
public override void Load()
{
var context = new MyContext();
context.Database.Initialize(false);
Bind<MyContext>().ToConstant(context).InSingletonScope();
Bind<IRepository>().To<DefaultRepository>();
Bind<IPagePropertyProvider>().To<DefaultPagePropertyProvider>().InSingletonScope();
}
}
public class DefaultPagePropertyProvider : IPagePropertyProvider
{
IRepository _repository;
object _syncLock = new object();
public DefaultPagePropertyProvider (IRepository repository)
{
_repository = repository;
}
public string GetValue(string pageName, string propertyName
{
lock (_syncLock)
{
var prop = page.PageProperties.FirstOrDefault(x => x.Property.Equals(propertyName) && x.PageName.Equals(pageName)).Value;
return prop;
}
}
public void SetValue(string pageName, string propertyName, string value)
{
var pageProp = _repository.PageProperties.FirstOrDefault(x => x.Property.Equals(propertyName) && x.PageName.Equals(pageName));
pageProp.Value = value;
_repository.SaveSingleEntity(pageProp);
}
}
In my view I am doing 3 ajax calls, one to get a list from contacts to fill out a table, one ajax call to determine how many pages i have depending on the page size I'm using, and one ajax call to set the page size that I want to use. so a select box changes the page size (How many contacts per page: [ 30]) , a table that displays the contacts (generated from jquery which decifers json), and finally a div containing a list of page numbers to click. The workflow is, call GetContacts(), this function then queries the PageProperties to find out the page size to use, then call GetPages(), this function also queries PageProperties to find out what page size to use, SetPageSize() which sets the page size. So GetContacts() and GetPages() is used when a page is selected from the div, SetPageSize() then GetContacts() and GetPages() is called when the select box change event is fired. GetContacts() and GetPages() is only called when the first SetPageSize() $.ajax request is done() and there is a success from that function.
Now, before I added lock(syncLock) in the DefaultPageProperty service and before I added InSingletonScope to both that service and the context, I was getting two errors.
The connection was not closed. The connection's current state is connecting.
An EdmType cannot be mapped to CLR classes multiple times
I assumed because the connection was in a connecting state, that the context was being reused and reused and reused, so I thought putting that to SingletonScope() would mean that only one connection was made, then I thought the same about DefaultPageProperty and then because I was making async calls to that service, I should put a lock over the database querying.
It works, and the problems don't exist. But I don't know if what I have done is correct within the pattern I'm using, I'm wondering if I've missed something fundamental? My question is, is this a proper/viable solution which won't create any caveats later down the road? Have I actually solved the issue or just created more?
I redesigned the way I do my context now.
I have my context then I implement IDbContextFactory<TContext> called DefaultContextFactory<MyContext> and I inject them.
In the Repository I have in the public constructor _context = contextFactory.Create();.
Then throughout the repository i just use _context.WhatEver() and its fine.
I also did in the ModuleLoader Bind<IRepository>().To<DefaultRepository>().InTransientScope() in order to make every call to it create a new repository!
I don't need a repository factory because I only have one repository!