In the process of refactoring an ASP.NET MVC 5 web project, I see an opportunity to move some select lists to another class where multiple controllers can access them. This would allow me to remove duplicate code.
In this instance, the select lists require a trip to the database. To hand-code the lists, which might change over time, would not be feasible (hence, the database query).
Although I have no compiler errors and the page appears to work as intended, I am not sure if I am creating other problems by taking this approach. Is the code approach shown below a "bad" way to achieve this outcome? Is there a better way to do this?
In summary, this is what I am doing:
The class and methods within are static
A private static readonly database context is defined at the top
The two functions shown query the database and produce the desired results.
Because this is a static class, there is no dispose method.
The Class:
public static class ElpLookupLists
{
private static readonly EllAssessmentContext Db = new EllAssessmentContext();
// code...
internal static IEnumerable<SelectListItem> StandardSelectList(string selectedDomain)
{
return Db.ElpStandardLists.Where(m => m.Domain == selectedDomain)
.Select(m => m.Standard).Distinct()
.Select(z => new SelectListItem { Text = z.ToString(), Value = z.ToString() }).OrderBy(z => z.Value)
.ToList();
}
internal static IEnumerable<SelectListItem> PerformanceIndicatorSelectList(string selectedDomain, int? selectedStandard,
string selectedSubConcept)
{
var query =
Db.ElpStandardLists.Where(m => m.Domain == selectedDomain).Where(m => m.Standard == selectedStandard);
if (!string.IsNullOrEmpty(selectedSubConcept)) query = query.Where(m => m.SubConcept == selectedSubConcept);
var list =
query.Select(m => m.PerformanceIndicator)
.Distinct().OrderBy(m => m)
.Select(z => new SelectListItem { Text = z.ToString(), Value = z.ToString() })
.OrderBy(z => z.Text).ToList();
return list;
}
}
In my opinion a better alternative would be to create a separate controller with methods to get this data, and you could OutputCache this method. You can then call this method in other controllers, and it won't make the database trip every time. The return value will be cached. You can control the cache settings of course.
The advantage of this technique over yours is that in your case, the database trip will always happen when the application starts because the method is static, irrespective of whether or not you are going to use it. Whereas by using a cached method, you make the database trip the first time you call the method.
Related
I know my title sounds a little bit confusing but here's the better explanation:
I have an API made with ASP.NET CORE and .NET 5 where I use the repository pattern to perform all my DB queries.
For one of these queries I need to have some kind of calculation made and set the result of that calculation as the value of one of the entity's property in the database before it returns the entire object, but the calculation needs to happen before the query returns because I need the query to also return the new value inside the object.
In other words, here's what I had so far:
public async Task<IReadOnlyList<Place>> GetAllPlacesByCategoryWithRelatedDataAndFilters(Guid id, string city)
{
// I need the ratio to be calculated before I return "places"
var places = await context.Places
.Include(d => d.Category)
.Where(d => d.CategoryId == id)
.ToListAsync();
foreach(var place in places)
{
// Ratio is a double
place.Ratio = CalculateRatioByMultiplying(place.Amount1, place.Amount2);
// Here's where I don't know how to really update the value inside of the
// database, if I leave it like this then it will show the new value when
// the query is performed but the value is still the default value in the
// database
// I've also tried this but this time nothing happens,
// it doesn't even throw an Exception or anything and the value
// is still the default value in the database
var ratio = CalculateRatioByMultiplying(place.Amount1, place.Amount2);
place.Ratio = ratio;
await placeRepository.UpdateAsync(place);
// Also the CalculateRatioByMultiplying is a method I've added to the
// IPlaceRepository and so in this PlaceRepository too that just
// multiplies arg1 with arg2 and returns a double
}
// Re-do the query with the new Ratio values for each of the places
var places = await context.Places
.Include(d => d.Category)
.Where(d => d.CategoryId == id)
.Where(d => d.Ratio != 0
&& d.City == city)
.OrderByDescending(d => d.Ratio)
.Take(10)
.ToListAsync();
return places;
}
However, I'm also using the mediator pattern to query from the controllers ( I'm using clean architecture ) and some times I get confused as to should the calculation be made in the "GetAllPlacesByCategoryWithRelatedDataAndFiltersQueryHandler" ( where the query is called ) or in the repository directly.
Unfortunately I can't share the project or any sample code as it is a private project but I can answer any question you have of course.
Thanks for your help
You're trying to update something (executing a command) while querying the data. Not sure if you're using CQRS or not, but it's still a separation of concerns problem.
May I suggest you entirely remove the update of place.Ratio from this query. And in all the commands that might have effects on place.Ratio, you could publish/dispatch a PlaceRatioRequiredChangedEvent event (it's an INotification for MediatR) then handle this event/notification.
I know it's a handful of work, but it'll set a better business flow for later development.
I have an Entity Framework v5 model created from a database. The table Season has a corresponding entity called Season. I need to calculate the Season's minimum start date and maximum end date for each year for a Project_Group. I then need to be able to JOIN those yearly min/max Season values in other LINQ queries. To do so, I have created a SeasonLimits class in my Data Access Layer project. (A SeasonLimits table does not exist in the database.)
public partial class SeasonLimits : EntityObject
{
public int Year { get; set; }
public DateTime Min_Start_Date { get; set; }
public int Min_Start_Date_ID { get; set; }
public DateTime Max_End_Date { get; set; }
public int Max_End_Date_ID { get; set; }
public static IQueryable<SeasonLimits> QuerySeasonLimits(MyEntities context, int project_Group_ID)
{
return context
.Season
.Where(s => s.Locations.Project.Project_Group.Any(pg => pg.Project_Group_ID == project_Group_ID))
.GroupBy(x => x.Year)
.Select(sl => new SeasonLimits
{
Year = sl.Key,
Min_Start_Date = sl.Min(d => d.Start_Date),
Min_Start_Date_ID = sl.Min(d => d.Start_Date_ID),
Max_End_Date = sl.Max(d => d.End_Date),
Max_End_Date_ID = sl.Max(d => d.End_Date_ID)
});
}
}
// MVC Project
var seasonHoursByYear =
from d in context.AuxiliaryDateHours
from sl in SeasonLimits.QuerySeasonLimits(context, pg.Project_Group_ID)
where d.Date_ID >= sl.Min_Start_Date_ID
&& d.Date_ID < sl.Max_End_Date_ID
group d by new
{
d.Year
} into grp4
orderby grp4.Key.Year
select new
{
Year = grp4.Key.Year,
HoursInYear = grp4.Count()
};
In my MVC project, whenever I attempt to use the QuerySeasonLimits method in a LINQ query JOIN, I receive the message,
"LINQ to Entities does not recognize the method
'System.Linq.IQueryable`1[MyDAL.SeasonLimits]
QuerySeasonLimits(MyDAL.MyEntities, MyDAL.Project_Group)' method, and
this method cannot be translated into a store expression."
Is this error being generated because SeasonLimits is not an entity that exists in the database? If this can't be done this way, is there another way to reference the logic so that it can be used in other LINQ queries?
EF is trying to translate your query to SQL and as there is no direct mapping between your method and the generated SQL you're getting the error.
First option would be not to use the method and instead write the contents of the method directly in the original query (I'm not sure at the moment if this would work, as I don't have a VS running). In the case this would work, you'll most likely end up with a very complicated SQL with a poor performance.
So here comes the second option: don't be afraid to use multiple queries to get what you need. Sometimes it also makes sense to send a simpler query to the DB and continue with modifications (aggregation, selection, ...) in the C# code. The query gets translated to SQL everytime you try to enumerate over it or if you use one of the ToList, ToDictionary, ToArray, ToLookup methods or if you're using a First, FirstOrDefault, Single or SingleOrDefault calls (see the LINQ documentation for the specifics).
One possible example that could fix your query (but most likely is not the best solution) is to start your query with:
var seasonHoursByYear =
from d in context.AuxiliaryDateHours.ToList()
[...]
and continue with all the rest. This minor change has fundamental impact:
by calling ToList the DB will be immediately queried and the whole
AuxiliaryDateHours table will be loaded into the application (this will be a performance problem if the table has too many rows)
a second query will be generated when calling your QuerySeasonLimits method (you could/should also include a ToList call for that)
the rest of the seasonHoursByYear query: where, grouping, ... will happen in memory
There are a couple of other points that might be unrelated at this point.
I haven't really investigated the intent of your code - as this could lead to further optimizations - even total reworks that could bring you more gains in the end...
I eliminated the SeasonLimits object and the QuerySeasonLimits method, and wrote the contents of the method directly in the original query.
// MVC Project
var seasonLimits =
from s in context.Season
.Where(s => s.Locations.Project.Project_Group.Any(pg => pg.Project_Group_ID == Project_Group_ID))
group s by new
{
s.Year
} into grp
select new
{
grp.Key.Year,
Min_Start_Date = grp.Min(x => x.Start_Date),
Min_Start_Date_ID = grp.Min(x => x.Start_Date_ID),
Max_End_Date = grp.Max(x => x.End_Date),
Max_End_Date_ID = grp.Max(x => x.End_Date_ID)
};
var seasonHoursByYear =
from d in context.AuxiliaryDateHours
from sl in seasonLimits
where d.Date_ID >= sl.Min_Start_Date_ID
&& d.Date_ID < sl.Max_End_Date_ID
group d by new
{
d.Year
} into grp4
orderby grp4.Key.Year
select new
{
Year = grp4.Key.Year,
HoursInYear = grp4.Count()
};
On a web-app I'm working on it usingNHibernate with ASP.NET-MVC, we were experiencing some major lag on page loads, so I was given the task to look into it and do some refactoring.
When I looked through the code, there were many calls like this:
public static IEnumerable<Client> GetAllClients()
{
return new EntityManager<Client>()
.GetAll<Client>()
.Where(x => x.IsDeleted == false || x.IsDeleted == null);
}
After some debugging and searching I found out that this actually performs 2 separate calls to the database, one to GetAll then it applies the Where and does a second call, or something like that. I found another question on SO which made me decide to use a session instead:
public static IEnumerable<Client> GetAllClients()
{
using (ISession session = NHibernateSessionFactoryManager.Factory.OpenSession())
{
return session.QueryOver<Client>()
.Where(x => x.IsDeleted == false || x.IsDeleted == null)
.List();
}
}
this was a lot nicer as it took the Where clause into consideration before going to the database. However, there are still methods that I see calling GetAll() but then applying another Where on top of that:
var Clients = GetAllClients().Where(n => n.Location == "someCity");
My guess is this first calls the GetAll() using the session, then it applies the new Where and goes back to the database again? Is this true?
is there a better/more elegant way to do this?
It's a very common problem when you over abstract the data access.
The List method call in GetAllClients method will cause all client records being loaded into memory from database.
I would suggest using one NHibernate ISession per request, instead of one ISession per method call. And make your methods returning IQueryable instead of IEnumerable. Something like this (NHibernate 3.0+):
using NHibernate;
using NHibernate.Linq;
public class EntityManager<T> {
private ISession _session;
public EntityManager(ISession session) {
_session = session;
}
public IQueryable<T> GetAllClients() {
return _session.Query<T>();
}
}
Then you can filter by country:
GetAllClients().Where(it => it.Location == "city").ToList();
Or doing projection:
var clientShortInfos = GetAllClients().Where(it => it.Location == "city")
.Select(it => new
{
Id = it.Id,
FullName = it.FullName
})
.ToList();
This allows you to retrieve only needed fields instead of all fields from database.
Here are some helpful posts:
http://ayende.com/blog/3955/repository-is-the-new-singleton
http://ayende.com/blog/4101/do-you-need-a-framework
It depends on what is the result type of GetAll of EntityManager. If it is IQueryable<Clinet> then the where will be applied before hitting the database, if it is IEnumerable<Client>, or list, or array then Where will be performed in memory.
Because GetAllClients returns IEnumerable<Client> the subsequent calls to Where will be performed in memory.
Also, in the second example you probably want to use Query (Linq) instead of QueryOver. Linq and QueryOver are different query engines.
Yes, You absolutely made correct solution. I don't know much about NHibernate but this case resembles with Entity Framework as well. Using session will avoid hit on server and prepare a query first. Once query is prepared then it will hit server.
You can also find an example of same where clause in NHibernate over here
Where clause in NHibernate
You can also find a good tutorial related to IQueryable concept in Nhibernate over here.
http://ayende.com/blog/2227/implementing-linq-for-nhibernate-a-how-to-guide-part-1
I'm using the following queries to detect duplicates in a database.
Using a LINQ join doesn't work very well because Company X may also be listed as CompanyX, therefore I'd like to amend this to detect "near duplicates".
var results = result
.GroupBy(c => new {c.CompanyName})
.Select(g => new CompanyGridViewModel
{
LeadId = g.First().LeadId,
Qty = g.Count(),
CompanyName = g.Key.CompanyName,
}).ToList();
Could anybody suggest a way in which I have better control over the comparison? Perhaps via an IEqualityComparer (although I'm not exactly sure how that would work in this situation)
My main goals are:
To list the first record with a subset of all duplicates (or "near duplicates")
To have some flexibility over the fields and text comparisons I use for my duplicates.
For your explicit "ignoring spaces" case, you can simply call
var results = result.GroupBy(c => c.Name.Replace(" ", ""))...
However, in the general case where you want flexibility, I'd build up a library of IEqualityComparer<Company> classes to use in your groupings. For example, this should do the same in your "ignore space" case:
public class CompanyNameIgnoringSpaces : IEqualityComparer<Company>
{
public bool Equals(Company x, Company y)
{
return x.Name.Replace(" ", "") == y.Name.Replace(" ", "");
}
public int GetHashCode(Company obj)
{
return obj.Name.Replace(" ", "").GetHashCode();
}
}
which you could use as
var results = result.GroupBy(c => c, new CompanyNameIgnoringSpaces())...
It's pretty straightforward to do similar things containing multiple fields, or other definitions of similarity, etc.
Just note that your defintion of "similar" must be transitive, e.g. if you're looking at integers you can't define "similar" as "within 5", because then you'd have "0 is similar to 5" and "5 is similar to 10" but not "0 is similar to 10". (It must also be reflexive and symmetric, but that's more straightforward.)
Okay, so since you're looking for different permutations you could do something like this:
Bear in mind this was written in the answer so it may not fully compile, but you get the idea.
var results = result
.Where(g => CompanyNamePermutations(g.Key.CompanyName).Contains(g.Key.CompanyName))
.GroupBy(c => new {c.CompanyName})
.Select(g => new CompanyGridViewModel
{
LeadId = g.First().LeadId,
Qty = g.Count(),
CompanyName = g.Key.CompanyName,
}).ToList();
private static List<string> CompanyNamePermutations(string companyName)
{
// build your permutations here
// so to build the one in your example
return new List<string>
{
companyName,
string.Join("", companyName.Split(" ".ToCharArray(), StringSplitOptions.RemoveEmptyEntries);
};
}
In this case you need to define where the work is going to take place i.e. fully on the server, in local memory or a mixture of both.
In local memory:
In this case we have two routes, to pull back all the data and just do the logic in local memory, or to stream the data and apply the logic piecewise. To pull all the data just ToList() or ToArray() the base table. To stream the data would suggest using ToLookup() with custom IEqualityComparer, e.g.
public class CustomEqualityComparer: IEqualityComparer<String>
{
public bool Equals(String str1, String str2)
{
//custom logic
}
public int GetHashCode(String str)
{
// custom logic
}
}
//result
var results = result.ToLookup(r => r.Name,
new CustomEqualityComparer())
.Select(r => ....)
Fully on the server:
Depends on your provider and what it can successfully map. E.g. if we define a near duplicate as one with an alternative delimiter one could do something like this:
private char[] delimiters = new char[]{' ','-','*'}
var results = result.GroupBy(r => delimiters.Aggregate( d => r.Replace(d,'')...
Mixture:
In this case we are splitting the work between the two. Unless you come up with a nice scheme this route is most likely to be inefficient. E.g. if we keep the logic on the local side, build groupings as a mapping from a name into a key and just query the resulting groupings we can do something like this:
var groupings = result.Select(r => r.Name)
//pull into local memory
.ToArray()
//do local grouping logic...
//Query results
var results = result.GroupBy(r => groupings[r]).....
Personally I usually go with the first option, pulling all the data for small data sets and streaming large data sets (empirically I found streaming with logic between each pull takes a lot longer than pulling all the data then doing all the logic)
Notes: Dependent on the provider ToLookup() is usually immediate execution and in construction applies its logic piecewise.
I am working on project that allows a user to add Time to a Task. On the Task I have a field for EstimatedDuration, and my thoughts are I can get ActualDuration from the Time added to the Task.
I have a LINQ2SQL class for the Task, as well as and additional Task class (using partials).
I have the following for my query so far:
public IQueryable<Task> GetTasks(TaskCriteria criteria)
{
// set option to eager load child object(s)
var opts = new System.Data.Linq.DataLoadOptions();
opts.LoadWith<Task>(row => row.Project);
opts.LoadWith<Task>(row => row.AssignedToUser);
opts.LoadWith<Task>(row => row.Customer);
opts.LoadWith<Task>(row => row.Stage);
db.LoadOptions = opts;
IQueryable<Task> query = db.Tasks;
if (criteria.ProjectId.HasValue())
query = query.Where(row => row.ProjectId == criteria.ProjectId);
if (criteria.StageId.HasValue())
query = query.Where(row => row.StageId == criteria.StageId);
if (criteria.Status.HasValue)
query = query.Where(row => row.Status == (int)criteria.Status);
var result = query.Select(row => row);
return result;
}
What would be the best way to get at the ActualDuration, which is just a sum of the Units in the TaskTime table?
Add a property to your Task partial class, similiar to this:
public int ActualDuration
{
get {
YourDataContext db = new YourDataContext();
return
db.TaskDurations.Where(t => t.task_id == this.id).
Sum (t => t.duration);
}
}
Then you can reference the actual duration as Task.ActualDuration.
Update: You asked about how to do this with a partial class. Of course it hits the database again. The only way to get data from the database that you don't know yet is to hit the database. If you need to avoid this for performance reasons, write a subquery or SQL function that calculates the actual duration, and use a Tasks view that includes the calculated value. Now, the function/query will still have to aggregate the entered durations for every task row in the result set, so it will still be performance-intensive. If you have a very large task table and performance issues, keep a running tally on the task table. I think that even the partial class solution is fine for several 100,000's of tasks. You rarely retrieve large numbers at once, I would assume. Grid controls with paging only get a page at the time.