I have this Model
public class CPMC
{
public int CPMCId { get; set; }
public List<TPM> tpm = new List<TPM>();
public List<TPMC> tpmc = new List<TPMC>();
}
public class TPMC
{
public int Id { get; set; }
public Int64 Amount { get; set; }
public int PId { get; set; }
public Int64 PAmount { get; set; }
public int CPMCId { get; set; }
}
public class TPM
{
public int Type { get; set; }
public int Id { get; set; }
public Int64 Amount { get; set; }
public int VAT { get; set; }
public DateTime FromDate { get; set; }
public DateTime ToDate { get; set; }
public int CPMCId { get; set; }
}
The data for this List is 5k records of CPMCId and 50k records for each child list inside with condition is
List<int> CPMCIdList = aPP.Select(x => Convert.ToInt32(x.CPMCId)).Distinct().ToList();
List<CPMC> cpl = (from ap in CPMCIdList
select new CPMC
{
CPMCId = ap,
tpm = tpml.Where(x=>x.CPMCId == ap).ToList(),
tpmc = tpmcl.Where(x=>x.CPMCId == ap).ToList()
}).ToList();
But it takes a lot of time to fill data in List. Can you guys have a better implement for this solution
Thanks in advance
Due to the two inner loop linear searches (LINQ Where operators), your current implementation has O(K*N*M) time complexity, where K=CPMCIdList.Count, N=tpml.Count, M=tpmcl.Count.
It can be reduced to the much faster O(K+M+N) by using the LINQ Group Join operators which internally use a quite efficient hash based lookup:
var cpl =
(from cpmcId in CPMCIdList
join tpm in tpml on cpmcId equals tpm.CPMCId into tpmg
join tpmc in tpmcl on cpmcId equals tpmc.CPMCId into tpmcg
select new CPMC
{
CPMCId = cpmcId,
tpm = tpmg.ToList(),
tpmc = tpmcg.ToList()
}).ToList();
First, let's reduce your problem to the minimum case:
You have the following types:
public class A
{
public int Id { get; set; }
public List<B> bs = new List<B>();
public List<C> cs = new List<C>();
}
public class B
{
public int CPMCId { get; set; }
}
public class C
{
public int CPMCId { get; set; }
}
Apparently, you have a list of A's, B's and Cs
List<A> as;
List<B> bs;
List<C> cs;
you're looking to create a list of A's
Now first let's take a look at why your solution is slow.
What you're doing is first creat a list of all the ID's you want, and then, for each ID, search all records that match. That means you're scanning the child lists entirely for every ID. That's clearly not optimal.
The operation you are looking for is called Outer Join in SQL. Unfortunately, Linq doesn't have an equivalent operation out of the box.
So we're going to that ourselves. It's possible to make a generic version of this approach, but it's not entirely straightforward. What we're going to do is sort the A's and the B's by their CPMCId, and then take all the matching records that have a corresponding ID in the list of As:
IEnumerable<A> make_as(IEnumerator<B> ordered_bs, IEnumerator<C> ordered_cs, IEnumerator<int> ordered_ids) {
//make sure the current element of bs and cs is at the first element, not before it.
if(!ordered_bs.MoveNext() || !ordered_cs.MoveNext())
throw new ArgumentException("empty bs or cs");
while(ordered_ids.MoveNext()) {
nextid = ordered_ids.Current;
var a = new A(){
id = nextId;
};
//process the B's
while(ordered_bs.Current.CPMCId < nextid) //not in the list, skip it {
ordered_bs.MoveNext();
}
while(ordered_bs.Current.CPMCId == nextid) //matching, add to the list {
a.bs.add(ordered_cs.Current);
if(!orderd_bs.MoveNext()) break; //move bs forward. If b's is empty, we're done here
}
//do the same for the C's
while(ordered_cs.Current.CPMCId < nextid) {
ordered_cs.MoveNext();
}
while(ordered_cs.Current.CPMCId == nextid) {
a.cs.add(ordered_cs.Current);
if(!ordered_cs.MoveNext()) break;
}
yield return a;
}
}
var result = make_as(
bs.orderBy(b => b.PCMCId).GetEnumerator(),
cs.orderBy(c => c.PCMCId).GetEnumerator(),
as.Select(a => a.id).OrderBy(id => id).Distinct().GetEnumerator()
).ToList()
Some notes:
I'm getting the impression that this is a part of a solution that already had some processing done. When you know that you're going to need all ID's, you don't need the original list of A's at all, and the nextId will be the lowest Current of the A's and Bs
It's also quite possible that right now you're in a bit of a hole you dug yourself in to. It's quite possible that you could do this more efficiently - and more elegantly - further "upstream" in your code.
As a last note, this snippet does not work when either the list of B's or the list of C's contain no elements. In that case, a simple GroupBy is sufficient.
Related
In a .NET Winforms application I'm trying to query using Entity Framework. I created Models with Db-Scaffold Database-First from SQL-Server in which i created all relationships.
Now combobox cb2 has Location as Datasource.
public partial class Location
{
public Location()
{
AktiveSchichten = new HashSet<AktiveSchichten>();
}
public int Id { get; set; }
public string Name { get; set; }
public virtual ICollection<AktiveSchichten> AktiveSchichten { get; set; }
}
cb2's SelectedValue holds the LocationId. Now I'm trying to get every Shift in Shifts where location equals SelectedValue location.
public partial class Shifts
{
public int Id { get; set; }
public TimeSpan Start { get; set; }
public TimeSpan End { get; set; }
public int Location { get; set; }
public virtual Location LocationNavigation { get; set; }
}
}
My approach
var shifts = dbContext.AktiveSchichten.Where(a => a.Location = cb2.SelectedValue);
foreach (var shift in shifts)
{
//..
}
why can i not do this with an int when
var shifts = dbContext.Shifts.Where(a => a.Location == (int)cb2.SelectedValue);
doesnt throw an error
Your approach is syntactically incorrect: you have an assignment in the Where clause
a.Location = cb2.SelectedValue instead of a comparison
a.Location == cb2.SelectedValue
and you're not casting cb2.SelectedValue to an int (compared to the working statement var shifts =...).
Contrary to jdweng's comment casts will work with with LINQ - as long as the cast is valid (cb2.SelectedValue is of type object and holds an int). But you might be better off to assign the value before the LINQ statement:
var locationId = (int)cb2.SelectedValue;
var shifts = dbContext.Shifts.Where(a => a.Location == locationId);
Furthermore: Do you really have AktiveSchichten and Shifts on your dbContext? Maybe you would like to consolidate your naming (German, English; plural or singular for classes; class name for a property not representing an instance of the class Shifts.Location) to make your code easier to understand.
What I am trying to do is combine two lists of different types into a new type, on the Id property of each. Both lists have different properties that I need in the new list.
This snippet is working already, but I am not happy with the performance. Assuming both lists are the same length, and in whatever order they need to be, is it possible to do this more efficiently?
class A
{
public int Id { get; set; }
public string stuffA { get; set; }
//other properties that we aren't using
}
class B
{
public int Id { get; set; }
public string stuffB { get; set; }
//other properties we aren't using
}
class C
{
public int Id { get; set; }
public string stuffA { get; set; }
public string stuffB { get; set; }
}
public List<C> getList(List<A> listA, List<B> listB)
{
var listC = new List<C>();
foreach(var a in listA)
{
var b = listB.Where(x => x.Id == a.Id);
listC.Add(new C{ Id = a.Id, stuffA = a.stuffA, stuffB = b.stuffB});
}
return listC;
}
I've looked into the Enumerable.Zip method, which pairs up two lists in the order provided, but I can't seem to use that with objects. The only examples I can make work are with primitive types.
I've seen this question: How merge two lists of different objects? but this doesn't create a new type, only an anonymous type containing the old lists (I believe).
Any ideas?
Your looking for a simple Linq join, the order of items doesn't matter. For example:
public List<C> getList(List<A> listA, List<B> listB)
{
var listC = from a in listA
join b in listB on a.Id equals b.Id
select new C
{
Id = a.Id,
stuffA = a.stuffA,
stuffB = b.stuffB
};
return listC.ToList();
}
If you can ensure that both lists are in the same order with the same number of elements, you can loop through the lists and join based on index. This eliminates any searches required:
public List<C> getList(List<A> listA, List<B> listB)
{
var listC = new List<C>();
for(var i = 0; i < listA.Count; i++)
{
listC.Add(new C
{
Id = listA[i].Id,
stuffA = listA[i].stuffA,
stuffB = listB[i].stuffB
};
}
return listC;
}
This sounds a bit like a homework question but anyway... the trivial solution has a complexity of O(n^2). if efficiency is your goal, you need to use a structure that allows for fast searching like a hashtable, then your solution will be O(n). so add the elements of the first array to a hash table, then loop thru the second array and compose the result.
I have these three entities:
public class Dog
{
public int DogId { get; set; }
public string Name { get; set; }
public int Age { get; set; }
public bool Checked { get; set; }
public string DogImage { get; set; }
public virtual ICollection<Result> Results { get; set; }
}
public class Event
{
public int EventId { get; set; }
public string EventName { get; set; }
public string EventLocation { get; set; }
public string EventType { get; set; }
public string EventDate { get; set; }
public virtual ICollection<Result> Results { get; set; }
}
public class Result
{
public int ResultId { get; set; }
public int Track { get; set; }
public int Obedience { get; set; }
public int Protection { get; set; }
[ForeignKey("Dog")]
public int DogId { get; set; }
public virtual Dog Dog { get; set; }
[ForeignKey("Event")]
public int EventId { get; set; }
public virtual Event Event { get; set; }
}
I´ve been getting help from here before in order to set it up like this.
Entity Framework errors when trying to create many-to-many relationship
So the way it is now I guess the result is the "glue" that ties these classes together containing foreign keys to the two other tables.
What I have been trying to achieve for days now is to:
Create an event.
Add dogs to the event.
Add results to the dogs participating in the choosenEvent.
Lets say I create an event like this:
[HttpPost]
public ActionResult CreateEvent(Event newEvent)
{
newEvent.EventDate = newEvent.EventDate.ToString();
_ef.AddEvent(newEvent);
return View();
}
Now I guess the next step would be to add a list of dogs to this event and in order to do that I need to somehow use my result-class since that's the "glue"-class. Please let me know if I'm even on the right track here.
It is not really a good idea to do many to many relationships like how you've done. See here
In order to get a proper many to many relationship, mapped in the proper way in the database, that doesn't have pitfalls, I would try it this way:
public class Dog {}
public class Event {}
public class Result {}
// This is a linking table between Dog and Results
public class DogResult
{
public int Id {get;set;}
public int DogId {get;set;}
public int ResultId {get;set;}
}
// This is a linking table between Events and Results
public class EventResult
{
public int Id {get;set;}
public int EventId {get;set;}
public int ResultId {get;set;}
}
When you now write your query you can do this:
using (var context = new DbContext())
{
var dogs = context.Dogs();
var dogResults = context.DogResults();
var results = context.Results();
var dogsAndResults = dogs.Join(
dogResults,
d => d.Id,
r => r.DogId,
(dog, dogResult) => new { dog, dogResult })
.Join(
results,
a => a.dogResult.ResultId,
r => r.Id,
(anon, result) => new { anon.dog, result });
}
It is a bit nasty looking, but it will give you back a list of anonymous objects containing a Dog and its related Result. But obviously it would be better to do this in a stored proc:
using (var context = new DbContext())
{
var results = context.Database.ExecuteStoreQuery<SomeResultDto>("SELECT * .... JOIN ... ");
}
This is cleaner, because you are using SQL.
This is a more complex way of dealing with it. But far more performant, especially if you understand fully how entity framework executes LINQ.
Obviously if you want to create these links:
using (var context = new DbContext())
{
context.Dogs.AddRange(dogs); // dogs being a list of dog entities
context.Results.AddRange(results); // events being a list of results entities
context.DogResults.AddRange(dogResults); // a list of the links
}
It is completely up to you how you create these links. To turn this into a sproc as well, you want to create some custom User Defined Table Types and use them as a Table Value Parameter.
var dogResults = dogs.SelectMany( d => results.Select ( r => new DogResult { DogId = d.Id, ResultId = r.Id } ) );
That is a beast of a LINQ query and basically it gets every dog and links it to every result. Run it in LinqPad and Dump the values.
I've only done this using the fluent method (when I was learning I found you can do everything in fluent, but not with annotations, so I've not looked into them), the following creates a many to many between my Unit entity and my UnitService entity:
modelBuilder.Entity<Unit>()
.HasMany<UnitService>(u => u.Services)
.WithMany(us => us.Units);
This code is in the protected override void OnModelCreating(DbModelBuilder modelBuilder) method.
In your case Event is Unit and Dog is UnitService.
Oh ooops, you don't need that at all, your 'join' table is your results table, in my case I don't care about the join table so its all hidden.
Maybe something like:
modelBuilder.Entity<Result>()
.HasMany<Event>(e => e.Results);
modelBuilder.Entity<Result>()
.HasMany<Dog>(d => d.Results);
I have a Comment and Votes related to the comment.
[Table("QAComment")]
public class QaComment : IEntity
{
[Key, Column("QACommentID")]
public int Id { get; set; }
// ...
public virtual ICollection<QaCommentVote> Votes { get; set; }
[NotMapped]
public int OverallVote { get; set; }
}
[Table("QACommentVote")]
public class QaCommentVote : IEntity
{
[Key, Column("QACommentVoteID")]
public int Id { get; set; }
[ForeignKey("QAComment")]
public int QaCommentId { get; set; }
public int Value { get; set; }
public virtual QaComment QaComment { get; set; }
}
I need to get comments with the sum of their votes, not pulling all votes to the application.
The ways I can see to achive this:
1. Make a database view for Commment and calc votes sum in there.
Cons: dont wanna make extra-views
2. Via LINQ:
var comments =
Set<QaComment>()
.Select(c => new QaComment() {/* assign every property once again and calc OverallVote */});
Cons: don't like to assign allproperties once again.
Is there a better way devoid of that cons?
UPDATE
This is what I want as a result of LINQ:
SELECT
qac.*,
(SELECT SUM(v.Value)
FROM QACommentVote v
WHERE v.QACommentID = qac.QACommentID) as OverallVote
FROM QAComment qac
You can fetch QaComment and the sum you're looking for separately as anonymous type and merge them into one object using LINQ to Objects:
var comments
= Set<QaComment>()
.Select(c => new { c, sum = c.Votes.Sum(v => v.Value))
.AsEnumerable() // to make next query execute as LINQ to Objects query
.Select(x => { x.c.OverallVote = x.sum; return x.c; })
.ToList();
But to make point clear: I haven't tested that :)
2 classes
public class Student
{
public int StudentID { get; set;}
public string Name { get; set;}
public List<Fee> Fees {get;set;}
}
public class Fee
{
public int FeeID { get; set;}
public decimal FeeAmount { get; set; }
}
let say there are 10 students objects Student[] stud = new Student[10]
if stud[0] has 2 fees ( Fee[2] ) and they are
FeeID=1, FeeAmount=54.23
FeeID=2, FeeAmount=234.98
if stud[1] has 1 fees ( Fee[2] ) and they are
FeeID=1, FeeAmount=9.99
if stud[2] has 3 fees ( Fee[3] ) and they are
FeeID=1, FeeAmount=123.45
FeeID=2, FeeAmount=67.89
FeeID=3, FeeAmount=987.65
I need to sort the Student Collections by TotalAmount(Fee Collection)
TotalAmount of Fee
stud[0] = 54.23+234.98=289.21
stud[1] = =9,99
stud[2] = 123.45+67.89+987.65=1178.99
there for after sorted it should become
stud[0] = 123.45+67.89+987.65=1178.99
stud[1] = 54.23+234.98=289.21
stud[2] = =9,99
It sounds like you just want:
stud = stud.OrderByDescending(x => x.Fees.Sum(fee => fee.FeeAmount)).ToArray();
Gotta love LINQ :)
A couple of things to note:
This will still only calculate the sum of the fees once per student
This will not currently handle null elements. Do you need to? (You seem to have a fixed array size... perhaps use List<Student> instead?)
Unless you actually need it as an array afterwards, just drop the ToArray call. Be aware that it will sort it every time you iterate through it unless you use ToArray or ToList though.
var results = stud.OrderByDescending(s => s.Fees.Sum(f => f.FeeAmount)).ToArray();
A simple Linq query should do the job:
stud =
(from s in stud
orderby s.Fees.Sum(f => f.FeeAmount)
select s)
.ToArray();
var students = new List<Student>() { .. add students here ... };
students.OrderBy(x => x.Fees.Sum(y => y.FeeAmount));
And if you use an old .net framework (without Linq) :
public class Student : IComparable
{
public int StudentID { get; set; }
public string Name { get; set; }
public List<Fee> Fees { get; set; }
private decimal TotalAmount
{
get
{
decimal total = 0;
if (Fees != null)
foreach (var fee in Fees)
total += fee.FeeAmount;
return total;
}
}
public int CompareTo(object obj)
{
//Ascending
//return TotalAmount.CompareTo((obj as Student).TotalAmount);
//Descending
return (obj as Student).TotalAmount.CompareTo(TotalAmount);
}
}
public class Fee
{
public int FeeID { get; set; }
public decimal FeeAmount { get; set; }
}
List<Student> students = new List<Student>();
...
students.Sort();
Linq is better...