Develop Reference

How to populate objects with relationship from datatable?

I am having trouble designing an approach for taking data from a CSV into business objects. I'm starting by parsing the CSV and getting each row into a DataTable and that is where my mental block starts.
I've got the following classes where APDistribution is considered a child of Voucher with a 1:Many relationship:
public class Voucher
{
public string GPVoucherNumber { get; set; }
public string VendorID { get; set; }
public string TransactionDescription { get; set; }
public string Title { get; set; }
public string DocNumber { get; set; }
public DateTime DocDate { get; set; }
public decimal PurchaseAmount { get; set; }
public IEnumerable<APDistribution> Distributions { get; set; }
}
public class APDistribution
{
public string AccountNumber { get; set; }
public decimal Debit { get; set; }
public decimal Credit { get; set; }
public string DistributionReference { get; set; }
}
My CSV looks like this. Several fields can repeat representing the Voucher transaction (Vendor, Title Invoice Number, Invoice Amount, etc), and some fields are the Distribution detail (Journal Account Code, Journal Amount).
I began by thinking I could use Linq to project onto my business objects but I'm failing to see how I can structure the query to do that in one pass. I find myself wondering if I can do one query to project into a Voucher collection, one to project into an APDistribution collection, and then some sort of code to properly associate them.
I started with the following where I am grouping by the fields that should uniquely define a Voucher, but that doesn't work because the projection is dealing with an anonymous type instead of the DataRow.
var vouchers =
from row in invoicesTable.AsEnumerable()
group row by new { vendor = row.Field<string>("Vendor Code"), invoice = row.Field<string>("Vendor Invoice Number") } into rowGroup
select new Voucher
{ VendorID = rowGroup.Field<string>("Vendor Code") };
Is this achievable without introducing complex Linq that a future developer (myself included) could have difficulty understanding/maintaining? Is there a simpler approach without Linq that I'm overlooking?

The general idea is:
invoicesTable
.AsEnumerable()
.GroupBy(x=> new { row.Field<string>("Vendor Code"), row.Field<string>("Vendor Invoice Number")})
.Select(grouping =>
new Voucher
{
VendorID = grouping.First().Field<string>("VendorId") /* and so on */
Distributions = grouping.Select(somerow => new redistribution {AccountNumber = somerow.Field<string>("AccountNumber") /* and so on */}
}
But this is not the most elegant way.

You are looking for a Linq join. See the documentation here for more greater depth.
Where it appears that you are running into trouble however, is that on your 2 objects you need something for the query to compare against, like maybe adding public string VendorID { get; set; } to the APDistribution class, if possible. I would assume that the CSV files would have something that ties an APDistribution back to a Voucher, so whatever it is, make sure it's in both classes so you can relate one to the other. The name doesn't need to be the same in both classes but it should be. More importantly is that you now have something that an equality comparer can use for the join operation.
Now personally, I don't like big gnarly queries if I can break them apart and make things easier. Too much to reason about all at once, and you've indicated that you agree. So my approach is to divide and conquer as follows.
First, run queries to project the CSV data into discrete objects, like so:
var voucherRows =
from row in invoicesTable.AsEnumerable()
Select New Voucher {
VendorID = row.Field<string>("Vendor Code")
// other properties to populate
};
and
var distributionRows =
from row in distributionsTable.AsEnumerable()
Select New APDistribution {
VendorID = row.Field<string>("Vendor Code"),
// other properties to populate
};
At this point you have 2 data sets that are related in domain terms but not yet associated in code. Now you can compose the queries together in the Join query and the join starts to look a lot easier, maybe something like:
var vouchers =
from row in voucherRows
join dist in distributionRows
on row.VendorId equals dist.VendorId
into distGroup
select new Voucher
{ VendorID = row.VendorID,
// other properties to populate
Distributions = distGroup.ToList()
};
You'll have to modify the queries to your needs, but this breaks them down into 3 distinct operations that are all designed to do 1 thing, thus easier to read, reason about, debug, and modify later. If you need to group the vouchers you can at this point, but this should get you moving. Also, if needed, you can add a validation step or other processing in between the initial CSV queries and the join and you don't have to rewrite your queries, with the exception of changing some input variable names on the join.
Also, disclaimer that I did NOT build these queries in an IDE before posting so you may have some typos or missed symbols to deal with, but I'm pretty sure I have it right. Sorry in advance if you find anything aggravating.

While Linq can be cool and add efficiencies, it doesn't add value if you can't be sure the code is correct today, and can't understand it tomorrow. Maybe using Linq in this case is Premature Optimization.
Start with a non-Linq solution that is verifiably accurate without being needlessly inefficient, and then optimize later if performance becomes a problem.
Here's how I might tackle this:
var vouchers = new Dictionary<string, Voucher>();
foreach (DataRow row in invoicesTable)
{
string vendor = row.Field<string>("Vendor Code");
string invoice = row.Field<string>("Vendor Invoice Number");
string voucherKey = vendor + "|" + invoice;
if (!vouchers.ContainsKey(voucherKey))
{
vouchers.Add(voucherKey, new Voucher { VendorID = vendor, DocNumber = invoice });
}
vouchers[voucherKey].Distributions.Add(new APDistribution { AccountNumber = row.Field<string>("Journal Account Code") });
}
If this will be processing a large number of rows, you can tune this a bit by preallocating the Dictionary to an estimate of the number of unique vendors:
var vouchers = new Dictionary<string, Voucher>(invoicesTable.Rows.Count * 0.8);

How To Select All Columns From Inherited Class with LINQ and C#

I've got an entity I created using Entity Framework's auto code gen and don't want to modify that entry, as I regenerate that code on every table alter. For the sake of a single EF query, I've created an inherited entity that adds one extra column, SessionId as showing below.
public class SpeakerWithSessionIdAdded : Speaker
{
public int SessionId { get; set; }
}
Where Speaker is defined as below but with about 100 total column (I do know there should be less columns, but it is was it is right now).,
public class Speaker {
public int Id {get;set;}
public string firstName {get;set;}
public string lastName {get;set;}
... MANY MORE COLUMNS
}
I have a LINQ query as follows where I want to get out all the columns plus the one I added (SessionId). Is there any easy way to do this without using either reflection, or by hand listing every column I want to show?
var speakersWithSessionIdAdded = await
(from sessionPresenter in _dbContext.SessionPresenter
join speaker in _dbContext.Speakerson sessionPresenter.SpeakerId equals attendee.Id
where sessionIds.Contains(sessionPresenter.SessionId)
select new SpeakerWithSessionIdAdded
{
SessionId = sessionPresenter.SessionId,
Id = attendee.Id,
UserFirstName = attendee.UserFirstName,
UserLastName = attendee.UserLastName,
Email = attendee.Email,
Username = attendee.Username,
UserBio = attendee.UserBio,
AllowHtml = attendee.AllowHtml
.. I DON'T WANT TO LIST OUT 100 MORE LINES HERE FOR EVERY PROPERTY
}).ToListAsync();
return speakersWithSessionIdAdded.ToLookup(r => r.SessionId);
}

Querying Many to Many relationships Entitty Framework (doing wrong?? )

I've been doing some research on this topic and figure out a way to achieve this queries in my project but I'm not sure if something here is wrong. please help.
in summary I've created the entities like this:
class Student
{
public int StudentId { get; set; }
public string Name { get; set; }
public ICollection<Courses> Courses {get;set;} //or public List <Courses> {get;set;}
}
class Course
{
public int CourseId { get; set; }
public string Name { get; set; }
public ICollection<Students> Students {get;set;} //or public List<Students> {get;set;}
}
// We can see here that the database creates the Join Table Correctly
What I want to do:
Display in a grid view each student and for each of the students display the courses in wich they are enrolled.
If I made a simple query like
dbContex.Students.ToList(); 
and we look at the list the Collection of courses value is null. What is happening here?, shoulden't EF map this and make a query to SQL to get the info?
After this y could not solve the problem because the info that I found was using other approach of the framework (Diagram First ,i think) and they set up things in the entities diagram.
 
How did I work out the problem :
Find out in a Wordpress Post a Query that I haven´t tried out and add some other lines of code to achieve what I wanted:
aux_S = contexto.Students.ToList();
foreach(var element in aux_S)
         
   {
                
element.Courses= contexto.Courses.Where(c => c.Students.Any(s => s.StudentId == element.StudentId)).ToList();
          
  }
// I know I can make a projection to dismiss all the fields that I do not need , this is just to try it out
Am I wrong  doing this ?
It worked, but how is it possible?

One of the slower parts of a database query is the transfer of the data to your machine. So it is good practice to transfer only the data you plan to use.
When you use LINQ in entity framework, using Queryable.Select is a good way to specify exactly what data you want to transfer. This is usually done just before your final ToList / ToDictionary / FirstOrDefault / Single / ...
You want all Students, each with all his Courses. If you look at your tables, you'll see that there is more data in the tables then you want. For instance, each Student has an Id, each of his Courses have the same value for StudentId. So if a Student attends 20 Courses, you would have transferred the same value for StudentId 21 times.
So to make your query efficient: Select only the Properties of Students you plan to use, with only the Properties of the Courses of these Students you are interested in.
This will automatically solve your problem:
var result = myDbcontext.Students
// if you don't want all Students, use a Where:
.Where(student => student.City = "Guadalajara")
// Select only the properties you plan to use:
.Select(student => new
{
Id = student.Id,
Name = student.Name,
Birthday = student.Birthday,
Address = new
{
Street = student.Street,
City = student.City,
...
}
Courses = student.Courses
// if you don't want all courses: use a where
.Where(course => course.Start.Year == 2018)
// again: select only the properties you plan to use
{
Name = course.Name,
Location = course.Location,
...
// One of the useless properties to transfer:
// StudentId = course.StudentId
})
.ToList();
});

If you perform this query:
var studentslist = dbContex.Students.ToList();
Each item on studentslist will have the 'Courses' collection null, because, although the connection/relation exists (between each table), you didn't specify that you wanted that collection populated. For that to happen you can change your query accordingly:
var studentslist = dbContex.Students.Include(p => p.Courses).ToList();
Now, after running the last query, if you get an empty list on one/any of the items, then it means those items (students), aren't linked to any courses.

You are not lazy loading, if you add virtual like: public virtual ICollection<Courses> Courses {get;set;} you should get the courses loaded.
However, I'd advise using lazy loading since it may cause performance issues down the road, what you want to do is eager loading.
So when you are querying your student you would simply do this:
dbContex.Students.Include(c => c.Courses).ToList();

ServiceStack LoadReferences when using SQL Query

Is it possible to Load References when instead of using the code below:
SqlExpression<Customer> q = db.From<Customer>();
q.Join<Customer,CustomerAddress>((cust,address) => cust.Id == address.CustomerId);
List<Customer> dbCustomers = db.LoadSelect(q);
Using this:
public class KpiTotal : IKpiTotal
{
public DateTime Date { get; set; }
public int TeamId { get; set; }
public Team Team { get; set; }
public int AccountId { get; set; }
public Account Account { get; set; }
public double Total { get; set; }
}
var result = dbCon.SelectFmt<KpiTotal>(#"select convert(date, t.TransactionDate) [Date], tm.TeamId,a.AccountNumber, count(distinct(t.RequisitionNumber)) Total
from task.tblTransactions t
inner join task.tblRequisitions r on r.RequisitionNumber = t.RequisitionNumber
inner join task.tblAccounts a on a.AccountNumber = r.AccountNumber
inner join Team tm on tm.DivisionId = a.DivisionId
where t.TransactionTypeNumber = 201 and a.IsActive = 1
and t.TransactionDate between {0} and {1}
group by convert(date, t.TransactionDate), tm.TeamName, a.AccountName
order by 1,2 desc", dateRange.Start, dateRange.End);
Because my result object (KpiTotal) has references to two child tables, and I would like to automatic load the references, instead of getting it with a foreach block.

I'm assuming you want to load in Team and Account from the above query. The LoadSelect method sniffs the POCO model and generates a query that pulls back all related DB records based on the foreign key relationships to the core object you're querying. It generates a query similar to this for each referenced / joined POCO (very pseudo-coded):
SELECT * FROM Team /* Related POCO */
WHERE Team.Id IN (SELECT TeamId FROM [original query with WHERE clase])
Basically, it does a single query to bring back all Teamss or Accounts.
With ServiceStack.OrmLite v4.0.40, there is now a new Merge extension method that will stitch together object references based in a more manual process.
In your case, you can query your KpiTotal results, then run just two separate queries to fetch back Team and Account lists, then merge them in. Basically:
var result = dbCon.SelectFmt<KpiTotal>(/* gnarly SQL */);
var teams = dbCon.Select(/* get all relevant teams */);
var accounts = dbCon.Select(/* get all relevant accounts */);
result.Merge(teams);
result.Merge(accounts);
Debug.WriteLine(result.Dump()); // Output to console / debug window, whatever

How to to write subqueries as aliases using the Criteria Api with NHibernate

hey guys, say I have Entities and mappings like this:
public class Episode
{
Guid Id {get;set;}
String Title {get;set;}
List<Group> Groups {get;set;}
}
public class Group
{
Guid Id {get;set;}
DateTime PubDate {get;set;}
}
public class EpisodeMap : ClassMap<Episode>
{
public EpisodeMap()
{
//other mappings..
Map.HasMany(ep => ep.Groups);
}
}
So, basically, an Episode has many Groups. Each Group has a PubDate, and so an Episode has many PubDates.
I'm trying to write a query using the NHibernate Criteria API which lets me query Episodes and Order them by PubDate given that I have a Group Id.
Essentially, how do I write the equivalent Criteria API query for this SQL query:
Select
e.*,
(Select top 1 ReleaseDate From EpisodeGroups where EpisodeFk = e.Id and GroupFk = #GroupId) as myPubDate
From Episodes e
Order By myPubDate
Please help! cheers guys

public DetachedCriteria BuildCriteria(int episodeId, int groupId)
{
var groupCriteria = DetachedCriteria.For<Groups>()
.Add(Restrictions.Eq("this.Id", groupId))
.Add(Restrictions.Eq("Group.Id", groupId))
.AddOrder(Order.Asc("Group.PubDate"));
return DetachedCriteria.For<EpisodeGroups>()
.Add(Restrictions.Eq("this.Id", episodeId))
.Add(Subqueries.PropertyIn("this.Groups", groupCriteria)
.SetMaxResult(1);
}
Then you can do something like this...
var episodes = _repository.ExectueCriteria<EpisodeGroups>(BuildCriteria(episodeId, groupId))
Just as a special consideration, the reason why you would want to eagerly load the Group entity is in case you would like to use LINQ to compare the PUBDATE later on in your business logic as opposed to using a strict detached criteria.
We have found that by eagerly loading the attributes of an entity will reduce the total number of calls to our database.
No guarantees this code works... but it should at least get you started, good luck :)