Combine datatables with specific conditions - c#

UPDATED: Forgot to mention, the table may contain more than one type of itemcode
I've got a problem for datatable. There're total 14 items with same itemcode. And now there're 2 tables come from different source. One is grouped by itemcode and sum up the quantity, when count equals to 10, go to next row, and each row contains specific information such as shipment and remarks. Another table contains more detail information.
Source1: Grouped table
ItemCode|TotalQty|Shipment|Remarks|Line
=========================================
ITEM01 | 1000 | S001 | R001 | 1 <==10 items here
ITEM01 | 400 | S002 | R002 | 2 <==4 items here
Source2: Detail table (14 items & rows)
RefNo|ItemCode|Quantity|Weight|From
=======================================
R001 | ITEM01 | 100 | 50 | US
R002 | ITEM01 | 100 | 50 | US
R003 | ITEM01 | 100 | 50 | US
. | . | . | . | .
. | . | . | . | .
R013 | ITEM01 | 100 | 50 | US
R014 | ITEM01 | 100 | 50 | US
I would like to combine source1 and source2 and get the result as below
Shipment|Line|Remarks|ItemCode|TotalQty|RefNo|Quantity|Weight|From
===================================================================
S001 | 1 | R001 | ITEM01 | 1000 | R001| 100 | 50 | US \\1
S001 | 1 | R001 | ITEM01 | 1000 | R002| 100 | 50 | US \\2
S001 | 1 | R001 | ITEM01 | 1000 | R003| 100 | 50 | US \\3
S001 | 1 | R001 | ITEM01 | 1000 | R004| 100 | 50 | US \\4
S001 | 1 | R001 | ITEM01 | 1000 | R005| 100 | 50 | US \\5
S001 | 1 | R001 | ITEM01 | 1000 | R006| 100 | 50 | US \\6
S001 | 1 | R001 | ITEM01 | 1000 | R007| 100 | 50 | US \\7
S001 | 1 | R001 | ITEM01 | 1000 | R008| 100 | 50 | US \\8
S001 | 1 | R001 | ITEM01 | 1000 | R009| 100 | 50 | US \\9
S001 | 1 | R001 | ITEM01 | 1000 | R010| 100 | 50 | US \\10
S002 | 2 | R002 | ITEM01 | 400 | R011| 100 | 50 | US \\11
S002 | 2 | R002 | ITEM01 | 400 | R012| 100 | 50 | US \\12
S002 | 2 | R002 | ITEM01 | 400 | R013| 100 | 50 | US \\13
S002 | 2 | R002 | ITEM01 | 400 | R014| 100 | 50 | US \\14
Is there any way (Linq or looping) to get the above result? Thanks for your help!

It can be done with LINQ (which I assume you're after, since you've included the LINQ tag), but not in what I would consider a nice fashion.
Given two DataTable objects with the above formats and data, named grouped and detail, here's a LINQ expression that will stitch the data together in the way you want:
IEnumerable<object[]> qry =
(
from DataRow rDetail in detail.Rows
let dgrp = detail.Rows.IndexOf(rDetail) / 10
join DataRow rGroup in grouped.Rows
on dgrp equals grouped.Rows.IndexOf(rGroup)
orderby rDetail["RefNo"]
select new object[] {
rGroup["Shipment"], rGroup["Line"], rGroup["Remarks"], rGroup["ItemCode"], rGroup["TotalQty"],
rDetail["RefNo"], rDetail["Quantity"], rDetail["Weight"], rDetail["From"]
}
);
Now you need another DataTable to pump those results into:
DataTable res = new DataTable();
res.Columns.Add("Shipment", typeof(string));
res.Columns.Add("Line", typeof(Int32));
res.Columns.Add("Remarks", typeof(string));
res.Columns.Add("ItemCode", typeof(string));
res.Columns.Add("TotalQty", typeof(Int32));
res.Columns.Add("RefNo", typeof(string));
res.Columns.Add("Quantity", typeof(Int32));
res.Columns.Add("Weight", typeof(Int32));
res.Columns.Add("From", typeof(string));
And finally, fill the res table with the results of the LINQ query:
foreach (object[] rowdata in qry)
res.Rows.Add(rowdata);
The code above works for this specific set of data, but I can't promise you anything more than that. It is heavily dependent on the row order in the source tables, and because I'm using DataTable.Rows.IndexOf to get the order it's quite possible that this will be terribly slow on large collections of data.
But then you're already using DataTables instead of a properly typed collection, so all bets are off in the performance- and code-sanity stakes anyway.
This is one case where I'd suggest not using LINQ to perform the task. IMHO this would be better done as an iterative loop rather than a query. You don't get much (if any) improvement over the iterative version, you lose a lot of clarity, and you have all sorts of fun getting things set up just right before you can use it.
And because I can't leave well-enough alone, here's a full (long, lots of code) solution using a combination of LINQ, classes to hold the data being processed and iteration to generate the table:
public DataTable MergeShippingData(DataTable groupTable, DataTable detailTable)
{
// convert group table to array of GroupEntry objects
var groupList =
(
from DataRow grouprow in groupTable.Rows
let ent = GroupEntry.FromRow(grouprow)
where ent != null
select ent
).ToArray();
// convert detail table to sequence of DetailEntry objects
var detailSeq =
from DataRow detailrow in detailTable.Rows
let ent = DetailEntry.FromRow(detailrow)
where ent != null
select ent;
// Create output DataTable
DataTable output = CreateOutputTable();
// Process all detail lines into shippings
foreach (var detail in detailSeq)
{
// Find available shipping group for the item code with enough remaining capacity
var grp = groupList.First (g => g.ItemCode == detail.ItemCode && g.Remainder >= detail.Quantity);
if (grp == null)
throw new Exception("No available shipping found for detail item...");
// update remaining space in shipping group
grp.Remainder -= detail.Quantity;
// add data to output table
output.Rows.Add(new object[] {
grp.Shipment, grp.Line, grp.Remarks, grp.ItemCode, grp.TotalQty,
detail.RefNo, detail.Quantity, detail.Weight, detail.From
});
}
return output;
}
// Class to hold the shipping groups while processing
public class GroupEntry
{
// fields from source DataTable
public string ItemCode;
public int TotalQty;
public string Shipment;
public string Remarks;
public int Line;
// process variable, holds remaining quantity value
public int Remainder;
// Convert DataRow into GroupEntry
public static GroupEntry FromRow(DataRow r)
{
try
{
return new GroupEntry
{
ItemCode = r.Field<string>(0),
TotalQty = r.Field<int>(1),
Shipment = r.Field<string>(2),
Remarks = r.Field<string>(3),
Line = r.Field<int>(4),
Remainder = r.Field<int>(1)
};
}
catch { }
return null;
}
}
// Class to hold shipping Detail records during processing
public class DetailEntry
{
public string RefNo;
public string ItemCode;
public int Quantity;
public int Weight;
public string From;
// Convert DataRow into DetailEntry
public static DetailEntry FromRow(DataRow r)
{
try
{
return new DetailEntry
{
RefNo = r.Field<string>(0),
ItemCode = r.Field<string>(1),
Quantity = r.Field<int>(2),
Weight = r.Field<int>(3),
From = r.Field<string>(4)
};
}
catch { }
return null;
}
}
// Create output DataTable
public DataTable CreateOutputTable()
{
DataTable res = new DataTable();
res.Columns.Add("Shipment", typeof(string));
res.Columns.Add("Line", typeof(Int32));
res.Columns.Add("Remarks", typeof(string));
res.Columns.Add("ItemCode", typeof(string));
res.Columns.Add("TotalQty", typeof(Int32));
res.Columns.Add("RefNo", typeof(string));
res.Columns.Add("Quantity", typeof(Int32));
res.Columns.Add("Weight", typeof(Int32));
res.Columns.Add("From", typeof(string));
return res;
}
Add some error handling and you're good to go.

Related

Retrieving a row value based on other columns value using linq

I have three data tables that basically contain the following data:
table 1:
ID FContractID | WaitingTime
1 | 1 | 85
2 | 1 | 98
3 | 1 | 122
4 | 1 | 45
5 | 1 | 234
6 | 1 | 101
etc.
Table 2:
PricingCriterionItemId PricingCriterionName PricingCriterionUnit
1 | WaitingTimeMax | min
2 | WaitingTimePeriod | min
3 | WaitingTimeOverdue | €/period
4 | OverDuePriceMax | €
Table 3:
PricingCriterionId ContractID PricingCriterionItemId PricingCriterionValue
1 | 1 | 1 | 70
2 | 1 | 2 | 30
3 | 1 | 3 | 30,00
4 | 1 | 4 | 120,00
I want to add to the table 1 a column that contains waiting time cost. The waiting time cost would be calculated like
WaitingTimeCost = min(((WaitingTime - WaitingTimeMax) / WaitingTimePeriod) * WaitingTimeOverdue, OverDuePriceMax)
I can easily join tables 2 and 3 into one table:
Table 4
PricingCriterionId ContractID PricingCriterionName PricingCriterionValue PricingCriterionUnit
1 | 1 | WaitingTimeMax | 70 | min
2 | 1 | WaitingTimePeriod | 30 | min
3 | 1 | WaitingTimeOverdue | 30,00 | €/period
4 | 1 | OverDuePriceMax | 120,00 | €
Is it possible using linq to assign a column's value on a certain row using other columns value?
Something like
var result = from WaitingData in table1
join PricingCriteria in table4
on WaitingData.ContractId equals PricingCriteria.ContractId
let WaitingTimeMax = (select PricingCriterionValue from table4 where PricingCriterionName = "WaitingTimeMax")
let ...
let WaitingTimeCost = min(((WaitingTime - WaitingTimeMax) / WaitingTimePeriod) * WaitingTimeOverdue, OverDuePriceMax)
select new
{
ID,
WaitingTimeCost
}
How to formulate this properly using linq?
You don't need to do everything in the database query.
Database is only IO device, which responsible only for reading and writing data.
Load all required data and calculate.
var contractsId = waitingData.Select(data => data.ContractId).ToList();
var pricingCriterias = table4.Where(criteria => contractsId.Contains(criteria.ContractId)
.ToLookup(criteria => criteria.ContractId);
var maxWaitingTime =
pricingCriterias.SelectMany(group => group)
.Where(criteria => criteria.PricingCriterionName = "WaitingTimeMax")
.Max(criteria => criteria.PricingCriterionValue);
foreach (var waitingItem in waitingData)
{
// Calculate others waiting values
var waitingPerPeriod = (WaitingTime - maxWaitingTime) / WaitingTimePeriod);
var waitingPrice = waitingPerPeriod * WaitingTimeOverdue;
var WaitingTimeCost = Math.Min(waitingPrice, OverDuePriceMax)
}
Below example with DataTables.
However DataTable is heavy and not strongly typed data structure and as developer it takes more time to deal with it.
Instead transform data in DataTable to the plain strong typed objects with descriptive property names - you will get IntelliSense for free ;)
var contractsId =
waitingData.AsEnumerable()
.Select(row => row.Field<int>("ContractId"))
.ToList();
var pricingCriterias =
table4.AsEnumerable()
.Where(row => contractsId.Contains(row => row.Field<int>("ContractId"))
.ToLookup(row => row.Field<int>("ContractId"));
var maxWaitingTime =
pricingCriterias.SelectMany(group => group)
.Where(row => row.Field<string>("PricingCriterionName") = "WaitingTimeMax")
.Max(row => row.Field<int>("PricingCriterionValue"));
foreach (var waitingItem in waitingData)
{
// Calculate others waiting values
var waitingPerPeriod = (WaitingTime - maxWaitingTime) / WaitingTimePeriod);
var waitingPrice = waitingPerPeriod * WaitingTimeOverdue;
var WaitingTimeCost = Math.Min(waitingPrice, OverDuePriceMax)
}

Is there an easy way to INNER join , OUTER join , LEFT OUTER join, RIGHT OUTER join, or UNION two (or more) DataTables in C#?

I am writing a c# application that connects separate database systems. These systems could be flat-file db's, Oracle, Sql, Excel Files, ext. The job of the C# application is to provide an outlet for making all of these sources available in one spot. So basically, the application accepts a list of queries and connection settings for the respective database systems and collects a bunch of results.
The goal is to output a singe DataTable with the result of all these queries all joined/unioned together(depending on settings). Does C# provide an easy way to perform any join/union operations on a list of DataTables?
For example:
Table1:
__________________________________________________________
|tb1_pk_id| tb1_name | tb1_data1 | tb1_data2 |
|---------|---------------|---------------|---------------|
| 1 | tb1name_blah1 | tb1dat1_blah1 | tb1dat2blah1 |
| 2 | tb1name_blah2 | tb1dat1_blah2 | tb1dat2blah2 |
| 3 | tb1name_blah3 | tb1dat1_blah3 | tb1dat2blah3 |
-----------------------------------------------------------
Table2:
__________________________________________________________
|tb2_pk_id| tb2_name | tb2_data1 | tb2_data2 |
|---------|---------------|---------------|---------------|
| 1 | tb2name_blah1 | tb2dat1_blah1 | tb2dat2blah1 |
| 2 | tb2name_blah2 | tb2dat1_blah2 | tb2dat2blah2 |
| 3 | tb2name_blah3 | tb2dat1_blah3 | tb2dat2blah3 |
-----------------------------------------------------------
Join Results:
__________________________________________________________ _______________________________________________
|tb1_pk_id| tb1_name | tb1_data1 | tb1_data2 | tb2_name | tb2_data1 | tb2_data2 |
|---------|---------------|---------------|---------------|---------------|---------------|---------------|
| 1 | tb1name_blah1 | tb1dat1_blah1 | tb1dat2blah1 | tb2name_blah1 | tb2dat1_blah1 | tb2dat2blah1 |
| 2 | tb1name_blah2 | tb1dat1_blah2 | tb1dat2blah2 | tb2name_blah2 | tb2dat1_blah2 | tb2dat2blah2 |
| 3 | tb1name_blah3 | tb1dat1_blah3 | tb1dat2blah3 | tb2name_blah3 | tb2dat1_blah3 | tb2dat2blah3 |
-----------------------------------------------------------------------------------------------------------
So far I have found the following code online (here) to do a merge on all the data:
private DataTable MergeAll(IList<DataTable> tables, String primaryKeyColumn)
{
if (!tables.Any())
throw new ArgumentException("Tables must not be empty", "tables");
if (primaryKeyColumn != null)
foreach (DataTable t in tables)
if (!t.Columns.Contains(primaryKeyColumn))
throw new ArgumentException("All tables must have the specified primarykey column " + primaryKeyColumn, "primaryKeyColumn");
if (tables.Count == 1)
return tables[0];
DataTable table = new DataTable("TblUnion");
table.BeginLoadData(); // Turns off notifications, index maintenance, and constraints while loading data
foreach (DataTable t in tables)
{
table.Merge(t); // same as table.Merge(t, false, MissingSchemaAction.Add);
}
table.EndLoadData();
if (primaryKeyColumn != null)
{
// since we might have no real primary keys defined, the rows now might have repeating fields
// so now we're going to "join" these rows ...
var pkGroups = table.AsEnumerable()
.GroupBy(r => r[primaryKeyColumn]);
var dupGroups = pkGroups.Where(g => g.Count() > 1);
foreach (var grpDup in dupGroups)
{
// use first row and modify it
DataRow firstRow = grpDup.First();
foreach (DataColumn c in table.Columns)
{
if (firstRow.IsNull(c))
{
DataRow firstNotNullRow = grpDup.Skip(1).FirstOrDefault(r => !r.IsNull(c));
if (firstNotNullRow != null)
firstRow[c] = firstNotNullRow[c];
}
}
// remove all but first row
var rowsToRemove = grpDup.Skip(1);
foreach (DataRow rowToRemove in rowsToRemove)
table.Rows.Remove(rowToRemove);
}
}
return table;
}
This works fine for doing a union, but I don't know if an easier way to do that already exists in .NET that will let me do ANY kind of join or union on a group of seprate DataTables (not just the union as in the code above) or do I have to custom code each type of join/union?
No, there is not a simple .Net way of doing this....
LINQ can come close... you can create table joins in LINQ, but they are typically "inner joins". Doing a "left join" is a bit more complicated and requires the GroupJoin keyword.
https://msdn.microsoft.com/en-us/library/bb386969(v=vs.110).aspx
If you'd like "do it yourself" with ADO.Net DataRelations, you might take a look at this old VB.Net article:
http://www.emmet-gray.com/Articles/DataRelations.html

C# ObservableCollection get all values where Id is identical

How could I get all rows, where my Id (it is meant to be like that) is x.
Example list:
| Id | Key | Value |
----------------------
| 0 | FName | Peter |
| 0 | LName | Griff |
| 0 | Phone | 12345 |
| 1 | FName | Sasha |
| 1 | LName | Un |
| 1 | Mail | SU#m.c|
| 2 | FName | Laura |
...
From the list I want to get all 0 and 1... differently to insert them into Sharepoint list.
This data has been inserted to this Collection before and Id is used to keep information for specific person.
All I could come up currently is to use loops, to do that
for (int i = 0; i < _dataVm.ADData.Count; i++)
{
foreach (DataFromAD k in _dataVm.ADData)
{
// Not sure, how could I sort out specific data here
}
}
You have to group data on Id using groupby to get seperated items by Id:
var result = from d in _dataVm.ADData
group d by d.Id into g
select new
{
Id = g.Key,
Data = g.ToList()
}

linq query sum products across multiple collections

I have two datasets that look like this:
+------------------------------------+
| Products |
+------------------------------------+
| Id | Name | Price |
+------------------------------------+
| 1 | apples | 1.00 |
| 2 | oranges | 2.00 |
| 3 | pomengrate | 3.00 |
+------------------------------------+
+-------------------------------+
| Sales |
+-------------------------------+
| CustId | ProductId | Quantity |
+-------------------------------+
| 1 | 1 | 5 |
| 1 | 2 | 4 |
| 1 | 3 | 2 |
| 2 | 1 | 8 |
| 2 | 3 | 7 |
+-------------------------------+
I want to get the amount that each customer is spending, essentially ending up with a result like this:
+----------------+
| CustId | Total |
+----------------+
| 1 | 19.00 |
| 2 | 29.00 |
+----------------+
I can do a sum across a single table but the Sum method in LINQ takes a lambda with only one argument: the reference to the table that the values being summed are in. These values are in different tables. How do I sum these together?
var totals =
from sale in Sales
group sale by sale.CustId into custSales
select new
{
CustId = custSales.Key,
Total = (
from custSale in custSales
select custSale.Product.Price *
custSale.Quantity)
.Sum()
};
Give this a shot. It gives the results you were looking for:
var results = sales.Join(products,
sale => sale.ProductID,
product => product.ID,
(sale, product) => new { CustID = sale.CustID, Total = sale.Quantity * product.Price })
.GroupBy(r => r.CustID)
.Select(g => new { CustID = g.Key, Total = g.Sum(gt => gt.Total) });
For completeness sake, here's the query syntax version (utilizing joins as opposed to a subselect):
var totals =
from sale in sales
join prod in product on sale.ProductId equals prod.Id
let saleProds = new { sale.CustId, Total = prod.Price * sale.Quantity }
group saleProds by saleProds.CustId into custSale
select new { Customer = custSale.Key, Total = custSale.Sum(tr => tr.Total) };
The key part is that you need to somehow transform the joined collection results (sale and prod) into a single entity that can then be grouped by.

Query tabels to sort sums across rows and count based on a value in LINQ

Here is my problem. i have 3-5 persons that is going to set a grade on one person and they use their own individual row to do so, and what I'm having trouble to do is to sum and average the grade from individual data across multiple rows on the same table.
in the select new statement i have made a pseudo answer of what i want
var users = from workRew in db.Reviews
select new
{
UserID = workRew.UserID.DistinctOfSomeSort
AvgGrade = workRew.Grade.Sum/CountOfSomeSort
};
Here i a illustration.
So if i have this table
| SomeID | UserID | Grade |
| 1 | 2 | 3 |
| 2 | 3 | 1 |
| 3 | 2 | 1 |
And this is the output i want from the LINQ query on the above (In theory ateast)
| UserID | AvgGrade |
| 2 | 2 |
| 3 | 1 |
EDIT: Simplified the whole case, to a great extent.
It should look something like this fragment:
group by user.UserID
select new
{
User = user.UserID
TotGradeCount = workRew.Grade.Sum()
Graders = workRew.Grade.Count()
}

Categories

Resources