I cannot find a specific example of this, so am posting the question. Any help appreciated.
I have two large generic lists, both with over 300K items.
I am looping through the first list to pull back information and generate a new item for a new list on the fly, but I need to search within the second list and return a value, based on THREE matching criteria, if found to add to the list, however as you can imagine, doing this 300k * 300k times is taking time.
Is there any way I can do this more efficiently?
My code:
var reportList = new List<StocksHeldInCustody>();
foreach (var correctDepotHolding in correctDepotHoldings)
{
var reportLine = new StocksHeldInCustody();
reportLine.ClientNo = correctDepotHolding.ClientNo;
reportLine.Value = correctDepotHolding.ValueOfStock;
reportLine.Depot = correctDepotHolding.Depot;
reportLine.SEDOL = correctDepotHolding.StockCode;
reportLine.Units = correctDepotHolding.QuantityHeld;
reportLine.Custodian = "Unknown";
reportLine.StockName = correctDepotHolding.StockR1.Trim() + " " + correctDepotHolding.StockR2.Trim();
//Get custodian info
foreach (var ccHolding in ccHoldList)
{
if (correctDepotHolding.ClientNo != ccHolding.ClientNo) continue;
if (correctDepotHolding.Depot != ccHolding.Depot) continue;
if (correctDepotHolding.StockCode != ccHolding.StockCode) continue;
if (correctDepotHolding.QuantityHeld != ccHolding.QuantityHeld) continue;
reportLine.Custodian = ccHolding.Custodian;
break;
}
reportList.Add(reportLine);
}
As Pranay says, a join is probably what you want:
var query = from correct in correctDepotHoldings
join ccHolding in ccHoldList
on new { correct.ClientNo, correct.Depot,
correct.StockCode, correct.QuantityHeld }
equals new { ccHolding.ClientNo, ccHolding.Depot,
ccHolding.StockCode, ccHolding.QuantityHeld }
// TODO: Fill in the properties here based on correct and ccHolding
select new StocksHeldInCustody { ... };
var reportList = query.ToList();
You could move the data from the lookup list into a dictionary, with the key being a unique hash of the 3 items you are searching on. Then you will have very quick lookups and save millions of iterations.
Check my full post : Linq Join on Mutiple columns using Anonymous type
Make use of Linq inner join that will do work for you.
var list = ( from x in entity
join y in entity2
on new { x.field1, x.field2 }
equals new { y.field1, y.field2 }
select new entity { fields to select}).ToList();
Join of linq on multiple field
EmployeeDataContext edb= new EmployeeDataContext();
var cust = from c in edb.Customers
join d in edb.Distributors on
new { CityID = c.CityId, StateID = c.StateId, CountryID = c.CountryId,
Id = c.DistributorId }
equals
new { CityID = d.CityId, StateID = d.StateId, CountryID = d.CountryId,
Id = d.DistributorId }
select c;
Use LINQ to join the lists and return it how you like.
eg
var list1 = GetMassiveList();
var list2 = GetMassiveList();
var list3 = from a in list1
join b in list2
on new { a.Prop1, a.Prop2 } equals
new { b.Prop1, b.Prop2 }
select new { a.Prop1, b.Prop2 };
To do your outter join, you can use DefaultIfEmpty()
This example is setting your RIGHT part of the join to a default object (often null) for the cases where a join wasn't made.
eg
from a in list1
join b in list2
on new { a.Prop1, a.Prop2 } equals
new { b.Prop1, b.Prop2 }
into outer
from b in outer.DefaultIfEmpty()
select new
Prop1 = a.Prop1,
Prop2 = b != null ? b.Prop2 : "Value for Prop2 if the b join is null"
}
Related
Have the following code: a entity linq query results in a select of two tables.
All data is available in the query, but can't get the result to split into two lists.
public Tuple<List<sale>, List<product>> SearchProduct(int productId = -1, string searchProduct = "")
{
//ToDo: searchProduct not working...gives nothing
var companyId = DalSession.DalGetCurrentUser().Company_ID;
using (var entity = new secondsoftEntities())
{
var query = (from s in entity.sales
join p in entity.products on s.Product_ID equals p.ProductID
where productId > 0 ? s.Product_ID == productId : s.Company_ID == companyId
select new { s, p }).ToList();
if (!string.IsNullOrEmpty(searchProduct))
{
query = query.FindAll(x => x.p.Description.ToLower().Contains(searchProduct.ToLower()));
}
// split s as List<sale> and p as List<product> to tuple output
return Tuple.Create(new List<sale>(), new List<product>() );
}
}
In the query result I see s and p, but how the exact them as a list with there properties in it so I can return them in Tuple.
Thanx Dinand
return Tuple.Create(query.Select(x => x.s).ToList(),
query.Select(x => x.p).ToList());
I am new to Entity, and have a question regarding a LINQ statement.
Below appears my code, it appears with exception to "TotalFoults" field. He always gives 0.
Could anyone help me because all the results in the "TotalFoults" column have a value above zero.
Thank you very much.
var result = (from sr in db.StudentsResult
join F in db.Fouls on sr.Enrollment equals F.Enrollment into F_join
from F in F_join.DefaultIfEmpty()
join S in db.Students on sr.Enrollment equals S.Enrollment into A_join
from A in A_join.DefaultIfEmpty()
where
F.Day != null
group new { sr, s } by new
{
sr.Enrollment ,
sr.Name,
sr.Number,
sr.Classes,
s.Discount
} into g
orderby
g.Key.Classes,
g.Key.Number,
g.Key.Name
select new frequencyMod()
{
Enrollment = g.Key.Enrollment ,
StudentName = g.Key.Name,
StudentFile= g.Key.Number,
Classes = g.Key.Classes,
TotalFoults = (from m0 in db.Foults
where
m0.Enrollment == g.Key.Enrollment
group m0 by new
{
m0.Enrollment
} into a
select new
{
Total = a.Sum(p => p.Foults)
}).FirstOrDefault().Total
}).ToList();
I am using the below Inner Join to retrive the Data between the two tables, but all data is not getting populated. I tried implementing Outer join by connecting using CCY1== CCY1 and PCODE == PCODE, but no luck.
var q = from g1 in TableCCY1.AsEnumerable()
join g2 in TableCCY2.AsEnumerable()
on g1.Field<string>("CCY1") equals g2.Field<string>("CCY1")
where g1.Field<string>("PCODE") == g2.Field<string>("PCODE")
select new
{
g1currency = g1.Field<string>("CCY1"),
g2currency = g2.Field<string>("CCY1"),
g1code = g1.Field<string>("PCODE"),
g2code = g2.Field<string>("PCODE"),
g1Amt1 = g1.Field<string>("AMT1"),
g2Amt2 = g2.Field<string>("AMT2")
};
Thanks for your help.
For left join you can use this approuch: http://msdn.microsoft.com/en-us/library/vstudio/bb397895.aspx
The code should be:
var q = from g1 in TableCCY1
join g2 in TableCCY2 on g1.CCY1 equals g2.CCY1 && g1.PCODE equals g2.PCODE into TableCCY3
from g3 in TableCCY3.DefaultIfEmpty()
select new
{
g1currency = g1.CCY1,
g2currency = (g3 == null ? String.Empty : g3.CCY1),
g1code = g1.PCODE,
g2code = (g3 == null ? String.Empty : g3.PCODE),
g1Amt1 = g1.AMT1,
g2Amt2 = (g3 == null ? 0 : g3.AMT2)
};
It looks like you just want to union/concat the two tables into one and then just group on those two columns. You're not logically joining the two tables. That actually makes it much easier.
var q = from row in TableCCY1.AsEnumerable().Concat(TableCCY2.AsEnumerable())
group row by new
{
CCY1 = row.Field<string>("CCY1"),
PCode = row.Field<string>("PCODE")
} into matches
select new
{
CCY1 = matches.Key.CCY1,
PCODE = matches.Key.PCode,
Sum = matches.Sum(match => match.Field<decimal?>("AMT2")),
};
I have two queries that I would like to merge. This might be a left outer join, but it seems different.
The first query selects distinct stuff from a table:
var d = from d in db.Data
select (d.ID, d.Label, Value = 0).Distinct;
Lets suppose this returns the following:
{1,"Apple",0}
{2,"Banana",0}
{3,"Cabbage",0}
I then have another query that makes a different selection:
var s = from d in db.Data
where d.Label != "Apple"
select (d.ID, d.Label, d.Value);
This returns:
{2,"Banana",34}
{3,"Cabbage",17}
I then want a third query that joins the d and s together based upon their ID and their Label. I want the result to look like this:
{1,"Apple",0}
{2,"Banana",34}
{3,"Cabbage",17}
I'm basically just updating the numbers in the third query, but I have no idea how I should be doing this. It feels like it should be a simple join, but I just cannot get it to work.
This should work:
var query1 = from d in db.Data
select new { d.ID, d.Label, Value = 0 }.Distinct();
var query2 = from d in db.Data
where d.Label != "Apple"
select new { d.ID, d.Label, d.Value };
var result =
from d1 in query1
join d2 in query2 on new { d1.ID, d1.Label } equals new { d2.ID, d2.Label } into j
from d2 in j.DefaultIfEmpty()
select new
{
d1.ID,
d1.Label,
Value = d2 != null ? d2.Value : d1.Value
};
Note: are you sure you want to join on the ID and the label ? It seems rather strange to me... the label shouldn't be part of the key, so it should always be the same for a given ID
Here is one using method chain, which is my personal favorite.
var one = db.Data.Select(f => new {f.Id, f.Label, Value = 0});
var two = db.Data.Select(f => f).Where(f => f.Label != "Apple");
var three = one.Join(two, c => c.Id, p => p.Id, (c, p) => new {c.Id, c.Label, p.Value});
Could you just do
var s = from d in db.Data
select new
{
Id = d.ID,
Label = d.Label,
Value = (d.Label == "Apple" ? 0 : d.Value)
};
How do I embed the ordinal number of element as its attribute in this linq query.
var AllSections = from s in xmlDoc.Descendants("section")
select new
{
id = s.Attribute("id").Value,
themeTitle = s.Element("themeTitle").Value,
themeText = s.Element("themeText").Value,
objects = (from a in AllObjects
join b in s.Descendants("object")
on a.Attribute("accessionNumber").Value equals
b.Attribute("accessionNumber").Value
//select a
select new
{
//index = insert ordinal id/index of element
ObjectTitle = a.Element("ObjectTitle").Value,
ObjectText = a.Element("textentry").Value,
}
)
};
You can't easily do it with a query expression - at least not without a horrible side effect. However, you can easily do it with dot notation for either Select or Where. Given that you've got quite a long query expression, it's probably easiest to embed an extra call to where at the start - assuming you do actually want the index of "s" in the original expression:
var AllSections =
from s in xmlDoc.Descendants("section")
select new
{
id = s.Attribute("id").Value,
themeTitle = s.Element("themeTitle").Value,
themeText = s.Element("themeText").Value,
objects = (from a in AllObjects.Select((Item,Index) => new {Item,Index})
join b in s.Item.Descendants("object")
on a.Item.Attribute("accessionNumber").Value equals
b.Attribute("accessionNumber").Value
//select a
select new
{
//index = insert ordinal id/index of element
Index = a.Index,
ObjectTitle = a.Element("ObjectTitle").Value,
ObjectText = a.Element("textentry").Value,
}
)
};
That's assuming you want the index of a within AllObjects.
#Jon Skeet gave you the appropriate overload of Select to use, and here is it in your query:
var AllSections = from s in xmlDoc.Descendants("section")
select new
{
id = s.Attribute("id").Value,
themeTitle = s.Element("themeTitle").Value,
themeText = s.Element("themeText").Value,
objects = (from a in AllObjects
join b in s.Descendants("object")
on a.Attribute("accessionNumber").Value
equals b.Attribute("accessionNumber").Value
select a).Select((a, index) =>
new
{
Index = index,
ObjectTitle = a.Element("ObjectTitle").Value,
ObjectText = a.Element("textentry").Value,
})
};