Develop Reference

how to add linq result to list with key

I have a linq which is inside a for loop,im adding the results to a list using addRange() but it will add whole thing in a single set,for example my first loop result has 16 items,the second has 10 items,...i want them to be added to list like this then i can see in list how many and which items has been added on each query
public List<statisticsDaily> dailyStat(List<string> id,string dtFrom,string dtTo)
{
List<StatisticsDaily> rsltofquery = new List<StatisticsDaily>();
for (int i = 0; i < id.Count; i++)
{
var rslt = (from d in db.statDaily
join s in db.masterData on d.m_turbine_id equals s.m_turbine_id
where d.m_turbine_id == IPAddress.Parse(id[i]) && d.m_date >= frm && d.m_date <= to
select new StatisticsDaily
{
m_wind_speed = d.m_wind_speed,
Date = d.m_date.ToString("yyyy-MM-dd"),
name = s.turbine_name,
Production = d.m_energy_prod,
Availability = d.m_corrected_av
}
).AsEnumerable().OrderBy(s => s.Date).ToList();
rsltofquery.AddRange(rslt);
}

You need to have collection of collections like List<List<StatisticsDaily>>.
So yours code will be:
public List<List<statisticsDaily>> dailyStat(List<string> id,string dtFrom,string dtTo)
{
List<List<StatisticsDaily>> rsltofquery = new List<List<StatisticsDaily>>();
for (int i = 0; i < id.Count; i++)
{
var rslt = (from d in db.statDaily
join s in db.masterData on d.m_turbine_id equals s.m_turbine_id
where d.m_turbine_id == IPAddress.Parse(id[i]) && d.m_date >= frm && d.m_date <= to
select new StatisticsDaily
{
m_wind_speed = d.m_wind_speed,
Date = d.m_date.ToString("yyyy-MM-dd"),
name = s.turbine_name,
Production = d.m_energy_prod,
Availability = d.m_corrected_av
}).AsEnumerable().OrderBy(s => s.Date).ToList();
rsltofquery.Add(rslt);
}
}
If you want to use all elements, not in parts, you can use SelectMany:
var x = dailyStat(id, dtFrom, dtTo);
foreach (var e in x.SelectMany(d => d)) ...

More efficient Linq expression

I have this Linq Join
var NewQuote = (from qw in Q
join NW in NewNotes on qw.RECID equals NW.RECID into temp
from j in temp
select new Quotes
{
QuoteNumber = qw.QuoteNumber,
CustPartNumber = qw.CustPartNumber,
ITEMGROUPID = qw.ITEMGROUPID,
LotSize = qw.LotSize,
EAU = qw.EAU,
CONTACTPERSONID = qw.CONTACTPERSONID,
QUOTATIONSTATUS = qw.QUOTATIONSTATUS,
QUOTESENTDATE = qw.QUOTESENTDATE,
PricePerPiece = qw.PricePerPiece,
QuoteValue = qw.QuoteValue,
Email = qw.Email,
RECID = qw.RECID,
Notes = j == null ? "" : j.NOTES
}).ToList();
Q is of the Quote class but I need to add the data to the Notes field from NewNotes. Is there a better way to do this than listing every field from the Quote class? If I have to add fields to Quote then I have to document to come to this section of code and update as well.

Why you create new instances of Quotes if you just want to update one property?
var query = from qw in Q join NW in NewNotes
on qw.RECID equals NW.RECID into temp
from j in temp
select new { Quote = qw, Notes = j?.Notes ?? "" };
foreach(var x in query)
{
x.Quote.Notes = x.Notes;
}

Foreach substitute in Linq

I have a query that I want to substitue the foreach with linq because the foreach is so slow
how can I write all this codein one query
this is my code:
ret = new List<ReportData>();
foreach (var item in Number)
{
string A = item.Substring(0, 11);
string B = item.Substring(14, 2);
string C = item.Substring(19, 11);
string D = item.Substring(33);
ret1 = (from a in Report
where a.A == A && a.B == B && a.C == C && a.D == D && Filter.Type.Contains(a.Y)
select new ReportData
{
X = a.X,
Y = a.Y,
});
if (ret1 != null && ret1.ToList().Count > 0)
{
ret.AddRange(ret1);
}
}

As already mentioned in the comments, LINQ will not make a foreach any faster; if you have to iterate the entire collection, then foreach will be faster than LINQ.
There is no need to check for null or if any results exists in the inner LINQ statement; just add the range since the LINQ query will return an Enumerable.Empty<ReportData> if nothing is returned from the query.
if (ret1 != null && ret1.ToList().Count > 0)
{
ret.AddRange(ret1);
}
// becomes
ret.AddRange(ret1);
Assumming Number is a collection of string, make sure there are no duplicates:
foreach (string item in Number.Distinct())
If Number is a large list, thousands of items or more, then consider using a Parallel.ForEach.

Linq will just enumerate the collection just like a foreach would, but you might see some benefit from a join:
var items = Number.Select( item => new {
A = item.Substring(0, 11),
B = item.Substring(14, 2),
C = item.Substring(19, 11),
D = item.Substring(33),
});
var ret = (from a in Report
join i in items
on new {a.A, a.B, a.C, a.D} equals new {i.A, i.B, i.C, i.D}
where Filter.Type.Contains(a.Y)
select new ReportData
{
X = a.X,
Y = a.Y,
});

MY Asp.Net MVC page takes 12 secs. to load

I have an MVC Controller like this
public ActionResult Index(int vendor=-1, int product = -1, string error="", string mode="Shortfall")
{
if (Session["UserId"] == null)
return RedirectToAction("Index", "Login");
var products = DbContext.GetAllProducts();
List<SurplusViewModel> surplusList = new List<SurplusViewModel>();
Dictionary<int, string> searchVendor = new Dictionary<int, string>();
Dictionary<int, string> searchProds = new Dictionary<int, string>();
if (products.Count() > 0)
{
foreach (var prod in products)
{
SurplusViewModel s = new SurplusViewModel(prod);
surplusList.Add(s);
foreach (var v in s.Vendors)
{
if (!searchVendor.ContainsKey(v.CorpId))
{
searchVendor.Add(v.CorpId, v.CorpName);
}
}
if(!searchProds.ContainsKey(s.ProductId))
searchProds.Add(s.ProductId, s.ProductVM.ProductCode + " / " + s.ProductVM.ProductPartNo);
}
}
ViewData["vendorList"] = searchVendor;
ViewData["productList"] = searchProds;
ViewData["selectVendor"] = vendor;
ViewData["selectProd"] = product;
ViewData["mode"] = mode;
ViewBag.Message = "";
ViewBag.Error = "";
IEnumerable<SurplusViewModel> finalList = surplusList.OrderBy(o => o.Difference).ToList();
if (vendor > 0)
{
Corporation searchcorp = DbContext.GetCorporation(vendor);
finalList = finalList.Where(x => x.VendorNames.IndexOf(searchcorp.CorpName) >= 0);
}
if (product > 0)
{
finalList = finalList.Where(x => x.ProductId == product);
}
if (vendor < 0 && product < 0)
{
if (mode.Equals("Shortfall"))
finalList = finalList.Where(f => f.VendorQuantity - (f.CMQuantity + f.OEMQuantity) < 0);
else if (mode.Equals("Surplus"))
finalList = finalList.Where(f => f.VendorQuantity - (f.CMQuantity + f.OEMQuantity) > 0);
}
return View(finalList);
//return View();
}
This takes about 20 seconds to load on localhost. What can I do to improve my app's loading time. If it takes 20 secs on localhost I assume it will be very slow on the internet. Any suggestions?
EDIT: Code for SurplusViewModel
public SurplusViewModel(Product product)
{
int productId = product.ProductId;
ProductId = productId;
ProductVM = new ProductViewModel(product);
var saleDetsCM = from s in DbContext.GetSalesOrderDetailsFromCM()
where s.ProductId == productId && s.SaleStatus.Equals("Open") && (s.OrderType.ToLower().Equals("prototype") || s.OrderType.ToLower().Equals("production"))
orderby s.SalDetId descending
select s;
var saleDetsOEM = from s in DbContext.GetSalesOrderDetailsFromOEMs()
where s.ProductId == productId && s.SaleStatus.Equals("Open") && (s.OrderType.ToLower().Equals("prototype") || s.OrderType.ToLower().Equals("production"))
orderby s.SalDetId descending
select s;
var shipQty = from s in DbContext.GetAllSalesDets()
where s.ProductId == productId && !s.SaleStatus.Equals("Open") && (s.OrderType.ToLower().Equals("prototype") || s.OrderType.ToLower().Equals("production"))
orderby s.SalDetId descending
select s;
CustomerOrdersFromCMs = saleDetsCM.ToList();
CustomerOrdersFromOEMs = saleDetsOEM.ToList();
VendorOrders = (from p in DbContext.GetPurchaseDetsForProduct(productId)
where p.OrderType != null && (p.OrderType.ToLower().Equals("prototype") || p.OrderType.ToLower().Equals("production"))
select p).ToList();
var poIds = from v in VendorOrders
select v.PodPOId;
BatchPurchaseDetails = DbContext.GetBatchPurchaseForProduct(productId).ToList();
VendorOrderCount = 0;
VendorQuantity = 0;
var purchaseOrds = (from po in DbContext.GetPurchaseOrdersForProduct(productId)
where poIds.Contains(po.POId)
select po).ToList();
List<int> vendIds = new List<int>();
foreach (var po in purchaseOrds)
{
vendIds.Add(po.VendorId.Value);
}
var vendors = from v in DbContext.GetAllCorps()
where vendIds.Contains(v.CorpId)
select v;
foreach (var podet in VendorOrders)
{
double totalbatchqty = 0;
var purdetBatch = DbContext.GetBatchDetailsForPurchaseDet(podet.PodId);
VendorQuantity += podet.Quantity;
foreach (var b in purdetBatch)
{
totalbatchqty += b.Quantity;
VendorQuantity -= b.Quantity;
}
if (totalbatchqty >= podet.Quantity)
{
}
else
{
VendorOrderCount++;
}
}
Vendors = vendors.ToList();
VendorNames = "";
foreach (var vnd in Vendors)
{
VendorNames += vnd.CorpName + ",";
}
if (VendorNames.Length > 0)
{
VendorNames = VendorNames.Substring(0, VendorNames.Length - 1);
}
OEMQuantity = 0;
foreach (var item in CustomerOrdersFromOEMs)
{
OEMQuantity += item.Quantity;
}
CMQuantity = 0;
foreach (var item in CustomerOrdersFromCMs)
{
CMQuantity += item.Quantity;
}
ShipQuantity = 0;
foreach (var item in shipQty)
{
ShipQuantity += item.Quantity;
}
Difference = VendorQuantity - (CMQuantity + OEMQuantity);
//TotalInsideSalesOrder = VendorOrders.Count();
}

You're doing a ton of things wrong, or at least very poorly. First, you're returning all records from various queries, the processing them in memory. This is fine if there are only a few records, but you mention that there are 50 products. How many Vendors? How many Corporations?
You're doing several database queries, and it sounds like you're doing even more queries in sub-objects you instantiate. All of those queries take time. That's why you want to minimize queries by writing more inclusive single queries, and performing as much work on the sql server as possible in a single (or as few as possible) queries.
Areas you can optimize... Don't do Count() > 0, instead use .Any(). .Any returns true after the first record it finds, rather than having to count everything and then compare that number to 0.
Another area, you're doing a foreach inside a foreach. This creates n * m loops. ie, if there's 2 products and 2 vendors, that's 4 loops, but if it's 3 of each, it's 9 loops, if it's 4 of each it's 16 loops. if it's 50 of each 2500 loops. And each one of those loops executes your SurplusViewModel constructor, which if it's a lot of code means it's going to be sloooooooow.
I see from your update, that SurplusViewModel executes at least 7 queries, maybe more.. it's hard to tell. so that's 2500 * 7 or 17,500 Queries (assuming 50 products, and 50 vendors). Are you beginning to see why it's so slow? Now, Imagine you had 100 products and 100 vendors. That's 10,000 loops with 7+ queries, that's at least 70,000 queries. This is not a scalable solution.
Let's look further.. what's in all these "Getxxx" methods? I assume there is some kind of query in each of those? Are you possibly performing double queries each time? Again, you're not including all the information.
To be honest, i'm quite surprised that this ONLY takes 20 seconds... i'd think it would be more like 20 minutes with any amount of data.

Comparing two large generic lists

I cannot find a specific example of this, so am posting the question. Any help appreciated.
I have two large generic lists, both with over 300K items.
I am looping through the first list to pull back information and generate a new item for a new list on the fly, but I need to search within the second list and return a value, based on THREE matching criteria, if found to add to the list, however as you can imagine, doing this 300k * 300k times is taking time.
Is there any way I can do this more efficiently?
My code:
var reportList = new List<StocksHeldInCustody>();
foreach (var correctDepotHolding in correctDepotHoldings)
{
var reportLine = new StocksHeldInCustody();
reportLine.ClientNo = correctDepotHolding.ClientNo;
reportLine.Value = correctDepotHolding.ValueOfStock;
reportLine.Depot = correctDepotHolding.Depot;
reportLine.SEDOL = correctDepotHolding.StockCode;
reportLine.Units = correctDepotHolding.QuantityHeld;
reportLine.Custodian = "Unknown";
reportLine.StockName = correctDepotHolding.StockR1.Trim() + " " + correctDepotHolding.StockR2.Trim();
//Get custodian info
foreach (var ccHolding in ccHoldList)
{
if (correctDepotHolding.ClientNo != ccHolding.ClientNo) continue;
if (correctDepotHolding.Depot != ccHolding.Depot) continue;
if (correctDepotHolding.StockCode != ccHolding.StockCode) continue;
if (correctDepotHolding.QuantityHeld != ccHolding.QuantityHeld) continue;
reportLine.Custodian = ccHolding.Custodian;
break;
}
reportList.Add(reportLine);
}

As Pranay says, a join is probably what you want:
var query = from correct in correctDepotHoldings
join ccHolding in ccHoldList
on new { correct.ClientNo, correct.Depot,
correct.StockCode, correct.QuantityHeld }
equals new { ccHolding.ClientNo, ccHolding.Depot,
ccHolding.StockCode, ccHolding.QuantityHeld }
// TODO: Fill in the properties here based on correct and ccHolding
select new StocksHeldInCustody { ... };
var reportList = query.ToList();

You could move the data from the lookup list into a dictionary, with the key being a unique hash of the 3 items you are searching on. Then you will have very quick lookups and save millions of iterations.

Check my full post : Linq Join on Mutiple columns using Anonymous type
Make use of Linq inner join that will do work for you.
var list = ( from x in entity
join y in entity2
on new { x.field1, x.field2 }
equals new { y.field1, y.field2 }
select new entity { fields to select}).ToList();
Join of linq on multiple field
EmployeeDataContext edb= new EmployeeDataContext();
var cust = from c in edb.Customers
join d in edb.Distributors on
new { CityID = c.CityId, StateID = c.StateId, CountryID = c.CountryId,
Id = c.DistributorId }
equals
new { CityID = d.CityId, StateID = d.StateId, CountryID = d.CountryId,
Id = d.DistributorId }
select c;

Use LINQ to join the lists and return it how you like.
eg
var list1 = GetMassiveList();
var list2 = GetMassiveList();
var list3 = from a in list1
join b in list2
on new { a.Prop1, a.Prop2 } equals
new { b.Prop1, b.Prop2 }
select new { a.Prop1, b.Prop2 };
To do your outter join, you can use DefaultIfEmpty()
This example is setting your RIGHT part of the join to a default object (often null) for the cases where a join wasn't made.
eg
from a in list1
join b in list2
on new { a.Prop1, a.Prop2 } equals
new { b.Prop1, b.Prop2 }
into outer
from b in outer.DefaultIfEmpty()
select new
Prop1 = a.Prop1,
Prop2 = b != null ? b.Prop2 : "Value for Prop2 if the b join is null"
}