Develop Reference

Match data from two list<T> where date from one list falls between dates from second list

I have a list of objects (Pulled from SQL DB) with a TransactionDate for each object “alarmHistoryList”.
I have another list of objects (Pulled from SQL DB) and each object has a StartDate a FinishDate and an ID “RunLogList”.
There will be a Many to One relationship where “List1” will be the many and “RunLogList” the one. Each Run may have many Alarms.
I want every object in “alarmHistoryList” returned with the ID of the object in “RunLogList” where the TransactionDate fall between the StartDate and the FinishDate.
private void MatchRunData()
{
foreach (var alarm in _alarmHistoryList)
{
var AlarmTransTime = alarm.TransactionTime;
var filteredData = _FARunLogList.Where(t =>
t.TrackInTime > AlarmTransTime && t.TrackOutTime < AlarmTransTime);
}
}

Run logs with alarms matching the run log time window:
var runLogAlarms = new Dictionary<RunLog, IList<Alarm>>();
foreach (var alarm in _alarmHistoryList)
{
var alarmTransTime = alarm.TransactionTime;
var filteredData = _FARunLogList
.Where(t => t.TrackInTime > alarmTransTime && t.TrackOutTime < alarmTransTime)
.ToList();
foreach (var runLog in filteredData)
{
if (runLogAlarms.TryGetValue(runLog, out var alarmsValue))
{
alarmsValue.Add(alarm);
}
else
{
runLogAlarms[runLog] = new List<Alarm> { alarm };
}
}
}

I came up with an answer before Prolog that works. I am sure the answer Prolog gave works as well and is cleaner but I am posting my answer since it is the one I will be using.
private void MatchRunData()
{
foreach (var alarm in _alarmHistoryList)
{
var AlarmTransTime = alarm.TransactionTime;
foreach (var run in _FARunLogList)
{
var TrackInTime = run.TrackInTime;
var TrackOutTime = run.TrackOutTime;
var ID = run.LogId;
if (AlarmTransTime > TrackInTime && AlarmTransTime < TrackOutTime)
{
_MergedalarmHistoryList.Add
(new AlarmHistoryDefinition()
{ AlarmDesc = alarm.AlarmDesc, AlarmID = alarm.AlarmID, ToolID = alarm.ToolID,
TransactionTime = alarm.TransactionTime, GlobalToolID = alarm.GlobalToolID,
RUnLogID = run.LogId });
}
}
_MergedalarmHistoryList.Add(new AlarmHistoryDefinition()
{ AlarmDesc = alarm.AlarmDesc, AlarmID = alarm.AlarmID, ToolID = alarm.ToolID,
TransactionTime = alarm.TransactionTime, GlobalToolID = alarm.GlobalToolID,
RUnLogID = 00000 });
}
}

Can try this
private void MatchRunData()
{
foreach (var alarm in _alarmHistoryList)
{
var filteredData = _FARunLogList.Where(t =>
t.TrackInTime > alarm.TransactionTime && t.TrackOutTime < alarm.TransactionTime);
alarm.RunLogListId = filteredData.RunLogListId;
}
}

SQL: how to check if a date in the list has changed in iteration asp.net

The scenario that I have so far is, I am fetching the logs from a device and storing it in the db sequentially with Checktype as I and O saying that his first entry was check-in and then check-out sequentially. This worked fine until I came up to an issue like;
Lets assume an emp has 3 logs in the device, 1 log for the date 15-feb-2018 and 2 logs on the date 16-feb-2018, what the query will do is it will insert the records as;
15-feb-2018 I
16-feb-2018 O
16-feb-2018 I
which is definitely wrong. it should be like
15-feb-2018 I
16-feb-2018 I
16-feb-2018 O
Current code snippet:
public static bool inn = true;
public ActionResult GetLogs()
{
if (isDevice2Connected)
{
//.. some code that fetches the logs
if (lstMachineInfo != null && lstMachineInfo.Count > 0)
{
var lastRecord = db.AttLogs.OrderByDescending(x => x.DateTime).FirstOrDefault();
List<Employee> empList = db.Emps.ToList();
var checkSingle = db.Perms.Where(x => x.Name == "Single" && x.IsPermitted == true).ToList();
if (lastRecord != null)
{
lstMachineInfo = lstMachineInfo.Where(x => x.DateTime > lastRecord.DateTime).ToList();
}
foreach(var emp in empList)
{
//this is where it should have some `Date` check
var empLogs = lstMachineInfo.Where(x => x.RegisterationId == int.Parse(emp.EnrollNumber)).ToList();
foreach (var p in empLogs)
{
if (checkSingle.Count > 0)
{
if (inn)
{
inn = false;
p.CheckType = "I";
}
else
{
inn = true;
p.CheckType = "O";
}
}
else
{
p.CheckType = "SINGLE DEVICE DEACTIVATED IN PERMISSIONS CHECK";
}
db.AttLogs.Add(p);
}
}
db.SaveChanges();
}
}
return View("GetAllUserInfo");
}
UPDATE:
trying to get the date from the empsLogs list so I can check if it has changed?
foreach (var p in empLogs)
{
if (checkSingle.Count > 0)
{
if (empLogs.Where(x => x.Date > NEXTDATEINLIST? ))
inn = true;

You're on the right track, all you need is to use p.Date instead of the variable and also a way to increment it like:
foreach(var emp in empList)
{
var empLogs = lstMachineInfo.Where(x => x.RegisterationId == int.Parse(emp.EnrollNumber)).ToList();
var prevDate = (from obj in empLogs select obj.Date).FirstOrDefault();
foreach (var p in empLogs)
{
if (checkSingle.Count > 0)
{
if (prevDate < p.Date) {
inn = true;
prevDate = p.Date;
}
if (inn)
{
inn = false;
p.CheckType = "I";
}
else
{
inn = true;
p.CheckType = "O";
}
}
else
{
p.CheckType = "SINGLE DEVICE DEACTIVATED IN PERMISSIONS CHECK";
}
db.AttendanceLogs.Add(p);
}
}
db.SaveChanges();

MY Asp.Net MVC page takes 12 secs. to load

I have an MVC Controller like this
public ActionResult Index(int vendor=-1, int product = -1, string error="", string mode="Shortfall")
{
if (Session["UserId"] == null)
return RedirectToAction("Index", "Login");
var products = DbContext.GetAllProducts();
List<SurplusViewModel> surplusList = new List<SurplusViewModel>();
Dictionary<int, string> searchVendor = new Dictionary<int, string>();
Dictionary<int, string> searchProds = new Dictionary<int, string>();
if (products.Count() > 0)
{
foreach (var prod in products)
{
SurplusViewModel s = new SurplusViewModel(prod);
surplusList.Add(s);
foreach (var v in s.Vendors)
{
if (!searchVendor.ContainsKey(v.CorpId))
{
searchVendor.Add(v.CorpId, v.CorpName);
}
}
if(!searchProds.ContainsKey(s.ProductId))
searchProds.Add(s.ProductId, s.ProductVM.ProductCode + " / " + s.ProductVM.ProductPartNo);
}
}
ViewData["vendorList"] = searchVendor;
ViewData["productList"] = searchProds;
ViewData["selectVendor"] = vendor;
ViewData["selectProd"] = product;
ViewData["mode"] = mode;
ViewBag.Message = "";
ViewBag.Error = "";
IEnumerable<SurplusViewModel> finalList = surplusList.OrderBy(o => o.Difference).ToList();
if (vendor > 0)
{
Corporation searchcorp = DbContext.GetCorporation(vendor);
finalList = finalList.Where(x => x.VendorNames.IndexOf(searchcorp.CorpName) >= 0);
}
if (product > 0)
{
finalList = finalList.Where(x => x.ProductId == product);
}
if (vendor < 0 && product < 0)
{
if (mode.Equals("Shortfall"))
finalList = finalList.Where(f => f.VendorQuantity - (f.CMQuantity + f.OEMQuantity) < 0);
else if (mode.Equals("Surplus"))
finalList = finalList.Where(f => f.VendorQuantity - (f.CMQuantity + f.OEMQuantity) > 0);
}
return View(finalList);
//return View();
}
This takes about 20 seconds to load on localhost. What can I do to improve my app's loading time. If it takes 20 secs on localhost I assume it will be very slow on the internet. Any suggestions?
EDIT: Code for SurplusViewModel
public SurplusViewModel(Product product)
{
int productId = product.ProductId;
ProductId = productId;
ProductVM = new ProductViewModel(product);
var saleDetsCM = from s in DbContext.GetSalesOrderDetailsFromCM()
where s.ProductId == productId && s.SaleStatus.Equals("Open") && (s.OrderType.ToLower().Equals("prototype") || s.OrderType.ToLower().Equals("production"))
orderby s.SalDetId descending
select s;
var saleDetsOEM = from s in DbContext.GetSalesOrderDetailsFromOEMs()
where s.ProductId == productId && s.SaleStatus.Equals("Open") && (s.OrderType.ToLower().Equals("prototype") || s.OrderType.ToLower().Equals("production"))
orderby s.SalDetId descending
select s;
var shipQty = from s in DbContext.GetAllSalesDets()
where s.ProductId == productId && !s.SaleStatus.Equals("Open") && (s.OrderType.ToLower().Equals("prototype") || s.OrderType.ToLower().Equals("production"))
orderby s.SalDetId descending
select s;
CustomerOrdersFromCMs = saleDetsCM.ToList();
CustomerOrdersFromOEMs = saleDetsOEM.ToList();
VendorOrders = (from p in DbContext.GetPurchaseDetsForProduct(productId)
where p.OrderType != null && (p.OrderType.ToLower().Equals("prototype") || p.OrderType.ToLower().Equals("production"))
select p).ToList();
var poIds = from v in VendorOrders
select v.PodPOId;
BatchPurchaseDetails = DbContext.GetBatchPurchaseForProduct(productId).ToList();
VendorOrderCount = 0;
VendorQuantity = 0;
var purchaseOrds = (from po in DbContext.GetPurchaseOrdersForProduct(productId)
where poIds.Contains(po.POId)
select po).ToList();
List<int> vendIds = new List<int>();
foreach (var po in purchaseOrds)
{
vendIds.Add(po.VendorId.Value);
}
var vendors = from v in DbContext.GetAllCorps()
where vendIds.Contains(v.CorpId)
select v;
foreach (var podet in VendorOrders)
{
double totalbatchqty = 0;
var purdetBatch = DbContext.GetBatchDetailsForPurchaseDet(podet.PodId);
VendorQuantity += podet.Quantity;
foreach (var b in purdetBatch)
{
totalbatchqty += b.Quantity;
VendorQuantity -= b.Quantity;
}
if (totalbatchqty >= podet.Quantity)
{
}
else
{
VendorOrderCount++;
}
}
Vendors = vendors.ToList();
VendorNames = "";
foreach (var vnd in Vendors)
{
VendorNames += vnd.CorpName + ",";
}
if (VendorNames.Length > 0)
{
VendorNames = VendorNames.Substring(0, VendorNames.Length - 1);
}
OEMQuantity = 0;
foreach (var item in CustomerOrdersFromOEMs)
{
OEMQuantity += item.Quantity;
}
CMQuantity = 0;
foreach (var item in CustomerOrdersFromCMs)
{
CMQuantity += item.Quantity;
}
ShipQuantity = 0;
foreach (var item in shipQty)
{
ShipQuantity += item.Quantity;
}
Difference = VendorQuantity - (CMQuantity + OEMQuantity);
//TotalInsideSalesOrder = VendorOrders.Count();
}

You're doing a ton of things wrong, or at least very poorly. First, you're returning all records from various queries, the processing them in memory. This is fine if there are only a few records, but you mention that there are 50 products. How many Vendors? How many Corporations?
You're doing several database queries, and it sounds like you're doing even more queries in sub-objects you instantiate. All of those queries take time. That's why you want to minimize queries by writing more inclusive single queries, and performing as much work on the sql server as possible in a single (or as few as possible) queries.
Areas you can optimize... Don't do Count() > 0, instead use .Any(). .Any returns true after the first record it finds, rather than having to count everything and then compare that number to 0.
Another area, you're doing a foreach inside a foreach. This creates n * m loops. ie, if there's 2 products and 2 vendors, that's 4 loops, but if it's 3 of each, it's 9 loops, if it's 4 of each it's 16 loops. if it's 50 of each 2500 loops. And each one of those loops executes your SurplusViewModel constructor, which if it's a lot of code means it's going to be sloooooooow.
I see from your update, that SurplusViewModel executes at least 7 queries, maybe more.. it's hard to tell. so that's 2500 * 7 or 17,500 Queries (assuming 50 products, and 50 vendors). Are you beginning to see why it's so slow? Now, Imagine you had 100 products and 100 vendors. That's 10,000 loops with 7+ queries, that's at least 70,000 queries. This is not a scalable solution.
Let's look further.. what's in all these "Getxxx" methods? I assume there is some kind of query in each of those? Are you possibly performing double queries each time? Again, you're not including all the information.
To be honest, i'm quite surprised that this ONLY takes 20 seconds... i'd think it would be more like 20 minutes with any amount of data.

Error while Adding rows to a table having 2 foriegn keys

I am using Entity framework 4, SQLite database with database first approach. I have attached my table structure as a snapshot.
I am able to write/ read data in the Bnb_Company and Bnb_Player tables, but when I try to enter a row into the Bnb_Game table, I get the following error.
Entities in 'MockStockDbEntities.Bnb_Game' participate in the 'Bnb_CompanyBnb_Game' relationship. 0 related 'Bnb_Company' were found. 1 'Bnb_Company' is expected.!
//Reset company values
if (context.Bnb_Company.Count() > 0)
{
var companydetails = context.Bnb_Company.ToList();
foreach (var company in companydetails)
{
company.ESC = 0;
company.MarketValue = company.FaceValue;
}
context.SaveChanges();
}
if (context.Bnb_Player.Count() > 0)
{
var playerDetails = context.Bnb_Player.ToList();
//Reset Player values
foreach (var player in playerDetails)
{
player.NetAmount = 0;
player.Amount = 1000;
if (context.Bnb_Company.Count() > 0)
{
var companyDetails = context.Bnb_Company.ToList();
foreach (var company in companydetails)
{
if (context.Bnb_Game.Count() > 0 && (context.Bnb_Game.Where(b => b.PlayerId == player.Id && b.CompanyId == company.Id).Count() > 0))
{
var specificCompanyShare = context.Bnb_Game.Where(b => b.PlayerId == player.Id && b.CompanyId == company.Id).FirstOrDefault();
specificCompanyShare.Shares = company.DefaultShares;
}
else
{
Bnb_Game playerCompanyShare = new Bnb_Game();
playerCompanyShare.CompanyId = company.Id;
playerCompanyShare.PlayerId = player.Id;
playerCompanyShare.Shares = company.DefaultShares;
context.Bnb_Game.AddObject(playerCompanyShare);
}
}
}
}
context.SaveChanges();
}

Try using the Entity reference instead of the the Id value.
Bnb_Game playerCompanyShare = new Bnb_Game();
playerCompanyShare.Bnb_Company = company;
playerCompanyShare.Bnb_Player = player;
playerCompanyShare.Shares = company.DefaultShares;
context.Bnb_Game.AddObject(playerCompanyShare);

LINQ to SQL dividing filters for better performance

I have these below LINQ to SQL queries
var kayitlarFiltreli = from rows in db.TBLP1CARIs
orderby rows.ID descending
where rows.HESAPADI.ToLower().Contains(filter.ToLower()) ||
(rows.CARITURU == "Bireysel" ?
rows.B_ADSOYAD.ToLower().Contains(filter.ToLower()) :
rows.K_FIRMAADI.ToLower().Contains(filter.ToLower())) ||
rows.ID.ToString().Contains(filter)
select rows;
var kayitlarBakiyeli = from rows in kayitlarFiltreli
select new
{
HESAPNO = rows.ID,
HESAPADI = rows.HESAPADI,
CARIADI = (rows.CARITURU == "Bireysel" ? rows.B_ADSOYAD : rows.K_FIRMAADI),
Bakiye = get_bakiye(rows.ID, rows.LISTEPARABIRIMI)
};
var kayitlarSon = from rows in kayitlarBakiyeli
select new
{ rows.HESAPNO,
rows.HESAPADI,
rows.CARIADI,
Bakiye = rows.Bakiye.Contains(".") == true ?
rows.Bakiye.TrimEnd('0').TrimEnd('.') :
rows.Bakiye
};
I am having performance problem I mean the queries response at least after 15secs, and when it is deployed to the website it takes at least 5 secs for the page which is using these queries to fill a GridView.get_bakiye(p1,p2,..) is a long method with a for, a foreach and a Linq-to-SQL query in it.I think the most of time is spent on get_bakiye I struggled with it already and reduced the response time like 2 secs, however it is still slow.And I am trying to get the above queries work faster.
I tried
var kayitlarSirali = from rows in db.TBLP1CARIs
orderby rows.ID descending
select rows;
var kayitlarFiltreli = from rows in kayitlarSirali
where rows.HESAPADI.ToLower().Contains(filter.ToLower()) ||
(rows.CARITURU == "Bireysel" ?
rows.B_ADSOYAD.ToLower().Contains(filter.ToLower()) :
rows.K_FIRMAADI.ToLower().Contains(filter.ToLower())) ||
rows.ID.ToString().Contains(filter)
select rows;
And the rest is the same.
Basically I just seperated the filtering part with Contains(), which I am not sure if that helps so much.
Is it good to seperate where's I mean filters when querying the database, and is it better for performance to query the database once and get the results into an in-memory IQueryable and do the rest on it?
What do you recommend for these queries to work faster?
This is the get_bakiye() method which is not something I wrote fully but I am supposed to make it perform faster.
public static string get_bakiye(int cari_id, string birim_kod)
{
return get_bakiye(cari_id, DAL.DAOCari.GetEntity(cari_id).LISTEPARABIRIMI, null,false);
}
public static string get_bakiye(int cari_id, string birim_kod, List<BAL.P_CariBakiyeTablosu> custom_rapor, bool borcluTespit)
{
VeriyazDBDataContext db = new VeriyazDBDataContext(); db.Connection.ConnectionString = System.Configuration.ConfigurationManager.ConnectionStrings["LocalSqlServer"].ConnectionString;
decimal return_bakiye = 0;
if (birim_kod == null || birim_kod.Trim() == "") //default
birim_kod = "TL";
//devir bakiyesini hesapla:
List<BAL.P_CariBakiyeTablosu> bakiyeler = new List<BAL.P_CariBakiyeTablosu>();
if (custom_rapor == null)
bakiyeler = CariBakiyeRaporuOlustur(cari_id, true);
else
bakiyeler = custom_rapor;
bakiyeler.RemoveAt(0);
List<TBLP1DOVIZTANIMLARI> dovizTanimlariTumListe = DAL.DAOdoviztanimlari.SelectAll().ToList();
//devirleri hesaplarken döviztanimlari tablosundaki varsayılan kuru kullanıyor
for (int i = 0; i < bakiyeler.Count; i++)
{
if (bakiyeler[i].DOVIZ == birim_kod)
{
return_bakiye = return_bakiye + Convert.ToDecimal(bakiyeler[i].DEVIR);
}
else
{
decimal from_kur = 1;
from_kur = from_kur = dovizTanimlariTumListe.Where(rows => rows.TBLP1DOVIZLER.KOD == bakiyeler[i].DOVIZ).FirstOrDefault().VARSAYILANKUR.GetValueOrDefault(1);
decimal to_kur = 1;
to_kur = dovizTanimlariTumListe.Where(rows => rows.TBLP1DOVIZLER.KOD == birim_kod).First().VARSAYILANKUR.GetValueOrDefault(1);
return_bakiye = return_bakiye + (Convert.ToDecimal(bakiyeler[i].DEVIR) * (from_kur / to_kur));
}
}
//islem bakiyesini hesapla:
var islemler = from rows in db.TBLP1ISLEMs
where
rows.CARI_ID == cari_id &&
rows.TEKLIF.GetValueOrDefault(false) == false &&
rows.SOZLESME.GetValueOrDefault(false) == false &&
(rows.SIPARISDURUMU == "İşlem Tamamlandı" ||
rows.SIPARISDURUMU == "Hazırlanıyor" ||
rows.SIPARISDURUMU == "" ||
rows.SIPARISDURUMU == null)
select rows;
//var dovizKuruOlanIslemler = from dovizKuruRow in db.TBLP1DOVIZKURUs
// select dovizKuruRow.ISLEM_ID;
foreach (var item in islemler)
{
decimal from_kur = 1;
decimal fromKurVarsayilan = 1;
//belirtilen dövizin varsayılanını çekiyor
fromKurVarsayilan = dovizTanimlariTumListe.Where(rows => rows.TBLP1DOVIZLER.KOD == item.PARABIRIMI).FirstOrDefault().VARSAYILANKUR.GetValueOrDefault(1);
try
{
from_kur = item.KURDEGERI.Value;
//aşağıdaki satırda dövizkuru tablosundan işleme ait kuru çekerek hesap yapıyordu, işlem tablosuna KURDEGERİ kolonu ekleyince
//buna gerek kalmadı, yukarıdaki satırda işleme ait kur değeri işlem tablosundan çekiyor.
//from_kur = item.TBLP1DOVIZKURUs.Where(rows => rows.DOVIZBIRIM == item.PARABIRIMI).FirstOrDefault().KUR.GetValueOrDefault();
}
catch
{
from_kur = fromKurVarsayilan;
}
//carinin para biriminin varsayılan kurunu çekiyor
decimal to_kur = 1;
decimal toKurVarsayilan = 1;
toKurVarsayilan = dovizTanimlariTumListe.Where(rows => rows.TBLP1DOVIZLER.KOD == birim_kod).FirstOrDefault().VARSAYILANKUR.GetValueOrDefault(1);
to_kur = toKurVarsayilan;
if (item.CARIISLEMTURU == "BORC")
{
return_bakiye = return_bakiye + (Convert.ToDecimal(item.GENELTOPLAM) * (from_kur / to_kur));
}
if (item.CARIISLEMTURU == "ALACAK")
{
return_bakiye = return_bakiye - (Convert.ToDecimal(item.GENELTOPLAM) * (from_kur / to_kur));
}
}
string returnBakiyeParaFormatli = DAL.Format.ParaDuzenle.ParaFormatDuzenle(return_bakiye.ToString());
if (borcluTespit==true)
{
return return_bakiye.ToString();
}
if (returnBakiyeParaFormatli.Contains(".") == true)
{
return returnBakiyeParaFormatli.TrimEnd('0').TrimEnd('.') + " " + birim_kod;
}
else
{
return returnBakiyeParaFormatli + " " + birim_kod;
}
}
}

In general i think you need to understand what causes Linq to Sql to execute a query against your database. In general, extension methods such as ToList(), First(), FirstOrDefault(), Single() will cause Linq To Sql to execute a command against the database. One line that does concern me is:
List<TBLP1DOVIZTANIMLARI> dovizTanimlariTumListe = DAL.DAOdoviztanimlari.SelectAll().ToList();
This seems to be getting every row from the database table that DAOdoviztanimlari is mapped to. The result of this is then queried in memory.
This then happens for every record in the queries that call get_bakiye()!
Ultimately (perfect world) you want get_bakiye() to not contain any of the extension methods i have mentioned and to return IQueryable<string> then let Linq to SQL descide how it optimizes and executes the SQL.

First profile your database and see how long the generated queries take to execute and how often do they execute.
If the results from the profiling show that you are executing the same query against the database or the query takes too long to execute perhaps you should consider loading values into memory and access them from there. Or even consider compiled queries as an alternative if that's plausible.
I had a similar problem a little while ago that I solved by creating a separate class that handles the loading of values that I needed into a collection and updating the values when necessary.