Look at the following code example.
What it does:
Iterates a bunch of customers. If it already knows the customer, it retrieves the existing database object for that customer (this is the problem-ridden part). Otherwise, it creates a new object (this works fine).
All loans where the social security number matches (CPR) will be added to the new or existing customer.
The problem: it works for new customer objects, but when I retrieve an existing customer object, the loans lose their relation to the customer when saved (CustomerID = null). They are still saved to the database.
Any ideas?
protected void BuildCustomerData()
{
Console.WriteLine(" Starting the customer build.");
var counter = 0;
var recycleCount = 100;
var reportingCount = 100;
var sTime = DateTime.Now;
var q = from c in db.IntermediaryRkos
select c.CPR;
var distincts = q.Distinct().ToArray();
var numbersToProcess = distincts.Count();
Console.WriteLine(" Identified " + numbersToProcess + " customers. " + (DateTime.Now - sTime).TotalSeconds);
foreach (var item in distincts)
{
var loans = from c in db.IntermediaryRkos
where c.CPR == item
select c;
var existing = db.Customers.Where(x => x.CPR == item).FirstOrDefault();
if (existing != null)
{
this.GenerateLoanListFor(existing, loans);
db.Entry(existing).State = System.Data.EntityState.Modified;
}
else
{
var customer = new Customer
{
CPR = item,
};
this.GenerateLoanListFor(customer, loans);
db.Customers.Add(customer);
db.Entry(customer).State = System.Data.EntityState.Added;
}
counter++;
if (counter % recycleCount == 0)
{
this.SaveAndRecycleContext();
}
if (counter % reportingCount == 0)
{
Console.WriteLine(" Processed " + counter + " customers of " + numbersToProcess + ".");
}
}
db.SaveChanges();
}
protected void GenerateLoanListFor(Customer customer, IQueryable<IntermediaryRko> loans)
{
customer.Loans = new List<Loan>();
foreach (var item in loans.Where(x => x.DebtPrefix == "SomeCategory").ToList())
{
var transformed = StudentLoanMap.CreateFrom(item);
customer.Loans.Add(transformed);
db.Entry(transformed).State = System.Data.EntityState.Added;
}
}
EDIT 1:
As pointed out, I am manually setting the state. This is due to the RecycleContext call, which is implemented for maximum db transaction performance:
protected void SaveAndRecycleContext()
{
db.SaveChanges();
db.Dispose();
db = new SolutionDatabase();
db.Configuration.AutoDetectChangesEnabled = false;
db.Configuration.ValidateOnSaveEnabled = false;
}
Existing loan or not, you wipe out the customer.Loans Property when you call
customer.Loans = new List<Loan>();
Related
I'm trying to add multi-selected items to list _AT.SOrderDetails = new List<SOrderDetail>(); but the list is overwritten by the last record selected. The list at right Datagrid only shows data of the last selected record of the left Datagrid instead of all records
foreach (int i in ((GridView)gridControl3.MainView).GetSelectedRows())
{
DataRowView oSOrder = (DataRowView)((GridView)gridControl3.MainView).GetRow(i);
int id = Convert.ToInt32(oSOrder.Row.ItemArray[0]);
if (oSOrder == null)
{
MessageBox.Show("select an item to edit", "Item not yet selected", MessageBoxButtons.OK, MessageBoxIcon.Information);
return;
}
int x = Convert.ToInt32(oSOrder["SOrderID"]);
_Order = db.SOrderTables.FirstOrDefault(o => o.QSOrdersID == x);
SOrderTable oOrder = null;
oOrder = _Order;
if (_Order != null)
{
_AT.SOrderDetails = new List<SOrderDetail>();
if (_Order.SOrderDetails != null)
{
foreach (SOrderDetail oPODItem in _Order.SOrderDetails.ToList())
{
_OrderDetail = new SOrderDetail();
Product oProduct = db.Products.FirstOrDefault(o => o.ProductID == oPODItem.ProductID);
_OrderDetail.ProductID = oPODItem.ProductID;
_OrderDetail.Description = oProduct.Description;
_OrderDetail.Quantity = oPODItem.Quantity;
_OrderDetail.Form = oPODItem.Form;
_OrderDetail.Price = oPODItem.Price;
_AT.SOrderDetails.Add(_OrderDetail);
}
}
}
}
The list at right Datagrid only shows data of the last selected record of the left datagrid
How do I add all values of the selected records, I need help I'm still new, thank you.
i created a new list _tapiwa.SOrderDetails = new List<SOrderDetail>(); So i was now saving information from list _Order.SOrderDetails.ToList(); to the new list i created.
`foreach (int i in ((GridView)gridControl3.MainView).GetSelectedRows())
{
DataRowView oSOrder = (DataRowView)((GridView)gridControl3.MainView).GetRow(i);
int id = Convert.ToInt32(oSOrder.Row.ItemArray[0]);
if (oSOrder == null)
{
MessageBox.Show("select an item to edit", "Item not yet selected", MessageBoxButtons.OK, MessageBoxIcon.Information);
return;
}
int x = Convert.ToInt32(oSOrder["SOrderID"]);
_Order = db.SOrderTables.FirstOrDefault(o => o.QSOrdersID == x);
label1.Text += (oSOrder.Row.ItemArray[0]).ToString()+" "+ (oSOrder.Row.ItemArray[1]).ToString()+" " + (oSOrder.Row.ItemArray[2]).ToString()+ "\n";
SOrderTable oOrder = null;
// db = new MedriveEntities();
oOrder = _Order;
if (_Order != null)
{
if (_tapiwa == null)
{
_tapiwa = new SOrderTable();
_tapiwa.SOrderDetails = new List<SOrderDetail>();
}
foreach (SOrderDetail oPODItem in oOrder.SOrderDetails.ToList())
{
_OrderDetail = new SOrderDetail();
// label2.Text += oPODItem.ProductID.ToString() + " " + oPODItem.Description.ToString() + " " + oPODItem.Quantity.ToString() + "\n";
Product oProduct = db.Products.FirstOrDefault(o => o.ProductID == oPODItem.ProductID);
_OrderDetail.ProductID = oPODItem.ProductID;
_OrderDetail.Description = oProduct.Description;
_OrderDetail.Quantity = oPODItem.Quantity;
_OrderDetail.Form = oPODItem.Form;
_OrderDetail.Price = oPODItem.Price;
_tapiwa.SOrderDetails.Add(_OrderDetail);
}
}
}
RefreshList1();`
I am currently migrating data from an old MS Infopath based application to an asp.net core app. I have written a .net console app to read from four tables (tblcreated, tblPDI, tblHistory, tblDekadenplanung, tblAdv) containing the old data, manipulating some attributes (e.g. dates) and writing it back to the new table structures (table VehicleFiles, Deliveries, CustomerPayment, comments). Because VehicleFiles holds the foreign keys of the deliveries and customer payment data, i have to save them to DB to get the ID and then store the ID in the vehiclefiles row, which might cause a high load due to the 2 separate write processes). Its 30.000 rows in total.
When I have the migration app running on my web server to avoid latency, it starts very fast, processing around 20 rows per second. But as soon as it hits the 300 rows mark it gradually starts to slow down. At the 25000 mark it took about 3 to 4 minutes to process 10 rows yesterday. I just post the code, maybe someone will find obvious flaws or memory leaks in my code and give me notice.
static void Main(string[] args) {
var logRepository = LogManager.GetRepository(Assembly.GetEntryAssembly());
XmlConfigurator.Configure(logRepository, new FileInfo("log4net.config"));
log.InfoFormat("Running as {0}", WindowsIdentity.GetCurrent().Name);
var Configuration = new ConfigurationBuilder().SetBasePath(Path.Combine(AppContext.BaseDirectory)).AddJsonFile("appsettings.json", optional: true).Build();
var serviceProvider = new ServiceCollection().AddDbContext < AutolineContext > ...
// ...Removed bc password
var autolineContext = serviceProvider.GetRequiredService < AutolineContext > ();
var samContext = serviceProvider.GetRequiredService < SAMContext > ();
var luxWebContext = serviceProvider.GetRequiredService < LUX_WEB_SAMContext > ();
var tblCreated = samContext.TblCreated.OrderBy(x = >x.Id).ToList();
var tblPDIAll = samContext.TblPdi.ToList();
var tblHistoryAll = samContext.TblTerminHistorie.ToList();
var tblDekadenplanungAll = samContext.TblDekadenplanung.ToList();
var tblAdvAll = samContext.TblAdV.ToList();
var tblPDI = new TblPdi();
var tblHistory = new TblTerminHistorie();
var tblDekadenplanung = new TblDekadenplanung();
var tblAdv = new TblAdV();
int counter = 0;
var begin = DateTime.Now;
var lastTime = DateTime.Now;
Deliveries delivery = new Deliveries();
CustomerPayments customerPayments = new CustomerPayments();
foreach(var item in tblCreated) {
counter++;
if (counter % 10 == 0) {
Console.WriteLine("\nNächstes Element: " + item.Id + "\nVerarbeitet: " + counter + "\nSekunden seit Beginn: " + (DateTime.Now - begin).TotalSeconds + "\nDauer letzte Zehn: " + (DateTime.Now - lastTime).Seconds);
}
VehicleFiles vf = luxWebContext.OrderNumbers.Where(x = >x.CommissionNumber == item.Kommissionsnummer).Select(x = >x.FkVehicleFileNavigation).SingleOrDefault();
delivery = new Deliveries();
customerPayments = new CustomerPayments();
tblPDI = tblPDIAll.SingleOrDefault(x = >x.Kommissionsnummer == item.Kommissionsnummer);
tblHistory = tblHistoryAll.Where(x = >x.Kommissionsnummer == item.Kommissionsnummer).OrderBy(x = >x.CreationDate).LastOrDefault();
tblDekadenplanung = tblDekadenplanungAll.SingleOrDefault(x = >x.Kommissionsnummer == item.Kommissionsnummer);
tblAdv = tblAdvAll.SingleOrDefault(x = >x.Kommissionsnummer == item.Kommissionsnummer);
// PDI
delivery.Pdichecker = tblPDI.PdiPruefer;
delivery.Pdistatus = tblPDI.PdiOk == "OK" ? "1": "";
if (!string.IsNullOrEmpty(tblPDI.Pdi)) {
log.Info("Speichere PDI-Datum: " + DateTime.ParseExact(tblPDI.Pdi, "dd/MM/yyyy", CultureInfo.InvariantCulture));
delivery.Pdidate = DateTime.ParseExact(tblPDI.Pdi, "dd/MM/yyyy", CultureInfo.InvariantCulture);
}
DateTime dateValue;
if (!string.IsNullOrEmpty(tblPDI.ZurReinigung) && DateTime.TryParse(tblPDI.ZurReinigung.Replace("h", ":"), out dateValue)) {
log.Info("Speichere Reinigung-Datum: " + DateTime.Parse(tblPDI.ZurReinigung.Replace("h", ":")));
delivery.ForCleaning = DateTime.Parse(tblPDI.ZurReinigung.Replace("h", ":"));
}
// Registrierung
delivery.PlateMounted = tblPDI.Montiert == "OK";
delivery.RegistrationStatus = tblPDI.AnmeldungOk == "Ok";
// Auslieferung
if (tblHistory != null && !string.IsNullOrEmpty(tblHistory.DeliveryDate)) {
log.Info("Speichere Plan-Delivery-Datum: " + DateTime.Parse(tblHistory.DeliveryDate.Split(" ")[0]));
delivery.PlanDeliveryDate = DateTime.Parse(tblHistory.DeliveryDate.Split(" ")[0]);
}
if (!string.IsNullOrEmpty(item.DatumAuslieferung)) {
log.Info("Speichere Delivery-Datum: " + DateTime.Parse(item.DatumAuslieferung));
delivery.DeliveryActual = DateTime.Parse(item.DatumAuslieferung);
}
delivery.DeliveryPerson = tblDekadenplanung.Auslieferer;
if (tblDekadenplanung.Teamleiter != null) delivery.TeamLead = DateTime.Parse(tblDekadenplanung.Teamleiter);
delivery.CreationDate = DateTime.Now;
delivery.CreationUser = "MigrationJob";
// Zahlungsart
decimal value;
customerPayments.Value1 = Decimal.TryParse(tblDekadenplanung.Betrag, out value) ? Convert.ToDecimal(tblDekadenplanung.Betrag) : default;
customerPayments.Value2 = Decimal.TryParse(tblDekadenplanung.Betrag2, out value) ? Convert.ToDecimal(tblDekadenplanung.Betrag2) : default;
customerPayments.PaymentTyp1 = string.IsNullOrEmpty(customerPayments.PaymentTyp1) ? PaymentMethodConverter(tblDekadenplanung.Zahlungsart) : null;
customerPayments.PaymentTyp2 = string.IsNullOrEmpty(customerPayments.PaymentTyp2) ? PaymentMethodConverter(tblDekadenplanung.Zahlungsart2) : null;
customerPayments.CreationDate = DateTime.Now;
customerPayments.CreationUser = "MigrationJob";
customerPayments.LastEditedBy = tblDekadenplanung.LastUser;
if (!string.IsNullOrEmpty(tblDekadenplanung.UpdateDate)) customerPayments.LastEditedDate = DateTime.Parse(tblDekadenplanung.UpdateDate);
log.Info("Schreibe CreationDate: " + item.Creationdate.Value.ToUniversalTime());
vf.CreationDate = item.Creationdate.HasValue ? item.Creationdate.Value.ToUniversalTime() : new DateTime();
vf.CreationUser = item.AnlageUser;
vf.Active = item.Archive == "False" ? false: true;
vf.Concluded = item.Abgeschlossen == "Ja" ? true: false;
vf.ConclusionDate =
default; // Nicht gesetzt im alten SAM
vf.HolUndBringDauer = item.HolBringDauer != "0" ? Convert.ToInt32(item.HolBringDauer) : default;
if (delivery.DeliveryActual.HasValue) vf.HolUndBringEnde = delivery.DeliveryActual.Value.AddMonths(Convert.ToInt32(item.HolBringDauer)).AddDays( - 1);
// Kommentare
if (!string.IsNullOrEmpty(tblDekadenplanung.KommentarAus) && tblDekadenplanung.KommentarAus != "-") {
CommentsSam deliveryComment = new CommentsSam {
FkVehicleFile = vf.PkVehicleFile,
Comment = tblDekadenplanung.KommentarAus,
User = "Sys",
CommentType = "Delivery",
Date = DateTime.Now
};
luxWebContext.Add(deliveryComment);
}
if (!string.IsNullOrEmpty(tblPDI.KommentarLog)) {
CommentsSam pdiComment = new CommentsSam {
FkVehicleFile = vf.PkVehicleFile,
Comment = tblPDI.KommentarLog,
User = "Sys",
CommentType = "PDI",
Date = DateTime.Now
};
luxWebContext.Add(pdiComment);
}
if (!string.IsNullOrEmpty(tblAdv.KommentarZah)) {
CommentsSam advComment = new CommentsSam {
FkVehicleFile = vf.PkVehicleFile,
Comment = tblAdv.KommentarZah,
User = "Sys",
CommentType = "ADV",
Date = DateTime.Now
};
luxWebContext.Add(advComment);
}
try {
luxWebContext.Add(delivery);
luxWebContext.Add(customerPayments);
luxWebContext.SaveChanges();
vf.FkDelivery = delivery.PkDelivery;
vf.FkCustomerPayment = customerPayments.PkCustomerPayment;
luxWebContext.Update(vf);
log.Info("Daten erfolgreich geschrieben. VehicleFile-ID: " + vf.PkVehicleFile);
lastTime = DateTime.Now;
delivery = null;
customerPayments = null;
}
catch(Exception e) {
log.Error("Fehler beim Schreiben der Daten: " + e);
}
}
The process memory in visual studio 2019 is capped at ~261 mb. This is what the diagnosis looks like in the first 6 mins:
I have found this blog post that deals with the same problem but I don't know how to apply it to my specific case:
https://weblog.west-wind.com/posts/2014/dec/21/gotcha-entity-framework-gets-slow-in-long-iteration-loops
One of the issues with your code
foreach(var item in tblCreated) {
// ..
VehicleFiles vf = luxWebContext.OrderNumbers.Where(x = >x.CommissionNumber == item.Kommissionsnummer).Select(x = >x.FkVehicleFileNavigation).SingleOrDefault();
assuming luxWebContext is your EF context. For each item in tblCreated you fire a query to SQL server. This is a huge performance hit.
Rather fetch your data before your for loop and convert it to a Dictionary or Lookup.
// Before loop
var orderNumerbsByCommsionNumber = luxWebContext.OrderNumbers.ToLookup(x => x.CommissionNumber)
// Or ToDictionary(x => x.CommissionNumber) if the CommissionNumber is unique.
foreach(var item in tblCreated) {
// ..
VehicleFiles vf = orderNumerbsByCommsionNumber[item.Kommissionsnummer].Select(x => x.FkVehicleFileNavigation).SingleOrDefault();
You can do the same for every where/first/single in your loops. In your example:
tblPDI = tblPDIAll.SingleOrDefault(x = >x.Kommissionsnummer == item.Kommissionsnummer);
tblHistory = tblHistoryAll.Where(x = >x.Kommissionsnummer == item.Kommissionsnummer).OrderBy(x = >x.CreationDate).LastOrDefault();
tblDekadenplanung = tblDekadenplanungAll.SingleOrDefault(x = >x.Kommissionsnummer == item.Kommissionsnummer);
tblAdv = tblAdvAll.SingleOrDefault(x = >x.Kommissionsnummer == item.Kommissionsnummer);
if you want you can batch commit, add this line to the end of your loop
if(counter % 100 = 0)
{
luxWebContext.SaveChanges(); // Or async version
}
This commits the changes to the database every time the counter is divisible by 100.
Alright, I figured out how to solve this issue. I added the disposal and new creation of my luxWebContext on every foreach iteration with the using keyword, which disposes the luxWebContext automatically after leaving the brackets. Looks like this:
foreach(var item in tblCreated) {
counter++;
using(var luxWebContext = new LUX_WEB_SAMContext(optionsBuilder.Options)) {
if (counter % 100 == 0) {
Console.WriteLine("\nNächstes Element: " + item.Id + "\nVerarbeitet: " + counter + "\nSekunden seit Beginn: " + (DateTime.Now - begin).TotalSeconds + "\nDauer letzte Zehn: " + (DateTime.Now - lastTime).Milliseconds);
lastTime = DateTime.Now;
}
...
I have to update one field in the row of the table after fetching two records from the same row. As an easiest practice I have fetched two records individually, created a new value and then updating that particular property through Entity framework. I think there is a better way to do the same thing with less code. If any body can suggest please.
if (objModel.amountpaid==0)
{
using (estatebranchEntities db=new estatebranchEntities())
{
int rentVar = Convert.ToInt32(db.PropertyDetails.Where(m => m.propertyid == objVM.propertyid).Select(m => m.rent).SingleOrDefault());
int balanceVar = Convert.ToInt32(db.PropertyDetails.Where(m => m.propertyid == objVM.propertyid).Select(m => m.balance).SingleOrDefault());
int balanceUpdateVar = (rentVar + balanceVar);
var propInfo = new PropertyDetail() { balance = balanceUpdateVar };
//var result = (from a in db.PropertyDetails
// where a.propertyid == objVM.propertyid
// select new PropertyDetail
// {
// rent = a.rent,
// balance = a.balance
// }).ToList();
db.PropertyDetails.Attach(propInfo);
db.Entry(propInfo).Property(z => z.balance).IsModified = true;
db.SaveChanges();
}
}
Here is what I think you can do.
Fetch the data once and update once.
using (estatebranchEntities db=new estatebranchEntities())
{
var propDetails = db.PropertyDetails.FirstOrDefault(m => m.propertyid == objVM.propertyid);
if (propDetails != null)
{
int rentVar = Convert.ToInt32(propDetails.rent);
int balanceVar = Convert.ToInt32(propDetails.balance);
int balanceUpdateVar = rentVar + balanceVar;
//now do the update
propDetails.balance = balanceUpdateVar;
db.Entry(proDetails).State = EntityState.Modified;
db.SaveChanges();
}
}
if you need to use the rentVar,balanceVar or the balanceUpdateVar, outside of the using statement then declare them outside it.
I am calling a web api and saving the records on the database through the controller, i want each time im calling the api to check if the record exists in the database if yes then dont save, if not then save.
var client = new WebClient();
var text = client.DownloadString("https://www.test.com/api/all-users?name=testusername%20&pass=334432");
var wclients = JsonConvert.DeserializeObject<dynamic>(text);
List<apicli> list1 = new List<apicli>();
var clie = new apicli();
if (wclients.message == "success")
{
var data = wclients.data;
//var account = wclients.account;
ViewBag.test = data;
foreach(var item in ViewBag.test)
{
clie.Email = item.email;
clie.Name = item.name;
clie.Aff = item.affiliated_id;
foreach(var item1 in #item.account.real)
{
clie.Login = item1.login;
clie.password = item1.pass;
}
list1.Add(clie);
db.apiclis.AddRange(list1);
db.SaveChanges();
};
}
I would assume you need something like this, although you need to check what is the unique id of each record:
foreach(var item in data){
var c = new apicli {
Email = item.email,
Name = item.name,
Aff = item.affiliated_id
Login = item.account.real.LastOrDefault()?login??"",
Login = item.account.real.LastOrDefault()?pass??""
}
if(!db.apiclis.Any(a => a.Email == c.Email && a.Name == c.Name && a.Aff == c.Aff)){
db.apiclis.Add(c);
}
}
Here I assume that email+name+aff = unique identificator.
I have mission system in MVC. and I give a same mission or diffrent mission for a lot of users. and I show title, description, start date , finish date and who are/is in mission. These are showing view page in grid.mvc. but when I login I can see every mission. I dont want to every mission I just want to see only my mission. of course other users see their missions.
in my controller, I split names.
this is my Codes,
Controller:
TicketDbContext db = new TicketDbContext();
public ActionResult Index()
{
var result = db.Missions.OrderByDescending(x=>x.GivenDate).ToList();
string ad = string.Empty;
foreach (var item in result)
{
if (!string.IsNullOrEmpty(item.GivenUsers))
{
string[] Ids = item.GivenUsers.Split(',');
for (int i = 0; i < Ids.Length; i++)
{
int id = Convert.ToInt32(Ids[i]);
item.GivenUsers += db.Users.FirstOrDefault(x => x.Id == id).Name+ " " + db.Users.FirstOrDefault(x => x.Id == id).Surname+ ",";
}
}
}
return View(result);
}
ScreenShot of my Grid
firstly, I recommend that you keep the assigned users ids in separated table
This is can handle your case..
var currentUserId = "";// set current userId from Where are you holding (session,identity etc..)
var result = db.Missions.OrderByDescending(x=>x.GivenDate).ToList();
result = result.Where(x=> x.GivenUsers.Split(',').Contains(currentUserId));
foreach (var item in result)
{
if (!string.IsNullOrEmpty(item.GivenUsers))
{
int[] Ids = Array.ConvertAll(item.GivenUsers.Split(','),
delegate(string s) { return int.Parse(s); }) ;
string[] names = db.Users.Where(u => Ids.Contains(u.Id)).Select(u => u.Name+ " " + u.Surname).ToArray();
item.GivenUsers = string.Join(",",names);
}
}