I am writing a small application that does a lot of feed processing. I want to use LINQ EF for this as speed is not an issue, it is a single user app and, in the end, will only be used once a month.
My questions revolves around the best way to do bulk inserts using LINQ EF.
After parsing the incoming data stream I end up with a List of values. Since the end user may end up trying to import some duplicate data I would like to "clean" the data during insert rather than reading all the records, doing a for loop, rejecting records, then finally importing the remainder.
This is what I am currently doing:
DateTime minDate = dataTransferObject.Min(c => c.DoorOpen);
DateTime maxDate = dataTransferObject.Max(c => c.DoorOpen);
using (LabUseEntities myEntities = new LabUseEntities())
{
var recCheck = myEntities.ImportDoorAccess.Where(a => a.DoorOpen >= minDate && a.DoorOpen <= maxDate).ToList();
if (recCheck.Count > 0)
{
foreach (ImportDoorAccess ida in recCheck)
{
DoorAudit da = dataTransferObject.Where(a => a.DoorOpen == ida.DoorOpen && a.CardNumber == ida.CardNumber).First();
if (da != null)
da.DoInsert = false;
}
}
ImportDoorAccess newIDA;
foreach (DoorAudit newDoorAudit in dataTransferObject)
{
if (newDoorAudit.DoInsert)
{
newIDA = new ImportDoorAccess
{
CardNumber = newDoorAudit.CardNumber,
Door = newDoorAudit.Door,
DoorOpen = newDoorAudit.DoorOpen,
Imported = newDoorAudit.Imported,
RawData = newDoorAudit.RawData,
UserName = newDoorAudit.UserName
};
myEntities.AddToImportDoorAccess(newIDA);
}
}
myEntities.SaveChanges();
}
I am also getting this error:
System.Data.UpdateException was unhandled
Message="Unable to update the EntitySet 'ImportDoorAccess' because it has a DefiningQuery and no element exists in the element to support the current operation."
Source="System.Data.SqlServerCe.Entity"
What am I doing wrong?
Any pointers are welcome.
You can do multiple inserts this way.
I've seen the exception you're getting in cases where the model (EDMX) is not set up correctly. You either don't have a primary key (EntityKey in EF terms) on that table, or the designer has tried to guess what the EntityKey should be. In the latter case, you'll see two or more properties in the EDM Designer with keys next to them.
Make sure the ImportDoorAccess table has a single primary key and refresh the model.
Related
I have function which inserts record in database. I want to make sure that there are no duplicate entries in database. Function first checks if there is query string parameter. If there is, then it acts like edit mode otherwise insert mode. There is a function which can return currently added records in database. I need to check duplication based on two columns before insertion in database.
myService = new myService();
myFlow mf = new myFlow();
if (!string.IsNullOrEmpty(Request["myflowid"]))
{
mf = myService.Getmyflow(Convert.ToInt32(Request["myflowid"]));
}
int workcount = 0;
int.TryParse(txtWorkCount.Text, out workcount);
mf.Name = txtName.Text.Trim();
mf.Description = txtDescription.Text.Trim();
mf.FunctionCode = txtFunctioneCode.Text.Trim();
mf.FunctionType = txtFunctioneType.Text.Trim();
mf.WorkCount = workcount;
if (mf.WorkFlowId == 0)
{
mf.SortOrder = 0;
mf.Active = true;
mf.RecordDateTime = DateTime.Now;
message = "Saved Successfully";
}
else
{
_editMode = true;
message = "Update Successfully";
}
}
int myflowId = mfService.AddEditmyflow(mf);
I want to check duplication based on functiontype and functioncode. Another function mfService.Getmyflows() can return currently added records in database.
How can I check duplication using Linq?
First of all, what database do you use? Many databases support upsert behavior (update or insert depending of was data found or not). For example, MERGE in ms sql, MERGE in oracle, INSERT .. ON DUPLICATE in mysql and so on. This could be preferred solution. Upsert is usually an atomic operation.
In your particular case do you you transactions? Are you sure no one will insert data after you ensured about duplicates but before you have inserted your record? Example:
#1 thread #2 thread
look for duplicates
... look for duplicate
no duplicates found ...
no duplicates found
insert data_1
insert data_1
This will end up with duplicates you trying to avoid.
According to your code you populating data from GUI and adding only one item.
If you have access to myService code you could add method to query item by your two columns, instead of querying all items via mfService.Getmyflows() and looking through this collection inside your code. It would be more performant (especially if you have indexes in that columns) and more memory efficient.
And finally, existing of a single element inside collection can be easily done:
var alreadyExist = mfService.Getmyflows()
.Any(x => x.Column1 == value1 && x.Column2 == value2);
I am trying to add an entry into a table and use the primary key of that added entry to create an additional entry into another table.
The error I am getting is
The transaction manager has disabled its support for remote/network
transactions. (Exception from HRESULT: 0x8004D024)
I believe this is caused by creating multiple connections within a single TransactionScope, but I am doing everything within one context / using statement, so I do not believe that I should be receiving this error.
Service
[OperationBehavior(TransactionScopeRequired = true)]
public void CreateGroup(NewGroupData data)
{
var groupRepo = _GroupRepo ?? new InvestigatorGroupRepository();
groupRepo.CreateGroup(data.UserId, data.InvestigatorGroupName, data.HasGameAssignment, data.InstitutionId);
}
Repository
public void CreateGroup(string userId, string investigatorGroupName, bool hasGameAssignment, int institutionId)
{
using (var context = new GameDbContext())
{
var newGroup = new InvestigatorGroup()
{
InvestigatorGroupName = investigatorGroupName,
HasGameAssignment = hasGameAssignment,
InstitutionId = institutionId,
IsTrashed = false
};
int institutionUserId =
context.InstitutionUsers.Where(
iu => !iu.IsTrashed && iu.APUser.UserId == userId && iu.InstitutionId == institutionId).Select(iu => iu.InstitutionUserId).Single();
var newGroupUser = new InvestigatorGroupUser()
{
InstitutionUserId = institutionUserId,
InvestigatorGroup = newGroup,
CreationDate = DateTime.Now
};
context.InvestigatorGroupUsers.Add(newGroupUser);
context.SaveChanges();
}
}
You start with a wrong assumption.
The line...
int newGroupId = context.InvestigatorGroups.Add(newGroup).InvestigatorGroupId;
...will always assign 0 to newGroupId. The Add method only marks the entity for insert, but doesn't actually insert it. Only SaveChanges writes data to the database, not any other method in Entity Framework.
So the assignment...
InvestigatorGroupId = newGroupId,
...is faulty as well. You have to assign the new InvestigatorGroup to a navigation property in InvestigatorGroupUser:
InvestigatorGroup = newGroup,
Add this navigation property to InvestigatorGroupUser if you haven't got it yet.
If you have that, it's enough to execute these lines:
context.InvestigatorGroupUsers.Add(newGroupUser);
context.SaveChanges();
No need to Add the newGroup object too, It will be added by adding newGroupUser.
So if you do that, the only transaction you need is the one that SaveChanges uses internally by default. For the code you show, you don't need a TransactionScope. If this is part of a greater WCF transaction the story may be different, but I think at least you needed some misconceptions to be straightened out.
I will try to describe my issue in details.I have the following scenario.
1.) I have 3 tables : business, customoffice(Custom Office) and cusdesc(custom office description)
The relationship is that a business has on customoffice and one customoffice has many cusdesc.
The table business has a field customofficeno which is a foreign key to the field cuscode of the customoffice table.The table cusdesc has a field cuscode which is a foreign key to the field cuscode of the customoffice table.
The objective is to select a business including the custom office and custom office description using entity framework.
2.) Code
I have a procedure FillData which fills a datagrid. My objective is to display fields from the 3 tables. I managed to display data from tables "Business" and "Customoffice" but i need to display the description of a custom office via table "cusdesc" and be more specific the field "CSNAME".
3.) My issue is that when I include the ("CUSTOMSOFFICE.CUSDESC") the results do not contain data from table "CUSDESC" but only how many records much the criteria so I cannot access the field "CSNAME"
Hereafter is the procedure:
using (var _context = new ReftabEntities())
{
try
{
SetGlobalValues();
ObjectQuery<BUSINESS> q_business = _context.BUSINESS.Where("it.BUSINESSNO=" + int.Parse(pv_businessno)).Where(string.Format("(it.BUSINESSSTART <= DATETIME'{0:yyyy-MM-dd HH:mm}') and (it.BUSINESSCLOSED >= DATETIME'{0:yyyy-MM-dd HH:mm}')", pv_date)).Include("CUSTOMSOFFICE").Include("CUSTOMSOFFICE.CUSDESC");
gvBusinessList.Caption = "Total records selected: " + q_business.Count();
gvBusinessList.DataSource = q_business;
gvBusinessList.DataBind();
}
catch (Exception e)
{
errorPopup.Text = e.Message;
errorPopup.ShowOnPageLoad = true;
}
finally
{
_context.Dispose();
}
}
}
Can you please give a hint what I do wrong.
Thanks in advance.
The Include operator just asks EF to load a related entity with the query. If you don't use "Include", EF will only extract the properties of BUSINESS and will not extract the properties of the CUSTOMSOFFICE. There is no need for the "Include("CONSOMSOFFICE.CUSDESC")" since you already loaded the entire CUSTOMSOFFICE entity in the first Include.
I see that you're binding the result to the grid view, and if I understand correctly, the issue is that the "CUSTOMSOFFICE.CUSDESC" is not being displayed in the gridview. I believe this is because the gridview tries to render its representation of the "CUSTOMSOFFICE" object itself, since that's the direct property of the items you are binding. To have more control over the "columns" of your gridview, I suggest using LINQ to transform the results of your query into what you explicitly want to display.
I am assuming that BUSINESSNO, BUSINESSSTART, and BUSINESSCLOSED are properties of your Business entity itself, and that BUSINESSNO is the Primary Key. Let me rewrite your query into this:
var q_business = _context.BUSINESS.Include(b=>b.CUSTOMSOFFICE)
.Where(p => p.BUSINESSNO == int.Parse(pv_businessno)
&& p.BUSINESSSTART <= DateTime.Parse(pv_date)
&& p.BUSINESSCLOSED >= DateTime.Parse(pv_date) )
.FirstOrDefault();
This query would extract the details of the Business (including the related CUSTOMSOFFICE details) that matches the given pv_businessno and falls within your date criteria. But you can't bind this to your gridview yet because you might encounter the same problem where the CUSTOMSOFFICE.DESC is not displayed. To ensure proper display, you must identify what properties you want to include. For example, if you only want to display the set of properties below:
BUSINESS.BUSINESSNO
BUSINESS.BUSINESSNAME
BUSINESS.CUSTOMSOFFICE.CUSCODE
BUSINESS.CUSTOMSOFFICE.CUSDESC
You should transform your output to explicitly and immediately include these properties.
var q_business = _context.BUSINESS.Include(b=>b.CUSTOMSOFFICE)
.Where(b => b.BUSINESSNO == int.Parse(pv_businessno)
&& b.BUSINESSSTART <= DateTime.Parse(pv_date)
&& b.BUSINESSCLOSED >= DateTime.Parse(pv_date) )
.Select(b => new {BusinessNo = b.BUSINESSNO,
BusinessName = b.BUSINESSNAME,
CustomsOfficeCode = b.CUSTOMSOFFICE.CUSCODE,
CustomsOfficeDesc = b.CUSTOMSOFFICE.CUSDESC } ) //This Select statement creates a new anonymous type that has Businessno, BusinessName, CustomsOfficeCode, and CustomsOfficeDesc properties
.FirstOrDefault();
When you bind this to your gridview, it should be able to display the value of the CUSDESC property.
I am currently trying to create a new order (which will be shown below) in a web service, and then send that data to insert a new row into the database. For some reason my DBML / Data Context does not allow me to use InsertOnSubmit.
Any ideas? I haven't used Linq to Sql in about 7 months.
Thanks in advance.
[WebMethod]
public string InsertOrderToDatabases()
{
//Start Data Contexts ------
DataContext db = new DataContext(System.Configuration.ConfigurationManager.AppSettings["RainbowCMSConnectionString"]);
DataContext dcSqlOES = new DataContext(System.Configuration.ConfigurationManager.AppSettings["OESConnectionString"]);
//Get table from local database
Table<Schedule> Schedule = db.GetTable<Schedule>();
//Find last order number in databases
var lastOrderNumber = from lOrder in Schedule
orderby lOrder.templ_idn descending
select lOrder.templ_idn;
int firstOrderID;
var firstOrder = lastOrderNumber.FirstOrDefault();
firstOrderID = firstOrder.Value + 1;
qrOrder qrOrd = new qrOrder
{
.... data in here creating a new order
};
//TODO: fix below with an insert on submit
if (qrOrd != null)
{
// **Schedule.InsertOnSubmit(qrOrd);**
}
//db.GetTable<Schedule>().InsertOnSubmit(qrOrd);
try
{
//Submit the changes to the database
db.SubmitChanges();
return "Orders were sent to the databases.";
}
catch ()
{
}
}
Based on your response, it appears that you are using the wrong table, or perhaps the wrong data type. I also noticed that when you declare your localSchedule variable, you declare it as type Table<Schedule>, which means it should contain Schedule entities, not qrOrder entities.
Table<TEntity>.InsertOnSubmit expects a specific strongly typed entity to be passed in. In your case, it is expecting Web_Service.Schedul‌e, but you are trying to pass in a qrOrder.
Schedule.InsertOnSubmit(qrOrd);
That line will not treat to submit changes to connected entity , Try this
db.Schedule.InsertOnSubmit(qrOrd);
db.SubmitChanges();
you can try with
db.GetTable(typeof(Schedule)).InsertOnSubmit(qrOrd);
Or
db.GetTable(qrOrd.GetType()).InsertOnSubmit(qrOrd);
I am getting a voialation of Primary key error with the following code. I can't see for looking as to why this may be and I need to sort it out. Can anybody help with a fresh pair of eyes?
var events = (from e in nodes.Descendants("event")
select new Event
{
Event_ID = int.Parse(e.Attribute("event_id").Value),
Name = e.Attribute("name").Value,
Code = e.Attribute("code").Value,
Minute = e.Attribute("minute").Value != String.Empty ? int.Parse(e.Attribute("minute").Value) : 0,
Minute_Extra = e.Attribute("minute_extra").Value != String.Empty ? int.Parse(e.Attribute("minute_extra").Value) : 0,
Team = GetTeam(e.Attribute("team_id")),
Last_Updated = DateTime.Parse((FormatDateTime(e.Attribute("last_updated").Value)))
});
foreach (Event matchEvent in events)
{
//Check to see if this event exists
if (match.Events.Any(o => o.Event_ID == matchEvent.Event_ID))
{
Event theEvent = (from m in match.Events
where m.Event_ID == matchEvent.Event_ID
select m).FirstOrDefault();
//There is an event with this ID, so check its last updated flag
if (theEvent.Last_Updated < matchEvent.Last_Updated)
{
//Update the current event
theEvent.Event_ID = matchEvent.Event_ID;
theEvent.Name = matchEvent.Name;
theEvent.Code = matchEvent.Code;
theEvent.Minute = matchEvent.Minute;
theEvent.Minute_Extra = matchEvent.Minute_Extra;
theEvent.Team = matchEvent.Team;
theEvent.Last_Updated = matchEvent.Last_Updated;
}
}
//If the event is not there we need to add it
else
{
match.Events.Add(matchEvent);
}
myDb.SaveChanges();
UPDATE 1: The following is the error I get when SaveChanges() is called:
{"Violation of PRIMARY KEY constraint 'PK_Matches_1'. Cannot insert duplicate key in object 'dbo.Matches'.\r\nThe statement has been terminated."}
UPDATE 2: I am not using identity insert on the DB table for this as this is an import from a 3rd party Web Service where I need to retain all Ids. I am not sure if this will affect the update process with entity framework?
UPDATE 3: Ok well when I turn on identity insert the update is successful however I dont wish to have indentity insert on this table as the Ids are passed in from a WebService and I need to retain these Ids.
I'm not sure, because I'm not too hot on Entity Framework, but do you need this line?
theEvent.Event_ID = matchEvent.Event_ID;
It comes just after
//There is an event with this ID, so check its last updated flag
if (theEvent.Last_Updated < matchEvent.Last_Updated)
and I would think it's redundant, and also might cause a Primary Key error, as I don't think you can assign to a primary key once it's been created.
Update
Did a quick search, and you can't update a primary key once it's been created, so I'm pretty sure this is where your error is.
See this SO answer: Update primary key value using entity framework
I believe that your problem lies when you are updating the Event_ID property. The object you requested from your database through the match DBContext already contains that same Event_ID as the Web service. Therefore, there is no need to update this value
if (theEvent.Last_Updated < matchEvent.Last_Updated)
{
//Update the current event
theEvent.Event_ID = matchEvent.Event_ID; // <-- Delete this line.
...
}
You may ask, why it matters since both values are the same? As it so happens the DBContext keeps track of your objects and its changes. In the context itself, each property has an original and current value. When you assigned the same value to the Event_ID the context is going to interpret it as a compleately different value even though is the same.
I hope this helps.