I am writing a parser for one of website, which have products connected to categories. I am trying to build my own database with these items.
I have decided to use Entity Framework, but I am new to this framework, so here's my problem:
During parsing i have multiple items with same category. But categories are kind of trees. I mean, category have a reference to parentCategory.
During of parsing i have a list of category inheritance f.e : category1 -> category1.1 -> category1.1.1
Each product I parse and add to database need to verify if that category exist and go through category inheritance to create non existing categories.
Code looks like this:
Category parentCategory = null;
foreach (var sCategory in categories)
{
var currentCategory = d.CategorySet.SingleOrDefault(category => category.Name == sCategory && category.Parent == parentCategory);
if (currentCategory == null)
{
currentCategory = new Category(){Name = sCategory,Parent = parentCategory};
if(parentCategory != null)
d.Entry(parentCategory).State = EntityState.Unchanged;
}
parentCategory = currentCategory;
}
But in this case, SingleOrDefault LinQ does not work because of exception:
Unable to create a constant value of type 'DataBaseModel.Category'. Only primitive types or enumeration types are supported in this context.
I know that I should compare IDs of category, but in this case it needs to saveChanges into db every time I add sth to DB.
Is there any other possibility to handle that?
I have solved this issue by creating local Dictionaries of Categories and before usage, fill this dictionaries by data from database.
_categoriesDictionary.Clear();
foreach (var category in this.Container.CategorySet)
{
Category temp = category;
string fullCategoryString = "";
while (temp != null)
{
fullCategoryString = fullCategoryString.Insert(0, temp.Name + ";");
temp = temp.Parent;
}
_categoriesDictionary.Add(fullCategoryString, category);
}
And then when analyzing the record:
Category parentCategory = null;
string fullCatString = "";
foreach (var sCategory in categories)
{
fullCatString += sCategory + ";";
Category currentCategory;
if (!_categoriesDictionary.TryGetValue(fullCatString, out currentCategory))
{
currentCategory = new Category()
{
Name = sCategory,
Parent = parentCategory
};
this.Container.CategorySet.Add(currentCategory);
_categoriesDictionary.Add(fullCatString, currentCategory);
}
parentCategory = currentCategory;
}
result.Category = parentCategory;
This has another adventage from my point of view:
Its collecting data on start, and then do not query DB every time
Related
I am trying to send data snapshots to Firebase database. Which is mostly working fine, but I am having an issue where, instead of child objects being attached directly to their intended parents, they are being added to an additional child that is attached to the intended parent.
public void CreateCampaign()
{
campaignName = campaignNameText.text;
ownerName = pInfo.userName;
if (string.IsNullOrEmpty(campaignName))
{
DebugLog("invalid campaign name.");
return;
}
DebugLog(String.Format("Attempting to add campaign ", campaignName, ownerName));
DatabaseReference reference = FirebaseDatabase.DefaultInstance.GetReference("Users").Child(pInfo.userName).Child("Campaigns").Push();
DebugLog("Running Transaction...");
reference.RunTransaction(AddCampaignTransaction)
.ContinueWith(task =>
{
if (task.Exception != null)
{
DebugLog(task.Exception.ToString());
}
else if (task.IsCompleted)
{
DebugLog("Campaign " + campaignName + " added successfully.");
}
});
}
TransactionResult AddCampaignTransaction(MutableData mutableData)
{
List<object> Campaigns = mutableData.Value as List<object>;
if (Campaigns == null)
{
Campaigns = new List<object>();
}
Dictionary<string, object> newCampaignMap = new Dictionary<string, object>();
newCampaignMap["CampaignName"] = campaignName;
newCampaignMap["Owner"] = pInfo.userName;
newCampaignMap["Members"] = 0;
Campaigns.Add(Child(newCampaignMap));
mutableData.Value = Campaigns;
return TransactionResult.Success(mutableData);
InitializeCampaign();
}
So with this all my data is added to the database, however my data structure looks like this.
Users
User Name
Campaigns
pushID
0
Campaign Name
Owner Name
Memebrs
What i need to know is; how can I prevent the child "0" from being placed between pushID and the three keys I'm adding, so that my data structure looks like this.
Users
User Name
Campaigns
pushID
Campaign Name
Owner Name
Members
Like mentioned in Gazihan's answer, the problem arises from uploading a list (Campaigns) to the database and not the node you desire (newCampaignMap).
Transactions are used where you need to modify data that already exists on the server in a ordered fashion. Because of your use of Push() to generate a new database reference, this is not needed and SetValueAsync can be used instead.
In your code for AddCampaignTransaction above, you generate an empty list (because there is never existing data), add the value of newCampaignMap to that list and then upload the list to the database instead of just the new value. In this function, you are also making use of shared internal variables from the CreateCampaign function which is bad practice, especially when dealing with asynchronous code.
Instead, I propose using the following code:
public void CreateCampaign()
{
// take local copies of values
string campaignName = campaignNameText.text;
string ownerName = pInfo.userName;
if (string.IsNullOrEmpty(campaignName))
{
DebugLog("invalid campaign name.");
return;
}
DebugLog(String.Format("Attempting to add campaign '{0}' for '{1}'... ", campaignName, ownerName));
// structure data
Dictionary<string, object> newCampaignMap = new Dictionary<string, object>();
newCampaignMap["CampaignName"] = campaignName;
newCampaignMap["Owner"] = pInfo.userName;
newCampaignMap["Members"] = 0;
DebugLog("Adding to database... ");
// get reference and upload data
DatabaseReference reference = FirebaseDatabase.DefaultInstance.GetReference("Users").Child(pInfo.userName).Child("Campaigns").Push();
reference.SetValueAsync(newCampaignMap)
.ContinueWith(task =>
{
if (task.IsFaulted)
{
DebugLog(task.Exception.ToString());
return;
}
DebugLog("Campaign " + campaignName + " added successfully.");
});
}
You are pushing a List with a single element. The 0 signifies that it is the 0th element. If there were more elements in the List, you would see 1, 2, etc.
You should replace this line
mutableData.Value = Campaigns;
with this line
mutableData.Value = newCampaignMap;
and try again.
Also you can get rid of the Campaigns etc. that are not used anymore.
it is my query:
from customer in db.tblCustomers
select new
{
ID = customer.CustomerID,
Mobile = customer.Mobile1,
LastName = customer.Family
};
for every customer there is tow mobile phones, I need to add a new entry if the second mobile phone is not null. also I should change the LastName for second entry to something like "Second Mobile". How can I get two different entry from one customer using linq query?
Using the same generated type you can't have one with only one property of phone number and another with two. You can do:
from customer in db.tblCustomers
select new
{
ID = customer.CustomerID,
Mobile = customer.Mobile1,
SecondMobile = customer.Mobile2, // will be null if no second mobile exists
LastName = customer.Family
};
Otherwise what you can do is create a custom type Customer that will have a single phone number and a derived type ExtendedCustomer with two - and just instantiate the one or the other. Something along the psudo:
from customer in db.tblCustomers
select customer.Mobile2 != null ? new Customer(...) : new ExtendedCustomer(...);
If what you mean is having two different objects in the resulted collection then use union:
List<Customer> result = new List<Customer>();
foreach(var item in db.tblCustomers)
{
result.Add(new Customer(/*data for first mobile phone*/);
if(item.Mobile2 != null)
{
result.Add(new Customer(/*data for second mobile phone*/);
}
}
Could you please try this if it helps?
var customers = db.tblCustomers.SelectMany(x => x.GetMultipleRow()).ToList();
GetMultipleRow is an extension method as below.
public static class CustomerExtensions
{
public static IEnumerable<Customer> GetMultipleRow(this Customer cust)
{
yield return new Customer { CustomerID = cust.CustomerID, Mobile1 = cust.Mobile1, Family = cust.Family };
/* Data for first mobile*/
if (cust.Mobile2 != null)
yield return new Customer { CustomerID = cust.CustomerID, Mobile1 = cust.Mobile2, Family = cust.Family };
/* Data for second mobile*/
}
}
I have this list of 2000+ categories that need to be organized in a tree before being sent to the controller and the View so the javascript plugin can render them correctly.
I am already doing this but the performance is terrible. It is taking like 30 seconds to assemble the tree.
I can't see what is dropping performance here. Can you guys help me to improve this code?
var allCategories = dal.Listar();
List<Model.Entity.CategoriaCursoEADVO> nestedCategories = new List<Model.Entity.CategoriaCursoEADVO>();
foreach (Model.Entity.CategoriaCursoEAD item in allCategories)
{
if (item.IdCategoriaPai == null)
{
CategoriaCursoEADVO child = new CategoriaCursoEADVO();
child.id = item.Id;
child.text = item.Descricao;
nestedCategories.Add(child);
FillChild(allCategories, child, item.Id);
}
}
And here is the FillChild method:
public int FillChild(IEnumerable<CategoriaCursoEAD> categorias, CategoriaCursoEADVO parent, int IID)
{
var childCategories = categorias.Where(w => w.IdCategoriaPai.Equals(IID));
parent.children = new List<CategoriaCursoEADVO>();
if (childCategories.Count() > 0)
{
foreach (CategoriaCursoEAD cat in childCategories)
{
CategoriaCursoEADVO child = new CategoriaCursoEADVO();
child.id = cat.Id;
child.text = cat.Descricao;
parent.children.Add(child);
FillChild(categorias, child, cat.Id);
}
return 0;
}
else
{
return 0;
}
}
I think the problem is with the new instances and tried using Parallel loops with no satisfatory level of improvement.
This is a pretty good time to use a HashTable (Dictionary). Something like the below code should help.
// Convert the flat list into a hash table with the ID
// of the element as the key
var dict = allCategories.ToDictionary (i => i.Id);
// Group categories by the parent id
var parentGrouping = allCategories.Where(c => c.IdCategoriaPai != null).GroupBy(c => c.ParentId);
// Since we group the items by parent id, we can find
// the parent by id in the dictionary and add the children
// that have that particular id.
foreach(var groupItem in parentGrouping)
if(groupItem.Key != null)
dict[(int)groupItem.Key].children.AddRange(groupItem);
// Get the root elements.
var hierarchicalCategories = allCategories.Where(item => item.IdCategoriaPai == null);
// Do what you need to do here.
This code will create a tree of categories. hierarchicalCategories will contain direct references to the root elements (categories that do not have a parent), assuming that your data is structured that way.
So i have a function that gets a list of students from a web service and also query the localdb for all the students in there. the data is placed in two different list. So i want to check to see if a new student already exists in the localdb List. if it does, update it and it if doesn't then add it. i unable to get it working . I am trying to perform this using LINQ, but i can't seem to get it working right. My LINQ skills are amateurish at best.
public async Task GetStudents()
{
String controllerName = "Students";
List<Students> newStudentData = await RunGetAsync<Students>(controllerName);
// get all the service types that already exists in the localStudent Db
List<Students> currentStudentData = db.Studentss.ToList();
foreach (Students existingStudents in currentStudentData)
{
foreach (Students newStudents in newStudentData)
{
IEnumerable<Students> selectStudents = from student in newStudentData // check if Students exist in the database
where student.Id == existingStudents.Id
select student;
if (selectStudents == null) // didn't find it, then add it
{
db.Students.Add(newStudents);
}
if (selectStudents != null) // found it , then update the informations
{
Students updatedStudents = new Students();
foreach (var field in selectStudents)
{
updatedStudents.FName = field.FName;
updatedStudents.LName = field.LName;
updatedStudents.ZipCode = field.ZipCode;
updatedStudents.AccessCode = field.AccessCode;
}
db.Entry(updatedStudents).State = System.Data.Entity.EntityState.Modified;
}
}
}
db.SaveChanges();
}
Thank you very much for your help.
you're looping more than you need :
foreach (Students newStudents in newStudentData)
{
var student = currentStudentData.FirstOrDefault(s => s.Id == newStudents.Id);
if(student == null)
{
//add
}
else
{
//update
}
}
with FirstOrDefault you can find out if it exists and get a reference to it at the same time, if it does.
You could use Intersect and Except like below:
//Find students that already exist to update
var updateStudents = currentStudentData.Intersect(newStudentData);
//Find new students to add
var addStudents = newStudentData.Except(currentStudentData);
Say, I have the following conceptual model, there are strories that have tags (more than one, so it's a many-to-many relationship), plus each tag belongs to a particular category.
My data comes from an external source and before inserting it I want to make sure that no duplicated tags are added.
Updated code snippet:
static void Main(string[] args)
{
Story story1 = new Story();
story1.Title = "Introducing the Entity Framework";
story1.Tags.Add(new Tag { Name = ".net", });
story1.Tags.Add(new Tag { Name = "database" });
Story story2 = new Story();
story2.Title = "Working with Managed DirectX";
story2.Tags.Add(new Tag { Name = ".net" });
story2.Tags.Add(new Tag { Name = "graphics" });
List<Story> stories = new List<Story>();
stories.Add(story1);
stories.Add(story2);
EfQuestionEntities db = new EfQuestionEntities();
Category category = (from c in db.Categories
where c.Name == "Programming"
select c).First();
foreach (Story story in stories)
{
foreach (Tag tag in story.Tags)
{
Tag currentTag = tag;
currentTag = GetTag(tag.Name, category, db);
}
db.Stories.AddObject(story);
}
db.SaveChanges();
}
public static Tag GetTag(string name, Category category, EfQuestionEntities db)
{
var dbTag = from t in db.Tags.Include("Category")
where t.Name == name
select t;
if (dbTag.Count() > 0)
{
return dbTag.First();
}
var cachedTag = db.ObjectStateManager.GetObjectStateEntries(EntityState.Added).
Where(ose => ose.EntitySet == db.Tags.EntitySet).
Select(ose => ose.Entity).
Cast<Tag>().Where(x => x.Name == name);
if (cachedTag.Count() != 0)
{
return cachedTag.First();
}
Tag tag = new Tag();
tag.Name = name;
tag.Category = category;
db.Tags.AddObject(tag);
return tag;
}
However, I get an exception about an object with the same EntityKey that is already present in the ObjectContext.
Also, if I remove the else statement I will get an exception about violating an FK constraint, so it seems like its Category attribute is set to null.
I 've had the same problem with EF. Here's what I ended up doing:
Instead of doing story1.Tags.Add(new Tag { Name = ".net", }) yourself, routed all Tag creation through a helper method like this: story1.Tags.Add(GetTag(".net")).
The GetTag method checks the tags in the context to see if it should return an existing entity, like you do. If it does, it returns that.
If there is no existing entity, it checks the ObjectStateManager to see if there are Tag entities added to the context but not already written to the db. If it finds a matching Tag, it returns that.
If it still has not found the Tag, it creates a new Tag, adds it to the context, and then returns it.
In essence this will make sure that no more than one instance of any Tag (be it already existing or just created) will be used throughout your program.
Some example code lifted from my project (uses InventoryItem instead of Tag, but you get the idea).
The check in step 3 is done like this:
// Second choice: maybe it's not in the database yet, but it's awaiting insertion?
inventoryItem = context.ObjectStateManager.GetObjectStateEntries(EntityState.Added)
.Where(ose => ose.EntitySet == context.InventoryItems.EntitySet)
.Select(ose => ose.Entity)
.Cast<InventoryItem>()
.Where(equalityPredicate.Compile())
.SingleOrDefault();
if (inventoryItem != null) {
return inventoryItem;
}
If the Tag is not found in step 3, here's the code for step 4:
inventoryItem = new InventoryItem();
context.InventoryItems.AddObject(inventoryItem);
return inventoryItem;
Update:
It should be used like this:
Story story1 = new Story();
story1.Title = "Introducing the Entity Framework";
story1.Tags.Add(GetTag(".net", category, db));
story1.Tags.Add(GetTag("database", category, db));