Develop Reference

Is there any better way to filter the data based on grand grand child property in C#?

I have a model which consist multi level of sub models. Now I want to filter the data based on most bottom level of child element. Please see the code (it's just for reference)
var treatmentItemss = new List<TreatmentItemViewModel>();
IEnumerable<IPublishedContent> countries = UmbracoContext.Content.GetById(contentId: 1063).Children();
foreach (var country in countries)
{
IEnumerable<IPublishedContent> countys = country.Children();
foreach (var county in countys)
{
IEnumerable<IPublishedContent> cities = county.Children();
foreach (var city in cities)
{
IEnumerable<IPublishedContent> clinics = city.Children();
foreach (var clinic in clinics)
{
IEnumerable<IPublishedContent> clinics2 = clinic.Children();
foreach (var clinic2 in clinics2)
{
IEnumerable<IPublishedContent> treatmentMenus = clinic2.Children();
foreach (var treatmentMenu in treatmentMenus)
{
IEnumerable<IPublishedContent> treatmentCategories = treatmentMenu.Children();
foreach (var treatmentCategory in treatmentCategories)
{
dynamic treatmentItems = treatmentCategory.Children();
foreach (var item in treatmentItems)
{
treatmentItemss.Add(new TreatmentItemViewModel
{
SurgicalProcedure = item.SurgicalProcedure?.Name,
NonSurgicalTreatment = item.NonSurgicalTreatment?.Name,
Title = item.Title,
Id = item.Id,
});
}
}
}
}
}
}
}
}
In above code I want to fetch the data base on SurgicalProcedure.Name
for example, I want to get all the data of countries in which SurgicalProcedure.Name == "Angioplasty".

You could consider writing an iterator method, that does the dirty job of doing the deep dive in the model, and exposes all the TreatmentItemViewModel as an enumerable sequence. Iterators are methods that contain yield return statements:
IEnumerable<TreatmentItemViewModel> GetTreatmentItems(int contentId)
{
IEnumerable<IPublishedContent> countries = UmbracoContext.Content
.GetById(contentId).Children();
foreach (var country in countries)
{
foreach (var county in country.Children())
{
foreach (var city in county.Children())
{
foreach (var clinic in city.Children())
{
foreach (var clinic2 in clinic.Children())
{
foreach (var treatmentMenu in clinic2.Children())
{
foreach (var treatmentCategory in treatmentMenu.Children())
{
foreach (var item in treatmentCategory.Children())
{
yield return new TreatmentItemViewModel()
{
SurgicalProcedure = item
.SurgicalProcedure?.Name,
NonSurgicalTreatment = item
.NonSurgicalTreatment?.Name,
Title = item.Title,
Id = item.Id,
};
}
}
}
}
}
}
}
}
}
You could then query this sequence with the Where LINQ operator:
TreatmentItemViewModel[] results = GetTreatmentItems(contentId: 1063)
.Where(x => x.SurgicalProcedure == "Angioplasty")
.ToArray();
Enumerating all these TreatmentItemViewModel entities could result in a lot of garbage, so you could consider defining this type as a struct instead of class.

Having nested code like this is the not good at all.
I think you should change your design to support unlimited level of tree grandchildren and ancestors.
To understand the topic checkout this question and answers.

Sorting a list if internal list is empty

I have a list inside of another list. I need to add to the list only those elements whose internal list is not empty. The data is pulled from the server. The code contains 4 lists, Each inside of another - Companies - Shops - Campaigns - Coupons. I need to check if the Coupons list is not empty and add the elements related to that. I would be very thankfull if someone could help with that.
foreach (var row in Companies)
{
reportData.companies.Add(new Companies { ID = row.Id, name = row.BrandName });
var Shops = logicService.CompanyShopService.GetShopsByCompany(row.Id);
var Campaigns = logicService.CampaignService.GetCampaignsByCompany(row.Id);
foreach (var shop in Shops)
{
reportData.companies.Where(item => item.ID == shop.ClientID).FirstOrDefault().shops.Add(new Distribution_Shops { Id = shop.Id, Name = shop.Name });
var shop_campaigns = Campaigns.Where(item => item.ShopID == shop.Id);
foreach (var campaign in shop_campaigns)
{
var coupons = logicService.CouponsService.GetAllByCampaign(campaign.Id, false);
foreach (var company in reportData.companies)
{
foreach (var tmp_shop in company.shops)
{
if (tmp_shop.Id == shop.Id)
{
tmp_shop.campaigns.Add(new Distribution_Campaign { Id = campaign.Id, Name = campaign.CampaignName, coupons = coupons});
}
}
}
}
}
}

A lot if and else conditions repeating code

I have a scenario where I do filters depending of checkbox checked, now I have only 2 checkbox and I need to cover all scenarios into if, else conditionals like:
//List Example:
var projectTechnicians = (from DataRow dr in dtEmployeGuid.Rows
where dr["Title"].ToString().Contains("Project Technician")
select new
{
EmpGuid = (Guid)dr["EmpGuid"]
}).ToList();
if (!chkProjectTechs.Checked && !chkTeamLeader.Checked)
{
foreach (DataRowView list in lstTech.SelectedItems)
{
var selectedEmpGuid = (Guid)list[0];
EmpGuid.Add(selectedEmpGuid);
}
parameters = ToDataTable(EmpGuid);
}
else if (!chkTeamLeader.Checked && chkProjectTechs.Checked)
{
foreach (var technician in projectTechnicians)
{
EmpGuid.Add(technician.EmpGuid);
}
parameters = ToDataTable(EmpGuid);
}
else if (!chkProjectTechs.Checked && chkTeamLeader.Checked)
{
foreach (var teamLeader in teamLeaders)
{
EmpGuid.Add(teamLeader.EmpGuid);
}
parameters = ToDataTable(EmpGuid);
}
else if (chkProjectTechs.Checked && chkTeamLeader.Checked)
{
foreach (var technician in projectTechnicians)
{
EmpGuid.Add(technician.EmpGuid);
}
parameters = ToDataTable(EmpGuid);
foreach (var teamLeader in teamLeaders)
{
EmpGuid.Add(teamLeader.EmpGuid);
}
parameters = ToDataTable(EmpGuid);
}
But I need to Add more checkbox, but foreach checkbox I will add to my form I need to add it to each conditional, and at the final of the day I will get a very long code. Is there another way to do this?

There is an issue in you last else if condition. parameters is getting overwritten by teamleaders empGuids
foreach (var technician in projectTechnicians)
{
EmpGuid.Add(technician.EmpGuid);
}
parameters = ToDataTable(EmpGuid);
foreach (var teamLeader in teamLeaders)
{
EmpGuid.Add(teamLeader.EmpGuid);
}
parameters = ToDataTable(EmpGuid); // Overwriting projectTechnicians added above.
You can simplify your code by using if condition only, check my comments in the code.
// add EmpGuids from lstTech if no check box is selected.
// you need to add all checkboxes not selected in the condition
if (!chkProjectTechs.Checked && !chkTeamLeader.Checked)
{
foreach (DataRowView list in lstTech.SelectedItems)
{
var selectedEmpGuid = (Guid)list[0];
EmpGuid.Add(selectedEmpGuid);
}
}
// if projectTechnicians is checked, then add projectTechnicians EmpGuids to parameters
if (chkProjectTechs.Checked)
{
foreach (var technician in projectTechnicians)
{
EmpGuid.Add(technician.EmpGuid);
}
}
// add team leader Empguids if checked.
if (chkTeamLeader.Checked)
{
foreach (var teamLeader in teamLeaders)
{
EmpGuid.Add(teamLeader.EmpGuid);
}
}
// Add new checkbox conditions here
//
// perform your next operation on parameters here.
// At this point you will have either lstTech employees or employees based on checkbox selection in EmpGuid list.
parameters = ToDataTable(EmpGuid);
EDIT
if you are using LINQ, then you can use below LINQ query to add EmpGuids instead of using foreach, for example.
EmpGuid.AddRange(teamLeader.Select(x=>x.EmpGuid));

Improve code performance

I have a cron-job method which constructs a user's story feed based on user's featured stories, followed categories and followed users.
The final feed is added in the below database table in correct order:
UserFeed table:
Uid StoryListIds
1 3, 23, 45, 3, 6, 234, .....
2 3, 23, 45, 6, 87, 44, .....
3 3, 23, 45, 32, 4, 62, .....
The method is below and contains comments.
The code:
public void ConstructUserFeed()
{
try
{
//get all user ids
var userIds = userBL.GetIds();
//get all featured stories
var featStories = storyBL.GetFeaturedStories();
if (userIds != null && userIds.Count > 0)
{
foreach (var userId in userIds)
{
//integer List to store the stories ids in correct order
List<int> storyIdsLst = new List<int>();
//integer List for excluding duplicates
List<int> exceptStoryIds = new List<int>();
//user feed List to store the stories
var userFeed = new List<UserFeedDTO>();
//string to store the id list so we can add it later in db
string storyIds = "";
if (featStories != null && featStories.Count > 0)
{
foreach (var featStory in featStories)
{
//first add all the ids of featured stories except own user stories, ordered by date
if (featStory.authorId != userId)
{
storyIdsLst.Add(featStory.id);
exceptStoryIds.Add(featStory.id);
}
}
}
//get user's followed categories ids
var followedCategoryIds = userCategoriesBL.GetFollowedCategoryIds(userId);
if (followedCategoryIds != null && followedCategoryIds.Count > 0)
{
foreach (var categoryId in followedCategoryIds)
{
//get the user's 5 latest stories for every followed category
//except own stories and previous stories
var storiesByCateg = storyBL.GetByCategory(5, categoryId, userId, exceptStoryIds);
if (storiesByCateg != null && storiesByCateg.Count > 0)
{
foreach (var storyByCateg in storiesByCateg)
{
userFeed.Add(storyByCateg);
exceptStoryIds.Add(storyByCateg.id);
}
}
}
}
//get user's followed users ids
var followedUserIds = userFollowersBL.GetFollowedUserIds(userId);
if (followedUserIds != null && followedUserIds.Count > 0)
{
foreach (var followedId in followedUserIds)
{
//get the user's 5 latest stories for every followed user
//except own stories and previous stories
var storiesByFollowedUsers = storyBL.GetByFollowedUser(5, followedId, userId, exceptStoryIds);
if (storiesByFollowedUsers != null && storiesByFollowedUsers.Count > 0)
{
foreach (var storyByFollowedUsers in storiesByFollowedUsers)
{
userFeed.Add(storyByFollowedUsers);
}
}
}
}
// order the stories by date
userFeed = userFeed.OrderByDescending(story => story.dateAdded).ToList();
if (userFeed != null && userFeed.Count > 0)
{
foreach (var story in userFeed)
{
//add the story ids after the featured story ids
storyIdsLst.Add(story.id);
}
}
//comma separated list of story ids as string so we can store it in db
storyIds = string.Join(",", storyIdsLst.Select(n => n.ToString()).ToArray());
//create the UserFeed model
UserFeed userFeedModel = new UserFeed();
userFeedModel.userId = userId;
userFeedModel.storyListId = storyIds;
userFeedModel.lastUpdateTime = DateTime.Now;
userFeedBL.AddOrUpdateUserFeed(userFeedModel);
}
uof.Save();
}
}
catch (Exception ex)
{
Console.WriteLine("Error occurred in processing job. Error : {0}", ex.Message);
}
}
The above method takes ~35 seconds to complete for 30 users.
Q: How can I improve my code and performance?

It's hard to say exactly what's causing it to execute so slowly, for that I recommend profiling your question with SQL Server Profiler. Check that your queries is properly asked and doesn't do anything unnecessary.
After that, I would consider asking fewer questions. Since you're doing it in a loop you may benefit from doing fewer, but heavier questions. For example (assuming userFollowersBL.* is querying the DB):
var followedCategoryIds = userCategoriesBL.GetFollowedCategoryIds(userId);
I assume that the signature is something like:
IEnumerable<int> GetFollowedCategoryIds(userId);
Consider changing it to:
IDictionary<int, IEnumerable<int>> GetFollowedCategoryIds(IEnumerable<int> userIds);
Then you'll have a dictionary with the userids and each of their followedCategoryIds in memory before you start your foreach. This way you can send all your result from userBL.GetIds() in one query. It may speed up performance to do it once, instead of 30 times. The same thing goes for userFollowersBL.GetFollowedUserIds(userId).
Now you have decreased the number of queries to you DB with approx 58 times.
public void ConstructUserFeed()
{
try
{
//get all user ids
var userIds = userBL.GetIds();
//get all featured stories
var featStories = storyBL.GetFeaturedStories();
// Fetch all in one query.
IDictionary<int,IEnumerable<int>> allFollowedCategoryIds= userCategoriesBL.GetFollowedCategoryIds(userIds);
// Fetch all in one query
IDictionary<int,IEnumerable<int>> allUserFollowers = userFollowersBL.GetFollowedUserIds(userIds);
if (userIds != null && userIds.Count > 0)
{
foreach (var userId in userIds)
{
//integer List to store the stories ids in correct order
List<int> storyIdsLst = new List<int>();
//integer List for excluding duplicates
List<int> exceptStoryIds = new List<int>();
//user feed List to store the stories
var userFeed = new List<UserFeedDTO>();
//string to store the id list so we can add it later in db
string storyIds = "";
if (featStories != null && featStories.Count > 0)
{
foreach (var featStory in featStories)
{
//first add all the ids of featured stories except own user stories, ordered by date
if (featStory.authorId != userId)
{
storyIdsLst.Add(featStory.id);
exceptStoryIds.Add(featStory.id);
}
}
}
//get user's followed categories ids
var followedCategoryIds = allFollowedCategoryIds[userId]
if (followedCategoryIds != null && followedCategoryIds.Count > 0)
{
foreach (var categoryId in followedCategoryIds)
{
//get the user's 5 latest stories for every followed category
//except own stories and previous stories
var storiesByCateg = storyBL.GetByCategory(5, categoryId, userId, exceptStoryIds);
if (storiesByCateg != null && storiesByCateg.Count > 0)
{
foreach (var storyByCateg in storiesByCateg)
{
userFeed.Add(storyByCateg);
exceptStoryIds.Add(storyByCateg.id);
}
}
}
}
//get user's followed users ids
var followedUserIds = allUserFollowers[userId];
if (followedUserIds != null && followedUserIds.Count > 0)
{
foreach (var followedId in followedUserIds)
{
//get the user's 5 latest stories for every followed user
//except own stories and previous stories
var storiesByFollowedUsers = storyBL.GetByFollowedUser(5, followedId, userId, exceptStoryIds);
if (storiesByFollowedUsers != null && storiesByFollowedUsers.Count > 0)
{
foreach (var storyByFollowedUsers in storiesByFollowedUsers)
{
userFeed.Add(storyByFollowedUsers);
}
}
}
}
// order the stories by date
userFeed = userFeed.OrderByDescending(story => story.dateAdded).ToList();
if (userFeed != null && userFeed.Count > 0)
{
foreach (var story in userFeed)
{
//add the story ids after the featured story ids
storyIdsLst.Add(story.id);
}
}
//comma separated list of story ids as string so we can store it in db
storyIds = string.Join(",", storyIdsLst.Select(n => n.ToString()).ToArray());
//create the UserFeed model
UserFeed userFeedModel = new UserFeed();
userFeedModel.userId = userId;
userFeedModel.storyListId = storyIds;
userFeedModel.lastUpdateTime = DateTime.Now;
userFeedBL.AddOrUpdateUserFeed(userFeedModel);
}
uof.Save();
}
}
catch (Exception ex)
{
Console.WriteLine("Error occurred in processing job. Error : {0}", ex.Message);
}
}
But honestly, this is mostly speculations and may differ from time to time. In my experience you tend to benefit from asking for more data once, than the same data several times.

Very slow runtime with Entity Framework nested loop (using nav properties)

Right now, I'm trying to write a method for a survey submission program that utilizes a very normalized schema.
I have a method that is meant to generate a survey for a team of people, linking several different EF models together in the process. However, this method runs EXTREMELY slowly for anything but the smallest team sizes (taking 11.2 seconds to execute for a 4-person team, and whopping 103.9 seconds for an 8 person team). After some analysis, I found that 75% of the runtime is taken up in the following block of code:
var TeamMembers = db.TeamMembers.Where(m => m.TeamID == TeamID && m.OnTeam).ToList();
foreach (TeamMember TeamMember in TeamMembers)
{
Employee employee = db.Employees.Find(TeamMember.EmployeeID);
SurveyForm form = new SurveyForm();
form.Submitter = employee;
form.State = "Not Submitted";
form.SurveyGroupID = surveygroup.SurveyGroupID;
db.SurveyForms.Add(form);
db.SaveChanges();
foreach (TeamMember peer in TeamMembers)
{
foreach (SurveySectionDetail SectionDetail in sectionDetails)
{
foreach (SurveyAttributeDetail AttributeDetail in attributeDetails.Where(a => a.SectionDetail.SurveySectionDetailID == SectionDetail.SurveySectionDetailID) )
{
SurveyAnswer answer = new SurveyAnswer();
answer.Reviewee = peer;
answer.SurveyFormID = form.SurveyFormID;
answer.Detail = AttributeDetail;
answer.SectionDetail = SectionDetail;
db.SurveyAnswers.Add(answer);
db.SaveChanges();
}
}
}
}
I'm really at a loss as to how I might go about cutting back the runtime. Is this just the price I pay for having this many related entities? I know that joins are expensive operations, and that I've essentially got 3 Or is there some inefficiency that I'm overlooking?
Thanks for your help!
EDIT: As requested by Xiaoy312, here's how sectionDetails and attributeDetails are defined:
SurveyTemplate template = db.SurveyTemplates.Find(SurveyTemplateID);
List<SurveySectionDetail> sectionDetails = new List<SurveySectionDetail>();
List<SurveyAttributeDetail> attributeDetails = new List<SurveyAttributeDetail>();
foreach (SurveyTemplateSection section in template.SurveyTemplateSections)
{
SurveySectionDetail SectionDetail = new SurveySectionDetail();
SectionDetail.SectionName = section.SectionName;
SectionDetail.SectionOrder = section.SectionOrder;
SectionDetail.Description = section.Description;
SectionDetail.SurveyGroupID = surveygroup.SurveyGroupID;
db.SurveySectionDetails.Add(SectionDetail);
sectionDetails.Add(SectionDetail);
db.SaveChanges();
foreach (SurveyTemplateAttribute attribute in section.SurveyTemplateAttributes)
{
SurveyAttributeDetail AttributeDetail = new SurveyAttributeDetail();
AttributeDetail.AttributeName = attribute.AttributeName;
AttributeDetail.AttributeScale = attribute.AttributeScale;
AttributeDetail.AttributeType = attribute.AttributeType;
AttributeDetail.AttributeOrder = attribute.AttributeOrder;
AttributeDetail.SectionDetail = SectionDetail;
db.SurveyAttributeDetails.Add(AttributeDetail);
attributeDetails.Add(AttributeDetail);
db.SaveChanges();
}
}

There is several points that you can improve :
Do not SaveChanges() on each Add() :
foreach (TeamMember TeamMember in TeamMembers)
{
...
// db.SaveChanges();
foreach (TeamMember peer in TeamMembers)
{
foreach (SurveySectionDetail SectionDetail in sectionDetails)
{
foreach (SurveyAttributeDetail AttributeDetail in attributeDetails.Where(a => a.SectionDetail.SurveySectionDetailID == SectionDetail.SurveySectionDetailID) )
{
...
// db.SaveChanges();
}
}
}
db.SaveChanges();
}
Consider to reduce the numbers of round trips to the database. This can be done by : they are memory-intensive
using Include() to preload your navigation properties; or
cashing the partial or whole table with ToDictionary() or ToLookup()
Instead of Add(), use AddRange() or even BulkInsert() from EntityFramework.BulkInsert if that fits your setup :
db.SurveyAnswers.AddRange(
TeamMembers.SelectMany(p =>
sectionDetails.SelectMany(s =>
attributeDetails.Where(a => a.SectionDetail.SurveySectionDetailID == s.SurveySectionDetailID)
.Select(a => new SurveyAnswer()
{
Reviewee = p,
SurveyFormID = form.SurveyFormID,
Detail = a,
SectionDetail = s,
}))));

Use Include to avoid SELECT N + 1 issue.
SurveyTemplate template = db.SurveyTemplates.Include("SurveyTemplateSections")
.Include("SurveyTemplateSections.SurveyTemplateAttributes")
.First(x=> x.SurveyTemplateID == SurveyTemplateID);
Generate the whole object graph and then save to DB.
List<SurveySectionDetail> sectionDetails = new List<SurveySectionDetail>();
List<SurveyAttributeDetail> attributeDetails = new List<SurveyAttributeDetail>();
foreach (SurveyTemplateSection section in template.SurveyTemplateSections)
{
SurveySectionDetail SectionDetail = new SurveySectionDetail();
//Some code
sectionDetails.Add(SectionDetail);
foreach (SurveyTemplateAttribute attribute in section.SurveyTemplateAttributes)
{
SurveyAttributeDetail AttributeDetail = new SurveyAttributeDetail();
//some code
attributeDetails.Add(AttributeDetail);
}
}
db.SurveySectionDetails.AddRange(sectionDetails);
db.SurveyAttributeDetails.AddRange(attributeDetails);
db.SaveChanges();
Load all employees you want before the loop, this will avoids database query for every team member.
var teamMemberIds = db.TeamMembers.Where(m => m.TeamID == TeamID && m.OnTeam)
.Select(x=>x.TeamMemberId).ToList();
var employees = db.Employees.Where(x => teamMemberIds.Contains(x.EmployeeId));
create a dictionary for attributeDetails based on their sectionDetailId to avoid query the list on every iteration.
var attributeDetailsGroupBySection = attributeDetails.GroupBy(x => x.SectionDetailId)
.ToDictionary(x => x.Key, x => x);
Move saving of SurveyAnswers and SurveyForms to outside of the loops:
List<SurveyForm> forms = new List<SurveyForm>();
List<SurveyAnswer> answers = new List<SurveyAnswer>();
foreach (int teamMemberId in teamMemberIds)
{
var employee = employees.First(x => x.Id == teamMemberId);
SurveyForm form = new SurveyForm();
//some code
forms.Add(form);
foreach (int peer in teamMemberIds)
{
foreach (SurveySectionDetail SectionDetail in sectionDetails)
{
foreach (SurveyAttributeDetail AttributeDetail in
attributeDetailsGroupBySection[SectionDetail.Id])
{
SurveyAnswer answer = new SurveyAnswer();
//some code
answers.Add(answer);
}
}
}
}
db.SurveyAnswers.AddRange(answers);
db.SurveyForms.AddRange(forms);
db.SaveChanges();
Finally if you want faster insertions you can use EntityFramework.BulkInsert. With this extension, you can save the data like this:
db.BulkInsert(answers);
db.BulkInsert(forms);