Develop Reference

Splitting a large dataset into smaller groups

I'm trying to build a sidebar search navigation filters of check boxes and radio buttons. I'm getting the values from a database. Something like the following, but with 12 filter categories total:
Color
[] red
[] green
[] blue
Size
[] small
[] medium
[] large
Shape
[] square
[] circle
[] triangle
It is working for me using something like the code below. But it seems really inefficient to make a database call for each of the sub-categories:
public ActionResult Index ()
{
SearchBarViewModel model = new SearchBarViewModel();
model.Color = GetValuesFromDb();
model.Size = GetValuesFromDb();
model.Shape = GetValuesFromDb();
return View(model)
}
I'm guessing there is a more efficient way to do this by making a single database query, returning a large dataset that contains all the category values and them split then into groups with linq? I'm just not sure how this would be done?
Database Schema*
SearchKey SearchValue
--------- -----------------
Id Name Id KeyId Value
--------- -----------------
1 Color 1 1 Red
2 Size 2 1 Green
3 Shape 3 1 Blue
4 2 Small
5 2 Medium
6 2 Large
Sql Query
SELECT sv.Id, sv.Value
FROM SearchKey sk
JOIN SearchValue sv ON sv.KeyId = sk.Id
WHERE sk.Name = #ValuePassedToSP

It might or might not be a little early in your development to be concerned about performance of db calls. If the menu values are not changing often or in different contexts, it can make more sense to have the menu structure stored in the database like you do. If the menu values are not changing often, it might be better to store them in a program code or settings file that is only loaded when your app is first loaded, or maybe at demand after that.
I think the linq in context you are looking for might go something like this, where the "GetALLSearchValuesFromDb()" method returns an IEnumerable generated by a SQL statement like you have already, only without the WHERE clause:
public ActionResult Index ()
{
SearchBarViewModel model = new SearchBarViewModel();
var searchvalues = GetALLSearchValuesFromDb();
model.Color = searchvalues.Where(sv => sv.Name == "Color");
model.Size = searchvalues.Where(sv => sv.Name == "Size");
model.Shape = searchvalues.Where(sv => sv.Name == "Shape");
return View(model)
}

LINQ To SQL Include existing object in insert

In this method, I am inserting a new item (Room) into the database. That process functions as expected.
But, in addition to that, each time I add a room, I want to add a piece of furniture as the initial piece. Each item of type Furniture has a "RoomID" to designate its location. Thus, Room contains a collection of Furniture. Below, I am the piece of "primary" furniture from the database, adding it to the room's furniture collection, and submitting changes. The room gets added to the database, but the Furniture.RoomID column remains as null.
public void AddResidentToUniverse(int residentID, int universeID)
{
Universe uni = _context.Universes.FirstOrDefault(u => u.UniverseID == universeID);
Resident res = _context.Residents.FirstOrDefault(r=>r.ResidentID == residentID);
if (uni != null && res!=null)
{
Room e = new Room();
Furniture primary = _context.Furnitures.FirstOrDefault(p => p.FurnitureID == new FurnitureController().GetPrimary(universeID).FurnitureID);
e.UniverseID = uni.UniverseID;
e.RoomName = res.RootName;
e.ResidentID = residentID;
e.Expired = null;
e.Furniture.Add(primary);
uni.Rooms.Add(e);
_context.SubmitChanges();
}
}

You need to add a line that tells your database what you want to insert. For example,
uni.Rooms.InsertOnSubmit(Room object);
uni.Furniture.InsertOnSubmit(furniture piece);
after this, you can write your
uni.SubmitChanges();
line.

I finally bit the bullet and erase my dbml, dropped and recreated the tables, and recreated my dbml. The Furniture.RoomID column updates correctly now. A totally unsatisfying, ham-handed and brute-force approach, I know.

C# Reading and Summarizing Text File with LINQ

I've read MANY different solutions for the separate functions of LINQ that, when put together would solve my issue. My problem is that I'm still trying to wrap my head about how to put LINQ statements together correctly. I can't seem to get the syntax right, or it comes up mish-mash of info and not quite what I want.
I apologize ahead of time if half of this seems like a duplicate. My question is more specific than just reading the file. I'd like it all to be in the same query.
To the point though..
I am reading in a text file with semi-colon separated columns of data.
An example would be:
US;Fort Worth;TX;Tarrant;76101
US;Fort Worth;TX;Tarrant;76103
US;Fort Worth;TX;Tarrant;76105
US;Burleson;TX;Tarrant;76097
US;Newark;TX;Tarrant;76071
US;Fort Worth;TX;Tarrant;76103
US;Fort Worth;TX;Tarrant;76105
Here is what I have so far:
var items = (from c in (from line in File.ReadAllLines(myFile)
let columns = line.Split(';')
where columns[0] == "US"
select new
{
City = columns[1].Trim(),
State = columns[2].Trim(),
County = columns[3].Trim(),
ZipCode = columns[4].Trim()
})
select c);
That works fine for reading the file. But my issue after that is I don't want the raw data. I want a summary.
Specifically I need the count of the number of occurrences of the City,State combination, and the count of how many times the ZIP code appears.
I'm eventually going to make a tree view out of it.
My goal is to have it laid out somewhat like this:
- Fort Worth,TX (5)
- 76101 (1)
- 76103 (2)
- 76105 (2)
- Burleson,TX (1)
- 76097 (1)
- Newark,TX (1)
- 76071 (1)
I can do the tree thing late because there is other processing to do.
So my question is: How do I combine the counting of the specific values in the query itself? I know of the GroupBy functions and I've seen Aggregates, but I can't get them to work correctly. How do I go about wrapping all of these functions into one query?
EDIT: I think I asked my question the wrong way. I don't mean that I HAVE to do it all in one query... I'm asking IS THERE a clear, concise, and efficient way to do this with LINQ in one query? If not I'll just go back to looping through.
If I can be pointed in the right direction it would be a huge help.
If someone has an easier idea in mind to do all this, please let me know.
I just wanted to avoid iterating through a huge array of values and using Regex.Split on every line.
Let me know if I need to clarify.
Thanks!
*EDIT 6/15***
I figured it out. Thanks to those who answered it helped out, but was not quite what I needed. As a side note I ended up changing it all up anyways. LINQ was actually slower than doing it other ways that I won't go into as it's not relevent. As to those who made multiple comments on "It's silly to have it in one query", that's the decision of the designer. All "Best Practices" don't work in all places. They are guidelines. Believe me, I do want to keep my code clear and understandable but I also had a very specific reasoning for doing it the way I did.
I do appreciate the help and direction.
Below is the prototype that I used but later abandoned.
/* Inner LINQ query Reads the Text File and gets all the Locations.
* The outer query summarizes this by getting the sum of the Zips
* and orders by City/State then ZIP */
var items = from Location in(
//Inner Query Start
(from line in File.ReadAllLines(FilePath)
let columns = line.Split(';')
where columns[0] == "US" & !string.IsNullOrEmpty(columns[4])
select new
{
City = (FM.DecodeSLIC(columns[1].Trim()) + " " + columns[2].Trim()),
County = columns[3].Trim(),
ZipCode = columns[4].Trim()
}
))
//Inner Query End
orderby Location.City, Location.ZipCode
group Location by new { Location.City, Location.ZipCode , Location.County} into grp
select new
{
City = grp.Key.City,
County = grp.Key.County,
ZipCode = grp.Key.ZipCode,
ZipCount = grp.Count()
};

The downside of using File.ReadAllLines is that you have to pull the entire file into memory before operating over it. Also, using Columns[] is a bit clunky. You might want to consider my article describing using DynamicObject and streaming the file as an alternative implemetnation. The grouping/counting operation is secondary to that discussion.

var items = (from c in
(from line in File.ReadAllLines(myFile)
let columns = line.Split(';')
where columns[0] == "US"
select new
{
City = columns[1].Trim(),
State = columns[2].Trim(),
County = columns[3].Trim(),
ZipCode = columns[4].Trim()
})
select c);
foreach (var i in items.GroupBy(an => an.City + "," + an.State))
{
Console.WriteLine("{0} ({1})",i.Key, i.Count());
foreach (var j in i.GroupBy(an => an.ZipCode))
{
Console.WriteLine(" - {0} ({1})", j.Key, j.Count());
}
}

There is no point getting everything into one query. It's better to split the queries so that it would be meaningful. Try this to your results
var grouped = items.GroupBy(a => new { a.City, a.State, a.ZipCode }).Select(a => new { City = a.Key.City, State = a.Key.State, ZipCode = a.Key.ZipCode, ZipCount = a.Count()}).ToList();
Result screen shot
EDIT
Here is the one big long query which gives the same output
var itemsGrouped = File.ReadAllLines(myFile).Select(a => a.Split(';')).Where(a => a[0] == "US").Select(a => new { City = a[1].Trim(), State = a[2].Trim(), County = a[3].Trim(), ZipCode = a[4].Trim() }).GroupBy(a => new { a.City, a.State, a.ZipCode }).Select(a => new { City = a.Key.City, State = a.Key.State, ZipCode = a.Key.ZipCode, ZipCount = a.Count() }).ToList();

Dynamic List Values

Dictionary<string, EmployeeInfo> employeeInfoList = new Dictionary<string, EmployeeInfo>();
employeeInfoList = EmployeeInfoProxy.GetAllEmployeeInfo(TenantId);
if (employeeInfoList != null)
{
List<EmployeeInfo> employee = new List<EmployeeInfo>(employeeInfoList.Values);
ViewData["Name"] = employee[0].Name;
ViewData["Salary"] = employee[0].Salary;
ViewData["Department"] = employee[0].Department;
ViewData["Designation"] = employee[0].Designation;
ViewData["Address"] = employee[0].Address;
ViewData["Address1"] = employee[0].Address1;
}
the above code is working fine.if the employee has only one record,so i had hardcoded employee[0].
if there are more rows,how to pass those index to employee(for eg employee[\i should get the value dynamically here].

If I understand your question correctly, given a List you could use linq to grab a single employee.
EmployeeInfo singleEmployee = employee.Where( x => x.Id).SingleOrDefault();
if (singleEmployee != null)
{
do something here
}
that said, I don't know if you have a 'key' against the employee like Id or similar.
To get by index, you have a bigger round trip (outputting the emploees to a grid or something similar, allowing a click that passes in the index of the employee you want, and access the single eployee that way).
Unless you're heavily employing caching in the 'EmployeeInfoProxy.GetAllEmployeeInfo()' method though, I'd say you want a 'GetEmployeeInfoById()' and again, I'll harp back to the need for your EmployeeInfo to have a unique Id of some description - you will make your life so much easier.
Hope this helps.
Cheers,
Terry
[in addition]
if employeeListInfo has 3 entries, then to access each of them I'd say the best way would be to create a viewmodel (it looks like this is MVC), but worst case scenario you could do:
List<EmployeeInfo> employee = new List<EmployeeInfo>(employeeInfoList.Values);
ViewData["Employees"] = employee;
then in your view, just do:
<% foreach(EmployeeInfo employeeDetails in ViewData["Employees"] as List<EmployeeInfo>) { %>
<!-- you will be looping through each employee here, so you can do as you wish
<% } %>
I wouldn't say this is best practice - using a ViewModel would likely represent a better way to do this, though the above will certainly give you a solution that will work in your case.

How can I shorten this linq to sql query?

I am am making a calendar and to make it easier on myself I break up appointments that span over multiple weeks.
For instance Jan 1st to Jan 31st spans like 6 weeks(my calendar is always 42 cells - 6 by 7). So I would basically have 6 rows stored in my database.
However somethings I do require to me to put all these rows back together into one row. For instance if I want to export my calendar in Ical format.
I have a field in my database called bindingClassName all these rows get the same unquie id to that group of tasks so I am able to get all the weeks easily.
// get all of the task rows by binding class name.
var found = plannerDb.Calendars.Where(u => u.UserId == userId && u.BindingClassName == bindingClassName)
.GroupBy(u => u.BindingClassName);
List<Calendar> allAppoingments = new List<Calendar>();
// go through each of the results and add it to a list of calendars
foreach (var group in found)
{
foreach (var row in group)
{
Calendar appointment = new Calendar();
appointment.AppointmentId = row.AppointmentId;
appointment.AllDay = row.AllDay;
appointment.BindingClassName = row.BindingClassName;
appointment.Description = row.Description;
appointment.EndDate = row.EndDate;
appointment.StartDate = row.StartDate;
appointment.Title = row.Title;
appointment.Where = row.Where;
appointment.UserId = row.UserId;
allAppoingments.Add(appointment);
}
}
// order
var test = allAppoingments.OrderBy(u => u.StartDate);
var firstAppointment = test.First();
var LastAppointment = test.Last();
Calendar newAppointment = new Calendar();
newAppointment.UserId = firstAppointment.UserId;
newAppointment.Description = firstAppointment.Description;
newAppointment.AllDay = firstAppointment.AllDay;
newAppointment.StartDate = firstAppointment.StartDate;
newAppointment.Title = firstAppointment.Title;
newAppointment.Where = firstAppointment.Where;
newAppointment.BindingClassName = firstAppointment.BindingClassName;
newAppointment.EndDate = LastAppointment.EndDate;
return newAppointment;
So basically that big blob finds all the appointments with the same binding name. Then I go through each one and make it into a Calendar object then finally once it is all made I get the first and last record to get the startDate and endDate.
So I am not good with linq but I am not sure if I can just add something after the groupBy to do what I want.
Edit.
I am trying group all my appointments together once I get all of them from the user.
So I have this so far
I tried something like this.
var allApointments = calendar.GetAllAppointments(userId);
var group = allApointments.GroupBy(u => u.BindingClassName).Select(u => new Calendar()).ToList
I was hoping that it would fill each group automatically but it does not. So I am not sure if don't need groupby again.
Edit # admin
Hi thanks for explaining sorting and grouping. How you explained it though it seems either one would work.
Like the code you have for getting the first and last date works great and does what I wanted it to.
I think grouping might have worked because in the end though I am looking just to have one row that has the startdate of the first record and the end date of the last record all the other information would be the same.
So I don't know if it would harder to write that instead or what but like I said your query does what I want.
However that query is used on a single basis. Like I use that query only when a user clicks to view that appointment on my calendar. By clicking on the appointment I get all the information about that appointment and thats where I need to look at if that task spans over multiple days and figure out when the appointment started and when it is going to end.
Now I need another query and I think it would be better if I could actually group them as how I understand it from your explanation it will make one row. the reason I think this is because I want to export all the records in the table from that user.
So if I order them into one continues block by binding name I still going to need some loops that goes through all the records and gets the first and start date. So if I could just group it in one go and the final result would be just one record for each group of binding names and it would have the first start date and the last end date from the first and last record would be better.

Why are you grouping the appointments if you aren't actually using the group? It looks like you're just using them individually. In any case, you're already filtering the rows on a single value for BindingClassName in the Where clause, so you would only end up with 1 (or 0) group(s) anyway.
You can rewrite that series of foreach loops into a Select and ToList() like this:
var allAppointments =
plannerDb.Calendars.Where(
row => row.UserId == userId &&
row.BindingClassName == bindingClassName).OrderBy(
row => row.StartDate).Select(
row => new Calendar()
{
AppointmentId = row.AppointmentId,
AllDay = row.AllDay,
BindingClassName = row.BindingClassName,
Description = row.Description,
EndDate = row.EndDate,
StartDate = row.StartDate,
Title = row.Title,
Where = row.Where,
UserId = row.UserId
}).ToList();
This will give you back the full list in the order you wanted. However, I'm curious why you're retrieving the whole list when it looks like you're only interested in the first and last appointment. You could instead do this:
var baseQuery =
plannerDb.Calendars.Where(
row => row.UserId == userId &&
row.BindingClassName == bindingClassName);
var first = baseQuery.OrderBy(row => row.StartDate).First();
var last = baseQuery.OrderByDescending(row => row.StartDate).Select(
row => row.EndDate).First();
return new Calendar()
{
AppointmentId = first.AppointmentId,
AllDay = first.AllDay,
BindingClassName = first.BindingClassName,
Description = first.Description,
EndDate = last,
StartDate = first.StartDate,
Title = first.Title,
Where = first.Where,
UserId = first.UserId
});
This should produce outputs that are the same as what you have now. I would question, however, if this is exactly what you want. Say you have two appointments:
Appointment 1 starts January 5 and ends on January 10
Appointment 2 starts January 6 and ends on January 7
Using this (and your) logic, you would get the end date as January 7, since Appointment 2 has the larger start date, but Appointment 1 actually ends later. I would recommend changing the second query to this:
var last = baseQuery.OrderByDescending(row => row.EndDate).Select(
row => row.EndDate).First();
This will give you the largest end date, which I think is what you're actually after.
EDIT
I think you're making the (very common) mistake of confusing grouping with sorting. When you say you want to "group the appointments by the binding name", it sounds like you want a full, complete list of appointments, and you want those appointments arranged in such a way as all appointments with a particular binding name form a contiguous block. If that's the case, you want to order the list by the binding name, not group them. Grouping takes the whole list and produces one row per grouping clause and allows you to perform aggregation functions on the remaining columns. For example, let's say I group the appointments on the binding name. This means that my result set will contain one row per binding name, and I can then do things like find the maximum start or end date or something like that; more formally, you can specify aggregation operations, which are operations that take a set of data (i.e. a list of start dates) and return a single piece of data (i.e. the maximum start date).
Unless I'm misunderstanding, it sounds like you still want to retrieve all of the individual assignments, you just want them arranged by binding name. If this is the case, just OrderBy(row => row.BindingName) and it will do the trick. In addition, you may want to avoid using the word "group", as people will think you mean the sort of grouping that I described above.

Just as a side point not concerning the linq, have you looked at AutoMapper? I am currently using this for populating data objects from linq and I've found it really useful for getting rid of the large sections of code where you just map to dtos. It wouldn't make the query parts of your code any shorter but would reduce:
return new Calendar()
{
AppointmentId = first.AppointmentId,
AllDay = first.AllDay,
BindingClassName = first.BindingClassName,
Description = first.Description,
EndDate = last,
StartDate = first.StartDate,
Title = first.Title,
Where = first.Where,
UserId = first.UserId
});
to:
return Mapper.Map(first,new Calendar{EndDate = last});