We are using an extractor application that will export data from the database to csv files. Based on some condition variable it extracts data from different tables, and for some conditions we have to use UNION ALL as the data has to be extracted from more than one table. So to satisfy the UNION ALL condition we are using nulls to match the number of columns.
Right now all the queries in the system are pre-built based on the condition variable. The problem is whenever there is change in the table projection (i.e new column added, existing column modified, column dropped) we have to manually change the code in the application.
Can you please give some suggestions how to extract the column names dynamically so that any changes in the table structure do not require change in the code?
My concern is the condition that decides which table to query. The variable condition is
like
if the condition is A, then load from TableX
if the condition is B then load from TableA and TableY.
We must know from which table we need to get data. Once we know the table it is straightforward to query the column names from the data dictionary. But there is one more condition, which is that some columns need to be excluded, and these columns are different for each table.
I am trying to solve the problem only for dynamically generating the list columns. But my manager told me to make solution on the conceptual level rather than just fixing. This is a very big system with providers and consumers constantly loading and consuming data. So he wanted solution that can be general.
So what is the best way for storing condition, tablename, excluded columns? One way is storing in database. Are there any other ways? If yes what is the best? As I have to give at least a couple of ideas before finalizing.
Thanks,
A simple query like this helps you to know each column name of a table in Oracle.
Select COLUMN_NAME from user_tab_columns where table_name='EMP'
Use it in your code :)
Ok, MNC, try this for size (paste it into a new console app):
using System;
using System.Collections.Generic;
using System.Linq;
using Test.Api;
using Test.Api.Classes;
using Test.Api.Interfaces;
using Test.Api.Models;
namespace Test.Api.Interfaces
{
public interface ITable
{
int Id { get; set; }
string Name { get; set; }
}
}
namespace Test.Api.Models
{
public class MemberTable : ITable
{
public int Id { get; set; }
public string Name { get; set; }
}
public class TableWithRelations
{
public MemberTable Member { get; set; }
// list to contain partnered tables
public IList<ITable> Partner { get; set; }
public TableWithRelations()
{
Member = new MemberTable();
Partner = new List<ITable>();
}
}
}
namespace Test.Api.Classes
{
public class MyClass
{
private readonly IList<TableWithRelations> _tables;
public MyClass()
{
// tableA stuff
var tableA = new TableWithRelations { Member = { Id = 1, Name = "A" } };
var relatedclasses = new List<ITable>
{
new MemberTable
{
Id = 2,
Name = "B"
}
};
tableA.Partner = relatedclasses;
// tableB stuff
var tableB = new TableWithRelations { Member = { Id = 2, Name = "B" } };
relatedclasses = new List<ITable>
{
new MemberTable
{
Id = 3,
Name = "C"
}
};
tableB.Partner = relatedclasses;
// tableC stuff
var tableC = new TableWithRelations { Member = { Id = 3, Name = "C" } };
relatedclasses = new List<ITable>
{
new MemberTable
{
Id = 2,
Name = "D"
}
};
tableC.Partner = relatedclasses;
// tableD stuff
var tableD = new TableWithRelations { Member = { Id = 3, Name = "D" } };
relatedclasses = new List<ITable>
{
new MemberTable
{
Id = 1,
Name = "A"
},
new MemberTable
{
Id = 2,
Name = "B"
},
};
tableD.Partner = relatedclasses;
// add tables to the base tables collection
_tables = new List<TableWithRelations> { tableA, tableB, tableC, tableD };
}
public IList<ITable> Compare(int tableId, string tableName)
{
return _tables.Where(table => table.Member.Id == tableId
&& table.Member.Name == tableName)
.SelectMany(table => table.Partner).ToList();
}
}
}
namespace Test.Api
{
public class TestClass
{
private readonly MyClass _myclass;
private readonly IList<ITable> _relatedMembers;
public IList<ITable> RelatedMembers
{
get { return _relatedMembers; }
}
public TestClass(int id, string name)
{
this._myclass = new MyClass();
// the Compare method would take your two paramters and return
// a mathcing set of related tables that formed the related tables
_relatedMembers = _myclass.Compare(id, name);
// now do something wityh the resulting list
}
}
}
class Program
{
static void Main(string[] args)
{
// change these values to suit, along with rules in MyClass
var id = 3;
var name = "D";
var testClass = new TestClass(id, name);
Console.Write(string.Format("For Table{0} on Id{1}\r\n", name, id));
Console.Write("----------------------\r\n");
foreach (var relatedTable in testClass.RelatedMembers)
{
Console.Write(string.Format("Related Table{0} on Id{1}\r\n",
relatedTable.Name, relatedTable.Id));
}
Console.Read();
}
}
I'll get back in a bit to see if it fits or not.
So what you are really after is designing a rule engine for building dynamic queries. This is no small undertaking. The requirements you have provided are:
Store rules (what you call a "condition variable")
Each rule selects from one or more tables
Additionally some rules specify columns to be excluded from a table
Rules which select from multiple tables are satisfied with the UNION ALL operator; tables whose projections do not match must be brought into alignment with null columns.
Some possible requirements you don't mention:
Format masking e.g. including or excluding the time element of DATE columns
Changing the order of columns in the query's projection
The previous requirement is particularly significant when it comes to the multi-table rules, because the projections of the tables need to match by datatype as well as number of columns.
Following on from that, the padding NULL columns may not necessarily be tacked on to the end of the projection e.g. a three column table may be mapped to a four column table as col1, col2, null, col3.
Some multi-table queries may need to be satisfied by joins rather than set operations.
Rules for adding WHERE clauses.
A mechanism for defining default sets of excluded columns (i.e. which are applied every time a table is queried) .
I would store these rules in database tables. Because they are data and storing data is what databases are for. (Unless you already have a rules engine to hand.)
Taking the first set of requirements you need three tables:
RULES
-----
RuleID
Description
primary key (RuleID)
RULE_TABLES
-----------
RuleID
Table_Name
Table_Query_Order
All_Columns_YN
No_of_padding_cols
primary key (RuleID, Table_Name)
RULE_EXCLUDED_COLUMNS
---------------------
RuleID
Table_Name
Column_Name
primary key (RuleID, Table_Name, Column_Name)
I've used compound primary keys just because it's easier to work with them in this context e.g. running impact analyses; I wouldn't recommend it for regular applications.
I think all of these are self-explanatory except the additional columns on RULE_TABLES.
Table_Query_Order specifies the order in which the tables appear in UNION ALL queries; this matters only if you want to use the column_names of the leading table as headings in the CSV file.
All_Columns_YN indicates whether the query can be written as SELECT * or whether you need to query the column names from the data dictionary and the RULE_EXCLUDED_COLUMNS table.
No_of_padding_cols is a simplistic implementation for matching projections in those UNION ALL columns, by specifying how many NULLs to add to the end of the column list.
I'm not going to tackle those requirements you didn't specify because I don't know whether you care about them. The basic thing is, what your boss is asking for is an application in its own right. Remember that as well as an application for generating queries you're going to need an interface for maintaining the rules.
MNC,
How about creating a dictionary of all the known tables involved in the application process up front (irrespective of the combinations - just a dictionary of the tables) which is keyed on tablename. the members of this dictionary would be a IList<string> of the column names. This would allow you to compare two tables on both the number of columns present dicTable[myVarTableName].Count as well as iterating round the dicTable[myVarTableName].value to pull out the column names.
At the end of the piece, you could do a little linq function to determine the table with the greatest number of columns and create the structure with nulls accordingly.
Hope this gives food for thought..
Related
I have a query regarding generating SQL insert statement using c# classes.
So I have a class called students.
There is a function which gets list of students and dump that in database.
Student Model
public class Student
{
public string ID { get; set; } = ""; // DB column name is studentID
public string Name { get; set; } = ""; // DB column name is studentName
public string address { get; set; } // DB column name is studentAddress
}
Function to dump Data
public async Task<Error> DumpStudentAsync()
{
List<Student> students = new List<Student>();
StringBuilder query = new StringBuilder();
query = query.Append("INSERT INTO Student(studentID,studentName,studentAddress) VALUES");
string columnList = "(#studentID{0},#studentName{0},#studentAddress{0})";
for (int i = 0; i < students.Count; i++)
{
query.AppendFormat($"{columnList},", i);
}
query = query.Replace(',', ';', query.Length - 1, 1);
SqlCommand cmd = new SqlCommand
{
CommandText = query.ToString(),
};
for (int i = 0; i < students.Count; i++)
{
cmd.Parameters.AddWithValue($"#studentID{i}", students[i].ID);
cmd.Parameters.AddWithValue($"#studentName{i}", students[i].Name);
cmd.Parameters.AddWithValue($"#studentAddress{i}", students[i].address);
}
SQLWrapper db = new SQLWrapper(_ctx, "", DSConfig.SQLConnectionKey);
return await db.ExecuteStatementAsync(cmd, "");
}
So here I want to make this function generic in such a way that if I add a new field in my student object there should be no code change done in my function.
I tried searching the answers but I didn't get anything.
Here firstly I'm appending the insert format in query where I have hard-coded.
Secondly I'm appending variables for command parameters on the basis of students count which is also hard-coded.
Also the class properties and database columns names are different how can I make use of class properties as DB column names?
Can I do something like this ?
StringBuilder studentQuery = new StringBuilder();
string columns = "";
// Add column list on the basis of students properties
foreach (var property in Student.Properties)
{
columns += "property.ID,"; // It should be studentID
}
// Add variables on the basis of students count
// Add command parameters value on the basis of students count
FYI: I'm using ADO.NET code to perform DB activities not Entity framework.
This kind of automatic is not easily done. .NET is strongly typed. Unless you go into stuff like reflection or dynamic code to do it. And I would not advise it. Strong Typisation is your friend. Without it you end up in the JavaScript and PHP examples for this comic. Hint: JS does the wrong thing in both cases.
For me at least, having to do some minor changes on the frontend for changes on the Database is acceptable work. If anything that is the smalest, least dangerous part of the whole process. So I can only advise against trying this.
However for databases and only databases, stuff like Entity Framework might be a good idea. It can generate your classes from the Database.
i know it is not complicated but i struggle with it.
I have IList<Material> collection
public class Material
{
public string Number { get; set; }
public decimal? Value { get; set; }
}
materials = new List<Material>();
materials.Add(new Material { Number = 111 });
materials.Add(new Material { Number = 222 });
And i have DbSet<Material> collection
with columns Number and ValueColumn
I need to update IList<Material> Value property based on DbSet<Material> collection but with following conditions
Only one query request into database
The returned data from database has to be limited by Number identifier (do not load whole database table into memory)
I tried following (based on my previous question)
Working solution 1, but download whole table into memory (monitored in sql server profiler).
var result = (
from db_m in db.Material
join m in model.Materials
on db_m.Number.ToString() equals m.Number
select new
{
db_m.Number,
db_m.Value
}
).ToList();
model.Materials.ToList().ForEach(m => m.Value= result.SingleOrDefault(db_m => db_m.Number.ToString() == m.Number).Value);
Working solution 2, but it execute query for each item in the collection.
model.Materials.ToList().ForEach(m => m.Value= db.Material.FirstOrDefault(db_m => db_m.Number.ToString() == m.Number).Value);
Incompletely solution, where i tried to use contains method
// I am trying to get new filtered collection from database, which i will iterate after.
var result = db.Material
.Where(x=>
// here is the reasonable error: cannot convert int into Material class, but i do not know how to solve this.
model.Materials.Contains(x.Number)
)
.Select(material => new Material { Number = material.Number.ToString(), Value = material.Value});
Any idea ? For me it is much easier to execute stored procedure with comma separated id values as a parameter and get the data directly, but i want to master linq too.
I'd do something like this without trying to get too cute :
var numbersToFilterby = model.Materials.Select(m => m.Number).ToArray();
...
var result = from db_m in db.Material where numbersToFilterBy.Contains(db_m.Number) select new { ... }
I'm struggling to find the best way to store and represent the data I have in SQL (MySQL DB) and C# windows form.
My data when mapped to classes which looks like this;
public class Parent
{
public string UniqueID { get; set; } //Key
public DateTime LoadTime { get; set; }
public string Reference { get; set; }
private List<Child> Elements { get; set; }
}
public class Child
{
public int MemberCode { get; set; } //Composite key
public int ElementCode { get; set; } //Composite key
public Object Data { get; set; }
}
My data is very dynamic. So a parent record can have any number of child records.
In the child record then the MemberCode and ElementCode are actually foreign keys to other tables/classes, which when a look-up is performed gives me details of what the data actually is. For example
MemberCode = 1 & ElementCode = 1 means data is a Date
MemberCode = 1 & ElementCode = 3 means data is a telephone number
MemberCode = 2 & ElementCode = 11 means data is a Product Code
MemberCode = 2 & ElementCode = 12 means data is a Service Code
etc
These effectively combine to indicate what the column name is, and these are always the same (so MemberCode = 1 & ElementCode = 1 will always be a Date no matter which child object it is associated with).
At the moment these are references/lookups but I could also put the data in a variable in the class as that might make it easier. Then it would be more like a Key Value Pair.
At the moment in my DB I have these stored as two tables, with the child record also containing the UniqueID from the parent. But I'm, not sure that this is the best way as I will explain.
My tables are created as such
CREATE TABLE `PARENT` (
`ID` INT(11) NOT NULL AUTO_INCREMENT,
`LOADTIME` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
`REFERENCE` VARCHAR(100) NOT NULL,
PRIMARY KEY (`ID`)
)
CREATE TABLE `CHILD` (
`ID` INT(11) NOT NULL,
`MEMBER_CODE` INT(11) NOT NULL,
`ELEMENT_CODE` INT(11) NOT NULL,
`DATA` VARCHAR(4000) NULL DEFAULT NULL,
PRIMARY KEY (`ID`, `MEMBER_CODE`, `ELEMENT_CODE`),
CONSTRAINT `fk_ID` FOREIGN KEY (`ID`) REFERENCES `Parent` (`ID`)
)
Now what I want to do is to flatten out this data so that I can display a single parent record with all child records as a single row. I ideally want to display it in an ObjectListView (http://objectlistview.sourceforge.net/cs/index.html) but can consider datagrid if it makes life easier.
Because my data is dynamic, then I'm struggling to flatten this out and if I select 10 parent records then each can have different number of child elements, and each can have different MemberCodes and ElementCode, which means that they are effectively different columns.
So my data could look like the following (but on a larger scale);
But because of the dynamic nature of the data, then I struggling to do this. Either in SQL or in Objects in my code. Maybe there is even another way to store my data which would suit it better.
After many many days working on this then I have managed to resolve this issue myself. What I done was the following;
In my original child class then the MemberCode and ElementCode make a unique key that basically stated what the column name was. So I took this a step further and added a "Column_Name" so that I had
public class Child
{
public int MemberCode { get; set; } //Composite key
public int ElementCode { get; set; } //Composite key
public string Column_Name { get; set; } //Unique. Alternative Key
public Object Data { get; set; }
}
This was obviously reflected in my database table as well.
My SQL to extract the data then looked like this;
select p.UniqueID, p.LoadTime, p.reference, c.MemberCode, c.ElementCode , c.column_name, c.Data
from parent as p, child as c
where p.UniqueID = c.UniqueID
//aditional filter criteria
ORDER BY p.UniqueID, MemberCode, ElementCode
ordering by the UniqueID first is critical to ensure the records are in the right order for later processing.
The I would use a dynamic and a ExpandoObject() to store the data.
So I iterate over the result to the convert the sql result into this structure as follows;
List<dynamic> allRecords = new List<dynamic>(); //A list of all my records
List<dynamic> singleRecord = null; //A list representing just a single record
bool first = true; //Needed for execution of the first iteration only
int lastId = 0; //id of the last unique record
foreach (DataRow row in args.GetDataSet.Tables[0].Rows)
{
int newID = Convert.ToInt32(row["UniqueID"]); //get the current record unique id
if (newID != lastId) //If new record then get header/parent information
{
if (!first)
allRecords.Add(singleRecord); //store the last record
else
first = false;
//new object
singleRecord = new List<dynamic>();
//get parent information and store it
dynamic head = new ExpandoObject();
head.Column_name = "UniqueID";
head.UDS_Data = row["UniqueID"].ToString();
singleRecord.Add(head);
head = new ExpandoObject();
head.Column_name = "LoadTime";
head.UDS_Data = row["LoadTime"].ToString();
singleRecord.Add(head);
head = new ExpandoObject();
head.Column_name = "reference";
head.UDS_Data = row["reference"].ToString();
singleRecord.Add(head);
}
//get child information and store it. One row at a time
dynamic record = new ExpandoObject();
record.Column_name = row["column_name"].ToString();
record.UDS_Data = row["data"].ToString();
singleRecord.Add(record);
lastId = newID; //store the last id
}
allRecords.Add(singleRecord); //stores the last complete record
Then I have my information stored dynamically in the flat manner that I required.
Now the next problem was the ObjectListView I wanted to use. This could not accept such dynamic types.
So I had the information stored within my code as I wanted, but I could still not display it as was required.
The solution was that was to use a variant of the ObjectListView known as the DataListView. This is effectively the same control but can be data bound.
Another alternative would also be to use a DatagridView, but I wanted to stick to the ObjectListView for other reasons.
So now I had to convert my dynamic data into a Datasource. This I done as follows;
DataTable dt = new DataTable();
foreach (dynamic record in allRecords)
{
DataRow dr = dt.NewRow();
foreach (dynamic item in record)
{
var prop = (IDictionary<String, Object>)item;
if (!dt.Columns.Contains(prop["Column_name"].ToString()))
{
dt.Columns.Add(new DataColumn(prop["Column_name"].ToString()));
}
dr[prop["Column_name"].ToString()] = prop["UDS_Data"];
}
dt.Rows.Add(dr);
}
Then I simply assign my datasource to the DataListView, generate the columns, and hey presto I now have my dynamic data extracted, flattened and displayed how I require.
Is it possible to dynamically limit the number of columns returned from a LINQ to SQL query?
I have a database SQL View with over 50 columns. My app has a domain object with over 50 properties, one for each column. In my winforms project I bind a list of domain objects to a grid. By default only a few of the columns are visible however the user can turn on/off any of the columns.
Users are complaining the grid takes too long to load. I captured the LINQ generated SQL query then executed it within SQL Server Management Studio and verified its slow. If I alter the SQL statement, removing all the invisible columns, it runs almost instantly. There is a direct correlation between performance and the number of columns in the query.
I'm wondering if its possible to dynamically alter the number of columns returned from the LINQ generated SQL query? For example, here is what my code currently looks like:
public List<Entity> GetEntities()
{
using (var context = new CensusEntities())
{
return (from e in context.Entities
select e).ToList();
}
}
The context.Entities object was generated from a SQL View that contains over 50 columns so when the above executes it generates SQL like "SELECT Col1, Col2, Col3, ... Col50 FROM Entity INNER JOIN...". I would like to change the method signature to look like this:
public List<Entity> GetEntities(string[] visibleColumns)
{
using (var context = new CensusEntities())
{
return (from e in context.Entities
select e).ToList();
}
}
I'm not sure how to alter the body of this method to change the generated SQL statement to only return the column values I care about, all others can be NULL.
Something like this should work:
List<string> columns = new List<string>();
columns.Add("EmployeeID");
columns.Add("HireDate");
columns.Add("City");
Add columns to your list ^.
var result = Class.ReturnList(columns);
Pass the List to a method ^.
public static List<Entity> ReturnList(List<string> VisibleColumns)
{
StringBuilder SqlStatement = new StringBuilder();
SqlStatement.Append("Select ");
for (int i = 0; i < VisibleColumns.Count; i++)
{
if (i == VisibleColumns.Count - 1)
{
SqlStatement.Append(VisibleColumns[i]);
}
else
{
SqlStatement.Append(VisibleColumns[i]);
SqlStatement.Append(",");
}
}
SqlStatement.Append(" FROM Entity");
using (var ctx = new DataClasses1DataContext())
{
var result = ctx.ExecuteQuery<Entity>(SqlStatement.ToString());
return result.ToList();
}
}
This basically just makes a SELECT statement with all the fields you passed in with the VisibleColumns list.
In this case, the SQL statement that will be generated by the strings in the VisibleColumns list is:
Select EmployeeID, HireDate, City From Employee
(note: i used the Northwind database to try this out, hence the EmployeeID etc column names. You should replace them with your own, obviously.)
It is not trivial to do this dynamically, but if you have a limited set of combinations of columns you want to retreive you can do an explicit select like this:
public List<Entity> GetEntities()
{
using (var context = new CensusEntities())
{
return (from e in context.Entities
select new
{
col1 = e.col1,
col4 = e.col4,
col5 = e.col5,
}
).ToList()
.Select(x=>new Entity{col1 = x.col1, col4 = x.col4, col5 = x.col5}).ToList();
}
}
The extra select step is necessary because LINQ2SQL won't create partial entities for you.
Create a method for each common combination of columns (especially the initial) the users wants to retrieve.
However to make this dynamic you can build a query with you entity stored as a property in an anonymous class and collect your result properties in another anonymous class in second property in the same anonymous class. Finally you select your entities from the collected objects into objects of the correct type.
public List<Entity> GetEntities()
{
using (var context = new CensusEntities())
{
var combinedResult = (from e in context.Entities
select new {
Entity = e,
CollectedValues = new
{
// Insert default values of the correct type as placeholders
col1 = 0, // or "" for string or false for bool
col2 = 0, // or "" for string or false for bool
// ...
col49 = 0, // or "" for string or false for bool
col50 = 0, // or "" for string or false for bool
}
);
// Then copy each requested property
// col1
if (useCol1)
{
var combinedResult = (from e in combinedResult
select new {
Entity = e,
CollectedValues = new
{
col1 = e.Enitity.col1, // <-- here we update with the real value
col2 = e.CollectedValues.col2, // <-- here we just use any previous value
// ...
col49 = e.CollectedValues.col49, // <-- here we just use any previous value
col50 = e.CollectedValues.col50, // <-- here we just use any previous value }
);
}
// col2
if (useCol2)
{
// same as last time
col1 = e.CollectedValues.col1, // <-- here we just use any previous value
col2 = e.Enitity.col2, // <-- here we update with the real value
// ...
}
// repeat for all columns, update the column you want to fetch
// Just get the collected objects, discard the temporary
// Entity property. When the query is executed here only
// The properties we actually have used from the Entity object
// will be fetched from the database and mapped.
return combinedResult.Select(x => x.CollectedValues).ToList()
.Select(x=>new Entity{col1 = x.col1, col2 = x.col2, ... col50 = x.col50}).ToList();
}
}
There will be lots of code, and a pain to maintain, but it should work.
If you are going this route I suggest that you build a code generator that builds this code with reflection from your LINQ context.
Try something like this
using (var context = new CensusEntities())
{
var q = from e in context.Entities
select e.myfield1,e.myfield2;
return q.Tolist();
}
The resulting query should be lighter and also all the data conversion that goes underneath.
But if you really need to build dynamic input, I think some dynamic sql should be involved. So
build the dynamic SQL and get a data table
use a datatable to a dynamic object conversion as shown here
How can I convert a DataTable into a Dynamic object?
BTW a lot of hard work, I think you should considered using the first block of code.
I have need to select a number of 'master' rows from a table, also returning for each result a number of detail rows from another table. What is a good way of achieving this without multiple queries (one for the master rows and one per result to get the detail rows).
For example, with a database structure like below:
MasterTable:
- MasterId BIGINT
- Name NVARCHAR(100)
DetailTable:
- DetailId BIGINT
- MasterId BIGINT
- Amount MONEY
How would I most efficiently populate the data object below?
IList<MasterDetail> data;
public class Master
{
private readonly List<Detail> _details = new List<Detail>();
public long MasterId
{
get; set;
}
public string Name
{
get; set;
}
public IList<Detail> Details
{
get
{
return _details;
}
}
}
public class Detail
{
public long DetailId
{
get; set;
}
public decimal Amount
{
get; set;
}
}
Normally, I'd go for the two grids approach - however, you might also want to look at FOR XML - it is fairly easy (in SQL Server 2005 and above) to shape the parent/child data as xml, and load it from there.
SELECT parent.*,
(SELECT * FROM child
WHERE child.parentid = parent.id FOR XML PATH('child'), TYPE)
FROM parent
FOR XML PATH('parent')
Also - LINQ-to-SQL supports this type of model, but you need to tell it which data you want ahead of time. Via DataLoadOptions.LoadWith:
// sample from MSDN
Northwnd db = new Northwnd(#"c:\northwnd.mdf");
DataLoadOptions dlo = new DataLoadOptions();
dlo.LoadWith<Customer>(c => c.Orders);
db.LoadOptions = dlo;
var londonCustomers =
from cust in db.Customers
where cust.City == "London"
select cust;
foreach (var custObj in londonCustomers)
{
Console.WriteLine(custObj.CustomerID);
}
If you don't use LoadWith, you will get n+1 queries - one master, and one child list per master row.
It can be done with a single query like this:
select MasterTable.MasterId,
MasterTable.Name,
DetailTable.DetailId,
DetailTable.Amount
from MasterTable
inner join
DetailTable
on MasterTable.MasterId = DetailTable.MasterId
order by MasterTable.MasterId
Then in psuedo code
foreach(row in result)
{
if (row.MasterId != currentMaster.MasterId)
{
list.Add(currentMaster);
currentMaster = new Master { MasterId = row.MasterId, Name = row.Name };
}
currentMaster.Details.Add(new Detail { DetailId = row.DetailId, Amount = row.Amount});
}
list.Add(currentMaster);
There's a few edges to knock off that but it should give you the general idea.
select < columns > from master
select < columns > from master M join Child C on M.Id = C.MasterID
You can do it with two queries and one pass on each result set:
Query for all masters ordered by MasterId then query for all Details also ordered by MasterId. Then, with two nested loops, iterate the master data and create a new Master object foreach row in the main loop, and iterate the details while they have the same MasterId as the current Master object and populate its _details collection in the nested loop.
Depending on the size of your dataset you can pull all of the data into your application in memory with two queries (one for all masters and one for all nested data) and then use that to programatically create your sublists for each of your objects giving something like:
List<Master> allMasters = GetAllMasters();
List<Detail> allDetail = getAllDetail();
foreach (Master m in allMasters)
m.Details.Add(allDetail.FindAll(delegate (Detail d) { return d.MasterId==m.MasterId });
You're essentially trading memory footprint for speed with this approach. You can easily adapt this so that GetAllMasters and GetAllDetail only return the master and detail items you're interested in. Also note for this to be effective you need to add the MasterId to the detail class
This is an alternative you might consider. It does cost $150 per developer, but time is money too...
We use an object persistence layer called Entity Spaces that generates the code for you to do exactly what you want, and you can regenerate whenever your schema changes. Populating the objects with data is transparent. Using the objects you described above would look like this (excuse my VB, but it works in C# too):
Dim master as New BusinessObjects.Master
master.LoadByPrimaryKey(43)
Console.PrintLine(master.Name)
For Each detail as BusinessObjects.Detail in master.DetailCollectionByMasterId
Console.PrintLine(detail.Amount)
detail.Amount *= 1.15
End For
With master.DetailCollectionByMasterId.AddNew
.Amount = 13
End With
master.Save()