I have written a generic database helper method, which returns the records of a particular entity.
here is how I do it:
I have a class called Customer having 10 properties also having a property called TableName.
There is a Method which just take Type parameter, and return an array of passed type.
How the method work is, by using reflection it got a table name, and fire a select statement, and on the basis of DataReader it loops through each colum and Properties of passed Type.
So, the problem is suppose there are 1 million records and 10 properties. It loops for 10 (Properties) * (1,000,000 records) = 10,000,000 times
is there any optimized way to do this, something like using LINQ against a Datareader?
Here is a code
object[] LoadAll(Type type)
{
try
{
object obj = Activator.CreateInstance(type);
SqlConnection conn = new SqlConnection("connection string");
string tableName = type.GetField("TableName").GetValue(obj) as string;
SqlCommand cmd = conn.CreateCommand();
cmd.CommandText = string.Format("select * from {0}", tableName);
conn.Open();
List<object> list = new List<object>();
SqlDataReader reader = cmd.ExecuteReader();
while (reader.Read())
{
object obj1 = Activator.CreateInstance(type);
foreach (PropertyInfo propertyInfo in type.GetProperties())
{
obj.GetType().GetProperty(propertyInfo.Name).SetValue(obj1,reader[propertyInfo.Name],null);
}
list.Add(obj1);
}
}
Thanx
Try an object-relational mapper like NHibernate.
Sounds like you are calculating the size for every instance of an entity. You should have some meta-data controller which caches the size of an entity for each table name. Assuming I understand your problem right, for the same table name, the size will always be the same.
If I understad the problem correctly, it sounds like you could simply make the DB do the work for you. You say "and fire a select statement": Can you not fire a smarter select statement that does what you are explaining?
I don't fully understand what you are trying to do when you say that you are looping through each column. But look into the "Group by" and "aggrop" aka "Aggregate operators" and see if any of those can help you out.
For optimization point of view , there is no need to maintain a connection to db for 1 million records, mean u are interacting with database until your loop ends. :( .
for optimization , you cache the whole table record in some dataset and then iterate it. not take the connection live to database for long time . hope it will be ans of your question . :)
You could probably tighten up the loop a bit to reduce calls involving reflection. You don't need to create that initial obj either:
PropertyInfo[] properties = type.GetProperties();
while (reader.Read())
{
object obj = Activator.CreateInstance(type);
foreach (PropertyInfo propertyInfo in properties)
{
propertyInfo.SetValue(obj, reader[propertyInfo.Name], null);
}
list.Add(obj);
}
But to get it even faster you could give pass the LoadAll() function a way to map the row to a new object, something along the lines of:
IEnumerable<T> LoadAll<T>(Func<DataReader, T> map) {
var tablename = typeof(T).GetField("TableName)......
// other connection and query stuff
while (reader.Read()) {
yield return map(reader);
}
}
// use it like:
var kittens = LoadAll<Kitten>(reader => new Kitten {
Name = (string)reader["Name"],
Colour = (string)reader["Colour"]
});
This also gives you more control over the mapping from the data layer to your domain object, for example your method using reflection would take a lot of modification to handle an enum property, which would be straightforward to code in an explicit map function.
You could try installing the free trial of ReSharper, its inspection tools can suggest a variety of ways to optimize your code from the ground up.
Related
I am trying to build an extension to populate an object with values from a SQLDataReader.
The code I have so far is
public static T RetunObject<T>(this Type source, SqlDataReader dr)
{
Type type = source.GetType();
PropertyInfo[] properties = type.GetProperties();
foreach (PropertyInfo property in properties)
{
property.SetValue(property, dr[property.Name]);
type.GetProperty(property.Name).SetValue(type, dr[property.Name]);
}
return type.Cast<T>();
}
One thing for sure is the last line is not right and not exactly sure if there is a way for this to work at this point.
The goal results would be to use the code like
MyClass myclass = new MyClass();
var results = myclass.ReturnObject(myDataReader);
Reflection has never been my strong suit. so I am pretty sure I am pretty far off.
UPDATED CODE WHICH APPEARS TO WORK
public static object RetunObject(this object source, SqlDataReader dr)
{
while (dr.Read())
{
foreach (PropertyInfo property in source.GetType().GetRuntimeProperties())
{
if(dr.GetSchemaTable().Select("ColumnName='"+ property.Name+"'").Count() == 1)
{
var readervalue = dr.GetValue(dr.GetOrdinal(property.Name));
property.SetValue(source, readervalue);
}
}
}
return source;
}
it is used like
return (Role)role.RetunObject(sqlcomm.ExecuteReader());
Takes a SQLDataReader and returns an generic object which is able to be cast to the object type you need it to be. Obviously if you try to cast to an invalid type of object then it won't work but so far this seems to work. My next trial will be to test nested class to see if it will work there as well.
In your example there is plenty errors and it is completely unclear what you try to accomplish. Probably you tried to just apply properties from reader to your object:
public static T RetunObject<T>(this T source, SqlDataReader dr)
{
var type = typeof(T);
var properties = type.GetProperties();
foreach (var property in properties)
{
property.SetValue(source, dr[property.Name]);
}
return source;
}
But, I recommend you to use Entity Framework for this, because you probably creating another bicycle by doing this. This task was completed a lot better/faster/safer than the solution you try to accomplish.
Try to spend a little of your time on things people commonly use before trying to implement something, that no one beside you will understand/support.
It will be a lot more useful and faster than spending time on potentially buggy code. Here is good start in EF Code First: https://msdn.microsoft.com/en-us/library/jj193542(v=vs.113).aspx
Is petapoco capable of achieving the following :
1.Unlimited joins in one query
2.Unlimited One to Many relations in one query
I have looked at PetaPOCO and it seems like it is not capable of doing more than 4 joins, the longest signature looks like :
db.Query<T1, T2, T3 , T4>
Also seems like it supports a one to many relation , but only for one composite object such as below :
db.FetchOneToMany<T1, T2> where T2 is a foreign key of T1
I'm testing some of the micro ORMs out there to stick to the best one. Do you know of any of them that can handle these situations and if none of the micro ORMs are supporting this feauture, how do you deal with an object that is like the following :
class A
{
List<B> member1;
List<C> member2;
Z member3; //Composit object
Z1 member4; //Composit object
Z2 member5; //Composit object
Z3 member6; //Composit object
Z4 member7; //Composit object
}
And then even more complicated is , what if member one (type B) has some composite object within itself ? What if we have :
class B
{
G member0;
}
Please don't propose a solution to hit database multiple times, coz it's going to be way too many calls when the objects become just a little bit complex.
Oh and i also know that one other way of tackling the case of unlimited joins is creating a very flat object that hols all fields combined. It's not an elegant solution at all.
The T1..T$ Query() overloads all pass through to the main Query(..Type[]..) method. You can either add more Query() overloads yourself to handle more T parameters, or pass in all the types you need in a Type array (which is what the T1-T4 functions do) :
Query<TRet>( new Type[]{typeof(Poco1), typeof(Poco2), typeof(Poco3), typeof(Poco4), typeof(Poco5)}, null, sql, args);
You can have multiple one to many relationships but Schotime is right, you need to be very careful of swathes of duplicate data coming back in your result set. Write the sql query and look at the result set, is the amount of duplication acceptable to you? If so then in Petapoco there is a concept of relator callbacks where you write a small class that handles the different pocos in a single result row and add each poco to the list properties on the parent poco.
http://www.toptensoftware.com/Articles/115/PetaPoco-Mapping-One-to-Many-and-Many-to-One-Relationships
I've never had to do this with multiple one to many but quoted from the above
"If you're joining more than two tables you'll need something more
complex but it's really just extensions of the above."
Another option is to have a stored procedure that does all the work in a single database request and have it return multiple result sets which I believe Schotime has achieved in his branch of petapoco but I've not used it myself yet so I can't really comment on if it will help here :
http://schotime.net/blog/index.php/2011/11/20/petapoco-multiple-result-sets/
If I absolutely had to wire up all the data in one go for objects as complex and nested as you are suggesting then I would use a stored procedure (a single db call) and stitch it all together with code. Only then would I figure out how to do this in Petapoco. However if your UI doesn't show all the nested data until the user clicks on an expander button (or similar) I'd use an AJAX call at that point rather than get all the data initially.
The answer is correct, but I came to this page from another forum and no one there could make this work, so I thought I would chip in what I did to make things clearer. Basically, I had code like the following:
var sql = "select * from someTable where tableId = #0";
var listOfStuff = _petapoco.Fetch<FirstType, SecondType, ThirdType, FourthType, FirstType>(new RelatorClass().MapIt, sql, idVar);
Since I needed to add in a fifth poco, and all the Fetch methods eventually lead to the master Query method listed above in the accepted answer, I had to do this:
var sql = "select * from someTable where tableId = #0";
Func<FirstType, SecondType, ThirdType, FourthType, FifthType, FirstType> mapIt = new RelatorClass().MapIt;
var listOfStuff = _petapoco.Query<FirstType>(new[] { typeof (FirstType), typeof (SecondType), typeof (ThirdType), typeof (FourthType), typeof(FifthType)}, mapIt, sql, idVar).ToList();
Now I can query with 5 pocos and I didn't have to modify the PetaPoco code. The only other thing to do would be to add to your relator class so you can tell PetaPoco where to map the new data and you're good to go.
Delegate:
Note: you don't have to return the (UserActivity,int) anonymous type like this, you can return a single type without the parenthesis! I'm just lazy and don't want to create a new model for the return type.
private delegate (UserActivity, int) GetIt(int fk_AccountTypeValue, UserActivityModel ua, User u, Client c, Client_Account ca);
SQL Operation:
public List<(UserActivity,int)> SomeMethodName(int orgnizationID)
{
var sql = Sql.Builder
.Select("TOP(200) at.FK_AccountTypeValue, ua.*, u.*, c.*, ca.*")
.From("UserActivity ua")
.LeftJoin("Users u").On("u.PK_UserID = ua.FK_UserID")
.LeftJoin("Client c").On("c.FK_UserID = u.PK_UserID")
.LeftJoin("Client_Account ca").On("ca.FK_ClientID = c.PK_ClientID")
.LeftJoin("AccountType at").On("at.PK_AccountType = c.FK_AccountTypeID")
.Where("u.FK_OrganizationID =#0", orgnizationID)
.OrderBy("ua.Timestamp desc");
GetIt obj = new GetIt(youKnowIt);
var typs = new Type[]{typeof(int), typeof(UserActivityModel), typeof(User), typeof(Client), typeof(Client_Account)};
var uaList = _database.Query<(UserActivity, int)>(typs, obj, sql.SQL, sql.Arguments).ToList();
return uaList;
}
Method pointed to by the delegate:
private (UserActivity,int) youKnowIt(int fk_AccountTypeValue, UserActivityModel ua, CurrentDesk.Models.User u, CurrentDesk.Models.Client c, CurrentDesk.Models.Client_Account ca)
{
// do stuff
var uam = new UserActivity()
{
// assign stuff
};
return (uam, fk_AccountTypeValue);
}
I am new to C# and this may end up being a dumb question but i need to ask anyway.
Is there a mechanism with C# to deserialize a result from an executed SQL statement into a c# object?
I have a C# program that reads a table from an sql server storing the row in an object - i am assigning each column value to an object member manually so i was wondering if there is a way to serialize the row automagically into an object. Or even better, a whole table in a collection of objects of the same type.
My environment is C#, VS2010, .NET4, SQLServer2008.
The assumption is that i know the columns i need, it's not a select * query.
A link to a neat example will also be appreciated.
Thanks.
You could use an ORM to do this. ADO.NET Entity Framework (checkout the video tutorials) and NHibernate are popular choices.
If the columns named as per the table names, you can do this with LINQ-to-SQL without any mapping code - just using ExecuteQuery:
using(var dc = new DataContext(connectionString)) {
var objects = dc.ExecuteQuery<YourType>(sql); // now iterate object
}
Additionally, the sql can be automatically parameterized using string.Format rules:
class Person {
public int Id {get;set;}
public string Name {get;set;}
public string Address {get;set;}
}
using(var dc = new DataContext(connectionString)) {
List<Person> people = dc.ExecuteQuery(#"
SELECT Id, Name Address
FROM [People]
WHERE [Name] = {0}", name).ToList(); // some LINQ too
}
Of course, you can also use the VS tools to create a typed data-context, allowing things like:
using(var dc = new SpecificDataContext(connectionString)) {
List<Person> people =
(from person in dc.People
where person.Name == name
select person).ToList();
}
I'm just getting my head round C#. I've been creating classes and objects so say i created a class called Member:
public class Member
{
public int MemberID;
public string FirstName;
public string LastName;
public string UserName;
}
and I create a new object of that class by doing this:
Member Billy = new Member();
Billy.UserName = "Jonesy";
Billy.FirstName = "Billy";
Billy.LastName = "Jones";
That's all fine but what if I've queried a database and gotten back 5 members, can I create objects on the fly? Or what is the best way to store these members in memory?
I've used VB.Net where I would just add them into a datatable. But I've never really done any object-oriented programming before and thought since I'm learning C#, now's the best time to learn OOP.
If you don't go with LINQ to SQL (or the Entity Framework) then using a regular ADO.NET DataReader you would loop through the results, instantiate a new object with the details, and add it to a list.
Roughly it would look like this:
List<Member> members = new List<Member>();
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
using (SqlCommand command = new SqlCommand(queryString, connection))
{
using (SqlDataReader reader = command.ExecuteReader())
{
while (reader.Read())
{
Member member = new Member();
member.UserName = reader.GetString(0);
member.FirstName = reader.GetString(1);
member.LastName = reader.GetString(2);
members.Add(member);
}
}
}
}
foreach(Member member in members)
{
// do something
}
This is a common problem. Fortunately there is a good answer to this: Linq To Sql! You can read about it here: http://weblogs.asp.net/scottgu/archive/2007/05/19/using-linq-to-sql-part-1.aspx
What it basically does is that it creates a class, one per table you choose in your database. This makes it very easy to get all your objects directly from the database into object oriented programming.
Saving is as easy as calling a function "SubmitChanges()". There are more providers for this but I think Linq will suit you well as a beginner as it abstracts a lot of the annoying parts.
I'd recommend that you look at LINQ to SQL. Then you can write code like this to query the database to get a specific user:
Member member = db.Members.Single(member => member.UserName == "Jonesy");
or to get users matching a criterion:
IQueryable<Member> members = db.Members
.Where(member => member.LastName == "Jones");
LINQ to SQL also takes care of writing the boilerplate code to declare the classes based on the database structure.
Linq2Sql suggested twice is not the only way, but having in mind case when all objects can be one to one mapped to tables in your database it works just fine. Personally I would go for EF instead however, because it allows you to have one more layer of abstraction.
Also I can suggest you to look at db4o and etc, there you just can save you poco and that's all.
I need to loop through the properties of a custom object type that I'm getting back from the database and only show the columns that contain data.
This means I cannot simply bind the list of objects to the datagrid.
I don't want to loop through each object and see if the column is empty/null and determine in the UI to display it.
What I'm thinking is in my business layer before I send the object back I would send an IEnumerable back with only those columns that should be visible. Thus I was thinking of using Linq to Object to do this, but I'm not sure that would be very pretty.
Does anyone know of a solution that I could use without a ton of IF statements that I could do to check through a large object (30 or so columns) to determine what should be shown or not.
Foreach (CustomerData customerdata in Customers)
{
if (!customerdata.address.Equals(""))
{
dgvCustomerData.Column["Address"].visible = false;
}
//Continue checking other data columns...
}
I wish to avoid all of this in the UI and all the IFs...
I'm having a brain fart on this one can anyone help me?
Thanks
You could do the following to simplify it a bit
Action<T,string> del = (value,name) => {
if ( value.Equals("") ) {
dgvCustomerData.Column[name].Visible = false;
}
};
foreach ( var data in Customers ) {
del(data.address,"Address");
del(data.name, "Name");
...
}
Take a look at the .NET Reflection Libraries. You can use reflection to get ahold of all of an object's properties, and loop through them to find out if they are null or not. Then you could return a collection of KeyValuePair objects where Key = property name, and Value = true/false. You'd then use the keyvaluepairs to set column visibility...