OleDb / DataTable to Model

OleDb / DataTable to Model - c#

have to extract data from a mdb file on the server. I can open and access the data. Now i have to map it to a model and i have no Idea how to receive the data as a model. My idea was to loop through the received DataTabel data and assign it to values of the model type:
OleDbCommand command = new OleDbCommand(sqlcommand, DbConnection);
adapter = new OleDbDataAdapter(command);
builder = new OleDbCommandBuilder(adapter);
dt = new DataTable();
try
{
DbConnection.Open();
adapter.Fill(dt);
foreach (DataRow row in dt.Rows)
{
var examplemodel= new exampleModel(
Id = row.ItemArray[0],
...
);
}
}
catch (Exception ex)
{
}
finally
{
DbConnection.Close();
}
The problem here is that I can not assign row.ItemArray[x] as an element of the model since row.ItemArray[x] is of type object and I cannot convert it to a int string or whatever.
Also I thought that there is maybe a simpler and cleaner approach to this problem.
Any ideas or suggestion are much appreciated.

You can use this
var Entity=(from DataRow dataRow in data.Rows select YourEntity<exampleModel>(dataRow)).ToList();
The Helper Class
public static T YourEntity<T>(DataRow row) where T : new()
{
var entity = new T();
var properties = typeof(T).GetProperties();
foreach (var property in properties)
{
//Get the description attribute
var descriptionAttribute = (DescriptionAttribute)property.GetCustomAttributes(typeof(DescriptionAttribute), true).SingleOrDefault();
if (descriptionAttribute == null)
continue;
property.SetValue(entity, row[descriptionAttribute.Description]);
}
return entity;
}
and Decorate the Entity with proper datatable header
public class exampleModel
{
....
[Description("Subentity_datatable_header_header")]
public string Subentity { get; set; }
....
}

I prefer simple Linq. Whatever approach it may be, it is very clean and safe if we use the column names instead of index.
You could do something like.
dt.Rows.AsEnumerable()
.Select(r=> new exampleModel()
{
Id = r.Field<int>("col1"),
Name = r.Field<string>("col2"),
...
});

Related

Dump rows from Oracle.ManagedDataAccess.Client.OracleDataReader to a collection that survives closed connection

I'm trying to write a simple helper function that connects to Oracle database using Oracle.ManagedDataAccess.Core library, and returns the rows.
Returned object should survive when connection to database is closed. Something similar to the following. Not sure what the will be.
public <sometype> GetOracleResults(string connectionString, string cmdText, List<OracleParameter> oracleParameters)
{
<sometype> results = null;
try
{
using (OracleConnection oracleConnection = new OracleConnection(connectionString))
{
oracleConnection.Open();
using (OracleCommand oracleCommand = new OracleCommand(cmdText, oracleConnection))
{
foreach (var param in oracleParameters)
{
oracleCommand.Parameters.Add(param);
}
OracleDataReader oracleDataReader = oracleCommand.ExecuteReader();
if(oracleDataReader.HasRows)
{
results = new <sometype>();
while (oracleDataReader.Read())
{
//loop through the reader and add results
return results;
}
}
}
}
}
catch (Exception)
{
//todo
throw;
}
}

Does it have to be strongly typed? If not, a coworker showed me sticking the data in a List of Dictionary items, E.G.:
var columns = new List<string>();
for (int i = 0; i < myReader.FieldCount; i++)
{
columns.Add(myReader.GetName(i).Trim());
}
while (myReader.Read() && results.Count < 200)
{
Dictionary<string, string> row = new Dictionary<string, string>();
foreach (var column in columns)
{
row.Add(column, Convert.ToString(myReader[column], CultureInfo.InvariantCulture).Trim());
}
results.Add(row);
}
I suppose if you know the schema so that the generic type coming into the method had property names matching the column names, then you could use reflection to match up the column results with the C# object.

Best way to populate datatable into datamodel with huge number of properties in C#

I have a mainframe database from which I am exporting 500 000 records using ODBC . Now I have to populate the datatable into data model(object).
The data model or class has more than 150 properties which needs to be populated from datatable. I am using reflection to do so inside every iteration of each datarow. I could have use list or reader properties to populate from datatable but there I have to define 150 members. To avoid and make it more generic I choosed to use reflection.
DataModel model = null;
//DataModel is a class containing various properties
//i.e public class DataModel
//{
// public string Name {get;set;}
// public string Role {get;set;}
//etc....150 properties
//}
dt = new DataTable();
using(OdbcDataReader reader = cmd.ExecuteReader())
{
DatatTable dtSchema = reader.GetSchemaTable();
List<DataColumn> listCols = new List<DataColumn>();
if(dtSchema != null)
{
foreach(DataRow drow in dtSchema.Rows)
{
string ColName = Convert.ToString(drow["ColumnName"]);
DataColumn column = new DataColumn(columnName,(Type)(drow["DataType"]));
listCols.Add(column);
dt.Columns.Add(column);
}
PropertyInfo[] properties = typeof(DataModel).GetProperties();
while(reader.Read())
{
DataRow dataRow = dt.NewRow();
for(int i = 0; i < listCols.COunt; i++)
{
dataRow[((DataCOlumn)listCols[i])]
}
dt.Rows.Add(dataRow);
model = new DataModel();
foreach(PropertyInfo property in properties)
{
if(dt.Columns.Contains(property.Name))
SetValue(model,property.name,dataRow[property,name]);
}
if(model != null) lst.Add(model);
}
}
I am getting exception by using this approach of reflection. Can the above be rectified and any different approach can be used? I am using reflection to do so inside every iteration of each datarow. Please tell me is there any other good or more performance efficient method apart from reflection to do so. (Please note I have more than 150 properties that needs to be populated)

extracting contents of Datatable into list C#

I am experimenting to take datatable contents into a list. I am using following code but its not working correctly.
public List<object> ShowMessage()
{
List<object> obj = new List<object>();
DataTable dt = new DataTable();
dt.Columns.Add("ID");
dt.Columns.Add("Name");
dt.Rows.Add("1","AAA");
dt.Rows.Add("2", "BBB");
dt.Rows.Add("3", "CCC");
foreach (DataRow dr in dt.Rows)
{
obj.Add(dr);
}
return obj;
}
I am new and not sure I am doing in a right way or I need to use some thing else. Any suggestion will be highly appreciated.
Thanks.

Converting your DataTable to list of name strings (with help of Linq to DataSet):
List<string> names =
dt.AsEnumerable().Select(r => r.Field<string>("Name")).ToList();
Which is same as
List<string> names = new List<string>();
foreach(DataRow r in dt.Rows)
names.Add((string)r["Name"]);

I think you are making a too abstract example. This one use a possible class named Person
public class Person
{
public int PersonID;
public string Name;
// other fields will follow in future
}
public List<Person> GetPersonList()
{
List<Person> people = new List<Person>();
// This is just as example, because in real code
// you get this table from a database
DataTable dt = new DataTable();
dt.Columns.Add("ID");
dt.Columns.Add("Name");
dt.Rows.Add(1,"John");
dt.Rows.Add(2, "Mark");
dt.Rows.Add(3, "Steve");
// Loop over the rows and construct a Person instance for every row
// Add that row to the List<Person> to return
foreach (DataRow dr in dt.Rows)
{
Person p = new Person() {PersonID =Convert.ToInt32(dr[0]), Name = dr[1].ToString());
people.Add(p);
}
return people;
}
By the way, this pattern of code, is exactly what a good ORM do for you. A little research for Entity Framework or Dapper would be very useful

#Karni: you were close to what you need. however below is the modified version of your code example so that you achieve what you need..
public class Obj
{
public int ID { get; set; }
public string Name { get; set; }
}
public class ListObj : List<Obj>
{
}
class Program
{
static void Main(string[] args)
{
DataTable dt = new DataTable();
dt.Columns.Add("ID");
dt.Columns.Add("Name");
dt.Rows.Add("1", "AAA");
dt.Rows.Add("2", "BBB");
dt.Rows.Add("3", "CCC");
ListObj objListObj = new ListObj();
//to fill the list / collection
for (int i = 0; i < dt.Rows.Count; i++)
{
objListObj.Add(new Obj() { ID = Convert.ToInt16(dt.Rows[i][0]), Name = dt.Rows[i][1].ToString() });
}
//To verify if the collection is filled.
foreach (var item in objListObj)
{
Console.WriteLine(item.ID + " : " + item.Name);
}
Console.Read();
}
}

How to convert DataTable to List<object>?

I have a strongly typed class PersonExport. I initially get data into a DataTable and call the following method on the DataTable to convert it to List<PersonExport>:
public static List<T> ConvertToList<T>(DataTable dt, out string message)
{
message = string.Empty;
var list = new List<T>();
try
{
var columnNames = dt.Columns.Cast<DataColumn>()
.Select(c => c.ColumnName)
.ToList();
var properties = typeof(T).GetProperties();
list = dt.AsEnumerable().Select(row =>
{
var objT = Activator.CreateInstance<T>();
foreach (var pro in properties)
{
if (columnNames.Contains(pro.Name))
{
var value = row[pro.Name];
var typeName = value.GetType().FullName;
if (typeName == "MySql.Data.Types.MySqlDateTime")
{
var mySqlDateTime = (MySqlDateTime) value;
if (mySqlDateTime.IsValidDateTime)
{
value = Convert.ToDateTime(mySqlDateTime.ToString());
pro.SetValue(objT, value, null);
}
}
else
{
pro.SetValue(objT, row.IsNull(pro.Name) ? null : value, null);
}
}
}
return objT;
}).ToList();
}
catch (Exception ex)
{
message = (ex.InnerException != null) ? ex.InnerException.Message : ex.Message;
}
return list;
}
However, once I start removing columns from the DataTable returned, it no longer works because the columns in the DataTable don't match up with the properties in the PersonExport list.
I am eventually using the exported list here to export to excel, but it is not working since I have modified by DataTable and it can't Deserialize into a List<PersonExport>:
//Trying to get data into List<object>
List<object> persons = GetPersonExport(query, out message);
var exportData = new Dictionary<string, List<object>> { { "xldata", persons} };
//Deserialize to List<object> to export
var persons = JsonConvert.DeserializeObject<List<object>>(args["xldata"]);
The above line just returns a List of empty objects.
A few things got me thinking, but I am wondering what might be the best approach. I am using the EPPLUS library to export data to excel and it has the option of hiding columns, so would it be better to just export the whole object and hide columns you don't want, this way you avoid the anonymous type or what I can do is still get the whole object, but then convert it to a DataTable and then remove the columns? Thoughts?

All that you want is:
public IEnumerable<object> GetListOfObject()
{
foreach (var prod in TenMostExpensiveProducts().Tables[0].AsEnumerable())
{
yield return prod;
}
}
Or:
TenMostExpensiveProducts().Tables[0].AsEnumerable().Select (x => x).ToList<object>()
But you can get it work more elegant via linq like this:
from prod in TenMostExpensiveProducts().Tables[0].AsEnumerable()
where prod.Field<decimal>("UnitPrice") > 62.500M
select prod
Or like this (AsDynamic is called directly on DataSet):
TenMostExpensiveProducts().AsDynamic().Where (x => x.UnitPrice > 62.500M)
I prefer the last approach while is is the most flexible.
P.S.: Don't forget to connect System.Data.DataSetExtensions.dll reference

List<Dictionary<string, object>> rows = new List<Dictionary<string, object>>();
Dictionary<string, object> row;
foreach (DataRow dr in dt.Rows)
{
row = new Dictionary<string, object>();
foreach (DataColumn col in dt.Columns)
{
row.Add(col.ColumnName, dr[col]);
}
rows.Add(row);
}
StringBuilder sbRes = new StringBuilder();
jSon.Serialize(rows, sbRes);
ret = sbRes.ToString();

Database context and Return Dynamic Result Set in ASP.NET MVC

In MVC 4 and EF 5 i want to run dynamic query.
var returndata = Context.Database.SqlQuery(Type, strsql, null);
i don't know, how many fields it will return and name. Out of this result i want to make table structure that will display on view.
Question : What should i passed as Type?
my query return below result:
Field 1, Field 2, Field 3, Field 4, Field 5
Row1...
Row2..
Appreciate any suggestion.

You could use a raw SQL query because EF doesn't support that:
private static IEnumerable<object[]> Read(DbDataReader reader)
{
while (reader.Read())
{
var values = new List<object>();
for (int i = 0; i < reader.FieldCount; i++)
{
values.Add(reader.GetValue(i));
}
yield return values.ToArray();
}
}
and then:
public ActionResult Index()
{
using (var ctx = new UsersContext())
using (var cmd = ctx.Database.Connection.CreateCommand())
{
ctx.Database.Connection.Open();
cmd.CommandText = "SELECT * FROM UserProfile";
using (var reader = cmd.ExecuteReader())
{
var model = Read(reader).ToList();
return View(model);
}
}
}
and finally in your view:
#model IEnumerable<object[]>
<table>
<tbody>
#foreach (var row in Model)
{
<tr>
#foreach (var column in row)
{
<td>#column</td>
}
</tr>
}
</tbody>
</table>

This method loads data from SQL select (with parameters) to the list of rows, where each row is the dictionary of columns (the key is the column name).
private static List<Dictionary<string, object>> LoadData(string sqlSelect, params object[] sqlParameters)
{
var table = new List<Dictionary<string, object>>();
using (var ctx = new DbEntities())
{
ctx.Database.Connection.Open();
using (var cmd = ctx.Database.Connection.CreateCommand())
{
cmd.CommandText = sqlSelect;
foreach (var param in sqlParameters)
cmd.Parameters.Add(param);
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
var row = new Dictionary<string, object>();
for (int i = 0; i < reader.FieldCount; i++)
row[reader.GetName(i)] = reader[i];
table.Add(row);
}
}
}
}
return table;
}

Finally i made is using TypeBuilder option suggested by "Mortalus" and ExpandoObject object. It has little performance overhead right now.
Take Typebuilder code from "Mortalus" answer then i made code according to my requirement as below.
List<Dictionary<string, object>> expandolist = new List<Dictionary<string, object>>();
foreach (var item in returndata)
{
IDictionary<string, object> expando = new ExpandoObject();
foreach (PropertyDescriptor propertyDescriptor in TypeDescriptor.GetProperties(item))
{
var obj = propertyDescriptor.GetValue(item);
expando.Add(propertyDescriptor.Name, obj);
}
expandolist.Add(new Dictionary<string, object>(expando));
}
return expandolist;
so now, I have "Dictionary" object from dynamic object. and using it you can work easily at design time rather then wait until runtime using "dynamic" object.

I have recently stumbled upon this example:
http://www.markzhou.com/blog/post/2011/06/02/Use-dynamic-type-in-Entity-Framework-41-SqlQuery()-method.aspx
I haven't had the time to test it myself but it seems that it is possible with some additional work to construct the dynamic type.
In short you would want to do something like this:
TypeBuilder builder = Program.CreateTypeBuilder(
"MyDynamicAssembly", "MyModule", "MyType");
Program.CreateAutoImplementedProperty(builder, "name", typeof(string));
Program.CreateAutoImplementedProperty(builder, "type", typeof(string));
Program.CreateAutoImplementedProperty(builder, "id", typeof(int));
Type resultType = builder.CreateType();
dynamic queryResult = context.Database.SqlQuery(
resultType, "SELECT * FROM sys.sysobjects");
Where TypeBuilder is described in details in the post I have attached.

Without knowing anything about the type that is returned, I think you might be out of luck.
If you know what patterns it might fall under, you could use some try { } catch () { }'s on interfaces that match those parameters on your otherwise dynamic query, but that seems like it might be a bit painful.

Unfortunately, EF won't materialize objects unless it knows their Type.
If this is really necessary for you, I think your best bet would be to fall back to ADO.NET and DataTable.

Similarly post by Darin Dimitrov, but it returns DataTable
public DataTable QueryToTable(Entities db, string queryText, SqlParameter[] parametes)
{
using ( DbDataAdapter adapter = new SqlDataAdapter())
{
adapter.SelectCommand = db.Database.Connection.CreateCommand();
adapter.SelectCommand.CommandText = queryText;
if (parametes != null)
adapter.SelectCommand.Parameters.AddRange(parametes);
DataTable table = new DataTable();
adapter.Fill(table);
return table;
}
}
Use
SqlParameter[] parametes = new[]
{
new SqlParameter("date_from", dateFrom)
};
DataTable tab = QueryToTable(new Entities(),
"Select * From SomeTable Where ADate >= #date_from", parametes);
Example for MS SQL Server

Adding to Petr Voborník's answer, dynamic query, I add dynamic insert of ResultSet, my application takes the dynamic query of all tables of the entire database, a chunk at a time and then inserts the dynamic results into a remote database, using Always Encrypted (omitted here). Passing a sb command and parameter object.
public void StoreData(DbContext dbContext, Dictionary<string, string> columnInfo, List<Dictionary<string, object>> multiInsertObj, string tableName)
{
_ctx = dbContext;
_columnInfo = columnInfo;
var sb = new StringBuilder();
sb.Append(BuildSqlCommand(tableName, columnInfo, multiInsertObj.Count));
ExecuteSqlCommand(sb, GetParamsObject(columnInfo, multiInsertObj));
}
private static StringBuilder BuildSqlCommand(string tableName, Dictionary<string, string> variableInfo, int variableCount)
{
//Build sql command
var sb = new StringBuilder();
sb.Append("INSERT INTO dbo." + tableName + "(");
foreach (var variable in variableInfo)
{
sb.Append(variable.Key);
sb.Append(", ");
}
sb.Append("SystemNumber, ");
sb.Remove(sb.Length - 2, 2).Append(") VALUES ");
for (var i = 0; i < variableCount; i++)
{
sb.Append("(");
foreach (var name in variableInfo.Keys)
{
sb.Append("#" + name + "_" + i + ",");
}
sb.Append("#SystemNumber" + "_" + i + ",");
sb.Remove(sb.Length - 1, 1).Append("),");
}
sb.Remove(sb.Length - 1, 1);
return sb;
}
private static object[] GetParamsObject(Dictionary<string, string> columnInfo, List<Dictionary<string, object>> multiInsertObj)
{
var variableCount = multiInsertObj.Count;
var rowCount = multiInsertObj[0].Keys.Count;
var objectLength = (rowCount + 1) * variableCount;
var variableDataTypes = columnInfo.Values.ToList();
var paramObj = new object[objectLength];
var j = 0;
var i = 0;
foreach (var row in multiInsertObj)
{
var k = 0;
foreach (var data in row)
{
var sb = new StringBuilder();
sb.Append("#");
sb.Append(data.Key);
sb.Append("_" + i);
paramObj[j] = new SqlParameter(sb.ToString(), SetSqlDataType(variableDataTypes[k])) { Direction = Input, Value = data.Value };
j++;
k++;
}
paramObj[j] = new SqlParameter(("#SystemNumber" + "_" + i), SetSqlDataType("int")) { Direction = Input, Value = _systemNumber };
i++;
j++;
}
return paramObj;
}
private static void ExecuteSqlCommand(StringBuilder sb, params object[] sqlParameters)
{
using (_ctx)
{
_ctx.Database.Connection.Open();
using (var cmd = _ctx.Database.Connection.CreateCommand())
{
cmd.CommandText = sb.ToString();
foreach (var param in sqlParameters)
cmd.Parameters.Add(param);
try
{
cmd.ExecuteNonQuery();
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
}
}
}

These approaches seemed a bit complicated to me, at least in my situation. All I needed was to return a DataTable so it could be previewed. Meaning every value could be a string.
So I created two classes:
public class DynamicResultModel
{
public DynamicResultColumnModel headers = new();
public List<DynamicResultColumnModel> rows = new();
}
public class DynamicResultColumnModel
{
public List<string> columns = new();
}
Then created a simple helper class to basically serialize and deserialize a DataTable into and out of a DynamicResultModel
public static DynamicResultModel DataTableToDynamic(DataTable dtData)
{
DynamicResultModel result = new();
DynamicResultColumnModel headers = new();
foreach (DataColumn col in dtData.Columns)
{
result.headers.columns.Add(col.ColumnName);
}
foreach (DataRow row in dtData.Rows)
{
DynamicResultColumnModel rowData = new();
foreach (var item in row.ItemArray)
{
rowData.columns.Add(item.ToString());
}
result.rows.Add(rowData);
}
return result;
}
The only caveat was that I could not return a DynamicResultModel from my controller, I had to serialize it into a string first. Regardless, this worked for me and saved me a lot of time.
So my API endpoint definition was
Task<ApiResponse<string>>
Instead of
Task<ApiResponse<DynamicResponseModel>>

Develop Reference

C# (C-Sharp) is a programming language developed by Microsoft that runs on the .NET Framework.

OleDb / DataTable to Model - c#

I prefer simple Linq. Whatever approach it may be, it is very clean and safe if we use the column names instead of index. You could do something like. dt.Rows.AsEnumerable() .Select(r=> new exampleModel() { Id = r.Field<int>("col1"), Name = r.Field<string>("col2"), ... });

Related

Dump rows from Oracle.ManagedDataAccess.Client.OracleDataReader to a collection that survives closed connection

Best way to populate datatable into datamodel with huge number of properties in C#

extracting contents of Datatable into list C#

How to convert DataTable to List<object>?

Database context and Return Dynamic Result Set in ASP.NET MVC

Categories

Resources