Develop Reference

A generic function that will return the results of a SELECT query from a local MySQL database - C# .Net 4.6.1

I'm using a MySQL local database, connecting to the database is not a problem (anymore). I have a small-scale database with around 6 different tables, each with around 4-6 columns, and rows <100 (not working with large data).
I am creating a WPF application that only ever needs to SELECT data from these databases, it never needs to add to them. The database is filled with static data which I will need to run SELECT statements on it and then use the results to display in my WPF app.
I need to make a function in my DBHandler class which can then be called from any other class in my system, to query the database with a specified SELECT statement, and then use the results. The problem is that my queries will vary - sometimes I might be calling for one column, such as;
(SELECT id FROM students WHERE name = 'Conor')
Sometimes I might be calling for multiple rows in a more complex statement.. such as this (pseudo):
(SELECT name, address FROM destinations WHERE long, lat intersects_with (SELECT long, lat FROM trains))
Whenever I call this function with a query, I will always be expecting the format of the data response, so if I just return a List<> or array, it should be no problem accessing the data even though the function is generic and not specific for one query or table.
So far I have tried this:
public static MySqlDataReader Query(string SQLQuery)
{
using (MySqlConnection con = new MySqlConnection(connectionString))
{
con.Open();
MySqlCommand command = new MySqlCommand(SQLQuery, con);
MySqlDataReader reader = command.ExecuteReader();
return reader;
}
}
// Some other class
MySqlDataReader reader = DBHandler.Query("SELECT * FROM destinations");
while (reader.Read())
{
MessageBox.Show(reader[0].ToString());
}
This doesn't work, because it complains the reader is closed. I presume I can't simply return a MySqlDataReader object.
My next thought process would be to do the actual query and return all the data in this Query function, and store all the results which can then be returned. But how I return the data is my main issue, because it needs to be generic for variable SELECT queries, so it can't have a fixed size for number of rows or columns returned. I thought maybe I could store it in a List<>, or a List<> within a List<>, but I'm really not sure on how to lay it out.
I know this is asking a lot but it is boggling my mind - I don't know how to make this generic SELECT function, but I know it will be really helpful as I will just need to call this whenever I need to get data in another part of the system.
Thank you!

You cannot try to use a DataReader when its connection has been closed. So, when your code exits the using block, the connection is closed as well the reader. However, you can pass to your Query method an Action delegate that receives a MySqlDataReader. This function will be defined by the caller of Query so you can customize it for your different tables while keeping a generic approach to the boilerplate code used to open, query and read the database.
public static MySqlDataReader Query(string SQLQuery, Action<MySqlDataReader> loader)
{
using (MySqlConnection con = new MySqlConnection(connectionString))
{
con.Open();
using(MySqlCommand command = new MySqlCommand(SQLQuery, con))
using(MySqlDataReader reader = command.ExecuteReader())
{
// here you can pass the reader, you are still inside the using block
while(reader.Read())
loader.Invoke(reader)
}
}
}
In the caller code you could write
List<Destination> destinations = new List<Destination>();
MySqlDataReader reader = DBHandler.Query("SELECT * FROM destinations", dataLoaderForDestination);
Console.WriteLine("Loaded " + destinations.Count + " destinations");
private void dataLoaderForDestination(MySqlDataReader reader)
{
Destination dest = new Destination();
dest.Address = reader.GetString(0);
dest.Nation = reader.GetInt32(1);
...
destinations.Add(dest);
}
Of course in a different point of your code you could pass the reference to a different Action delegate tailored for a different set of data returned by your query
List<Student> students = new List<Student>();
private void dataLoaderForStudents(MySqlDataReader reader)
{
Student st = new Student();
st.Name = reader.GetString(0);
st.Class = reader.GetInt32(1);
students.Add(st);
}

a reader is online, you need to loop inside (using connection), because if you leave the using, the connction is disposed and closed

C# getAll Function advice

Hi I am trying to create CRUD functions in C# but am stuck on my first one which is FetchALL, as so far it says not all code path returns a value.
Heres my code so far
public SqlDataReader FetchAll(string tableName)
{
using (SqlConnection conn = new SqlConnection(_ConnectionString,))
{
string query = "SELECT * FROM " + tableName;
SqlCommand command = new SqlCommand(query, conn);
using (SqlDataReader reader = command.ExecuteReader())
conn.Open();
conn.Close();
}
}
}
}
I can give you more information, thanks

You have a return type of SqlDataReader, but you aren't returning anything anywhere in your code. At the very least you should declare your data reader and return it like this:
public SqlDataReader FetchAll(string tableName)
{
SqlDataReader reader;
using (SqlConnection conn = new SqlConnection(_ConnectionString))
{
string query = "SELECT * FROM " + tableName;
// added using block for your command (thanks for pointing that out Alex K.)
using (SqlCommand command = new SqlCommand(query, conn))
{
conn.Open(); // <-- moved this ABOVE the execute line.
reader = command.ExecuteReader(); // <-- using the reader declared above.
//conn.Close(); <-- not needed. using block handles this for you.
}
}
return reader;
}
Note, I've noted a few other problems I saw as well, which you can see by my comments.
Also, I want to point out something very important: you should always avoid string concatenation in queries as this opens you up to the risk of a SQL injection attack (as gmiley has duly pointed out). In this case, you should create an enum which contains values associated with all the possible table names, and then use a dictionary to look up the table names based on their enum values. If a user provides an invalid/unknown value, you would then thrown an argument exception.
This isn't the end of your problems, though (as Default has pointed out). You can't create the connection in a using block, which disposes and closes as soon as it exits the block, and then use the SqlDataReader that is returned from the method. If I were you, I'd return a DataSet instead of a SqlDataReader. Here's how I'd do it:
First, create your enum of possible table values:
public enum Table
{
FirstTable,
SecondTable
}
And a dictionary that maps table enum values to the table names (which you will populate in your static constructor):
private static Dictionary<Table, string> _tableNames = new Dictionary<Table, string>(); // populate this in your static constructor.
And then here is your method to fetch the data:
public static System.Data.DataSet FetchAll(Table fromTable)
{
var ret = new System.Data.DataSet();
using (var conn = new System.Data.SqlClient.SqlConnection(_connectionString))
{
string tableName = "";
if (!_tableNames.TryGetValue(fromTable, out tableName)) throw new ArgumentException(string.Format(#"The table value ""{0}"" is not known.", fromTable.ToString()));
string query = string.Format("SELECT * FROM {0}", tableName);
using (var command = new System.Data.SqlClient.SqlCommand(query, conn))
{
using (var adapter = new System.Data.SqlClient.SqlDataAdapter(command))
{
adapter.Fill(ret);
}
}
}
return ret;
}
One final note, I'd advise you name your class-level variables with lower camel case per convention, e.g. _connectionString.

Firstly you aren't returning anything from the method. I'd add, are you sure you want to return a SqlDataReader? It is declared within a using block, so it will be closed by the time you return it anyway. I think you should re-evaluate what this function should return.

You need a return statment for the method to return a value.

Most efficient way to populate List<int> via ADO.NET call to stored procedure

I’m calling a stored procedure via ADO.NET as shown (in simplified form) below. The database is a MySQL database.
The stored procedure returns a list of ID values that correspond to calls that need to have their call times rescheduled.
The C# code stores them in a list.
My question is: Is there a more efficient way to get the values into the C# list, instead of using the DataReader as I’m doing?
I don’t know yet if the way I’m doing it is too inefficient for our application (that will be determined during testing), I’m just looking for a faster strategy, if one exists.
I looked at using a DataSet but, from what I’ve read, that could be slower if the list of ID-s is large (which it could be).
Also, from what I’ve read, LINQ might be slower, as well.
I only need to store the list into callsToRescheduleList; ie, I don’t need to do any random access of the ID-s, so those features of the DataSet are not needed.
I’m just looking for the fastest way to get the data into the list.
Any suggestions?
The C# code:
private void GetCallsToRescheduleList()
{
MySqlCommand cmd = new MySqlCommand
(
"`phytel`.`spPhy_GetCallsToRescheduleListPreviousDays`",
(MySqlConnection) DatabaseConnection, workerTransaction
);
cmd.CommandType = CommandType.StoredProcedure;
MySqlDataReader reader = cmd.ExecuteReader();
if (reader.HasRows)
{
while( reader.Read())
{
callsToRescheduleList.Add(reader.GetInt32(0));
}
}
}
The MySQL stored procedure:
CREATE PROCEDURE `spPhy_GetCallsToRescheduleListPreviousDays` ()
BEGIN
SELECT
id
FROM callrequest
WHERE
dialerCampaignId = 'CATH001’
AND
status = 'SCHEDULED’
;
END

AFAIK that's about as good as it gets - the select is narrow, and use of the reader ordinal overload in the tight loop is good.
Some checks:
Ensure that there is an index (key) on (dialerCampaignId, status) in MySql - I'm assuming there is better selectivity on dialerCampaignId here.
If there is a chance that your List<Int> consumer won't iterate the full list every time, as an alternative to rolling the data up into a list, consider also using an enumerable and yield return:
public IEnumerable<int> GetCallsToRescheduleListPreviousDays()
{
using (var cmd = new MySqlCommand
(
"`phytel`.`spPhy_GetCallsToRescheduleListPreviousDays`",
(MySqlConnection) DatabaseConnection, workerTransaction
)
{
cmd.CommandType = CommandType.StoredProcedure;
using (var reader = cmd.ExecuteReader())
{
if (reader.HasRows)
{
while( reader.Read())
{
yield return reader.GetInt32(0);
}
}
}
}
}
Also remember to Dispose of IDisposable resources like Commands and Readers

What is a more efficient way to query MySQL using C#?

Based on links around the StackOverflow site (references below), I've come up with this block of code to perform queries from my C# application to a MySQL database.
using (var dbConn = new MySqlConnection(config.DatabaseConnection))
{
using (var cmd = dbConn.CreateCommand())
{
dbConn.Open();
cmd.CommandType = CommandType.Text;
cmd.CommandText = "SELECT version() as Version";
using (IDataReader reader = cmd.ExecuteReader())
{
if (reader.Read())
{
Console.WriteLine("Database Version: " + reader.GetString(reader.GetOrdinal("Version")));
}
}
}
}
The problem I have with this, is that I have to build up this massive block of code every time I have a group of queries to make because I don't (and shouldn't) leave the connection open for the life of the application.
Is there a more efficient way to build the supporting structure (the nested usings, opening the connection, etc), and instead pass my connection string and the query I want to run and get the results back?
Referenced questions:
Use of connections with C# and MySql - Specifically the answer
by tsells
Mysql select where and C#
Update a mysql table using
C#
That is three of the ones I looked at. There were a few more, but my Google-fu can't refind them right now. All of these provide answers for how to perform a single query. I want to perform separate business logic queries - a few of them repeatedly - and don't want to repeat unneeded code.
What I've tried:
Based on the comment from nawfal, I have these two methods:
private MySqlDataReader RunSqlQuery(string query)
{
Dictionary<string, string> queryParms = new Dictionary<string, string>();
MySqlDataReader QueryResult = RunSqlQuery(query, queryParms);
return QueryResult;
}
private MySqlDataReader RunSqlQuery(string query, Dictionary<string, string> queryParms)
{
MySqlDataReader reader = null;
if (queryParms.Count > 0)
{
// Assign parameters
}
try
{
using (var dbConn = new MySqlConnection(config.DatabaseConnection))
{
using (var cmd = dbConn.CreateCommand())
{
dbConn.Open();
cmd.CommandType = CommandType.Text;
cmd.CommandText = query;
using (reader = cmd.ExecuteReader())
{
return reader;
}
}
}
}
catch (MySqlException ex)
{
// Oops.
}
return reader;
}
The problem with this attempt is that the reader closes when it is returned from the method.

Have you considered using an Object Relational Mapper (ORM)? I'm fond of Castle Active Record and NHibernate myself, but there's plenty of others. Entity Framework and Linq to SQL are popular Microsoft solutions too.
With these tools, your queries become pretty simple CRUD method calls that do the connection and session handling for you (mostly).

Instead of creating the reader in a using statement inside your RunSqlQuery method you could return it directly:
return cmd.ExecuteReader();
Then wrap the call to RunSqlQuery in a using statement:
using( var reader = RunSqlQuery(....) )
{
// Do stuff with reader.
}

You could use Actions or Funcs to get what I think you are after.
invoked like this...
RunSqlQuery("SELECT * FROM ...", reader => ReadResult(reader));
private bool ReadResult(MySqlDataReader reader)
{
//Use the reader to read the result
if (!success)
return false;
return true;
}
implemented like this...
private bool RunSqlQuery(string query, Func<MySqlDataReader, bool> readerAction)
{
Dictionary<string, string> queryParms = new Dictionary<string, string>();
return RunSqlQuery(query, readerAction, queryParms);
}
private bool RunSqlQuery(string query, Func<MySqlDataReader, bool> readerAction, Dictionary<string, string> queryParms)
{
MySqlDataReader reader = null;
if (queryParms.Count > 0)
{
// Assign parameters
}
try
{
using (var dbConn = new MySqlConnection(config.DatabaseConnection))
{
using (var cmd = dbConn.CreateCommand())
{
dbConn.Open();
cmd.CommandType = CommandType.Text;
cmd.CommandText = query;
using (reader = cmd.ExecuteReader())
{
return readerAction.Invoke(reader);
}
}
}
}
catch (MySqlException ex)
{
// Oops.
return false;
}
}

Why do you want to return the datareader from the method? It will be closed once u wrap it in inside the using block. Also you can assign parameters only after getting an instance of IDbCommand, so I have moved that part to inside of the using block.
If you strictly want to return the datareader, then better return IEnumerable<IDataRecord> using the yield keyword.
private IEnumerable<IDataRecord> RunSqlQuery(string query,
Dictionary<string, string> queryParms)
{
using (var dbConn = new MySqlConnection(config.DatabaseConnection))
{
using (var cmd = dbConn.CreateCommand())
{
if (queryParms.Count > 0)
{
// Assign parameters
}
cmd.CommandText = query;
cmd.Connection.Open();
using (var reader = cmd.ExecuteReader())
foreach (IDataRecord record in reader as IEnumerable)
yield return record;
}
}
}
Or even better is to read the data there itself and return the data back, as in this question. That way you dont have to rely on classes in db namespaces outside your db class.

I have been down that road. Along the lines of suggesting ORMs, I would recommend EF Code First. Sorry to be a bit off topic, but I have never had a second thought about going back to this pattern after using EF Code First.
Before Code First, EF was quite a pain, but now it has matured and if you had a DB you are potentially modifying structure, i.e. a new app feature requires a new table or column, then EF Code First approach is my recommendation. If it is a third party database or database for another app, that someone else manages its structure, then you only need to refresh your data model whenever they deploy changes, then I would not use Code First, and instead just use traditional EF where you generate/update your model based on some existing database.
Note you could adopt EF and begin using it while you keep your existing code base as-is. This depends on how much of your framework is dependent on using ADO objects though. EF Power Tools extension has a way to generate a Code First model, or you could just use the traditional non-Code First EF to generate a modal from database.
When you want to query, you can get right to the business of what you are trying to query without having alot of infrastructure code or wrappers. The other thing about wrappers like the above, is there are edge cases that you will have to go back to using the ADO API instead of your RunSqlQuery helper.
This is a trivial example, as usually I don't have methods like GetActivePeopleNames, but just put the query where ever it is needed. There is little overhead in terms of fluff code, so it isn't obtrusive to have my query among everything else. Although I do exercise some presenter patterns to abstract the query and data transformation from the business logic.
HREntities db = new HREntities();
private ICollection<string> GetActivePeopleNames()
{
return db.People.Where(p => p.IsActive).Select(p => p.FirstName + " " + p.LastName)
.ToList();
}
I didn't have to create a parameter object. I could have used some variable for Where(p => p.IsActive == someBool) and it would have been safe from SQL injection in that context. The connection is handled automatically. I can use .Include to grab related objects in the same connection if needed.

Populating an object based on a one-to-many table relationship in SQL

I have an object in C# like this:
private ClassWidget
{
public int ID;
public List<int> WidgetFavoriteNumbers;
}
Let's say I have two tables in SQL, one defines widget properties, and the other holds many records for a single widget, let's say the widget's favorite numbers:
widgets
-----------
id (int, not null)
// other properties ...
widget_nums
----------
widget_id (int, not null)
num (int)
I find myself frequently executing two SQL queries to populate this object even though I know I can join the tables to create just one query. The reason is that it seems simpler to populate the object with just the data I need rather than iterating over result sets that have a lot of duplicate data. Of course this widget example is much simplified compared to the real scenario. Here's the example:
int WidgetID = 8;
ClassWidget MyWidget = new ClassWidget();
using (SqlConnection conn = GetSQLConnection())
{
using (SqlCommand cmd = conn.CreateCommand())
{
conn.Open();
cmd.CommandText = #"SELECT id FROM widgets WHERE id = #WidgetID;";
cmd.Parameters.AddWithValue("WidgetID", WidgetID);
using (SqlDataReader Reader = cmd.ExecuteReader())
{
if (Reader.HasRows)
MyWidget.ID = GetDBInt("id", Reader); // custom method to read database result
}
cmd.CommandText = #"SELECT num FROM widget_nums WHERE widget_id = #WidgetID;";
using (SqlDataReader Reader = cmd.ExecuteReader())
{
if (Reader.HasRows)
while (Reader.Read())
MyWidget.WidgetFavoriteNumbers.Add(GetDBInt("num", Reader));
}
conn.Close();
}
}
My question is whether I should continue using this type of approach, or if performing a table join would be recommended. If the table join is recommended, what is the best design pattern to populate the object? My problem is that I have to create some logic to filter out duplicate rows, and is especially complicated when I am getting all widgets rather than just one.

I would use a table join. It is pretty simple to create a method which will traverse the results. You can use this method even when querying for multiple widgets and and their widget_nums
private IEnumerable<ClassWidget> MapReaderToWidget(IDataReader reader) {
var dict = new Dictionary<int, ClassWidget>();
while (reader.Read()) {
var id = (int)reader["id"];
ClassWidget widget;
if (!dict.TryGetValue(id, out widget)) {
widget = new ClassWidget {
ID = id,
WidgetFavoriteNumbers = new List<int>();
};
dict.Add(id, widget);
}
widget.WidgetFavoriteNumbers.Add((int)reader["num"]);
}
return dict.Values;
}
Then rewrite your method as following:
using (SqlConnection conn = GetSQLConnection())
{
using (SqlCommand cmd = conn.CreateCommand())
{
conn.Open();
cmd.CommandText = #"SELECT id FROM widgets INNER JOIN widget_nums on .... WHERE id = #WidgetID;";
cmd.Parameters.AddWithValue("WidgetID", WidgetID);
using (SqlDataReader Reader = cmd.ExecuteReader()) {
return MapReaderToWidget(reader).FirstOrDefault();
}
}
}

Use the table join. It uses a single SQL query, and it's extremely fast (far faster than your current approach). And for logic to filter out duplicate rows, you can come up with a query for that, I'd imagine; take some time to develop a query that gives you what you want out of the database, and you'll be pleased with the results.

I think you should start moving to Ado Entity Framework or LinQ to SQL as you data provideer as it will save you a lot of time and it will do exactly what you want in an efficient way.