Develop Reference

Why does DELETE statement fail silently with MySQL Connector/net when it works in MySql Workbench and PHP/PDO?

I have a MariaDB database containing a table with information about mobile devices (both android and ios) running on a CentOS 7 server with Mono JIT compiler version 5.4.0.201 and MySql Connector/NET (latest version 8.0.13).
All is working great, except I can't delete devices based on push tokens from the Devices table via Connector/NET. Table definition is as follows:
CREATE TABLE `Devices` (
`idDevices` int(11) NOT NULL AUTO_INCREMENT,
`notificationId` varchar(160) DEFAULT NULL,
`deviceId` varchar(64) NOT NULL,
`isAndroid` tinyint(1) NOT NULL,
`deviceModel` varchar(64) DEFAULT NULL,
`appMajor` tinyint(4) DEFAULT NULL,
`appMinor` tinyint(4) DEFAULT NULL,
PRIMARY KEY (`idDevices`),
UNIQUE KEY `isAndroid` (`isAndroid`,`deviceId`),
UNIQUE KEY `notificationId_UNIQUE` (`isAndroid`,`notificationId`),
KEY `appVersion` (`appMajor`,`appMinor`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
When I run the DELETE statement from my Mono application, no exceptions occur, but the int value returned from ExecuteNonQuery() is always 0, and no rows have been deleted from the table. No errors or warnings are logged in mariadb.log (log_warning=4).
If I copy the exact same SQL statement to MySql Workbench and execute the statement there, the rows are deleted as expected.
If I create a PHP script and use PDO connection to the same db, it also works as expected. All three test cases use the same user (root). So it seems to be a problem specific to Connector/NET
var tokenCount = 0;
var sql = new StringBuilder ("DELETE FROM Devices WHERE isAndroid = 0 AND notificationId IN ('");
while (...) {
sql.Append (token);
sql.Append ("','");
tokenCount++;
}
sql.Length -= 2;
sql.Append (')');
using (var connection = new MySqlConnection (connectionString)) {
connection.Open ();
using (var command = new MySqlCommand (sql.ToString (), connection)) {
var affectedRows = command.ExecuteNonQuery ();
if (affectedRows != tokenCount) {
//Always logs affectedRows(0)
log.Warn ($"Remove tokenCount({tokenCount}) != affectedRows({affectedRows}). sql={sql}");
}
}
}
UPDATE: I enabled general_log and I see the correct db is initialized AND the statement is run - but the table is not changed! Output from log:
3 Init DB mydb
3 Query DELETE FROM Devices WHERE isAndroid = 0 AND notificationId IN ('002B1C477DB4F20868D157A806DF70E05D61D3950562C03E092707CA9C5CCF23')
UPDATE #2: I tried changing the DELETE statement into a SELECT statement using the same WHERE clause, and lo and behold, no rows are returned. So it seems that the matching on notificationId is not working for some reason. Could it be an encoding issue? Why would it work in other clients but not my Connector/NET client?

OK, so I found the problem after hours of debugging. Turns out it has nothing to do with Connector/NET. The problem was in how I was constructing the tokens, I accidentally appended some characters which would not print in my log4net log, so the SQL seemed OK on the client side-- but the actual SQL string which was sent to the database server contained invalid tokens!

Concurrency checks with Entity Framework and Stored Procedures

I am using Entity Framework and manipulating data in a sqlserver database via stored procedures (per client request).
Data is pulled from the database via stored procedures and the results of these stored procedures populates a SQLite db in the Winforms Application.
SQLite is used for additional querying and changing of data and then pushed back via update stored procedure to the sql server db when the user syncs
all stored procedures are on sql server (no in text / in line sql in the application)
I am faced with the scenario where multiple users can potentially attempt to update the same field, which poses 2 problems for me.
If they call the same stored procedure at the same time (select or update).
I am not sure what my options are here from a programming level, I don't have rights to make server changes.
if the field they are trying to update has already been updated.
for problem 2 I am trying to build in a check by date stamping the modification. ie. when a user syncs sql server adds that sync date to a date modified column, if a another user tries to modify the same field i want to check the date modified on his sqlite db and compare that to date modified in sql server, if sql server's date modified is more recent, keep sql server values, if syncing user's modified date is more recent use his...
I have looked into Resolving optimistic concurrency with a condition where the client wins.
using (var context = new BloggingContext())
{
var blog = context.Blogs.Find(1);
blog.Name = "The New ADO.NET Blog";
bool saveFailed;
do
{
saveFailed = false;
try
{
context.SaveChanges();
}
catch (DbUpdateConcurrencyException ex)
{
saveFailed = true;
// Update original values from the database
var entry = ex.Entries.Single();
entry.OriginalValues.SetValues(entry.GetDatabaseValues());
}
} while (saveFailed);
}
but this seems to only work when you directly query the db with Entity Framework and not when you want to update via stored procedure.
what can I use to perform these types of checks?

Ok, This is probably not the best solution, but it is what I was able to come up with, and although not tested extensively initial once over seems to be ok-ish.
I am not going to mark this as the answer, but its what i got working based on my question above.
calling stored procedure at same time, created a class for the transactions
public class TransactionUtils
{
public static TransactionScope CreateTransactionScope()
{
var transactionOptions = new TransactionOptions();
transactionOptions.IsolationLevel = IsolationLevel.ReadCommitted;
transactionOptions.Timeout = TransactionManager.DefaultTimeout;
return new TransactionScope(TransactionScopeOption.Required, transactionOptions);
}
}
and then in code use it as follows:
var newTransactionScope = TransactionUtils.CreateTransactionScope();
try
{
using (newTransactionScope)
{
using (var dbContextTransaction = db_context.Database.BeginTransaction(/*System.Data.IsolationLevel.ReadCommitted*/))
{
try
{
db_context.Database.CommandTimeout = 3600;
db_context.Database.SqlQuery<UpdateData>("UpdateProc #Param1, #Param2, #Param3, #Param4, #Param5, #Param6, #DateModified",
new SqlParameter("Param1", test1),
new SqlParameter("Param2", test2),
new SqlParameter("Param3", test3),
new SqlParameter("Param4", test4),
new SqlParameter("Param6", test5),
new SqlParameter("DateModified", DateTime.Now)).ToList();
dbContextTransaction.Commit();
}
catch (TransactionAbortedException ex)
{
dbContextTransaction.Rollback();
throw;
}
As for issue 2 (concurrency)
I could not find a way to use built in concurrency checks between data on SQL Server and the data that I want to update from SQLite (2 different contexts)
So I am storing Date modified in both sql server and sqlite.
the sqlite date modified is updated when the user modifies a record,
date modified on sql server is updated when a sync runs.
Before syncing I query the sqlServer db for the record to be updated's date modified and compare it with the sqlite's date modified for that record in a if statement and then either run the update stored procedure for that record or not

Unable to begin a distributed transaction using Entiy Framework

I'm having the following error executing this piece of code:
private bool _updateList(SysInfo _sysInfo, List<pList> _pList)
{
try
{
foreach (var p in _pList)
{
_context.spUpdatePListApprovalFlow(p.countryID, _sysInfo.User.JobRoleID, p.src, p.id, p.status, _sysInfo.User.Username);
}
return true;
}
catch (Exception ex) //debug only
{
throw; //throws error to the main try catch
}
}
ERROR
The operation could not be performed because OLE DB provider "MSDASQL"
for linked server "AS400_LINKEDSRV" was unable to begin a distributed
transaction.
However, everything works fine when I run the Stored Procedure in SQL Management Studio:
exec [dbo].[spUpdatePListApprovalFlow]
#CountryID = 123456,
#UserTypeID = 23456,
#Src = 1,
#Id = '123456789',
#Status = 30,
#Username = 'username'
I'm tired of digging for an answer nothing works... Few things I've tried:
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
insert multiple transactions in the stored procedure
This sp has 2 sub stored procedures on it. One that writes into a table of the application's database, and another that updates a table in as400.

In EF6 stored procedures are called in an explicit transaction, by default. You can turn this off for a particular DbContext instance by changing its configuration after creating it, or for all instances of a DbContext type by changing it in the constructor. EG
using (var db = new Db())
{
db.Configuration.EnsureTransactionsForFunctionsAndCommands = false;
//. . .
}
See: https://msdn.microsoft.com/en-us/library/system.data.entity.infrastructure.dbcontextconfiguration.ensuretransactionsforfunctionsandcommands

Ok, after half a day trying to solve this I've finally solved it.
Resolution
Downgraded from Entity Framework 6 to Entity Framework 5 and the
distribuited transactions error has gone.
Just pay attention, if you're going to do this, you have to change some usings in your code. ( in auto generated code in the Data Model as well)
EF 6 uses
using System.Data.Entity.Core.Objects
EF 5 uses
using System.Data.Objects;

If you don't need distributed transactions you can try to disable them in the settings of the linked server:
EXEC master.dbo.sp_serveroption
#server=N'AS400_LINKEDSRV',
#optname=N'remote proc transaction promotion',
#optvalue=N'false'

Refer to this Microsoft page on Linked Servers.
Your System Administrator and/or DBA will probably need to make changes to address the missing linked server definition to your AS/400 server.
Another possible issue is that the AS/400 server (from IBM) lacks software support for the OLE DB data sources. This too would be something that the System Administration staff may need to address.

SQL Slow in code not SSMS

I am running into issues working with a very large table from C# .Net 4.0.
For reference the table has ~120 Million rows.
I can't do even a simple query like
SELECT TOP(50) *
FROM TableName
WHERE CreatedOn < '2015-06-01';
From code it will timeout (Default setting - 15 seconds I believe), but in SSMS it is instant.
There is an index on the column in the WHERE clause. I have also tried explicitly casting the string to a DateTime, and using a DateTime parameter instead of a literal value.
I tried a different query that filters by the PK (bigint, identity, clustered index) If I do something like "Where TableRowID = 1" it works fine from code, but if I try to use "<" or "<=" instead it will timeout (returns instantly in SSMS), regardless of how many rows are turned.
The execution plans are very simple and are exactly the same.
I have tried changing ARITHABORT but that has had no effect.
I tried having the Application connect with my own account (SSPI) instead of its own credentials with no effect.
I have been researching this issue for a few days, but everything I have found blames different execution plans.
Does anyone have any idea what could be causing this issue?
The .Net code looks like this:
private DataSet ExecuteQuery(string query, string db, List<SqlParameter> parms = null)
{
string connectionString = ConfigurationManager.ConnectionStrings[db].ToString();
SqlConnection con = new SqlConnection(connectionString.Trim());
SqlDataAdapter sqlDataAdapter = new SqlDataAdapter();
try
{
con.Open();
DataSet ds = new DataSet();
sqlDataAdapter.SelectCommand = new SqlCommand(query, con);
sqlDataAdapter.SelectCommand.CommandType = CommandType.Text;
if (parms != null)
{
foreach (SqlParameter p in parms)
{
sqlDataAdapter.SelectCommand.Parameters.Add(p);
}
}
sqlDataAdapter.Fill(ds);
if (ds.Tables.Count > 0 && ds.Tables[0].Rows.Count > 0)
{
return ds;
}
return null;
}
finally
{
if (sqlDataAdapter != null)
sqlDataAdapter.Dispose();
if (con != null)
con.Dispose();
}
}
The error message I get in .Net is the standard timeout message:
Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

Here are my experience when dealing the issues.
Very C# sql code passed to sql server is
SELECT TOP(50) *
FROM TableName
WHERE CreatedOn < '2015-06-01'
Make sure the criteria. If it takes "instant time" to retrieve records on SSMS, the sql statement and db optimization is ok.
As other people have pointed out, you should post your C# codes and see what happens. There could be other issues. Could that be network? Could that be web.config? Do you call directly from C# code? Do you call via web service?
when you said time out? Does it time out at the time you execute the query. There is very little information you provide. Do you use third party routines (such as written by vendor or your company) to execute queries? If it possible, put the break point at the code that execute sql statement. What I mean is dig all the way to native codes, and put the break codes.
120 million records. Looks like the database has be optimized if it runs very fast on SSMS. I would take look outside SQL server.
good luck

My first step there would be to look at what your code is sending to the sql server. I'd begin by running the sql profiler. if you're not familiar with it. Here is a link on how to use it.
https://www.mssqltips.com/sqlservertip/2040/use-sql-server-profiler-to-trace-database-calls-from-third-party-applications/
After this you may want to look into network traffic times between the 2 servers.
Then look at IIS and see how it's setup. Could be a setting is wrong.
Check the error logs and see if you have any errors as well.

When you execute code in SSMS, the system is setting some default values on your connection. Specifically, look at Tools --> Options --> Query Execution --> Sql Server --> Advanced (and also ANSI). These statements are executed on your behalf when you open a new query window.
When you create your connection object in C#, are you setting these same options? If you don't explicitly set them here, you are taking the default values as defined in the actual SQL Server instance. In SSMS, you can get this by viewing the properties of the server instance and choosing the Connections page. This shows the default connection options.
You can also get this information without using the UI (you can do this in a separate application that uses the same connection string, for example). This article on MSSQLTips should guide you in the right direction.

Can't Create The Same Table Twice in SQL Server

i have a C# Windows Form Application. I push a button it should create a Table and insert a value (int).
I create the initial database as a Service-Database (Add New Item > Data > Service-Database).
The code for the button is (output follows below it):
private void button1_Click(object sender, EventArgs e)
{
SqlConnection thisConnection = new SqlConnection(Properties.Settings.Default.Database1ConnectionString);
SqlCommand nonqueryCommand = thisConnection.CreateCommand();
try
{
thisConnection.Open();
nonqueryCommand.CommandText = "CREATE TABLE MyTable1 (intColumn int)";
Console.WriteLine(nonqueryCommand.CommandText);
Console.WriteLine("Number of Rows Affected is: {0}", nonqueryCommand.ExecuteNonQuery());
nonqueryCommand.CommandText = "INSERT INTO MyTable1 VALUES (99)";
Console.WriteLine(nonqueryCommand.CommandText);
Console.WriteLine("Number of Rows Affected is: {0}",
nonqueryCommand.ExecuteNonQuery());
}
catch (SqlException ex)
{
Console.WriteLine(ex.ToString());
}
finally
{
thisConnection.Close(); // close connection
Console.WriteLine("Connection Closed.");
}
}
OUTPUT:
CREATE TABLE MyTable1 (intColumn int)
Number of Rows Affected is: -1
INSERT INTO MyTable1 VALUES (99)
Number of Rows Affected is: 1
Connection Closed.
Nothing Shows up on Server Explorer Though No additional Tables even if I close it down and reconnect.
If i push the button to make it issue the same again i get:
System.Data.SqlClient.SqlException (0x80131904): There is already an object named 'MyTable1' in the database.
but still nothing on server explorer.

The exception told you exactly what's going on. Your table already exists. You can't create it again. You need to DROP the table if it already exists.

Nothing Shows up on Server Explorer Though No additional Tables even if I close it down and reconnect.
Whenever you execute your program, Visual Studio's project management automatically deploy (copy that database) .mdf database file at deployment folder Debug\Bin. I think your code uses a database which is located at Debug\Bin folder (which will be not shown in server explorer) and Server Explorer shows a database (.mdf) which is located at root of project folder and it is empty.

Try spceifying the commandType property of the SqlCommand = CommandType.Text
Also make sure that you are connecting to the same instance of SQL. You can get your code's by breakpointing the line after you open the connection (for it's when you know it works) and going looking for the servername.
Note that you can have multiple SQL instances in one machine... so you could be working on the right server (say localhost) and yet not be accessing the correct instance (say SQLEXPRESS instead of MSSQLSERVER).