Develop Reference

How To DEALLOCATE PREPARE Statement In C# Using Mysql.Data Client?

hope you guys are fine?
OK.. i am using MySQL.Data client/library to access and use MySQL database. I was using happily it for sometimes on quite a few project. But suddenly facing a new issue that causing me hold on my current project. :(
Because current project makes some (looks like it's a lot) db queries. and i am facing following exception :
Can't create more than max_prepared_stmt_count statements (current value: 16382)
i am closing and disposing the db engine/connection every time i am done with it. But getting damn confused why i am still getting this error.
​here is the sample code just to give you idea.. (trimmed out unnecessary parts)
//this loop call an API with pagination and get API response
while(ContinueSalesOrderPage(apiClient, ref pageNum, days, out string response, window) == true)
{
//this handle the API date for the current page, it's normally 500 entry per page, and it throws the error on 4th page
KeyValueTag error = HandleSalesOrderPageData(response, pageNum, out int numOrders, window);
}
private KeyValueTag HandleSalesOrderPageData(string response, int pageNum, out int numOrders, WaitWindow window)
{
numOrders = json.ArrayOf("List").Size;
//init db
DatabaseWriter dbEngine = new DatabaseWriter()
{
Host = dbHost,
Name = dbName,
User = dbUser,
Password = dbPass,
};
//connecting to database
bool pass = dbEngine.Connect();
//loop through all the entry for the page, generally it's 500 entries
for(int orderLoop = 0; orderLoop < numOrders; orderLoop++)
{
//this actually handle the queries, and per loop there could be 3 to 10+ insert/update query using prepared statements
KeyValueTag error = InsertOrUpdateSalesOrder(dbEngine, item, config, pageNum, orderLoop, numOrders, window);
}
//here as you can see, i disconnect from db engine, and following method also close the db connection before hand
dbEngine.Disconnect();
}
//code from DatabaseWriter class, as you see this method close and dispose the database properly
public void Disconnect()
{
_CMD.Dispose();
_engine.Close();
_engine.Dispose();
}
so, as you can see i close/dispose the database connection on each page processing, but still it shows me that error on 4th page. FYI, 4th page data is not the matter i checked that. If i skip the page and only process the 4th page, it process successfully.
and after some digging more in google, i found prepare statement is saved in database server and that needs to be close/deallocate. But i can't find any way to do that using MySQL.Data Client :(
following page says:
https://dev.mysql.com/doc/refman/8.0/en/sql-prepared-statements.html
A prepared statement is specific to the session in which it was created. If you terminate a session without deallocating a previously prepared statement, the server deallocates it automatically. 
but that seems incorrect, as i facing the error even after closing connection on each loop
so, i am at dead end and looking for some help here? 
thanks in advance
best regards

From the official docs, the role of max_prepared_stmt_count is
This variable limits the total number of prepared statements in the server.
Therefore, you need to increase the value of the above variable, so as to increase the maximum number of allowed prepared statements in your MySQL server's configuration
Open the my.cnf file
Under the mysqld section, there is a variable max_prepared_stmt_count. Edit the value accordingly(remember the upper end of this value is 1048576)
Save and close the file. Restart MySQL service for changes to take place.

You're probably running into bug 77421 in MySql.Data: by default, it doesn't reset connections.
This means that temporary tables, user-declared variables, and prepared statements are never cleared on the server.
You can fix this by adding Connection Reset = True; to your connection string.
Another fix would be to switch to MySqlConnector, an alternative ADO.NET provider for MySQL that fixes this and other bugs. (Disclaimer: I'm the lead author.)

How to add hashmap to hashmap that alredy exists without overriding the information?

I want to make list (hashmap in hashmap) of usernames in the firsestore.
I don't want to run over the data in participants every time that Running the action.
HashMap innermap = new HashMap();
HashMap outmap= new HashMap();
DocumentReference documentReference =Signup2.Getdatabase(Context).Collection("teams").Document(userdata.GetString("teamcode", "0000"))
.Collection("workout").Document(position.GetString("workoutdateposition", ""));
innermap.Put("username", userdata.GetString("username", ""));
outmap.Put("participants", innermap);
documentReference.Set(outmap);

If you want to merge the data in your API call with the existing data in the document, you have two options:
Call the UpdateAsync API
documentReference.UpdateAsync(outmap);
The UpdateAsync only works if the document already exists. If you don't know whether the document already exists or not, you can call Set or SetAsync with SetOptions.MergeAll:
documentReference. SetAsync(outmap, SetOptions.MergeAll);

SQL query in Crystal Reports doesn't update

I'm writing an application which changes Crystal Reports database access parameters in report files. I open reports witin a .NET windows forms app and apply the SDK functionality to change driver type (ODBC/OLEDB), server name, database name, user, password, authentication type etc. I'm having a problem with the database name. My code DOES change the specific properties of the table ConnectionInfo (in subreports too) but fails to update the general SQL Query within the report. This results in the report still accessing the old database.
So if the original report was configured to access database_1 and I'm changing it to database_2, it will have all table properties properly changed to database_2 (verifiable in the Designer). It will still have database_1 in the query though. The database name remains unchanged in both the SDK RowsetController.GetSQLStatement() result and in the Crystal Reports Developer query view (Database->Show SQL Query...).
Also I have to have both databases (database_1 and database_2) online while the conversion takes place, otherwise I get exceptions on either GetSQLStatement(when database_1 is offline; becuase it still refers to it) or SetTableLocation (when database_2 is offline - this is expected and acceptable behavior though). If both db are online, there are no errors.
Here is exactly what I'm using:
1) CrystalDecisions.CrystalReports.Engine.ReportDocument.Load(filePath, OpenReportMethod.OpenReportByTempCopy)
(...)
2) Make and fill CrystalDecisions.ReportAppServer.DataDefModel.PropertyBag
3) Iterate through CrystalDecisions.ReportAppServer.DataDefModel.Tables and apply all properties with SetTableLocaiton() for each one.
4) Repeat with each subreport
5) RowsetController.GetSQLStatement() to view the report's sql query.
Is there some way to update the query basing on the new table ConnectionInfos (which seem to be set properly)? I don't even see any possibility of manually updating the query (GET, search&replace, SET).
I'm using:
.NET 4.5,
Visual Studio 2012,
CR for VS 13.0.5,
Crystal Reports Developer 9.2.2.693 for results verification (source reports are also created with it)

Answer: set propper QualifiedName for each table. The QualifiedName is the full name of the table including the DbName. This later appears in the SQL Query of the report. By qualified name we understand:
myDatabase.mySchema.myTableName
Code example:
CrystalDecisions.ReportAppServer.DataDefModel.Table boTable = new CrystalDecisions.ReportAppServer.DataDefModel.Table();
CrystalDecisions.ReportAppServer.DataDefModel.PropertyBag boMainPropertyBag = new CrystalDecisions.ReportAppServer.DataDefModel.PropertyBag();
CrystalDecisions.ReportAppServer.DataDefModel.PropertyBag boInnerPropertyBag = new CrystalDecisions.ReportAppServer.DataDefModel.PropertyBag();
// Custom function to fill property bags with values which influence the table properties as seen in CR Developer
FillPropertyBags(boMainPropertyBag, boInnerPropertyBag);
CrystalDecisions.ReportAppServer.DataDefModel.ConnectionInfo boConnectionInfo = new CrystalDecisions.ReportAppServer.DataDefModel.ConnectionInfo();
boConnectionInfo.Attributes = boMainPropertyBag;
boConnectionInfo.Kind = CrystalDecisions.ReportAppServer.DataDefModel.CrConnectionInfoKindEnum.crConnectionInfoKindCRQE;
boTable.ConnectionInfo = boConnectionInfo;
CrystalDecisions.ReportAppServer.DataDefModel.Tables boTables = boReportDocument.ReportClientDocument.DatabaseController.Database.Tables;
for (int i = 0; i < boTables.Count; i++)
{
boTable.Name = boTables[i].Name;
// the QualifiedName is directly taken into the CR general query so this is a quick fix to change it
boTable.QualifiedName = boTables[i].QualifiedName.Replace("oldDbName", "newDbName");
boTable.Alias = boTables[i].Alias;
boReportDocument.ReportClientDocument.DatabaseController.SetTableLocation(boTables[i], boTable);
}
Uhh...Researching for whole day and found answer after publishing question on SO.

SQL Server CE 4.0 - DbUpdate Exception was unhandled

I am struggling with SQL Server CE 4.0 almost all day. I get this error when I try to save new record (entity) to my database.
First a little preview - I installed SQL Server CE 4.0 via NuGet and I created a database for my project. Then I created the Entity Framework model layer and started working with that.
When I get this error my inner exception says this :
The column cannot be modified. [ Column name = Id ]
I did a little research and find out that this may be cause because of my settings for the Id property so I changed it like that :
I already have 17 records that I have inserted manually for testing purposes but I doubt this may cause any problem.
So what I try to do is this:
public override void Save()
{
using (RalBaseEntities ctx = new RalBaseEntities())
{
System.Data.Entity.DbSet<MainInfo> mainInfoEntity = ctx.Set<MainInfo>();
MainInfo entity = new MainInfo();
entity.Manager = txtManager.Text;
entity.Broker = txtBroker.Text;
mainInfoEntity.Add(entity);
ctx.SaveChanges();
}
}
So when I try to execute the Save() method I get the error above. I wrote that in previous versions one should have to create the id manually, but it's fixed in v. 4.0 and judging by the settings that I show as an image here I don't see a reason just to get a new record with an auto generated unique Id.
The word Update in DbUpdate Exception that I get is worrying me a bit. Maybe I'm trying to save the data in a wrong way but I spend a lot of time googling and it seems to be the right way.

EzAPI OLE DB Destination

I've searched all over and I now have to ask SO. I'm trying to construct a simple dataflow using EzAPI. It's been anything but easy, but I'm committed to figuring this out. What I can't figure out is how to get the EzOleDBDestination working. Here's my complete code
var a = new Application();
// using a template since it's impossible to set up an ADO.NET connection to MySQL
// using EzAPI and potentially even with the raw SSIS API...
var pkg = new EzPackage(a.LoadPackage(#"C:\...\Package.dtsx", null));
pkg.Name = "Star";
var df = new EzDataFlow(pkg);
df.Name = "My DataFlow";
var src = new EzAdoNetSource(df);
src.Name = "Source Database";
src.SqlCommand = "SELECT * FROM enum_institution";
src.AccessMode = AccessMode.AM_SQLCOMMAND;
src.Connection = new EzConnectionManager(pkg, pkg.Connections["SourceDB"]);
src.ReinitializeMetaData();
var derived = new EzDerivedColumn(df);
derived.AttachTo(src);
derived.Name = "Prepare Dimension Attributes";
derived.LinkAllInputsToOutputs();
derived.Expression["SourceNumber"] = "id";
derived.Expression["Name"] = "(DT_STR,255,1252)description";
// EDIT: reordered the operation here and I no longer get an error, but
// I'm not getting any mappings or any input columns when I open the package in the designer
var dest = new EzOleDbDestination(df);
dest.AttachTo(derived, 0, 0);
dest.Name = "Target Database";
dest.AccessMode = 0;
dest.Table = "[dbo].[DimInstitution]";
dest.Connection = new EzConnectionManager(pkg, pkg.Connections["TargetDB"]);
// this comes from Yahia's link
var destInput = dest.Meta.InputCollection[0];
var destVirInput = destInput.GetVirtualInput();
var destInputCols = destInput.InputColumnCollection;
var destExtCols = destInput.ExternalMetadataColumnCollection;
var sourceColumns = derived.Meta.OutputCollection[0].OutputColumnCollection;
foreach(IDTSOutputColumn100 outputCol in sourceColumns) {
// Now getting COM Exception here...
var extCol = destExtCols[outputCol.Name];
if(extCol != null) {
// Create an input column from an output col of previous component.
destVirInput.SetUsageType(outputCol.ID, DTSUsageType.UT_READONLY);
var inputCol = destInputCols.GetInputColumnByLineageID(outputCol.ID);
if(inputCol != null) {
// map the input column with an external metadata column
dest.Comp.MapInputColumn(destInput.ID, inputCol.ID, extCol.ID);
}
}
}
Basically, anything that involves calls to ReinitializeMetadata() results in 0xC0090001, because that method is where the error happens. There's no real documentation to help me, so I have to rely on any gurus here.
I should mention that the source DB is MySQL and the target DB is SQL Server. Building packages like this using the SSIS designer works fine, so I know it's possible.
Feel free to tell me if I'm doing anything else wrong.
EDIT: here's a link to the base package I'm using as a template: http://www.filedropper.com/package_1 . I've redacted the connection details, but any MySQL and SQL Server database will do. The package will read from MySQL (using the MySQL ADO.NET Connector) and write to SQL Server.
The database schema is mostly irrelevant. For testing, just make a table in MySQL that has two columns: id (int) and description (varchar), with id being the primary key. Make equivalent columns in SQL Server. The goal here is simply to copy from one to the other. It may end up being more complex at some point, but I have to get past this hurdle first.

I can't test this now BUT I am rather sure that the following will help you get it working:
Calling ReinitializeMetadata() causes the component to fetch the table metadata. This should only be called after setting the AccessMode and related property. You are calling it before setting AccessMode...
Various samples including advice on debugging problems
define the derived column(s) directly in the SQL command instead of using a EzDerivedColumn
try to get it working with 2 SQL Server DBs first, some of the available MySQL ADO.NET provider have some shortcomings under some circumstances
UPDATE - as per comments some more information on debugging this and a link to a complete end-to-end sample with source:
http://blogs.msdn.com/b/mattm/archive/2009/08/03/looking-up-ssis-hresult-comexception-errorcode.aspx
http://blogs.msdn.com/b/mattm/archive/2009/08/03/debugging-a-comexception-during-package-generation.aspx
Complete working sample with source

I've had this exact same issue and been able to resolve it with a lot of experimentation. In short you must set the connection for both the source and destination, and then call the attachTo after both connections are set. You must call attachTo for every component.
I've written a blog about starting with an SSIS package as a template, and then manipulating it programmatically to produce a set of new packages.
The article explains the issue more.