Working example with ReliableSqlConnection and Azure - c#

I have done everything i read on the internet, tutorials, but nothing seem to work!
https://www.google.com/search?q=reliablesqlconnection+azure
http://geekswithblogs.net/ScottKlein/archive/2012/01/27/understanding-sql-azure-throttling-and-implementing-retry-logic.aspx
i already install all the hand on lab:
http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=6932
The NuGets
PM> Install-Package EnterpriseLibrary.WindowsAzure.TransientFaultHandling
PM> Install-Package CommonServiceLocator
All the config I found to solve specific problems (just to mention one).
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section name="RetryPolicyConfiguration" type="Microsoft.Practices.EnterpriseLibrary.WindowsAzure.TransientFaultHandling.Configuration.RetryPolicyConfigurationSettings, ... />
<section name="typeRegistrationProvidersConfiguration" type="Microsoft.Practices.EnterpriseLibrary.Common.Configuration.TypeRegistrationProvidersConfigurationSection, Microsoft.Practices.EnterpriseLibrary.Common... />
</configSections>
<RetryPolicyConfiguration defaultRetryStrategy="Fixed Interval Retry Strategy">
<incremental name="Incremental Retry Strategy" />
<fixedInterval name="Fixed Interval Retry Strategy" />
<exponentialBackoff name="Exponential Backoff Retry Strategy" />
</RetryPolicyConfiguration>
<typeRegistrationProvidersConfiguration>
<add sectionName="RetryPolicyConfiguration" name="RetryPolicyConfiguration" />
</typeRegistrationProvidersConfiguration>
</configuration>
I cant get it work! I keep getting errors like
Could not load file or assembly 'Microsoft.Practices.ServiceLocation,
OR
The type RetryManager cannot be constructed. You must configure the container to supply this value
OR
Activation error occured while trying to get instance of type RetryManager, key "
OR it keep looking for *.cs files when debugging!
And more and more!!
Is someone there! with a simple azure ReliableSqlConnection sample! that i can download and run?
Please! Preferable using the latest dlls?
Thanks.
This is one of my simple testing code in a new WinForm solution
I have tried many combinations! like
ReliableSqlconnection with ExecuteReader or
SqlConnection with ExecuteReaderWithRetry or
ReliableSqlconnection with ExecuteReaderWithRetry
I just cant get it work! using SqlConnection with ExecuteReader, works perfect.! but is not reliable connection! so i will keep getting connection errors.
using (var cnn = new ReliableSqlConnection(connString))
{
cnn.Open();
using (var cmd = cnn.CreateCommand())
{
cmd.CommandText = "SELECT * FROM MyTable";
using (var rdr = cmd.ExecuteReaderWithRetry())
{
if (rdr.Read())
{
Console.Write(rdr.GetString(1));
}
}
}
}

Enterprise library 6 is all about the retry logic! No more ReliableSqlConnection.
This same logic will work for all azure storage services:
SQL Azure, Windows Azure Storage, Windows Azure Caching, or the Windows Azure Service Bus
You can even use it for all retry needs using your own retry logic class with the Interface ITransientErrorDetectionStrategy
So here is a working example (Console, WinForm, Website, WebMethod):
1.- Installing the NuGet (V. 6.0)
PM> Install-Package EnterpriseLibrary.TransientFaultHandling.WindowsAzure.Storage
2.- WebConfig:
<connectionStrings>
<add name="MyConnectionString" connectionString="Server=tcp:********.database.windows.net,1433;Database=DATABASENAME;User ID=*********;Password=********;Trusted_Connection=False;Encrypt=True;"/>
</connectionStrings>
3.- Using
using System;
using System.Configuration;
using System.Data;
using System.Data.SqlClient;
using System.Diagnostics;
using Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling;
4.- De example class
public class ReliableAzureConnection
{
string ConnectionString;
RetryPolicy RetryPolicy;
/// <summary>
/// Initialize the retryPolicy
/// Load the connection string from App.config
/// </summary>
public ReliableAzureConnection()
{
ConnectionString = ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString;
//This means, 3 retries, first error, wait 0.5 secs and the next errors, increment 1 second the waiting
Incremental RetryStrategy = new Incremental(3, TimeSpan.FromMilliseconds(500), TimeSpan.FromSeconds(1));
// You can use one of the built-in detection strategies for
//SQL Azure, Windows Azure Storage, Windows Azure Caching, or the Windows Azure Service Bus.
//You can also define detection strategies for any other services that your application uses.
RetryPolicy = new RetryPolicy<StorageTransientErrorDetectionStrategy>(RetryStrategy);
}
public DataTable GetTable(string commandText)
{
DataTable DataTable = null;
DataTable TempDataTable = null;
try
{
TempDataTable = new DataTable();
//This is the function that will retry,
//dont try to make your retry logic your self!
//there are so many error codes. Not all can retry
RetryPolicy.ExecuteAction(() =>
{
// Here you can add any logic!
//1.-Fill DataSet, NonQueries, ExecuteScalar
using (SqlConnection SqlConnection = new SqlConnection(ConnectionString))
{
SqlConnection.Open();
using (SqlCommand SqlCommand = new SqlCommand(commandText, SqlConnection))
{
TempDataTable.Load(SqlCommand.ExecuteReader());
}
}
DataTable = TempDataTable;
TempDataTable = null;
});
}
catch (SqlException ex)
{
//You can manage you own errors, for example bad queries or bad connections.
Debug.WriteLine(ex.Message);
throw;
}
finally
{
if (TempDataTable != null) TempDataTable.Dispose();
}
return DataTable;
}
//Example using ExecuteAction<TResult>
public DataTable GetTableUsingTResult(string commandText)
{
return RetryPolicy.ExecuteAction<DataTable>(() =>
{
DataTable DataTable = new DataTable();
using (SqlConnection SqlConnection = new SqlConnection(ConnectionString))
{
SqlConnection.Open();
using (SqlCommand SqlCommand = new SqlCommand(commandText, SqlConnection))
{
DataTable.Load(SqlCommand.ExecuteReader());
}
}
return DataTable;
});
}
}
5.- Call
ReliableAzureConnection ReliableAzureConnection = new ReliableAzureConnection();
DataTable MyTable = ReliableAzureConnection.GetTable("SELECT * FROM YourTable");
Debug.WriteLine(MyTable.Rows.Count);
I hope it help someone there. thanks.

As of .NET 4.6.1 the SqlConnection class now has retrying built in.
Troubleshoot, diagnose, and prevent SQL connection errors and transient errors for SQL Database
.NET SqlConnection parameters for connection retry
If your client program connects to to Azure SQL Database by using the .NET Framework class System.Data.SqlClient.SqlConnection, you should use .NET 4.6.1 or later (or .NET Core) so you can leverage its connection retry feature. Details of the feature are here.
When you build the connection string for your SqlConnection object, you should coordinate the values among the following parameters:
ConnectRetryCount (Default is 1. Range is 0 through 255.)
ConnectRetryInterval (Default is 1 second. Range is 1 through 60.)
Connection Timeout (Default is 15 seconds. Range is 0 through 2147483647)
Specifically, your chosen values should make the following equality true:
Connection Timeout = ConnectRetryCount * ConnectionRetryInterval
For example, if the count = 3, and interval = 10 seconds, a timeout of only 29 seconds would not quite give the system enough time for its 3rd and final retry at connecting: 29 < 3 * 10.

ReliableSqlConnection is technically part of EL 6 but it is outdated and is not appropriate for modern applications which care about scalability because that class, as well as all other Sql specific extension methods in the EL 6, doesn't support async operations, so, I believe the only good solution is to use generic retry EL 6 logic async methods to wrap async methods from ADO.NET.

Related

ASP.NET MVC kills my IIS - too many request

I facing a big issuse. I've build software that now over 100 users use at once.
Since than my MVC ASP.NET Application is dying. The IIS Crashes over 30-40 times a day.
i dont have any recursively code
Main Probleme is that all users fetch a boolean which tell's them if they need to get new data.
That fetch-http method is called 1-5 times a second from diffrent users.
But the SQL-Reader is slower than the request made.
Error Message: There is already an open DataReader associated with this Command which must be closed first.
or
Error Message: Internal connection fatal error. Error state: 15, Token : 97
My method:
[HttpGet]
[Route("fetch")]
public IHttpActionResult fetch()
{
SqlConnection connection = new SqlConnection(con);
string sql = "SELECT whentime FROM did_datachange WHERE ux_admin_id = " + id;
connection.Open();
using (SqlCommand command = new SqlCommand(sql, connection))
{
using (SqlDataReader reader = command.ExecuteReader())
{
while (reader.Read())
{
if (reader.FieldCount == 1)
dateTime = reader.GetDateTime(0);
}
}
}
connection.Close();
....following code...
}
i know that lock cloud solve that problem, but using lock slows down the code significantly. what can i do?
it was bad code (multi threading that threw alot of execeptions)

SQLCipher integration into Xamarin.Forms; SQLite Library doesn't support encryption exception

I am implementing SQLCipher into a Xamarin.Forms application. I thought everything was working until I noticed that the DB that was being created by the X.F. application was actually a SQLite3 DB w/o encryption or a password. After looking into it for a while, I haven't been able to find a solution. I am encountering an exception that says
System.InvalidOperationException: 'You specified a password in the connection string, but the native SQLite library you're using doesn't support encryption.'
I currently have 4 projects in this solution. The standard 3 in XamarinForms (Default PCL for cross platform stuff, Project.Android, and Project.iOS). In addition to those 3, I have a custom PCL that is labeled Project.Core. This PCL is responsible for all DataAccess since it implements the Repository Pattern, Unit Of Work, DbContext, etc.
In this 4th project, and within my DbContext.cs class, I have this:
// Added for more context
using System;
using System.IO;
using Microsoft.Data.Sqlite;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using Xamarin.Forms;
private SqliteConnection connection;
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
string connStr = Path.Combine(
path1: Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments),
path2: "App.db");
string passStr = deviceIdentifier;
string path = Path.GetDirectoryName(connStr);
if (!Directory.Exists(path))
{
Directory.CreateDirectory(path);
}
// Check if db file exists
if (!File.Exists(connStr))
{
FileStream stream = File.Create(connStr);
stream.Close();
}
// DOCS => https://learn.microsoft.com/en-us/dotnet/standard/data/sqlite/encryption?tabs=netcore-cli
// => https://www.bricelam.net/2016/06/13/sqlite-encryption.html
var connectionString = new SqliteConnectionStringBuilder()
{
DataSource = connStr,
Mode = SqliteOpenMode.ReadWriteCreate,
Password = passStr
}.ToString();
// NOTE: THIS IS WHERE THE EXCEPTION IS THROWN!!!
// THE CODE BELOW THIS IS AN ALTERNATE ROUTE THAT DOENS'T WORK EITHER
**connection.Open();**
// This code doesn't throw anything, but it doesn't key the DB either
using (SqliteCommand command = connection.CreateCommand())
{
command.CommandText = "SELECT quote($password);";
command.Parameters.AddWithValue("$password", passStr);
string escapedPassword = (string)command.ExecuteScalar(); // Protects against SQL injection
command.CommandText = "PRAGMA key = " + escapedPassword /*+ ";"*/;
command.Parameters.Clear();
command.ExecuteNonQuery();
}
#if DEBUG
optionsBuilder.EnableSensitiveDataLogging();
#endif
optionsBuilder.UseSqlite(connection);
SQLitePCL.Batteries_V2.Init();
}
Through my research it appears there might be an issue with one of the SQLite/SQLCipher packages in this PCL (the PCL is targeting .NET Standard 2.0 for reference).
I currently have:
Microsoft.Data.Sqlite.Core 3.1.1 (w/ dependencies on Microsoft.Data.Sqlite.dll & SQLitePCLRaw.core 2.0.2)
SQLitePCLRaw.bundle_sqlcipher 1.1.14 (w dependencies on SQLitePCLRaw.core 2.0.2, SQLitePCLRaw.batteries_sqlcipher.dll, SQLitePCLRaw.batteries_v2.dll)
A couple of other things to note:
When viewing SQLitePCL namespace, it shows the package as being sqlitepclraw.bundle_e_sqlite3 instead of having a reference to sqlcipher.
\.nuget\packages\sqlitepclraw.bundle_e_sqlite3\2.0.2\lib\netstandard2.0\SQLitePCLRaw.batteries_v2.dll
I believe there may be an issue with that dependency, but I'm not sure and would appreciate any assistance!
Thanks in advance.
PS - Can provide more information as requested
Found a working solution.
After looking into the packages, I found that replacing the existing SQLitePCLRaw bundle package with SQLitePCLRaw.bundle_zetetic found here, resolved the issues connecting and maintaining an encrypted database.
Working code snippet is:
// StringBuilder here, and the SqliteConnection below are
// from the Microsoft.Data.Sqlite namespace v3.1.1
var connectionString = new SqliteConnectionStringBuilder()
{
DataSource = connStr,
Mode = SqliteOpenMode.ReadWriteCreate,
Password = passStr
}.ToString();
connection = new SqliteConnection(connectionString);
connection.Open();

SSIS C# Script Task Error

All, I'm using SQL 2014 and Visual Studio 2013.
I have a script task that is not firing. It's being read, but not working. This task creates tables and inserts data into them. I need to do this with a script task as there are 100's of TSV files and the fields may change month to month and it's a pain maintaining individual nodes for each table.
If you look at the code snippet, the message boxes (1) do fire, but the script errors right after - I believe at (2):
The error message is:
I think this error refers to variables that are not accessible in the task or are misspelled, etc. I've checked these Ad nauseam - don't think that's it.
Any help is greatly appreciated. Thank you.
The problem in your code is that you are creating an ADO.NET (C# standard) connection in your script code, but the base of this - DBconn connection manager - is an OLEDB connection manager. These two connections could not be casted into one another.
Suggestions:
If possible, create DBconn connection manager as an ADO.NET. Then your code should work.
In case you have to keep DBconn as an OLEDB connection manager, you have to create a SqlConnection connection in script task based on DBconn. I have done that building connection string for ADO.NET from OLEDB conn string and creating a new SqlConnection with that connection string.
Below is a code sample for function generating Connection String.
using RuntimeWrapper = Microsoft.SqlServer.Dts.Runtime.Wrapper;
using System.Data.OleDb;
using System.Data.SqlClient;
using Microsoft.SqlServer.Dts.Runtime;
static string Get_ManagedConnString(string Src_Name, ConnectionManager CM)
{
if (CM.CreationName != "OLEDB")
throw new Exception(string.Format("Cannot get Conn String from non-OLEDB Conn manager {0}", CM.Name));
RuntimeWrapper.IDTSConnectionManagerDatabaseParameters100 cmParams_Src = CM.InnerObject as RuntimeWrapper.IDTSConnectionManagerDatabaseParameters100;
OleDbConnection oledbConn_Src = cmParams_Src.GetConnectionForSchema() as OleDbConnection;
OleDbConnectionStringBuilder oledbCSBuilder_Src = new OleDbConnectionStringBuilder(oledbConn_Src.ConnectionString);
SqlConnectionStringBuilder sqlCSBuilder_Src = new SqlConnectionStringBuilder();
sqlCSBuilder_Src.DataSource = oledbCSBuilder_Src["Data Source"].ToString();
sqlCSBuilder_Src.InitialCatalog = oledbCSBuilder_Src["Initial Catalog"].ToString();
if (oledbCSBuilder_Src["integrated security"].ToString() == "SSPI")
{
sqlCSBuilder_Src.IntegratedSecurity = true;
}
else
{
sqlCSBuilder_Src.UserID = oledbCSBuilder_Src["User ID"].ToString();
sqlCSBuilder_Src.Password = oledbCSBuilder_Src["Password"].ToString();
}
return sqlCSBuilder_Src.ConnectionString;
}

Using SignalR with Entity Framework to poll the database

I am trying to implement SignalR in a.NET MVC5 application. I am using EF 6.0.0 and SignalR 2.2.3. I want to update the client screen whenever there's some change in the database (update,insert, delete). I've searched around and found a few articles that show how to setup SignalR to send real time messages like chat, then there are some articles that show how to do what I want to do but with ADO.NET instead of EF. Here's an article I found that uses ADO.NET to achive the same thing. What can I do to implement the same thing but with EF? Here's a code snippet from the article that handles the ADO.NET part of the process:
public class JobInfoRepository {
public IEnumerable<JobInfo> GetData()
{
using (var connection = new SqlConnection(ConfigurationManager.ConnectionStrings["DefaultConnection"].ConnectionString))
{
connection.Open();
using (SqlCommand command = new SqlCommand(#"SELECT [JobID],[Name],[LastExecutionDate],[Status]
FROM [dbo].[JobInfo]", connection))
{
// Make sure the command object does not already have
// a notification object associated with it.
command.Notification = null;
SqlDependency dependency = new SqlDependency(command);
dependency.OnChange += new OnChangeEventHandler(dependency_OnChange);
if (connection.State == ConnectionState.Closed)
connection.Open();
using (var reader = command.ExecuteReader())
return reader.Cast<IDataRecord>()
.Select(x => new JobInfo(){
JobID = x.GetInt32(0),
Name = x.GetString(1),
LastExecutionDate = x.GetDateTime(2),
Status = x.GetString(3) }).ToList();
}
}
}
I've also read the comments in the article and some people are saying dependency_OnChange method is firing continuously. Shouldn't it be fired only when we change something in a database? If it is supposed to fire continuously, we might as well use simple ajax polling and poll the db every second, right?
Is there another article explaining EF with SignalR that I can look into? Thanks
After some more research, I have found out that EF can not be used instead of ADO.NET when configuring the hub:
using (SqlCommand command = new SqlCommand(#"SELECT [JobID],[Name],[LastExecutionDate],[Status]
FROM [dbo].[JobInfo]", connection))
Since it uses SqlCommand and not any lamba expression.

MySQL .Net connection pool connection.Open() very slow

Version 6.4.4:
Using the most basic implementation of MySqlConnection, the following code takes 2-5 seconds per connection when preloading the connection pool to reach the "Min Pool Size" configured in my connection string.
Any ideas why it is taking so long, how to fix, or workarounds?
Connection String:
<add name="users" connectionString="server=192.168.1.2;User Id=dbuser;password=dbpassword;database=users;Pooling=true;Min Pool Size=10;Max Pool Size=50;Persist Security Info=True;" />
Code
private static void MySqlConnectionTester()
{
string connectionString = ConfigurationManager.ConnectionStrings["users"].ConnectionString;
using (var connection = new MySqlConnection(connectionString))
{
using (var command = connection.CreateCommand())
{
command.CommandText = "select * from users;";
try
{
connection.Open(); // This line hangs until "Min Pool Size" is reached.
using (var reader = command.ExecuteReader())
{
while(reader.Read())
{
// Read results
}
}
}
catch(Exception ex)
{
// Log exception
}
finally
{
connection.Close();
}
}
}
}
At MySQL's homepage they write that you should avoid creating, opening and closing the connection object youself, instead you should use the helper class which should work better with connectionpooling.
I have not tested it, but was something I just read :)
The issue was also present on our side with MySql Connector .Net 6.6.5. The MySql Connector .Net 6.8.3 is solving this issue.
Fabrice

Categories

Resources