I have an Azure function app that triggers when data is enqueued to a service bus queue. In the Azure function, I implemented a charging method. First I get the particular row from the database and check the account points.
SELECT * FROM UsersAccountPoints WHERE UserId = #UserId
Then I update the points.
UPDATE UsersAccountPoints
SET FundsChange = #ChargeAmount, FundsAmount -= #ChargeAmount
WHERE UserId = #UserId
When I run this on local, it runs perfectly fine. But when I deploy it on Azure, the function app starts scaling. Then parallel processors start updating the same row of the UsersAccountPoints table, giving me confusing results.
Then I tried UPDLOCK.
SELECT *
FROM UsersAccountPoints WITH (UPDLOCK)
WHERE UserId = #UserId
But this also gives me the same result. I use the Azure premium function, SQL Server database, and Dapper for ORM. Is there a better way to handle this kind of scenario than UPDLOCK?
Related
I'm developing an cryptocurrency project. When a customer tries to withdraw his money, I use the code shown here to ensure that this customer has enough balance to make the withdrawal, then I pay the desired amount into customer wallet.
public async Task EnsureCustomerHasEnoughBalance(decimal withdrawAmount, Guid customerId)
{
var currentBalance = await _transactionService.GetCustomerBalance(customerId);
if (currentBalance < withdrawAmount)
throw new Exception("Insufficient balance");
}
The problem is if someone calls my async API many times and quickly, some of the requests will be processed the same time in different threads. So, any customer can hack my code and withdraw more money than his balance allows.
I've tried using lock, semaphore and etc but none of them work in async. Do you have any solution?
Thanks for any help
I have some experiences for this case. To be honest when running a payment application, and with the critical action relate to balance, we need to be really carefully. In this case i don't use interal lock since we would deploy the app in many nodes, that can not guarantee the atomic.
In my project, we have two options:
Centralize locking by using redis (redlock).
Using transaction, we write procedure and wrap it in transaction.
CREATE PROCEDURE withraw(in amount int)
BEGIN
START TRANSACTION;
select balance from accounts where id = account_id into #avaiable_balance for update;
if(#avaiable_balance > amount )
then
update accounts set balance = balance -amount,balance=balance-amount where id = account_id;
select 1;
else
select -1;
COMMIT;
END
Do you use Nethereum nuget package?
If you use Nethereum package, it's ok.
I used Nethereum package for withdraw money.
There is no problem.
Please check this site.
https://nethereum.com/
https://docs.nethereum.com/en/latest/nethereum-block-processing-detail/
I have a ASP .NET Core application using EF and an Azure SQL database. We recently migrated the database to the Hyperscale service tier. The database has 2 vCores and 2 secondary replicas. When we have a function query a secondary replica (by either modifying the connection string to include ApplicationIntent=READONLY; or by using a new services.AddDbContext() from our Startup.cs) we find that functions take 20-30x longer to execute.
For instance, this function:
public async Task<List<StaffWorkMuchModel>> ExemptStaffWorkMuchPerWeek(int quarterId, int facilityId) {
using (var dbConnection = (IDbConnection) _serviceProvider.GetService(typeof(IDbConnection))) {
dbConnection.ConnectionString += "ApplicationIntent=READONLY;";
dbConnection.Open();
return (await dbConnection.QueryAsync<StaffWorkMuchModel>("ExemptStaffWorkMuchPerWeek", new {
id_qtr = quarterId,
id_fac = facilityId
}, commandType: CommandType.StoredProcedure, commandTimeout: 150)).ToList();
}
}
We have tried to query the secondary replica directly using SQL Server Management Studio and have found that the queries all return in less than a second. Also, when we add breakpoints in our code, it seems like the queries are returning results immediately. Most of the pages we are having issues with use ajax to call 4+ functions very similar to the one above. It almost seems like they are not running asynchronously.
This same code runs great when we comment out:
dbConnection.ConnectionString += "ApplicationIntent=READONLY;";
Any idea what could be causing all of our secondary replica functionss to load so slow?
We are using Hangfire 1.7.2 within our ASP.NET Web project with SQL Server 2016. We have around 150 sites on our server, with each site using Hangfire 1.7.2. We noticed that when we upgraded these sites to use Hangfire, the DB server collapsed. Checking the DB logs, we found out there were multiple locking queries. We have identified one RPC Event “sys.sp_getapplock;1” In the all blocking sessions. It seems like Hangfire is locking our DB rendering whole DB unusable. We noticed almost 670+ locking queries because of Hangfire.
This could possibly be due to these properties we setup:
SlidingInvisibilityTimeout = TimeSpan.FromMinutes(30),
QueuePollInterval = TimeSpan.FromHours(5)
Each site has around 20 background jobs, a few of them run every minute, whereas others every hour, every 6 hours and some once a day.
I have searched the documentation but could not find anything which could explain these two properties or how to set them to avoid DB locks.
Looking for some help on this.
EDIT: The following queries are executed at every second:
exec sp_executesql N'select count(*) from [HangFire].[Set] with (readcommittedlock, forceseek) where [Key] = #key',N'#key nvarchar(4000)',#key=N'retries'
select distinct(Queue) from [HangFire].JobQueue with (nolock)
exec sp_executesql N'select count(*) from [HangFire].[Set] with (readcommittedlock, forceseek) where [Key] = #key',N'#key nvarchar(4000)',#key=N'retries'
irrespective of various combinations of timespan values we set. Here is the code of GetHangfirServers we are using:
public static IEnumerable<IDisposable> GetHangfireServers()
{
// Reference for GlobalConfiguration.Configuration: http://docs.hangfire.io/en/latest/getting-started/index.html
// Reference for UseSqlServerStorage: http://docs.hangfire.io/en/latest/configuration/using-sql-server.html#configuring-the-polling-interval
GlobalConfiguration.Configuration
.SetDataCompatibilityLevel(CompatibilityLevel.Version_170)
.UseSimpleAssemblyNameTypeSerializer()
.UseRecommendedSerializerSettings()
.UseSqlServerStorage(ConfigurationManager.ConnectionStrings["abc"]
.ConnectionString, new SqlServerStorageOptions
{
CommandBatchMaxTimeout = TimeSpan.FromMinutes(5),
SlidingInvisibilityTimeout = TimeSpan.FromMinutes(30),
QueuePollInterval = TimeSpan.FromHours(5), // Hangfire will poll after 5 hrs to check failed jobs.
UseRecommendedIsolationLevel = true,
UsePageLocksOnDequeue = true,
DisableGlobalLocks = true
});
// Reference: https://docs.hangfire.io/en/latest/background-processing/configuring-degree-of-parallelism.html
var options = new BackgroundJobServerOptions
{
WorkerCount = 5
};
var server = new BackgroundJobServer(options);
yield return server;
}
The worker count is set just to 5.
There are just 4 jobs and even those are completed (SELECT * FROM [HangFire].[State]):
Do you have any idea why the Hangfire is hitting so many queries at each second?
We faced this issue in one of our projects. The hangfire dashboard is pretty read heavy and it polls the hangfire db very frequently to refresh job status.
Best solution that worked for us was to have a dedicated hangfire database.
That way you will isolate the application queries from hangfire queries and your application queries won't be affected by the hangfire server and dashboard queries.
There is a newer configuration option called SlidingInvisibilityTimeout when configuring SqlServerStorage that causes these database locks as part of newer fetching non-transactional message fetching algorithm. It is meant for long running jobs that may cause backups of transactional logs to error out (as there is a database transaction that is still active as part of the long running job).
.UseSqlServerStorage(
"connection_string",
new SqlServerStorageOptions { SlidingInvisibilityTimeout = TimeSpan.FromMinutes(5) });
Our DBA did not like the database locks, so I just removed this SlidingInvisibilityTimeout option to use the old transactional based message fetching algorithm since I didn't have any long running jobs in my queue.
Whether you enable this option or not is dependent on your situation. You may want to consider moving your queue database outside of your application database if it isn't already and enable the SlidingInvisibilityTimeout option. If your DBA can't live with the locks even if the queue is a separate database, then maybe you could refactor your tasks into many more smaller tasks that are shorter lived. Just some ideas.
https://www.hangfire.io/blog/2017/06/16/hangfire-1.6.14.html
SqlServerStorage runs Install.sql which takes an exclusive schema lock on the Hangfire-schema.
DECLARE #SchemaLockResult INT;
EXEC #SchemaLockResult = sp_getapplock #Resource = '$(HangFireSchema):SchemaLock',
#LockMode = 'Exclusive'
From the Hangfire documentation:
"SQL Server objects are installed automatically from the SqlServerStorage constructor by executing statements
described in the Install.sql file (which is located under the tools folder in the NuGet package). Which contains
the migration script, so new versions of Hangfire with schema changes can be installed seamlessly, without your
intervention."
If you don't want to run this script everytime you could set SqlServerStorageOptions.PrepareSchemaIfNecessary to false.
var options = new SqlServerStorageOptions
{
PrepareSchemaIfNecessary = false
};
var sqlServerStorage = new SqlServerStorage(connectionstring, options);
Instead run the Install.sql manually by using this line:
SqlServerObjectsInstaller.Install(connection);
Although there are many questions on this topic going around, none of the suggestions seem to work.
I've got a couple of stored procedures which should be run on a daily basis - some of these stored procedures are quite straight forward, others a bit more tricky. But even the simplest of procedures will run indefinitely when called from a C# program (console) using the SqlClient.
This client is running on the server and should be promoted to a windows service when it's actually functioning.
What I've tried so far.
Add ARITHABORT ON (or OFF) as first execute after connection initialization.
Add ARITHABORT ON (or OFF) as first command in the Stored Procedure
Using WITH RECOMPILE
Add ARITHABORT as a global configuration thing.
(EXEC sys.sp_configure N'user options', N'64'
GO
RECONFIGURE WITH OVERRIDE
GO)
The stored procedures (all of them) have no input parameters and the simplest (the only one I currently use) is this:
CREATE PROCEDURE [dbo].[_clean_messageLog]
WITH RECOMPILE
AS
BEGIN
SET NOCOUNT ON;
set arithabort on;
DELETE FROM MessageLog WHERE Moment < GETDATE() - 60;
DELETE FROM MessageLog WHERE Moment < GETDATE() - 30 AND [Status] = 200;
END
There are no messages to be actually deleted and in SSMS the stored procedures executes (as expected) within milliseconds.
From the C# Console Application however it takes forever (literally).
Main-method:
const int TIME_OUT = 900000; // 15 minutes
timer.Stop();
foreach (var command in commands.Where(command => !string.IsNullOrWhiteSpace(command)))
{
var start = DateTime.Now;
WriteEvent(string.Format("Starting: {0}", command), EventLogEntryType.Information);
using (var sql = new Lib.Data.SqlServerHelper(connectionString))
{
sql.newCommand(command.Trim());
sql.execute(TIME_OUT);
}
WriteEvent(string.Format("Done in {0} seconds", DateTime.Now.Subtract(start).TotalSeconds), EventLogEntryType.Information);
}
Does anyone have suggestions?
EDIT
The sqlHelper is just a basic (very simple) wrapper. But even when I change the above code to this:
foreach (var command in commands.Where(command => !string.IsNullOrWhiteSpace(command)))
{
var start = DateTime.Now;
WriteEvent(string.Format("Starting: {0}", command), EventLogEntryType.Information);
using (var sql = new SqlConnection(connectionString))
{
sql.Open();
var sqlCommand = new SqlCommand(command.Trim(), sql) {CommandType = CommandType.StoredProcedure};
sqlCommand.ExecuteNonQuery();
}
WriteEvent(string.Format("Done in {0} seconds", DateTime.Now.Subtract(start).TotalSeconds), EventLogEntryType.Information);
}
It's exactly the same.
EDIT #2
Is there an option I can schedule these stored procedures to be run by SQL Server itself on an interval or specific time?
SOLVED
Kinda, although I've never found an actual C# solution to my problem using the SQL Server Agent did the trick. The C# processes were locked due to deadlock issues - which sometimes also occur on the jobs (not as many as the console program), but we're working on that.
Is there an option I can schedule these stored procedures to be run by
SQL Server itself on an interval or specific time?
Yes, SQL Server Agent can run jobs based on specific time or interval.
Creating SQL Server Job
SSMS -> SQL Server Agent -> Right-Click -> New Job -> Select Name, Database, Code and Schedule
When you finish you can click Script button and get script that create job (if needed).
You can also start Job using T-SQL (for example from application/another stored procedure or trigger):
EXEC msdb.dbo.sp_start_job N'JobName';
I have a WCF service and client developed using C# which handles data transfer from SQL server database on the server to the SQL server database on the client end. I am facing some issues with the current architecture and planning to modify it to an idea I have, and would like to know if it is possible to achieve it, or how best can I modify the architecture to suite my needs.
The Server side database server is SQL 2008 R2 SP1 and client side servers are SQL 2000
Before I state the idea, below is the overview and current shortcomings of the architecture design I am using.
Overview:
Client requests for a table’s data.
WCF service queries the Server database for all pending data for the requested table. This data is loaded into a dataset.
WCF Compresses the Dataset using GZIP compression and converts it to byte for the client to download.
Client receives the Byte stream, un-compresses it and replicates the data from the Dataset to the physical table on the client database. This data is inserted row by row since in need the Primary key column filed to be returned to the server so that it can be flagged of as transferred.
Once the client has finished replicating the data, it uploads the successful rows Primary key fields back to the server, and in turn the server update each field one by one.
The above procedure uses a basic http binding, with streamed transfer mode.
Shortcomings:
This works great for little data, but when it comes to bulk data, maintaining the dataset in memory as the download is ongoing and also at the client side as replication is ongoing, is becoming impossible as sometimes the dataset size goes up to 4gb. The server can hold this much data since it’s a 32gb RAM server, but at the client side I get System out of memory exception since the client machine has 2gb RAM.
There are numerous deadlocks as the select query is running and also when updating since I am using transaction mode as read committed.
For bulk data it is very slow and completely hangs the client machine when the DTS is ongoing.
Idea in mind:
Maintain the same service and logic of row by row transfer since I cannot change this due to sensitivity of the data, but rather than downloading bulk data I plan to use the sample given in http://code.msdn.microsoft.com/Custom-WCF-Streaming-436861e6.
Thus the new flow will be as:
Upon receiving the download request, the server will open a connection to the DB using snapshot isolation as the transaction level.
Build the row by row object on the server and send it to the client on the requested channel, as the client receives each row object, it gets processed and a success or failure response is sent back to the server on the same method same channel, as I need to update the data on the same snapshot transaction.
This way I will reduce bulk objects in memory, and rely on SQL for the snapshot data that will be maintained in temdb once the transaction is initiated.
Challenge:
How can I send the row object and wait for a confirmation before sending the next one, as the update to the server row has to occur on the same snapshot transaction. Since if I create another method on the service to perform the flagging off, the snapshots will be different and this will cause issues in the integrity of the data in case the data undergoes changes after the snapshot transaction was initiated.
If this is the wrong approach, then please suggest a better one, as I am open to any suggestions.
If my understanding of the snapshot isolation is wrong, then please correct me as I am new to this.
Update 1:
I would like to achieve something like this when the client is the one requesting:
//Client Invokes this method on the server
public Stream GetData(string sTableName)
{
//Open the Snapshot Transaction on the Server
SqlDataReader rdr = operations.InitSnapshotTrans("Select * from " + sTableName + " Where Isnull(ColToCheck,'N') <> 'Y'");
//Check if there are rows available
if(rdr.HasRows)
{
while rdr.read()
{
SendObj sendobj = Logic.CreateObejct(rdr);
//Here is where i am stuck
//At this point I want to write the object to the Stream
...Write sendobj to Stream
//Once the client is done processing it reverts with a true for success or false for failuer.
if (returnObj == true)
{
operations.updateMovedRecord(rdr);
}
}
}
}
For the server sending i have written the code as Such (I used Pub Sub Model for this):
public void ServerData(string sServerText)
{
List<SubList> subscribers = Filter.GetClients();
if (subscribers == null) return;
Type type = typeof(ITransfer);
MethodInfo publishMethodInfo = type.GetMethod("ServerData");
foreach (SubList subscriber in subscribers)
{
try
{
//Open the Snapshot Transaction on the Server
SqlDataReader rdr = operations.InitSnapshotTrans("Select * from " + sTableName + " Where Isnull(ColToCheck,'N') <> 'Y'");
//Check if there are rows available
if(rdr.HasRows)
{
while rdr.read()
{
SendObj sendobj = Logic.CreateObejct(rdr);
bool rtnVal = Convert.ToBoolean(publishMethodInfo.Invoke(subscriber.CallBackId, new object[] { sendobj }));
if (rtnVal == true)
{
operations.updateMovedRecord(rdr);
}
}
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
}
Just off the top of my head, this sounds like it might take longer. That may or may not be a concern.
Given the requirement in challenge 1 (that everything happen in the context of one method call), it sounds like what actually needs to happen is for the server to call a method on the client, sending a record, and then waiting for the client to return confirmation. That way, everything that needs to happen, happens in the context of a single call (server to client). I don't know if that's feasible in your situation.
Another option might be to use some kind of double-queue system (perhaps with MSMQ?) so that the server and client can maintain an ongoing conversation within a single session.
I assume there's a reason why you can't just divide the data to be downloaded into manageable chunks and repeatedly execute the original process on the chunks. That sounds the least ambitious option, but you probably would have done it already if it met all your needs.