I understand how to use entity framework to implement a unit of work and only commit changes after the full unit is performed, but how do i take this a step further? For example, the following needs to all happen under one transaction
CreateUser(...)
{
//1.) New up user object
//2.) Add newly created object to database
//3.) Send Email
//4.) Commit transaction ( ensures email is successfully sent AND object is created in database, else transaction fails
}
I'm not quite sure how to make sure that sending of email and saving of the user within the database all happens within one transaction. Any advice is greatly appreciated
Not all resources are transactional. Email is one of them. So one cannot expect sending email is rolled back when other resources fails in a transaction.
There are some alternative solutions:
1) invoke non-transactional resource at the last step.
Just like you did in the create user example. In this case, all business constraints are checked and pass, the only reason of failure is infrastructure issue which does not frequently happen in real world(with carefully maintainence). Make some compensation when the infrastructure failure does happen. This could be done automatically or manually, depends on how often it happens. For example, if the email is sent, but the user setup is rolled back, you may tell him/her that you're very sorry(this is very import :P ) but there is something wrong with the setup, please try again.
2) apply eventually consistency
Use transactional resource instead to trigger the process. For example, you may use messaging (globally transactional if using 2 phase commit) to notify to send emails.
CreateUser(...)
{
//1.) New up user object
//2.) Add newly created object to database
//3.) Publish user setup event by messaging
//4.) Commit transaction ( ensures message is successfully sent AND object is created in database, else transaction fails
}
In any case, failures and unconsistency occurs more or less. You need to evalute is it acceptable.
Related
If you don't know C# but you are familiar with the actor model please read my problem below as it is more about architecture and data management.
I'm a very junior C# developer and trying to understand what the actor model is. I'm kind of done with it but it's left one point that I cannot get.
Before I tell you a problem let me describe the context in order to provide you with better undersrtanding.
As a test example I want to build an app for an imaginary bank. I'm going to implement this application by using both akka.net and Orleans on learning purpose and to be able to compare them.
Use cases:
As a user I want to be able to create a new account;
As a user I want to be able to login to the app using my account unique number;
As a user I want to be able to deposit money to my account;
As a user I want to be able to select another user and transfer a specified sum of money to their account;
As a user I want to be able to withdraw a sum of money from my account.
So, there are the following entities:
User;
Account.
Identifying one to one relationship between the user and their account.
I'm going to use ORM to store this data in my Database. Obviously the models look something like this:
public class User
{
public Guid Id { get; set; }
public string FullName { get; set; }
....
}
public class Account
{
public Guid Id { get; set; }
public string UniqueNumber { get; set; }
public string Balance { get; set; }
...
}
And I also want to have two actors/grains:
AccountActor;
TransactionService;
Their interfaces:
//Or IAccountGrain
public interface IAccountActor
{
void Deposit(Money amount);
void Withdraw(Money amount);
}
//Or ITransactionGrain
public interface ITransactionActor
{
void Transfer(IAccountActor from, IAccountActor to, Money amount);
}
The thing that I don't understand is how to treat the data in the relation database.
Let's imagine the following scenario:
50 users are online and vigorously making request via the client app to the REST API of the app.
They are withdrawing, depositing and transferring money almost without any pauses.
The question is:
Should I create one actor per user account? I'm pretty sure that I
need because how then I can implement thousands of transactions
between different accounts.
How can I associate the user account with the AccountActor? Is it correct If I load the data from database using repository before
actor's activation/start and set up the state?
And the main problem:
How to save state back to the database?
Let's image an account A that has 1000$.
And It occurs about 100 transactions initiated by the users where this account is involved.
Account A changes its state from message to message.
What is the best approach to save these changes to database? I read that If I use calls to the database directly from the actor, I will lose all the benefits because of blocked operations.
Should I create one more actor to process messages from other actors and writes changes to database using repositories?
I mean I can send from AccountActor messages about the account changes to the new actor where I will call the appropriate repository. But, Isn't it a bottleneck? Let's imagine 1000 users online and about 100 000 transactions between accounts. Then the actor that is responsible for saving accounts changes to the database can have too many messages to process.
Sorry for the long text. I tried to find examples of applications that use Orleans or Akka.net but I haven't found anything what utilizes a database.
Thank you for you attention.
There are a couple ideas you’re missing here, but let’s take the questions in order.
Should I create one actor per user account? I'm pretty sure that I
need because how then I can implement thousands of transactions
between different accounts.
I assume the alternative you’re considering is multiple actors per user account, and this would be wrong. There can be only one actor per user account or else you will run into the problem you describe that simultaneous requests can withdraw the same money twice.
How can I associate the user account with the AccountActor?
You are missing a UserActor which owns the AccountActors. An account cannot exist without an owner, otherwise we don’t know who owns the money in an account. In the real world, one typically does not send money to a random account. They want to send it to a person, and uses the senders User persona account to do so.
Is it correct If I load the data from database using repository before
actor's activation/start and set up the state?
Yes, in fact that is mandatory. Without the state in the actor, the actor is not good for much.
What is the best approach to save these changes to database? I read
that If I use calls to the database directly from the actor, I will
lose all the benefits because of blocked operations. Should I create
one more actor to process messages from other actors and writes
changes to database using repositories?
You’re on the right track but not quite there yet. Saves of actor state are done with asynchronous methods to write to the DB. With an async method, the main thread does not block waiting for the DB write to happen, thus the processing thread can continue with its business.
When only one actor is involved in an action, it can save its own state via the async method. In banking registers there are always 2 accounts involved and writes to both must succeed or fail, never one succeed and one fail. Therefore, the TransactionActor will open a DB transaction and tell each AccountActor to save its state using that DB transaction. If either fails, it aborts the transaction and both fail. Note that this method is a private async method on the TransactionActor so you get the benefits of parallel processing.
BTW, you can't find any examples of writing to the DB in Orleans because that is all handled for you by the framework. The save methods are automatically async and they interact with the DB. All you do in Orleans is reference the Actor and the state is automatically pulled from the DB for you.
This question already has answers here:
How to monitor SQL Server table changes by using c#?
(11 answers)
Closed 6 years ago.
I want to get notified when a certain change occurs in Database table. Consider the case: I want to perform a certain action when the column in a row changes its value to 5. How can I achieve it. I am using C# and entity framework to access the database.
For this you have to make a schedule job which will continuously(like interval of 5 minutes) ping database and notify you as like Facebook's notification bar.
Also you can write trigger on that table which will insert/update notification table and from there you will get notify.
The short answer is that you should probably try and manage this outside of SQL server. I have to assume that you have some application logic executing outside of SQL server that is the source of the update. Ideally your notification logic should be placed in your application tier before or after the database is updated.
Should you not be able to achieve this, three other options I can offer are:
polling You build a service that reads the value from SQL server in a loop. The loop should read the value periodically, and perform the notification. Most engineers avoid polling as from a best practices standpoint it is typically contra indicated due to adding persistent load to the database. Although polling should be avoided, it's surprisingly common in the field.
msmq You update the value via a stored procedure, and use this article to send a message to MSMQ when the value is 5. You will need to write a service to consume the MSMQ message and process the notification. You may use a WCF service using MSMQ transport to make this easy.
email You send an email using sp_send_dbmail in the update stored procedure, and build the necessary notification consumer(s). It should be noted that this method will likely also involve polling if you consume the email electronically. You can avoid that by using IMAP IDLE to process the email notifications. Try MailKit
Reporting services also apparently offers notifications, but I am not familiar with them.
using(var context = new FooEntities)
{
try
{
var customer = context.Customers.First(i=> i.CustomerID = 23);
customer.Name = "Bar";
context.SaveChanges();
//Write your notification code here
}
catch(Exception ex)
{
//Write notification along with the error you want to display.
}
}
Search in google there's many different way of displaying a notification.
I have a question concerning redis in a distributed architecture.
Assume I have n clients, either windows desktop applications or asp.net web/web api servers.
One of the clients, lets say client A, hits the cache for a data and has a miss (the data is not in the cache). The client then starts to get the real data (from lets say a database) and then sets it in the cache when it's done.
Client B comes along and wants the same data, does a fetch to the cache and since it's a miss, does the same processing.
Is there a way for Client B to ...(N) not to do the processing (i.e go to the database) until the data is in the cache and fetch the data from the cache instead when it's available?
I understand that on a single app (or web server), using threads it's easy to check that, but in a distributed architecture?
Is this also a correct way of thinking as well? for the wait process that is
If so then could Client A put a flag somewhere stating that he's loading Data X and that all other clients should wait until he's done?
Otherwise, the idea then would be something along the lines of :
Client A requests Data X
Miss in cache
Processes Data X
Looks if Data X is now in cache
If not, add Data X to cache, otherwise, use it and don't store it in cache
Thanks!
As Kevin said, it's called cache stampede.
One of the best documents to do with this problem I have read is Using memcached: How to scale your website easily (comes from Josef Finsel):
What we need in this instance is some way to tell our program that
another program is working on fetching the data. The best way to
handle that is by using another memcached entry as a lock.
When our program queries memcached and fails to find data, the first
thing it attempts to do is to write a value to a specific key. In our
example where we are using the actual SQL request for the key
name we can just append ":lock" to the SQL to create our new key.
What we do next depends on whether the client supports returning
success messages on memcached storage commands. If it does,
then we attempt to ADD the value. If we are the first one to attempt
this then we’ll get a success message back. If the value exists then
we get a failure indication and we know that another process is trying
to update the data and we wait for some predetermined time
before we try to get the data again.
When the process that’s updating
the cache is done, it deletes the lock key.
IDE: Visual Studio, C# .net 4.0, Winforms application
Is there any way in SQL Server to implement a column in a table which can set itself automatically to 0 when it has not received any ACK (acknowledgement) signal from the application side?
The requirement is I want to keep a column which can keep track that Is application is open or it has been closed?
One way to implement is using the OnClose() event, i.e. on Close() I can change it's value to 0. but the problem is suppose application got hanged for some reason or Power is gone than the value in database will not be updated to zero.
So I want to create an automated column which can track this situation and make itself zero when the application is not sending any request or idle means closed.
please suggest how to handle this.
You can't do that. The only thing you can do is to save GETDATE() in a column in a table as the last activity time of the application and invoke the stored procedure from a high-priority thread every 10 seconds for example.
When you want to know if the application is alive or not, just check this value, if more than 10 seconds is passed since then, you app is gone.
UPDATE:
A more precise but complex approach would be to listen on a socket inside your application and then whenever you want to know if the application is alive, send a request from your sql script to PING the application. You should use CLR programming for this approach, but I think the first one will be practically enough.
Considering it will be a multiple instance scenario where multiple instances of the application can point to same database. You can try the following:
Create a separate table to maintain sessions. This table would contain three columns 1)Machine name or IP 2) Session Id (GUID) and 3) TimeStamp.
Whenever application starts create a new session id and make an entry into this table it means new session is started.
Keep on updating timestamp on every request based on session id.
Have a timeout configured somewhere in web.config or database which will come in handy later.
Now when application is exiting gracefully then delete the row from the table.
Now if application crashes the row won't be delete so next time when application starts you can check if the row already exists. Use machine name or IP to query to the table.
Use the timeout setting configured in (3) above to determine since how long the application is idle.
I am using a web application and a windows service on the same machine as Appfabric.
Both applications reuse same DAL code (dll) which is EF (Entity Framework) Code-First based and accessing the same cache in Appfabric. The code in the windows service is implemented as a Job as part of Quartz.Net
The web application has to support multiple requests off course, and the windows service multiple threads( scheduler and events).
For both, the shared DAL dll creates a DbContext object per http session and thread ContextID or just Thread ContextID for the later. The DAL uses the EFCachingProviders from here. Also, my EF solution uses Optimistic concurrency with a timestamp columns and IsRowVersion in the mapping.
As stated here, the benefit of having a 2nd level cache is to have access to a representation of the original state across processes! But that does not seem to work for me, I get 'OptimisticConcurrencyException' in my use case as following:
restart cache cluster, restart windows service, restart iis -> clean slate :)
Using web app (firefox), I insert a new object A with reference to existing object B. I can see the new row in the database. All ok.
Using webapp in another browser (chrome) = new session, i can see the new object.
Next, the windows service tries to do some background processing and tries to update object B. This results in an 'OptimisticConcurrencyException'. Apparently the process in the windows service is holding a version of Object B with a dated rowversion.
If i restart the windows service, it tries the same logic again and works with no exception....
So both applications are multithreaded, use same DAL code, connect to same database, and same cache cluster and same cache. I would expect the update and insert to be in the appfabric cache. I would expect the EF context of the windows service to use the newest information. Somehow, it seems, that it's 1st level cache in holding on old information...
or something else is going wrong.
Please advice...
Update
Ok, after digging around, i fixed the Update problem of my windows service. Each Manager object with queries the DAL uses a DbContext bound to its Process ID + Thread ID. So in the Execute function of my Quartz Job, all Managers (of different object types) should share the same DbContext which is created by the first Manager.
The problem was, that after the function finished, the DbContext was not Disposed (which happens automatically in the HTTP Session based DbContext manager). So the next time the Job was executed, the same DbContext was found and used, which by that time was dated already (old first level cache???). The 2nd level cache should not be a problem, because that is shared and SHOULD contain newest objects... if any.
So this part is fixed.
New problem
So the web-app creates a new object A, updates an existing object B, the windows-service now works and is able to update the existing (changed) object B with no problem.
Problem:
When i do a refresh of the webapp, it does not see the changes (by the windows service) of object B....
So if the webapp changed a count to 5, 10 minutes later the windows service change that count to 6 and I open the web-app in same or new window/browser, i still see 5, not 6!
A restart of the webapp (iis) does not help, also an iisreset doesn't.
When i do Restart-CacheCluster.... it works and shows 6....
So it looks like the item is in the cache. The windows service updates it, but does not invalidate the item, which is old and used by the webapp....
Or... although the same object, the webapp has its own entry in the cache and the win-app has its own entry (which does get invalidated)....
Which one?
Solution
I solved this myself. The EF wrapper uses the query string as a key to store items in the cache, it seems. So 2 different queries (does not matter if they originate from 2 different application sharing same distributed cache or same application) referencing the same data in the database will have different keys (different query string) and so different places in the cache. Perhaps its not this black-and-white but something like this...
I don't think internally some way of algorithm is used to check if a query touches existing cached objects.
This causes my problem where my windows service does an update and the webapp still sees the old one from the cache which could only be solved by doing a Restart-CacheCluster command.
So how i fixed this:
My windows Service is a batch job triggered by the Quartz Scheduler. After it is done
I clear the whole cache:
private void InvalidateCache()
{
try
{
DataCache myCache = ...
foreach (String region in myCache.GetSystemRegions())
{
myCache.ClearRegion(region);
}
}
catch (Exception ex)
{
eventLog.WriteEntry("InvalidateCache exception : " + ex.Message);
}
}
I don't have an answer, but I hope the thoughts below might point you into the right direction.
If this is only an issue on updates, I would go for reading a fresh instance of the record on every update from the database, and update that. This would avoid optimistic concurrency errors. Note that the DbContext is not thread safe - I don't know if this would cause the issue, but reading every time new would address it.
If you are having this issue on reads, then you would have to track down where the various caches are and which one is not getting updated and why. I am guessing there are various configuration options for caching at each point of usage. Good luck with that.... :)