I am developing the MVC web application.
Which means I am creating views, models, view models. I use linq-to-sql as a database provider, I have implemented custom Unit-of-Work pattern, and also the Data Source, Repositories and Service patterns which looks perfectly (in my implementation) and completely separated from the direct SQL code. Actually, from any database, which is designed for unit testing, so I could test my application with in-memory data sources with no affect on the database.
And eventually I am stuck with one problem: I have no protection agains cross thread (or cross process, because as far as I know, IIS can create more than one app domain with a single web app) operations.
For example, I have a table with some records. And every now and again a web request happens which (inside controller and then service and then repository) picks the SQL table' row on the maximum of let's say TicketId column and then inserts in that table another row with (that column value + 1).
In the case two or more threads or processes do the same thing, the duplicated values can appear in the database. Some time ago, when my webapp was somekind smaller, I used the direct SQL code and simple UPDLOCK in SELECT statements which inside TransactionScope using block was locking the record I'm modifying (or anything else) preventing all other database clients to wait until I finish.
With all these patterns I forgot one thing:
How do I actually implement database multi-access protection issue?
Without any direct SQL code.
How do I actually implement database multi-access protection issue?
It's the database engine's job to do that. It's your job to ensure your app fails gracefully should there be any issues reported back. See Locking in the Database Engine.
For example, I have a table with some records. And every now and again a web request happens which
(inside controller and then service and then repository) picks the SQL table' row on the maximum > of let's say TicketId column and then inserts in that table another row with (that column value + 1).
I get the impression here that you don't seem to have much faith in your database considering you are trying to replicate it's behaviour. Set your field to be auto-increment and that should solve your issue. If you do have to implement your own manual auto-increment then you need to use some form of locking because what you essentially have is a race condition e.g.
private static object lockObject = new Object();
lock(lockObject)
{
... DB stuff
}
For cross-process locking, you would need to look at possibly a named Mutex.
Why not use a single table to represent a ticket number and the use transaction with Seriliazable transaction isolation level?
int GetTicketNumber(bool shouldIncrement)
{
if(shouldIncrement)
{
// use transaction with serilizable isolation level
select and update
}
else
{
//just select
}
}
Related
I am trying to find working and optimal solution for the following situation:
I have WebService in the cloud written on c#;
Endpoint of this service can be called in a random time with random frequency;
During handling of request we call method where we have read \ write operations on the table in our DB (I have simplified flow):
we read data from table
apply some validation rules
and according to specific rules update some fields and write data back to the table
commit transaction
The real problem is when we have 2 or 3 simultaneous request for one user - we easily get to the situation when we read invalid data from the table, because previous operation didn't save results (commit transaction) yet.
I've tried to setup isolation level, but it is reason for deadlock.
Usage of Upsert operation on DB level requires moving logic to stored procedure - it is not desirable.
So the question: what is the best way to synchronize work with DB correctly? Thank you for any suggestions.
Tools:
Entity Framework 6.0, Azure SQL, Azure
This might seem like an odd question, but it's been bugging me for a while now. Given that i'm not a hugely experienced programmer, and i'm the sole application/c# developer in the company, I felt the need to sanity check this with you guys.
We have created an application that handles shipping information internally within our company, this application works with a central DB at our IT office.
We've recently switch DB from mysql to mssql and during the transition we decided to forgo the webservices previously used and connect directly to the DB using Application Role, for added security we only allow access to Store Procedures and all CRUD operations are handle via these.
However we currently have stored procedures for updating every field in one of our objects, which is quite a few stored procedures, and as such quite a bit of work on the client for the DataRepository (needing separate code to call the procedure and pass the right params for each procedure).
So i'm thinking, would it be better to simply update the entire object (in this case, an object represents a table, for example shipments) given that a lot of that data would be change one field at a time after initial insert, and that we are trying to keep the network usage down, as some of the clients will run with limited internet.
Whats the standard practice for this kind of thing? or is there a method that I've overlooked?
I would say that updating all the columns for the entire row is a much more common practice.
If you have a proc for each field, and you change multiple fields in one update, you will have to wrap all the stored procedure calls into a single transaction to avoid the database getting into an inconsistent state. You also have to detect which field changed (which means you need to compare the old row to the new row).
Look into using an Object Relational Mapper (ORM) like Entity Framework for these kinds of operations. You will find that there is not general consensus on whether ORMs are a great solution for all data access needs, but it's hard to argue that they solve the problem of CRUD pretty comprehensively.
Connecting directly to the DB over the internet isn't something I'd switch to in a hurry.
"we decided to forgo the webservices previously used and connect directly to the DB"
What made you decide this?
If you are intent on this model, then a single SPROC to update an entire row would be advantageous over one per column. I have a similar application which uses SPROCs in this way, however the data from the client comes in via XML, then a middleware application on our server end deals with updating the DB.
The standard practice is not to connect to DB over the internet.
Even for small app, this should be the overall model:
Client app -> over internet -> server-side app (WCF WebService) -> LAN/localhost -> SQL
DB
Benefits:
your client app would not even know that you have switched DB implementations.
It would not know anything about DB security, etc.
you, as a programmer, would not be thinking in terms of "rows" and "columns" on client side. Those would be objects and fields.
you would be able to use different protocols: send only single field updates between client app and server app, but update entire rows between server app and DB.
Now, given your situation, updating entire row (the entire object) is definitely more of a standard practice than updating a single column.
It's better to only update what you change if you know what you change (if using an ORM like entity Framework for example), but if you're going down the stored proc route then yes definately update everything in a row at once that's way granular enough.
You should take the switch as an oportunity to change over to LINQ to entities however if you're already in a big change and ditch stored procedures in the process whenever possible
I have ERP database "A" has only read permission, where i cant create trigger on the table.
A is made for ERP system (Unknown Program for me ). I have another Database "B" that is private to my application this application work on both databases. i want to reflect A's changes(for any insert/Update/Delete) instantly to B.
Is there any Functionality in c# that can work exactly as trigger works in database???
You have few solutions, best one depends on which kind of database you have to support.
Generic solution, changes in A database aren't allowed
If you can't change master database and this must work with every kind of database then you have only one option: polling.
You shouldn't check too often (so forget to do it more or less instantly) to save network traffic and it's better to do in in different ways for insert/update/delete. What you can do depends on how database is structured, for example:
Insert: to catch an insert you may simply check for highest row ID (assuming what you need to monitor has an integer column used as key).
Update: for updates you may check a timestamp column (if it's present).
Delete: this may be more tricky to detect, a first check would be count number of rows, if it's changed and no insert occured then you detected a delete else just subtract the number of inserts.
Generic solution, changes in A database are allowed
If you can change the original database you can decrease network traffic (and complexity) using triggers on database side, when a trigger is fired just put a record in an internal log table (just few columns: one for the change type, one for affected table, one for affected record).
You will need to poll only on this table (using a simple query to check if number of rows increased). Because action (insert/update/delete) is stored in the table you just need to switch on that column to execute proper action.
This has a big disadvantage (in my point of view): it puts logic related to your application inside the master database. This may be terrible or not but it depends on many many factors.
SQL Server/Vendor specific
If you're application is tied to Microsoft SQL Server you can use SqlDependency class to track changes made. It works for SS only but I think there may be implementations for other databases. Disadvantage is that this will always bee specific to a specific vendor (so if A database will change host...you'll have to change your code too).
From MSDN:
SqlDependency was designed to be used in ASP.NET or middle-tier services where there is a relatively small number of servers having dependencies active against the database. It was not designed for use in client applications, where hundreds or thousands of client computers would have SqlDependency objects set up for a single database server.
Anyway if you're using SQL Server you have other options, just follow links in MSDN documentation.
Addendum: if you need a more fine control you may check TraceServer and Object:Altered (and friends) classes. This is even more tied to Microsoft SQL Server but it should be usable on a more wide context (and you may keep your applications unaware of these things).
You may find useful, depending on your DBMS:
Change Data Capture (MS SQL)
http://msdn.microsoft.com/en-us/library/bb522489%28v=SQL.100%29.aspx
Database Change Notification (Oracle)
http://docs.oracle.com/cd/B19306_01/appdev.102/b14251/adfns_dcn.htm
http://www.oracle.com/technetwork/issue-archive/2006/06-mar/o26odpnet-093584.html
Unfortunately, there's no SQL92 solution on data change notification
Yes There is excellent post are here please check this out..
http://devzone.advantagedatabase.com/dz/webhelp/advantage9.1/mergedprojects/devguide/part1point5/creating_triggers_in_c_with_visual_studio_net.htm
If this post solve your question then mark as answered..
Thanks
What I want is pretty simple conceptually but I can't figure out how it would be best to implement such a thing.
In my web application I have services which access repositories which access EF which interacts with the SQL Server database. All of these are instanced once per web request.
I want to have an extra layer between the repositories and EF (or the services and the repositories?) which statically keeps track of objects being pulled from and pushed to the database.
The goal, assuming DB-access is only accomplished through the application, would be that we know for a fact that unless some repository access EF and commits a change, the object set didn't really change.
An example would be:
Repository invokes method GetAllCarrots();
GetAllCarrots() performs a query on SQL Server retrieving a List<Carrot>, if nothing else happens in between, I would like to prevent this query from being actually made on the SQL Server each time (regardless of it being on a different web request, I want to be able to handle that scenario)
Now, if a call to BuyCarrot() adds a Carrot to the table, then I want that to invalidate the static cache for Carrots, which would make GetAllCarrots(); require a query to the database once again.
What are some good resources on database caching?
You can use LinqToCache for this.
It allows you to use the following code inside your repository:
var queryTags = from t in ctx.Tags select t;
var tags = queryTags.AsCached("Tags");
foreach (Tag t in tags)
{
...
}
The idea is that you use SqlDependency to be notified when the result of a query changes. As long as the result doesn't change you can cache it.
LinqToCache keeps track of your queries and returns the cached data when queried. When a notification is received from SqlServer the cache is reset.
I recommend you reading the http://rusanu.com/2010/08/04/sqldependency-based-caching-of-linq-queries/ .
I had a similar challenge, and due to EF's use and restrictions, i've decided to implement the cache as an additional service between the client and server's service, using an IoC. Monitoring all service methods that could affect the cached data.
Off course is not a perfect solution when you have a farm of servers running the services, if the goal is to support multiple servers i would implement using the SqlDependency.
In the project my team is working on, there is a windows service which iterates through all the entities in a certain table, and updates some of their fields based on some rules we defined. We use NHibernate as our ORM tool. Currently, we open one session and one transaction for the entire proccess, which means the transaction is commited after all the entities have been proccessed. I think this approach isn't good, and I wanted to hear some more opinios:
Should we keep our current way of managing the session, Or should move to a different approach?
One option I thought about is opening a transaction per entity, and another suggestion was to open a new session for each entity.
What approach you think will work best?
There isn't a single way to do it; it all depends on the specific cases.
In the app I'm working on, I have examples of the three approaches, and there's a reason for choosing each one. For example:
The whole process must have transactional atomicity: use a single transaction
The process has a lot of common data, but each record in the "master" table can be considered a unit of work: use a single session, multiple transactions
Processing each record in the master table should be independent from the others (including error handling): use a session per record