I'm considering implementing sql database caching using the following scheme:
In the ASP.NET webapplication I want a continuously running thread that check's a table, say dbStatus, to see if field dbDirty has been set true. If so, the local in-memory cache is updated, querying a view in which all needed tables are present.
When any of the tables in the view is updated, a trigger on that table is fired setting dbStatus.dbDirty true. So this would mean I have to add a on insert,update,delete trigger on those tables
One of the reasons I want to implement such a caching scheme is that the same database is used in a Winform version of this application.
My question: is this a viable approach?
Many thanks in advance for helping me with this one, Paul
This is a viable approach.
The main problem you need to be aware of is that ASP.NET worker processes can exit at any time for many reasons (deployment, recycle, reboot, bluescreen, bug, ...). This means that your code must tolerate being aborted (in fact just disappearing) at any time.
Also, consider that your app can run two times at the same time during worker recycling and if you run multiple servers for HA.
Also, cross-request state in a web app requires you to correctly synchronize your actions. This sounds like you might need to solve some race conditions.
Besides that this approach works.
Consider incrementing a version number instead of a boolean. That makes it easier to avoid synchronization issues such as lost updates because there is no need to reset the flag. There is only one writer. That's easier than multiple writers.
Related
We have a process that needs to run every so soften against a DB used by a web app, and we need to prevent all other updates during this process execution. Is there any global way to do this maybe thru nHibernate, .NET or maybe directly in Oracle?
The original idea was to have a one-record DB table to indicate if the process is running or not, but with this we will need to go back to every single save/update method and make changes to verify if this record exist or not prior to the save/update call.
My reaction to that kind of requirement is to review the design as it is highly unusual outside of doing application upgrades. Other than that there are a couple option:
Shutdown the DB, open it in exclusive mode, make changes, and then open it up for everyone.
Attempt to lock all the required tables with LOCK TABLE. That might generate deadlock exceptions depending on the order of doing the locks.
I am building a ASP.NET MVC 4 app using Entity Framework where multiple threads can access a table at the same time (add, delete row, etc.) Right now I am doing using (UserDBContext db = new UserDBContext()) within each controller (so a new DBContext is created for each request since MVC framework creates a seperate thread for each request). From what i read, this is safe; however, I am curious about:
What happens when two threads access the same table, but not the same row? Are both threads allowed to access simultaneously?
What happens when two threads modify the same row? say, one tries to read while the other tries to delete? Is one thread blocked (put to sleep), then gets waken up automatically when the other is done?
Thanks!
1: Locking in the database. Guaranteeing multi user scenarios is one of the top priority of databases. Learn the basics. There are good books.
2: Locking. Again. One will have to wait.
This is extremely fundamental so I would suggest you take 2 steps back and get something like "SQL for dummies" and learn about ACID conditions that any decent database guarantees. NOTHING in here has to do with EF by the way. This is all done on the database internal level.
I am using LINQ to SQL in order to read/write a database on a server. Right now only the creator of an db-entry can change the according information.
Is it guaranteed that thereby no concurrency issues should appear? I mean it is not possible that one entry is changed simultaniously form two different locations. Or are there also read-concurrent problems that could appear?
Thanks in advance
No; that won't help at all.
A single user can still send multiple concurrent requests from different tabs or browsers or machines.
Excluding the obvious unlikely case of a "malicious" user using mutiple windows I wonder if strange scenarios could happen:
The user sends its first request which is executed on a thread by the web server but for whatever reason this thread is preempted.
Then later the same user emits another request that is in conflict with the first one, but this request is executed on a thread which is not preempted and directly writes a new value to the DB.
The first thread is then awakened and ends its works writing the old value
So when you have any doubt put some safeguards because we're never too careful, especially when programming ;)
In an ideal world yes. But there are several things to consider.
Can the same user submit changes from multiple locations?
If the TCP/IP connection is broken and reestablished will overlapping/out of order requests be an issue?
Is there any possiblity of needing moderator or admin access to user data?
I need a method to run every so often that does some database processing. However, I may need it to be triggerable by an admin on the site. But I don't want this method being run more than once at the same time, as this could cause issues with the way it hits the database.
For example, could I...
Create a singleton class that runs the method on a timer, and instantiate it in the global.asax file. Then, since it's a singleton, I can call it from my normal .aspx pages and call the method whenever I want. I would probably need to use that "lock" feature of C# to check to see if the method is already running.
I heard some talk lately that Singletons are "evil", but this seems like the perfect fit for it. What do you think? Thanks in advance.
Timers and locks (that are intended to synchronize access to the database) are a bad idea on the web; you may have zero, one or many app-pools on different servers. They may recycle at any time, and won't be spun up until needed. Basically, this won't prevent you hammering the db from multiple sources.
Personally, I'd be tempted to either write a service to do this work (either db-polling, or via WCF etc), or use the db (a SP or similar) - set a flag in a table-row to say "in progress", do the work at the db, and clear the flag (duplicate attempts exit immediately while in progress).
I would do it this way
Build a normal ASP.NET page which does the processing
StealBorrow LFSR Consultings idea for a flag in the DB which does the work of checking if the process is currently running
Use normal cronjob or windows task scheduler to call the web page on a regular basis.
And Singletons aren't evil they just get abused easily.
Another option which Joel Spolsky mentioned in one of the SO Podcasts, i believe it was #20 something. Is to set an empty Cache object on application start with a certain expiration date, and in the CacheItemRemovedCallback make a call out to page or do some work and then reset the empty cache object.
I'm probably horribly mis-quoting him, so I recommend you listen or look through the transcripts for yourself.
What about just setting up a flag in the database and checking that to determine if the job is running or not? Seems simpler IMO.
The canonical way to write a singleton ends up not being thread safe. Especially in a webby environment, where threads needn't even be on the same machine!
If you really want to do a "singleton", think of it as a service that you only ever deploy to one machine. Then use the transactional semantics of your database like Marc Gravell suggests to synchronize the locks.
We've done similar things by using a Web Service to do the backend processing, then writing a Desktop App to call it on whatever schedule we need. We can then run that app on a server, or an admin can run it directly from their PC to trigger the job.
Edit: After I saw your revision that you don't want them to run simulatenously, we have usually just controlled that with a database flag like a few others have said, nothing fancy but it gets the job done
Set an Application wide variable to denote that the process is running. That should be a little easier than storing the variable in the database, right?
At the moment I am working on a project admin application in C# 3.5 on ASP.net. In order to reduce hits to the database, I'm caching a lot of information using static variables. For example, a list of users is kept in memory in a static class. The class reads in all the information from the database on startup, and will update the database whenever changes are made, but it never needs to read from the datebase.
The class pings other webservers (if they exist) with updated information at the same time as a write to the database. The pinging mechanism is a Windows service to which the cache object registers using a random available port. It is used for other things as well.
The amount of data isn't all that great. At the moment I'm using it just to cache the users (password hashes, permissions, name, email etc.) It just saves a pile of calls being made to the database.
I was wondering if there are any pitfalls to this method and/or if there are better ways to cache the data?
A pitfall: A static field is scoped per app domain, and increased load will make the server generate more app domains in the pool. This is not necessarily a problem if you only read from the statics, but you will get duplicate data in memory, and you will get a hit every time an app domain is created or recycled.
Better to use the Cache object - it's intended for things like this.
Edit: Turns out I was wrong about AppDomains (as pointed out in comments) - more instances of the Application will be generated under load, but they will all run in the same AppDomain. (But you should still use the Cache object!)
As long as you can expect that the cache will never grow to a size greater than the amount of available memory, it's fine. Also, be sure that there will only be one instance of this application per database, or the caches in the different instances of the app could "fall out of sync."
Where I work, we have a homegrown O/RM, and we do something similar to what you're doing with certain tables which are not expected to grow or change much. So, what you're doing is not unprecedented, and in fact in our system, is tried and true.
Another Pitfall you must consider is thread safety. All of your application requests are running in the same AppDomain but may come on different threads. Accessing a static variable must account for it being accessed from multiple threads. Probably a bit more overhead than you are looking for. Cache object is better for this purpose.
Hmmm... The "classic" method would be the application cache, but provided you never update the static variables, or understand the locking issues if you do, and you understand that they can disappear at anytime with an appdomain restart then I don't really see the harm in using a static.
I suggest you look into ways of having a distributed cache for your app. You can take a look at NCache or indeXus.Net
The reason I suggested that is because you rolled your own ad-hoc way of updating information that you're caching. Static variables/references are fine but they don't update/refresh (so you'll have to handle aging on your own) and you seem to have a distributed setup.