What I want is pretty simple conceptually but I can't figure out how it would be best to implement such a thing.
In my web application I have services which access repositories which access EF which interacts with the SQL Server database. All of these are instanced once per web request.
I want to have an extra layer between the repositories and EF (or the services and the repositories?) which statically keeps track of objects being pulled from and pushed to the database.
The goal, assuming DB-access is only accomplished through the application, would be that we know for a fact that unless some repository access EF and commits a change, the object set didn't really change.
An example would be:
Repository invokes method GetAllCarrots();
GetAllCarrots() performs a query on SQL Server retrieving a List<Carrot>, if nothing else happens in between, I would like to prevent this query from being actually made on the SQL Server each time (regardless of it being on a different web request, I want to be able to handle that scenario)
Now, if a call to BuyCarrot() adds a Carrot to the table, then I want that to invalidate the static cache for Carrots, which would make GetAllCarrots(); require a query to the database once again.
What are some good resources on database caching?
You can use LinqToCache for this.
It allows you to use the following code inside your repository:
var queryTags = from t in ctx.Tags select t;
var tags = queryTags.AsCached("Tags");
foreach (Tag t in tags)
{
...
}
The idea is that you use SqlDependency to be notified when the result of a query changes. As long as the result doesn't change you can cache it.
LinqToCache keeps track of your queries and returns the cached data when queried. When a notification is received from SqlServer the cache is reset.
I recommend you reading the http://rusanu.com/2010/08/04/sqldependency-based-caching-of-linq-queries/ .
I had a similar challenge, and due to EF's use and restrictions, i've decided to implement the cache as an additional service between the client and server's service, using an IoC. Monitoring all service methods that could affect the cached data.
Off course is not a perfect solution when you have a farm of servers running the services, if the goal is to support multiple servers i would implement using the SqlDependency.
Related
Newbie to the whole ASP.Net conundrum !!
So I have a set of Web API's (Actions) in a controller which will be called in succession to each other from the client. All the API's depend on a data model object fetched from the database. Right now I have a DAO layer which fetches this data and transparently caches it. So there is no immediate issue as there is no round trip to database for each API call. But the DAO layer is maintained by a different team and makes no guarantee that the cache will continue to exist or it's behaviour won't change.
Now the properties or attributes of the model object would change but not too often. So If I refer the API calls made in succession from a client as a bundle, then I can safely assume that the bundle can actually query this data once and use it without having to worry about change in value. How can I achieve this ? Is there a design pattern somewhere in ASP.Net world which I can use ? What I would like is to fetch this value at a periodic interval and refresh it in case one of the API calls failed indicating the underlying values have changed.
There are a few techniques that might be used. First of all, is there a reason for the need of a second cache, because your Data Access Layer already has it, right?
You can place a cache on the Web API response level by using a third party library called Strathweb.CacheOutput, and:
CacheOutput will take care of server side caching and set the appropriate client side (response) headers for you.
You can also cache the data from your data access layer by using a more manual approach by using the MemoryCache from System.Runtime.Caching.
Depending on what's the infrastructure available, distributed caches like Cassandra or Redis may be the best choice.
I am working on an application where users in a local network are notified when a "record" from the database changes. So if person "A" changes Record 1 and person "B" has Record 1 in their view, person "B" will be notified that Record 1 has changed.
We are currently using what is called a "duplex" service to send these notifications through a WCF service. The service is rather slow and ideally we would like to remove it. Are there any ways to keep data in real-time (or close to it) for users, when another user has updated the database?
If you are just looking to replace your notification service my suggestion would be to use SignalR to keep your clients synchronized with your DB.
I had a similar problem once and ended up crossing over to a NoSql cloud solution called Firebase with the Firesharp library. Firebase - Features, Firesharp.
It is blindingly fast and actually really easy to use. So if your application is still quite small and the business risk of migrating to a NoSql setup is small it might be a worthwhile solution.
How long they keep the record in view?
What is the mechanism that is checking if the value changed?
what kind of client application are you using?
You could create an async method that is validating the status of the item the user is using against a nodejs server that will be querying the db, for that particular record, reply with the value and the async method can validate if the current value is different from the one from the DB and then it can inform the user or update the record.
Those have small footprint and should run more efficiently.
I am looking at options for caching data at service layer off my web application (server layer gets data from other systems and at Web Front End I dont want to go on round trip for that data each time - I would like to cache it for say 20 mins and if it is not null load it from cache if not go and retrieve it
I have looked at Dynacache which basically looks as if it should do exactly what I want but I have been having problems getting it working with SimpleInjector my DI Framework. Has anyone used a similar NuGet package or got an example of doing similar?
I typically set up my web service layer with as little caching as possible, and leave the caching up to the client. If a website needs to only cache a set of data, then that's its own responsibility. If another web application needs real-time access, then I don't want to hinder that.
If I DO need to cache, say a static list that hardly changes, then I typically use something like MemoryCache and set a rolling timeout. For this, I usually write a wrapper for this that utilizes a lambda Func in the .Get() property of my caching service as the source of the cache for that key if the value happens to be null.
This might seem like an odd question, but it's been bugging me for a while now. Given that i'm not a hugely experienced programmer, and i'm the sole application/c# developer in the company, I felt the need to sanity check this with you guys.
We have created an application that handles shipping information internally within our company, this application works with a central DB at our IT office.
We've recently switch DB from mysql to mssql and during the transition we decided to forgo the webservices previously used and connect directly to the DB using Application Role, for added security we only allow access to Store Procedures and all CRUD operations are handle via these.
However we currently have stored procedures for updating every field in one of our objects, which is quite a few stored procedures, and as such quite a bit of work on the client for the DataRepository (needing separate code to call the procedure and pass the right params for each procedure).
So i'm thinking, would it be better to simply update the entire object (in this case, an object represents a table, for example shipments) given that a lot of that data would be change one field at a time after initial insert, and that we are trying to keep the network usage down, as some of the clients will run with limited internet.
Whats the standard practice for this kind of thing? or is there a method that I've overlooked?
I would say that updating all the columns for the entire row is a much more common practice.
If you have a proc for each field, and you change multiple fields in one update, you will have to wrap all the stored procedure calls into a single transaction to avoid the database getting into an inconsistent state. You also have to detect which field changed (which means you need to compare the old row to the new row).
Look into using an Object Relational Mapper (ORM) like Entity Framework for these kinds of operations. You will find that there is not general consensus on whether ORMs are a great solution for all data access needs, but it's hard to argue that they solve the problem of CRUD pretty comprehensively.
Connecting directly to the DB over the internet isn't something I'd switch to in a hurry.
"we decided to forgo the webservices previously used and connect directly to the DB"
What made you decide this?
If you are intent on this model, then a single SPROC to update an entire row would be advantageous over one per column. I have a similar application which uses SPROCs in this way, however the data from the client comes in via XML, then a middleware application on our server end deals with updating the DB.
The standard practice is not to connect to DB over the internet.
Even for small app, this should be the overall model:
Client app -> over internet -> server-side app (WCF WebService) -> LAN/localhost -> SQL
DB
Benefits:
your client app would not even know that you have switched DB implementations.
It would not know anything about DB security, etc.
you, as a programmer, would not be thinking in terms of "rows" and "columns" on client side. Those would be objects and fields.
you would be able to use different protocols: send only single field updates between client app and server app, but update entire rows between server app and DB.
Now, given your situation, updating entire row (the entire object) is definitely more of a standard practice than updating a single column.
It's better to only update what you change if you know what you change (if using an ORM like entity Framework for example), but if you're going down the stored proc route then yes definately update everything in a row at once that's way granular enough.
You should take the switch as an oportunity to change over to LINQ to entities however if you're already in a big change and ditch stored procedures in the process whenever possible
I am developing the MVC web application.
Which means I am creating views, models, view models. I use linq-to-sql as a database provider, I have implemented custom Unit-of-Work pattern, and also the Data Source, Repositories and Service patterns which looks perfectly (in my implementation) and completely separated from the direct SQL code. Actually, from any database, which is designed for unit testing, so I could test my application with in-memory data sources with no affect on the database.
And eventually I am stuck with one problem: I have no protection agains cross thread (or cross process, because as far as I know, IIS can create more than one app domain with a single web app) operations.
For example, I have a table with some records. And every now and again a web request happens which (inside controller and then service and then repository) picks the SQL table' row on the maximum of let's say TicketId column and then inserts in that table another row with (that column value + 1).
In the case two or more threads or processes do the same thing, the duplicated values can appear in the database. Some time ago, when my webapp was somekind smaller, I used the direct SQL code and simple UPDLOCK in SELECT statements which inside TransactionScope using block was locking the record I'm modifying (or anything else) preventing all other database clients to wait until I finish.
With all these patterns I forgot one thing:
How do I actually implement database multi-access protection issue?
Without any direct SQL code.
How do I actually implement database multi-access protection issue?
It's the database engine's job to do that. It's your job to ensure your app fails gracefully should there be any issues reported back. See Locking in the Database Engine.
For example, I have a table with some records. And every now and again a web request happens which
(inside controller and then service and then repository) picks the SQL table' row on the maximum > of let's say TicketId column and then inserts in that table another row with (that column value + 1).
I get the impression here that you don't seem to have much faith in your database considering you are trying to replicate it's behaviour. Set your field to be auto-increment and that should solve your issue. If you do have to implement your own manual auto-increment then you need to use some form of locking because what you essentially have is a race condition e.g.
private static object lockObject = new Object();
lock(lockObject)
{
... DB stuff
}
For cross-process locking, you would need to look at possibly a named Mutex.
Why not use a single table to represent a ticket number and the use transaction with Seriliazable transaction isolation level?
int GetTicketNumber(bool shouldIncrement)
{
if(shouldIncrement)
{
// use transaction with serilizable isolation level
select and update
}
else
{
//just select
}
}