Develop Reference

ASP.NET MVC Cache is clearing itself

I have a ASP.NET MVC 4 application which is running on a Windows 2012 R2 with IIS 8.5 machine. There is a wierd behavior which I cannot figure it out. The cache seems is clearing itself. By cache, I mean MemoryCache.Default, HttpContext.Current.Cache, and also OutputCache. I'm googling the issue for hours and seems nothing is wrong. Can you list cause of clearing cache? I mean is there a checklist which I can test the server against it? Thanks in advance.

Those caches are held in memory, and as such are volatile. They are held against the W3wp process that IIS spawns to handle those requests.
After a period of inactivity, IIS closes down the processes, so these caches will be cleared.
IIS also closes down the processes (recycles the app pool)
By default after 1746 mins (IIRC)
After the memory used by a thread reaches a set threshold
Caches are also isolated by thread, so if you have a web farm/web garden, these caches are not shared, again IIRC
If you need to persist these items longer, then you will need to look at caching the objects in a persisted storage area, such as a db or application state managed server.

Inproc and Outproc session state modes

I have basic understanding of session state modes,i have gone through this article on MSDN but not able to understand When to use InProc,StateServer and SqlServer session modes?more specifically confused between when to use State server and when to use sqlserver?

The 3 various modes help spread out your state in different ways to make your application more scalable across a farm and to make it more robust in its own operation.
InProc
InProc is the most basic session management scenario where the the session is stored attached to the process that is actually running it. This means it has the fastest response time because the server doesn't have to go to an alternate source to fetch the data it needs. While it's technically the fastest, it's also the weakest because it can only be used on the server which is running the website. It also is prone to memory dumps. If your site crashes for any reason, the sessions are dumped along with the process. For small, very stable sites, InProc is perfectly acceptable and possibly even ideal. InProc also has the benefit of being able to hold any memory object in session. This can also be problematic if you attempt to hold enormous objects.
StateServer
StateServer refers to the ASP.Net state server service that can reside on any particular machine. It typically operates on port 42424, and can service a single machine or many machines. It is intended to be faster than the SQL server state management methods, but I would argue that the difference in speed is negligible. Perhaps in very large enterprise environments the difference becomes noticeable, but for the web farms I've seen it is not. StateServer requires that any objects in session be Serializable in order to be stored and transferred properly. This means that not just any object can be placed in session, so you have plan ahead when constructing your classes. The state server can be on the machine your website is on or it can be on a machine that is accessible through the 42424 port. This means that session data is decoupled from the IIS process and is therefore "immune" to crashes and hangs. This allows you to have a farm of servers using a common state server, and load balancing becomes simple if the client does not need to be restricted to a specific server. While the state server service is fairly rapid, it operates on a port that many network administrators consider to be simply another "attack vector" for intrusions. Which leads to the SQL state server.
SqlServer
SqlServer mode operates much the same as StateServer. Objects must be serialized, the sql server can be local or it can be remote making it less prone to single server crashes in a farm. Network administrators tend to prefer sql servers for state management because they reduce intrusion vectors. Since your website will likely need a sql server to perform data access anyway, this is just piggy backing. Sql server also allows you to visibly inspect what is in the state tables.
My preference typically is for the StateServer. It is very easy to get up and running and you can have a common one that holds state for many environments that are not related (re: dev, qa, etc). It requires no actual maintenance and is very easy to set up. It also does not require licenses to run as sql server does. However, as your need to decentralization and security increases, sql server becomes a much more friendly option. Use InProc for only the most basic sites or sites with limited traffic.

Session State Server vs Custom Session State Provider

I have been tasked to scale out the session for an application. From my research the most obvious choice is to use the State Server session provider, because I don't need the users sessions to persist (SQL Server Session provider)
About the app:
Currently using InProc session provider
All objects stored in session are serializable
All objects are small (mostly simple objects (int, string) and a few simple class instances)
Before I dive head-first into the IT side of things and with the ability to provide a custom session provider with ASP.NET 4, should I even consider a custom session state provider. Why or why not? Are there any "good" ones out there?
Thanks!
User feedback:
Why are we using session: persistence of data between postbacks (e.g. user selections)
How: user makes a selection, selection is stored. User leaves a page and returns,
selections are restored. etc. etc.
Will be creating a web farm

I've provided some links you can read up on on properly scaling session using the state server.
Useful links from Maarten Balliauw's blog:
http://blog.maartenballiauw.be/post/2007/11/ASPNET-load-balancing-and-ASPNET-state-server-%28aspnet_state%29.aspx
http://blog.maartenballiauw.be/post/2008/01/ASPNET-Session-State-Partitioning.aspx
http://blog.maartenballiauw.be/post/2008/01/ASPNET-Session-State-Partitioning-using-State-Server-Load-Balancing.aspx
My State Server related projects:
http://www.codeproject.com/KB/aspnet/p2pstateserver.aspx (latest code at https://github.com/tenor/p2pStateServer)
http://www.codeproject.com/KB/aspnet/stateserverfailover.aspx
Hope that helps.

It depends on what you mean by "scaling" session storage. If your simply talking about session state performance, your not going to beat the in-process session state provider. Switching to the State Server provider will actually make things slower -- due to the extra overhead of serializing and transferring objects across process boundaries.
Where the State Server can help you scale, is that it allows multiple machines in a load balanced web-farm to share a single session state. It is limited by machine memory, however, and if you will have lots of concurrent sessions you may want to use the SQL Session State provider.
For the most performance in a web farm, you can also try using AppFabric as was previously suggested. I haven't done it myself but it is explained here.
Also, here's a link for using Memcached as a Session State provider. Again, I haven't used it, so I can't offer an opinion on it...
EDIT: As #HOCA mentions in the comments, there are 3rd party solutions if cost isn't an issue. One I've heard of (but not used) is ScaleOut SessionServer.

I would highly recommend that before you look in to scaling out session that you first evaluate whether session was even needed in the first place.
Session variables are serialized and deserialized for every single page load whether the page uses that data or not. (EDIT: Craig pointed out that you have some level of control over this in .net 4 http://msdn.microsoft.com/en-us/library/system.web.sessionstate.sessionstatebehavior.aspx However, this still has drawbacks, see comments to this answer.)
For single server instances this is okay as you are just pulling it from the local memory of your web server. The load on these apps tend to be pretty small so caching user specific information locally makes sense.
However, as soon as you move storage of session to another server you have increased the network requirements and page load times of your application. Namely, every page will result in the session data to be moved from the remote server, across the network wire, and into memory of the web server.
At this point you have to ask yourself: is the load to pull this information from the database server directly as necessary more than pulling it from the session server every single time?
There are few instances where pulling it from the database server as needed takes longer or results in more traffic than grabbing it from a remote session server.
Bearing in mind that a lot of people set up their database server(s) to also be session servers and you start to see why use of session doesn't make any sense.
The only time I would consider using session for load balanced web apps is if the time to acquire the data exceeded a "reasonable" amount of time. For example, if you have a really complex query to return a single value and this query would have to be run for lots of pages. But even then there are better ways that reduce the complexity of dealing with remote session data.

I would advise against the use of session state, regardless of the provider.
Especially with your "very small objects" use viewstate. Why?
Scales best of all. NOTHING to remember on the server.
NO worries about session timeout.
No worries about webfarms.
No worries about wasted resources for sessions that will never come back.
Especially in ASP.NET 4 viewstate can be very manageable and small.

Is caching infrequently changed data to speed client application load times an appropriate use of Isolated Storage?

I'm working on an enterprise application re-write using Silverlight. The project is still in its early stages of development, but there is a heavy initial data load on the application start as it pulls several sets of business objects from the server. Some of these data sets are infrequently changed once they're set up by the user; like the list of all customized data types in use by the user.
In these cases, the thought is to cache the data objects (probably in a serialized form) in Isolated Storage so there's no wait on an asynchronous call to the server to grab the data after the first application load.
I thought that Isolated Storage is meant to store configuration data such as user preferences, or to share across the in-browser and out-of-browser version of an app...that it works a lot like a cookie store.
My main concern is that I'm unsure of how secure Isolated Storage is, and I don't trust caching application data in it. To be fair, the user would also have access to the Silverlight .xap file.
Is this an appropriate use for Isolated Storage, why or why not?

It's a fair use of isolated storage, if you're comfortable with the caveats.
The first caveat in my mind is that whatever you store in isolated storage on one machine will not be available when the user fires up your app on another machine - you lose the mobility advantage of web applications over desktop installed apps. If the user spends some time configuring their preferences, etc, they will be irritated that they have to do it all over again just because they switched to a different computer to view your web app. To solve this, you should replicate the user's customizations to cloud storage so that it can be copied down to whatever machine they choose to run your web app on. Treat the isolated storage as a performance optimization cache for data that officially lives in the cloud.
I believe Silverlight isolated storage is written to disk in the user's private data area in the file system. \users\\AppData or similar. This will keep it isolated away from other users on the same machine, but will not provide any protection from other programs running for the same user. I don't recall if Silverlight isolated storage is encrypted on disk. I highly doubt it.
A second caveat is that Silverlight isolated storage has a quota limit, and it's fairly small by default (1MB). The quota can be increased with a call to IncreaseQuotaTo(), which will prompt the end user to ok the request.
The third caveat is that if you're going to use local storage as a cache of data that lives in the cloud, you have to manage the data synchronization yourself. If the user makes changes locally, you need to push that up to the storage of authority in the cloud, and you'll have to decide when or how often to refresh the local cache from the cloud, and what to do when both have been changed at the same time (collision).
The browser cookie store is not a great metaphor for describing Silverlight isolated storage. Browser cookies for a given domain are attached to every http request that is made from the client to the server. The cookies are transmitted to the server constantly. The data in Silverlight isostorage is only accessible to the Silverlight code running on the client machine - it is never transmitted anywhere by Silverlight or the browser.
Treat Silverlight's isolated storage as a local cache of cloud data and you should be fine. Treat isostorage as a permanent storage and you'll piss off your customers because the data won't follow them everywhere they can use your web app.

Not a complete answer to your story but a data point to consider:
Beware the IO speeds of IsolatedStorage. While there has been considerable effort put into speeding it up, you may want to consider other options if you plan to do multiple small reads/writes as it can be extremely slow. (That, or use appropriate buffering techniques to ensure your reads/writes are larger and infrequent.)

ASP.NET Session size limitation

Is there some kind of Session size limitation or advisable value to not to surpass ?
In my web application I create a few DataTables to store user selections which are stored in session until user approves selections so I add those values to database.
The problem is that I don't know whether session is reliable enough to keep few objects in or not ?
Thanks!
more info
Session size is about 10-20KB max.

Here's a few notes about Session state:
InProc (mode="InProc") session state is limited to the amount of memory available to the worker process. Only object references are stored, not the objects themselves.
Out of process state management serialises objects before persisting them:
Out of Process state management using the session state server (mode="StateServer") is limited to the amount of memory available to the state service.
Out of Process state management using SQL Server (mode="SQLServer") is bound only by the maximum size of the SQL image data type or the maximum permitted size of the database.
Obviously there still has to be enough memory available to the worker process to be able to pull an out of session object back into memory and re-hydrate for the duration of the http request.
As I mentioned previously, out of process state management serialises objects before persisting them.
This means that objects must be serialisable which excludes, for example, XmlDocument or anything that inherits from MarshalByRef.
Attempting to serialise objects of this sort will result in the following exception:
Unable to serialize the session state. In 'StateServer' and
'SQLServer' mode, ASP.NET will serialize the session state objects,
and as a result non-serializable objects or MarshalByRef objects are
not permitted. The same restriction applies if similar serialization
is done by the custom session state store in 'Custom' mode.

Yes, it is reliable enough. It just isn't very scalable, so plan ahead. This will totally grind to a halt when you run it on more than 1 server.
And there is sort of a limit: Number-of-concurrent-Users * SizeOf-Session < Available-Mem
It depends of course on the size of the tables, storing a few kB is usually acceptable (although high traffic sites will try to keep it smaller).
If your users can share tables, then you can place that data in the Application object, a great saving.
And a session object is limited to the TimeOut setting, default is 20 min. One way to optimize memory consumption is reducing that, but that is a trade off with user convenience.

I am assuming that you are having the session stored in "inProc" mode. In this mode, ASP.NET applications' session, cache etc are stored in the web server's RAM (via aspnet_wp.exe process). And .NET does not get to use all of it. There is a setting in machine.config which tells the threshold limit (by default 60%). Once this threshold is reached, IIS will recycle the worker process, and all the session information is lost.
Note that, if your server hosts more than one asp.net application, the 60% of memory is to be shared by all apps. So if the cumulative memory usage reaches the threshold the worker process still gets recycled.
Alternative to this, besides optimizing your application to use session sparingly,is to set the application to use session in out of process mode (using a stateserver or sqlserver to store session information).
Out of process mode can bring down your system performance.
Refer to this article for more information on Session State Management.

You should always assume session is a very valuable storage and very limited. The consumption should be as little as possible, because you can never know how many users the application is going to support.
DataTable can be too large to store in sessions, unless it can be kept small enough.