I'm doing some close to the metal HTTP tangling with Owin.
I have a owin middleware that outputs javascripts. It looks like this (relevant parts)
public override Task Invoke(IOwinContext context)
{
var response = context.Response;
response.ContentType = "application/javascript";
response.StatusCode = 200;
if (ClientCached(context.Request, scriptBuildDate))
{
response.StatusCode = 304;
response.Headers["Content-Length"] = "0";
response.Body.Close();
response.Body = Stream.Null;
return Task.FromResult<Object>(null);
}
response.Headers["Last-Modified"] = scriptBuildDate.ToUniversalTime().ToString("r");
return response.WriteAsync(js);
}
private bool ClientCached(IOwinRequest request, DateTime contentModified)
{
string header = request.Headers["If-Modified-Since"];
if (header != null)
{
DateTime isModifiedSince;
if (DateTime.TryParse(header, out isModifiedSince))
{
return isModifiedSince >= contentModified;
}
}
return false;
}
It will output 200 if its not client cached and add a Last-Modified date to the header, if its client cached it will output 304 "Not modified".
The problem is that the client will not call the url again unless they are doing a hard F5 in the browser. My understanding of Last modified caching is that it should call each time to check if the content has been modified?
Update:
Control: must-revalidate
Chrome
F5 and ctrl+F5 will call server, opening site in new tab or restarting browser will call server, typing the address in same tab will not call server. If-Modified-Since only cleared when doing Ctrl+F5 which means it can be used to return 304 correctly when content not modified
IE10
F5 and ctrl+F5 will call server, opening site in new tab will not call server, typing the address in same tab will not call server. If-Modified-Since cleared when doing Ctrl+F5 OR when restarting browser
Cache-Control: no-cache and Pragma: no-cach
Chrome
Will call server for every action If-Modified-Since only cleared when doing Ctrl+F5
Will call server for every action If-Modified-Since cleared for both restarting browser and Ctrl+F5
Conclusion
Looks like no-cache is might better if you want to be sure it calls to check for 304 each time
From the HTTP/1.1 spec (RFC2616, my emphasis):
13.2.2 Heuristic Expiration
Since origin servers do not always provide explicit expiration times,
HTTP caches typically assign heuristic expiration times, employing
algorithms that use other header values (such as the Last-Modified
time) to estimate a plausible expiration time. The HTTP/1.1
specification does not provide specific algorithms, but does impose
worst-case constraints on their results. Since heuristic expiration
times might compromise semantic transparency, they ought to used
cautiously, and we encourage origin servers to provide explicit
expiration times as much as possible.
Providing a Last-Modified header is not equivalent to asking user agents to check for updates every time they need a resource from your server.
Ideally, you should add an Expires header whenever possible. However, adding the header Cache-Control: must-revalidate should help.
Related
Is it possible to set Max Age for Cookies in Web Forms Application? I know, that it's okey to set Expire, but is there a way to set Max Age?
Asp.Net doesn't specifically provide this property on HttpCookie, probably because they are very Microsoft-centric, and IE doesn't support max-age (as least, as of IE11)
However, you can still do it. Here's some code demonstrating the proper and invalid ways to set this cookie with max-age:
// doesn't work:
var mytestcookie = new HttpCookie("regular_httpcookie", "01");
mytestcookie.Values.Add("max-age", "300");
Response.Cookies.Add(mytestcookie);
// *does* work:
Response.Headers.Add("set-cookie", "testingmaxage=01;max-age=300; path=/");
And it renders like this in the HTTP response:
Set-Cookie testingmaxage=01;max-age=300; path=/
X-AspNet-Version 4.0.30319
Set-Cookie regular_httpcookie=01&max-age=300; expires=Fri, 10-Jun-2016 15:02:15 GMT; path=/
As you can see above, if you are also setting cookies using HttpCookie, this will create a second "set-cookie" header on the response , but the browser won't mind, it will just add it to the list of cookies.
I tested on IE11 and Chrome and this is a non-issue - the cookies will all go in, as long as they have differing names. If the cookie name conflicts with one already set in HttpCookies, the last one in wins. Check out the text of your HTTP response to see which one goes in last. (Best to simply make sure they don't conflict though)
As I mentioned at the beginning, when testing on IE11, I noted that it's ignoring the max-age property of the cookie. Here's a link to a way to settle that issue:
Set-Cookie: Expire property, clock skew and Internet Explorer issue
I'm using HttpClient 0.6.0 from NuGet.
I have the following C# code:
var client = new HttpClient(new WebRequestHandler() {
CachePolicy =
new HttpRequestCachePolicy(HttpRequestCacheLevel.CacheIfAvailable)
});
client.GetAsync("http://myservice/asdf");
The service (this time CouchDB) returns an ETag value and status code 200 OK. There is returned a Cache-Control header with value must-revalidate
Update, here are the response headers from couchdb (taken from the visual studio debugger):
Server: CouchDB/1.1.1 (Erlang OTP/R14B04)
Etag: "1-27964df653cea4316d0acbab10fd9c04"
Date: Fri, 09 Dec 2011 11:56:07 GMT
Cache-Control: must-revalidate
Next time I do the exact same request, HttpClient does a conditional request and gets back 304 Not Modified. Which is right.
However, if I am using low-level HttpWebRequest class with the same CachePolicy, the request isn't even made the second time. This is the way I would want HttpClient also behave.
Is it the must-revalidate header value or why is HttpClient behaving differently? I would like to do only one request and then have the rest from cache without the conditional request..
(Also, as a side-note, when debugging, the Response status code is shown as 200 OK, even though the service returns 304 Not Modified)
Both clients behave correctly.
must-revalidate only applies to stale responses.
When the must-revalidate directive is present in a response received by a cache, that cache MUST NOT use the entry after it becomes stale to respond to a
subsequent request without first revalidating it with the origin server. (I.e., the cache MUST do an end-to-end revalidation every time, if, based solely on the origin server's Expires or max-age value, the cached response is stale.)
Since you do not provide explicit expiration, caches are allowed to use heuristics to determine freshness.
Since you do not provide Last-Modified caches do not need to warn the client that heuristics was used.
If none of Expires, Cache-Control: max-age, or Cache-Control: s- maxage (see section 14.9.3) appears in the response, and the response does not include other restrictions on caching, the cache MAY compute a freshness lifetime using a heuristic. The cache MUST attach Warning 113 to any response whose age is more than 24 hours if such warning has not already been added.
The response age is calculated based on Date header since Age is not present.
If the response is still fresh according to heuristic expiration, caches may use the stored response.
One explanation is that HttpWebRequest uses heuristics and that there was a stored response with status code 200 that was still fresh.
Answering my own question..
According to http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.4 I would say that
a "Cache-Control: must-revalidate" without expiration states that the resource should be validated on every request.
In this case it means a conditional GET should be done every time the resource is made. So in this case System.Net.Http.HttpClient is behaving correctly and the legacy (Http)WebRequest is doing invalid behavior.
In ASP.NET MVC, now we can response 304 code to browser, which means that the content in the server has not been changed, the browser can use its local cache for this url.
public ActionResult Image(int id){
var image = _imageRepository.Get(id);
if (image == null)
throw new HttpException(404, "Image not found");
if (!String.IsNullOrEmpty(Request.Headers["If-Modified-Since"]))
{
CultureInfo provider = CultureInfo.InvariantCulture;
var lastMod = DateTime.ParseExact(Request.Headers["If-Modified-Since"], "r", provider).ToLocalTime();
if (lastMod == image.TimeStamp.AddMilliseconds(-image.TimeStamp.Millisecond))
{
Response.StatusCode = 304;
Response.StatusDescription = "Not Modified";
return Content(String.Empty);
}
}
var stream = new MemoryStream(image.GetImage());
Response.Cache.SetCacheability(HttpCacheability.Public);
Response.Cache.SetLastModified(image.TimeStamp);
return File(stream, image.MimeType);
}
But I am a little confused about the logic in the browser. For example, when we first ask for a page http://www.test.com/index.html,it will load a javascript file aaa.js. But when the browser ask another page http://www.test.com/index2.html, this page also contains aaa.js.
Here comes the question. We know that the browser has a logic for http cache. I assume that when the browser asks for index2.html, it will check that it has aaa.js locally, which is available, so it will not communicate with server about this file. So here, no 304 is returned, because the browser has not request anything about this file.Is this the right logic?
Or every time it will communicate with the server to check the version of the file? In this situation, if we don't write any C# code to return 304 status, every time it will return the whole file.So I guess this is not the logic.
What is the relationship between the browser cache and 304 status?
Depending on the server response to the first request to aaa.js the browser may or may not request the file again the second time.
If no specific caching headers are sent by the server with the file, on the second page load the browser will send a request for aaa.js again. If the browser doesn't have the JS file in its cache, it will send the request the same as it did the first time. If aaa.js is in the browser cache it will send a request to the server containing an If-Modified-Since header with the date the file was previously downloaded. The server then checks if the file has been modified: if so it sends the new file; otherwise it sends the 304 header.
Now let's spool back to the beginning. In the initial request to aaa.js the server could include a Cache-control header telling the browser how long to cache the file for. Let's say Cache-control: max-age=3600 which instructs to cache the file for one hour (3600 seconds).
If the user visits the second page within one hour, the browser won't even send a request to the server for aaa.js, it will just use the cached file without question.
Once the hour is up and a new page is loaded, the browser request aaa.js again.
I am trying to return a ActionResult from an MVC api, and if you give it a redirect uri in the post body it should redirect that uri. Right now if you do not give it a uri it performs fine. But if you give it a uri where it should redirect to that uri it just returns a 302 and does not navigate anywhere.
Now I think one issue I am having is that the returned location in the response header is the base URI for the MVC and then the input URI. One problem I have is that all documentation on this I have found, well sucks.
So my question is how do I use the RedirectResult action to actually redirect to a different URI? Also can I redirect to a URI that is outside the current domain? I am worried I cannot since it is appending the base URI to the location in the response. It does not force the clients browser to navigate to another URI.
Here is my action result.
[AllowAnonymous]
public ActionResult Dispatch(Dto.GoogleAnalyticsEvent evt)
{
log.DebugFormat("Dispatch()");
if (evt.Redirect != null)
{
return new RedirectResult(evt.Redirect, false);
}
else
{
return Dispatch<Dto.GoogleAnalyticsEvent>(evt, _api.DispatchAnalyticsEvent, true);
}
}
Here is the returned response if you give it a uri in the post body. The response code is actually a 302 found.
Access-Control-Allow-Head... Content-Type
Access-Control-Allow-Meth... GET,POST
Cache-Control private
Content-Length 139
Content-Type text/html; charset=utf-8
Date Wed, 22 May 2013 22:42:25 GMT
Location /events/www.google.com
Server Microsoft-IIS/7.5
access-control-allow-orig... *
Thanks in advance for the help.
Just to clarify, I was looking for the response from the API to force the browser to navigate to a URI upon response. Though now that I type that out loud it seems that browsers probably would not allow that. How is a 302 found supposed to affect the browser?
If you are calling this action via AJAX (or some other form of direct call to server) the browser will have no way to know that server returned 302 (or 40x, 50x, 200 or any other kind of response).
The behavior is exactly the same as if you get 200+data response for AJAX request - browser will not magically navigate to that response page.
Solution: if you need to redirect as result of AJAX request you need to redesign your protocol to return some 200 response that says "please redirect to this Url" and set window.location.href accordingly on client side when you get such response.
Note that AJAX call will follow 302 redirect and result of the call will be page you are redirecting to (if on the same domain) or "access denied" of some sort if it is cross domain and CORS not turned on destination. See How to prevent ajax requests to follow redirects using jQuery for more background on it.
While writing a custom IHttpHandler I came across a behavior that I didn't expect concerning the HttpCachePolicy object.
My handler calculates and sets an entity-tag (using the SetETag method on the HttpCachePolicy associated with the current response object). If I set the cache-control to public using the SetCacheability method everything works like a charm and the server sends along the e-tag header. If I set it to private the e-tag header will be suppressed.
Maybe I just haven't looked hard enough but I haven't seen anything in the HTTP/1.1 spec that would justify this behavior. Why wouldn't you want to send E-Tag to browsers while still prohibiting proxies from storing the data?
using System;
using System.Web;
public class Handler : IHttpHandler {
public void ProcessRequest (HttpContext ctx) {
ctx.Response.Cache.SetCacheability(HttpCacheability.Private);
ctx.Response.Cache.SetETag("\"static\"");
ctx.Response.ContentType = "text/plain";
ctx.Response.Write("Hello World");
}
public bool IsReusable { get { return true; } }
}
Will return
Cache-Control: private
Content-Type: text/plain; charset=utf-8
Content-Length: 11
But if we change it to public it'll return
Cache-Control: public
Content-Type: text/plain; charset=utf-8
Content-Length: 11
Etag: "static"
I've run this on the ASP.NET development server and IIS6 so far with the same results. Also I'm unable to explicitly set the ETag using
Response.AppendHeader("ETag", "static")
Update: It's possible to append the ETag header manually when running in IIS7, I suspect this is caused by the tight integration between ASP.NET and the IIS7 pipeline.
Clarification: It's a long question but the core question is this: why does ASP.NET do this, how can I get around it and should I?
Update: I'm going to accept Tony's answer since it's essentially correct (go Tony!). I found that if you want to emulate the HttpCacheability.Private fully you can set the cacheability to ServerAndPrivate but you also have call cache.SetOmitVaryStar(true) otherwise the cache will add the Vary: * header to the output and you don't want that. I'll edit that into the answer when I get edit permissions (or if you see this Tony perhaps you could edit your answer to include that call?)
I think you need to use HttpCacheability.ServerAndPrivate
That should give you cache-control: private in the headers and let you set an ETag.
The documentation on that needs to be a bit better.
Edit: Markus found that you also have call cache.SetOmitVaryStar(true) otherwise the cache will add the Vary: * header to the output and you don't want that.
Unfortunately if you look at System.Web.HttpCachePolicy.UpdateCachedHeaders() in .NET Reflector you see that there's an if statement specifically checking that the Cacheability is not Private before doing any ETag stuff. In any case, I've always found that Last-Modified/If-Modified-Since works well for our data and is a bit easier to monitor in Fiddler anyway.
If like me you're unhappy with the workaround mentioned here of using Cacheability.ServerAndPrivate, and you really want to use Private instead - perhaps because you are customising pages individually for users and it makes no sense to cache on the server - then at least in .NET 3.5 you can set ETag through Response.Headers.Add and this works fine.
N.B. if you do this you have to implement the comparison of the client headers yourself and the HTTP 304 response handling - not sure if .NET takes care of this for you under normal circumstances.