I created a timeout middleware that works basically like this:
public async Task InvokeAsync(HttpContext httpContext)
{
var stopwatch = Stopwatch.StartNew();
using (var timeoutTS = CancellationTokenSource.CreateLinkedTokenSource(httpContext.RequestAborted))
{
var delayTask = Task.Delay(config.Timeout);
var res = await Task.WhenAny(delayTask, _next(httpContext));
Trace.WriteLine("Time taken = " + stopwatch.ElapsedMilliseconds);
if (res == delayTask)
{
timeoutTS.Cancel();
httpContext.Response.StatusCode = 408;
}
}
}
In order to test it, I created a controller action:
[HttpGet]
public async Task<string> Get(string timeout)
{
var result = DateTime.Now.ToString("mm:ss.fff");
if (timeout != null)
{
await Task.Delay(2000);
}
var rng = new Random();
result = result + " - " + DateTime.Now.ToString("mm:ss.fff");
return result;
}
The configured timeout to 500ms and the Time Taken reported is usually 501-504 ms (which is a very acceptable skid).
The problem is that every now and then I was seeing an error on the output windows saying that the response had already started. And I thought to myself: this cant be! this is happening 1 second earlier than the end of the Task.Delay on the corresponding controller.
So I opened up fiddler and (to my surprise) several requests are returning in 1.3-1.7 seconds WITH A FULL RESPONSE BODY.
By comparing the reported time written on the response body with the timestamp on fiddler "statistic" tab I can guarantee that the response im looking at does not belong to that request at hand!
Does anyone knows what's going on? Why is this "jumbling" happening?
Frankly, you're not using middleware in the way it is designed for.
You might want to read this middleware docs.
The ASP.NET Core request pipeline consists of a sequence of request delegates, called one after the other.
In your case, your middleware is running in parallel with the next middleware.
When a middleware short-circuits, it's called a terminal middleware because it prevents further middleware from processing the request.
If I understand you correctly, you might want to create such terminal middleware, but clearly your current one is not.
In your case, you have invoked the _next middleware, which means the request has already handed off to the next middleware in the request pipeline. The subsequent middleware components can start the response before the timeout has elapsed. i.e. a race condition between your middleware and a subsequent middleware.
To avoid the race condition, you should always check HasStarted before assigning the status code. And if the response has started, all you can do might only be aborting the request if you don't want the client to wait for too long.
static void ResetOrAbort(HttpContext httpContext)
{
var resetFeature = httpContext.Features.Get<IHttpResetFeature>();
if (resetFeature is not null)
{
resetFeature.Reset(2);
}
else
{
httpContext.Abort();
}
}
app.Use(next =>
{
return async context =>
{
var nextTask = next(context);
var t = await Task.WhenAny(nextTask, Task.Delay(100));
if (t != nextTask)
{
var response = context.Response;
// If response has not started, return 408
if (!response.HasStarted)
{
// NOTE: you will still get same exception
// because the above check does not eliminate
// any race condition
try
{
response.StatusCode = StatusCodes.Status408RequestTimeout;
await response.StartAsync();
}
catch
{
ResetOrAbort(context);
}
}
// Otherwise, abort the request
else
{
ResetOrAbort(context);
}
}
};
});
Related
I try to understand solution of this channel CancellationTokenSource callback is called for all request
In dotnet 5
I have the code bellow
public class AppTimeout
{
...
public AppTimeout(RequestDelegate next, IOptions<ServerOptions> options)
{
...
}
public async Task Invoke(HttpContext httpContext)
{
var cancellationTokenSource = CancellationTokenSource.CreateLinkedTokenSource(httpContext.RequestAborted);
cancellationTokenSource.CancelAfter(_timeout);
httpContext.RequestAborted = cancellationTokenSource.Token;
var requestPath = httpContext.Request.Path;
cancellationTokenSource.Token.Register(() =>
{
logger.Message("... requestPath...");
});
await _next(httpContext);
}
}
After on request have been in timeout, The callback will be triggers for all request
But it is not case and the callback will be called only for the request in timeout just if I replace
cancellationTokenSource.Token.Register(() =>
by
using var registration = cancellationTokenSource.Token.Register(() =>
My question is why ?
Could you explain me please?
Without using statements both cancellationTokenSource and registration will continue to be active after the requests ends and eventually you will get a timeout where the callback is invoked even though the request already ended. The timeout is not a timeout of the request. It's a result of a bug in the code. I'm not sure I understand why this only starts to occur after a "real" timeout but I wouldn't waste energy trying to understand that. You have to dispose registration and I believe also cancellationTokenSource when the requests ends (e.g. by using using statements).
How to efficiently limit request length timeout on server side ? I'm using Microsoft.Owin.Host.HttpListener and there are cases when (due to call to external service) serving request takes ridiculous amount of time. This is not a problem - but web server should give up sooner than - well never (I did some tests, but after 5 minutes I stopped it).
Is there a way how to limit time for serving single request (similar to <httpRuntime maxRequestLength="..." /> in IIS ecosystem) ?
Sample controller code:
public async Task<HttpResponseMessage> Get() {
// ... calls to 3pty services here
await Task.Delay(TimeSpan.FromMinutes(5));
}
Starting web server:
WebApp.Start(this.listeningAddress, new Action<IAppBuilder>(this.Build));
Note: I've read about limiting http listener, but that just limits incoming request properties, it doesn't cancel request that is slow due to slow server processing:
var listener = appBuilder.Properties[typeof(OwinHttpListener).FullName] as OwinHttpListener;
var timeoutManager = listener.Listener.TimeoutManager;
timeoutManager.DrainEntityBody = TimeSpan.FromSeconds(20);
timeoutManager.EntityBody = TimeSpan.FromSeconds(20);
timeoutManager.HeaderWait = TimeSpan.FromSeconds(20);
timeoutManager.IdleConnection = TimeSpan.FromSeconds(20);
timeoutManager.RequestQueue = TimeSpan.FromSeconds(20);
Related:
https://github.com/aspnet/AspNetKatana/issues/152
Conceptually "older" web server solutions - i.e. IIS are using one-thread-per-request separation and ThreadAbortException to kill slow requests. Owin is using different philosophy - i.e. it fires new task per request and forcibly cancelling task is best avoided. There are two sides of this problem:
shus client away if it takes too long
cancel server processing if it takes too long
Both can be achieved using middleware component. There also is a cancellation token provided directly by owin infrastructure for cases when client disconnects (context.Request.CallCancelled where context is IOwinContext)
If you're interested only in cancelling server flow ASAP when it takes to long, I'd recommend something like
public class MyMiddlewareClass : OwinMiddleware
{
// 5 secs is ok for testing, you might want to increase this
const int WAIT_MAX_MS = 5000;
public MyMiddlewareClass(OwinMiddleware next) : base(next)
{
}
public override async Task Invoke(IOwinContext context)
{
using (var source = CancellationTokenSource.CreateLinkedTokenSource(
context.Request.CallCancelled))
{
source.CancelAfter(WAIT_MAX_MS);
// combined "client disconnected" and "it takes too long" token
context.Set("RequestTerminated", source.Token);
await Next.Invoke(context);
}
}
}
And then in controller
public async Task<string> Get()
{
var context = this.Request.GetOwinContext();
var token = context.Get<CancellationToken>("RequestTerminated");
// simulate long async call
await Task.Delay(10000, token);
token.ThrowIfCancellationRequested();
return "Hello !";
}
Shusing the client away is more complex. The middleware will look like this:
public static async Task ShutDownClientWhenItTakesTooLong(IOwinContext context,
CancellationToken timeoutToken)
{
await Task.Delay(WAIT_MAX_MS, timeoutToken);
if (timeoutToken.IsCancellationRequested)
{
return;
}
context.Response.StatusCode = (int)HttpStatusCode.ServiceUnavailable;
}
public async Task ExecuteMainRequest(IOwinContext context,
CancellationTokenSource timeoutSource, Task timeoutTask)
{
try
{
await Next.Invoke(context);
}
finally
{
timeoutSource.Cancel();
await timeoutTask;
}
}
public override async Task Invoke(IOwinContext context)
{
using (var source = CancellationTokenSource.CreateLinkedTokenSource(
context.Request.CallCancelled))
using (var timeoutSource = new CancellationTokenSource())
{
source.CancelAfter(WAIT_MAX_MS);
context.Set("RequestTerminated", source.Token);
var timeoutTask = ShutDownClientWhenItTakesTooLong(context, timeoutSource.Token);
await Task.WhenAny(
timeoutTask,
ExecuteMainRequest(context, timeoutSource, timeoutTask)
);
}
}
I have a ASP.NET project which involves sending HTTP requests via the Web-API Framework. The following exception is only raised when debugging:
The server committed a protocol violation. Section=ResponseStatusLine
The project runs perfectly if I "Start Without Debugging".
How should I resolve this exception?
Any help is appreciated!
Update
The problem seems related to the ASP.NET MVC Identity Framework.
To access other Web-API methods, the client application has to first POST a login request (The login request does not need to be secure yet, and so I am sending the username and password strings directly to the Web-API POST method). If I comment out the login request, no more exception is raised.
Below are the relevant code snippets:
The Post method:
UserManager<ApplicationUser> UserManager = new UserManager<ApplicationUser>(new UserStore<ApplicationUser>(new ApplicationDbContext()));
AccountAccess ac = new AccountAccess();
public async Task<HttpResponseMessage> Post()
{
string result = await Request.Content.ReadAsStringAsync();
LoginMessage msg = JsonConvert.DeserializeObject<LoginMessage>(result);
HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);
var user = UserManager.Find(msg.username, msg.password);
if (user == null)
return response;
if (user.Roles == null)
return response;
var role = from r in user.Roles where (r.RoleId == "1" || r.RoleId == "2") select r;
if (role.Count() == 0)
{
return response;
}
bool task = await ac.LoginAsync(msg.username, msg.password);
response.Content = new StringContent(task.ToString());
return response;
}
The Account Access class (simulating the default AccountController in MVC template):
public class AccountAccess
{
public static bool success = false;
public AccountAccess()
: this(new UserManager<ApplicationUser>(new UserStore<ApplicationUser>(new ApplicationDbContext())))
{
}
public AccountAccess(UserManager<ApplicationUser> userManager)
{
UserManager = userManager;
}
public UserManager<ApplicationUser> UserManager { get; private set; }
public async Task<bool> LoginAsync(string username, string password)
{
var user = await UserManager.FindAsync(username, password);
if (user != null)
{
await SignInAsync(user, isPersistent: false);
return true;
}
else
{
return false;
}
}
~AccountAccess()
{
if (UserManager != null)
{
UserManager.Dispose();
UserManager = null;
}
}
private IAuthenticationManager AuthenticationManager
{
get
{
return HttpContext.Current.GetOwinContext().Authentication;
}
}
private async Task SignInAsync(ApplicationUser user, bool isPersistent)
{
AuthenticationManager.SignOut(DefaultAuthenticationTypes.ExternalCookie);
var identity = await UserManager.CreateIdentityAsync(user, DefaultAuthenticationTypes.ApplicationCookie);
AuthenticationManager.SignIn(new AuthenticationProperties() { IsPersistent = isPersistent }, identity);
}
}
Below are the relevant code snippets:
In client application:
public static async Task<List<T>> getItemAsync<T>(string urlAction)
{
message = new HttpRequestMessage();
message.Method = HttpMethod.Get;
message.RequestUri = new Uri(urlBase + urlAction);
HttpResponseMessage response = await client.SendAsync(message);
string result = await response.Content.ReadAsStringAsync();
List<T> msgs = JsonConvert.DeserializeObject<List<T>>(result);
return msgs;
}
In Web-API controller:
public HttpResponseMessage Get(string id)
{
HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);
if (id == "ItemA")
{
List<ItemAMessage> msgs = new List<ItemAMessage>();
// some code...
response.Content = new StringContent(JsonConvert.SerializeObject(msgs));
}
else if (id == "ItemB")
{
List<ItemBMessage> msgs = new List<ItemBMessage>();
// some code...
response.Content = new StringContent(JsonConvert.SerializeObject(msgs));
}
return response;
}
Some observations I have:
I thought that I may need to send the request asynchronously (with the async-await syntax), but the exception still persists that way.
If I step through the code, the request does enter the HTTP method, but the code breaks at random line (Why?!) before returning the response, so I assume no response is being sent back.
I have tried the following solutions, as suggested in answers to similar questions, none of which works for me:
Setting useUnsafeHeaderParsing to true
Adding the header Keep-Alive: false
Changing the port setting of Skype (I don't have Skype, and port 80 and 443 are not occupied)
Additional information, in case they matter:
Mac OS running Windows 8.1 with VMware Fusion
Visual Studio 2013
.NET Framework 4.5
IIS Express Server
Update 2
The exception is resolved, but I am unsure of which modification did the trick. AFAIK, either one or both of the following fixed it:
I have a checkConnection() method, which basically sends a GET request and return true on success. I added await to the HttpClient.SendAsync() method and enforced async all the way up.
I retracted all code in the MainWindow constructor, except for the InitializeComponent() method, into the Window Initialized event handler.
Any idea?
Below are relevant code to the modifications illustrated above:
the checkConnectionAsync method:
public static async Task<bool> checkConnectionAsync()
{
message = new HttpRequestMessage();
message.Method = HttpMethod.Get;
message.RequestUri = new Uri(urlBase);
try
{
HttpResponseMessage response = await client.SendAsync(message);
return (response.IsSuccessStatusCode);
}
catch (AggregateException)
{
return false;
}
}
Window Initialized event handler (retracted from the MainWindow constructor):
private async void Window_Initialized(object sender, EventArgs e)
{
if (await checkConnectionAsync())
{
await loggingIn();
getItemA();
getItemB();
}
else
{
logMsg.Content = "Connection Lost. Restart GUI and try again.";
}
}
Update 3
Although this may be a little off-topic, I'd like to add a side note in case anyone else falls into this – I have been using the wrong authentication approach for Web-API to start with. The Web-API project template already has a built-in Identity framework, and I somehow "replaced" it with a rather simple yet broken approach...
This video is a nice tutorial to start with.
This article provides a more comprehensive explanation.
In the Client Application you are not awaiting task. Accessing Result without awaiting may cause unpredictable errors. If it only fails during Debug mode, I can't say for sure, but it certainly isn't the same program (extra checks added, optimizations generally not enabled). Regardless of when Debugging is active, if you have a code error, you should fix that and it should work in either modes.
So either make that function async and call the task with the await modifier, or call task.WaitAndUnwrapException() on the task so it will block synchronously until the result is returned from the server.
Make sure URL has ID query string with value either as Item A or Item B. Otherwise, you will be returning no content with Http status code 200 which could lead to protocol violation.
When you use SendAsync, you are required to provide all relevant message headers yourself, including message.Headers.Authorization = new AuthenticationHeaderValue("Basic", token); for example.
You might want to use GetAsync instead (and call a specific get method on the server).
Also, are you sure the exception is resolved? If you have some high level async method that returns a Task and not void, that exception might be silently ignored.
I have a rest endpoint on asp.net mvc:
[HttpGet]
public List<SomeEntity> Get(){
// Lets imagine that this operation lasts for 1 minute
var someSlowOperationResult = SomeWhere.GetDataSlow();
return someSlowOperationResult;
}
On the frontEnd I have a next javascript:
var promise = $.get("/SomeEntities");
setTimeout(function(){promise.abort()}, 100);
How to force Thread to die after abort call, to prevent slow calculation to be done?
Thanks in advance.
I found that Response have isClientConnected property. So we can use next approach:
[HttpGet]
public List<SomeEntity> Get(){
var gotResult = false;
var result = new List<SomeEntity>();
var tokenSource2 = new CancellationTokenSource();
CancellationToken ct = tokenSource2.Token;
Task.Factory.StartNew(() =>
{
// Do something with cancelation token to break current operation
result = SomeWhere.GetSomethingReallySlow();
gotResult = true;
}, ct);
while (!gotResult)
{
if (!Response.IsClientConnected)
{
tokenSource2.Cancel();
return result;
}
Thread.Sleep(100);
}
return result;
}
Can we? Or I miss something?
UPDATE:
Yes, it works
The backend has no idea that you have called abort() and if the request has already been sent then the server-side logic will run until it completes. In order to stop it from running you will have to send another request to your controller which notifies that controller that you've aborted the request and the controller will have to access the instance that is currently running you slow operation and this instance should have a method which forces the calculations to cancel.
I have a problem in my app with the HttpClient. When I change the date or time on my phone the HttpClient cache gets messed up. For example, when I change the date backwards and run the app again every response is taken from the cache, and when I move the date forward, no request is cached, nor taken from the cache. I don't know where the problem is, so help if you encountered a similiar problem.
Is there any way to clear the HttpClient cache programatically? I would do that on every app start.
EDIT:
I just used Fiddler to look at responses and found out that the time in the 'Date' response header is the correct time (on server), and maybe HttpClient uses this header for its calculations in combination with the time on the device, which returns wrong results
EDIT2:
One use case:
When the user opens the app and fetches an url (http://my.website.com/jsondata) where the cache control header is set to 10 seconds, and then closes the app and changes the time backwards or forwards, and then opens the app again in some time, then in the first case the httpclient Always gets the response of the same url from its cache, and in the second case never uses the cache again (if the user requests the same url multiple times in 10 seconds)
My code is :
public async Task<StringServerResponse> DownloadJsonAsync(Uri uri, DecompressionMethods decompressionMethod = DecompressionMethods.None, int timeout = 30000, int retries = 3)
{
if (retries < 1 || retries > 10) retries = 3;
int currentRetries = 0;
// Baseline delay of 1 second
int baselineDelayMs = 1000;
// Used for exponential back-off
Random random = new Random();
StringServerResponse response = new StringServerResponse();
if (decompressionMethod == DecompressionMethods.GZip)
{
//Use decompression handler
using (var compressedHttpClientHandler = new HttpClientHandler())
{
if (compressedHttpClientHandler.SupportsAutomaticDecompression)
{
compressedHttpClientHandler.AutomaticDecompression = System.Net.DecompressionMethods.GZip;
}
using (var httpClient = new HttpClient(compressedHttpClientHandler))
{
httpClient.Timeout = TimeSpan.FromMilliseconds(timeout);
do
{
++currentRetries;
try
{
var httpResponse = await httpClient.GetAsync(uri, HttpCompletionOption.ResponseContentRead);
if (httpResponse.IsSuccessStatusCode)
{
response.StatusCode = httpResponse.StatusCode;
response.Content = await httpResponse.Content.ReadAsStringAsync();
return response;
}
}
catch (Exception)
{
}
int delayMs = baselineDelayMs + random.Next((int) (baselineDelayMs*0.5), baselineDelayMs);
if (currentRetries < retries)
{
await Task.Delay(delayMs);
DebugLoggingService.Log("Delayed web request " + delayMs + " seconds.");
}
// Increment base-delay time
baselineDelayMs *= 2;
} while (currentRetries < retries);
}
}
}
else
{
// same but without gzip handler
}
return null;
}
You can try to add one more parameter to your input queries in URL. You can follow my another post: Why does the HttpClient always give me the same response?