I made a simple app where chunks of a file are streamed from client to server. Server-side I have handled exceptions such that a response of my own is returned to the client. When I throw an exception before reading from the stream has been completed, however, even though it gets caught and a custom response is returned, client-side I still get an unhandled RpcException with status Cancelled.
public override async Task<UploadFileResponse> UploadFile(
IAsyncStreamReader<UploadFileRequest> requestStream,
ServerCallContext context)
{
try
{
bool moveNext = await requestStream.MoveNext();
using (var stream = System.IO.File.Create($"foo.txt"))
{
while (moveNext)
{
// If something goes wrong here, before the stream has been fully read, an RpcException
// of status Cancelled is caught in the client instead of receiving an UploadFileResponse of
// type 'Failed'. Despite the fact that we catch it in the server and return a Failed response.
await stream.WriteAsync(requestStream.Current.Data.ToByteArray());
moveNext = await requestStream.MoveNext();
throw new Exception();
}
// If something goes wrong here, when the stream has been fully read, we catch it and successfully return
// a response of our own instead of an RpcException.
// throw new Exception();
}
return new UploadFileResponse()
{
StatusCode = UploadStatusCode.Ok
};
}
catch (Exception ex)
{
return new UploadFileResponse()
{
Message = ex.Message,
StatusCode = UploadStatusCode.Failed
};
}
}
Perhaps the way I approach implementing this operation is wrong. I can see why the server would return a Cancelled RPC exception because we indeed cancel the call before the stream has been fully read but I don't understand why it overrides the custom response. It might be that handling both would have to be done client-side - a failed response and a potential RPC exception.
I found some materials on the topic - Server and Client.
Apparently it is common to throw RpcExceptions whenever there should be an invalid response as also shown in the official gRPC Github repository here.
Related
I am currently trying to write an ASP.NET Core API middleware which opens a SQL transaction before the underlying MVC action is executed. The transaction uses the Serializable isolation level, and is used by all SQL requests in the underlying MVC action. Then, when the MVC action exits:
if it succeeded, the middleware should commit the transction ;
if it failed with a serialization error, the middleware should reset everything and retry the MVC action (max. N times) ;
otherwise, the middleware should roll back the transaction and rethrow the error.
What I ended up with is:
public async Task InvokeAsync(HttpContext context, IDatabaseDriver databaseDriver)
{
context.Request.EnableBuffering(REQUEST_BUFFER_THRESHOLD);
int attempt = 0;
while (true)
{
attempt++;
try
{
await this._next(context);
await databaseDriver.Commit();
break;
}
catch (PostgresException ex)
when (ex.SqlState == PostgresErrorCodes.SerializationFailure &&
attempt <= MAX_RETRIES)
{
// SQL serialization failure: rollback and retry
await databaseDriver.Rollback();
context.Request.Body.Seek(0, SeekOrigin.Begin);
}
catch
{
// Unhandled error: rollback and throw
await databaseDriver.Rollback();
throw;
}
}
}
Unfortunately, this doesn't work properly because SQL serialization exeptions sometimes happen at the await databaseDriver.Commit() step, which is executed after the action returned successfully and started writing to the HTTP response stream. This results in duplicate JSON data in the response body.
What would be the best approach to solve this problem?
Let the API client reexecute the query (use a dedicated error code like HTTP 419) and never reexecute the ASP.NET action from a middleware. Using request buffering is a bad thing anyway and there might be other undesirable side effects when rerunning the MVC pipeline.
Commit the request transaction in each MVC action before it returns instead of doing so from the outer middleware.
Commit the transaction in a global action filter (only if no exception is thrown), which is run before the response stream is touched, thus avoiding the duplicate "commit" instruction in each action from the previous approach.
Somehow delay the ASP.NET MVC pipeline from writing to the response stream until the transaction is commited (is that even possible?).
Anything else.
I ended up solving this issue by resetting the response stream before each retry. This is normally not possible because the response stream is not seekable, but you can use a temporary MemoryStream to replace the response stream while the middleware is running:
public async Task InvokeAsync(HttpContext context, IDatabaseDriver databaseDriver)
{
context.Request.EnableBuffering(REQUEST_BUFFER_THRESHOLD);
Stream originalResponseBodyStream = context.Response.Body;
using var buffer = new MemoryStream();
context.Response.Body = buffer;
try
{
int attempt = 0;
while (true)
{
attempt++;
try
{
// Process request then commit transaction
await this._next(context);
await databaseDriver.Commit();
break;
}
catch (PostgresException ex)
when (ex.SqlState == PostgresErrorCodes.SerializationFailure &&
attempt <= MAX_RETRIES)
{
// SQL serialization failure: rollback and retry
await databaseDriver.Rollback();
context.Request.Body.Seek(0, SeekOrigin.Begin);
context.Response.Body.SetLength(0);
}
catch
{
// Unhandled error: rollback and throw
await databaseDriver.Rollback();
throw;
}
}
}
finally
{
context.Response.Body = originalResponseBodyStream;
buffer.Seek(0, SeekOrigin.Begin);
await buffer.CopyToAsync(context.Response.Body);
}
}
I am receiving error
taskcanceledexception a task was canceled without any inner exception details, and I am not receiving taskcanceled exception in Sentry. How can I see what the stack trace for this exception is or what changes I need to make to the code ?
Thanks
private T CallDiffbotAndDeserialise<T>(string baseUrl, string pageUrl, int maxTags, int minimumTagConfidencePercentage)
{
var client = diffBotConnection.GetClient();
client.BaseAddress = new Uri(baseUrl);
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
try
{
HttpResponseMessage response = client.GetAsync($"?token={settings.DiffBotToken}&maxTags={maxTags}&tagConfidence={minimumTagConfidencePercentage / 100}&url={Uri.EscapeDataString(pageUrl)}&ts={DateTime.Now.ToSafeCacheString()}").Result;
string responseString = response.Content.ReadAsStringAsync().Result;
T diffBotResponse = JsonConvert.DeserializeObject<T>(responseString);
return diffBotResponse;
}
catch (AggregateException e) // If the task is cancelled or times out
{
return default(T);
};
}
API connection:
public abstract class APIConnection : IDisposable
{
protected HttpClient Client;
private bool disposed = false;
protected APIConnection() : this(3000) { }
protected APIConnection(int timeOut)
{
Client = new HttpClient()
{
Timeout = TimeSpan.FromMilliseconds(timeOut)
};
}
public HttpClient GetClient() => Client;
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
protected virtual void Dispose(bool disposing)
{
if (!disposed)
{
if (disposing)
{
Client.Dispose();
}
disposed = true;
}
}
You are calling .Result which always throws AggregateException.
That means you are not only catching TaskCancelledException or OperationCancelledException, you'll catch anything thrown by the two calls to .Result.
Since you are handling the exception and hiding the fact it ever happened (by catch and returning) Sentry won't know about it. If you want to send that event to Sentry, you'd need to call the Sentry client manually.
With SharpRaven:
var ravenClient = new RavenClient("dsn"); // Initialize the client
ravenClient.CaptureEvent(new SentryEvent(exception));
With the new SDK Sentry is developing (which is still a preview release):
// Initialize the SDK only once, at the start of the app
using (SentrySdk.Init("dsn"))
{
SentrySdk.AddBreadcrumb($"Starting a web request to: {baseUrl}");
try
{
// make request
}
catch (Exception e)
{
SentrySdk.CaptureException(exception);
}
}
In this example I added a breadcrumb which in case of an event (for example capturing an exception explicitly like above) is sent together with the event.
Also note that the new SDK automatically detects exceptions that go unhandled. That is not the case of your exemple since you are explicitly catching it.
I think it's important to mention that ideally you would avoid blocking the thread by calling .Result and instead would use async/await.
The await keyword unwraps the Exception from the faulted Task.
That means that your catch block could now catch OperationCancelledException instead. Any other error like a total failure to connect to the server would not go into your catch block and instead would bubble up the stack.
I need to translate an error in an RX stream (IObservable) into an exception in the method that contains the subscription to the stream
(because of this issue https://github.com/aspnet/SignalR/pull/1331 , Whereby errors arent serialised to clients.) Once this issue is fixed I will revert to handling error properly
e.g.
I have the following method
public IObservable<StreamItem> LiveStream()
{
_mySvc.Start();
return _mySvc.ThingChanged();
}
So I have tried to subscribe to the stream and rethrow the error, but it still doesnt get transmitted to the client:
public IObservable<StreamItem> LiveStream()
{
_mySvc.Start();
_mySvc.ThingChanged().Subscribe(item => {}, OnError, () => {});
return _mySvc.ThingChanged();
}
private void OnError(Exception exception)
{
throw new Exception(exception.Message);
}
What I need is the equivelent of throwing in the LiveStream method
e.g. this error is propogated to the client
public IObservable<StreamItem> LiveStream()
{
_mySvc.Start();
throw new Exception("some error message");
return _mySvc.ThingChanged();
}
any ideas how to achieve this?
I have found this as well, especially with a "contained" reactive pipeline—that is, one with a well-defined beginning and end. In situations like those, it may suffice to simply allow underlying exceptions to bubble up to the containing scope. But as you have found, that concept is rather foreign to Rx generally: what happens in the pipeline stays in the pipeline.
The only way out of this that I have found in a contained scenario is to "slip" the error out of the stream using Catch(), and hand back an empty IObservable to allow the stream to halt naturally (otherwise, you'll hang if you're awaiting an IObservable for completion).
This will not work within your LiveStream() method, because that context/scope should have passed out of existence long before you're consuming your stream. So, this will have to happen in the context that contains the whole pipeline.
Exception error = null;
var source = LiveStream()
.Catch<WhatYoureStreaming, Exception>(ex => {error = ex; return Observable.Empty<WhatYoureStreaming>(); })
...
await source; // if this is how you're awaiting completion
// not a real exception type, use your own
if (error != null) throw new ContainingException("oops", error);
Just don't throw error there at the end, you'll lose the original stack trace.
Try this code:
public IObservable<StreamItem> LiveStream()
{
_mySvc.Start();
return
_mySvc
.ThingChanged()
.Materialize()
.Do(x =>
{
if (x.Kind == NotificationKind.OnError)
{
OnError(x.Exception);
}
})
.Dematerialize();
}
I'm not sure that this is the best way to go - throwing exceptions like this can cause you grief inside a stream where you end up with the wrong exception handlers firing. You might need to find another solution.
I have a common method that I'm using to handle a specific error that may come back from a number of functions:
protected async Task<T> RunMyMethod<T>(Func<T> method)
{
try
{
var returnValue = await Task.Run<T>(method);
return returnValue;
}
catch (MyCustomException)
{
// Force a clean shutdown of the software
ShutdownApplication();
return default(T);
}
}
Here's an example of how that is then used in a derived class:
private async Task<IEnumerable<MyData>> GetMyData()
{
var returnValue = await base.RunMyMethod<IEnumerable<MyData>>(() =>
{
var returnval = GetMyDataFromServer();
return returnval;
});
return returnValue;
}
When an exception of type MyCustomException occurs in GetMyDataFromServer() the software doesn't drop into the catch block. I get the following error in the function GetMyData():
An exception of type 'System.ServiceModel.FaultException`1' occurred in mscorlib.dll but was not handled in user code
Additional information: Exception of type 'MyCustomException' was thrown.
This is with only User-unhandled exceptions turned on.
GetMyDataFromServer() communicates with a WCF service. This service is what throws the error.
ChannelFactory<TChannel> cf = new ChannelFactory<TChannel>(endPointName);
Binding binding = new BasicHttpBinding(BasicHttpSecurityMode.TransportCredentialOnly);
var clientCredentials = new ClientCredentials();
. . .
channel = cf.CreateChannel();
var data = channel.CallWCFService();
Having looked around on-line, it appeared that the correct way to handle this was to change the base method as follows:
protected async Task<T> RunMyMethod<T>(Func<T> method)
{
var returnValue = await Task.Run<T>(method).ContinueWith(e =>
{
ShutdownApplication();
return default(T);
}, TaskContinuationOptions.OnlyOnFaulted);
return returnValue;
}
When I run this, I'm obviously not trapping for the correct error message, but I'm just getting a TaskCancellationException.
So, I have two questions: is my conclusion about how to handle this exception correct and, if so, how do I trap a specific error; and why am I getting a TaskCancellationException?
You get TaskCancellationException because the continuation is cancelled as it's conditional (i.e. TaskContinuationOptions.OnlyOnFaulted) and the condition isn't met since the antecedent task wasn't faulted.
There's no reason to use that method of adding a continuation. Using async-await like you did at the start is good enough (and even simpler).
The issue is that you are trying to catch MyCustomException but that isn't the exception being thrown. Since you're using WCF the exception is FaultException. You can check the "real" exception stored in FaultException.InnerException.
Is there anyway we can get HttpStatus code when exception caught? Exceptions could be Bad Request, 408 Request Timeout,419 Authentication Timeout? How to handle this in exception block?
catch (Exception exception)
{
techDisciplines = new TechDisciplines { Status = "Error", Error = exception.Message };
return this.Request.CreateResponse<TechDisciplines>(
HttpStatusCode.BadRequest, techDisciplines);
}
I notice that you're catching a generic Exception. You'd need to catch a more specific exception to get at its unique properties. In this case, try catching HttpException and examining its status code property.
However, if you are authoring a service, you may want to use Request.CreateResponse instead to report error conditions.
http://www.asp.net/web-api/overview/web-api-routing-and-actions/exception-handling has more information
I fell into the same trap when doing error handling in my WebAPI controllers. I did some research on best practices for exception handling and finally ended up with following stuff that works like a charm (hope it will will help :)
try
{
// if (something bad happens in my code)
throw new HttpResponseException(new HttpResponseMessage(HttpStatusCode.BadRequest) { Content = new StringContent("custom error message here") });
}
catch (HttpResponseException)
{
// just rethrows exception to API caller
throw;
}
catch (Exception x)
{
// casts and formats general exceptions HttpResponseException so that it behaves like true Http error response with general status code 500 InternalServerError
throw new HttpResponseException(new HttpResponseMessage(HttpStatusCode.InternalServerError) { Content = new StringContent(x.Message) });
}