Develop Reference

IOException: Cannot determine the frame size or a corrupted frame was received

Please explain why this doesn't work with "https://localhost:5001" but works correctly with "https://localhost:44386". If you will send a link to any documentation, it will be large plus.
2 asp.net core 5.0 WebAPI applications
one application sends an http request to another
throwing an exception in the first option
public class SomeHttpClient: ISomeHttpClient
{
private readonly HttpClient _httpClient;
public ServerHealthCheckServiceHttpClient(HttpClient httpClient, IOptions<SomeOptions> options)
{
_httpClient = httpClient;
_httpClient.BaseAddress = options.Value.SomeAddress;
}
public async Task<bool> DoSomething(CancellationToken token)
{
var request = new HttpRequestMessage(HttpMethod.Get, "someMethod");
var response = await _httpClient.SendAsync(request, token);
}
}
First variant error System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. ---> System.IO.IOException: Cannot determine the frame size or a corrupted frame was received.

Change port number. That will unblock you

Polly HandleTransientHttpError not catching HttpRequestException

I've created a retry policy on my HttpClient in the Startup.ConfigureServices method. Note also that by default, asp.net core 2.1 logs 4 [Information] lines for each call made by the HttpClient which are shows in the logs at the end of my question.
services.AddHttpClient("ResilientClient")
.AddPolicyHandler(
Policy.WrapAsync(
PollyRetryPolicies.TransientErrorRetryPolicy(),
Policy.TimeoutAsync<HttpResponseMessage>(TimeSpan.FromSeconds(60))));
The policy is defined as follows. Note that I write the retry attempt to logs, so I will know if the retry policy is invoked.
public static IAsyncPolicy < HttpResponseMessage > TransientErrorRetryPolicy() {
return HttpPolicyExtensions
.HandleTransientHttpError()
.Or < TimeoutRejectedException > ()
.WaitAndRetryAsync(sleepDurations: ExponentialBackoffPolicy.DecorrelatedJitter(3, SEED_DELAY, MAX_DELAY),
onRetry: (message, timespan, attempt, context) => {
context.GetLogger() ? .LogInformation($ "Retrying request to {message?.Result?.RequestMessage?.RequestUri} in {timespan.TotalSeconds} seconds. Retry attempt {attempt}.");
});
}
HandleTransientHttpError() is a Polly extension that states in it's comments:
The conditions configured to be handled are:
• Network failures (as System.Net.Http.HttpRequestException)
My httpclient usage is like this:
using (HttpResponseMessage response = await _httpClient.SendAsync(request))
{
response.EnsureSuccessStatusCode();
try
{
string result = await response.Content.ReadAsStringAsync();
if (result == null || result.Trim().Length == 0) {
result = "[]";
}
return JArray.Parse(result);
} catch (Exception ex) {
_logger.LogInformation($ "Failed to read response from {url}. {ex.GetType()}:{ex.Message}");
throw new ActivityException($ "Failed to read response from {url}.", ex);
}
}
The following logs are captured:
[Information] System.Net.Http.HttpClient.ResilientClient.LogicalHandler: Start processing HTTP request GET https://api.au.... obfuscated
[Information] System.Net.Http.HttpClient.ResilientClient.CustomClientHandler: Sending HTTP request GET https://api.au..... obfuscated
[Information] System.Net.Http.HttpClient.ResilientClient.CustomClientHandler: Received HTTP response after 2421.8895ms - 200
[Information] System.Net.Http.HttpClient.ResilientClient.LogicalHandler: End processing HTTP request after 2422.1636ms - OK
Unknown error responding to request: HttpRequestException:
System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException: The server returned an invalid or unrecognized response.
at System.Net.Http.HttpConnection.FillAsync()
at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.CopyToAsyncCore(Stream destination, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.HttpConnectionResponseContent.SerializeToStreamAsync(Stream stream, TransportContext context, CancellationToken cancellationToken)
at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
--- End of inner exception stack trace ---
at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at nd_activity_service.Controllers.ActivityController.GetND(String url) in /codebuild/output/src251819872/src/src/nd-activity-service/Controllers/ActivityController.cs:line 561
The Http call succeeds, and I can see it returns 200 - OK. But then the HttpRequestException is thrown. I assume the policy is not being invoked because the HttpClient message pipeline has already resolved, as we can see it returned 200 - OK. So how is it throwing an exception outside of this?
And how do I handle it? Wrap another policy around the method that handles HttpRequestExceptions specifically?
This error does appear to be transient. It is a scheduled job and works the next time it is called.

Your policy is defined against the HttpClient not against the HttpResponseMessage.
So, the response.EnsureSuccessStatusCode() will not trigger retry even if you receive for example 428.
The HandleTransientHttpError will trigger retry if you receive 408 or 5XX status codes from the downstream system. And when the SendAsync throws the HttpRequestException
Because your exception StackTrace looks like this:
System.Net.Http.HttpRequestException: Error while copying content to a stream.
System.IO.IOException: The server returned an invalid or
unrecognized response.
that's why my educated guess is that this exception is thrown by the HttpContent class while you try to read the response body (ReadAsStringAsync).
This will not trigger retry since you have defined your policy on the HttpClient.
If you want to retry in those cases as well when either the response.EnsureSuccessStatusCode() throws HRE or when the response.Content.ReadAsStringAsync() does then you have to wrap your whole http communication and response processing logic into a retry policy.
Let me show you how to do that.
First use a PolicyRegistry instead of AddPolicyHandler:
//services.AddHttpClient("ResilientClient")
// .AddPolicyHandler(
// Policy.WrapAsync(
// TransientErrorRetryPolicy(),
// Policy.TimeoutAsync<HttpResponseMessage>(TimeSpan.FromSeconds(60))));
services.AddHttpClient("ResilientClient");
var registry = services.AddPolicyRegistry();
registry.Add("retry", Policy.WrapAsync(
TransientErrorRetryPolicy(),
Policy.TimeoutAsync<HttpResponseMessage>(TimeSpan.FromSeconds(60))));
Then ask the DI for the register, for example:
private readonly IHttpClientFactory factory;
private readonly IReadOnlyPolicyRegistry<string> registry;
public TestController(IHttpClientFactory factory, IReadOnlyPolicyRegistry<string> registry)
{
this.factory = factory;
this.registry = registry;
}
Finally retrieve the combined policy and execute the http call:
var retryPolicy = registry.Get<IAsyncPolicy<HttpResponseMessage>>("retry");
await retryPolicy.ExecuteAsync(async () => await IssueRequest());
private async Task<HttpResponseMessage> IssueRequest()
{
var _httpClient = factory.CreateClient("ResilientClient");
HttpResponseMessage response = await _httpClient.GetAsync("http://httpstat.us/428");
response.EnsureSuccessStatusCode();
return response;
}
I've used the httpstat.us to simulate 428 response.

gRPC client in C# does not work with gRPC server in Go with mTLS support

I have a gRPC server in Golang that has mTLS enabled using the following ServerOptions:
// getServerOptions returns a list of GRPC server options.
// Current options are TLS certs and opencensus stats handler.
func (h *serviceHandler) getServerOptions() []grpc.ServerOption {
tlsCer, err := tls.LoadX509KeyPair(tlsDir+"tls.crt", tlsDir+"tls.key")
if err != nil {
logger.WithError(err).Fatal("failed to generate credentials")
}
cfg := &tls.Config{
Certificates: []tls.Certificate{tlsCer},
ClientAuth: tls.RequireAndVerifyClientCert,
GetConfigForClient: func(*tls.ClientHelloInfo) (*tls.Config, error) {
h.certMutex.RLock()
defer h.certMutex.RUnlock()
return &tls.Config{
Certificates: []tls.Certificate{tlsCer},
ClientAuth: tls.RequireAndVerifyClientCert,
ClientCAs: h.caCertPool,
}, nil
},
}
// Add options for creds and OpenCensus stats handler to enable stats and tracing.
return []grpc.ServerOption{grpc.Creds(credentials.NewTLS(cfg)), grpc.StatsHandler(&ocgrpc.ServerHandler{})}
}
The server works fine for a gRPC client in Golang, but fails for the following gRPC c# client after the cert exchange handshake.
static async Task Main(string[] args)
{
string baseAddress = "x.x.x.x";
var x509Cert = new X509Certificate2("client.pfx", "123");
var client = CreateClientWithCert("https://" + baseAddress + ":443", x509Cert);
try {
var response = await client.PostAllocateAsync(new AllocationRequest {Namespace = "Default"});
Console.Write(response.State.ToString());
}
catch(RpcException e)
{
Console.WriteLine($"gRPC error: {e.Status.Detail}");
}
catch
{
Console.WriteLine($"Unexpected error calling agones-allocator");
throw;
}
}
public static AllocationService.AllocationServiceClient CreateClientWithCert(
string baseAddress,
X509Certificate2 certificate)
{
var loggerFactory = LoggerFactory.Create(logging =>
{
logging.AddConsole();
logging.SetMinimumLevel(LogLevel.Trace);
});
// Add client cert to the handler
var handler = new HttpClientHandler();
handler.ClientCertificates.Add(certificate);
handler.ServerCertificateCustomValidationCallback =
HttpClientHandler.DangerousAcceptAnyServerCertificateValidator;
// Create the gRPC channel
var channel = GrpcChannel.ForAddress(baseAddress, new GrpcChannelOptions
{
HttpClient = new HttpClient(handler),
LoggerFactory = loggerFactory,
});
return new AllocationService.AllocationServiceClient(channel);
}
}
Here is the trace log:
dbug: Grpc.Net.Client.Internal.GrpcCall[1]
Starting gRPC call. Method type: 'Unary', URI: 'https://x.x.x.x/v1alpha1.AllocationService/PostAllocate'.
dbug: Grpc.Net.Client.Internal.GrpcCall[18]
Sending message.
trce: Grpc.Net.Client.Internal.GrpcCall[21]
Serialized 'V1Alpha1.AllocationRequest' to 9 byte message.
trce: Grpc.Net.Client.Internal.GrpcCall[19]
Message sent.
fail: Grpc.Net.Client.Internal.GrpcCall[6]
Error starting gRPC call.
System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.IO.IOException: The response ended prematurely.
at System.Net.Http.HttpConnection.FillAsync()
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request)
dbug: Grpc.Net.Client.Internal.GrpcCall[8]
gRPC call canceled.
fail: Grpc.Net.Client.Internal.GrpcCall[3]
Call failed with gRPC error status. Status code: 'Internal', Message: 'Error starting gRPC call: An error occurred while sending the request.'.
dbug: Grpc.Net.Client.Internal.GrpcCall[4]
Finished gRPC call.
gRPC error: Error starting gRPC call: An error occurred while sending the request.
Can someone please help me understand what is the reason for the failure? Using SslCredentials also fails.
x.x.x.x is the IP replacement for the privacy reasons.

I wrote some sample codes to reproduce the cross platform incompatibility issue reported here.
The root cause is when using GetConfigForClient in golang server, which in this case is used to refresh the client CA certificates, the requests from C# client fails. However, when it was replaced with VerifyPeerCertificate, the issue got resolved.

WCF HandleError System.InvalidOperationException A task may only be disposed if it is in a completion state

I am stuck on an issue that may be conceptual or something simple I am missing. I have a Desktop application that is making an async call to the server.
This involves multiple nested calls through several classes. I have kept it async the whole way down the call chain.
The main issue which I have been able to reproduce consistently is when I am passing an HTTP client to another service which makes an async Post.
The code snippets aren't %100 correct as is, just trying to show basic
public async Task<bool> SomeTask(IEnumerable<Dto> dtoList)
{
var response = await _corepointService.PostCore(request, _idbContext);
if (response.StatusCode == HttpStatusCode.BadRequest)
{
success = false;
}
}
return success;
}
public Task<HttpResponseMessage> PostCore(Dto request, IDBContext
dbContext, HttpClient _httpClient)
{
var dto = new HttpRequestMessage
{
RequestUri = new Uri(request.Resource),
Method = HttpMethod.Post,
Content = new StringContent(request.Body, Encoding.UTF8,
"application/json")
};
return _httpClient.SendAsync(dto);
}
This is the other alternative I have tried, not passingin the HttpClient and creating a new one.
public Task<HttpResponseMessage> PostCore(Dto request, IDBContext
dbContext)
{
var _httpClient = new HttpClient();
var dto = new HttpRequestMessage
{
RequestUri = new Uri(request.Resource),
Method = HttpMethod.Post,
Content = new StringContent(request.Body, Encoding.UTF8,
"application/json")
};
return _httpClient.SendAsync(dto);
}
The error I am getting is
WCF HandleError - System.InvalidOperationException: A task may only be disposed if it is in a completion state (RanToCompletion, Faulted or Canceled).
at System.Threading.Tasks.Task.Dispose(Boolean disposing)
at System.Threading.Tasks.Task.Dispose()
at System.ServiceModel.Dispatcher.MessageRpc.DisposeParametersCore(Boolean excludeInput)
WCF HandleError - System.InvalidOperationException: A task may only be disposed if it is in a completion state (RanToCompletion, Faulted or Canceled).
at System.Threading.Tasks.Task.Dispose(Boolean disposing)
at System.Threading.Tasks.Task.Dispose()
at System.ServiceModel.Dispatcher.MessageRpc.DisposeParametersCore(Boolean excludeInput)
WCF HandleError - System.InvalidOperationException: A task may only be disposed if it is in a completion state (RanToCompletion, Faulted or Canceled).
at System.Threading.Tasks.Task.Dispose(Boolean disposing)
at System.Threading.Tasks.Task.Dispose()
at System.ServiceModel.Dispatcher.MessageRpc.DisposeParametersCore(Boolean excludeInput)
I am consistenly getting this error when the Async Post is being made in the final service. I am just not sure if it is even related, or is some kind of race relation. Any help or links to async information is much appreciated. I have tried to find any related posts on SO and didn't find anything specifically regarding this.
Thanks!

Please ensure the following:
async in PostCore method definition
public async Task<HttpResponseMessage> PostCore(Dto request,
IDBContext dbContext, HttpClient _httpClient){}
and 'await' the SendAsync call
return await _httpClient.SendAsync(dto);
Hope this helps!

This was caused by a wrapper class in the Desktop client that was returning a task and expecting the method it was awaiting to be synchronous.
It was unrelated to the HttpPost or any of the async code in the server itself.

Custom DelegatingHandler never return after await base.SendAsync

I'm building an API consumer using HttpClient. Because the provider require the consumer to authenticate using Digest Authentication, so I need to write a custom DelegatingHandler like below:
public class DigestAuthDelegatingHandler : DelegatingHandler
{
public DigestAuthDelegatingHandler(HttpMessageHandler innerHandler) : base(innerHandler) { }
protected async override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var response = await base.SendAsync(request, cancellationToken);
if (!response.IsSuccessStatusCode && response.StatusCode == HttpStatusCode.Unauthorized)//This line of code is never reached
{
//Generate the Digest Authorization header string and add to the request header,
//then try to resend the request to the API provider
}
return response;
}
}
I create a HttpClient and add my custom DelegatingHandler to the message handlers pineline
HttpClient httpClient = new HttpClient(new DigestAuthDelegatingHandler(new HttpClientHandler()));
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
httpClient.BaseAddress = new Uri("http://127.0.0.1/");
HttpResponseMessage response = httpClient.GetAsync("api/getTransactionInfo?TransactionNumber=1000).Result;
After doing that, it look like that my consumer runs forever. When I add a break point AFTER the code line await base.SendAsync above, I see that the code will never return, so I have no way to check if the response is get an 401 unauthorized to extract the Digest authorization header details. Nothing wrong at the API provider because I've successfully built another API consumer site using the traditional WebHttpRequest support Digest Authenticate and it works well.
IMPORTANT NOTE: if I switch to write consumer as a Console Application then it works well. So, I'm not sure but I think it's problem related to ASP.NET thread when running in asynchronous mode?
Is there anything wrong I'm doing?

I agree with Darrel - most probably, task is faulted and there is no result... you can use explicit continuation to inspect task state - for example,
return base.SendAsync(request, cancellationToken).ContinueWith(task =>
{
// put the code to check the task state here...
});
On different note, I am not sure if you need to create a custom DelegatingHandler for authenticating ... try using HttpClientHandler with Credentials properties (or UseDefaultCredentials to pass default credentials of current user)
var httpClient = new HttpClient(new HttpClientHandler() {
PreAuthenticate = true,
Credentials = new NetworkCredentials(...
});
EDIT: Found the example with digest authentication being used with http client using credential cache - see this SO Q & A: HttpRequestMessage and Digest Authentication
This should solve your actual problem w.r.t. digest authentication without building your own handler.

My guess is the .Result is blocking the continuation in your handler. Try changing the .Result to a .ContinueWith

Having run into this same issue, this is what ended up working for me. The problem was that the endpoint threw a 500 Internal Server Error and thus blocked the thread.
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
return await base.SendAsync(request, cancellationToken)
.ContinueWith<HttpResponseMessage>(task =>
{
return task.Result;
});
}
Note that the return inside the anon function returns to the SendAsync function, and then we actually return the Result.