Develop Reference

debug .net library using HttpClient? (Get request hangs/time out)

Update/Edit
I pull their repo build it and step through it, I find the HttpClient is simply timing out here
var result = Await client.SendAsync(request) //Times out after a while
If i change it to a blocking call, it works
var result = Await client.SendAsync(request).GetAwaiter().GetResult() //Works
The next call, gets the results Async, and it works as expected
var body = await result.Content.ReadAsStringAsync(); //Works if the Send is synchronous
Why would sending the request Async timeout, yet performing the send blocking and then reading the result async work? I don't quite understand what is happening here?
Original Post
I'm struggling to debug an issue i'm facing using a 3rd party library that handles the HTTP requests to their API. I'm targeting .Net 4.6.1 - It's a mess of a web project with both WebApi and webforms.
The package is using HttpClient() for it's requests (Postmark.Net)
PostMark Package URL
https://github.com/wildbit/postmark-dotnet/tree/71c05efafd266a4403f8e6c18a8b485252082fd3
Issue
When I run the project, the application just appears to hang. It does not timeout and return back to the caller, it just appears stuck. In my debug window it just shows the threads exiting with code 0.
Sample Code
Dim cc = New PostmarkDotNet.PostmarkClient("key")
Dim msg = cc.GetOutboundMessageDetailsAsync("some-id").GetAwaiter.GetResult()
Log.Debug(msg.Status.ToString & " - " & msg.ReceivedAt.ToString() & " - " & msg.To.FirstOrDefault.Email.ToString())
Note: Intentionally using getAwaiter to block the thread to help troubleshoot.
What I've Tried:
Created a new .net project targeting the same framework version (4.7.1) - The library and code functions as expected; so it must be related to this project
Clean / Rebuild the project
Running Fiddler to see what is happening on the wire - i see an outbound request with a response with the SSL cert from the API - then nothing. I'm thinking perhaps some issue is occurring around the SSL handshake - however, i doubt it is a TLS issue since a test project on same machine worked as expected.
Further Questions:
How can I debug or step into the library itself to see what is happening there?
The package requires .Net.Http 4.3.4, but in my web.config i'm showing a bindingRedirect to 4.2.0.0, but in my test project i threw together seeing the same thing and not having an issue.
Suggestions?

Adding Cipher suite to TLS1.2 of HttpClient of dotnetcore 3.1

I encounter the folowing exception when connecting to the website of Western digital:
website of Western digital
22:02:34,803 | HttpGrabber | DEBUG | Grabbing: GET https://shop.westerndigital.com/de-de/products/internal-drives/wd-red-sata-2-5-ssd#WDS200T1R0A
22:02:34,858 | HttpGrabber | DEBUG | System.Net.Http.SocketsHttpHandler.Http2Support: True
22:02:34,865 | HttpGrabber | DEBUG | System.Net.Http.UseSocketsHttpHandler: True
22:02:35,067 | HttpGrabber | ERROR | System.AggregateException: One or more errors occurred. (The SSL connection could not be established, see inner exception.)
---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
---> System.ComponentModel.Win32Exception (0x80090326): Le message reçu était inattendu ou formaté de façon incorrecte.
--- End of inner exception stack trace ---
I think that the C# code is correct as I have 3/4 Unit test Passing:
[TestCase("https://allianz-fonds.webfg.net/sheet/fund/FR0013192572/730?date_entree=2018-04-04")]
[TestCase("https://www.galaxus.de/de/s1/product/zotac-zbox-magnus-en72070v-intel-core-i7-9750h-0gb-pc-13590721")]
[TestCase("https://www.hystou.com/Gaming-Mini-PC-F7-with-Nvidia-GeForce-GTX-1650-p177717.html")]
[TestCase("https://shop.westerndigital.com/de-de/products/internal-drives/wd-red-sata-2-5-ssd#WDS200T1R0A")]
The SSL diagnostic done by ssllabs gives a list of supported cipher suites handled by the website of Western digital:
Firefox connects succesfully to the website, and Wireshark spots that firefox has 1 cipher in the list:
However my dotnet core application has a fatal in the ssl handshake because it has not a single cipher common with WD :
I took a lot of time to understand that the error comes from here.... if it really comes from here.
Hence 2 questions comes from this analysis:
Is it possible to add a cipher suite in my dot net core 3.1 application , written in C# to be compliant with this website ?
I have seen discussion on internet stipulating that maybe the Us company which is Microsoft is not allowed to export strong cryptographic algorithms... if this is true, what about firefox (Usa too) using the same suite as Western digital (Usa too).
Is there a possibility to use in C# another library ( I think about open SSl) but the other library does provide all layers of https (ie propose equivalent of httpClient) / what about crossplatform to avoid loosing the cross platform feature of dotnetcore ....
Remark: Even Fiddler has this problem ! Which is understandable as it is also relying on the dot net framework technology:
To answer comment of #Steffen Ullrich I run this stuff on Win7:

I had the same issue, my autotests (dotnetcore3.1) are run on the WS 2012 R2 machines and I have to call third-party API which accepts only TWO cipher suites: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030).
C# HttpClient relies on cipher suites in the host system, in contrast to Chrome, Firefox, and Curl which have their own Security and Cryptography systems. WS 2012 R2 doesn't have those two ciphers and I know no way how to add them to the machines, there are no windows updates with those ciphers.
I've chosen using a pretty cool NuGet packet CurlThin as the solution. Using it we can set up our own cipher suites for requests, so we need to do nothing with the server side.
I've installed two packets: CurlThin itself and CurlThin.Native.
The resulting code for a GET request with a header, to an HTTPS endpoint, looks like this:
using CurlThin;
using CurlThin.Enums;
using CurlThin.Helpers;
using CurlThin.Native;
using CurlThin.SafeHandles;
using System.Text;
private static string GetToken()
{
//This string is for extracting libcurl and ssl libs to the bin directory.
CurlResources.Init();
var global = CurlNative.Init();
var easy = CurlNative.Easy.Init();
string content;
try
{
var dataCopier = new DataCallbackCopier();
CurlNative.Easy.SetOpt(easy, CURLoption.URL, "https://someendpoints.net/thatendpoint?fake=true");
CurlNative.Easy.SetOpt(easy, CURLoption.WRITEFUNCTION, dataCopier.DataHandler);
//This string is needed when you call a https endpoint.
CurlNative.Easy.SetOpt(easy, CURLoption.CAINFO, CurlResources.CaBundlePath);
var headers = CurlNative.Slist.Append(SafeSlistHandle.Null, "Authorization: Bearer blablabla");
CurlNative.Easy.SetOpt(easy, CURLoption.HTTPHEADER, headers.DangerousGetHandle());
//Your set of ciphers, full list is here https://curl.se/docs/ssl-ciphers.html
CurlNative.Easy.SetOpt(easy, CURLoption.SSL_CIPHER_LIST, "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256");
CurlNative.Easy.Perform(easy);
content = Encoding.UTF8.GetString(dataCopier.Stream.ToArray());
}
finally
{
easy.Dispose();
if (global == CURLcode.OK)
CurlNative.Cleanup();
}
return content;
}

.NET Core uses the ciphers supported by the native TLS stack, i.e. SChannel. Which ciphers are supported depend on the version of Windows. Which ciphers are supported by your OS (is documented in TLS Cipher Suites in Windows 7. As you can see, none of the ciphers offered by the server are supported by your OS.
With Firefox or Chrome browser the situation is different. These come with their own stack and are thus not limited on what the OS offers. That's why they work.

Indeed the same code PASS succefully when executed on Windows 10:

Kestrel request per second issue

i'm a newbie to asp.net core
i'm write a web api service, which store passed data to database. in theory there is about 300-400 request per second to server in future and response time must be less than 10 seconds
but first of all i try to run some load test with locust.
i write simple app with one controller and only one post method which simple return Ok() without any processing.
i try to create load to this service for 1000 users. my service run under ubuntu 16.04 with .net core 2.1 (2 Xeon 8175M with 8 GB of RAM). Locust run from dedicated computer
but i see only ~400 RPS and response time about 1400 ms. For empty action it is very big value.
i'm turn off all loging, run in production mode but no luck - still ~400 rps.
in system monitor (i use nmon) i see that both cpu loads only for 12-15% (total 24-30%). I have about 3 GB free ram, no network usage (about 200-300 KB/s), no disk usage, so system have hardware resource for handling request.
so i think, that there is problem with some configuration or may be with system resource like sockets, handles etc
i also try to use libuv instead of managed socket, but result is same
in kestrel configuration i setup explicitly Limit.MaxConnection and MaxUpgradedConnection to null (but it is default value)
so, i have two question:
- in theory, can kestrel provide high rps?
- if first is true, can you give me some advise for start point (links, articles and so on)

Azure ASP .net WebApp The request timed out

I have deployed an ASP .net MVC web app to Azure App service.
I do a GET request from my site to some controller method which gets data from DB(DbContext). Sometimes the process of getting data from DB may take more than 4 minutes. That means that my request has no action more than 4 minutes. After that Azure kills the connection - I get message:
500 - The request timed out. The web server failed
to respond within the specified time.
This is a method example:
[HttpGet]
public async Task<JsonResult> LongGet(string testString)
{
var task = Task.Delay(360000);
await task;
return Json("Woke", JsonRequestBehavior.AllowGet);
}
I have seen a lot of questions like this, but I got no answer:
Not working 1
Cant give other link - reputation is too low.
I have read this article - its about Azure Load Balancer which is not available for webapps, but its written that common way of handling my problem in Azure webapp is using TCP Keep-alive. So I changed my method:
[HttpGet]
public async Task<JsonResult> LongPost(string testString)
{
ServicePointManager.SetTcpKeepAlive(true, 1000, 5000);
ServicePointManager.MaxServicePointIdleTime = 400000;
ServicePointManager.FindServicePoint(Request.Url).MaxIdleTime = 4000000;
var task = Task.Delay(360000);
await task;
return Json("Woke", JsonRequestBehavior.AllowGet);
}
But still get same error.
I am using simple GET request like
GET /Home/LongPost?testString="abc" HTTP/1.1
Host: longgetrequest.azurewebsites.net
Cache-Control: no-cache
Postman-Token: bde0d996-8cf3-2b3f-20cd-d704016b29c6
So I am looking for the answer what am I doing wrong and how to increase request timeout time in Azure Web app. Any help is appreciated.
Azure setting on portal:
Web sockets - On
Always On - On
App settings:
SCM_COMMAND_IDLE_TIMEOUT = 3600
WEBSITE_NODE_DEFAULT_VERSION = 4.2.3

230 seconds. That's it. That's the in-flight request timeout in Azure App Service. It's hardcoded in the platform so TCP keep-alives or not you're still bound by it.
Source -- see David Ebbo's answer here:
https://social.msdn.microsoft.com/Forums/en-US/17305ddc-07b2-436c-881b-286d1744c98f/503-errors-with-large-pdf-file?forum=windowsazurewebsitespreview
There is a 230 second (i.e. a little less than 4 mins) timeout for requests that are not sending any data back. After that, the client gets the 500 you saw, even though in reality the request is allowed to continue server side.
Without knowing more about your application it's difficult to suggest a different approach. However what's clear is that you do need a different approach --
Maybe return a 202 Accepted instead with a Location header to poll for the result later?

I just changed my Azure Web Site from Shared Enviroment to Standard, and it works.

Trying to get SuperWebSocket server working under mono/c#

I was using Nugget before (http://nugget.codeplex.com/), but when I upgraded to Chrome 14, it stopped working, and that's when I found this project: http://superwebsocket.codeplex.com/.
I've read this bit here: Is superwebsocket available in asp.net by default?, and I've also built the mono version of SuperSocket. Connecting with Chrome 14, and the websocket just shows as pending in Chrome.
I see a message like this in the server side logfile:
INFO 2011-09-22 23:44:03,924 17166ms uer - WebSocket Server - Session: 1a2dc865-9d02-468e-ac7b-26f3d0b96a2a/127.0.0.1:49261
New SocketSession was accepted!
But the WebSocketServer.NewSessionConnected event never fires. I do see the WebSocketServer.SessionClosed event firing however.
Anyone have any ideas why the new session connection event never fires, and/or why Chrome never receives any response from the socket server?

Which version of SuperWebSocket are you using?
The latest source support Chrome 14, but there is no drop for it for now.
So you should download source code by yourself.
Another problem is SuperSocket, SuperSocket 1.4 SP1 which SuperWebSocket base on has no MONO assemblies in release package, so you need build it by yourself.