Develop Reference

Adding Cipher suite to TLS1.2 of HttpClient of dotnetcore 3.1

I encounter the folowing exception when connecting to the website of Western digital:
website of Western digital
22:02:34,803 | HttpGrabber | DEBUG | Grabbing: GET https://shop.westerndigital.com/de-de/products/internal-drives/wd-red-sata-2-5-ssd#WDS200T1R0A
22:02:34,858 | HttpGrabber | DEBUG | System.Net.Http.SocketsHttpHandler.Http2Support: True
22:02:34,865 | HttpGrabber | DEBUG | System.Net.Http.UseSocketsHttpHandler: True
22:02:35,067 | HttpGrabber | ERROR | System.AggregateException: One or more errors occurred. (The SSL connection could not be established, see inner exception.)
---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
---> System.ComponentModel.Win32Exception (0x80090326): Le message reçu était inattendu ou formaté de façon incorrecte.
--- End of inner exception stack trace ---
I think that the C# code is correct as I have 3/4 Unit test Passing:
[TestCase("https://allianz-fonds.webfg.net/sheet/fund/FR0013192572/730?date_entree=2018-04-04")]
[TestCase("https://www.galaxus.de/de/s1/product/zotac-zbox-magnus-en72070v-intel-core-i7-9750h-0gb-pc-13590721")]
[TestCase("https://www.hystou.com/Gaming-Mini-PC-F7-with-Nvidia-GeForce-GTX-1650-p177717.html")]
[TestCase("https://shop.westerndigital.com/de-de/products/internal-drives/wd-red-sata-2-5-ssd#WDS200T1R0A")]
The SSL diagnostic done by ssllabs gives a list of supported cipher suites handled by the website of Western digital:
Firefox connects succesfully to the website, and Wireshark spots that firefox has 1 cipher in the list:
However my dotnet core application has a fatal in the ssl handshake because it has not a single cipher common with WD :
I took a lot of time to understand that the error comes from here.... if it really comes from here.
Hence 2 questions comes from this analysis:
Is it possible to add a cipher suite in my dot net core 3.1 application , written in C# to be compliant with this website ?
I have seen discussion on internet stipulating that maybe the Us company which is Microsoft is not allowed to export strong cryptographic algorithms... if this is true, what about firefox (Usa too) using the same suite as Western digital (Usa too).
Is there a possibility to use in C# another library ( I think about open SSl) but the other library does provide all layers of https (ie propose equivalent of httpClient) / what about crossplatform to avoid loosing the cross platform feature of dotnetcore ....
Remark: Even Fiddler has this problem ! Which is understandable as it is also relying on the dot net framework technology:
To answer comment of #Steffen Ullrich I run this stuff on Win7:

I had the same issue, my autotests (dotnetcore3.1) are run on the WS 2012 R2 machines and I have to call third-party API which accepts only TWO cipher suites: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030).
C# HttpClient relies on cipher suites in the host system, in contrast to Chrome, Firefox, and Curl which have their own Security and Cryptography systems. WS 2012 R2 doesn't have those two ciphers and I know no way how to add them to the machines, there are no windows updates with those ciphers.
I've chosen using a pretty cool NuGet packet CurlThin as the solution. Using it we can set up our own cipher suites for requests, so we need to do nothing with the server side.
I've installed two packets: CurlThin itself and CurlThin.Native.
The resulting code for a GET request with a header, to an HTTPS endpoint, looks like this:
using CurlThin;
using CurlThin.Enums;
using CurlThin.Helpers;
using CurlThin.Native;
using CurlThin.SafeHandles;
using System.Text;
private static string GetToken()
{
//This string is for extracting libcurl and ssl libs to the bin directory.
CurlResources.Init();
var global = CurlNative.Init();
var easy = CurlNative.Easy.Init();
string content;
try
{
var dataCopier = new DataCallbackCopier();
CurlNative.Easy.SetOpt(easy, CURLoption.URL, "https://someendpoints.net/thatendpoint?fake=true");
CurlNative.Easy.SetOpt(easy, CURLoption.WRITEFUNCTION, dataCopier.DataHandler);
//This string is needed when you call a https endpoint.
CurlNative.Easy.SetOpt(easy, CURLoption.CAINFO, CurlResources.CaBundlePath);
var headers = CurlNative.Slist.Append(SafeSlistHandle.Null, "Authorization: Bearer blablabla");
CurlNative.Easy.SetOpt(easy, CURLoption.HTTPHEADER, headers.DangerousGetHandle());
//Your set of ciphers, full list is here https://curl.se/docs/ssl-ciphers.html
CurlNative.Easy.SetOpt(easy, CURLoption.SSL_CIPHER_LIST, "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256");
CurlNative.Easy.Perform(easy);
content = Encoding.UTF8.GetString(dataCopier.Stream.ToArray());
}
finally
{
easy.Dispose();
if (global == CURLcode.OK)
CurlNative.Cleanup();
}
return content;
}

.NET Core uses the ciphers supported by the native TLS stack, i.e. SChannel. Which ciphers are supported depend on the version of Windows. Which ciphers are supported by your OS (is documented in TLS Cipher Suites in Windows 7. As you can see, none of the ciphers offered by the server are supported by your OS.
With Firefox or Chrome browser the situation is different. These come with their own stack and are thus not limited on what the OS offers. That's why they work.

Indeed the same code PASS succefully when executed on Windows 10:

.net httpclient on Azure w/ TLS/SNI: The request was aborted: Could not create SSL/TLS secure channel

Similar questions have been asked multiple times before - but before flagging this one as duplicate please read on. Most of these questions are very old. I have worked through a lot of questions and answers and did not find a suitable solution.
We have an Azure Cloud Service project in .net 4.5. It connects to dozens of our customers' APIs (not necessarily cloud hosted) without any problems, but a single API fails with this error message:
The request was aborted: Could not create SSL/TLS secure channel
What am I missing here?
This is the code (slightly condensed) that I am using to connect to the API (this runs per API, so the base URL does not change):
ServicePointManager.ServerCertificateValidationCallback += ValidateRemoteCertificate;
ServicePointManager.Expect100Continue = true;
ServicePointManager.SecurityProtocol =
SecurityProtocolType.Tls12 |
SecurityProtocolType.Tls11 |
SecurityProtocolType.Tls |
SecurityProtocolType.Ssl3;
ApiClient = HttpClientFactory.Create();
ApiClient.DefaultRequestHeaders.Authorization = null;
ApiClient.DefaultRequestHeaders.TryAddWithoutValidation("Authorization", "Basic {passwordToken}");
ApiClient.DefaultRequestHeaders.Accept.Clear();
ApiClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
SemaphoreSlim throttler = new SemaphoreSlim(initialCount: 50);
var sp = ServicePointManager.FindServicePoint(new Uri(baseUrl));
sp.SetTcpKeepAlive(true, 30000, 30000);
foreach (var request in urls)
{
Result = new HttpResponseMessage();
Result = await ApiClient.GetAsync(url);
...
}
This is what makes it hard to debug:
This problem only occurs in production, i.e. when running as an Azure Cloud Service. Not when debugging locally.
It only occurs with requests sent through HttpClient. Not with WebClient.
Further research (comparing the APIs) revealed that this API is the only one that has enabled SNI and ONLY supports TLS1.2.
Suggestions considered from other questions/answers regarding SNI in .net Framework:
To prevent misunderstandings: This is about the cloud service connecting to an API, not about a connection that is being made to the cloud service.
The HttpClient instance is being reused for all requests to a single API. (This is important as this answer suggests that the SNI tag will be created with the domain HttpClient has been initialized with). I have also tried configuring TLS after the Factory instantiated the HttpClient. No change.
The certificates are valid of course. No self-signed certificates but regular trusted ones off the shelf. Opening the API in any browser also works like a charm.
TLS1.2 is not enabled by default in .net framework 4.5, but the line ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12should actually enable it. Is there anything wrong with the way I am doing it?
Calling the API in curl (WSL and Azure remote bash) with curl --user 'user:pwd' https://myurl also works perfectly and returns the expected data.
Testing tls1.2 with openssl like openssl s_client -connect hostname:443 -tls1_2 does not reveal any issues. The chain is displayed correctly, and a TLSv1.2 session is confirmed. Testing the server's SNI feature with openssl with openssl s_client -connect host:443 -tls1_2 -servername host -tlsextdebug -msg reveals SNI support by returning TLS server extension "server name" (id=0), len=0 I get the same certificate if I provide a completely different fantasy hostname though.
I captured the TLS/SNI handshake when debugging locally (see screenshot below). No issues. My ability to debug ends with the cloud service. I would love to see the handshake between the cloud service and the API in WireShark, but I don't know of any option to analyze network traffic at that layer on an Azure cloud service. But if anyone knows how to capture the handshake process, I'd appreciate some hints.
The server selects ECDHE-RSA-AES256-GCM-SHA384 as a cipher suite during the handshake with openssl which is pretty much default for TLS1.2. I don't have access to the cloud services list of cipher suites that it would provide at Client Hello - any idea how to find out?
I don't have any proof that SNI is actually causing the problem, but this is the only difference between this API and dozens of others I can spot.
Stack trace:
System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The request was aborted: Could not create SSL/TLS secure channel. at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar) --- End of inner exception stack trace --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult() at WCFServiceWebRole1.Controllers... in [the line calling GetAsync()]

I ended up re-creating a simple API on one of my servers and configured the software in such a way to send its requests there. That way I could capture the TLS handshake and analyze it in Wireshark. These are the supported cipher suites (client side, which is Azure cloud service):
And these are the cipher suites supported by the API which is not working:
I would assume that there should be a match so that server and client can agree on one. However, I cannot find a match... guess that is what is causing the problem. In fact, the list of supported cipher suites is much longer in a local debugging session - and there is at least one match which explains why it works locally.

SSL Error "The message received was unexpected or badly formatted" for a .NET application on one specific machine only

I have a .NET Core 3.1 C# application which is calling an API via HTTPS (and presenting its public key as part of getting the token as that certificate is later used to decrypt information sent back separately). On just about all our machines, it is working, but on one Windows 8.1 machine, we get the following series of exceptions when we try to initially connect for an authentication token:
The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
---> System.ComponentModel.Win32Exception (0x80090326): The message received was unexpected or badly formatted.
The exception is thrown from System.Net.Http.HttpClient.FinishSendAsyncBuffered so I suspect it is happening at the HTTPS level and our certificate stuff is not really relevant here anyway.
Our code to get the token looks like this:
The constructor for the auth service:
public XXXXAuthService(IXXDbService dbService, XXXXApiConfig config)
{
_dbService = dbService;
_config = config;
// try forcing TLS1.2 for SSL connection exceptions thrown in some operating environments
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
_httpClient = new HttpClient {BaseAddress = new Uri(config.BaseUrl)};
_httpClient.DefaultRequestHeaders.Accept.Clear();
_httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
}
Code to get the auth token:
private async Task<string> GetXXXXBearerToken(string userId, DateTime creationTime)
{
var token = await GenerateProviderJwtForXXXX(userId, creationTime);
var kvp = new List<KeyValuePair<string, string>>
{
new KeyValuePair<string, string>("grant_type", "urn:ietf:params:oauth:grant-type:token-exchange"),
new KeyValuePair<string, string>("subject_token", token),
new KeyValuePair<string, string>("subject_token_type", "urn:ietf:params:oauth:token-type:jwt")
};
var data = new FormUrlEncodedContent(kvp);
var publicKey = await GetXXXXPublicKey();
_httpClient.DefaultRequestHeaders.Remove("X-XXXX-Public-Cert");
_httpClient.DefaultRequestHeaders.Add("X-XXXX-Public-Cert", publicKey);
var response = await _httpClient.PostAsync("Identity/token", data);
if (!response.IsSuccessStatusCode)
throw new Exception("XXXX Token Server Error: " + response.ReasonPhrase);
var result = await response.Content.ReadAsStringAsync();
var authResponse = JsonConvert.DeserializeObject<OAuthResponse>(result);
if (!string.IsNullOrEmpty(authResponse.access_token))
return authResponse.access_token;
System.Diagnostics.Trace.WriteLine("Token Exchange Result: " + result);
if (!string.IsNullOrEmpty(authResponse.error))
{
var outcome = new XXX.XXXX.Model.OperationOutcome();
outcome.Issue.Add(new XXX.XXXX.Model.OperationOutcome.IssueComponent()
{
//some code to throw an error is here
}
throw new XXX.XXXX.Rest.XXXXOperationException("Bearer Token Exchange failed", response.StatusCode);
}
Unfortunately none of the existing questions/advice anywhere on Stack Overflow, or the rest of the web, for this particular error seems to have helped. They are primarily about version discrepancies between client and server which seems not to be the case here as I am forcing TLS 1.2 (which is active and enabled on the failing machine).
Interestingly, I can visit the server URL in a browser via HTTPS just fine, which suggests there is something about my code that is the problem rather than the machine, but it works everywhere else.
I have confirmed that:
The certificate I am using to authenticate the connection on the machine is valid and has a chain of trust (though as above I don't think we are getting that far as the TLS connection itself is failing)
The server we are calling supports TLS 1.2 (by forcing it)
I can get to the website for the URL independently via the browser
Is there something I need to do either in the code or on the machine to get this call to work everywhere?
Things I have tried to resolve the issue
Installing all Windows 8.1 updates to present day
Forcing TLS 1.2 in the code (see above code sample)
Limiting VM to TLS 1.2 only

I might be able to at least point you in the right direction…
Same Symptoms
I had a .NET Core 3.1 web app running on IIS (Windows Server 2012 R2) that got the exact same error and stacktrace when it tried to connect to another server using TLS 1.2. I also had the symptom where I could connect with the browser (Chrome), but not with the app. (Would have been interesting to see if Internet Explorer browser worked though.)
Root Cause
The TLS handshake was failing because the two servers were unable to agree on a common cipher suite. (Using Wireshark, I discovered that when my app tried to connect it provided a more limited set of cipher suites than when the Chrome browser made the call.)
Solution
In my case, I used IIS Crypto (a small free tool: https://www.nartac.com/Products/IISCrypto/) to enable additional cipher suites on my web app's server. I downloaded and ran IIS Crypto, checkmarked additional cipher suites on its Cipher Suites tab, and then restarted the machine.
One of the new cipher suites worked with my app and the destination server, so the TLS handshake was successful and the error was resolved.
One quick caveat: Some cipher suites are more secure than others, so you'll want to read up on best practices.
Addendum
If you want to further diagnose the failure, I'd recommend installing Wireshark (another free tool: https://www.wireshark.org/#download) on the machine with your .NET Core app. If a TLS Handshake Failure is the issue, you will see a message like: Alert (Level: Fatal, Description: Handshake Failure)
This primer on wireshark output helped me:
https://blog.catchpoint.com/2017/05/12/dissecting-tls-using-wireshark/

I faced a simular issue, and in order to help others here's what I concluded:
Sucessfully executing this code doesn't mean that your application supports the specified protocol version, and the "SSL Error" can still occur later on when trying to establish a connection:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
In my case I was trying to force Tls13 and found out that my app configuration didn't actually support it:
net core 3.0 running on a Windows Server Datacenter 2019, version 1809
So I had to change my configuration to the following which provides support for the protocol version I needed:
Net framework 5.0 on a Windows Server Datacenter 2022, OS build 20348.288
I was trying to connect to an endpoint that suddenly dropped Tls 1.2 support (not sure why) and from then on only accepted Tls 1.3.

A call to SSPI failed, see inner exception using FileZilla Server and System.Net.FtpClient

I am setting up an FTP Server (FileZilla) and using the netftp codeplex project System.Net.FtpClient to connect to it over SSL/TLS using an X509Certificate2.
I've downloaded FileZilla Server from sourceforge.
I've downloaded System.Net.FtpClient v14.06.17 from codeplex.
I think I have isolated the problem, but maybe someone has some additional thoughts as to the underlying cause. A workaround is provided at the bottom of this post.
An exception occurs calling client.GetListing() and client.GetNameListing(), however, uploading (OpenWrite) and downloading (OpenRead) both work fine. I've tested many versions of FileZilla to isolate what version it started breaking.
0.9.42 works (2013-12-16)
0.9.43 bad (2014-01-02)
0.9.46 bad (2014-08-03)
0.9.50 bad (2015-03-19) - current
The exception that occurs is:
outer message: A call to SSPI failed, see inner exception.
inner message: The message received was unexpected or badly formatted
The C# code sample that causes the error:
FtpClient client = new FtpClient();
client.Credentials = new NetworkCredential("blah", "blah123");
client.Host = "127.0.0.1";
client.Port = 21;
client.DataConnectionEncryption = true;
client.EncryptionMode = FtpEncryptionMode.Explicit;
//client.ClientCertificates.Add(cert); // tried both with and without
client.ValidateCertificate += (cli, e) => { e.Accept = true; };
client.Connect();
var list = client.GetNameListing(); // exception on this line
On the Codeplex Documentation page, jptrosclair wrote:
You do not use pem certificates, use p12 instead. See this Stack
Overflow thread for more information. If you get SPPI exceptions
with an inner exception about an unexpected or badly formatted
message, you are probably using the wrong type of certificate.
The link refers to asp.net, not FileZilla. My guess is that IIS doesn't support PEM and was most likely sending a bunch of gibberish through the stream. However, if you use a certificate format that your server supports, then there should be no problem. As far as I have read, FileZilla only supports PEM format (or CRT/CER which are still PEM format).
Also, this doesn't explain why OpenRead and OpenWrite succeed, but GetListing and GetNameListing fail.
Regardless, I still tested multiple different certificate generation tools, including FileZilla's built-in generator, C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Bin\makecert.exe, and OpenSSL (windows). The problem still persisted.
Here is a link similar discussion from skovachev https://netftp.codeplex.com/discussions/535815
His workaround was to clone the connection before getting the listing.
Workaround:
I've tested the following with all FileZilla versions listed above. The fix is to download the System.Net.FtpClient source code from codeplex and in the FtpSocketStream.cs class edit the following method:
public void ActivateEncryption(string targethost, X509CertificateCollection clientCerts)
change:
m_sslStream.AuthenticateAsClient(targethost, clientCerts, SslProtocols.Tls | SslProtocols.Ssl3 | SslProtocols.Ssl2, true);
to:
m_sslStream.AuthenticateAsClient(targethost, clientCerts, SslProtocols.Tls | SslProtocols.Ssl3, true);
(Basically just remove the Ssl2 flag).
My guess is something changed in FileZilla v0.9.43 when the Ssl2 flag is set when connecting to the server. It is strange that the first time AuthenticateAsClient is called it will succeed, but the second time it will fail.
As for uploading and downloading succeeding, the default EnableThreadSafeDataConnections is true, which causes the client to be cloned and reconnect to the server. However, GetListing and GetNameListing do not clone the connection. Putting a breakpoint in the ActivateEncryption method, the AuthenticateAsClient will be called multiple times. Maybe there is some connection pooling happening under the hood in the SslStream class and multiple AuthenticateAsClient are causing problems. It's worth noting that the FtpClient.ValidateCertificate event is only called once even though AuthenticateAsClient is called twice. This gives some evidence of some caching is happening.

HttpWebRequest NameResolutionFailure exception in .NET (with Mono on Ubuntu)

I have a .NET program running on Ubuntu via Mono 2.10
The program downloads a webpage via an HttpWebRequest every minute or so which works fine most of the time:
String result;
WebResponse objResponse;
WebRequest objRequest = System.Net.HttpWebRequest.Create(url);
using (objResponse = objRequest.GetResponse())
{
using (StreamReader sr =
new StreamReader(objResponse.GetResponseStream()))
{
result = sr.ReadToEnd();
// Close and clean up the StreamReader
sr.Close();
}
}
The problem is that after few days I start getting exceptions thrown:
DateTime: 01/25/2012 08:15:41
Type: System.Net.WebException
Error: Error: NameResolutionFailure
Stack:
at System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult) [0x00000] in <filename unknown>:0
at System.Net.HttpWebRequest.GetResponse () [0x00000] in <filename unknown>:0
at socks_server.Program.readHtmlPage (System.String url) [0x00000] in <filename unknown>:0
at socks_server.Program.getAccessKeysProc () [0x00000] in <filename unknown>:0
The server is still abel to resolve DNS, for example
wget http://www.google.com
Will return the file without any probelm as will ping and other commands that resolve DNS.
My program however will continue to throw that exception until I restart it. After restarting the application it will start working again as it should.
I have checked open file counts on the system (400 ish), memory usage (327mb of 4gb), CPU usage (2-3%) and all are OK.
Any ideas?

You can solve it by translating the host name to ip and add the host name to Headers collection or to Host property.
If your url is http://example.com/uri. Resolve the host yourself. Suppose its 1.2.3.4. It'll be http://1.2.3.4/uri. Now add Host: example.com header to your request. I think it can be done by setting HttpWebRequest.Host property.

I know this is an old post, but was facing the same error, so thought to share the solution.
The best solution I found, when that exception occurs while the Wifi
is connected, is just to retry my server call with a slight sleep in
between. It works most of the time, otherwise if the second call
fails I cancel the request.
This error can also raise if the user's Wifi is very unstable or the
signal is very low. The same error occurs if there is no internet
connection at all, even if connected to Wifi.
This is in line with my ans on :
System.Net.WebException: Error: NameResolutionFailure when Calling WCF Services throwing exception in mono android application

Well I use the HttpClient - but it might be a similar problem. I had the same issue on a Android device (it worked on a Windows Phone)... But after I added the Host to the header it worked!
client.DefaultRequestHeaders.Host = "mydomain.com";
You can still use the name in the url (you don't have to use the IP address)

I was experiencing the same issue in my mono application on raspbian. I've tried different solutions described in this and other threads but none worked. Eventually, I was able to fix the problem by changing the name servers in /etc/resolv.conf to the google ones https://developers.google.com/speed/public-dns/
Mirko

I was getting this error when I started the mobile app (android or iOS it does not matter) without internet connection. After restored the connection every request returns "NameResolutionFailure exception". I had to wait 120 seconds for having the http request working again. Setting the following line of code anywhere in the app startup the error was finally gone.
System.Net.ServicePointManager.DnsRefreshTimeout = 0;
The default DnsRefreshTimeout value is 120 seconds.