Develop Reference

SSL Error "The message received was unexpected or badly formatted" for a .NET application on one specific machine only

I have a .NET Core 3.1 C# application which is calling an API via HTTPS (and presenting its public key as part of getting the token as that certificate is later used to decrypt information sent back separately). On just about all our machines, it is working, but on one Windows 8.1 machine, we get the following series of exceptions when we try to initially connect for an authentication token:
The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
---> System.ComponentModel.Win32Exception (0x80090326): The message received was unexpected or badly formatted.
The exception is thrown from System.Net.Http.HttpClient.FinishSendAsyncBuffered so I suspect it is happening at the HTTPS level and our certificate stuff is not really relevant here anyway.
Our code to get the token looks like this:
The constructor for the auth service:
public XXXXAuthService(IXXDbService dbService, XXXXApiConfig config)
{
_dbService = dbService;
_config = config;
// try forcing TLS1.2 for SSL connection exceptions thrown in some operating environments
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
_httpClient = new HttpClient {BaseAddress = new Uri(config.BaseUrl)};
_httpClient.DefaultRequestHeaders.Accept.Clear();
_httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
}
Code to get the auth token:
private async Task<string> GetXXXXBearerToken(string userId, DateTime creationTime)
{
var token = await GenerateProviderJwtForXXXX(userId, creationTime);
var kvp = new List<KeyValuePair<string, string>>
{
new KeyValuePair<string, string>("grant_type", "urn:ietf:params:oauth:grant-type:token-exchange"),
new KeyValuePair<string, string>("subject_token", token),
new KeyValuePair<string, string>("subject_token_type", "urn:ietf:params:oauth:token-type:jwt")
};
var data = new FormUrlEncodedContent(kvp);
var publicKey = await GetXXXXPublicKey();
_httpClient.DefaultRequestHeaders.Remove("X-XXXX-Public-Cert");
_httpClient.DefaultRequestHeaders.Add("X-XXXX-Public-Cert", publicKey);
var response = await _httpClient.PostAsync("Identity/token", data);
if (!response.IsSuccessStatusCode)
throw new Exception("XXXX Token Server Error: " + response.ReasonPhrase);
var result = await response.Content.ReadAsStringAsync();
var authResponse = JsonConvert.DeserializeObject<OAuthResponse>(result);
if (!string.IsNullOrEmpty(authResponse.access_token))
return authResponse.access_token;
System.Diagnostics.Trace.WriteLine("Token Exchange Result: " + result);
if (!string.IsNullOrEmpty(authResponse.error))
{
var outcome = new XXX.XXXX.Model.OperationOutcome();
outcome.Issue.Add(new XXX.XXXX.Model.OperationOutcome.IssueComponent()
{
//some code to throw an error is here
}
throw new XXX.XXXX.Rest.XXXXOperationException("Bearer Token Exchange failed", response.StatusCode);
}
Unfortunately none of the existing questions/advice anywhere on Stack Overflow, or the rest of the web, for this particular error seems to have helped. They are primarily about version discrepancies between client and server which seems not to be the case here as I am forcing TLS 1.2 (which is active and enabled on the failing machine).
Interestingly, I can visit the server URL in a browser via HTTPS just fine, which suggests there is something about my code that is the problem rather than the machine, but it works everywhere else.
I have confirmed that:
The certificate I am using to authenticate the connection on the machine is valid and has a chain of trust (though as above I don't think we are getting that far as the TLS connection itself is failing)
The server we are calling supports TLS 1.2 (by forcing it)
I can get to the website for the URL independently via the browser
Is there something I need to do either in the code or on the machine to get this call to work everywhere?
Things I have tried to resolve the issue
Installing all Windows 8.1 updates to present day
Forcing TLS 1.2 in the code (see above code sample)
Limiting VM to TLS 1.2 only

I might be able to at least point you in the right direction…
Same Symptoms
I had a .NET Core 3.1 web app running on IIS (Windows Server 2012 R2) that got the exact same error and stacktrace when it tried to connect to another server using TLS 1.2. I also had the symptom where I could connect with the browser (Chrome), but not with the app. (Would have been interesting to see if Internet Explorer browser worked though.)
Root Cause
The TLS handshake was failing because the two servers were unable to agree on a common cipher suite. (Using Wireshark, I discovered that when my app tried to connect it provided a more limited set of cipher suites than when the Chrome browser made the call.)
Solution
In my case, I used IIS Crypto (a small free tool: https://www.nartac.com/Products/IISCrypto/) to enable additional cipher suites on my web app's server. I downloaded and ran IIS Crypto, checkmarked additional cipher suites on its Cipher Suites tab, and then restarted the machine.
One of the new cipher suites worked with my app and the destination server, so the TLS handshake was successful and the error was resolved.
One quick caveat: Some cipher suites are more secure than others, so you'll want to read up on best practices.
Addendum
If you want to further diagnose the failure, I'd recommend installing Wireshark (another free tool: https://www.wireshark.org/#download) on the machine with your .NET Core app. If a TLS Handshake Failure is the issue, you will see a message like: Alert (Level: Fatal, Description: Handshake Failure)
This primer on wireshark output helped me:
https://blog.catchpoint.com/2017/05/12/dissecting-tls-using-wireshark/

I faced a simular issue, and in order to help others here's what I concluded:
Sucessfully executing this code doesn't mean that your application supports the specified protocol version, and the "SSL Error" can still occur later on when trying to establish a connection:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
In my case I was trying to force Tls13 and found out that my app configuration didn't actually support it:
net core 3.0 running on a Windows Server Datacenter 2019, version 1809
So I had to change my configuration to the following which provides support for the protocol version I needed:
Net framework 5.0 on a Windows Server Datacenter 2022, OS build 20348.288
I was trying to connect to an endpoint that suddenly dropped Tls 1.2 support (not sure why) and from then on only accepted Tls 1.3.

SocketException and IOException when attempting to store an HTML file as a string

I've tried a few methods at this point to pull this website and stuff it into a variable to later be parsed, and all of them have thrown the same errors:
IOException: Unable to write data to the transport connection: An established connection was aborted by the software in your host machine.
SocketException: An established connection was aborted by the software in your host machine
So far I've tried WebClient.DownloadString() in the .Net lib, something similar in the .Net.Http lib, as well as the HtmlDocument.Load() from the HtmlAgilityPack lib all with the same result. In the .Net lib I've tried the commonly suggested solution of disabling keepalive, and am just assuming at this point that there is some setting on my computer causing this.
Any ideas? And thanks in advance for reading. =)
Here is my current code:
string tcgpSite;
using (var wc = new System.Net.WebClient())
tcgpSite = wc.DownloadString("http://shop.tcgplayer.com/magic?newSearch=true");
Console.WriteLine(tcgpSite);

It could be a probably with the way your using statement is set up in your actual code.
The following works for me:
var wc = new System.Net.WebClient()
string tcgpSite = wc.DownloadString("http://shop.tcgplayer.com/magic?newSearch=true");
As does the code you posted.
Otherwise, you may have some kind of fire wall or other network issue closing your connection. The error seems to indicate the same.
SocketException: An established connection was aborted by the software in your host machine.
Try temporarily disabling any firewall or anti virus your machine might be running.

WCF client has inconsistent behavior with multiple network interfaces

I have the WCF client shown below:
public static string Execute(string a)
{
WebHttpBinding b = new WebHttpBinding();
b.Security.Mode = WebHttpSecurityMode.Transport;
WebChannelFactory<IAnimalService> f = new WebChannelFactory<IAnimalService>(b, new Uri(a));
f.Endpoint.Behaviors.Add(new WebHttpBehavior());
IWebService client = f.CreateChannel();
return client.SayHello("moo");
}
I am testing the client (in a console host) against its counterpart service from 2 different computers (A, B) and and i get different outcome. Here are the similarities and differences between A and B:
Both A and B are Windows Server 2012 R2 have multiple network interfaces. And each of them have one interface connected to the Internet (so as to reach the service in question).
When running on machine A, everything works as expected i get expected response ("cow says 'moo'") in the client.
When running on machine B, i get different behaviours:
Via the web-browser (firefox), i can get to the service by constructing a request in the address bar. I can see the request going on the external network interface. And everything works as in 2 above.
Via the app (i.e. code above), i get the exception below and more over, when looking at the request in wireshark, it does not go on the external network interface. The exception comes the fact that on that network interface there is a proxy and the proxy is rejecting the request (X-Squid-Error -> ERR_ACCESS_DENIED 0).
Exception:
Unhandled Exception: System.ServiceModel.Security.MessageSecurityException: The HTTP request was forbidden with client authentication scheme
'Anonymous'. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.
at System.Net.HttpWebRequest.GetResponse()
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
--- End of inner exception stack trace ---
I also consulted the routing table on machine B and it shows that request should have gone on the external network interface. I have assumed that the internal routing table would be consulted by the lower layer of the TCP-IP stack in order to forward the packet to the correct (network) medium.
So, I am wondering if someone would have an idea how i could get my client to pick the correct interface on machine B. I am not sure how to go about to troubleshoot the problem further. So any kind of advice/tips on how to get to the bottom of this situation on machine B would be most welcomed.

Try to use this snippet in you config file, so you will be able to use your default credentials to autenticate in the proxy server.
<system.net>
<defaultProxy useDefaultCredentials="true" />
</system.net>

It would appear that by default, the binding seeks to look for the default proxy setting on machine B. Adding the following binding configuration solved the problem:
b.UseDefaultWebProxy = false;

HttpWebRequest.GetResponse() returning 404 Error

I have some code that calls HttpWebRequest's GetResponse() method to retrieve HTML from a URL and return it to the calling method.
This has been working perfectly fine within my Development and QA environments but now that I have uploaded it to my UAT server, I keep getting the following error:
The remote server returned an error: (404) Not Found.
The main difference between Dev/QA and UAT is that UAT uses SSL/HTTPS based URLs whereas Dev/QA uses HTTP. I introduced the following line of code to help progress me a little futher:
ServicePointManager.ServerCertificateValidationCallback = new System.Net.Security.RemoteCertificateValidationCallback(AcceptAllCertifications);
where AcceptAllCertifications always returns true but I still get my 404 error.
I that people who previously had this error have been able to resolve the issue by merely ensuring the URI used for the HttpWebRequest doesn't have a slash at the end (see: Simple HttpWebRequest over SSL (https) gives 404 Not Found under C#) but this does not make a difference to me.
I have now tried what was suggested at this post (see: HttpWebResponse returns 404 error) where I render the exception on the page. This bypassed the yellow-warning screen and gives me a bit more informtion, including the URL it is trying to get a response from. However, when I copy and paste the URL into my browser, it works perfectly fine and renders the HTML on the page. I'm quite happy therefore that the correct URL is being used in the GetResponse call.
Has anyone got any ideas as to what may be causing me this grief? As said, it only seems to be a problem on my UAT server where I am using SSL.
Here is my code to assist:
public static string GetHtmlValues()
{
var webConfigParentUrlValue = new Uri(ConfigurationManager.AppSettings["ParentUrl"]);
var destinationUrl = HttpContext.Current.Request.Url.AbsoluteUri;
var path = "DestinationController" + "/" + "DestinationAction" + "?destinationUrl=" + destinationUrl;
var redirect = new Uri(webConfigParentUrlValue, path).AbsoluteUri;
ServicePointManager.ServerCertificateValidationCallback = new System.Net.Security.RemoteCertificateValidationCallback(AcceptAllCertifications);
var request = (HttpWebRequest)WebRequest.Create(redirect);
//Ensures that if the user has already signed in to the application,
// their authorisation is carried on through to this new request
AttachAuthorisedCookieIfExists(request);
HttpWebResponse result;
try
{
result = (HttpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
result = ex.Response as HttpWebResponse;
}
String responseString;
using (Stream stream = result.GetResponseStream())
{
StreamReader reader = new StreamReader(stream, Encoding.UTF8);
responseString = reader.ReadToEnd();
}
return responseString;
}
More details of the error as it is rendered on the page:

I ran into a similar situation, but with a different error message. My problem turned out to be that my UAT environment was Windows 2008 with .NET 4.5. In this environment the SSL handshake/detecting is performed differently than most web browsers. So I was seeing the URL render without error in a web browser but my application would generate an error. My error message included "The underlying connection was closed: An unexpected error occurred on a send". This might be your issue.
My solution was to force the protocol change. I detect the specific error, then I force a change in the security protocol of my application and try again.
This is the code I use:
catch (Exception ex)
{
if(ex.Message.Contains("The underlying connection was closed: An unexpected error occurred on a send."))
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;
// retry the retrieval
}
}

I finally found the solution to my problem...
The first clue to get me on the right track was the wrong physical path being displayed in the 404 error from IIS. It turns out that this incorrect physical path was mapped to another site in my IIS setup. This particular naturally had a binding also; port 443. As you may know, port 443 is the default port for https.
Now looking at my URL that I was trying to pass into the HTTPWebRequest.GetResponse() method, it looked something like this:
https://www.my-web-site.com
Taking this into account, when this application was hosted on IIS within the bounds of SSL, the error was occuring as follows:
Code enters the aforementioned method GetHtmlValues()
The code gets https://www.my-web-site.com from the web.config file
A response is requested from https://www.my-web-site.com
At this point, as no port has been specified and application is now out there on the open internet, it tries to get a response from https://www.my-web-site.com:443
The problem is, my application isn't hosted via IIS on port 443. A different application lives here. Subsequently, as the page can't be found on port 443, a 404 error is produced.
Now for the solution...
Looking in IIS, I found the port that my application sits on. Let's say port 16523.
Whereas previously in my web.config I had my key of ParentUrl decalred with a value of https://www.my-web-site.com, this is to be changed to http://www.my-web-site.com:16523
Note how the https has become http and the port number is specified at the end. Now when the application tries to get the response, it no longer uses the default ssl port as the correct one was specified.

TF400324 error, but only when using the TFS API

I've seen a lot of discussion about "TF400324: Team Foundation services are not available from server", but everything I've read relates to DNS or proxies, and reflect being unable to connect to TFS at all, through any channel. My case is different: I cannot reach my TFS server when using the .NET library, but it works fine using Visual Studio's workflow tools, and I can reach the same URL just fine in a browser.
Zee code, it is here:
private TfsConfigurationServer _server;
...
Uri url = new Uri(serverName + rootFolder);
var creds1 = new NetworkCredential(username, password, Environment.UserDomainName);
var creds2 = new BasicAuthCredential(creds1);
var creds3 = new TfsClientCredentials(creds2) { AllowInteractive = false };
_server = new TfsConfigurationServer(url, creds3);
// Throws Microsoft.TeamFoundation.TeamFoundationServiceUnavailableException
_server.Authenticate();
The creds are needlessly elaborate because I've tried various suggested solutions, but I don't think that's relevant; I get the same behavior with any other creds I've tried, or no creds at all.
If I copy the exact URL to a browser, I can authenticate and proceed. Within Visual Studio I can connect to TFS using Source Control Explorer and other tools, without explicitly authenticating. What might the library want handled differently?
Additional detail from the error:
Team Foundation services are not available from server https://tfs.imahufflepuff.com:8080/tfs/Root/Project.
Technical information (for administrator):
The underlying connection was closed: An unexpected error occurred on a send.
---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send.
---> System.IO.IOException: The handshake failed due to an unexpected packet format.
We don't have an in-house TFS admin, otherwise I'd kick this issue over to him. I've tried to use Fiddler to get additional detail, but VS consistently refuses to show up there. I can reach an externally-hosted API while debugging, so I don't think there's a network or proxy problem locking down VS.

Have you tried using TfsTeamProjectCollection class instead of TfsConfigurationServer ?
E.g.
Uri url = new Uri(serverName + rootFolder);
var creds = new NetworkCredential(username, password, Environment.UserDomainName);
var server = new TfsTeamProjectCollection(url, creds);
server.Authenticate();
You can also try debugging this issue using Fiddler. You'll have to change VS proxy settings before starting Fiddler:
Either set the registry key reg add hkcu\SOFTWARE\Microsoft\VisualStudio\12.0\TeamFoundation\RequestSettings
/v BypassProxyOnLocal /t REG_SZ /d False
or
Set environment variable TFS_BYPASS_PROXY_ON_LOCAL=0

Seems like an SSL handshake issue at its root, which has nothing to do with the TFS and HTTP protocol and authentication, they sit above SSL. So make sure you have a valid certificate, matching hostname, good validity, accessible CRL where the cert is not revoked, etc. Also check in other browsers or openssl.exe ("openssl.exe s_client -connect servername:8080") for more diagnostic info. You didn't mention but a proxy or an SSL-level certificate authentication could also cause problems, should be easily debuggable by the methods I mentioned above.