Develop Reference

Get encrypted data from HTTPS request to use certificate manually C#

I am trying for my project this:
I want to download the (root) certificate from given url (or from diff location in later stages)
i want then get data from given url and use certificate i download in step one to "decrypt them"
check response on given url if step 1 and 2 get me same results as just response from the server
Basically I am trying to create something that check that given certificates works on given url same as the one automatically given.
Preferably do all 3 steps in one request (so if given url has counter for access, whole procedure is just one access on website)
I can do quite nice step 1)
var client = new TcpClient(address.ToString(), 443);
var certValidation = new RemoteCertificateValidationCallback(delegate (object snd, X509Certificate certificate, X509Chain chainLocal, SslPolicyErrors sslPolicyErrors)
{
return true; //Accept every certificate, even if it's invalid
});
// Create an SSL stream and takeover client's stream
using (var sslStream = new SslStream(client.GetStream(), true, certValidation))
{
sslStream.AuthenticateAsClient(InputWWW.Text);
var serverCertificate = sslStream.RemoteCertificate;
cert = new X509Certificate2(serverCertificate);
}
But I am not able to find any tips hot to get that raw data locally.
So far I found only something like this, where I use certificate in request handler
WebRequestHandler handler = new WebRequestHandler();
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.ServerCertificateValidationCallback = (a, b, c, d) => { return true; };
handler.ClientCertificates.Add(certificate);
which is usage of certificate.
I want something like this:
var binary/StringBlob = webRequest.getRawData(url,port);
var serverResponseManualy = applyCertificate/publicKey(binary/StringBlob, X509Certificate2 certificate );
checkBodyEquals(serverResponseManually, webRequest.GetResponse());
Do you know how to do it, or what should I search for?
Is it possible to do all this in one request to server or not?
Thanks

So i was able to get more info about https, after that i discover that the way i ask this question is not correct. Because certificate (asymmetric cryptography) is used only to create symmetric key, which is used for encryption/description. So encrypted message by private key cannot be obtained

HttpClient - The message received was unexpected or badly formatted

I am trying to connect to an API that requires two way SSL configured. I have the the certificate and the private key which I configured in postman as shown in the below image.
This works fine in Postman, but SSL fails when I try to implement it in C# using HttpClient. The error that I get is "The message received was unexpected or badly formatted.". I believe it has something to do with incorrect configuration.
I have referred this StackOverflow post to implement my code: Associate a private key with the X509Certificate2 class in .net
Below is what I have tried:
byte[] publicCertificateBytes = File.ReadAllBytes("<Public Certificate>");
var publicCertificate = new X509Certificate2(publicCertificateBytes);
byte[] privateKey = Convert.FromBase64String(File.ReadAllText("<private key file>").Replace("-----BEGIN PRIVATE KEY-----", "").Replace("-----END PRIVATE KEY-----", ""));
using (var rsa = RSA.Create())
{
rsa.ImportPkcs8PrivateKey(privateKey, out _);
publicCertificate = publicCertificate.CopyWithPrivateKey(rsa);
publicCertificate = new X509Certificate2(publicCertificate.Export(X509ContentType.Pkcs12));
}
var httpService = new GenericUtilityManager().ResolveUtility<IHttpService>();
var handler = new HttpClientHandler();
handler.SslProtocols = SslProtocols.Tls12;
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.ClientCertificates.Add(publicCertificate);
handler.ServerCertificateCustomValidationCallback = (sender, cert, chain, sslPolicyErrors) => true;
httpService.InitializeHttpClientHandler(handler);
Please help.

Try to load the certicate using the CreateFromPem static function, this method returns a new certificate with the private key.
var certificateCrt = File.ReadAllText("<Public Certificate>");
var privateKey = File.ReadAllText("<private key file>");
using var certificate = X509Certificate2.CreateFromPem(certificateCrt, privateKey);
If that doesn't work, try to creating a .pfx certificate using OpenSSL:
openssl pkcs12 -export -out certificate.pfx -inkey privateKey.key -in certificate.crt
I had this problem when I needed to use a certificate on a Windows OS, to work I had to create a .pfx certificate, but on other OS like Mac and Linux it worked fine without a .pfx certificate.

Is it possible to generate on the fly SSL certificates from a root CA in C#?

I'm setting up a server under my own root CA to generate SSL certificates on the fly under .NET Core.
I'm able to generate self-signed certificates using the CertificateRequest class. However, these certs obviously aren't trusted by clients with my own root CA. I'm using the CertificateRequest.CreateSelfSigned() method to do this. I cannot use my root CA to sign these new certs, however. Using the CertificateRequest.Create() method will generate my new cert, but it will not provide a private key.
public static X509Certificate2 CreateSelfSignedCertificate(string domain)
{
SubjectAlternativeNameBuilder sanBuilder = new SubjectAlternativeNameBuilder();
sanBuilder.AddDnsName(domain);
X500DistinguishedName distinguishedName = new X500DistinguishedName($"CN=On-The-Fly Generated Cert");
using (RSA rsa = RSA.Create(2048))
{
var request = new CertificateRequest(distinguishedName, rsa, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);
request.CertificateExtensions.Add(
new X509KeyUsageExtension(X509KeyUsageFlags.DataEncipherment | X509KeyUsageFlags.KeyEncipherment | X509KeyUsageFlags.DigitalSignature, false));
request.CertificateExtensions.Add(
new X509EnhancedKeyUsageExtension(
new OidCollection { new Oid("1.3.6.1.5.5.7.3.1") }, false));
request.CertificateExtensions.Add(sanBuilder.Build());
var ca = new X509Certificate2(File.ReadAllBytes(#"E:\testing_ca_certificate.pfx"), "password"); //Open my root CA cert, generated in OpenSSL
//Generates a cert, but does not provide a private key.
var certificate = request.Create(ca, new DateTimeOffset(DateTime.UtcNow.AddDays(-1)), new DateTimeOffset(DateTime.UtcNow.AddDays(365)), new byte[] { 0, 1, 2, 3 });
//Generates a usable cert, but is not under my root CA
//var certificate = request.CreateSelfSigned(new DateTimeOffset(DateTime.UtcNow.AddDays(-1)), new DateTimeOffset(DateTime.UtcNow.AddDays(365)));
return new X509Certificate2(certificate.Export(X509ContentType.Pfx, "password"), "password", X509KeyStorageFlags.DefaultKeySet);
}
}
Using the CertificateRequest.Create() method, I get a valid cert without a private key. I should have this private key so I can encrypt SSL traffic.

Using the CertificateRequest.Create() method, I get a valid cert without a private key.
Real certificate authorities should not have the private key of the CA and the private key matching the certificate to be created in the same place at the same time. Aside from the fact that there is currently no way to decode a Certification Request (CSR), this method assumes it's being used with a public-only key provided to the CertificateRequest constructor.
Following the flow, the "expected" model is:
Client determines the need for a new certificate
Client generates public/private keypair
Client sends public key and other necessary information to CA (PKCS#10 CertificationRequest, or other means)
CA validates request
CA builds up the CertificateRequest object using the client public key
CA produces a certfificate with CertificateRequest.Create()
CA sends the certificate back to the client (cert.RawData)
Client instantiates X509Certificate2 instance (only has public key)
Client uses the CopyWithPrivateKey (extension) method to associate the private key to a new certificate object.
Client does whatever client wants now.
So, after
//Generates a cert, but does not provide a private key.
var certificate = request.Create(ca, new DateTimeOffset(DateTime.UtcNow.AddDays(-1)), new DateTimeOffset(DateTime.UtcNow.AddDays(365)), new byte[] { 0, 1, 2, 3 });
you should add
certificate = certificate.CopyWithPrivateKey(rsa);
(Though, really, you should have your certificate objects in using statements so they get disposed when no longer needed, in which case you'd need a second variable to hold the cert-with-key)

Given the certificate request was created using your rsa instance, you should be able to export the private key from it. RSA.ToXmlString() allows for exporting the key to a format that later that can be later imported using RSA.FromXmlString().

WebApi HttpClient not sending client certificate

I am trying to secure my RESTful WebApi service with ssl and client authentication using client certificates.
To test; I have generated a self signed certificate and placed in the local machine, trusted root certification authorities folder and i have generated a "server" and "client" certificates.
Standard https to the server works without issue.
However I have some code in the server to validate the certificate, this never gets called when I connect using my test client which supplies my client certificate and the test client is returned a 403 Forbidden status.
This imples the server is failing my certificate before it reaches my validation code.
However if i fire up fiddler it knows a client certificate is required and asks me to supply one to My Documents\Fiddler2. I gave it the same client certificate i use in my test client and my server now works and received the client certificate i expect!
This implies that the WebApi client is not properly sending the certificate, my client code below is pretty much the same as other examples i have found.
static async Task RunAsync()
{
try
{
var handler = new WebRequestHandler();
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.ClientCertificates.Add(GetClientCert());
handler.ServerCertificateValidationCallback += Validate;
handler.UseProxy = false;
using (var client = new HttpClient(handler))
{
client.BaseAddress = new Uri("https://hostname:10001/");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/xml"));
var response = await client.GetAsync("api/system/");
var str = await response.Content.ReadAsStringAsync();
Console.WriteLine(str);
}
} catch(Exception ex)
{
Console.Write(ex.Message);
}
}
Any ideas why it would work in fiddler but not my test client?
Edit: Here is the code to GetClientCert()
private static X509Certificate GetClientCert()
{
X509Store store = null;
try
{
store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
store.Open(OpenFlags.OpenExistingOnly | OpenFlags.ReadOnly);
var certs = store.Certificates.Find(X509FindType.FindBySubjectName, "Integration Client Certificate", true);
if (certs.Count == 1)
{
var cert = certs[0];
return cert;
}
}
finally
{
if (store != null)
store.Close();
}
return null;
}
Granted the test code does not handle a null certificate but i am debugging to enssure that the correct certificate is located.

There are 2 types of certificates. The first is the public .cer file that is sent to you from the owner of the server. This file is just a long string of characters. The second is the keystore certificate, this is the selfsigned cert you create and send the cer file to the server you are calling and they install it. Depending on how much security you have, you might need to add one or both of these to the Client (Handler in your case). I've only seen the keystore cert used on one server where security is VERY secure. This code gets both certificates from the bin/deployed folder:
#region certificate Add
// KeyStore is our self signed cert
// TrustStore is cer file sent to you.
// Get the path where the cert files are stored (this should handle running in debug mode in Visual Studio and deployed code) -- Not tested with deployed code
string executableLocation = Path.GetDirectoryName(AppDomain.CurrentDomain.RelativeSearchPath ?? AppDomain.CurrentDomain.BaseDirectory);
#region Add the TrustStore certificate
// Get the cer file location
string pfxLocation = executableLocation + "\\Certificates\\TheirCertificate.cer";
// Add the certificate
X509Certificate2 theirCert = new X509Certificate2();
theirCert.Import(pfxLocation, "Password", X509KeyStorageFlags.DefaultKeySet);
handler.ClientCertificates.Add(theirCert);
#endregion
#region Add the KeyStore
// Get the location
pfxLocation = executableLocation + "\\Certificates\\YourCert.pfx";
// Add the Certificate
X509Certificate2 YourCert = new X509Certificate2();
YourCert.Import(pfxLocation, "PASSWORD", X509KeyStorageFlags.DefaultKeySet);
handler.ClientCertificates.Add(YourCert);
#endregion
#endregion
Also - you need to handle cert errors (note: this is BAD - it says ALL cert issues are okay) you should change this code to handle specific cert issues like Name Mismatch. it's on my list to do. There are plenty of example on how to do this.
This code at the top of your method
// Ignore Certificate errors need to fix to only handle
ServicePointManager.ServerCertificateValidationCallback = MyCertHandler;
Method somewhere in your class
private bool MyCertHandler(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors error)
{
// Ignore errors
return true;
}

In the code you are using store = new X509Store(StoreName.My, StoreLocation.LocalMachine);.
Client certificates are not picked up from LocalMachine, you should instead use StoreLocation.CurrentUser.
Checking MMC -> File -> Add or Remove Snap-ins -> Certificates -> My user account you will see the certificate that fiddler uses. If you remove it from My user account and only have it imported in Computer account you will see that Fiddler can not pick it up either.
A side note is when finding certificates you also have to address for culture.
Example:
var certificateSerialNumber= "‎83 c6 62 0a 73 c7 b1 aa 41 06 a3 ce 62 83 ae 25".ToUpper().Replace(" ", string.Empty);
//0 certs
var certs = store.Certificates.Find(X509FindType.FindBySerialNumber, certificateSerialNumber, true);
//null
var cert = store.Certificates.Cast<X509Certificate>().FirstOrDefault(x => x.GetSerialNumberString() == certificateSerialNumber);
//1 cert
var cert1 = store.Certificates.Cast<X509Certificate>().FirstOrDefault(x =>
x.GetSerialNumberString().Equals(certificateSerialNumber, StringComparison.InvariantCultureIgnoreCase));

try this.
Cert should be with the current user store.
Or give full rights and read from a file as it is a console application.
// Load the client certificate from a file.
X509Certificate x509 = X509Certificate.CreateFromCertFile(#"c:\user.cer");
Read from the user store.
private static X509Certificate2 GetClientCertificate()
{
X509Store userCaStore = new X509Store(StoreName.My, StoreLocation.CurrentUser);
try
{
userCaStore.Open(OpenFlags.ReadOnly);
X509Certificate2Collection certificatesInStore = userCaStore.Certificates;
X509Certificate2Collection findResult = certificatesInStore.Find(X509FindType.FindBySubjectName, "localtestclientcert", true);
X509Certificate2 clientCertificate = null;
if (findResult.Count == 1)
{
clientCertificate = findResult[0];
}
else
{
throw new Exception("Unable to locate the correct client certificate.");
}
return clientCertificate;
}
catch
{
throw;
}
finally
{
userCaStore.Close();
}
}

MongoDB C# SSL Client Certificate

I am trying to establish a secure connection to MongoDB with the C# driver using certificate validation, but I am getting this error:
Unable to connect to server localhost:27017: Unable to read data from the transport connection: An established connection was aborted by the software in your host machine..
Heres the error from MongoDB:
[initandlisten] connection accepted from 127.0.0.1:26163 #2 (1 connection now open)
[conn2] ERROR: no SSL certificate provided by peer; connection rejected
[conn2] SocketException handling request, closing client connection: 9001 socket exception [CONNECT_ERROR]
When I connect to MongoDB through the mongo shell with the certificate it works.
var connectionString = "mongodb://localhost";
var clientSettings = MongoClientSettings.FromUrl(new MongoUrl(connectionString));
clientSettings.SslSettings = new SslSettings();
clientSettings.UseSsl = true;
clientSettings.SslSettings.ClientCertificates = new List<X509Certificate>()
{
new X509Certificate("cert.pem")
};
clientSettings.SslSettings.EnabledSslProtocols = SslProtocols.Default;
clientSettings.SslSettings.ClientCertificateSelectionCallback =
(sender, host, certificates, certificate, issuers) => clientSettings.SslSettings.ClientCertificates.ToList()[0];
clientSettings.SslSettings.ServerCertificateValidationCallback = (sender, certificate, chain, errors) => true;
var client = new MongoClient(clientSettings);
Does anyone know how to get this working?

Realize this is out of date but for the benefit of others...
If you're not handling cert revocation lists, you need to turn that setting off since it is enabled by default.
clientSettings.SslSettings.CheckCertificateRevocation = false;
Next, the X509Certificate2 you provide to the driver must include the private key. .NET doesn't seem to pick up private keys in pem files, so you need to provide certificate in .pfx format and include the passphrase.
To create a pfx file in openssl:
openssl pkcs12 -export -in mycert.cer -inkey mycert.key -out mycert.pfx
OpenSSL will prompt you for the export passphrase, use that when creating your X509Certificate2 object:
X509Certificate2 cert = new X509Certificate2("mycert.pfx","mypassphrase");

//struggled a lot to figure out this
using MongoDB.Bson;
using MongoDB.Driver;
namespace Mongo_AWS
{
internal class Program
{
private static void Main(string[] args)
{
//Mention cert file in connection string itself or put at your executable location
string connectionString = #"mongodb://user:pwd#localhost:9999/?ssl=true&ssl_ca_certs=C:\Users\sivaram\Downloads\my.pem";
MongoClientSettings settings = MongoClientSettings.FromUrl(new MongoUrl(connectionString));
//Disable certificate verification, if it is not issued for you
settings.VerifySslCertificate = false;
MongoClient client = new MongoClient(settings);
IMongoDatabase database = client.GetDatabase("test");
IMongoCollection<BsonDocument> collection = database.GetCollection<BsonDocument>("numbers");
System.Collections.Generic.List<BsonDocument> temp = collection.Find(new BsonDocument()).ToList();
BsonDocument docToInsert = new BsonDocument { { "sivaram-Pi", 3.14159 } };
collection.InsertOne(docToInsert);
}
}
}

,ssl_ca_certs = #"/path/my.pem", added this in connection string.
settings.VerifySslCertificate = false;
Use the above line if you are testing it from local/you have root certificate but not issued to your machine, may be issued to your production host.
Put the root certificate in the absolute path and refer that path directly in connection string. Mongo driver will take care of reading private key and all. No need to put it in the certificate store or somewhere.