Develop Reference

C# Connect to Ravendb keeps failing with error: This server requires client certificate for authentication, but none was provided by the client

No mather what i try, in an new .net 5 API project i cannot connect to RavenDB. It is given me an error:
This server requires client certificate for authentication, but none
was provided by the client.
The way I connect to the database:
byte[] certificateBytes = _certificateProvider.GetCertificate("CERT-NAME").Result;
string passphrase = _secretProvider.GetSecret("CERT-PASSPHRASE").Result;
X509Certificate2 certificate = new X509Certificate2(certificateBytes, passphrase);
var documentStore = new DocumentStore()
{
Urls = new[] { databaseOptions.DbUrl },
Conventions =
{
UseOptimisticConcurrency = true,
FindCollectionName = findCollectionName
},
Database = "DBNAME",
Certificate = certificate,
}.Initialize();
The certificate provider and the secret provider gets the data out of an Azure Key Vault. I validated the X509Certificate and it has the same thumbprint as I get in the admin panel of RavenDB. So that is loaded correctly. Also the certificate has the read/write rights on the requested database
But when i then do the following:
using (IAsyncDocumentSession session = documentStore.OpenAsyncSession())
{
var entity = await session.Query<EntityDTO>()
.SingleOrDefaultAsync();
}
Then i get the following error:
This server requires client certificate for authentication, but none
was provided by the client.
This while the certificate is given when initializing the document store. Anyone an idea how to continue with this as RavenCloud doesn't give more information then this?
In another project (.net core 3.1) the same code with the same certificate works. But could not find anything in the release notes of .net 5 what can cause this.

You need to load the certificate with the MachineKeySet flag.
X509Certificate2 certificate = new X509Certificate2(certificateBytes, passphrase, X509KeyStorageFlags.MachineKeySet);
This will solve the problem in Azure but your local dev machine might have a problem after this change. You need to allow full access for the user running RavenDB to the MachineKeySet folder on your local machine.
"C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys"
Please also note that if you don't dispose the certificate afterwards, the private keys pile up in that folder (a key is written to that folder every time the X509Certificate2 constructor opens the certificate).

In the RavenDB forum #iftah responded that you should set the storage flag to MachineKeySet because of the private key.
While that was not the case for me, it give me an direction to check. I was loading the certificate from Azure Key Vault using the CertificateClient and GetCertificateAsync. This only gives you the public key. So there never was an private key.
With the help of this post: KeyVault generated certificate with exportable private key. I've found out that i could use the SecretClient with GetSecretAsync. Then use Convert.FromBase64String() and that gives back also the private key.
So for the ones that have similar problems. Please verify when loading the certificate that you have the private key. My new code:
byte[] certificateBytes = Convert.FromBase64String(_secretProvider.GetSecret("CERT-NAME").Result);
X509Certificate2 certificate = new X509Certificate2(certificateBytes, string.Empty);
var documentStore = new DocumentStore()
{
Urls = new[] { databaseOptions.DbUrl },
Conventions =
{
UseOptimisticConcurrency = true,
FindCollectionName = findCollectionName
},
Database = "DBNAME",
Certificate = certificate,
}.Initialize();
And don't forget to load you certificate the following way if you want to have it working on Azure as #iftah suggested above:
X509Certificate2 certificate = new X509Certificate2(certificateBytes, string.Empty, X509KeyStorageFlags.MachineKeySet);

TLS handshake error from ... tls: client didn't provide a certificate

I have a C# RestFull client trying to connect to a Go server. Once I reach the TLS handshake stage it fails because client didn't provide a certificate.
I have verified that client certificate was associated with RestClient Object before the execution of the request.
// Initializing the client
RestClient client = new RestClient(Config.SERVER_DOMAIN);
// Adding client certificate (exported from a smart card with no private key attributes)
client.ClientCertificates = Global.cpf.getCertCollection();
// defining our request then executing it
var request = new RestRequest("users", Method.GET);
var response = await client.ExecuteTaskAsync(request);
It works only if the certificate was read from a .PFX file where the private component is there. But when I switch to smart card certificate (which has no private key attributes because the smart card doesn't want you to have them) the server doesn't receive any certificate from the client.
I understand that TLS needs a private key for the handshake stage, yet the client obj doesn't see any associated private key with the given certificate and therefore doesn't recognize the certificate as a valid one for TLS establishment.
I know that private keys can't be exported from the smart card, and I know that there has to be a way to tell RestClient object that in order to pass handshake stage, you should communicate with the smart card, however, I gave up!
Can someone point me to the right direction?

For most of the smartcards there should be minidriver or standalone CSP (Cryptographic Service Provider) available. These components act as a drivers that integrate the card with Windows cryptographic subsystem. They are usually provided by the device vendor and you should be able to get your certificate with reference to correct private key from your Windows Certificate Store once they're set up correctly.
Try to use the following method:
private static X509Certificate2 GetCertificate()
{
X509Store store = new X509Store(StoreName.My, StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadOnly);
try
{
// Note: X509Certificate2UI requires reference to System.Security
X509Certificate2Collection certs = X509Certificate2UI.SelectFromCollection(store.Certificates, null, null, X509SelectionFlag.SingleSelection);
if (certs != null && certs.Count > 0)
return certs[0];
}
finally
{
store.Close();
}
return null;
}

Using public key in asymmetric encryption with X.509 certificates

Sometimes ago I've read an article about using asymmetric keys like public and private keys to send data securely. What I've understood was that the server has 1 key (private key) that it use to encrypt data and all clients use the second key (public key) to decrypt it.
Now how should I expect to receive the key and how should I work with?
If I receive a Certificate from the server, wouldn't it contain both public and private keys?!
X509Store certStore = new X509Store(StoreName.My, StoreLocation.CurrentUser);
var cert = GetCertificate(certStore);
var privatekey= cert.PrivateKey;
var publicKey= cert.PublicKey;
Is it possbile to remove the private key from the certificate? How? and how can I understand if the certificate has the private key?

First a little bit of clarification:
In Public-key cryptography the public key is used to encrypt the data and the private key is used (by the server) to decrypt the data.
The owner of the private key stores private key and only shares a public key.
Any certificate from a server should contain only a public key.
(It would be a big security issue if the certificate contains the private Key. You could decrypt the messages from any other user)
To check if the cerificate has a private key you can use the HasPrivateKey-Property
cert.HasPrivateKey;
And to get a certificate with only the public key you can use:
byte[] bytes = cert.Export(X509ContentType.Cert);
var publicCert = new X509Certificate2(bytes);

If I receive a Certificate from the server, wouldn't it contain both public and private keys?!
No, not necessarily.
There are two different ways of obtaining the certificate
PKCS#7/PEM - the file usually contains only the public part of the certificate
PKCS#12/PFX - the store contains certificates with private keys
Is it possbile to remove the private key from the certificate?
By exporting it to a format that lets you store only the public part of the certificate.
Open a web browser, navigate to any site that uses SSL.
Now click the lock icon and, from there, the certificate information. What you have at the client side (in the browser) is the certificate without the private key. You can save it and even import into the system cert store but still without the private key.
and how can I understand if the certificate has the private key?
If you load it with your C# code, accessing the HasPrivateKey property will be true only for certs with private keys available.

Use X509Certificate without requiring passphrase

I have a certificate that I need to use in order to access a web service. The problem is that whenever I try to use the X509 certificate it asks for a passphrase (PIN). Is there any way of providing the passphrase directly, without it popping up the same window every time?
The certificate uses a dongle made by Oberthur Technologies, if it's of any help. Here's the code I use to get the certificate:
X509Store store = new X509Store("MY",StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadOnly | OpenFlags.OpenExistingOnly);
X509Certificate2Collection collection = (X509Certificate2Collection)store.Certificates;
if(collection.Count != 0)
userCert = collection[0]; // everything's ok up to here
And here's where I use the certificate:
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(new Uri(url));
req.ClientCertificates.Add(userCert); // add the certificate I just got
// ...
WebResponse ret = req.GetResponse(); // here's where it asks me for my passphrase

The certificate's private key is stored on the Oberthur token. It is enforcing the user to enter the PIN to gain access to certificates private key. This is by design and cannot be overridden.

I am searching for a way to do it with managed code but haven't yet found one.
In the past I have just called the CAPI function CryptSetProvParam with option PP_SIGNATURE_PIN and the PIN as the data. This prevents the CSP from prompting the user for a PIN. It's kinda hard for a system service to enter that PIN :)
So, you will have to pinvoke CryptSetProvParam to do it.

Using client certificate not in certificate store

I'm trying to authenticate myself against WebService using my client certificate, but, for some reasons (I explain), I don't want to load certificate from store, rather read it from disc.
The following:
// gw is teh WebService client
X509Certificate cert = new X509Certificate(PathToCertificate);
_gw.ClientCertificates.Add(ClientCertificate());
ServicePointManager.ServerCertificateValidationCallback = (a,b,c,d) => true;
_gw.DoSomeCall();
returns always 403 - the Service doesn't authorize me. But, when I save that certificate into CertStore, it works. (As stated in MSDN.)
Is it possible to use certificate not in store?
(the reason is, that I got windows service(client) sometimes calling webservice(server), and after unspecified amount of time the service 'forgets' my certificates and doesnt authorize against server, with no apparent reason)

What type of file is PathToCertificate? If it's just a .cer file, it will not contain the private key for the certificate and trying to use that certificate for SSL/TLS will fail.
However, if you have a PKCS7 or PKCS12 file that includes the public and private key for the certificate, your code will work (you might need to use the overload that takes a password if the private key has one).
To test this, I went to http://www.mono-project.com/UsingClientCertificatesWithXSP and created my client.p12 file following those instructions. I also created a simple HTTPS server using HttpListener for testing.
Then I compiled the following program into 'client.exe' and run like:
client.exe https://<MYSSLSERVER>/ client.p12 password
where client.p12 is the PKCS12 file generated before and 'password' is the password I set for the private key of the certificate.
using System;
using System.IO;
using System.Net;
using System.Security.Cryptography.X509Certificates;
using System.Text;
public class HttpWebRequestClientCertificateTest : ICertificatePolicy {
public bool CheckValidationResult (ServicePoint sp, X509Certificate certificate,
WebRequest request, int error)
{
return true; // server certificate's CA is not known to windows.
}
static void Main (string[] args)
{
string host = "https://localhost:1234/";
if (args.Length > 0)
host = args[0];
X509Certificate2 certificate = null;
if (args.Length > 1) {
string password = null;
if (args.Length > 2)
password = args [2];
certificate = new X509Certificate2 (args[1], password);
}
ServicePointManager.CertificatePolicy = new HttpWebRequestClientCertificateTest ();
HttpWebRequest req = (HttpWebRequest) WebRequest.Create (host);
if (certificate != null)
req.ClientCertificates.Add (certificate);
WebResponse resp = req.GetResponse ();
Stream stream = resp.GetResponseStream ();
StreamReader sr = new StreamReader (stream, Encoding.UTF8);
Console.WriteLine (sr.ReadToEnd ());
}
}
Let me know if you want me to upload the server code and the certificates used on both sides of the test.

The potential problem could be caching of SSL sessions (Schannel cache). Only first request negotiates the SSL handshake. Subsequent requests will use the same session ID and hope that the server accept it. If the server clears the SessionId, the requests will fail with 403 error. To disable local ssl session caching (and force SSL negotiation for each request) you have to open windows registry folder:
[HKEY_LOCAL_MACHINE][System][CurrentControlSet][Control][SecurityProviders][SCHANNEL]
and add the key named ClientCacheTime (DWORD) with value 0.
This issue is covered here:
http://support.microsoft.com/?id=247658

You have the potential for at least two problems...
First...
Your client certificate file cannot contain a private key unless it's accessed with a password. You should be using a PKCS #12 (*.pfx) certificate with a password so that your client has access to the private key. You client code will have to provide the password when opening the certificate as others have already posted. There are several ways to create this, the easiest is to use the following command-line to first generate the certificate, then use the MMC certificate manager to export the certificates private key:
Process p = Process.Start(
"makecert.exe",
String.Join(" ", new string[] {
"-r",// Create a self signed certificate
"-pe",// Mark generated private key as exportable
"-n", "CN=" + myHostName,// Certificate subject X500 name (eg: CN=Fred Dews)
"-b", "01/01/2000",// Start of the validity period; default to now.
"-e", "01/01/2036",// End of validity period; defaults to 2039
"-eku",// Comma separated enhanced key usage OIDs
"1.3.6.1.5.5.7.3.1," +// Server Authentication (1.3.6.1.5.5.7.3.1)
"1.3.6.1.5.5.7.3.2", // Client Authentication (1.3.6.1.5.5.7.3.2)
"-ss", "my",// Subject's certificate store name that stores the output certificate
"-sr", "LocalMachine",// Subject's certificate store location.
"-sky", "exchange",// Subject key type <signature|exchange|<integer>>.
"-sp",// Subject's CryptoAPI provider's name
"Microsoft RSA SChannel Cryptographic Provider",
"-sy", "12",// Subject's CryptoAPI provider's type
myHostName + ".cer"// [outputCertificateFile]
})
);
Second...
Your next problem is going to be server-side. The server has to allow this certificate. You have the right logic, but on the wrong side of the wire, move this line to the web server handling the request. If you cannot, you must then take the '.cer' file saved above to the server and add it to the server computer's trust list:
ServicePointManager.ServerCertificateValidationCallback = (a,b,c,d) => true;

Do you need a password for the certificate? If so, there is a field for it in the constructor.
X509Certificate cert = new X509Certificate(PathToCertificate,YourPassword);