Develop Reference

C# Connect to Ravendb keeps failing with error: This server requires client certificate for authentication, but none was provided by the client

No mather what i try, in an new .net 5 API project i cannot connect to RavenDB. It is given me an error:
This server requires client certificate for authentication, but none
was provided by the client.
The way I connect to the database:
byte[] certificateBytes = _certificateProvider.GetCertificate("CERT-NAME").Result;
string passphrase = _secretProvider.GetSecret("CERT-PASSPHRASE").Result;
X509Certificate2 certificate = new X509Certificate2(certificateBytes, passphrase);
var documentStore = new DocumentStore()
{
Urls = new[] { databaseOptions.DbUrl },
Conventions =
{
UseOptimisticConcurrency = true,
FindCollectionName = findCollectionName
},
Database = "DBNAME",
Certificate = certificate,
}.Initialize();
The certificate provider and the secret provider gets the data out of an Azure Key Vault. I validated the X509Certificate and it has the same thumbprint as I get in the admin panel of RavenDB. So that is loaded correctly. Also the certificate has the read/write rights on the requested database
But when i then do the following:
using (IAsyncDocumentSession session = documentStore.OpenAsyncSession())
{
var entity = await session.Query<EntityDTO>()
.SingleOrDefaultAsync();
}
Then i get the following error:
This server requires client certificate for authentication, but none
was provided by the client.
This while the certificate is given when initializing the document store. Anyone an idea how to continue with this as RavenCloud doesn't give more information then this?
In another project (.net core 3.1) the same code with the same certificate works. But could not find anything in the release notes of .net 5 what can cause this.

You need to load the certificate with the MachineKeySet flag.
X509Certificate2 certificate = new X509Certificate2(certificateBytes, passphrase, X509KeyStorageFlags.MachineKeySet);
This will solve the problem in Azure but your local dev machine might have a problem after this change. You need to allow full access for the user running RavenDB to the MachineKeySet folder on your local machine.
"C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys"
Please also note that if you don't dispose the certificate afterwards, the private keys pile up in that folder (a key is written to that folder every time the X509Certificate2 constructor opens the certificate).

In the RavenDB forum #iftah responded that you should set the storage flag to MachineKeySet because of the private key.
While that was not the case for me, it give me an direction to check. I was loading the certificate from Azure Key Vault using the CertificateClient and GetCertificateAsync. This only gives you the public key. So there never was an private key.
With the help of this post: KeyVault generated certificate with exportable private key. I've found out that i could use the SecretClient with GetSecretAsync. Then use Convert.FromBase64String() and that gives back also the private key.
So for the ones that have similar problems. Please verify when loading the certificate that you have the private key. My new code:
byte[] certificateBytes = Convert.FromBase64String(_secretProvider.GetSecret("CERT-NAME").Result);
X509Certificate2 certificate = new X509Certificate2(certificateBytes, string.Empty);
var documentStore = new DocumentStore()
{
Urls = new[] { databaseOptions.DbUrl },
Conventions =
{
UseOptimisticConcurrency = true,
FindCollectionName = findCollectionName
},
Database = "DBNAME",
Certificate = certificate,
}.Initialize();
And don't forget to load you certificate the following way if you want to have it working on Azure as #iftah suggested above:
X509Certificate2 certificate = new X509Certificate2(certificateBytes, string.Empty, X509KeyStorageFlags.MachineKeySet);

C# RestSharp Client Certificate getting exception "The SSL connection could not be established"

I programming in C# .Net Core 3.1 and .Net Standard library where this code is put.
I am not getting my http requests with RestSharp with a client certificate that are NOT installed on the server/computer i am not going to have access to the server/computer where this is going to be running later on..
My code:
// Create a RestSharp RestClient objhect with the base URL
var client = new RestClient(_baseAPIUrl);
// Create a request object with the path to the payment requests
var request = new RestRequest("swish-cpcapi/api/v1/paymentrequests");
// Create up a client certificate collection and import the certificate to it
X509Certificate2Collection clientCertificates = new X509Certificate2Collection();
clientCertificates.Import(_certDataBytes, _certificatePassword, X509KeyStorageFlags.MachineKeySet | X509KeyStorageFlags.PersistKeySet);
// Add client certificate collection to the RestClient
client.ClientCertificates = clientCertificates;
//X509Certificate2 certificates = new X509Certificate2(_certDataBytes, _certificatePassword);
//client.ClientCertificates = new X509CertificateCollection() { certificates };
// Add payment request data
request.AddJsonBody(requestData);
var response = client.Post(request);
var content = response.Content;
I have done what ever i can find on internet and i have used the service test environment with there own generated certificate and generated my own in there production environment with the same result..
I have tested set TLS 1.1 or TLS 1.2 to test, they specified TLS 1.1 on there own curl examples.
Anyone got any idea?
Update 2020-01-08 - I have got information from the service tech support that they only see me sending one certificate but when i trying to debug and checking the X509Certificate2Collection i finding 3 certificate. But they saying they only see one?

IMHO, if the service you are trying to access is protected via SSL cert with expecting public key/private key, then without the cert, you cant access it. If thats the intent of the service to be protected, i think you may only do to check the service health check or at max, check if the service is accessible (without having the client cert)
But just as HTTPS, then you can try to download the cert from the browser if you can access the service URL from your browser, say, to access their meta or a simple GET or something. In chrome, you may see a lock symbol (security icon) just before to the URL. click that, and you may possibly download the public cert. You can install that on your machine and try that.
In case, if that service has a pub cert and the access doesnt need the client cert, then above option may work. and i hope you are installing the CERT and access that via your code from the account that has access to the cert. Lets say, you install the cert under the LocalSystem, then you may need administrator access from code/solution to access that cert path. Or install the cert under the current_user path.
I would Check if am able to access the service from browser as the first step.
This is, just from what i have understood from your question. again, if you can explain whether the service is a protected service based on public key/private key, you need to have the access/cert to use that service.
UPDATE:
I have tried few options on my end, by creating a demo api and a demo client to use client certs. With what i have understood from your query, i assume below may be some of your options to try out.
// --- using the cert path if you have the path (instead of the byte[])
var myCert = new X509Certificate2("<path to the client cert.cer/.pfx>", "secure-password", X509KeyStorageFlags.MachineKeySet | X509KeyStorageFlags.PersistKeySet);
X509CertificateCollection clientCerts = new X509CertificateCollection();
clientCerts.Add(myCert);
OR
var certStore = new X509Store(StoreName.Root, StoreLocation.LocalMachine); //replace with appropriate values based on your cert configuration
certStore.Open(OpenFlags.ReadOnly);
//option 1 (ideally a client must have the cert thumbprint instead of a password
var cert = certStore.Certificates.Find(X509FindType.FindByThumbprint, "<cert Thumbprint>", false);
//option 2 (explore other options based on X509NameTypes
var cert = certStore.Certificates.OfType<X509Certificate2>()
.FirstOrDefault(cert => cert.GetNameInfo(X509NameType.DnsName, false) == "mycompany.dns.name.given in the cert");
client.ClientCertificates = new X509CertificateCollection(cert);

I recommend generate your own certificate
public static void MakeCert()
{
var ecdsa = ECDsa.Create(); // generate asymmetric key pair
var req = new CertificateRequest("cn=foobar", ecdsa, HashAlgorithmName.SHA256);
var cert = req.CreateSelfSigned(DateTimeOffset.Now, DateTimeOffset.Now.AddYears(10));
// Create PFX (PKCS #12) with private key
File.WriteAllBytes("c:\\temp\\mycert.pfx", cert.Export(X509ContentType.Pfx, "P#55w0rd"));
// Create Base 64 encoded CER (public key only)
File.WriteAllText("c:\\temp\\mycert.cer",
"-----BEGIN CERTIFICATE-----\r\n"
+ Convert.ToBase64String(cert.Export(X509ContentType.Cert), Base64FormattingOptions.InsertLineBreaks)
+ "\r\n-----END CERTIFICATE-----");
}
and then add to request
var client = new RestClient(url);
client.ClientCertificates = new X509CertificateCollection();
client.ClientCertificates.Add(new X509Certificate2("c:\\temp\\mycert.cer"));

How to determine if server has ssl certificate

I am creating a self-hosted owin server inside a windows service. I have implemented an ACME client to get a certificate from Let's Encrypt (to a domain given by the service's config.). However, if the service is run on a server which already has a certificate I do not need to request a new one. How can I determine, in code, if there is a certificate installed which applies for the domain set in the service's config?
The closest thing to a solution I found was to ignore existing certificates (if any) and always request a new one. Then when a certificate is received from Let's Encrypt, I save that certificate's serial to a file. On startup I then use the saved file (if any) to look for the existing certificate from the store:
public async Task<bool> NeedNewCertificate()
{
string certSerial = await AsyncFileHandler.ReadFileAsync(CERT_SERIAL_FILENAME);
using (var store = new X509Store(StoreName.My, StoreLocation.LocalMachine))
{
store.Open(OpenFlags.ReadOnly);
var certCollection = store.Certificates.Find(X509FindType.FindBySerialNumber, certSerial, true);
foreach (var cert in certCollection)
{
if (cert.NotBefore <= DateTime.Now && cert.NotAfter > DateTime.Now.AddDays(30)) // Let's Encrypt certificates are valid 90 days. They recommend renewing certificates every 30 days.
return false;
}
}
return true;
}

That's a good enough way of doing it I think.
Alternatively you can do a search in the certificate store that matches the parameters that you specify: the domain that the certificate is applied to, expiry date, etc. If you find the one that is valid, then you can use it.
That is in fact most likely the definition for what you mean by "certificate is installed on the server".
The attribute you probably would like to check is "Subject", e.g. you wold like these that have "CN = mydomain.com". You can have wildcard certificates as well, so you will need to figure out what types of certificates can be installed.

Loading Azure certificate when NOT using custom domain names

As I understand if someone doesn't want to use a custom domain name and instead plans on using *.azurewebsite.net domain assigned to the website by Azure, then HTTPS is already enabled with a certificate from Microsoft(I know this is not as secure as using a custom domain name). How would be I able to load this certificate programmatically. Currently I use the following method to load a certificate from local machine or Azure :
public static X509Certificate2 LoadFromStore(string certificateThumbprint,bool hostedOnAzure)
{
var s = certificateThumbprint;
var thumbprint = Regex.Replace(s, #"[^\da-zA-z]", string.Empty).ToUpper();
var store = hostedOnAzure ? new X509Store(StoreName.My, StoreLocation.CurrentUser) : new X509Store(StoreName.Root, StoreLocation.LocalMachine);
try
{
store.Open(OpenFlags.ReadOnly);
var certCollection = store.Certificates;
var signingCert = certCollection.Find(X509FindType.FindByThumbprint, thumbprint, false);
if (signingCert.Count == 0)
{
throw new FileNotFoundException(string.Format("Cert with thumbprint: '{0}' not found in certificate store. Also number of certificates in the sotre was {1}", thumbprint, store.Certificates.Count));
}
return signingCert[0];
}
finally
{
store.Close();
}
}
I assume the culprit is the following line of code :
new X509Store(StoreName.My, StoreLocation.CurrentUser)
because when I get an exception it tells me there is no certificate in the store although I pass the correct certificate Thumbprint(I grab the thumbprint from Chrome manually).

You will not be able to access this certificate programmatically in your WebApp as this certificate is not really installed on the Azure WebApp. Azure WebApps have a front-end server which does a "kind of" SSL Offloading so the WebApp actually never has access to this particular certificate. Why exactly you want to read this certificate though ?
Typically if there is a need for certificates in WebApps, you would install client certificates and pass them to services for Authentication as mentioned in https://azure.microsoft.com/en-us/blog/using-certificates-in-azure-websites-applications/ and those certificates you can access programmatically (code snippet mentioned in the same article)
But I am not sure what exactly you want to achieve by reading the server certificate

Azure API The server failed to authenticate the request

I have a task ( I tried with worker role and to upload a console app and run the .exe) that should run once a day and gather Azure Metrics of some of my VMs. This works flawlessly locally but on a cloud service I get this Error:
Unhandled Exception: Microsoft.WindowsAzure.CloudException: ForbiddenError: The server failed to authenticate the request. Verify that the certificate is valid and associated with this subscription. at Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSucces ... etc.
The line where this happens is:
MetricDefinitionListResponse metricListResponse = metricsClient.MetricDefinitions.List(resourceId, null,
nspace);
This is part of my code:
string subscriptionId = "fc4xxxx5-835c-xxxx-xxx-xxxxxxx";
// The thumbprint of the certificate.
string thumbprint = "‎f5 b4 xxxxxxxx f7 c2";
// Get the certificate from the local store.
//X509Certificate2 cert = GetCertificate(StoreName.My, StoreLocation.LocalMachine, thumbprint);
//cert = GetCertificate(StoreName.My, StoreLocation.CurrentUser, thumbprint) ?? new X509Certificate2(("manageAzure.cer"));
var cert = new X509Certificate2(("manageAzure.cer"));
Console.WriteLine("Certificate is : " + cert);
// Create the metrics client.
var metricsClient = new MetricsClient(new CertificateCloudCredentials(subscriptionId, cert));
Console.WriteLine("metricsClient is : " + metricsClient);
// The cloud service name and deployment name, found in the dashboard of the management portal.
string cloudServiceName = "abms2-carlsberg";
string deploymentName = "abms2-carlsberg";
// Build the resource ID string.
string resourceId = ResourceIdBuilder.BuildVirtualMachineResourceId(cloudServiceName, deploymentName);
string nspace = "WindowsAzure.Availability";
// Get the metric definitions.
MetricDefinitionListResponse metricListResponse = metricsClient.MetricDefinitions.List(resourceId, null,
nspace);
I have placed the management certificate in my solution, and I load it from there (it is set to always copy) and is the same (and the same way) I use when I run it locally.
So what "certificate" is it complaining about "to authenticate"? I can't seem to see what the problem is. Any help would be greatly appreciated as I have used the whole afternoon on this!
PS: I am already running this in elevated mode!

For someone else that may have this issue, I have solved it as explained here in the bottom : (http://www.dinohy.com/post/2013/11/12/403-Forbidden-when-Use-Azure-Management-REST-API-on-Role-instance.aspx)
Download the publishsettings file from:https://manage.windowsazure.com/publishsettings/index?client=vs&schemaversion=2.0 (This is a XML file, you can open it with notepad)
Find ManagementCertificate property, copy the value to a string. That a Base64 encoded string, you can use this string create a certificate: string base64Cer="The value of ManagementCertificate "
Use this string create a certificate. var certificate = new X509Certificate2(base64Cer);
although this last step is not exactly like passing the string directly (as the string is too long and will cause an exception) but rather is as follows:
var cert = new X509Certificate2(Convert.FromBase64String(base64cer));
Hope this will help someone else in my position too.

At a guess... You are loading the certificate you use to authenticate yourself from a .cer file. That doesn't have the private key so can't be used to authenticate you. I suspect that locally you probably have the private key stored in your private certificate store, assuming you generated the cert on your machine, which would probably make it work locally.
In short, try using a pfx file instead of a cer. The pfx for has the private key in it. If you generated the cert on your machine with makecert you will initially only have the .cer file. Use the local certificate manager to find the certificate in your personal store, then export it to a pfx file and include the private key.

This method helped me...
public static X509Certificate2 GetCertificate(string certificateString)
{
if (string.IsNullOrEmpty(certificateString))
return null;
var certificateAsBytes = Convert.FromBase64String(certificateString);
var certificate = new X509Certificate2(certificateAsBytes);
return certificate;
}
Also i came up with another scenario. Error occurs when your subscription id's mismatch.
Certificate:
<Subscription
ServiceManagementUrl="https://management.core.windows.net"
Id="00000000-0000-0000-0000-000000000000" /* -- Subscription Id -- */
Name="Visual Studio Premium with MSDN"
ManagementCertificate="" />
I'm trying to get credentials like this
Code:
string subscriptionId = "11111111-1111-1111-1111-111111111111"; // Subscription Id...
var credentials = new CertificateCloudCredentials(subscriptionId, x509Certificate2); // Will be varied here...
Where as my subscription id are mismatched. So, received this exception when i try to authenticate my request using "Certificates."
Hope this helps...