Develop Reference

X509Certificate2 usage issues since moving to Azure App Service

I have several remote servers that communicate with a central SOAP Service, where they can download the latest X509Certificate2 which can then be used to call a third-party API that requires this certificate to authenticate the requests.
Some of these remote servers are hosted by some of our clients on their own Windows Servers, which may be VMs or physical boxes, and others are hosted by us on Azure VMs.
We have had no previous issue with this functionality until recently when we moved our APIs from being hosted on a physical box to now being hosted in an Azure App Service (with an App Gateway handling requests).
What now happens is that all of the non-Azure servers, download the certificate successfully, but the certificate fails when used with the third-party rejecting the certificate with the error:
The request was aborted: Could not create SSL/TLS secure channel.
I have confirmed that if I manually copy the certificate to the servers, it works fine, but for some reason somewhere in the process of downloading it from the Azure SOAP API it now fails.
The code that we have to export the certificate on the API side so that it can be downloaded is something like:
[WebMethod]
public ClientCertificateMessage GetCertificate(LoginMessage login, string customer)
{
ClientCertificateMessage returnValue = new ClientCertificateMessage();
if (Authentication.VerifyLogin(login))
{
try
{
string filePathNameCertificate = ConfigurationManager.AppSettings["PathToLocalCert"].ToString() + customer + ".p12";
string filePathNamePassword = ConfigurationManager.AppSettings["PathToLocalCert"].ToString() + customer + ".txt";
string password = File.ReadAllText(filePathNamePassword);
X509Certificate2 x509Certificate2 = new X509Certificate2( filePathNameCertificate, password, X509KeyStorageFlags.Exportable);
returnValue.Certificate = x509Certificate2.Export(X509ContentType.Pkcs12, password);
}
catch (Exception ex)
{
// Log Error stuff here is removed
}
}
else
{
// error stuff here is removed
}
return returnValue;
}
The service that retrieves this certificate and uses it looks like:
public static X509Certificate2 GetCertificate(int certificateID, string password, string customer)
{
X509Certificate2 x509Certificate2 = null;
try
{
SoapClient client = new SoapClient();
Login login = CreateLogin();
ClientCertificateMessage clientCertificateMessage = null;
try
{
clientCertificateMessage = client.GetCertificate(login, customer);
}
catch (Exception ex)
{
// error logging removed
}
if ((clientCertificateMessage != null) && (clientCertificateMessage.Certificate != null))
{
using (CertificateData cd = new CertificateData())
{
dynamic revisedCertificate = new ExpandoObject();
revisedCertificate.Certificate = clientCertificateMessage.Certificate;
cd.Update(revisedCertificate, certificateID); // Save certificate data in database for later use
}
x509Certificate2 = new X509Certificate2(clientCertificateMessage.Certificate, password);
}
else
{
// handle logic got no certificate remove
}
}
catch (Exception ex)
{
// error logging removed
}
return x509Certificate2;
}
I cannot see anything that would explain why VMs on Azure continue to work, but all other VMs do not. We know those other servers can communicate and download the certificate, and when recreating the certificate from the byte array saved in the database the thumbprint and everything else matches.
I have seen other articles regarding Certificates on App Services where you need add Settings for WEBSITE_LOAD_CERTIFICATES or WEBSITE_LOAD_USER_PROFILE, however, we have not done this as the certificates do not fail to be generated.
Is there anything that I am missing where perhaps some odd Azure configuration or even some obvious technical reason for why a certificate downloaded fails, but the same certificate manually copied to the server works?
Thanks in advance for helping out.

Impersonation and potential Kerberos Double-Hop in Dotnet Core API

I'm trying to get an API to pass client credentials through to the database (on a different server) but experiencing something that smells very much like a Kerberos double-hop issue to me, however the systems people say that Kerberos delegation is set up appropriately on the servers in question. Below is a method I wrote to test this, and at the bottom is the response body I get in Postman when sending my own credentials to the API via NTLM. I get a similar response when shipping the credentials of another account I have access to. When shipping credentials I know to be bad, IIS stops my request & responds with a 401 without sending the request to the API; this is expected and required.
What can be causing this other than a Kerberos double-hop? Is there a flaw in my implementation of impersonation? I don't have access to either server in question, but I can supply the entire ugly exception encountered by the controller if needed.
/// <summary>
/// This endpoint should not support PUT. This method is simply a means by which to test impersonation
/// </summary>
[HttpPut]
public IActionResult PutMethod()
{
var bogusPutResult = new ImpersonationResult();
if (!(this.User.Identity is WindowsIdentity user)) return Problem("Cannot authenticate user");
var row1 = _myDbContext.MyTable.Find(1);
bogusPutResult.BeforeImpersonation = $"{WindowsIdentity.GetCurrent().Name} got a row with value {row1.Value}";
WindowsIdentity.RunImpersonated(user.AccessToken, () =>
{
string message;
try
{
var row2 = _myDbContext.MyTable.Find(2); // All domain users have SELECT access
message = $" got a row with value {row2.Value}";
}
catch (Exception e)
{
message = $" ate this: {e.GetType()}--{e.Message}";
}
bogusPutResult.DuringImpersonation = $"{WindowsIdentity.GetCurrent().Name} {message}";
});
bogusPutResult.AfterImpersonation = WindowsIdentity.GetCurrent().Name;
return Ok(bogusPutResult);
}
And the JSON returned by the API...
{
"BeforeImpersonation": "MYDOMAIN\\srv-apiserviceaccount got a row with value 34 ",
"DuringImpersonation": "MYDOMAIN\\finglixon ate this: Microsoft.Data.SqlClient.SqlException--Login failed for user 'NT AUTHORITY\\ANONYMOUS LOGON'.",
"AfterImpersonation": "MYDOMAIN\\srv-apiserviceaccount"
}

Why does Request Sync on HomeGraph API return 403 Forbidden?

Problem
When I call "Request Sync" on the Google HomeGraph API I receive a "403 Forbidden" response.
Background
I'm writing a Smart Home Action, and have successfully implemented SYNC, QUERY and EXECUTE. Testing on my mobile I can see and interact with devices okay. I'm now trying to implement Request Sync, but can't appear to interact with the API. I am making what seems to be successful requests for an Access Token. The token always begins with "ya29.c." which in my naïve understanding suggests an empty header and payload (trying it on https://jwt.io). However, when testing it at https://accounts.google.com/o/oauth2/tokeninfo?access_token= it appears valid, showing both my service account unique ID and the scope I intended. When I make a call to the API, either manually posting the data, or via Google's own code, it gives me a blunt 403 error. I do not know where I can get any more information on this error other than the exception objects. I'm new to GCP and couldn't find any sort of log. Given I've tried different methods and all return a 403 I'm inclined to suspect the issue is more a account or credential-related than the code, but can't be certain.
API Key
(I'm no longer able to reproduce any errors relating to API keys being missing or invalid).
Although the documentation doesn't show it, I've seen some people use an API key. When I don't include the API key with a p12 certificate, or include an incorrect one it errors (either with API key missing, or API key invalid accordingly). I have created an unrestricted API key in IAM, and am using that. I can't appear to explicitly relate this to HomeGraph API, but it says that it can call any API.
Code
This example fetches an access token, then tries to call the API via POST with and without the API key. It then tries to authenticate and call the API via the Google library code. Each fails with a 403.
using Google;
using Google.Apis.Auth.OAuth2;
using Google.Apis.HomeGraphService.v1;
using Google.Apis.HomeGraphService.v1.Data;
using Google.Apis.Services;
using Lambda.Core.Constants;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using static Google.Apis.HomeGraphService.v1.DevicesResource;
public class Example
{
public void RequestSync()
{
const string UrlWithoutKey = #"https://homegraph.googleapis.com/v1/devices:requestSync";
const string UrlWithKey = #"https://homegraph.googleapis.com/v1/devices:requestSync?key=" + OAuthConstants.GoogleApiKey;
string accessToken = this.GetAccessToken();
// Manual Attempt 1
try
{
string response = this.CallRequestSyncApiManually(accessToken, UrlWithoutKey);
}
catch (WebException ex)
{
// Receive 403, Forbidden
string msg = ex.Message;
}
// Manual Attempt 2
try
{
string response = this.CallRequestSyncApiManually(accessToken, UrlWithKey);
}
catch (WebException ex)
{
// Receive 403, Forbidden
string msg = ex.Message;
}
// SDK Attempt
try
{
this.CallRequestSyncApiWithSdk();
}
catch (GoogleApiException ex)
{
// Google.Apis.Requests.RequestError
// The caller does not have permission[403]
// Errors[Message[The caller does not have permission] Location[- ] Reason[forbidden] Domain[global]]
// at Google.Apis.Requests.ClientServiceRequest`1.ParseResponse(HttpResponseMessage response) in Src\Support\Google.Apis\Requests\ClientServiceRequest.cs:line 243
// at Google.Apis.Requests.ClientServiceRequest`1.Execute() in Src\Support\Google.Apis\Requests\ClientServiceRequest.cs:line 167
string msg = ex.Message;
}
}
private string GetAccessToken()
{
string defaultScope = "https://www.googleapis.com/auth/homegraph";
string serviceAccount = OAuthConstants.GoogleServiceAccountEmail; // "??????#??????.iam.gserviceaccount.com"
string certificateFile = OAuthConstants.CertificateFileName; // "??????.p12"
var oAuth2 = new GoogleOAuth2(defaultScope, serviceAccount, certificateFile); // As per https://stackoverflow.com/questions/26478694/how-to-produce-jwt-with-google-oauth2-compatible-algorithm-rsa-sha-256-using-sys
bool status = oAuth2.RequestAccessTokenAsync().Result;
// This access token at a glance appears invalid due to an empty header and payload,
// But verifies ok when tested here: https://accounts.google.com/o/oauth2/tokeninfo?access_token=
return oAuth2.AccessToken;
}
private string CallRequestSyncApiManually(string accessToken, string url)
{
string apiRequestBody = #"{""agentUserId"": """ + OAuthConstants.TestAgentUserId + #"""}";
var client = new HttpClient();
var request = (HttpWebRequest)WebRequest.Create(url);
var data = Encoding.ASCII.GetBytes(apiRequestBody);
request.Method = "POST";
request.Accept = "application/json";
request.ContentType = "application/json";
request.ContentLength = data.Length;
request.Headers.Add("Authorization", $"Bearer {accessToken}");
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
using (var stream = request.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
var response = (HttpWebResponse)request.GetResponse();
var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
return responseString;
}
private void CallRequestSyncApiWithSdk()
{
var certificate = new X509Certificate2(OAuthConstants.CertificateFileName, OAuthConstants.CertSecret, X509KeyStorageFlags.Exportable);
var credential = new ServiceAccountCredential(
new ServiceAccountCredential.Initializer(OAuthConstants.GoogleServiceAccountEmail)
{
Scopes = new[] { "https://www.googleapis.com/auth/homegraph" },
}.FromCertificate(certificate));
var service = new HomeGraphServiceService(
new BaseClientService.Initializer()
{
// Complains if API key is not provided, even though we're using a certificate from a Service Account
ApiKey = OAuthConstants.GoogleApiKey,
HttpClientInitializer = credential,
ApplicationName = OAuthConstants.ApplicationName,
});
var request = new RequestSyncRequest(
service,
new RequestSyncDevicesRequest
{
AgentUserId = OAuthConstants.TestAgentUserId
});
request.Execute();
}
}
Account Configuration
Account screenshots. (I'm not allowed to post images yet, so they're links)
HomeGraph is enabled
My API Key is unrestricted
My Service Account has Owner & Service Account Token Creator enabled
Updates
I have tried skipping manually obtaining the access token, as per Devunwired's suggestion. Whilst this does eliminate the error I was getting from not providing the API key, I still end up with the 403. My reasoning for doing the access token part manually was part of debugging a 403 I was getting with the API call. That way I could at least see part of the process working. I'm happy to use the library version for the solution as the access token doesn't appear to be the issue.
public void GoogleLibraryJsonCredentialExample()
{
try
{
GoogleCredential credential;
using (var stream = new FileStream(OAuthConstants.JsonCredentialsFileName, FileMode.Open, FileAccess.Read))
{
credential = GoogleCredential.FromStream(stream).CreateScoped(new[] { OAuthConstants.GoogleScope });
}
var service = new HomeGraphServiceService(
new BaseClientService.Initializer()
{
HttpClientInitializer = credential,
ApplicationName = OAuthConstants.ApplicationName,
});
var request = new RequestSyncRequest(
service,
new RequestSyncDevicesRequest
{
AgentUserId = OAuthConstants.TestAgentUserId
});
request.Execute();
}
catch (Exception ex)
{
// Receive 403, Forbidden
string msg = ex.Message;
}
}
Concerns
Is it possible that I need to be making the API call from a verified or white-listed domain? At the moment I'm running it from a console app running on my development machine. My understanding of domain verification is that it does not apply to incoming calls, and therefore shouldn't be the problem.

I am making what seems to be successful requests for an Access Token.
You should not need to manually request OAuth access tokens when using the Google client libraries. They generally handle this process internally using the credentials you provide from the GCP console.
Although the documentation doesn't show it, I've seen some people use an API key. Indeed, it is mandatory to include it for the SDK approach.
We do not recommend using the API key method to access the Home Graph API. You should be using service account credentials. API keys will technically work for the Request Sync method, but you will not be able to authenticate Report State using an API key.
The fact that you are receiving an error trying to build the HomeGraphServiceService without an API key may be indicative that the credential you are using isn't set up correctly (no private key or possibly missing scopes). The recommended method for supplying service account credentials is to download them in the JSON format rather than certificate, and the code to generate a credential from JSON should look something like this:
GoogleCredential credential;
using (var stream = new FileStream(serviceAccountCredentialFilePath, FileMode.Open, FileAccess.Read))
{
credential = GoogleCredential.FromStream(stream).CreateScoped(scopes);
}
You can find additional C# examples for authenticating APIs in the authentication guide.

The problem was nothing to do with my permission to talk to the HomeGraph API or that user. Instead it was where HomeGraph wanted to call my Smart Home Action, but the access token had expired. When attempting to refresh the token, an erroneous implementation on my part led to a blunt 403, which Google was then relaying back to me.
For those interested, the issue was that rather than omitting the expiry date for a token that should never expire, I was setting it to DateTime.MaxValue (subsequently sent through some further processing). Unfortunately when this is finally cast to an int, it is a value that exceeds int.Max. The subsequent time on the expiry was set to epoch (i.e. in the past), and therefore the token validation failed due to expiry.
For anyone else still having the same issue, double check your agentUserId matches exactly the value shown in your SYNC output payload. In my case I'd checked this.
Many thanks to anyone who's looked at this.

Error when calling any method on Service Management API

I'm looking to start an Azure runbook from a c# application which will be hosted on an Azure web app.
I'm using certificate authentication (in an attempt just to test that I can connect and retrieve some data)
Here's my code so far:
var cert = ConfigurationManager.AppSettings["mgmtCertificate"];
var creds = new Microsoft.Azure.CertificateCloudCredentials("<my-sub-id>",
new X509Certificate2(Convert.FromBase64String(cert)));
var client = new Microsoft.Azure.Management.Automation.AutomationManagementClient(creds, new Uri("https://management.core.windows.net/"));
var content = client.Runbooks.List("<resource-group-id>", "<automation-account-name>");
Every time I run this, no matter what certificate I use I get the same error:
An unhandled exception of type 'Hyak.Common.CloudException' occurred in Microsoft.Threading.Tasks.dll
Additional information: ForbiddenError: The server failed to authenticate the request. Verify that the certificate is valid and is associated with this subscription.
I've tried downloading the settings file which contains the automatically generated management certificate you get when you spin up the Azure account... nothing I do will let me talk to any of the Azure subscription
Am I missing something fundamental here?
Edit: some additional info...
So I decided to create an application and use the JWT authentication method.
I've added an application, given the application permissions to the Azure Service Management API and ensured the user is a co-administrator and I still get the same error, even with the token...
const string tenantId = "xx";
const string clientId = "xx";
var context = new AuthenticationContext(string.Format("https://login.windows.net/{0}", tenantId));
var user = "<user>";
var pwd = "<pass>";
var userCred = new UserCredential(user, pwd);
var result = context.AcquireToken("https://management.core.windows.net/", clientId, userCred);
var token = result.CreateAuthorizationHeader().Substring("Bearer ".Length); // Token comes back fine and I can inspect and see that it's valid for 1 hour - all looks ok...
var sub = "<subscription-id>";
var creds = new TokenCloudCredentials(sub, token);
var client = new AutomationManagementClient(creds, new Uri("https://management.core.windows.net/"));
var content = client.Runbooks.List("<resource-group>", "<automation-id>");
I've also tried using other Azure libs (like auth, datacentre etc) and I get the same error:
ForbiddenError: The server failed to authenticate the request. Verify that the certificate is valid and is associated with this subscription.
I'm sure it's just 1 tickbox I need to tick buried somewhere in that monolithic Management Portal but I've followed a few tutorials on how to do this and they all end up with this error...

public async Task StartAzureRunbook()
{
try
{
var subscriptionId = "azure subscription Id";
string base64cer = "****long string here****"; //taken from http://stackoverflow.com/questions/24999518/azure-api-the-server-failed-to-authenticate-the-request
var cert = new X509Certificate2(Convert.FromBase64String(base64cer));
var client = new Microsoft.Azure.Management.Automation.AutomationManagementClient(new CertificateCloudCredentials(subscriptionId, cert));
var ct = new CancellationToken();
var content = await client.Runbooks.ListByNameAsync("MyAutomationAccountName", "MyRunbookName", ct);
var firstOrDefault = content?.Runbooks.FirstOrDefault();
if (firstOrDefault != null)
{
var operation = client.Runbooks.Start("MyAutomationAccountName", new RunbookStartParameters(firstOrDefault.Id));
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
Also in portal:
1) Application is multitenant
2) Permissions to other applications section - Windows Azure Service Manager - Delegated permissions "Access Azure Service Management(preview)"

Ensure that your Management certificate has private key and was not made from the .CER file. The fact that you're not supplying a password when generating the X509Certificate object makes me think you're using public key only
Ensure that your Managemnet's certificate public key (.CER file) has been uploaded to the Azure management portal (legacy version, Management Certificate area)
Use CertificateCloudCredentials and not any other credential type of an object

Ok, stupid really but one of the tutorials I followed suggested installing the prerelease version of the libs.
Installing the preview (0.15.2-preview) has fixed the issue!

Does changing SSL Cert on a server break code?

EDIT:
What should I use to catch the error?
I have this where it fails to connect for an error message and defaults to a generic message.
public string ResponseError {
get {
string retVal = "";
try {
retVal = xmlElements.GetElementValue( FullResponse, "/VancoWS/Response/Errors/Error/ErrorDescription" );
} catch {
retVal = "There has been a problem processing your request. Please try again!";
}
return retVal;
}
}
A payment gateway said they made a change to their SSL Cert and since then our web application has not made a successful connection. The code was written in VS.NET 2008 .NET 3.5 I believe in C#. From what I can tell this looks like the code making the connection:
/* Method to perform web post */
private void SendBuffer(string strXml, out string fullResponse, out bool success)
{
String BaseAddress = Url + "?xml=";
try
{
System.Net.WebClient objRequest = new System.Net.WebClient();
objRequest.Encoding = System.Text.Encoding.ASCII;
byte[] buffer = System.Text.Encoding.ASCII.GetBytes(strXml);
byte[] responseBuffer = objRequest.UploadData(BaseAddress, "POST", buffer);
fullResponse = System.Text.Encoding.ASCII.GetString(responseBuffer);
success = true;
}
catch (Exception ex)
{
fullResponse = ex.ToString();
success = false;
}
}
The payment gateway says they are not getting any connections from our server to theirs. I contacted out webhost and they said they have not changed anything to block connections.
The host says they could access the URL fine with no invalid certificate.
URL in question: https://www.vancoservices.com/cgi-bin/ws2.vps
They also are still running SSLv3 so it's not a problem with protocol change or being forced to TLS 1.x
Any ideas as to what would break this?

Looks like they have set up a certificate which is either not signed by a valid authority or your web server doesn't have the authority that signed this certificate in its trusted CA.
To understand whether it is a problem with the certificate you may try connecting to the webserver that is executing this code, opening a webbrowser and attempting to connect to the address. The browser will tell you whether the certificate is valid or not (probably you will get a warning if it is not).
If you want to disable certificate validations in your code you may try the following:
System.Net.ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
Obviously this is not something you wanna be doing with a Production system, but you could just experiment with it to know whether it is a problem with the SSL certificate they set up and if it is self signed.

The "try" clause around the code is hiding any errors that are happening. One thing you could try is commenting out the try statement allowing the code to break and fall over with an error:
// try
{
...
}
or add a catch clause:
try
{
...
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}

just add the following to your config and change the log file path. this will give a sense if handshake/validation for cert is happening
http://msdn.microsoft.com/en-us/library/ty48b824(v=vs.110).aspx