Develop Reference

Why does Request Sync on HomeGraph API return 403 Forbidden?

Problem
When I call "Request Sync" on the Google HomeGraph API I receive a "403 Forbidden" response.
Background
I'm writing a Smart Home Action, and have successfully implemented SYNC, QUERY and EXECUTE. Testing on my mobile I can see and interact with devices okay. I'm now trying to implement Request Sync, but can't appear to interact with the API. I am making what seems to be successful requests for an Access Token. The token always begins with "ya29.c." which in my naïve understanding suggests an empty header and payload (trying it on https://jwt.io). However, when testing it at https://accounts.google.com/o/oauth2/tokeninfo?access_token= it appears valid, showing both my service account unique ID and the scope I intended. When I make a call to the API, either manually posting the data, or via Google's own code, it gives me a blunt 403 error. I do not know where I can get any more information on this error other than the exception objects. I'm new to GCP and couldn't find any sort of log. Given I've tried different methods and all return a 403 I'm inclined to suspect the issue is more a account or credential-related than the code, but can't be certain.
API Key
(I'm no longer able to reproduce any errors relating to API keys being missing or invalid).
Although the documentation doesn't show it, I've seen some people use an API key. When I don't include the API key with a p12 certificate, or include an incorrect one it errors (either with API key missing, or API key invalid accordingly). I have created an unrestricted API key in IAM, and am using that. I can't appear to explicitly relate this to HomeGraph API, but it says that it can call any API.
Code
This example fetches an access token, then tries to call the API via POST with and without the API key. It then tries to authenticate and call the API via the Google library code. Each fails with a 403.
using Google;
using Google.Apis.Auth.OAuth2;
using Google.Apis.HomeGraphService.v1;
using Google.Apis.HomeGraphService.v1.Data;
using Google.Apis.Services;
using Lambda.Core.Constants;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using static Google.Apis.HomeGraphService.v1.DevicesResource;
public class Example
{
public void RequestSync()
{
const string UrlWithoutKey = #"https://homegraph.googleapis.com/v1/devices:requestSync";
const string UrlWithKey = #"https://homegraph.googleapis.com/v1/devices:requestSync?key=" + OAuthConstants.GoogleApiKey;
string accessToken = this.GetAccessToken();
// Manual Attempt 1
try
{
string response = this.CallRequestSyncApiManually(accessToken, UrlWithoutKey);
}
catch (WebException ex)
{
// Receive 403, Forbidden
string msg = ex.Message;
}
// Manual Attempt 2
try
{
string response = this.CallRequestSyncApiManually(accessToken, UrlWithKey);
}
catch (WebException ex)
{
// Receive 403, Forbidden
string msg = ex.Message;
}
// SDK Attempt
try
{
this.CallRequestSyncApiWithSdk();
}
catch (GoogleApiException ex)
{
// Google.Apis.Requests.RequestError
// The caller does not have permission[403]
// Errors[Message[The caller does not have permission] Location[- ] Reason[forbidden] Domain[global]]
// at Google.Apis.Requests.ClientServiceRequest`1.ParseResponse(HttpResponseMessage response) in Src\Support\Google.Apis\Requests\ClientServiceRequest.cs:line 243
// at Google.Apis.Requests.ClientServiceRequest`1.Execute() in Src\Support\Google.Apis\Requests\ClientServiceRequest.cs:line 167
string msg = ex.Message;
}
}
private string GetAccessToken()
{
string defaultScope = "https://www.googleapis.com/auth/homegraph";
string serviceAccount = OAuthConstants.GoogleServiceAccountEmail; // "??????#??????.iam.gserviceaccount.com"
string certificateFile = OAuthConstants.CertificateFileName; // "??????.p12"
var oAuth2 = new GoogleOAuth2(defaultScope, serviceAccount, certificateFile); // As per https://stackoverflow.com/questions/26478694/how-to-produce-jwt-with-google-oauth2-compatible-algorithm-rsa-sha-256-using-sys
bool status = oAuth2.RequestAccessTokenAsync().Result;
// This access token at a glance appears invalid due to an empty header and payload,
// But verifies ok when tested here: https://accounts.google.com/o/oauth2/tokeninfo?access_token=
return oAuth2.AccessToken;
}
private string CallRequestSyncApiManually(string accessToken, string url)
{
string apiRequestBody = #"{""agentUserId"": """ + OAuthConstants.TestAgentUserId + #"""}";
var client = new HttpClient();
var request = (HttpWebRequest)WebRequest.Create(url);
var data = Encoding.ASCII.GetBytes(apiRequestBody);
request.Method = "POST";
request.Accept = "application/json";
request.ContentType = "application/json";
request.ContentLength = data.Length;
request.Headers.Add("Authorization", $"Bearer {accessToken}");
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
using (var stream = request.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
var response = (HttpWebResponse)request.GetResponse();
var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
return responseString;
}
private void CallRequestSyncApiWithSdk()
{
var certificate = new X509Certificate2(OAuthConstants.CertificateFileName, OAuthConstants.CertSecret, X509KeyStorageFlags.Exportable);
var credential = new ServiceAccountCredential(
new ServiceAccountCredential.Initializer(OAuthConstants.GoogleServiceAccountEmail)
{
Scopes = new[] { "https://www.googleapis.com/auth/homegraph" },
}.FromCertificate(certificate));
var service = new HomeGraphServiceService(
new BaseClientService.Initializer()
{
// Complains if API key is not provided, even though we're using a certificate from a Service Account
ApiKey = OAuthConstants.GoogleApiKey,
HttpClientInitializer = credential,
ApplicationName = OAuthConstants.ApplicationName,
});
var request = new RequestSyncRequest(
service,
new RequestSyncDevicesRequest
{
AgentUserId = OAuthConstants.TestAgentUserId
});
request.Execute();
}
}
Account Configuration
Account screenshots. (I'm not allowed to post images yet, so they're links)
HomeGraph is enabled
My API Key is unrestricted
My Service Account has Owner & Service Account Token Creator enabled
Updates
I have tried skipping manually obtaining the access token, as per Devunwired's suggestion. Whilst this does eliminate the error I was getting from not providing the API key, I still end up with the 403. My reasoning for doing the access token part manually was part of debugging a 403 I was getting with the API call. That way I could at least see part of the process working. I'm happy to use the library version for the solution as the access token doesn't appear to be the issue.
public void GoogleLibraryJsonCredentialExample()
{
try
{
GoogleCredential credential;
using (var stream = new FileStream(OAuthConstants.JsonCredentialsFileName, FileMode.Open, FileAccess.Read))
{
credential = GoogleCredential.FromStream(stream).CreateScoped(new[] { OAuthConstants.GoogleScope });
}
var service = new HomeGraphServiceService(
new BaseClientService.Initializer()
{
HttpClientInitializer = credential,
ApplicationName = OAuthConstants.ApplicationName,
});
var request = new RequestSyncRequest(
service,
new RequestSyncDevicesRequest
{
AgentUserId = OAuthConstants.TestAgentUserId
});
request.Execute();
}
catch (Exception ex)
{
// Receive 403, Forbidden
string msg = ex.Message;
}
}
Concerns
Is it possible that I need to be making the API call from a verified or white-listed domain? At the moment I'm running it from a console app running on my development machine. My understanding of domain verification is that it does not apply to incoming calls, and therefore shouldn't be the problem.

I am making what seems to be successful requests for an Access Token.
You should not need to manually request OAuth access tokens when using the Google client libraries. They generally handle this process internally using the credentials you provide from the GCP console.
Although the documentation doesn't show it, I've seen some people use an API key. Indeed, it is mandatory to include it for the SDK approach.
We do not recommend using the API key method to access the Home Graph API. You should be using service account credentials. API keys will technically work for the Request Sync method, but you will not be able to authenticate Report State using an API key.
The fact that you are receiving an error trying to build the HomeGraphServiceService without an API key may be indicative that the credential you are using isn't set up correctly (no private key or possibly missing scopes). The recommended method for supplying service account credentials is to download them in the JSON format rather than certificate, and the code to generate a credential from JSON should look something like this:
GoogleCredential credential;
using (var stream = new FileStream(serviceAccountCredentialFilePath, FileMode.Open, FileAccess.Read))
{
credential = GoogleCredential.FromStream(stream).CreateScoped(scopes);
}
You can find additional C# examples for authenticating APIs in the authentication guide.

The problem was nothing to do with my permission to talk to the HomeGraph API or that user. Instead it was where HomeGraph wanted to call my Smart Home Action, but the access token had expired. When attempting to refresh the token, an erroneous implementation on my part led to a blunt 403, which Google was then relaying back to me.
For those interested, the issue was that rather than omitting the expiry date for a token that should never expire, I was setting it to DateTime.MaxValue (subsequently sent through some further processing). Unfortunately when this is finally cast to an int, it is a value that exceeds int.Max. The subsequent time on the expiry was set to epoch (i.e. in the past), and therefore the token validation failed due to expiry.
For anyone else still having the same issue, double check your agentUserId matches exactly the value shown in your SYNC output payload. In my case I'd checked this.
Many thanks to anyone who's looked at this.

502.3 when calling AcquireTokenAsync in .NET Core 2.2

I'm encountering a problem. I am using Microsoft Graph to get the current logged in user via OnBehalfOfMsGraphAuthenticationProvider.cs as seen in the following solution.
This has been working flawlessly, but I have been doing some refactoring, and suddenly I get an error when trying to execute my authContext.AcquireTokenAsync() method.
HTTP Error 502.3 - Bad Gateway
The code in question looks like this:
public async Task AuthenticateRequestAsync(HttpRequestMessage request) {
var httpContext = _httpContextAccessor.HttpContext;
//Get the access token used to call this API
string token = await httpContext.GetTokenAsync("access_token");
//We are passing an *assertion* to Azure AD about the current user
//Here we specify that assertion's type, that is a JWT Bearer token
string assertionType = "urn:ietf:params:oauth:grant-type:jwt-bearer";
//User name is needed here only for ADAL, it is not passed to AAD
//ADAL uses it to find a token in the cache if available
var user = httpContext.User;
string userName =
user.FindFirst(ClaimTypes.Upn).Value ?? user.FindFirst(ClaimTypes.Email).Value;
var userAssertion = new UserAssertion(token, assertionType, userName);
//Construct the token cache
var cache = new DistributedTokenCache(user, _distributedCache,
_loggerFactory, _dataProtectionProvider);
var authContext = new AuthenticationContext(_configuration["AzureAd:Instance"] +
_configuration["AzureAd:TenantId"], true, cache);
var clientCredential = new ClientCredential(_configuration["AzureAd:ClientId"],
(string) _configuration["AzureAd:ClientSecret"]);
//Acquire access token
var result = await authContext.AcquireTokenAsync("https://graph.microsoft.com", clientCredential, userAssertion); //This is where it crashes
//Set the authentication header
request.Headers.Authorization = new AuthenticationHeaderValue(result.AccessTokenType, result.AccessToken);
}
I am calling it from my OrdersController:
// POST: api/Orders
[HttpPost]
public async Task<IActionResult> CreateAsync([FromBody] OrderDTO order) {
if (!ModelState.IsValid) {
return BadRequest(ModelState);
}
var graphUser = await this.graphApiService.GetUserProfileAsync();
The refactoring has consisted of dividing my solution into two class library projects and one web project - the latter has the controllers and the React app. GraphAPiClient and the provider are located in the Core library like this:
Screenshot of architecture

So, it turns out that the problem appeared when I upgraded the package Microsoft.IdentityModel.Clients.ActiveDirectory from v3.19.8 to v4.4.1. For some reason, no versions above v3.19.8 work with my code, causing it to crash when I try to make the call to https://graph.microsoft.com, but as soon as I downgraded the problem disappeared.

Try using AcquireToken instead of AcquireTokenAsync
azureAuthenticationContext.AcquireToken

Cannot access child value on JValue in Azure Mobile App

I recently implemented custom authentication with Azure Mobile App - All the server side works fine and also my web application which is using that mobile app service is working fine. I tested the server-side in details with POSTMAN and with different scenarios, everything works fine until I try to LoginAsync on Xamarin.
When I pass email and password in POSTMAN, I get the following response as a clear indication that it is working
but when I send a request from my app using LoginAsync I get the following error.
Cannot access child value on Newtonsoft.Json.Linq.JValue
My code to send request is fairly simple as following
public async Task<bool> Authenticate()
{
string username = "todo#gmail.com";
string password = "todo";
string message = string.Empty;
var success = false;
var credentials = new JObject
{
["email"] = username,
["password"] = password
};
try
{
MobileServiceUser user = await client.LoginAsync("CustomAuth", credentials);
if (user != null)
{
success = true;
CreateAndShowDialog("OK", "Auth");
}
}
catch (Exception ex)
{
CreateAndShowDialog(ex, "Auth Error");
}
return success;
}
where I am calling it as follows
private MobileServiceClient client;
client = new MobileServiceClient(applicationURL);
await Authenticate();
Any idea why I am getting Cannot access child value on Newtonsoft.Json.Linq.JValue error?
Cheers
EDIT POST
As a workaround, I am temporarily using InvokeApiAsync with JObject.FromObject instead of LoginAsync
await client.InvokeApiAsync("/.auth/login/CustomAuth", JObject.FromObject(credentials), HttpMethod.Post, null);
I am still not sure why LoginAsync does not work - Until I find a solution I will keep using InvokdeApiAsync as a workaround

AFAIK, your initialization for credentials is correct. For the below error:
Cannot access child value on Newtonsoft.Json.Linq.JValue
I checked your testing result via POSTMAN and found that you did not return userId to your client. The essential properties returned to your client would look like as follows:
{
"authenticationToken":"***",
"user":{
"userId":"***"
}
}
When using MobileServiceClient.LoginAsync, the client SDK would internally invoke LoginAsync() method under MobileServiceAuthentication.cs as follows:
JToken authToken = JToken.Parse(response);
// Get the Mobile Services auth token and user data
this.Client.CurrentUser = new MobileServiceUser((string)authToken["user"]["userId"]);
this.Client.CurrentUser.MobileServiceAuthenticationToken = (string)authToken[LoginAsyncAuthenticationTokenKey];
You would find that it would try to extract the userId property under user to construct the MobileServiceUser instance and assign to MobileServiceClient.CurrentUser.

Authentication to ASP.NET Web Api using ADFS

I am in the situation that I need to access a ASP.NET Web Api that is using ADFS for authentication. I can hit it reliably through my browser by going through the ADFS login portal and getting the relevant FedAuth cookie. Unfortunately I need to access it from outside of a dedicated browser for use in a mobile app. The project is pretty much a slightly modified version of the standard visual studio web api template set up for Work and School Authentication (on-premises) and set up for cookie authentication.
bit of code from Startup.Auth.cs:
public void Configuration(IAppBuilder app)
{
app.SetDefaultSignInAsAuthenticationType(CookieAuthenticationDefaults.AuthenticationType);
app.UseWsFederationAuthentication(
new WsFederationAuthenticationOptions
{
Wtrealm = realm,
MetadataAddress = adfsMetadata
});
app.UseCookieAuthentication(new CookieAuthenticationOptions
{
AuthenticationType = WsFederationAuthenticationDefaults.AuthenticationType
});
}
I can't seem to figure out where to start. I've tried requesting a access token from the ADFS and can get different versions of SAML assertions using relevant login info, but it gets rejected by the web API. Have I misunderstood how it's supposed to work?
From my understanding it's supposed to go like this:
How I think it's supposed to work
App requests a authentication token from the ADFS
ADFS gives the requestee an auth token if the information provided was correct
App makes request to the web API and sending the token along inside a cookie called FedAuth(by default anyway) as a base64 encoded string
Web Api sends the token to the ADFS to find out if the token is correct.
ADFS responds with some sort of success message
Web Api responds to the app either with a rejection or a piece of data depending on how authentication went.
This is what I have right now while trying to figure out how to get a hold of the correct tokens.
using System;
using System.IdentityModel.Protocols.WSTrust;
using System.IdentityModel.Tokens;
using System.Net;
using System.Net.Http;
using System.ServiceModel;
using System.ServiceModel.Security;
using Thinktecture.IdentityModel.Extensions;
using Thinktecture.IdentityModel.WSTrust;
namespace ConsoleApplication1
{
class Program
{
private const string UserName = "USERNAME";
private const string Password = "PASSWORD";
private const string Domain = "DOMAIN";
private const string ADFSEndpoint = "ADFS ENDPOINT";
private const string ApiBaseUri = "THE API";
private const string ApiEndPoint = "AN ENDPOINT";
static void Main(string[] args)
{
SecurityToken token = RequestSecurityToken(); // Obtain security token from ADFS.
CallApi(token); // Call api.
Console.ReadKey(); // Stop console from closing
}
private static SecurityToken RequestSecurityToken()
{
var trustChannelFactory =
new WSTrustChannelFactory(new UserNameWSTrustBinding(SecurityMode.TransportWithMessageCredential),
new EndpointAddress(new Uri(ADFSEndpoint)))
{
TrustVersion = TrustVersion.WSTrust13,
Credentials = { UserName = { UserName = UserName + "#" + Domain, Password = Password } },
};
var requestSecurityToken = new RequestSecurityToken
{
RequestType = RequestTypes.Issue,
KeyType = KeyTypes.Bearer,
AppliesTo = new EndpointReference(ApiBaseUri)
};
RequestSecurityTokenResponse response;
var securityToken = trustChannelFactory.CreateChannel().Issue(requestSecurityToken, out response);
return securityToken;
}
private static async void CallApi(SecurityToken securityToken)
{
using (var handler = new HttpClientHandler { CookieContainer = new CookieContainer() })
{
using (var client = new HttpClient(handler))
{
handler.CookieContainer.MaxCookieSize = 8000; // Trying to make sure I can fit it in the cookie
var cookie = new Cookie {
Name = "FedAuth",
Value = Base64Encode(securityToken.ToTokenXmlString()),
HttpOnly = true,
Secure = true
};
handler.CookieContainer.Add(new Uri(ApiBaseUri), cookie);
var response = client.GetAsync(new Uri(ApiBaseUri + ApiEndPoint)).Result;
string result = await response.Content.ReadAsStringAsync();
Console.WriteLine(result);
}
}
}
public static string Base64Encode(string plainText)
{
var plainTextBytes = System.Text.Encoding.UTF8.GetBytes(plainText);
return System.Convert.ToBase64String(plainTextBytes);
}
}
}
I can't quite remember what code I based my example of, but if anyone can point me in the right direction or tell me where I fucked up I'd appreciate it.
Edit: Sorry, forgot to add what I am getting.
The Web Api vomits out a bunch of debug information because an exception was thrown, telling me that a SecurityContextToken is expected instead of a saml:Assertion that I am apparently getting. Maybe my googlefoo is not powerful enough, but I can't seem to figure out where to start with this. Can I setup the api to accept SAML assertions or do I need to request the token in a different way?

You can't use WS-Fed to call a web API. You need OpenID Connect / OAuth as in Calling a web API in a web app using Azure AD and OpenID Connect.
It's for Azure AD but it does illustrate the flow.
What version of ADFS?
If 2.0, there is no OAuth support.
If 3.0, web API only - refer Securing a Web API with ADFS on WS2012 R2 Got Even Easier.
If 4.0, you have the full stack.

Mobile Service Client throws exception when response is not found

I am trying to port an application from an azure mobile service to an azure web app. (the mobile service was working). I have added microsoft account authentication to the web-app, and the web app api has a MobileAppController attribute. I have a Universal windows app front end that calls the api. The app first checks if a player is in the database, if not I get a not found response. If I call the method using the following code with the MobileServiceClient I get an exception.
private async Task<HttpResponseMessage> GetAZMAsyncP(string apiext, IDictionary<string,string> param )
{
string myuri = String.Format("{0}{1}", urlbase, apiext);
//client is the MobileServiceClient that is correctly logged in
//I do not get response which is 404 not found, I get an exception "The request could not be completed, Not Found"
var response = await client.InvokeApiAsync(myuri, System.Net.Http.HttpMethod.Get, param);
return response;
}
If I call the api from an httpclient and add my own headers, which the mobile client is supposed to do for me, then I get the response as requested. Here is the code:
private async static Task<HttpResponseMessage> GetAZAsync(string apiext)
{
string completeUrl = String.Format("{0}{1}", urlbase, apiext);
// Call out to AZ
using (var http = new HttpClient())
{
// http.BaseAddress = new Uri(completeUrl);
HttpRequestMessage rq = new HttpRequestMessage()
{
RequestUri = new Uri(completeUrl),
Method = HttpMethod.Get
};
addauthheader(rq);
var response = await http.SendAsync(rq);
return response;
}
}
private static void addauthheader(HttpRequestMessage rq)
{
MobileServiceUser user = App.client.CurrentUser;
rq.Headers.Add("X-ZUMO-FEATURES", "AT,QS");
rq.Headers.Add("X-ZUMO-INSTALLATION-ID",
"ff90f37e-0c03-4c52-a343-af711752e383");
rq.Headers.Add("X-ZUMO-AUTH", user.MobileServiceAuthenticationToken);
rq.Headers.Add("Accept", "application/json");
rq.Headers.Add("User-Agent", "ZUMO/2.1");
rq.Headers.Add("User-Agent",
"(lang = Managed; os = Windows Store; os_version = --; arch = X86; version = 2.1.40707.0)");
rq.Headers.Add("X-ZUMO-VERSION",
"ZUMO/2.1(lang = Managed; os = Windows Store; os_version = --; arch = X86; version = 2.1.40707.0)");
rq.Headers.Add("ZUMO-API-VERSION", "2.0.0");
}
You can try this out as it is live (and buggy).
https://gamenote2.azurewebsites.net/api/Players?displayname=Paul Goldschmidt&teamid=arizona-diamondbacks
Should give you a 404,
https://gamenote2.azurewebsites.net/api/Players?displayname=Chase Utley&teamid=los-angeles-dodgers
should give you a chase utley object. (YOu will be asked to log into a Microsoft Account).
So my questions: 1. Can I fix the mobileclient call to get a response instead of an execption
2. Is there any good reason for me to be spending so much time on this.

If you examine the exception, you will note that the status code is in there - it's just in a property that is not serialized. Just surround your InvokeApiAsync() call with a try/catch and test for the StatusCode. It should be a lot easier than writing your own HTTP Client code for the same purpose.
Specifically, MobileServiceInvalidOperationException contains the HttpResponse of the failed request, so you can check exception.Response.StatusCode value.