Develop Reference

Why does Request Sync on HomeGraph API return 403 Forbidden?

Problem
When I call "Request Sync" on the Google HomeGraph API I receive a "403 Forbidden" response.
Background
I'm writing a Smart Home Action, and have successfully implemented SYNC, QUERY and EXECUTE. Testing on my mobile I can see and interact with devices okay. I'm now trying to implement Request Sync, but can't appear to interact with the API. I am making what seems to be successful requests for an Access Token. The token always begins with "ya29.c." which in my naïve understanding suggests an empty header and payload (trying it on https://jwt.io). However, when testing it at https://accounts.google.com/o/oauth2/tokeninfo?access_token= it appears valid, showing both my service account unique ID and the scope I intended. When I make a call to the API, either manually posting the data, or via Google's own code, it gives me a blunt 403 error. I do not know where I can get any more information on this error other than the exception objects. I'm new to GCP and couldn't find any sort of log. Given I've tried different methods and all return a 403 I'm inclined to suspect the issue is more a account or credential-related than the code, but can't be certain.
API Key
(I'm no longer able to reproduce any errors relating to API keys being missing or invalid).
Although the documentation doesn't show it, I've seen some people use an API key. When I don't include the API key with a p12 certificate, or include an incorrect one it errors (either with API key missing, or API key invalid accordingly). I have created an unrestricted API key in IAM, and am using that. I can't appear to explicitly relate this to HomeGraph API, but it says that it can call any API.
Code
This example fetches an access token, then tries to call the API via POST with and without the API key. It then tries to authenticate and call the API via the Google library code. Each fails with a 403.
using Google;
using Google.Apis.Auth.OAuth2;
using Google.Apis.HomeGraphService.v1;
using Google.Apis.HomeGraphService.v1.Data;
using Google.Apis.Services;
using Lambda.Core.Constants;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using static Google.Apis.HomeGraphService.v1.DevicesResource;
public class Example
{
public void RequestSync()
{
const string UrlWithoutKey = #"https://homegraph.googleapis.com/v1/devices:requestSync";
const string UrlWithKey = #"https://homegraph.googleapis.com/v1/devices:requestSync?key=" + OAuthConstants.GoogleApiKey;
string accessToken = this.GetAccessToken();
// Manual Attempt 1
try
{
string response = this.CallRequestSyncApiManually(accessToken, UrlWithoutKey);
}
catch (WebException ex)
{
// Receive 403, Forbidden
string msg = ex.Message;
}
// Manual Attempt 2
try
{
string response = this.CallRequestSyncApiManually(accessToken, UrlWithKey);
}
catch (WebException ex)
{
// Receive 403, Forbidden
string msg = ex.Message;
}
// SDK Attempt
try
{
this.CallRequestSyncApiWithSdk();
}
catch (GoogleApiException ex)
{
// Google.Apis.Requests.RequestError
// The caller does not have permission[403]
// Errors[Message[The caller does not have permission] Location[- ] Reason[forbidden] Domain[global]]
// at Google.Apis.Requests.ClientServiceRequest`1.ParseResponse(HttpResponseMessage response) in Src\Support\Google.Apis\Requests\ClientServiceRequest.cs:line 243
// at Google.Apis.Requests.ClientServiceRequest`1.Execute() in Src\Support\Google.Apis\Requests\ClientServiceRequest.cs:line 167
string msg = ex.Message;
}
}
private string GetAccessToken()
{
string defaultScope = "https://www.googleapis.com/auth/homegraph";
string serviceAccount = OAuthConstants.GoogleServiceAccountEmail; // "??????#??????.iam.gserviceaccount.com"
string certificateFile = OAuthConstants.CertificateFileName; // "??????.p12"
var oAuth2 = new GoogleOAuth2(defaultScope, serviceAccount, certificateFile); // As per https://stackoverflow.com/questions/26478694/how-to-produce-jwt-with-google-oauth2-compatible-algorithm-rsa-sha-256-using-sys
bool status = oAuth2.RequestAccessTokenAsync().Result;
// This access token at a glance appears invalid due to an empty header and payload,
// But verifies ok when tested here: https://accounts.google.com/o/oauth2/tokeninfo?access_token=
return oAuth2.AccessToken;
}
private string CallRequestSyncApiManually(string accessToken, string url)
{
string apiRequestBody = #"{""agentUserId"": """ + OAuthConstants.TestAgentUserId + #"""}";
var client = new HttpClient();
var request = (HttpWebRequest)WebRequest.Create(url);
var data = Encoding.ASCII.GetBytes(apiRequestBody);
request.Method = "POST";
request.Accept = "application/json";
request.ContentType = "application/json";
request.ContentLength = data.Length;
request.Headers.Add("Authorization", $"Bearer {accessToken}");
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
using (var stream = request.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
var response = (HttpWebResponse)request.GetResponse();
var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
return responseString;
}
private void CallRequestSyncApiWithSdk()
{
var certificate = new X509Certificate2(OAuthConstants.CertificateFileName, OAuthConstants.CertSecret, X509KeyStorageFlags.Exportable);
var credential = new ServiceAccountCredential(
new ServiceAccountCredential.Initializer(OAuthConstants.GoogleServiceAccountEmail)
{
Scopes = new[] { "https://www.googleapis.com/auth/homegraph" },
}.FromCertificate(certificate));
var service = new HomeGraphServiceService(
new BaseClientService.Initializer()
{
// Complains if API key is not provided, even though we're using a certificate from a Service Account
ApiKey = OAuthConstants.GoogleApiKey,
HttpClientInitializer = credential,
ApplicationName = OAuthConstants.ApplicationName,
});
var request = new RequestSyncRequest(
service,
new RequestSyncDevicesRequest
{
AgentUserId = OAuthConstants.TestAgentUserId
});
request.Execute();
}
}
Account Configuration
Account screenshots. (I'm not allowed to post images yet, so they're links)
HomeGraph is enabled
My API Key is unrestricted
My Service Account has Owner & Service Account Token Creator enabled
Updates
I have tried skipping manually obtaining the access token, as per Devunwired's suggestion. Whilst this does eliminate the error I was getting from not providing the API key, I still end up with the 403. My reasoning for doing the access token part manually was part of debugging a 403 I was getting with the API call. That way I could at least see part of the process working. I'm happy to use the library version for the solution as the access token doesn't appear to be the issue.
public void GoogleLibraryJsonCredentialExample()
{
try
{
GoogleCredential credential;
using (var stream = new FileStream(OAuthConstants.JsonCredentialsFileName, FileMode.Open, FileAccess.Read))
{
credential = GoogleCredential.FromStream(stream).CreateScoped(new[] { OAuthConstants.GoogleScope });
}
var service = new HomeGraphServiceService(
new BaseClientService.Initializer()
{
HttpClientInitializer = credential,
ApplicationName = OAuthConstants.ApplicationName,
});
var request = new RequestSyncRequest(
service,
new RequestSyncDevicesRequest
{
AgentUserId = OAuthConstants.TestAgentUserId
});
request.Execute();
}
catch (Exception ex)
{
// Receive 403, Forbidden
string msg = ex.Message;
}
}
Concerns
Is it possible that I need to be making the API call from a verified or white-listed domain? At the moment I'm running it from a console app running on my development machine. My understanding of domain verification is that it does not apply to incoming calls, and therefore shouldn't be the problem.

I am making what seems to be successful requests for an Access Token.
You should not need to manually request OAuth access tokens when using the Google client libraries. They generally handle this process internally using the credentials you provide from the GCP console.
Although the documentation doesn't show it, I've seen some people use an API key. Indeed, it is mandatory to include it for the SDK approach.
We do not recommend using the API key method to access the Home Graph API. You should be using service account credentials. API keys will technically work for the Request Sync method, but you will not be able to authenticate Report State using an API key.
The fact that you are receiving an error trying to build the HomeGraphServiceService without an API key may be indicative that the credential you are using isn't set up correctly (no private key or possibly missing scopes). The recommended method for supplying service account credentials is to download them in the JSON format rather than certificate, and the code to generate a credential from JSON should look something like this:
GoogleCredential credential;
using (var stream = new FileStream(serviceAccountCredentialFilePath, FileMode.Open, FileAccess.Read))
{
credential = GoogleCredential.FromStream(stream).CreateScoped(scopes);
}
You can find additional C# examples for authenticating APIs in the authentication guide.

The problem was nothing to do with my permission to talk to the HomeGraph API or that user. Instead it was where HomeGraph wanted to call my Smart Home Action, but the access token had expired. When attempting to refresh the token, an erroneous implementation on my part led to a blunt 403, which Google was then relaying back to me.
For those interested, the issue was that rather than omitting the expiry date for a token that should never expire, I was setting it to DateTime.MaxValue (subsequently sent through some further processing). Unfortunately when this is finally cast to an int, it is a value that exceeds int.Max. The subsequent time on the expiry was set to epoch (i.e. in the past), and therefore the token validation failed due to expiry.
For anyone else still having the same issue, double check your agentUserId matches exactly the value shown in your SYNC output payload. In my case I'd checked this.
Many thanks to anyone who's looked at this.

Reading WebService Wsdl using c#.Net

We have a SOAP based web service and we are able to read its wsdl when we type in the url in Browser. We sit behind a proxy in our network but its not blocking anything and we are always able to read wsdl using browser.But when we enter the url in Browser say http://ist.services/CoreServices/coreservices?wsdl it asks for username and password which is not same as my windows credentials. So when i enter the username and password shared by the dev team , it returns the wsdl page. Please note that this webservice is developed and deployed on java based server.
How do i do the same in c#.net code and how do i pass the Security Crednetials in DiscoveryClientProtocol? I tried the below code which works for the webservices which doesn't ask for the Security credentials.
// Specify the URL to discover.
string sourceUrl = "http://ist.services/CoreServices/coreservices?wsdl";
string outputDirectory = "C:\\Temp";
DiscoveryClientProtocol client = new DiscoveryClientProtocol();
var credentials = new NetworkCredential("sunuser1", "xxxxxxx", "");
WebProxy proxy = new WebProxy("http://proxy.bingo:8000/", true) { Credentials = credentials };
client.Credentials = credentials;
// Use default credentials to access the URL being discovered.
//client.Credentials = credentials;//CredentialCache.DefaultCredentials;
client.Proxy = proxy;
String DiscoverMode = "DiscoverAny";
String ResolveMode = "ResolveAll";
try
{
DiscoveryDocument doc;
// Check to see if whether the user wanted to read in existing discovery results.
if (DiscoverMode == "ReadAll")
{
DiscoveryClientResultCollection results = client.ReadAll(Path.Combine("C:\\Temp", "results.discomap"));
//SaveMode.Value = "NoSave";
}
else
{
// Check to see if whether the user wants the capability to discover any kind of discoverable document.
if (DiscoverMode == "DiscoverAny")
{
doc = client.DiscoverAny(sourceUrl);
}
else
// Discover only discovery documents, which might contain references to other types of discoverable documents.
{
doc = client.Discover(sourceUrl);
}
// Check to see whether the user wants to resolve all possible references from the supplied URL.
if (ResolveMode == "ResolveAll")
client.ResolveAll();
else
{
// Check to see whether the user wants to resolve references nested more than one level deep.
if (ResolveMode == "ResolveOneLevel")
client.ResolveOneLevel();
else
Console.WriteLine("empty");
}
}
}
catch (Exception e2)
{
//DiscoveryResultsGrid.Columns.Clear();
//Status.Text = e2.Message;
Console.WriteLine(e2.Message);
}
// If documents were discovered, display the results in a data grid.
if (client.Documents.Count > 0)
Console.WriteLine(client);
}
}
Since the code didn't help me much , i opened the fiddler to trace the http calls when i manual read the wsdl in browser and i see it takes the credentials i entered as "Authorization: Basic cGDFDdsfdfsdsfdsgsgfg=" . In fiddler i see three calls with responses 401,302 and 200. But in my c#.net code i don't get the 200 response and it always throws me the 404 error.
I further debugged this and in httpresponse of client object i see the flag status as INVOCATION_FLAGS_INITIALIZED | INVOCATION_FLAGS_NEED_SECURITY
So looks like i need to pass the credentials as Security Credentials rather than Network credentials.

The below code has fixed the issue.
CredentialCache myCredentialCache = new CredentialCache { { new Uri(sourceUrl),
"Basic", networkCredential } };
discoveryClientProtocol.AllowAutoRedirect = true;
discoveryClientProtocol.Credentials = myCredentialCache;

setExpressCheckout and SSL/TLS error

I'm trying to develop a simple application that will enable users to purchase services off a website through the Paypal API. This application is running on ASP.NET with C#.
I have had very little luck trying to get the Paypal API to co-operate. The method I'm calling is SetExpressCheckout with all the appropriate variables.
I did my research and discovered that since I'm testing in Localhost, it may affect Paypal's ability to communicate with the application. So the next thing I tried was accessing my application through an open port and a publicly accessible IP address, but the same error occurs on the call to SetExpressCheckout.
Here is the error:
Exception Details: System.Net.WebException: The request was aborted: Could not create SSL/TLS secure channel.
Source Error:
Line 1790: [return: System.Xml.Serialization.XmlElementAttribute("SetExpressCheckoutResponse", Namespace="urn:ebay:api:PayPalAPI")]
Line 1791: public SetExpressCheckoutResponseType SetExpressCheckout([System.Xml.Serialization.XmlElementAttribute(Namespace="urn:ebay:api:PayPalAPI")] SetExpressCheckoutReq SetExpressCheckoutReq) {
Line 1792: object[] results = this.Invoke("SetExpressCheckout", new object[] {
Line 1793: SetExpressCheckoutReq});
Line 1794: return ((SetExpressCheckoutResponseType)(results[0]));
Source File: c:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files\anan_p2\730602d6\31a8d74e\App_WebReferences.c8vgyrf8.2.cs Line: 1792
I've also tried generating certificates using OpenSSL and uploading them to the Paypal account's encrypted seller option but still no effect.
Thank you very much for reading through my question!
Update: As requested here is the code being used.
String hostingOn = ConfigurationManager.AppSettings["default_site_url"];
reqDetails.ReturnURL = hostingOn + "marketplace_confirm.aspx";
reqDetails.CancelURL = hostingOn + "marketplace.aspx";
reqDetails.NoShipping = "1";
reqDetails.ReqConfirmShipping = "0";
reqDetails.OrderTotal = new BasicAmountType()
{
currencyID = CurrencyCodeType.CAD,
Value = payment_amt.Value,
};
SetExpressCheckoutReq req = new SetExpressCheckoutReq()
{
SetExpressCheckoutRequest = new SetExpressCheckoutRequestType()
{
Version = UtilPayPalAPI.Version,
SetExpressCheckoutRequestDetails = reqDetails
}
};
PayPalAPIAASoapBinding paypal = new PayPalAPIAASoapBinding();
paypal.SetExpressCheckout(req);
I am also using the https://api-aa-3t.paypal.com/2.0/ url for accessing the API

Since early 2016, Paypal started requiring TLS 1.2 protocol for communications in the Sandbox, and will enforce it for the live environment starting June 17. See here for reference.
In most .NET applications TLS 1.2 will come disabled by default, and therefore you'll need to enable it.
You need to add the following line, for example, at the beginning of you Application_Start method:
public class Site : HttpApplication
{
protected void Application_Start()
{
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
// other configuration
}
}

You're probably connecting to api.paypal.com or api.sandbox.paypal.com, and not sending along your API certificate. The API certificate is a client SSL certificate used to complete the SSL chain.
If you don't have or are not using an API certificate, you should connect to api-3t.paypal.com or api-3t.sandbox.paypal.com for Live or Sandbox respectively.

I've been working with a PayPal (NVP/Signature) Express Checkout integration and have been hit with this SSL/TLS error.
Nothing I did seemed to get around it but then I found the following code to add above my request. For reference, I'm using MVC3/.NET 4 so Tls1.2 isn't available to me by default (like in .NET 4.5 +). This first three lines of this code gets around that. I hope it helps people!
ServicePointManager.Expect100Continue = true;
ServicePointManager.SecurityProtocol = (SecurityProtocolType)3072;
ServicePointManager.DefaultConnectionLimit = 9999;
var url = "https://[paypal-api-url]/nvp";
var uri = new Uri(url);
var request = WebRequest.Create(uri);
var encoding = new UTF8Encoding();
var requestData = encoding.GetBytes(data);
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.Timeout = (300 * 1000);
request.ContentLength = requestData.Length;
using (var stream = request.GetRequestStream())
{
stream.Write(requestData, 0, requestData.Length);
}
var response = request.GetResponse();
...

Thanks a lot that really helps me.
For reference here is my code for establishing the interface in VB.NET
'Create a service Binding in code
Dim ppEndpointAddress As New System.ServiceModel.EndpointAddress("https://api-3t.sandbox.paypal.com/2.0/")
Dim ppBinding As New System.ServiceModel.BasicHttpBinding(System.ServiceModel.BasicHttpSecurityMode.Transport)
Dim ppIface As New PayPalAPI.PayPalAPIAAInterfaceClient(ppBinding, ppEndpointAddress)
Dim ppPaymentReq As New PayPalAPI.DoDirectPaymentReq()
ppPaymentReq.DoDirectPaymentRequest = ppRequest

Creating tweet with Twitterizer raises WebException 401

I am using Twitterizer, and am trying to get my ASP.Net app to upload reported traffic incidents to the official Twitter account.
I have looked at similar questions at SO, and the I tried all the recommendations (specify call-back url, check for careless errors and ensure the app has Read-Write permissions), but I still cannot solve the problem. I have some screenshots of the settings and code below. The callback URL does not exist, but is made up. Any help will be greatly appreciated.

Another item to look for is the computer time. Look at the server time in the Twitter response and compare it to the computer you're using. Also, here's a troubleshooting guide from the Twitter site:
https://dev.twitter.com/discussions/204

I came back to this problem, this time I tried regenerating my API keys, and now it is working. Perhaps my previous keys were duds.

public static void UploadTweet(string token, string tokensecreat, byte[] img, string Title)
{
Twitterizer.OAuthTokens tokens = new Twitterizer.OAuthTokens();
tokens.AccessToken = token;
tokens.AccessTokenSecret = tokensecreat;
tokens.ConsumerKey = TWITTER_CONSUMER_KEY;
tokens.ConsumerSecret = TWITTER_CONSUMER_SECRET;
byte[] photo = img;
TwitterResponse<TwitterStatus> response = TwitterStatus.UpdateWithMedia(tokens, Title, photo, new StatusUpdateOptions() { UseSSL = true, APIBaseAddress = "http://api.twitter.com/1.1/" });
if (response.Result == RequestResult.Success)
{
}
else
{
}
}

How do I use the Redmine REST API over https from .NET?

Our internal Redmine server only allows me to connect via HTTPS. Here's how I tried to use the REST API via HTTPS from .NET:
As suggested in Using the REST API with .NET, setting the host variable to "https://redmine.company.com/redmine/" and the apiKey to "ffffffffffffffffffffffffffffffffffffffff".
From scratch with the following code:
using System.IO;
using System.Net;
class Program
{
static void Main(string[] args)
{
ServicePointManager.ServerCertificateValidationCallback += (sender, cert, chain, error) => true;
var request = (HttpWebRequest)WebRequest.Create(
"https://redmine.company.com/redmine/issues/149.xml?key=ffffffffffffffffffffffffffffffffffffffff");
request.CookieContainer = new CookieContainer();
request.Method = "GET";
using (var response = request.GetResponse()) // Hangs here
using (var responseStream = response.GetResponseStream())
using (var memoryStream = new MemoryStream())
{
responseStream.CopyTo(memoryStream);
}
}
}
Of course, company.com and ffffffffffffffffffffffffffffffffffffffff are just placeholders for my real company and my real API key on my account page. Both attempts hang for some time before timing out with a WebException (see the Hangs here comment in attempt 2). I then tried to download other stuff from the Redmine server (like e.g. time_entries.csv, atom feeds, etc.), each time with exactly the same result.
So far so bad. However, if I copy-paste the URL https://redmine.company.com/redmine/issues/149.xml?key=ffffffffffffffffffffffffffffffffffffffff into my browser, I get exactly the response I would expect. So, it seems as though our Redmine server behaves as it should, but somehow I can't get it to work from .NET.
I have successfully downloaded stuff from other HTTPS sites and have managed to download issue data from http://demo.redmine.org with the code of attempt 2 (of course with adapted URLs, etc.). So, it seems there might be something special about how Redmine communicates over HTTPS.
If anybody is successfully using the Redmine REST API over HTTPS from .NET, I'd be really grateful for some pointers on what I'm doing wrong.
Also, suggestions on how to debug this from the client side would be greatly appreciated. So far I've tried Fiddler2, with no success. As soon as I enable its "Decrypt HTTPS traffic" setting then I no longer get an answer when I make the request in Internet Explorer.

We use redmine-net-api which supports HTTP/S connection and authentication based on API keys.
RedmineManager rm = new RedmineManager("https://&ltyour-address&gt", &ltapi-key&gt, "random-password");
IList&ltIssue&gt issues = rm.GetObjectList&ltIssue&gt(new NameValueCollection() { { "project_id", &ltproject-id&gt } });

Try this, it works for me:
// Allow every secure connection
ServicePointManager.ServerCertificateValidationCallback += (sender, cert, chain, error) => true;
// Create redmine manager (where URL is "https://10.27.10.10/redmine" for me and redmineKey is my redmine API key
RedmineManager redmineManager = new RedmineManager(redmineURL, redmineKey);
// Create your query parameters
NameValueCollection queryParameters = new NameValueCollection { { "project_id", "4" }, {"tracker_id", "17"}, { "offset", "0" } };
// Perform your query
int issuesFound = 0;
foreach (var issue in redmineManager.GetObjectList<Issue>(queryParameters, out issuesFound))
{
// By default you get the 25 first issues of the project_id and tracker_id specified.
// Play with the offset to get the rest
queryParameters["offset"] = ....
}

Explicit passing SecurityProtocolType.Tls12 value for securityProtocolType parameter solved the problem for my case:
RedmineManager redmineManager = new RedmineManager(_host, _apiKey,
securityProtocolType: SecurityProtocolType.Tls12);