Using C#, I am trying execute a query on Google Spanner db. I understand I can use the SpannerClient and all the current documentation explains how to execute a query quite simply, however these examples all assume default environment credentials.
I would like to execute a query against a given database but using custom credentials. So something like
var credentials = GoogleCredential.FromJson(jsonData);
SpannerClient client = new SpannerClient(connectionString, credentials)
var cmd = client.CreateSelectCommand("SELECT SingerId, AlbumId, AlbumTitle FROM Albums");
etc
I am currently unable to figure out how to do this?
Thanks
Currently this isn't as clean as we'd like it to be. You need to create a ChannelCredentials for the credentials, and provide that to the SpannerConnectionStringBuilder:
// First load the credentials, scope them, and convert to ChannelCredentials.
// You may want to move this to a separate method.
var googleCredential = GoogleCredential.FromJson(jsonData);
googleCredential = googleCredential.CreateScoped(SpannerClient.DefaultScopes);
// Use self-signed JWTs for service accounts.
// (This isn't strictly required, but reduces network usage.)
if (googleCredential.UnderlyingCredential is ServiceAccountCredential serviceCredential)
{
googleCredential = GoogleCredential.FromServiceAccountCredential(
serviceCredential.WithUseJwtAccessWithScopes(true));
}
// Note: this requires a using directive of "using Grpc.Auth;"
var channelCredentials = googleCredential.ToChannelCredentials();
// Now create a SpannerConnection with the SpannerCredentials
using var conn = new SpannerConnection(connectionString, credentials);
using var cmd = conn.CreateSelectCommand("SELECT ...");
...
We definitely hope to improve this - we have a tracking bug you might want to subscribe to so that you can simplify your code when it's fixed.
Related
I am using Google.Cloud.Vision.V1, Version=2.0.0.0 and the following below code from Google Vision API specify JSON file
using Google.Apis.Auth.OAuth2;
using Google.Cloud.Vision.V1;
using Grpc.Auth;
using Grpc.Core;
var credential = GoogleCredential.FromFile("VisionProject.json");
var channel = new Grpc.Core.Channel(ImageAnnotatorClient.DefaultEndpoint.ToString(), credential.ToChannelCredentials());
var client = ImageAnnotatorClient.Create(channel);
But its shows me this error No overload for method 'ImageAnnotatorClient.Create' takes 1 arguments.
I have found similar code in documentation https://googleapis.github.io/google-cloud-dotnet/docs/Google.Cloud.Vision.V1P2Beta1/api/Google.Cloud.Vision.V1P2Beta1.ImageAnnotatorClient.html
But for some reason, it's not working( unable to see the overload)
It seems that you are using newer version of API. Docs state that now authentication is set up(when needed) via environment variable:
Otherwise, the simplest way of authenticating your API calls is to download a service account JSON file then set the GOOGLE_APPLICATION_CREDENTIALS environment variable to refer to it. The credentials will automatically be used to authenticate. See the Getting Started With Authentication guide for more details.
So you can do something like this:
Environment.SetEnvironmentVariable("GOOGLE_APPLICATION_CREDENTIALS", "PathTo_VisionProject.json");
var client = ImageAnnotatorClient.Create();
Or set this environment variable some other way.
While setting the environment variable is certainly a simple way of specifying which service account file to use, it's not the only one. You can use the builder to specify the path very easily:
var client = new ImageAnnotatorClientBuilder
{
CredentialsPath = "VisionProject.json"
}.Build();
Another solution if you do not have access to the JSON file directly and you want to stay close to what you previously did with the GoogleCredential and Channel creation is something like:
var credential = GoogleCredential.FromFile("VisionProject.json");
// or if you have access to the content only
// var credential = GoogleCredential.FromJson(json);
var client = await new ImageAnnotatorClientBuilder
{
Endpoint = ImageAnnotatorClient.DefaultEndpoint,
ChannelCredentials = credential.ToChannelCredentials()
}.BuildAsync();
I have a nice Azure Active Directory set up with a dozen users. (All me!) So I have a Tenant ID, client ID and Client Secret.
I am also working on a simple console application that will function as a public client for this directory. This client also holds a list of usernames and passwords as this is just meant as a simple experiment. Not secure, I know. But I first need to understand how it works...
I do this:
IConfidentialClientApplication client = ConfidentialClientApplicationBuilder
.CreateWithApplicationOptions(options).Build();
And this creates my client app. Works fine.
I also get a token using "https://graph.microsoft.com/.default" and can use this to get all users as JSON:
string result = await GetHttpContentWithToken("https://graph.microsoft.com/v1.0/users",
token.AccessToken);
Although I might want it to be more user-friendly, JSON is fine for now.
How can I check if user is an authorized user?
And no, I don't want complex solutions that require various nuget packages. Just a plain and simple step-by-step explanation. I could probably Google this but I ended up with thousands of results and none were helpful... This should be easy, right?
[EDIT] I first wanted to get a list of users nut that failed because of a typo... (There's a dot before 'default'...)
It took some fooling around but it's not too difficult after all. There are a lot of libraries around Azure but it is all basically just a bunch of HTTP requests and responses. Even in a console application...
I started with making a PublicClientApplicationBuilder first:
var options = new PublicClientApplicationOptions()
{
ClientId = <**clientid**>,
TenantId = <**tenantid**>,
AzureCloudInstance = AzureCloudInstance.AzurePublic,
};
var client = PublicClientApplicationBuilder.CreateWithApplicationOptions(options).Build();
I can also create a ConfidentialClientApplication instead, but this allows me to log in interactively, if need be.
Next, set up the scopes:
var scopes = new List<string>() { "https://graph.microsoft.com/.default" };
As I wanted to log in using username and password, I have to use this:
var token = await client.AcquireTokenInteractive(scopes).ExecuteAsync();
But if I want to log in using code, I can also use this:
var password = new SecureString();
foreach (var c in <**password**>) { password.AppendChar(c); }
var token = await client.AcquireTokenByUsernamePassword(scopes, <**account**>, password).ExecuteAsync();
At this point, I'm authorized as the specified user. So, now all I need is to get whatever data I like, in JSON strings...
public static async Task<string> ExecCmd(string name, string url, string token)
{
HttpClient httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("bearer", token);
string result = await GetHttpContentWithToken(url, token);
JObject json = JsonConvert.DeserializeObject(result) as JObject;
File.WriteAllText(name, json.ToString());
return result;
}
As I just want to read the data as text files, I just execute the action in using a specific and write it as formatted JSON to the file . So, using this simple method I can now use this:
await ExecCmd("Profile.txt", "https://graph.microsoft.com/v1.0/me/", token.AccessToken);
await ExecCmd("Groups.txt", "https://graph.microsoft.com/v1.0/groups", token.AccessToken);
await ExecCmd("Users.txt", "https://graph.microsoft.com/v1.0/users", token.AccessToken);
These will provide me with (1) the profile of the current user, (2) the AD groups and (3) the AD users. And probably a bit more...
I can use this ExecCmd to retrieve a lot more data, if I want to. But there's something else to keep in mind! For it all to work, you also need to configure the Azure application and make sure all access rights are assigned and approved!
So, in Azure AD you have to add an "App registration" and fiddle around with the settings... (The Azure experts are horribly shocked now, but when you want to learn, you'd just have to try and fail until you succeed...)
Also set "Default client type" to "public client" for the registered app.
In Azure, with the registered app, you also need to set the proper API permissions! Otherwise, you won't have access. And as I want access to Active Directory, I need to add permissions to "Azure Active Directory Graph". I can do this inside Azure or by using the scope when I call AcquireTokenInteractive(). For example, by using "https://graph.windows.net/Directory.Read.All" instead of "https://graph.windows.net/.default".
Once you've accessed a token interactively, you can also get more tokens using client.AcquireTokenSilent(). It gets a bit tricky from here, especially if you want to access a lot of different items. Fortunately, Active Directory is mostly the directory itself, groups, users and members.
Personally, I prefer to grant access from the Azure website but this is quite interesting.
Anyways, I wanted to authenticate users with Azure and now I know how to do this. It still leaves a lot more questions but this all basically answers my question...
I'll use this as answer, as others might find it useful...
I'd like to use a GoogleCredential object (or similar) in order to create a Stackdriver logging client object (an instance of LoggingServiceV2Client class) using some custom credentials rather than the default application credentials.
I cannot see an appropriate overload of the LoggingServiceV2Client.Create method but the docstring for that method states:
Synchronously creates a
Google.Cloud.Logging.V2.LoggingServiceV2Client, applying defaults for
all unspecified settings, and creating a channel connecting to the
given endpoint with application default credentials where necessary.
See the example for how to use custom credentials.
which suggests it's possible somehow?
I have been unable to find a custom credentials example in the documentation anywhere. The only examples I see (eg this) read only the default application credentials from the GOOGLE_APPLICATION_CREDENTIALS environment variable which I'd prefer to avoid
It's possible, but far from obvious.
Add these two using statements to the top of your .cs:
using Google.Apis.Auth.OAuth2;
using Grpc.Auth;
Then instantiate the client like this:
var credential = GoogleCredential.FromFile(jsonPath)
.CreateScoped(LoggingServiceV2Client.DefaultScopes);
var channel = new Grpc.Core.Channel(
LoggingServiceV2Client.DefaultEndpoint.ToString(),
credential.ToChannelCredentials());
var client = LoggingServiceV2Client.Create(channel);
Other solutions didn't work for me using Google.Cloud.Logging.V2 - Version: 3.4.0, because of this line:
var client = LoggingServiceV2Client.Create(channel);
In version 3.4.0 there is no constructor that takes a channel as a parameter.
So I checked google documentation : LoggingServiceV2Client Create(), and it has this small note:
To specify custom credentials or other settings, use LoggingServiceV2ClientBuilder
So here is my working code using this approach:
var credential = GoogleCredential.FromFile(jsonPath).CreateScoped(LoggingServiceV2Client.DefaultScopes);
var client = new LoggingServiceV2ClientBuilder { ChannelCredentials = credential.ToChannelCredentials() }.Build();
I already apreciated to #Jeffrey Rennie. In my case, I am using Cloud Text-to-Speech and I had to use following code:
Usings:
using Google.Apis.Auth.OAuth2;
using Google.Cloud.TextToSpeech.V1;
using Grpc.Auth;
Code:
// Setting up credentials
string jsonPath = #"D:\my-test-project-0078ca7c0f8c.json";
var credential = GoogleCredential.FromFile(jsonPath).CreateScoped(TextToSpeechClient.DefaultScopes);
var channel = new Grpc.Core.Channel(TextToSpeechClient.DefaultEndpoint.ToString(), credential.ToChannelCredentials());
// Instantiate a client
TextToSpeechClient client = TextToSpeechClient.Create(channel);
// Perform the Text-to-Speech request, passing the text input with the selected voice parameters and audio file type
var response = client.SynthesizeSpeech(new SynthesizeSpeechRequest
{
Input = new SynthesisInput() { Text = "My test sentence" },
Voice = new VoiceSelectionParams() { LanguageCode = "en-US", SsmlGender = SsmlVoiceGender.Male },
AudioConfig = new AudioConfig { AudioEncoding = AudioEncoding.Mp3 };
});
Installed NuGet Packages:
Google.Cloud.TextToSpeech.V1 -Pre
Google.Apis.Auth
I have a Console Application project written in C# which I've added Application Insights to with the following NuGet packages.
Microsoft.ApplicationInsights
Microsoft.ApplicationInsights.Agent.Intercept
Microsoft.ApplicationInsights.DependencyCollector
Microsoft.ApplicationInsights.NLogTarget
Microsoft.ApplicationInsights.PerfCounterCollector
Microsoft.ApplicationInsights.Web
Microsoft.ApplicationInsights.WindowsServer
Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel
I've configured my InstrumentationKey in the config file and I'm firing up a TelemetryClient on startup using the with the following code:
var telemetryClient = new TelemetryClient();
telemetryClient.Context.User.Id = Environment.UserName;
telemetryClient.Context.Session.Id = Guid.NewGuid().ToString();
telemetryClient.Context.Device.OperatingSystem = Environment.OSVersion.ToString();
Everything is working well except AI is not capturing any requests that get sent to Mongo, I can see requests going off to SQL server in the 'Application map' but no sign of any other external requests. Is there any way that I can see telemetry of requests made to Mongo?
EDIT - Thanks to Peter Bons I ended up with pretty much the following which works like a charm and allows me to distinguish between success and failure:
var telemetryClient = new TelemetryClient();
var connectionString = connectionStringSettings.ConnectionString;
var mongoUrl = new MongoUrl(connectionString);
var mongoClientSettings = MongoClientSettings.FromUrl(mongoUrl);
mongoClientSettings.ClusterConfigurator = clusterConfigurator =>
{
clusterConfigurator.Subscribe<CommandSucceededEvent>(e =>
{
telemetryClient.TrackDependency("MongoDB", e.CommandName, DateTime.Now.Subtract(e.Duration), e.Duration, true);
});
clusterConfigurator.Subscribe<CommandFailedEvent>(e =>
{
telemetryClient.TrackDependency("MongoDB", $"{e.CommandName} - {e.ToString()}", DateTime.Now.Subtract(e.Duration), e.Duration, false);
});
};
var mongoClient = new MongoClient(mongoClientSettings);
I am not familiar with MongoDB but as far as I can tell there is no default support for it when it comes to Application Insights. But that does not mean you cannot do this, it will just involve some more code.
Again, I am not familiar with MongoDB but according to http://www.mattburkedev.com/logging-queries-from-mongodb-c-number-driver/ there is built-in support for logging the generated queries. Now, we only need to hook this up to Application Insights.
Since you already know how to use the TelemetryClient we can use the custom tracking methods provided by that class. See https://learn.microsoft.com/nl-nl/azure/application-insights/app-insights-api-custom-events-metrics for the available custom tracking methods.
All you need to do is to insert some code like this:
telemetryClient.TrackDependency(
"MongoDB", // The name of the dependency
query, // Text of the query
DateTime.Now, // Time that query is executed
TimeSpan.FromSeconds(0), // Time taken to execute query
true); // Indicates success
The class telemetryClient is thread-safe so you can reuse it.
Now, according to the referenced blogpost you should be able to do something like this:
var client = new MongoClient(new MongoClientSettings()
{
Server = new MongoServerAddress("localhost"),
ClusterConfigurator = cb =>
{
cb.Subscribe<CommandStartedEvent>(e =>
{
telemetryClient.TrackDependency(
"MongoDB", // The name of the dependency
e.Command.ToJson() // Text of the query
DateTime.Now, // Time that query is executed
TimeSpan.FromSeconds(0), // Time taken to execute query
true); // Indicates success
});
}
});
Again, I am not familiar with MongoDB but I hope this is a starting point for your imagination on how to adapt it to your needs using your knowledge of MongoDB.
EDIT:
If there is also a CommandCompletedEvent or similar event as opposed to the CommandStartedEvent event you should probably track the dependency there because you should then be able to calculate (or simpel read) the time spent and maybe get the actual value for the success indicator.
The short question is whether is this possible and if so, how?
Outline
I have a .NET application which currently uses a service account to access information across a Google Apps domain using the Google Drive API. This works fine using the google-api-dotnet-client library and code along the same lines as shown in the samples here - which are currently a very good basic example of what I'm doing.
What I want to do now is extend it so as well as using those APIs provided by the "new" google-api-dotnet-client library, it uses the older "GData" libraries, as provided for via the
older google-gdata library, specifically the Spreadsheets API (and perhaps more to come).
The Problem
This is where the difficulty arises. The former library does exactly what I want, as evidenced by the second link in the first paragraph above - and the fact I have it doing it myself. HOWEVER... although the second library has been updated to support OAuth 2.0 in addition to OAuth 1.0 and the other older auth techniques, it does not - as far as I can tell from extensive Googling and trail-and-error - allow the "service account on behalf of all my users" operation which I need.
My question is whether I'm missing something (possibly a hard to find or undocumented something) which would allow me to do what I want. Failing that, is there any way I could force this behaviour and make these two libraries operate side by side?
The ideal solution
Ideally I would love some way of having the Google.GData.Spreadsheets.SpreadsheetsService instance be able to take advantage of the Google.Apis.Authentication.Auth2Authenticator<AssertionFlowClient> instance I'm already using... somehow. Is such witchcraft possible? I'm I missing the obvious?
Failing that, I'm happy to do the whole OAuth2 "assertion flow client" dance again if I have to, in some way that the older library can handle.
Help?
Other Thoughts
I have considered - and rejected for the time being - the option of starting from scratch and writing my own library to make this happen. This is for two reasons:
The gdata library already exists, and has been developed by many people likely cleverer than myself. I'm not so arrogant that I believe I can do better.
I'm not certain the OAuth2 with service account approach is even supported/allowed on these older APIs.
An alternate approach which I've been hoping to avoid but may have to fall back to depending on the answers here will be to use 2-legged OAuth 1.0 for portions of this. I'd prefer not to, as having parts of the app rely on one old auth method whilst other parts do it the nice new way just feels wrong to me. And there's that much more to go wrong...
Updates
I have considered the possibility of subclassing GDataRequestFactory and GDataRequest so I can make my own request factory and have that take the instance of Google.Apis.Authentication.Auth2Authenticator<AssertionFlowClient> (well, an instance of Google.Apis.Authentication.IAuthenticator anyway) which could step in to authenticate the request just before it's called. However... the constructor for GDataRequest is internal, which has stopped me.
It's really looking like this isn't meant to be.
For the sake of other folks coming across this question (now that the solution linked to in the accepted answer uses deprecated code), here's how I solved it:
First, start in "new API" land (use the Google.Apis.Auth nuget package) by setting up a ServiceAccountCredential following Google's Service Account example:
//In the old api, this accessed the main api accounts' sheets, not anymore
//** Important ** share spreadsheets with the Service Account by inviting the "serviceAccountEmail" address to the sheet
string serviceAccountEmail = "12345697-abcdefghijklmnop#developer.gserviceaccount.com";
var certificate = new X509Certificate2(#"key.p12", "notasecret", X509KeyStorageFlags.Exportable);
ServiceAccountCredential credential = new ServiceAccountCredential(
new ServiceAccountCredential.Initializer(serviceAccountEmail)
{
Scopes = new[] { "https://spreadsheets.google.com/feeds", "https://docs.google.com/feeds" }
}.FromCertificate(certificate));
Tell the credential to request an Access Token:
credential.RequestAccessTokenAsync(System.Threading.CancellationToken.None).Wait();
Now it's time to switch back to "old API" land (use the Google.GData.Spreadsheets nuget package). Start by constructing the SpreadsheetsService (similar to Google's example):
SpreadsheetsService service = new SpreadsheetsService("MySpreadsheetIntegration-v1");
To use Service Account authentication, we'll create an instance of the GDataRequestFactory and set a custom Authorization header:
var requestFactory = new GDataRequestFactory("My App User Agent");
requestFactory.CustomHeaders.Add(string.Format("Authorization: Bearer {0}", credential.Token.AccessToken));
Finally, set the SpreadsheetsService's RequestFactory property to this new factory:
service.RequestFactory = requestFactory;
And go ahead and use the SpreadsheetsService as you would had you authenticated using any other technique. (Tip: share spreadsheets with the Service Account by inviting the serviceAccountEmail address to the sheet)
I managed to solve this by subclassing GDataRequestFactory and creating my own implementation of the interfaces implemented by GDataRequest. This implementation wraps an instance of GDataRequest instantiated via reflection, and adds in the necessary code to perform authentication using an instance of IAuthenticator (in my case, Auth2Authenticator).
I wrote a blog post on it and added an example as a Gist:
Blog: Using Google's Spreadsheet API using .NET, OAuth 2.0 and a Service Account
Gist 4244834
Feel free to use this if it helps you (BSD licence).
Hey just stumbled accross the same problem and produced a different solution:
Has anybody ever concidered of writing the parameters from the credentials-object directly to an OAuth2Parameters-Object?
I did this and it worked nicely:
public class OAuthTest
{
OAuth2Parameters param = new OAuth2Parameters();
public OAuthTest()
{
Debug.WriteLine("Calling: AuthGoogleDataInterface()");
bool init = AuthGoogleDataInterface();
if (init)
{
GOAuth2RequestFactory requestFactory = new GOAuth2RequestFactory(null, "My App User Agent", this.param);
//requestFactory.CustomHeaders.Add(string.Format("Authorization: Bearer {0}", credential.Token.AccessToken));
var service = new SpreadsheetsService("MyService");
service.RequestFactory = requestFactory;
SpreadsheetQuery query = new SpreadsheetQuery();
// Make a request to the API and get all spreadsheets.
SpreadsheetFeed feed = service.Query(query);
// Iterate through all of the spreadsheets returned
foreach (SpreadsheetEntry entry in feed.Entries)
{
// Print the title of this spreadsheet to the screen
Debug.WriteLine(entry.Title.Text);
}
}
Debug.WriteLine(m_Init);
}
private bool AuthGoogleDataInterface()
{
bool b_success;
try
{
Console.WriteLine("New User Credential");
// New User Credential
UserCredential credential;
using (var stream = new FileStream("client_secrets.json", FileMode.Open, FileAccess.Read))
{
GoogleClientSecrets GCSecrets = GoogleClientSecrets.Load(stream);
string[] ArrScope = new[] { "https://spreadsheets.google.com/feeds", "https://docs.google.com/feeds" };
credential = GoogleWebAuthorizationBroker.AuthorizeAsync(
GCSecrets.Secrets,
ArrScope,
"user", CancellationToken.None,
new FileDataStore("My.cal")).Result;
// put the Information generated for the credentials object into the OAuth2Parameters-Object to access the Spreadsheets
this.param.ClientId = GCSecrets.Secrets.ClientId; //CLIENT_ID;
this.param.ClientSecret = GCSecrets.Secrets.ClientSecret; //CLIENT_SECRET;
this.param.RedirectUri = "urn:ietf:wg:oauth:2.0:oob"; //REDIRECT_URI;
this.param.Scope = ArrScope.ToString();
this.param.AccessToken = credential.Token.AccessToken;
this.param.RefreshToken = credential.Token.RefreshToken;
}
Debug.WriteLine("AuthGoogleDataInterface: Success");
b_success = true;
}
catch (Exception e)
{
Debug.WriteLine(e.ToString());
b_success = false;
}
return b_success;
}
}