Specifying Google Credentials When Creating a SubscriberClient using C# Client Library - c#

I've setup a GCP PubSub processor for our service and it creates a SubscriberClient like so
var subscriptionClient = await SubscriberClient.CreateAsync(subscriptionName);.
And I have the GOOGLE_APPLICATION_CREDENTIALS environment variable set and pointing at a valid SA key. Everything works as expected.
However, how can I go about not using/relying on the GOOGLE_APPLICATION_CREDENTIALS environment variable on my local machine?
The Cloud Storage Client libraries allow you to create a storage client like so StorageClient.Create(GoogleCredentials gcpCredentials); and I was looking for something like this with the PubSub client libraries but did not find anything. There is ChannelCredentials but that does not appear to be for this purpose.
I do see that SubscriberServiceApiClientBuilder allows you to specify JsonCredentials but I'm not using that client for my use case. As the SubscriberClient and PublisherClient are more suitable for my purpose given the following from the documentation:
PublisherClient and SubscriberClient provide simpler APIs for message publishing and subscribing. These classes offer considerably higher performance and simplicity, especially when working with higher message throughput.
Thanks

Creating the ChannelCredentials manually, in a similar fashion as done in the PublisherClient and passing in ClientCreationSettings initialized with credentials set using GoogleCredentials.ToChannelCredentials() does the job.
var subscriptionName = SubscriptionName.FromProjectSubscription("projectId", "subscriptionId");
// create GoogleCredentials
var gcpCredentials = <code that creates valid GoogleCredentials>;
// create ClientCreationSettings with credentials based on the GoogleCredentials created above
var settings = new SubscriberClient.ClientCreationSettings(credentials: gcpCredentials.ToChannelCredentials());
var client = await SubscriberClient.CreateAsync(<SubscriptionName>, settings);
I asked this question in the GitHub googleapis/google-api-dotnet-client repo as well. if you want a bit more information about it: GitHub Issue 1764 Link

Related

How can I setup proxy settings in c# for Google Cloud Txt-To-Speech API

How to setup a proxy in c# code for Google Text-To-Speech API.
Does somebody knows where to put in the proxy settings for the Google TTS API in c#. Our project runs locally but not on the server behind a firewall, so it has to go via the proxy.
Hope you have a starting point for me ;-)
Thanks!
The intention is that if you've set the system proxy, or the HTTPS_PROXY environment variable, that should just work by default.
However, at the moment there's a bug in Grpc.Net.Client that causes that to fail. (Once it's been fixed and released, we'll obviously update our dependencies for the next release of Google.Cloud.TextToSpeech.V1 and other Cloud libraries.)
The simplest workaround is probably to still use the Grpc.Net.Client implementation of gRPC, but with an additional option:
var client = new TextToSpeechClientBuilder
{
GrpcAdapter = GrpcNetClientAdapter.Default.WithAdditionalOptions(
options => options.HttpHandler = new HttpClientHandler
{
Proxy = new WebProxy("your proxy here"),
UseProxy = true
})
}.Build();

Azure resourceManager

I'm trying to scale programmatically an Azure worker role and I can't find the right syntax for the resource name.
// Create the autoscale client.
AutoscaleClient autoscaleClient = new AutoscaleClient(new CertificateCloudCredentials(subscriptionId, cert));
AutoscaleSettingGetResponse get =autoscaleClient.Settings.Get(AutoscaleResourceIdBuilder.BuildCloudServiceResourceId("WRK-GROUP-NE01","CIW.GROUPS",true) );
AutoscaleSetting setting = get.Setting;
It returns : ResourceNotFound: The resource service name monitoring is not supported.
Can you point me to the right syntax?
The guide which you're following is based on the Azure Service Management model, aka Classic mode, which is deprecated and only exists mainly for backward compatibility support.
You should use the latest Microsoft.Azure.Insights nuget package for getting the autoscale settings.
Sample code using the nuget above is as below:
using Microsoft.Azure.Management.Insights;
using Microsoft.Rest;
//... Get necessary values for the required parameters
var client = new InsightsManagementClient(new TokenCredentials(token));
client.AutoscaleSettings.Get(resourceGroupName, autoScaleSettingName);
See the REST API Reference below for getting the autoscale settings.
GET
https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/microsoft.insights/autoscaleSettings/{autoscale-setting-name}?api-version={api-version}
For updating Autosale setting you can use below api
PUT https://management.azure.com/subscriptions/{subscriptionId}/resourcegroups/{resourceGroupName}/providers/microsoft.insights/autoscalesettings/{autoscaleSettingName}?api-version=2015-04-01
https://learn.microsoft.com/en-us/rest/api/monitor/autoscalesettings/createorupdate
Hope it helps.

Programmatically create service bus subscription using .net standard

My scenario: Website hosted on the cloud, where each instance creates a subscription to a Service Bus Topic for itself to listen for messages.
My question: How do I programmatically create subscriptions?
Microsoft.Azure.ServiceBus.3.1.0 allows to create a ManagementClient using the ConnectionString.
private async Task CreateTopicSubscriptions()
{
var client = new ManagementClient(ServiceBusConnectionString);
for (int i = 0; i < Subscriptions.Length; i++)
{
if (!await client.SubscriptionExistsAsync(TopicName, Subscriptions[i]))
{
await client.CreateSubscriptionAsync(new SubscriptionDescription(TopicName, Subscriptions[i]));
}
}
}
Original plan for the new Azure Service Bus client was not to include management plane at all and use Azure Active Directory route instead. This has proven to be too problematic, just like you've pointed out. Microsoft messaging team has put together a sample to demonstrate the basic operations.
Note that there's a pending PR to get it working with .NET Core 2.0
Moving forward, it was recognized that developers prefer to access Service Bass using a connection string like they used to over Azure Active Directory option. Management Operations issue is raised to track requests. Current plan is to provide a light weight management library for the .NET Standard client.
For now, the options are either to leverage the old client to create entities or use Microsoft.Azure.Management.ServiceBus (or Fluent) until the management package is available.
Update
Management operations were released as part of 3.1.0 version of the client.
Microsoft.Azure.ServiceBus has been deprecated. The new option is Azure.Messaging.ServiceBus and ManagementClient has been replaced by ServiceBusAdministrationClient.
string connectionString = "<connection_string>";
ServiceBusAdministrationClient client = new ServiceBusAdministrationClient(connectionString);
This new package also supports ManagedIdentity:
string fullyQualifiedNamespace = "yournamespace.servicebus.windows.net";
ServiceBusAdministrationClient client = new ServiceBusAdministrationClient(fullyQualifiedNamespace, new DefaultAzureCredential());
A little example:
var queueExists = await _administrationClient.QueueExistsAsync(queueName);
if(!queueExists)
await _administrationClient.CreateQueueAsync(queueName);
More info here.

How to specify AWS credentials in C# .NET core console program

I am trying to test a .NET core console program to publish a message to SNS. As I had issues trying to get it to work in Lambda, I want to try it in a non-Lambda environment. In Lambda, security is covered by the role, but in a console program, I presume that I have to specify my access key and secret somehow.
I've read this page: http://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/net-dg-config-creds.html#net-dg-config-creds-sdk-store, but still totally confused.
I'm running on my local development computer, not an EC2 instance. No intent to go to production with this, just trying to test some code.
I'm on Visual Studio 2015, .NET Core 1.0. I've used Nuget to get the following:
"AWSSDK.Extensions.NETCore.Setup": "3.3.3",
"AWSSDK.SimpleNotificationService": "3.3.0.23",
Based on the answer to How to set credentials on AWS SDK on NET Core? I created the /user/.aws/credentials file (assuming credentials was the file name and not the directory name).
But that question/answer doesn't address how to actually use this file. The code I'm running is below.
public static void Main(string[] args)
{
Console.WriteLine("Started");
//var awsCredentials = new Amazon.Runtime.AWSCredentials()
var client = new Amazon.SimpleNotificationService.AmazonSimpleNotificationServiceClient(Amazon.RegionEndpoint.EUWest2);
//var client = new Amazon.SimpleNotificationService.AmazonSimpleNotificationServiceClient(awsCredentials, Amazon.RegionEndpoint.EUWest2);
//Amazon.SimpleNotificationService.Model.PublishResponse publishResp = null;
SendMessage(client).Wait();
Console.WriteLine("Completed call to SendMessage: Press enter to end:");
Console.ReadLine();
}
The error I'm getting on the new client is:
An unhandled exception of type 'Amazon.Runtime.AmazonServiceException' occurred in AWSSDK.Core.dll
Additional information: Unable to find credentials
I see there is a way to pass an AWSCredentials object to that constructor, but I don't understand how to build it. Amazon.Runtime.AWSCredentials is an abstract class, so I can't use it in a "new" statement.
Based on Dan Pantry's answer, here is a simple short answer with code highlighted (note the region enum in the second line):
var awsCredentials = new Amazon.Runtime.BasicAWSCredentials("myaccesskey", "mysecretkey");
var client = new Amazon.SimpleNotificationService.AmazonSimpleNotificationSer‌​viceClient(
awsCreden‌​tials, Amazon.RegionEndpoint.EUWest2);
Use a role if possible, but above works when needed. Then the question is where to store the access key/secret key; could be environment variable, config file, prompt the user, or any of the usual suspects.
AWS-CLI and Python use credentials from here: c:\Users\username\.aws\credentials, so the C# could just read that file so as not to put the codes in the C# program itself. But then each user/developer that runs the program would need to set their credentials there.
There is also now a concept of running Lambda on your local machine, but I haven't tried it yet:
https://dzone.com/articles/run-aws-lambda-functions-locally-on-windows-machin#:~:text=Step%201%3A%20Download%20SAM%20local,version%20with%20the%20command%20below.&text=Step%203%3A%20Write%20your%20lambda,yaml%20on%20the%20root%20level.
So the point is that if you are going to do Lambda, but you need to test locally first, this would probably be worth trying.
You'll want to construct one of its child classes instead of the abstract one. You can take a look at the class hierarchy here.
For posterity, the options are:
AnonymousAWSCredentials - Authenticates as an anonymous user.
BasicAWSCredentials - You provide your credentials to the class constructor directly.
EnvironmentAWSCredentials - Credentials are pulled from the environment variables of the running executable.
InstanceProfileAWSCredentials - Pulls credentials from the Instance Profile of the EC2 instance running the executable. This, obviously, only works on EC2.
SessionAWSCredentials - Similar to BasicAWSCredentials, except utilises an AWS Session using a temporary session token from AWS STS.
RefreshingSessionAWSCredentials - Similar to SessionAWSCredentials, but refreshes when the STS token expires.
Note that the default strategy in the absence of a credentials object involves checking the Environment Variables and then the instance profile.
If you want to have the program pull credentials from ~/.aws/credentials, you'll need to do some legwork. There used to be a StoredProfileAWSCredentials class, but that appears to have been removed - you can find more information by looking at this github issue. This is only useful, really, in development as you won't be using ~/.aws/credentials in production but probably instance profiles - I'd suggest instead using the default strategy and using Environment AWS credentials in test or development environments.
I take this approach at work since we use a command line tool to grab us limited time tokens from AWS STS and plunk them into the current shell for use for the next hour.
EDIT: It appears you're using AWS Lambda. These have federated access to AWS resources based on the roles assigned to them, so this should work using the default credential strategy in the aws-sdk library which uses instance profiles. So this is only really necessary for development/testing, in which case I would again recommend just using environment variables.
This is a really old question, and the existing answers work, but I really don't like hard-coding my Access Key Id and Secret Key values directly into source code, even for throw-away projects I'm doing on my local machine. For one thing, I might revoke those keys in the future, so I want to leverage the credentials in my .aws\credentials file.
To do that for my .NET core apps (including console apps, etc), I first add two NuGet packages:
Microsoft.Extensions.Configuration.Json
AWSSDK.Extensions.NETCore.Setup
Then, I add an applications.json file to my project, which contains the following (note - you need to right-click the file, and set "Copy to output" as either "copy if newer" or "always"):
{
"AWS": {
"Profile": "default",
"ProfilesLocation": "C:\\Users\\my-user-profile-folder\\.aws\\credentials",
"Region": "us-west-2"
}
}
Finally, I create an instance of the AWS SDK client using the following:
var builder = new ConfigurationBuilder().AddJsonFile("appsettings.Development.json", optional: false, reloadOnChange: true);
var options = builder.Build().GetAWSOptions();
var s3client = options.CreateServiceClient<IAmazonS3>();
This way, if I update my credentials file, I'm fine. Or if my code gets zipped up and emailed to a friend or co-worker, I don't accidentally send them my credentials also.
There is another way to do this, without needing to add the NuGet packages also, which many people might prefer. You can use the new SharedCredentialsFile class and AWSCredentialsFactory, like this (using the "default" profile here, and assumes your credential file is in the default location, same as the other method):
var sharedFile = new SharedCredentialsFile();
sharedFile.TryGetProfile("default", out var profile);
AWSCredentialsFactory.TryGetAWSCredentials(profile, sharedFile, out var credentials);
var s3Client = new AmazonS3Client(credentials);
Note - I'm not checking that the two Try* methods are succeeding here, which you probably should do. Details on using these classes are here: https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/net-dg-config-creds.html#how-to-create-an-amazons3client-using-the-sharedcredentialsfile-class
While keeping your credentials in the shared "credentials" file, you can redefine the ProfilesLocation when creating the CredentialProfileStoreChain
//define your file location here:
var chain = new CredentialProfileStoreChain(#"C:\aws\credentials");
// input the name of your credentials here:
if (chain.TryGetAWSCredentials("nameofprofile", out AWSCredentials awsCredentials))
{
//executes if the credentials were found and inserted into awsCredentials
}
else
{
// executes if the credentials were not found
}
Taken from here: https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/creds-locate.html
For those struggling with profile names, here is where you can find it.
Contents of your ~/.aws/credentials:
[YOUR_PROFILE_NAME]
aws_access_key_id = ***
aws_secret_access_key = ***
aws_security_token = ***
aws_session_expiration = ***
aws_session_token = ***
So then in your application you access the credentials like this:
var chain = new CredentialProfileStoreChain();
var result = chain.TryGetAWSCredentials("YOUR_PROFILE_NAME", out var credentials);
Resources:
accessing credentials and profiles: https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/net-dg-config-creds.html#creds-locate
named profiles: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html

How can I force HTTP-only mode in the C# AWS SDK?

It looks like there used to be an option in the AmazonS3Config class to specify the communication protocol to use, but I don't see that anymore. Where did it go?
I'm trying to do some benchmarking, and one of the things I want to test is HTTP vs HTTPS. Since a lot of our objects are fairly small (less than 512k), I'm wondering if the HTTPS handshaking is contributing to our slowness in uploading.
The configuration is now called UseHttp. It's in the 'Amazon.Runtime.ClientConfig' class that is the base for all of the client-specific config classes.
var config = new AmazonS3Config { UseHttp = true, ... };
var client = new AmazonS3Client(config);
This works for all clients (unless they don't support http. You can still set it, but it will be ignored).

Categories

Resources