Develop Reference

Authentication between my services hosted on Google Cloud, using service accounts & signed JWT

I have a .NET Core API that will use OAuth2 for authorizing API calls. I've managed to successfully use Microsoft Identity Web API authentication, I can successfully play with my API via Swagger, however, my API will be hosted in Google Cloud and other services (also hosted there) will be using my API, it seems to be a good idea to use GCloud Service Accounts for authorization.
I've been following for instructions and the results leave me confused.
As mentioned in the link above, I've added SecurityDefinition and SecurityRequirement to my specification file. Code:
var openApiOAuthFlowExtensions = new Dictionary<string, IOpenApiExtension>();
openApiOAuthFlowExtensions.Add("x-google-issuer", new OpenApiString("test-api-consumer#my-project.iam.gserviceaccount.com"));
openApiOAuthFlowExtensions.Add("x-google-jwks_uri", new OpenApiString("https://www.googleapis.com/robot/v1/metadata/x509/test-api-consumer#my-project.iam.gserviceaccount.com"));
openApiOAuthFlowExtensions.Add("x-google-audiences", new OpenApiString("https://my-project.ew.r.appspot.com/"));
c.AddSecurityDefinition("service-1-auth", new OpenApiSecurityScheme()
{
Name = "service-1-auth",
Type = SecuritySchemeType.OAuth2,
Flows = new OpenApiOAuthFlows()
{
Implicit = new OpenApiOAuthFlow
{
AuthorizationUrl = new Uri("https://accounts.google.com/o/oauth2/auth"),
Scopes = new Dictionary<string, string>
{
{ "readAccess", "Access read operations" },
{ "writeAccess", "Access write operations" }
},
Extensions = openApiOAuthFlowExtensions
}
},
});
c.AddSecurityRequirement(new OpenApiSecurityRequirement
{
{
new OpenApiSecurityScheme
{
Reference = new OpenApiReference
{
Type = ReferenceType.SecurityScheme,
Id = "service-1-auth"
}
},
new string[] { }
}
});
That's what I get in my swagger.json file:
"securitySchemes": {
"service-1-auth": {
"type": "oauth2",
"flows": {
"implicit": {
"authorizationUrl": "https://accounts.google.com/o/oauth2/auth",
"scopes": {
"readAccess": "Access read operations",
"writeAccess": "Access write operations"
},
"x-google-issuer": "test-api-consumer#my-project.iam.gserviceaccount.com",
"x-google-jwks_uri": "https://www.googleapis.com/robot/v1/metadata/x509/test-api-consumer#my-project.iam.gserviceaccount.com",
"x-google-audiences": "https://my-project.ew.r.appspot.com/"
}
}
}
}
},
"security": [
{
"service-1-auth": [ ]
}
]
Then I wrote a Python script as following this step and all I am getting is 401 Unauthorized... The JWT token generated by the script is exactly what I wanted, I've checked the decoded values many times.
What am I doing wrong? How should I debug here..? My gut feeling says that there's something wrong with the API configuration or that I simply don't know that I'm missing some crucial steps. I get the same results locally and on test environment. It's my first time implementing API authorization on my own so I'm pretty lost here. What's even more confusing is that I followed the guidelines step by step and still have no result. Maybe I use wrong packages? It doesn't make sense though.
I'd highly appreciate any kind of help.

Refresh token missing in Google Oauth response file

I'm implementing Google OAuth in ASP.Net MVC application using Google's OAuth .Net library. Below is my code.
IAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow(
new GoogleAuthorizationCodeFlow.Initializer {
ClientSecrets = new ClientSecrets {
** ClientId ** , ** ClientSecret **
},
DataStore = new FileDataStore( ** responsepath ** , true),
Scopes = new [] {
"https://www.googleapis.com/auth/userinfo.email",
"https://www.googleapis.com/auth/gmail.send"
},
Prompt = "select_account"
});
var userId = "user";
var uri = Request.Url.ToString();
var code = Request["code"];
if (code != null) {
var token = flow.ExchangeCodeForTokenAsync(userId, code, uri.Substring(0, uri.IndexOf("?")), CancellationToken.None).Result;
var oauthState = AuthWebUtility.ExtracRedirectFromState(flow.DataStore, userId, Request["state"]).Result;
Response.Redirect(oauthState);
} else {
var result = new AuthorizationCodeWebApp(flow, uri, uri).AuthorizeAsync(userId, CancellationToken.None).Result;
if (result.RedirectUri != null) {
Response.Redirect(result.RedirectUri);
}
}
When user click's Google sign-in button, my page is redirected to Google authentication page. After successful authentication, my page is displayed again. When I check the responsepath, below file is created which contains access token, expiry time, etc.
Google.Apis.Auth.OAuth2.Responses.TokenResponse-user
When I run the above code locally in my visual studio debugging environment (IIS express), the above response file has "refresh_token" in it. When the same code is deployed in production environment (IIS), the "refresh_token" is missing is response file. I would like to know the reason behind it since I need refresh token for further processing.
Note: In both the cases, I revoked the application's access from my Google account before trying. So, this is not about "refresh_token" will be sent only for the first time.

Adding prompt=consent parameter while sending request to Google gives refresh token every time without fail.

Need to create a folder(and a file inside it) using C# inside Azure DevOps repository - be it Git or TFVC

From Azure DevOps portal, I can manually add file/ folder into repository irrespective of the fact that source code is cloned or not - Image for illustration.
However, I want to programmatically create a folder and a file inside that folder within a Repository from c# code in my ASP .NET core application.
Is there a Azure DevOps service REST API or any other way to do that? I'll use BASIC authentication through PAT token only.
Note : I'm restricted to clone the source code at local repository.
Early reply is really appreciated.
I tried HttpClient, GitHttpClient and LibGit2Sharp but failed.

Follow below steps in your C# code
call GetRef REST https://dev.azure.com/{0}/{1}/_apis/git/repositories/{2}/refs{3}
this should return the object of your repository branch which you can use to push your changes
Next, call Push REST API to create folder or file into your repository
https://dev.azure.com/{0}/{1}/_apis/git/repositories/{2}/pushes{3}
var changes = new List<ChangeToAdd>();
//Add Files
//pnp_structure.yml
var jsonContent = File.ReadAllText(#"./static-files/somejsonfile.json");
ChangeToAdd changeJson = new ChangeToAdd()
{
changeType = "add",
item = new ItemBase() { path = string.Concat(path, "/[your-folder-name]/somejsonfile.json") },
newContent = new Newcontent()
{
contentType = "rawtext",
content = jsonContent
}
};
changes.Add(changeJson);
CommitToAdd commit = new CommitToAdd();
commit.comment = "commit from code";
commit.changes = changes.ToArray();
var content = new List<CommitToAdd>() { commit };
var request = new
{
refUpdates = refs,
commits = content
};
var personalaccesstoken = _configuration["azure-devOps-configuration-token"];
var authorization = Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(string.Format("{0}:{1}", "", personalaccesstoken)));
_logger.LogInformation($"[HTTP REQUEST] make a http call with uri: {uri} ");
//here I making http client call
// https://dev.azure.com/{orgnizationName}/{projectName}/_apis/git/repositories/{repositoryId}/pushes{?api-version}
var result = _httpClient.SendHttpWebRequest(uri, method, data, authorization);

Azure Storage Emulator - Configuring CORS dynamically throws Server Authentication error

I'm using Microsoft.WindowsAzure.Storage C# library in .Net Core 2.0.
Using this library I'm trying to configure CORS dynamically in Azure Storage Emulator but getting error:
"Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature".
public async void ConfigureCors() {
ServiceProperties serviceProperties = await this.blobClient.GetServicePropertiesAsync();
serviceProperties.Cors = new CorsProperties();
serviceProperties.Cors.CorsRules.Clear();
serviceProperties.Cors.CorsRules.Add(new CorsRule() {
AllowedHeaders = allowedCorsHeaders,
ExposedHeaders = allowedCorsExposedHeaders,
AllowedOrigins = allowedCorsOrigin,
AllowedMethods = allowedCorsMethods,
MaxAgeInSeconds = allowedCorsAgeInSeconds
});
await blobClient.SetServicePropertiesAsync(serviceProperties);
}
I'm able to generate SAS key for upload files on local server directly, but is not able to configure CORS dynamically so that I can access storage via C# code.
Strange thing to note is that the above code is working perfectly fine when using Azure Storage Cloud but local emulator is throwing this error.
Version info:
WindowsAzure.Storage version is 8.4.0
Windows Azure Storage Emulator version 5.2.0.0
Azure storage explorer version is 0.9.01
Credentials used for connection:
AccountName=devstoreaccount1;
AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuF‌​q2UVErCz4I6tq/K1SZFP‌​TOtr/KBHBeksoGMGw==;

According to your description, I have created a test demo on my side. It works well.
I guess the reason why you get the Server failed to authenticate the request error is the wrong azure storage package version and storage emulator version.
I suggest you could update the storage version to 8.4.0 and storage emulator version to 5.2 firstly and try again.
More details about my test demo, you could refer to below codes:
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("UseDevelopmentStorage=true");
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
ServiceProperties serviceProperties = blobClient.GetServicePropertiesAsync().Result;
serviceProperties.Cors = new CorsProperties();
serviceProperties.Cors.CorsRules.Clear();
serviceProperties.Cors.CorsRules.Add(new CorsRule()
{
AllowedHeaders = new List<string>() { "*" },
ExposedHeaders = new List<string>() { "*" },
AllowedOrigins = new List<string>() { "*" },
AllowedMethods = CorsHttpMethods.Put | CorsHttpMethods.Get | CorsHttpMethods.Head | CorsHttpMethods.Post,
MaxAgeInSeconds = 1800
});
var re = blobClient.SetServicePropertiesAsync(serviceProperties);
Result:

How to configure CORS setting for Blob storage in windows azure

I have created several containers in a azure storage and also uploaded some files into these containers. Now I need to give domain level access to the container/blobs. So I tried it from code level like below.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
ServiceProperties blobServiceProperties = new ServiceProperties();
blobServiceProperties.Cors.CorsRules.Add(new CorsRule(){
AllowedHeaders = new List<string>() {"*"},
ExposedHeaders = new List<string>() {"*"},
AllowedMethods = CorsHttpMethods.Post | CorsHttpMethods.Put | CorsHttpMethods.Get | CorsHttpMethods.Delete ,
AllowedOrigins = new List<string>() { "http://localhost:8080/"},
MaxAgeInSeconds = 3600,
});
blobClient.SetServiceProperties(GetBlobServiceProperties());
But above code seems to be work if I am creating everything from code (Correct me if I am wrong). I also find setting like below Here,
<CorsRule>
<AllowedOrigins>http://www.contoso.com, http://www.fabrikam.com</AllowedOrigins>
<AllowedMethods>PUT,GET</AllowedMethods>
<AllowedHeaders>x-ms-meta-data*,x-ms-meta-target,x-ms-meta-source</AllowedHeaders>
<ExposedHeaders>x-ms-meta-*</ExposedHeaders>
<MaxAgeInSeconds>200</MaxAgeInSeconds>
</CorsRule>
But I didn't get where this code have to put. I mean in which file. Or is there any setting for CORS while creating container or blob from azure portal. Please assist. Any help would be appreciable. Thanks!

The following answers the question that was actually asked in the title. It appears the questioner already knew how to do this largely from his code, but here is my answer to this. Unfortunately the code samples MS has put out has been far from easy or clear, so I hope this helps someone else. In this solution all you need is a CloudStorageAccount instance, which you can call this function from then (as an extension method).
// USAGE:
// -- example usage (in this case adding a wildcard CORS rule to this account --
CloudStorageAccount acc = getYourStorageAccount();
acc.SetCORSPropertiesOnBlobService(cors => {
var wildcardRule = new CorsRule() { AllowedMethods = CorsHttpMethods.Get, AllowedOrigins = { "*" } };
cors.CorsRules.Add(wildcardRule);
return cors;
});
// CODE:
/// <summary>
/// Allows caller to replace or alter the current CorsProperties on a given CloudStorageAccount.
/// </summary>
/// <param name="storageAccount">Storage account.</param>
/// <param name="alterCorsRules">The returned value will replace the
/// current ServiceProperties.Cors (ServiceProperties) value. </param>
public static void SetCORSPropertiesOnBlobService(this CloudStorageAccount storageAccount,
Func<CorsProperties, CorsProperties> alterCorsRules)
{
if (storageAccount == null || alterCorsRules == null) throw new ArgumentNullException();
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
ServiceProperties serviceProperties = blobClient.GetServiceProperties();
serviceProperties.Cors = alterCorsRules(serviceProperties.Cors) ?? new CorsProperties();
blobClient.SetServiceProperties(serviceProperties);
}
It may be helpful to consider the properties of the CorsRule class:
CorsRule corsRule = new CorsRule() {
AllowedMethods = CorsHttpMethods.Get, // Gets or sets the HTTP methods permitted to execute for this origin
AllowedOrigins = { "*" }, // (IList<string>) Gets or sets domain names allowed via CORS.
//AllowedHeaders = { "*" }, // (IList<string>) Gets or sets headers allowed to be part of the CORS request
//ExposedHeaders = null, // (IList<string>) Gets or sets response headers that should be exposed to client via CORS
//MaxAgeInSeconds = 33333 // Gets or sets the length of time in seconds that a preflight response should be cached by browser
};

Let me try to answer your question. As you know, Azure Storage offers a REST API for managing storage contents. An operation there is Set Blob Service Properties and one of the things you do there is manage CORS rules for blob service. The XML you have included in the question is the request payload for this operation. The C# code you mentioned is actually storage client library which is essentially a wrapper over this REST API written in .Net. So when you use the code above, it actually invokes the REST API and sends the XML.
Now coming to options on setting up CORS rules, there're a few ways you can achieve that. If you're interested in setting them up programmatically, then you can either write some code which consumes the REST API or you could directly use .Net storage client library as you have done above. You could simply create a console application, put the code in there and execute that to set the CORS rule. If you're looking for some tools to do that, then you can try one of the following tools:
Azure Management Studio from Cerebrata: http://www.cerebrata.com
Cloud Portam: http://www.cloudportam.com (Disclosure: This product is built by me).
Azure Storage Explorer (version 6.0): https://azurestorageexplorer.codeplex.com/

Its not a good idea to give domain level access to your containers. You can make the container private, upload the files (create blob) and then share it by using Shared Access Policy.
The below code can help you.
static void Main(string[] args)
{
var account = CloudStorageAccount.Parse(ConfigurationManager.ConnectionStrings["AzureStorageAccount"].ConnectionString);
var bClient = account.CreateCloudBlobClient();
var container = bClient.GetContainerReference("test-share-container-1");
container.CreateIfNotExists();
// clear all existing policy
ClearPolicy(container);
string newPolicy = "blobsharepolicy";
CreateSharedAccessPolicyForBlob(container, newPolicy);
var bUri = BlobUriWithNewPolicy(container, newPolicy);
Console.ReadLine();
}
static void ClearPolicy(CloudBlobContainer container)
{
var perms = container.GetPermissions();
perms.SharedAccessPolicies.Clear();
container.SetPermissions(perms);
}
static string BlobUriWithNewPolicy(CloudBlobContainer container, string policyName)
{
var blob = container.GetBlockBlobReference("testfile1.txt");
string blobContent = "Hello there !!";
MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(blobContent));
ms.Position = 0;
using (ms)
{
blob.UploadFromStream(ms);
}
return blob.Uri + blob.GetSharedAccessSignature(null, policyName);
}
static void CreateSharedAccessPolicyForBlob(CloudBlobContainer container, string policyName)
{
SharedAccessBlobPolicy sharedPolicy = new SharedAccessBlobPolicy()
{
SharedAccessExpiryTime = DateTime.UtcNow.AddHours(24),
Permissions = SharedAccessBlobPermissions.Write | SharedAccessBlobPermissions.Read
};
var permissions = container.GetPermissions();
permissions.SharedAccessPolicies.Add(policyName, sharedPolicy);
container.SetPermissions(permissions);
}
<connectionStrings>
<add name="AzureStorageAccount" connectionString="DefaultEndpointsProtocol=https;AccountName=[name];AccountKey=[key]" />
</connectionStrings>