I want my application to be able to initialize the data protection key storage at startup.
But it will only work if I use the Azure Storage Access Key and not the MSI or Azure user in my Visual Studio (az login) that I expected. I have given the user and VM Scale Set the roles Owner, Contributor, Storage blob Data Contributor without success.
Is it a must to use SAS or the Access Key in order for the blob to be created automatically?
Error when using the token provider (MSI, az login)
Microsoft.AspNetCore.DataProtection.Internal.DataProtectionHostedService:
Information: Key ring failed to load during application startup.
Request Information
StatusMessage:The specified resource does not exist.
ErrorCode:ResourceNotFound
ErrorMessage:The specified resource does not exist.
public void ConfigureServices( IServiceCollection services )
{
var accessKey = "ddBU/...==";
var blobUri = new Uri( $"https://mystorageaccount.blob.core.windows.net/mycontainer/keys.xml" );
var tp = new Microsoft.Azure.Services.AppAuthentication.AzureServiceTokenProvider();
var token = tp.GetAccessTokenAsync( $"https://mystorageaccount.blob.core.windows.net/" ).Result;
// OK - creates and updates blob when neccessary
var sc = new Microsoft.Azure.Storage.Auth.StorageCredentials( "mystorageaccount", accessKey );
// NOK - can only read the blob
//sc = new Microsoft.Azure.Storage.Auth.StorageCredentials( token );
var cbb = new Microsoft.Azure.Storage.Blob.CloudBlockBlob( blobUri, sc );
services.AddDataProtection()
.PersistKeysToAzureBlobStorage( cbb )
.SetApplicationName( "MyFrontends" );
}
The role "Storage blob Data Contributor" does allow PersistKeysToAzureBlobStorage() to create and maintain the key in the Azure Storage Account.
I must have made a mistake while testing, maybe I did not wait long enough after applying the role or maybe I had not created the target container.
Related
I'm trying to get a list of App Services from a subscription so that I can filter staging sites and delete them using the Azure SDK. I'm running into the issue of not authenticating properly or not being able to see any resources on the subscription.
I have the "Owner" role on the subscription so I should have total access to the subscription and its resources however when trying to follow Microsoft's docs on authenticating (https://learn.microsoft.com/en-us/dotnet/azure/sdk/authentication) I can't seem to find a way to authenticate at the subscription level.
Here's the code I have:
var azurecreds = SdkContext.AzureCredentialsFactory
.FromServicePrincipal(
"???", //client ID
"???", //client secret
"have this", //tenant ID
AzureEnvironment.AzureGlobalCloud);
var azure = Azure
.Configure()
.Authenticate(azurecreds)
.WithSubscription("have this"); //subscription ID
//attempts with hard-coded values but not working
var appServicePlans = azure.AppServices.AppServicePlans.List();
var appServicePlans2 = azure.WebApps.List();
var appServicePlans2 = azure.AppServices.AppServicePlans.ListByResourceGroup("Staging");
As you are following this document : Authenticate with token credentials
So , as per the above document , you must have created a service principal using this command :
az ad sp create-for-rbac --sdk-auth
After you have create this service principal , you will get the below details :
From the above picture you have to copy the ClientID, Client Secret, TenantID and SubscriptionId. After you have taken a note of these mentioned details , you can put the same in the code .
var azurecreds = SdkContext.AzureCredentialsFactory
.FromServicePrincipal(
"ClientID copied from the above step", //client ID
"Client Secret Copied from the above step", //client secret
"have this", //tenant ID
AzureEnvironment.AzureGlobalCloud);
var azure = Azure
.Configure()
.Authenticate(azurecreds)
.WithSubscription("have this"); //subscription ID
//attempts with hard-coded values but not working
var appServicePlans = azure.AppServices.AppServicePlans.List();
var appServicePlans2 = azure.WebApps.List();
var appServicePlans2 = azure.AppServices.AppServicePlans.ListByResourceGroup("Staging");
I'm trying to test client side encryption with an azure storage account. So far I've created a resource group and put my KeyVault, Registered App on Active Directory and inside my keyVault I've created a secret.
I think im failing to map my secret to my storage account, but I figured that they should work if they are in the same resource group.
$key = "qwertyuiopasdfgh"
$b = [System.Text.Encoding]::UTF8.GetBytes($key)
$enc = [System.Convert]::ToBase64String($b)
$secretvalue = ConvertTo-SecureString $enc -AsPlainText -Force
$secret = Set-AzureKeyVaultSecret -VaultName 'ectotecStorageKeyVault' -Name 'ectotecSecret' -SecretValue $secretvalue -ContentType "application/octet-stream"
The problem is that im getting an invalid secret provided error with the following code:
namespace cifradoApp
{
class Program
{
private async static Task<string> GetToken(string authority, string resource, string scope)
{
var authContext = new AuthenticationContext(authority);
ClientCredential clientCred = new ClientCredential(
ConfigurationManager.AppSettings["clientId"],
ConfigurationManager.AppSettings["clientSecret"]);
AuthenticationResult result = await authContext.AcquireTokenAsync(resource, clientCred);
if (result == null)
throw new InvalidOperationException("Failed to obtain the JWT token");
return result.AccessToken;
}
static void Main(string[] args)
{
// This is standard code to interact with Blob storage.
StorageCredentials creds = new StorageCredentials(
ConfigurationManager.AppSettings["accountName"],
ConfigurationManager.AppSettings["accountKey"]
);
CloudStorageAccount account = new CloudStorageAccount(creds, useHttps: true);
CloudBlobClient client = account.CreateCloudBlobClient();
CloudBlobContainer contain = client.GetContainerReference(ConfigurationManager.AppSettings["container"]);
contain.CreateIfNotExists();
// The Resolver object is used to interact with Key Vault for Azure Storage.
// This is where the GetToken method from above is used.
KeyVaultKeyResolver cloudResolver = new KeyVaultKeyResolver(GetToken);
// Retrieve the key that you created previously.
// The IKey that is returned here is an RsaKey.
// Remember that we used the names contosokeyvault and testrsakey1.
var rsa = cloudResolver.ResolveKeyAsync("https://ectotecstoragekeyvault.vault.azure.net/secrets/ectotecSecret/dee97a40c78a4638bbb3fa0d3e13f75e", CancellationToken.None).GetAwaiter().GetResult();
// Now you simply use the RSA key to encrypt by setting it in the BlobEncryptionPolicy.
BlobEncryptionPolicy policy = new BlobEncryptionPolicy(rsa, null);
BlobRequestOptions options = new BlobRequestOptions() { EncryptionPolicy = policy };
// Reference a block blob.
CloudBlockBlob blob = contain.GetBlockBlobReference("BlobPruebaEncrypted.txt");
// Upload using the UploadFromStream method.
using (var stream = System.IO.File.OpenRead(#"C:\Users\moise\Desktop\ectotec stuff\Visual Studio\azureStorageSample\container\BlobPrueba.txt"))
blob.UploadFromStream(stream, stream.Length, null, options, null);
}
}
}
My app settings seems to be working fine, since i valide before with only my account and key to access the storage account, since I made tests without trying to do client side encryption, everything worked out just fine. The problem comes with the secret it seems.
ERROR WHEN I TRY TO UPLOAD SOMETHING TO MY STORAGE ACCOUNT CONTAINER(BLOB)
AdalException: {"error":"invalid_client","error_description":"AADSTS70002: Error validating credentials. AADSTS50012: Invalid client secret is provided.\r\nTrace ID: 52047a12-b950-4d8a-9206-120e383feb00\r\nCorrelation ID: e2ad8afe-4272-49aa-94c0-5dad435ffc45\r\nTimestamp: 2019-01-02 17:10:32Z","error_codes":[70002,50012],"timestamp":"2019-01-02 17:10:32Z","trace_id":"52047a12-b950-4d8a-9206-120e383feb00","correlation_id":"e2ad8afe-4272-49aa-94c0-5dad435ffc45"}: Unknown error
<appSettings>
<add key="accountName" value="sampleExample"/>
<add key="accountKey" value="KeyForMyApp"/>
<add key="clientId" value="app-id"/>
<add key="clientSecret" value="qwertyuiopasdfgh"/>
<add key="container" value="ectotec-sample2"/>
</appSettings>
I'm trying to replicate the example in this tutorial:
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-encrypt-decrypt-blobs-key-vault
You need to make sure that you have granted your appliation rights to read keys. This is seperate from the RBAC permissions on the Key Vault.
To do this, browse to teh Key Vault in the portal, on the menu on the left you will see a settings section, and under here an item called "access policies", click on this.
You then want to click the "Add New" button. In the window that opens, click on the "Select Principal" section, and then enter in the name or application ID of the application you want to have access. Select the appropriate permissions for keys, secrets or certificates and then click OK.
This will take you back to the list of authorised users, be sure to click save at the top left (it isn't obvious you need to do this), your app should then have access.
Getting an error when trying to use a SAS token.
I generate the token on the container like so:
var client = _account.CreateCloudBlobClient();
var container = client.GetContainerReference("2017-med");
var sas = container.GetSharedAccessSignature(new SharedAccessBlobPolicy
{
Permissions = SharedAccessBlobPermissions.Read,
SharedAccessExpiryTime = DateTimeOffset.UtcNow.AddYears(5)
});
I append it to the URL of a resource that is in the container and get this error message when I visit the URL:
Signature did not match. String to sign used was r
2023-05-14T05:48:34Z /blob/myblobname/2017-med 2017-07-29
Is it possible to use a container token for resources in the container? Or is there something else at play here?
As I have tested, you search the sasuri in the explorer and the container permission you set is only read.
I suggest that you could add a list permission in your sas policy like below:
var sas = container.GetSharedAccessSignature(new SharedAccessBlobPolicy
{
Permissions = SharedAccessBlobPermissions.Read | SharedAccessBlobPermissions.List
SharedAccessExpiryTime = DateTimeOffset.UtcNow.AddYears(5)
});
And then when you search it in your explorer you could list all the blob in your container to judge if the sasuri you generated is correct.
The List Blobs request may be constructed as follows.
HTTPS is recommended. Replace myaccount with the name of your storage account:
https://myaccount.blob.core.windows.net/mycontainer?restype=container&comp=list&sastoken
Actually, when you generate the sasuri for container, you could use the sasuri to access the blob in the container and do some operations within your permission.
string sasUri = uricontainer;
CloudBlockBlob blob = new CloudBlockBlob(new Uri(sasUri));
Update:
Signature did not match. String to sign used was r 2023-05-14T05:48:34Z /blob/myblobname/2017-med 2017-07-29
As you have provided, I found that your storagename is myblobname, which is very puzzled.
And I found there is indeed existing the storagename called myblobname. So, I guess that maybe you have a splicing error.
You could use var bloburi = container.Uri + sas; to get the full uri.
And if you want to show list or other operation with full uri, you could refer to the above code I provided.
I've got a basic web app in C# MVC (i'm new to MVC) which is connected to a database. In that database there is a table with a list of filenames. These files are stored in Azure Storage Blob Container.
I've used Scaffolding (creates a controller and view) to show data from my table of filenames and that works fine.
Now I would like to connect those filenames to the blob storage so that the user can click on and open them. How do I achieve this?
Do I edit the index view? Do I get the user to click on a filename and then connect to Azure storage to open that file? How is this done?
Please note that files on storage are private and is accessed using the storage key. Files cannot be made public.
Thanks for any advice.
[Update]
I've implemented the Shared Access Signature (SAS) using the code below.
public static string GetSASUrl(string containerName)
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
BlobContainerPermissions containerPermissions = new BlobContainerPermissions();
containerPermissions.SharedAccessPolicies.Add("twominutepolicy", new SharedAccessBlobPolicy()
{
SharedAccessStartTime = DateTime.UtcNow.AddMinutes(-1),
SharedAccessExpiryTime = DateTime.UtcNow.AddMinutes(2),
Permissions = SharedAccessBlobPermissions.Write | SharedAccessBlobPermissions.Read
});
containerPermissions.PublicAccess = BlobContainerPublicAccessType.Off;
container.SetPermissions(containerPermissions);
string sas = container.GetSharedAccessSignature(new SharedAccessBlobPolicy(), "twominutepolicy");
return sas;
}
public static string GetSasBlobUrl(string containerName, string fileName, string sas)
{
// Create new storage credentials using the SAS token.
StorageCredentials accountSAS = new StorageCredentials(sas);
// Use these credentials and the account name to create a Blob service client.
CloudStorageAccount accountWithSAS = new CloudStorageAccount(accountSAS, [Enter Account Name], endpointSuffix: null, useHttps: true);
CloudBlobClient blobClientWithSAS = accountWithSAS.CreateCloudBlobClient();
// Retrieve reference to a previously created container.
CloudBlobContainer container = blobClientWithSAS.GetContainerReference(containerName);
// Retrieve reference to a blob named "photo1.jpg".
CloudBlockBlob blockBlob = container.GetBlockBlobReference(fileName);
return blockBlob.Uri.AbsoluteUri + sas;
}
In order to access blobs that are not public, you'll need to use Shared Access Signatures, with that, you'll create access tokens valid for a period of time (you'll choose) and you can also restrict by IP address.
More info in here:
https://learn.microsoft.com/en-us/azure/storage/storage-dotnet-shared-access-signature-part-1
As they are not public, you'll need to add an additional step before pass the data to your view, which is concatenate the SAS token to the blob Uri. You can find a very good example in here: http://www.dotnetcurry.com/windows-azure/901/protect-azure-blob-storage-shared-access-signature
I'm looking to start an Azure runbook from a c# application which will be hosted on an Azure web app.
I'm using certificate authentication (in an attempt just to test that I can connect and retrieve some data)
Here's my code so far:
var cert = ConfigurationManager.AppSettings["mgmtCertificate"];
var creds = new Microsoft.Azure.CertificateCloudCredentials("<my-sub-id>",
new X509Certificate2(Convert.FromBase64String(cert)));
var client = new Microsoft.Azure.Management.Automation.AutomationManagementClient(creds, new Uri("https://management.core.windows.net/"));
var content = client.Runbooks.List("<resource-group-id>", "<automation-account-name>");
Every time I run this, no matter what certificate I use I get the same error:
An unhandled exception of type 'Hyak.Common.CloudException' occurred in Microsoft.Threading.Tasks.dll
Additional information: ForbiddenError: The server failed to authenticate the request. Verify that the certificate is valid and is associated with this subscription.
I've tried downloading the settings file which contains the automatically generated management certificate you get when you spin up the Azure account... nothing I do will let me talk to any of the Azure subscription
Am I missing something fundamental here?
Edit: some additional info...
So I decided to create an application and use the JWT authentication method.
I've added an application, given the application permissions to the Azure Service Management API and ensured the user is a co-administrator and I still get the same error, even with the token...
const string tenantId = "xx";
const string clientId = "xx";
var context = new AuthenticationContext(string.Format("https://login.windows.net/{0}", tenantId));
var user = "<user>";
var pwd = "<pass>";
var userCred = new UserCredential(user, pwd);
var result = context.AcquireToken("https://management.core.windows.net/", clientId, userCred);
var token = result.CreateAuthorizationHeader().Substring("Bearer ".Length); // Token comes back fine and I can inspect and see that it's valid for 1 hour - all looks ok...
var sub = "<subscription-id>";
var creds = new TokenCloudCredentials(sub, token);
var client = new AutomationManagementClient(creds, new Uri("https://management.core.windows.net/"));
var content = client.Runbooks.List("<resource-group>", "<automation-id>");
I've also tried using other Azure libs (like auth, datacentre etc) and I get the same error:
ForbiddenError: The server failed to authenticate the request. Verify that the certificate is valid and is associated with this subscription.
I'm sure it's just 1 tickbox I need to tick buried somewhere in that monolithic Management Portal but I've followed a few tutorials on how to do this and they all end up with this error...
public async Task StartAzureRunbook()
{
try
{
var subscriptionId = "azure subscription Id";
string base64cer = "****long string here****"; //taken from http://stackoverflow.com/questions/24999518/azure-api-the-server-failed-to-authenticate-the-request
var cert = new X509Certificate2(Convert.FromBase64String(base64cer));
var client = new Microsoft.Azure.Management.Automation.AutomationManagementClient(new CertificateCloudCredentials(subscriptionId, cert));
var ct = new CancellationToken();
var content = await client.Runbooks.ListByNameAsync("MyAutomationAccountName", "MyRunbookName", ct);
var firstOrDefault = content?.Runbooks.FirstOrDefault();
if (firstOrDefault != null)
{
var operation = client.Runbooks.Start("MyAutomationAccountName", new RunbookStartParameters(firstOrDefault.Id));
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
Also in portal:
1) Application is multitenant
2) Permissions to other applications section - Windows Azure Service Manager - Delegated permissions "Access Azure Service Management(preview)"
Ensure that your Management certificate has private key and was not made from the .CER file. The fact that you're not supplying a password when generating the X509Certificate object makes me think you're using public key only
Ensure that your Managemnet's certificate public key (.CER file) has been uploaded to the Azure management portal (legacy version, Management Certificate area)
Use CertificateCloudCredentials and not any other credential type of an object
Ok, stupid really but one of the tutorials I followed suggested installing the prerelease version of the libs.
Installing the preview (0.15.2-preview) has fixed the issue!