Develop Reference

Why does AWS Lambda not upload images to S3 properly in .NET?

I'm having an issue with .NET 6 framework regarding the uploading of image files to S3 using the AWS SDK.
When I POST to my endpoint running on a local IIES it works perfectly and I can see the generated file in S3 without any issues.
The problem is the following: After a serverless deployment to AWS Lambda, the same .NET Core endpoint that produced a perfect result in my local environment behaves way different when it's running on a lambda. when I try to open the image it shows a square dot at the center but no image.
I am using IFormFile and here is my code
public async Task<string> Upload(IFormFile formfile, string name)
{
var xbuilder = WebApplication.CreateBuilder();
var _AwsSetting = xbuilder.Configuration.GetSection("AwsCredentials").Get<AWSCredentials>();
var accessKey = _AwsSetting.AWSAccessKey;
var secretKey = _AwsSetting.AWSSecretAccessKey;
RegionEndpoint bucketRegion = RegionEndpoint.APSouth1;
var bucketName = _AwsSetting.AWSS3BucketName;
var location = $"{name + Path.GetExtension(formfile.FileName)}";
var contentType = formfile.ContentType;
var client = new AmazonS3Client(accessKey, secretKey, bucketRegion);
try
{
using (var stream = new MemoryStream())
{
await formfile.CopyToAsync(stream);
var putRequest = new PutObjectRequest()
{
Key = location,
BucketName = bucketName,
InputStream = stream,
CannedACL = S3CannedACL.PublicRead,
ContentType=contentType
};
await client.PutObjectAsync(putRequest);
string publicUrl = string.Empty;
publicUrl = $"https://{bucketName}.s3.{bucketRegion.SystemName}.amazonaws.com/{location}";
return publicUrl;
}
}
catch (Exception e)
{
throw e;
}
}

Azure Storage upload Bug

I do not know if this is a bug or not, but I have made a Xamarin app, that will connect to Azure storage to upload a file.
It doesn't want to upload nd I get this error
Azure service, to upload the file
I made the same application, using a console app (for testing fester)
var path = Path.Combine(projectPath, "universal.txt");
var fullpath = Path.GetFullPath(path);
var FileName = Path.GetFileName(fullpath);
using (StreamWriter sw = new StreamWriter(fullpath)) {
sw.WriteLine("Hello I want to go to Universal tomorrow");
}
var AzureStorage = new BlobContainerClient(ConectionString, ContanerName);
var blob = AzureStorage.GetBlobClient(FileName);
await blob.UploadAsync(fullpath);
My file get uploaded to Azure
File in storage

Use these functions to upload a file from xamarin.
static CloudBlobContainer GetContainer(ContainerType containerType)
{
var account = CloudStorageAccount.Parse(Constants.StorageConnection);
var client = account.CreateCloudBlobClient();
return client.GetContainerReference(containerType.ToString().ToLower());
}
public static async Task<string> UploadFileAsync(ContainerType containerType, Stream stream)
{
var container = GetContainer(containerType);
await container.CreateIfNotExistsAsync();
var name = Guid.NewGuid().ToString();
var fileBlob = container.GetBlockBlobReference(name);
await fileBlob.UploadFromStreamAsync(stream);
return name;
}
OR
client = new BlobServiceClient(storageConnectionString);
containerClient = await client.CreateBlobContainerAsync(containerName);
blobClient = containerClient.GetBlobClient(fileName);
await containerClient.UploadBlobAsync(fileName, memoryStreamFile);

How do you decrypt blobs with Azure KeyVault keys in Azure.Storage v12

Our code is currently using the old Microsoft.WindowsAzure.Storage libraries for blob storage access in Azure. I am trying to use the new v12 Azure.Storage.Blobs libraries to replace the old ones, however I cannot figure out how to decrypt/encrypt the blobs. The MS docs (https://learn.microsoft.com/en-us/azure/storage/blobs/storage-encrypt-decrypt-blobs-key-vault?tabs=dotnet) helpfully say that the v12 code snippets aren't ready yet, so there are no code examples.
The old code is like this:
var tokenProvider = new AzureServiceTokenProvider();
var cloudResolver = new KeyVaultKeyResolver(
new KeyVaultClient.AuthenticationCallback(_tokenProvider.KeyVaultTokenCallback));
var encryptionThingy = await cloudResolver.ResolveKeyAsync(<Key Vault URL> + "/keys/" + <key name>, CancellationToken.None);
var policy = new BlobEncryptionPolicy(encryptionThingy, cloudResolver);
var options = new BlobRequestOptions() { EncryptionPolicy = policy };
await <ICloudBlob Instance>.DownloadToStreamAsync(<stream>, null, options, null);
So far with the new code I've gotten here:
var azureKeys = new KeyClient(new Uri(<key vault url>), new DefaultAzureCredential());
var encKey = azureKeys.GetKey(<key name>);
ClientSideEncryptionOptions encryptionOptions = new ClientSideEncryptionOptions(ClientSideEncryptionVersion.V1_0)
{
KeyEncryptionKey = (IKeyEncryptionKey)key
};
var bsClient = new BlobServiceClient(cStr, new SpecializedBlobClientOptions() { ClientSideEncryption = encryptionOptions });
var containerClient = new BlobContainerClient(cStr, containerName);
bClient = containerClient.GetBlobClient(<blob name>);
Of course this throws an exception because KeyVaultKey cannot be converted to IKeyEncryptionKey. So my questions are
Can the key be converted to an IKeyEncryptionKey easily, and how?
Can a key resolver be easily retrieved from the Azure SDKs, and how so?
I'm presuming there are ways to do this that don't involve creating our own implementations of the interfaces, but MS in their infinite wisdom didn't see fit to add those few lines to their documentation.

I write a simple demo for you. Just try the C# console app below about azure blob client-encryption with azure KeyVault:
using System;
using Azure.Identity;
using Azure.Security.KeyVault.Keys.Cryptography;
using Azure.Storage;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Specialized;
namespace BlobEncyptionWithBlob
{
class Program
{
static void Main(string[] args)
{
string keyVaultName = "";
string keyName = "";
string kvUri = "https://" + keyVaultName + ".vault.azure.net/keys/" + keyName;
string storageConnStr = "";
string containerName = "";
string encyptBlob = "encypt.txt";
string localblobPath = #"C:\Users\Administrator\Desktop\123.txt";
string localblobPath2 = #"C:\Users\Administrator\Desktop\123-decode.txt";
//Below is to use recommended OAuth2 approach
//string clientID = "<OAuth Client ID>";
//string clientSecret = "<OAuth Secret>";
//string tenant = "<OAuth Tenant ID>";
//var cred = new ClientSecretCredential(tenant, clientID, clientSecret);
//This is what you use to directly replace older AppAuthentication
var cred = new DefaultAzureCredential();
CryptographyClient cryptoClient = new CryptographyClient(new Uri(kvUri), cred);
KeyResolver keyResolver = new KeyResolver(cred);
ClientSideEncryptionOptions encryptionOptions = new ClientSideEncryptionOptions(ClientSideEncryptionVersion.V1_0)
{
KeyEncryptionKey = cryptoClient,
KeyResolver = keyResolver,
KeyWrapAlgorithm = "RSA-OAEP"
};
BlobClientOptions options = new SpecializedBlobClientOptions() { ClientSideEncryption = encryptionOptions };
var blobClient = new BlobServiceClient(storageConnStr,options).GetBlobContainerClient(containerName).GetBlobClient(encyptBlob);
//upload local blob to container
blobClient.Upload(localblobPath);
//If you want to modify the meta data you have to copy the exisiting meta, think there is a bug in the library that will wipe out the encryptiondata metadata if you write your own meta
var myMeta = new Dictionary<string, string>();
myMeta.Add("comment", "dis file is da shiznit");
foreach (var existingMeta in blobClient.GetProperties().Value.Metadata)
{
if (!myMeta.ContainsKey(existingMeta.Key))
{
myMeta.Add(existingMeta.Key, existingMeta.Value);
}
}
blobClient.SetMetadata(myMeta);
//Download from container to see if it is decided
blobClient.DownloadTo(localblobPath2);
}
}
}
Result:
My local .txt file content:
Upload to blob and its content, it has been encrypted :
Download to local again and its content, it has been decoded:

Encryption with Azure Bob Storage v12 SDK for .Net

i want to migrate my code to the v12 SDK, but how can i use Azure Keyvault?
There ist no BlobEncryptionPolicy class.
This tutorial is outdatet. It is still based on the old SDK.
v11 SDK Code:
// Retrieve the key that you created previously.
// The IKey that is returned here is an RsaKey.
var rsa = cloudResolver.ResolveKeyAsync(
"https://contosokeyvault.vault.azure.net/keys/TestRSAKey1",
CancellationToken.None).GetAwaiter().GetResult();
// Now you simply use the RSA key to encrypt by setting it in the BlobEncryptionPolicy.
BlobEncryptionPolicy policy = new BlobEncryptionPolicy(rsa, null);
BlobRequestOptions options = new BlobRequestOptions() { EncryptionPolicy = policy };
// Reference a block blob.
CloudBlockBlob blob = contain.GetBlockBlobReference("MyFile.txt");
// Upload using the UploadFromStream method.
using (var stream = System.IO.File.OpenRead(#"C:\Temp\MyFile.txt"))
blob.UploadFromStream(stream, stream.Length, null, options, null);

Regarding the issue, please refer to the following steps. For more details, please refer to here.
Create a Service principal and set access policy in Azure key vault fro the sp
Code (Install package ``)
string tenantId = "<sp tenant>";
string clientId = "<sp appId>";
string clientSecret = "<sp secret>";
string connectionString = "";
ClientSecretCredential cred = new ClientSecretCredential(tenantId, clientId, clientSecret);
var vaultUri = new Uri("https://jimkey02.vault.azure.net/");
KeyClient keyClient = new KeyClient(vaultUri, cred);
// if you do not have key, please use following code to create
//KeyVaultKey rasKey = await keyClient.CreateRsaKeyAsync(new CreateRsaKeyOptions("blobKey"));
KeyVaultKey rasKey = await keyClient.GetKeyAsync("blobKey", "<key version>");
IKeyEncryptionKey key =new CryptographyClient(rasKey.Id, cred);
IKeyEncryptionKeyResolver keyResolver = new KeyResolver(cred);
ClientSideEncryptionOptions encryptionOptions = new ClientSideEncryptionOptions(ClientSideEncryptionVersion.V1_0)
{
KeyEncryptionKey = key,
KeyResolver = keyResolver,
// string the storage client will use when calling IKeyEncryptionKey.WrapKey()
KeyWrapAlgorithm = "RSA1_5"
};
BlobClientOptions options = new SpecializedBlobClientOptions() { ClientSideEncryption = encryptionOptions };
BlobClient blob = new BlobServiceClient(connectionString, options).GetBlobContainerClient("test").GetBlobClient("test.txt");
using (FileStream file = File.OpenRead(#"D:\test.txt"))
{
await blob.UploadAsync(file);
}
BlobDownloadInfo download = await blob.DownloadAsync();
using (StreamReader reader = new StreamReader(download.Content)) {
string text = await reader.ReadToEndAsync();
Console.WriteLine(text);
}

C# downloading from Windows Azure Storage returns an empty file without any exceptions

I'm trying to use Windows Azure Storage for my Windows Store App with Mobile Services to store images. I made uploading work by following this guide:
http://www.windowsazure.com/en-us/develop/mobile/tutorials/upload-images-to-storage-dotnet/
However, I couldn't find any material on downloading the files. I couldn't even find classes reference for windows store version! If someone could guide me to the documentation I would be grateful.
Anyway, I wrote the code but it doesn't seem work:
public static async System.Threading.Tasks.Task DownloadUserImage(SQLUser userData)
{
var usersFolder = await GetUsersFolder();
var imageUri = new Uri(userData.ImageUri);
var accountName = "<SNIP>";
var key = "<SNIP>";
StorageCredentials cred = new StorageCredentials(accountName, key);
CloudBlobContainer container = new CloudBlobContainer(new Uri(string.Format("https://{0}/{1}", imageUri.Host, userData.ContainerName)), cred);
CloudBlockBlob blob = container.GetBlockBlobReference(userData.ResourceName);
var imageFile = await usersFolder.CreateFileAsync(userData.Id.ToString() + ".jpg", CreationCollisionOption.ReplaceExisting);
using (var fileStream = await imageFile.OpenAsync(FileAccessMode.ReadWrite))
{
try
{
await blob.DownloadToStreamAsync(fileStream);
}
catch (Exception e)
{
Tools.HandleLiveException(e);
}
}
}
This code results in empty file creation, but it doesn't throw any exceptions whatsoever. If I paste the value of imageUri to my browser, it starts downloading the file and completes the download successfully. However, my program does not, for some reason.
Any help, please?

Apparently, I've been opening the stream in a wrong way. Here's a fix:
public static async System.Threading.Tasks.Task DownloadUserImage(SQLUser userData)
{
var usersFolder = await GetUsersFolder();
var imageUri = new Uri(userData.ImageUri);
var accountName = "<SNIP>";
var key = "<SNIP>";
StorageCredentials cred = new StorageCredentials(accountName, key);
CloudBlobClient client = new CloudBlobClient(new Uri(string.Format("https://{0}", imageUri.Host)), cred);
CloudBlobContainer container = client.GetContainerReference(userData.ContainerName);
var blob = await container.GetBlobReferenceFromServerAsync(userData.ResourceName);
var imageFile = await usersFolder.CreateFileAsync(userData.Id.ToString() + ".jpg", CreationCollisionOption.ReplaceExisting);
using (var fileStream = await imageFile.OpenStreamForWriteAsync())
{
try
{
await blob.DownloadToStreamAsync(fileStream.AsOutputStream());
}
catch (Exception e)
{
Tools.HandleLiveException(e);
}
}
}
It works perfectly now.