I'm having an issue with .NET 6 framework regarding the uploading of image files to S3 using the AWS SDK.
When I POST to my endpoint running on a local IIES it works perfectly and I can see the generated file in S3 without any issues.
The problem is the following: After a serverless deployment to AWS Lambda, the same .NET Core endpoint that produced a perfect result in my local environment behaves way different when it's running on a lambda. when I try to open the image it shows a square dot at the center but no image.
I am using IFormFile and here is my code
public async Task<string> Upload(IFormFile formfile, string name)
{
var xbuilder = WebApplication.CreateBuilder();
var _AwsSetting = xbuilder.Configuration.GetSection("AwsCredentials").Get<AWSCredentials>();
var accessKey = _AwsSetting.AWSAccessKey;
var secretKey = _AwsSetting.AWSSecretAccessKey;
RegionEndpoint bucketRegion = RegionEndpoint.APSouth1;
var bucketName = _AwsSetting.AWSS3BucketName;
var location = $"{name + Path.GetExtension(formfile.FileName)}";
var contentType = formfile.ContentType;
var client = new AmazonS3Client(accessKey, secretKey, bucketRegion);
try
{
using (var stream = new MemoryStream())
{
await formfile.CopyToAsync(stream);
var putRequest = new PutObjectRequest()
{
Key = location,
BucketName = bucketName,
InputStream = stream,
CannedACL = S3CannedACL.PublicRead,
ContentType=contentType
};
await client.PutObjectAsync(putRequest);
string publicUrl = string.Empty;
publicUrl = $"https://{bucketName}.s3.{bucketRegion.SystemName}.amazonaws.com/{location}";
return publicUrl;
}
}
catch (Exception e)
{
throw e;
}
}
Related
I do not know if this is a bug or not, but I have made a Xamarin app, that will connect to Azure storage to upload a file.
It doesn't want to upload nd I get this error
Azure service, to upload the file
I made the same application, using a console app (for testing fester)
var path = Path.Combine(projectPath, "universal.txt");
var fullpath = Path.GetFullPath(path);
var FileName = Path.GetFileName(fullpath);
using (StreamWriter sw = new StreamWriter(fullpath)) {
sw.WriteLine("Hello I want to go to Universal tomorrow");
}
var AzureStorage = new BlobContainerClient(ConectionString, ContanerName);
var blob = AzureStorage.GetBlobClient(FileName);
await blob.UploadAsync(fullpath);
My file get uploaded to Azure
File in storage
Use these functions to upload a file from xamarin.
static CloudBlobContainer GetContainer(ContainerType containerType)
{
var account = CloudStorageAccount.Parse(Constants.StorageConnection);
var client = account.CreateCloudBlobClient();
return client.GetContainerReference(containerType.ToString().ToLower());
}
public static async Task<string> UploadFileAsync(ContainerType containerType, Stream stream)
{
var container = GetContainer(containerType);
await container.CreateIfNotExistsAsync();
var name = Guid.NewGuid().ToString();
var fileBlob = container.GetBlockBlobReference(name);
await fileBlob.UploadFromStreamAsync(stream);
return name;
}
OR
client = new BlobServiceClient(storageConnectionString);
containerClient = await client.CreateBlobContainerAsync(containerName);
blobClient = containerClient.GetBlobClient(fileName);
await containerClient.UploadBlobAsync(fileName, memoryStreamFile);
Our code is currently using the old Microsoft.WindowsAzure.Storage libraries for blob storage access in Azure. I am trying to use the new v12 Azure.Storage.Blobs libraries to replace the old ones, however I cannot figure out how to decrypt/encrypt the blobs. The MS docs (https://learn.microsoft.com/en-us/azure/storage/blobs/storage-encrypt-decrypt-blobs-key-vault?tabs=dotnet) helpfully say that the v12 code snippets aren't ready yet, so there are no code examples.
The old code is like this:
var tokenProvider = new AzureServiceTokenProvider();
var cloudResolver = new KeyVaultKeyResolver(
new KeyVaultClient.AuthenticationCallback(_tokenProvider.KeyVaultTokenCallback));
var encryptionThingy = await cloudResolver.ResolveKeyAsync(<Key Vault URL> + "/keys/" + <key name>, CancellationToken.None);
var policy = new BlobEncryptionPolicy(encryptionThingy, cloudResolver);
var options = new BlobRequestOptions() { EncryptionPolicy = policy };
await <ICloudBlob Instance>.DownloadToStreamAsync(<stream>, null, options, null);
So far with the new code I've gotten here:
var azureKeys = new KeyClient(new Uri(<key vault url>), new DefaultAzureCredential());
var encKey = azureKeys.GetKey(<key name>);
ClientSideEncryptionOptions encryptionOptions = new ClientSideEncryptionOptions(ClientSideEncryptionVersion.V1_0)
{
KeyEncryptionKey = (IKeyEncryptionKey)key
};
var bsClient = new BlobServiceClient(cStr, new SpecializedBlobClientOptions() { ClientSideEncryption = encryptionOptions });
var containerClient = new BlobContainerClient(cStr, containerName);
bClient = containerClient.GetBlobClient(<blob name>);
Of course this throws an exception because KeyVaultKey cannot be converted to IKeyEncryptionKey. So my questions are
Can the key be converted to an IKeyEncryptionKey easily, and how?
Can a key resolver be easily retrieved from the Azure SDKs, and how so?
I'm presuming there are ways to do this that don't involve creating our own implementations of the interfaces, but MS in their infinite wisdom didn't see fit to add those few lines to their documentation.
I write a simple demo for you. Just try the C# console app below about azure blob client-encryption with azure KeyVault:
using System;
using Azure.Identity;
using Azure.Security.KeyVault.Keys.Cryptography;
using Azure.Storage;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Specialized;
namespace BlobEncyptionWithBlob
{
class Program
{
static void Main(string[] args)
{
string keyVaultName = "";
string keyName = "";
string kvUri = "https://" + keyVaultName + ".vault.azure.net/keys/" + keyName;
string storageConnStr = "";
string containerName = "";
string encyptBlob = "encypt.txt";
string localblobPath = #"C:\Users\Administrator\Desktop\123.txt";
string localblobPath2 = #"C:\Users\Administrator\Desktop\123-decode.txt";
//Below is to use recommended OAuth2 approach
//string clientID = "<OAuth Client ID>";
//string clientSecret = "<OAuth Secret>";
//string tenant = "<OAuth Tenant ID>";
//var cred = new ClientSecretCredential(tenant, clientID, clientSecret);
//This is what you use to directly replace older AppAuthentication
var cred = new DefaultAzureCredential();
CryptographyClient cryptoClient = new CryptographyClient(new Uri(kvUri), cred);
KeyResolver keyResolver = new KeyResolver(cred);
ClientSideEncryptionOptions encryptionOptions = new ClientSideEncryptionOptions(ClientSideEncryptionVersion.V1_0)
{
KeyEncryptionKey = cryptoClient,
KeyResolver = keyResolver,
KeyWrapAlgorithm = "RSA-OAEP"
};
BlobClientOptions options = new SpecializedBlobClientOptions() { ClientSideEncryption = encryptionOptions };
var blobClient = new BlobServiceClient(storageConnStr,options).GetBlobContainerClient(containerName).GetBlobClient(encyptBlob);
//upload local blob to container
blobClient.Upload(localblobPath);
//If you want to modify the meta data you have to copy the exisiting meta, think there is a bug in the library that will wipe out the encryptiondata metadata if you write your own meta
var myMeta = new Dictionary<string, string>();
myMeta.Add("comment", "dis file is da shiznit");
foreach (var existingMeta in blobClient.GetProperties().Value.Metadata)
{
if (!myMeta.ContainsKey(existingMeta.Key))
{
myMeta.Add(existingMeta.Key, existingMeta.Value);
}
}
blobClient.SetMetadata(myMeta);
//Download from container to see if it is decided
blobClient.DownloadTo(localblobPath2);
}
}
}
Result:
My local .txt file content:
Upload to blob and its content, it has been encrypted :
Download to local again and its content, it has been decoded:
I'm using .net core to upload and retrieve an image from a private Amazon S3 bucket.
I'm able to upload it successfully, and even view it after I download it from S3, however when I'm a bit unsure about how to return the stream/response back to the client for the actual API call (for example right now I'm just trying to use Postman/Fiddler proxy tools to get back the image from my API)
My code for S3 to retrieve the stream:
///Retrieve my image from my bucket
public async Task<string> ReadObjectData(MediaFolder key, String fileName)
{
string responseBody = "";
IAmazonS3 client;
using (client = new AmazonS3Client(accessKey, accessSecret, endpoint))
{
Amazon.S3.Model.GetObjectRequest request = new Amazon.S3.Model.GetObjectRequest
{
BucketName = bucket,
Key = key + "/" + fileName,
};
using (GetObjectResponse response = await client.GetObjectAsync(request))
using (Stream responseStream = response.ResponseStream)
using (StreamReader reader = new StreamReader(responseStream))
{
string title = response.Metadata["x-amz-meta-title"];
responseBody = reader.ReadToEnd();
}
}
return responseBody;
}
So now in my controller, I have the following action:
[HttpGet("ProfilePic")]
public async Task<IActionResult> GetProfilePicture()
{
var user = await GetUserFromBearerToken();
//Retrieve
var utf8ImageResponse = await _fileService.ReadObjectData(MediaFolder.Profiles, user.ProfileImageFileName);
//To return a file as a stream
var imageBytes = System.Text.Encoding.UTF8.GetBytes(utf8ImageResponse);
//Return the image, which I'll hardcode as jpeg for a test
return File(imageBytes, "image/jpeg");
}
When I make the call using Postman, it returns a little blank box (the box you'd see if you tried to return an image, but it wasn't a valid image or null in some way).
Right now I'm using Postman but ideally I'd want an app to present this image.
Any ideas what I'm doing wrong? I tried messing around with base64 encoding and other things but nothing seems to work.
Thanks!
This way you can retrieve the file as stream from S3 storage
public async Task<Stream> ReadObjectData(MediaFolder key, String fileName)
{
try
{
using (var client = new AmazonS3Client(accessKey, accessSecret, endpoint))
{
var request = new GetObjectRequest
{
BucketName = bucket,
Key = key + "/" + fileName
};
using (var getObjectResponse = await client.GetObjectAsync(request))
{
using (var responseStream = getObjectResponse.ResponseStream)
{
var stream = new MemoryStream();
await responseStream.CopyToAsync(stream);
stream.Position = 0;
return stream;
}
}
}
}
catch (Exception exception)
{
throw new Exception("Read object operation failed.", exception);
}
}
And then - return this stream as FileStreamResult:
[HttpGet("ProfilePic")]
public async Task<IActionResult> GetProfilePicture()
{
var user = await GetUserFromBearerToken();
Stream imageStream = await _fileService.ReadObjectData(MediaFolder.Profiles, user.ProfileImageFileName);
Response.Headers.Add("Content-Disposition", new ContentDisposition
{
FileName = "Image.jpg",
Inline = true // false = prompt the user for downloading; true = browser to try to show the file inline
}.ToString());
return File(imageStream, "image/jpeg");
}
Hello I have created a windows application which uploads image from hdd to google cloud server.
My code was working perfectly but after changing bucket name it is not working.
My both buckets are in the same project and I have given OAuth 2.0 to my project.
even there is no error showing while processing. Please help me.
string bucketForImage = ConfigurationManager.AppSettings["BucketName"];
string projectName = ConfigurationManager.AppSettings["ProjectName"];
string Accountemail = ConfigurationManager.AppSettings["Email"];
var clientSecrets = new ClientSecrets();
clientSecrets.ClientId = ConfigurationManager.AppSettings["ClientId"];
clientSecrets.ClientSecret = ConfigurationManager.AppSettings["ClientSecret"];
string gcpPath = #"D:\mrunal\tst_mrunal.png";
var scopes = new[] { #"https://www.googleapis.com/auth/devstorage.full_control" };
var cts = new CancellationTokenSource();
var userCredential = await GoogleWebAuthorizationBroker.AuthorizeAsync(clientSecrets, scopes, Accountemail, cts.Token);
var service = new Google.Apis.Storage.v1.StorageService();
var bucketToUpload = bucketForImage;
var newObject = new Google.Apis.Storage.v1.Data.Object()
{
Bucket = bucketToUpload,
Name = "mrunal.png"
};
fileStream = new FileStream(gcpPath, FileMode.Open);
var uploadRequest = new Google.Apis.Storage.v1.ObjectsResource.InsertMediaUpload(service, newObject,
bucketToUpload, fileStream, "image/png");
uploadRequest.OauthToken = userCredential.Token.AccessToken;
await uploadRequest.UploadAsync();
//uploadRequest.UploadAsync();
if (fileStream != null)
{
fileStream.Dispose();
}
Did you try this same code for the older bucket and it worked? It seems to me that there is an issue with line of code, uploadRequest.OauthToken = userCredential.Token.AccessToken. You are calling the Token.AccessToken directly from the userCredentials. These methods should be called from the userCredentials.Result.
I'm trying to use Windows Azure Storage for my Windows Store App with Mobile Services to store images. I made uploading work by following this guide:
http://www.windowsazure.com/en-us/develop/mobile/tutorials/upload-images-to-storage-dotnet/
However, I couldn't find any material on downloading the files. I couldn't even find classes reference for windows store version! If someone could guide me to the documentation I would be grateful.
Anyway, I wrote the code but it doesn't seem work:
public static async System.Threading.Tasks.Task DownloadUserImage(SQLUser userData)
{
var usersFolder = await GetUsersFolder();
var imageUri = new Uri(userData.ImageUri);
var accountName = "<SNIP>";
var key = "<SNIP>";
StorageCredentials cred = new StorageCredentials(accountName, key);
CloudBlobContainer container = new CloudBlobContainer(new Uri(string.Format("https://{0}/{1}", imageUri.Host, userData.ContainerName)), cred);
CloudBlockBlob blob = container.GetBlockBlobReference(userData.ResourceName);
var imageFile = await usersFolder.CreateFileAsync(userData.Id.ToString() + ".jpg", CreationCollisionOption.ReplaceExisting);
using (var fileStream = await imageFile.OpenAsync(FileAccessMode.ReadWrite))
{
try
{
await blob.DownloadToStreamAsync(fileStream);
}
catch (Exception e)
{
Tools.HandleLiveException(e);
}
}
}
This code results in empty file creation, but it doesn't throw any exceptions whatsoever. If I paste the value of imageUri to my browser, it starts downloading the file and completes the download successfully. However, my program does not, for some reason.
Any help, please?
Apparently, I've been opening the stream in a wrong way. Here's a fix:
public static async System.Threading.Tasks.Task DownloadUserImage(SQLUser userData)
{
var usersFolder = await GetUsersFolder();
var imageUri = new Uri(userData.ImageUri);
var accountName = "<SNIP>";
var key = "<SNIP>";
StorageCredentials cred = new StorageCredentials(accountName, key);
CloudBlobClient client = new CloudBlobClient(new Uri(string.Format("https://{0}", imageUri.Host)), cred);
CloudBlobContainer container = client.GetContainerReference(userData.ContainerName);
var blob = await container.GetBlobReferenceFromServerAsync(userData.ResourceName);
var imageFile = await usersFolder.CreateFileAsync(userData.Id.ToString() + ".jpg", CreationCollisionOption.ReplaceExisting);
using (var fileStream = await imageFile.OpenStreamForWriteAsync())
{
try
{
await blob.DownloadToStreamAsync(fileStream.AsOutputStream());
}
catch (Exception e)
{
Tools.HandleLiveException(e);
}
}
}
It works perfectly now.