Develop Reference

Renaming a container/folder in azure blob storage

I am trying to rename a container name for users when a condition is met. I made some research and found out that there is no rename function for containers in azure blob storage. But there is a way to accomplish this by copying the files and deleting it after copy. Below is the code I made.
string ContainerName = "old-container-name-user1";
string NewContainerName = "new-container-name-user2"
CloudStorageAccount sourceAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
CloudBlobClient sourceblobClient = sourceAccount.CreateCloudBlobClient();
CloudBlobContainer sourceBlobContainer = sourceblobClient.GetContainerReference(ContainerName);
CloudBlobContainer destBlobContainer = sourceblobClient.GetContainerReference(NewContainerName);
CloudBlockBlob blobOriginal = sourceBlobContainer.GetBlockBlobReference(ContainerName);
CloudBlockBlob blobNew = destBlobContainer.GetBlockBlobReference(NewContainerName);
blobNew.StartCopy(blobOriginal);
blobOriginal.Delete();
When I execute this code I got an error message. Below is the error.
Exception User-Unhandled
Microsoft.WindowsAzure.Storage.StorageException:'The remote server
returned an error: (404) Not Found.'
Inner Exception WebException: The remote server returned an error:
(404) Not Found.
When I also try "blobNew.StartCopyAsync(blobOriginal)" the code just goes through but when I check the containers in azure there is no container created. What do you think the problem is? Any tips on how to improve my code? Delete function does not work also.
UPDATE
I update my code and was able to copy the files from other to container to the new one. Below is the code.
string ContainerName = "old-container-name-user1"
string NewContainerName = "new-container-name-user2"
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(ContainerName);
CloudBlobContainer destcontainer = blobClient.GetContainerReference(NewContainerName);
destcontainer.CreateIfNotExists(BlobContainerPublicAccessType.Blob);
IEnumerable<IListBlobItem> IE = container.ListBlobs(useFlatBlobListing: true);
foreach (IListBlobItem item in IE)
{
CloudBlockBlob blob = (CloudBlockBlob)item;
CloudBlockBlob destBlob = destcontainer.GetBlockBlobReference(blob.Name);
destBlob.StartCopyAsync(new Uri(GetSharedAccessUri(blob.Name, container)));
}
AccessURI Method
private static string GetSharedAccessUri(string blobName, CloudBlobContainer container)
{
DateTime toDateTime = DateTime.Now.AddMinutes(60);
SharedAccessBlobPolicy policy = new SharedAccessBlobPolicy
{
Permissions = SharedAccessBlobPermissions.Read,
SharedAccessStartTime = null,
SharedAccessExpiryTime = new DateTimeOffset(toDateTime)
};
CloudBlockBlob blob = container.GetBlockBlobReference(blobName);
string sas = blob.GetSharedAccessSignature(policy);
return blob.Uri.AbsoluteUri + sas;
}
Now its working but another problem shows up. It says
System.InvalidCastException: 'Unable to cast object of type
'Microsoft.WindowsAzure.Storage.Blob.CloudBlobDirectory' to type
'Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob'.'
But this will be posted in another question. Thanks to our friend Gauriv for pointing out my problem. Check his answer below.
Update 2
By adding useFlatBlobListing: true in IEnumerable<IListBlobItem> IE = container.ListBlobs(blobListingDetails: BlobListingDetails.Metadata); I was able to fix my problem. This line of code was put in my display.
Final Code
IEnumerable<IListBlobItem> IE = container.ListBlobs(blobListingDetails: BlobListingDetails.Metadata, useFlatBlobListing: true);

If you look at your code for creating source and destination blob:
CloudBlockBlob blobOriginal = sourceBlobContainer.GetBlockBlobReference(ContainerName);
CloudBlockBlob blobNew = destBlobContainer.GetBlockBlobReference(NewContainerName);
You'll notice that you're passing the names of the blob container and not the name of the blob. Because you don't have a blob in the container by the name of the container, you're getting 404 error.
To copy a blob container, what you have to do is list all blobs from the source container and then copy them individually in the destination container. Once all the blobs have been copied, you can delete the source container.
If you want, you can use Microsoft's Storage Explorer to achieve "rename container" functionality. It also works by copying blobs from old container to the renamed container and then deletes the old container.
static void RenameContainer()
{
var connectionString = "DefaultEndpointsProtocol=https;AccountName=account-name;AccountKey=account-key";
var storageAccount = CloudStorageAccount.Parse(connectionString);
var client = storageAccount.CreateCloudBlobClient();
var sourceContainer = client.GetContainerReference("source-container");
var targetContainer = client.GetContainerReference("target-container");
targetContainer.CreateIfNotExists();//Create target container
BlobContinuationToken continuationToken = null;
do
{
Console.WriteLine("Listing blobs. Please wait...");
var blobsResult = sourceContainer.ListBlobsSegmented(prefix: "", useFlatBlobListing: true, blobListingDetails: BlobListingDetails.All, maxResults: 1000, currentToken: continuationToken, options: new BlobRequestOptions(), operationContext: new OperationContext());
continuationToken = blobsResult.ContinuationToken;
var items = blobsResult.Results;
foreach (var item in items)
{
var blob = (CloudBlob)item;
var targetBlob = targetContainer.GetBlobReference(blob.Name);
Console.WriteLine(string.Format("Copying \"{0}\" from \"{1}\" blob container to \"{2}\" blob container.", blob.Name, sourceContainer.Name, targetContainer.Name));
targetBlob.StartCopy(blob.Uri);
}
} while (continuationToken != null);
Console.WriteLine("Deleting source blob container. Please wait.");
//sourceContainer.DeleteIfExists();
Console.WriteLine("Rename container operation complete. Press any key to terminate the application.");
}

You can rename containers with Microsoft's "Microsoft Azure Storage Explorer" (after version 0.8.3).
Actual Answer:
Regarding your error message, If the client application receives an HTTP 404 (Not found) message from the server, this means that the object the client was attempting to use does not exist in the storage service. There are several possible reasons for this, such as:
· The client or another process previously deleted the object (Make sure name is correct)
· A Shared Access Signature (SAS) authorization issue
· Client-side code does not have permission to access the object
· Network failure
In order to identify the issue in detail, you can add a try/catch and see the actual error

Changing Metadata["Filename"] of blob and Container when copying and deleting container

I am trying to change the container name and specific files inside of it when a certain condition was met. The 1st code below works fine. It can change the container name and blob.name correctly. The issue is the 2nd code below.
Below is my code.
string ContainerName = "old-container-name";
string NewContainerName = "new-container-name";
var storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
var blobClient = storageAccount.CreateCloudBlobClient();
var sourcecontainer = blobClient.GetContainerReference(ContainerName);
var destcontainer = blobClient.GetContainerReference(NewContainerName);
destcontainer.CreateIfNotExists(BlobContainerPublicAccessType.Blob);
BlobContinuationToken continuationToken = null;
do
{
var blobsResult = sourcecontainer.ListBlobsSegmented(prefix: "", useFlatBlobListing: true, blobListingDetails: BlobListingDetails.All, maxResults: 1000, currentToken: continuationToken, options: new BlobRequestOptions(), operationContext: new OperationContext());
continuationToken = blobsResult.ContinuationToken;
var items = blobsResult.Results;
foreach (var item in items)
{
string newName = "";
var blob = (CloudBlob)item;
if (blob.Name.Contains("originalfilename"))
newName = blob.Name.Replace("originalfilename", "newfilename");
else
newName = blob.Name;
var targetBlob = destcontainer.GetBlobReference(newName);
targetBlob.StartCopy(blob.Uri);
}
}
while (continuationToken != null);
sourcecontainer.DeleteIfExists();
Before I added this condition inside the code above.
if (blob.Metadata["Filename"].Contains("originalfilename"))
{
blob.Metadata["Filename"] = blob.Metadata["Filename"].Replace("originalfilename", "newfilename");
targetBlob.SetMetadata();
}
I was able to change the metadata["filename"] of the file but I encountered an error while retrieving the files which before is not happening. I think the way I update the Metadata was wrong.
Below is the error message:
Exception Details: System.Collections.Generic.KeyNotFoundException:
The given key was not present in the dictionary.
Any tips on how to fix this one or a better way to change the metadata?

Based on my test, Filename is not an inbuilt metadata of blob. The inbuilt one is Name, which you cannot change. It always equals to the blob's name.
So, just as #Gaurav Mantri said in comment, did you add a Filename metadata for each blob manually?
If you have added Filename metadata for your blobs, then you can get and update it as following:
public static void Test()
{
string connString = "your_connection_string";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connString);
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference("pub");
CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference("CP4.png");
// Important, you need to fetch all attributes first!
cloudBlockBlob.FetchAttributes();
if (cloudBlockBlob.Metadata.ContainsKey("Filename"))
{
Console.WriteLine(cloudBlockBlob.Metadata["Filename"].ToString());
cloudBlockBlob.Metadata["Filename"] = "123";
cloudBlockBlob.SetMetadata();
}
else
{
cloudBlockBlob.Metadata.Add("Filename", "CP4.png");
cloudBlockBlob.SetMetadata();
}
}
In my test, the CP4.png blob's Filename metadata was updated to 123

Process more than one files at the same time in a synch way via azure functions

I have an Azure Function which extracts the name and file URLs from a blob container and then send this information to another function to process these files (unpack them and save them in Datalake)
For extartction of blobs:
string storageConnectionString = #"myconnstring";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(storageConnectionString);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("Container");
IEnumerable<IListBlobItem> blobs = new IListBlobItem[0];
foreach (IListBlobItem blobItem in container.ListBlobs())
{
if (blobItem is CloudBlobDirectory)
{
CloudBlobDirectory directory = (CloudBlobDirectory)blobItem;
blobs = directory.ListBlobs(true);
}
}
await ProcessBlobs(blobs);
and the function for process the bolbs:
public static async Task ProcessBlobs(IEnumerable<IListBlobItem> blobs)
{
var tasks = blobs.Select(currentblob =>
{
string FileUrl = currentblob.Uri.ToString();
string FileName = currentblob.Uri.Segments.Last();
//string content = "{ \"fileUri\": \""+ currentblob.Uri.ToString()+ "\" , \"fileName\": \""+ currentblob.Uri.Segments.Last()+"\"}";
var values = new Dictionary<string, string>
{
{ "fileUri", currentblob.Uri.ToString() },
{ "fileName", currentblob.Uri.Segments.Last() }
};
var content = new FormUrlEncodedContent(values);
string baseURL = #"https://<afu>.azurewebsites.net/api/process_zip_files_by_http_trigger?code=45"; ;
//string urlToInvoke = string.Format("{0}&name={1}", baseURL, FileUrl, FileName);
return RunAsync(baseURL, content);
});
await Task.WhenAll(tasks);
}
public static async Task RunAsync(string i_URL, FormUrlEncodedContent content)
{
var response = await client.PostAsync(i_URL, content);
var responseString = await response.Content.ReadAsStringAsync();
log.info(responseString);
}
the function RunAsyncprocess files asynchronous.
My question is now:
Is it generally possible to process blobs parallel, but in a synchronized way? Do you have any better and simpler idea, to implement my aim?

This is one of the best use cases for Durable Functions. Specifically, the Fan-Out/Fan-In Pattern.
You would need 4 functions in total
GetBlobList Activity Function
This is where you get the list of blobs to process. You would just get the list of blobs instead of the actual blobs instead.
ProcessBlob Activity Function
This function takes a blob path, fetches the blob and processes it.
Orchestrator Function
This is the function that calls the GetBlobList function, loops over the list of blob paths returned and calls the ProcessBlob function for each blob path.
Starter Function (Client Function)
This function simply triggers a run of the orchestration. This is usually a HTTP triggered function.
If you are new to Durable Functions, its best to walk through the quickstart doc to understand the different types of functions required.

CloudBlockBlob.UploadFromByteArrayAsync returns, but no image has been created

I have a method to upload my image to Azure blob storage. I have my account already created, and a name and key placed in my app. The behavior I'm seeing is that await UploadFromByteArrayAsync(...) returns and my method returns a URL. However, when I navigate to my azure blob storage in Microsoft Azure Storage Explorer, I can see that no blob has been created. Obviously, navigating to the URL returned by the method returns 404 also. The method has successfully created my container, so there is a definite connection with appropriate perms to my storage account, I have checked the content of the byte array and it contains actual data. Does anyone know why my image is never uploaded?
public async Task<string> UploadImage(byte[] imageByteArr)
{
// Retrieve storage account from the connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=https;AccountName=redacted;AccountKey=redacted;EndpointSuffix=core.windows.net");
// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve a reference to a previously created container.
CloudBlobContainer container = blobClient.GetContainerReference("user-images");
// Create the container if it doesn't already exist.
await container.CreateIfNotExistsAsync().ConfigureAwait(false);
var docId = Guid.NewGuid().ToString();
CloudBlockBlob blockBlob = container.GetBlockBlobReference(docId);
await blockBlob.UploadFromByteArrayAsync(imageByteArr, 0, imageByteArr.Length);
blockBlob.Properties.ContentType = "image/jpg";
await blockBlob.SetPropertiesAsync();
return blockBlob.Uri.ToString();
}

I had missed a step in the creation of the blobs tutorial I was following.
We need to call the following when creating the container in the codebehind so that we have public access to the image uploaded.
container.SetPermissions(
new BlobContainerPermissions { PublicAccess = BlobContainerPublicAccessType.Blob });

C# with azure storage getting exception when using DeleteIfExists()

I get an exception when using the method DeleteIfExists from the CloudBlockBlob class.
This is my code:
CloudBlobClient blobClient = this._storageAccount.CreateCloudBlobClient();
directory = directory.ToLower();
string containerDirectory = this.GetContainer(directory);
string relativePathWithoutContainer = this.GetRelativePathWithoutContainer(directory);
CloudBlobContainer container = blobClient.GetContainerReference(containerDirectory);
container.CreateIfNotExist();
container.SetPermissions(new BlobContainerPermissions() { PublicAccess = BlobContainerPublicAccessType.Blob });
foreach (HttpPostedFileBase file in files)
{
CloudBlockBlob blob = container.GetBlockBlobReference(string.Format("{0}/{1}", relativePathWithoutContainer, file.FileName.ToLower()));
blob.DeleteIfExists();
blob.UploadFromStream(file.InputStream,new BlobRequestOptions());
}
return true;
I get the exception at the line:
blob.DeleteIfExists();
The details of the exception are:
Server failed to authenticate the request. Make sure the value of
Authorization header is formed correctly including the signature.

Got an example from here try to add these and see if it works
// Delete the blob if it already exists, also deleting any snapshots.
BlobRequestOptions options = new BlobRequestOptions();
options.DeleteSnapshotsOption = DeleteSnapshotsOption.IncludeSnapshots;
blob.DeleteIfExists(options);