Below is the method where I am reading a csv file from an azure blob container and later calling a function to copy the contents in a tabular storage.
Now my requirement has bit changed and now .csv file will be compressed to .gz file in the blob container. I would like to know, how can I modify the below code so that I can read .gz file , decompress it and then pass the contents as I am already passing
public async Task<string> ReadStream(string BlobcontainerName, string fileName, string connectionString)
{
var contents = await DownloadBlob(BlobcontainerName, fileName, connectionString);
string data = Encoding.UTF8.GetString(contents.ToArray());
return data;
}
foreach (var files in recFiles)// recFiles are list of CSV files
{
string data = await ReadStream(containerName, files.Name, connectionString);}
public async Task<MemoryStream> DownloadBlob(string containerName, string fileName, string connectionString)
{
MemoryStream memoryStream = new MemoryStream();
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);
CloudBlobClient serviceClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = serviceClient.GetContainerReference(containerName);
CloudBlockBlob blob = container.GetBlockBlobReference(fileName);
if (blob.Exists())
{
using (memoryStream = new MemoryStream())
{
await blob.DownloadToStreamAsync(memoryStream);
}
}
return memoryStream;
}
Try this code:
public async Task<MemoryStream> DownloadBlob(string containerName, string fileName, string connectionString)
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);
CloudBlobClient serviceClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = serviceClient.GetContainerReference(containerName);
CloudBlockBlob blob = container.GetBlockBlobReference(fileName);
var memoryStream = new MemoryStream();
if (blob.Exists())
{
using (var gZipStream = new GZipStream(memoryStream, CompressionMode.Decompress))
{
await blob.DownloadToStreamAsync(gZipStream);
}
}
return memoryStream;
}
Related
I am trying to update a JSON file which is located in azure blob storage. when the program does the put call the saved file looks like this:
Zona de especial protecci\u00F3n
the accents and other characters are the matter, but that only happens when I download the file from the azure UI interface if I do a get from postman, that does not happen. this is my code:
SemanticDictionaryContent semanticDictionaryContent = new SemanticDictionaryContent()
{
Name = entity.Id + JSON_EXTENSION,
Content = BinaryData.FromObjectAsJson(entity)
};
Create a storage account in azure.
Create a container in azure.
Uploaded a Json file to a container using the below code.
I have used the below approach in Uploading / downloading / editing the Json file.
public static bool Upload()
{
try
{
var containerName = "mycontainer";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(ConnectionSting);
CloudBlobClient client = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = client.GetContainerReference(containerName);
var isCreated = container.CreateIfNotExists();
container.SetPermissionsAsync(new BlobContainerPermissions { PublicAccess = BlobContainerPublicAccessType.Blob });
using (FileStream fileStream = File.Open(#"C:\Tools\local.settings.json", FileMode.Open))
{
using (MemoryStream memoryStream = new MemoryStream())
{
memoryStream.Position = 0;
fileStream.CopyTo(memoryStream);
var fileName = "local.settings.json";
CloudBlockBlob blob = container.GetBlockBlobReference(fileName);
string mimeType = "application/unknown";
string ext = (fileName.Contains(".")) ? System.IO.Path.GetExtension(fileName).ToLower() : "." + fileName;
Microsoft.Win32.RegistryKey regKey = Microsoft.Win32.Registry.ClassesRoot.OpenSubKey(ext);
if (regKey != null && regKey.GetValue("Content Type") != null) mimeType = regKey.GetValue("Content Type").ToString();
memoryStream.ToArray();
memoryStream.Seek(0, SeekOrigin.Begin);
blob.Properties.ContentType = mimeType;
blob.UploadFromStream(memoryStream);
}
}
return true;
}
catch (Exception ex)
{
throw;
}
}
Uploaded Json file
Updated the Json file in azure and uploaded it using the below code.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(ConnectionSting);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
CloudBlockBlob jsonBlob = container.GetBlockBlobReference("local.settings.json");
string jsonText = jsonBlob.DownloadText();
dynamic jsonData = JsonConvert.DeserializeObject(jsonText);
jsonData.property1 = "Property1";
jsonData.property2 = "Property2";
jsonBlob.UploadText(JsonConvert.SerializeObject(jsonData));
And downloaded the Json file from azure manually and do not find any special characters.
Context: Encrypt and Decrypt an audio file (.wav) in Azure Storage.
Issue: inputStream must be seek-able (when encrypting) "await pgp.EncryptStreamAsync(sourceStream, outputStream);"
I'm not a C# Developer :)
Thank you for your help,
Here is the code i'm using:
static async Task Main()
{
string connectionString = Environment.GetEnvironmentVariable("AZURE_STORAGE_CONNECTION_STRING");
////Create a unique name for the container
string containerName = "audioinput";
string filename = "abc.wav";
//// Create a BlobServiceClient object which will be used to create a container client
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
BlobContainerClient sourcecontainer = blobServiceClient.GetBlobContainerClient(containerName);
BlobClient blobClient = sourcecontainer.GetBlobClient(filename);
if (sourcecontainer.Exists())
{
var sourceStream = new MemoryStream();
//Download blob to MemoryStream
await blobClient.DownloadToAsync(sourceStream);
sourceStream.Position = 0;
//OutputStream
await using var outputStream = new MemoryStream();
//Get encryptionkeys
EncryptionKeys encryptionKeys;
using (Stream publicKeyStream = new FileStream(#"...\public.asc", FileMode.Open))
encryptionKeys = new EncryptionKeys(publicKeyStream);
PGP pgp = new PGP(encryptionKeys);
await pgp.EncryptStreamAsync(sourceStream, outputStream);
}
else
{
Console.WriteLine("container doesn't exist");
}
}
I am uploading an excel file to azure storage container. When the file gets uploaded, and I try to download it back from the portal and open it, the open fails because the format of the file and extension do not match. Also, there is no size in the size column corresponding to the file. I cannot spot the error. The code is in asp.net core 3.1 with c#.
Here is my code
CloudStorageAccount cloudStorageAccount = CloudStorageAccount.Parse(connectionString); // to the azure account
CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();
CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference(containerName); // container in which the file will be uploaded
blob = cloudBlobContainer.GetBlockBlobReference(f); // f is file name
await blob.UploadFromStreamAsync(s); // this is a memory stream
blob.Properties.ContentType = fileType;
According to my test, we can use the following code to upload excel file to Azure blob storage
Install SDK
Install-Package Microsoft.Azure.Storage.Blob -Version 11.1.1
Install-Package Microsoft.AspNetCore.StaticFiles -Version 2.2.0
My excel file(.xlsx)
Code
static async Task Main(string[] args)
{
var filepath = #"D:\test.xlsx";
var storageAccount = CloudStorageAccount.Parse("<connection string>");
var cloudBlobClient = storageAccount.CreateCloudBlobClient();
var cloudBlobContainer = cloudBlobClient.GetContainerReference("test1");
var blob = cloudBlobContainer.GetBlockBlobReference(Path.GetFileName(filepath));
blob.Properties.ContentType = Get(Path.GetFileName(filepath));
using (var stream = File.OpenRead(filepath))
{
await blob.UploadFromStreamAsync(stream);
}
//download file
filepath = #"D:\test\" + blob.Name;
using (var stream = File.OpenWrite(filepath))
{
await blob.DownloadToStreamAsync(stream);
}
}
// get the file content type
static string Get(string fileName)
{
var provider = new FileExtensionContentTypeProvider();
string contentType;
if (!provider.TryGetContentType(fileName, out contentType))
{
contentType = "application/octet-stream";
}
return contentType;
}
I'm using Azure Blob Storage to save some files. I'm having issues downloading this stream & am not sure why. I don't get any errors, just an empty stream. I've verified that the file exists in the container, and even ran code to list all files in the container. Any help would be greatly appreciated.
private async Task<MemoryStream> GetMemoryStreamAsync(string fileName)
{
var storageAccountName = Environment.GetEnvironmentVariable("storage_account_name");
var storageAccountKey = Environment.GetEnvironmentVariable("storage_access_key");
var storageContainerName = Environment.GetEnvironmentVariable("storage_container_name");
CloudStorageAccount storageAccount = new CloudStorageAccount(new Microsoft.WindowsAzure.Storage.Auth.StorageCredentials(storageAccountName, storageAccountKey), true);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(storageContainerName);
CloudBlockBlob blockBlob = container.GetBlockBlobReference(fileName);
MemoryStream stream = new MemoryStream();
await blockBlob.DownloadToStreamAsync(stream);
return stream;
}
You'll need to set the position to zero before returning the stream so that the consumer of the stream reads it from the beginning.
MemoryStream stream = new MemoryStream();
await blockBlob.DownloadToStreamAsync(stream);
stream.Position = 0;
return stream;
I have zipped a file and upload to a blob in azure , but I am unable to download it after unzipping it. I have tried the below code but it is throwing error:
public FileStream Download(string strPath)
{
Stream fs = GetFile(strPath);
using (ZipArchive zip = new ZipArchive(fs))
{
var entry = zip.Entries.First();
var memoryStream = entry.Open();
string filename = "Report_" + GetUploadTime();
using (var fileStream = new FileStream(filename,
FileMode.CreateNew,
FileAccess.ReadWrite))
{
memoryStream.CopyTo(fileStream); // fileStream is not populated
return fileStream;
}
}
}
System.UnauthorizedAccessException occurred in mscorlib.dll but was not handled in user code, I do not want to create any folder or keep it anywhere just unzip and download how to do it.
public Stream GetFile(string strPath)
{
try
{
var filename = Path.GetFileName(strPath);
string account = ConfigurationManager.AppSettings["BlobContainer"];
string key = ConfigurationManager.AppSettings["BlobKey"];
string connectionString =
string.Format("DefaultEndpointsProtocol=https;AccountName={0};AccountKey={1}",
account, key);
CloudStorageAccount storageAccount =
CloudStorageAccount.Parse(connectionString);
var blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("reportportalblob");
CloudBlockBlob blob = container.GetBlockBlobReference(filename);
Stream blobStream = blob.OpenRead();
return blobStream;
}
catch (Exception)
{
// download failed
// handle exception
throw;
}
}
I have search for some code but I am not getting anything, please help.
We could get the unzip stream with following code.
public MemoryStream GetFile(string strPath)
{
try
{
var filename = Path.GetFileName(strPath);
string account = ConfigurationManager.AppSettings["accountName"];
string key = ConfigurationManager.AppSettings["accountKey"];
string containerName = "test";
string connectionString =$"DefaultEndpointsProtocol=https;AccountName={account};AccountKey={key}";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionString);
var blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
CloudBlockBlob blob = container.GetBlockBlobReference(filename);
MemoryStream memory = new MemoryStream();
blob.DownloadToStream(memory);
var zipArchive = new ZipArchive(memory, ZipArchiveMode.Read, true);
var entry = zipArchive.Entries.First();
if (entry != null)
{
var stream = entry.Open();
memory = new MemoryStream();
stream.CopyTo(memory);
memory.Position = 0;
return memory;
}
return null;
}
catch (Exception)
{
// download failed
// handle exception
throw;
}
}
If we want to down to file, then we could use File.WriteAllBytes(strPath,memory.ToArray()); or fileStream.Write(memory.ToArray(),0,memory.ToArray().Length-1);
public void Download(string strPath)
{
var memory = GetFile(strPath);
string filename = "Report_" + DateTime.Now;
var fileStream = new FileStream(filename,
FileMode.CreateNew,
FileAccess.ReadWrite);
// File.WriteAllBytes(strPath,memory.ToArray());
fileStream.Write(memory.ToArray(),0,memory.ToArray().Length-1);
}