I am trying to create an controller that will allow me to save images into my database. So far I have this bit of code:
/// <summary>
/// Handles an upload
/// </summary>
/// <returns></returns>
[HttpPost]
[Route("")]
public async Task<IHttpActionResult> Upload()
{
// If the request is not of multipart content, then return a bad request
if (!Request.Content.IsMimeMultipartContent())
return BadRequest("Your form must be of type multipartcontent.");
// Get our provider
var provider = new MultipartFormDataStreamProvider(ConfigurationManager.AppSettings["UploadFolder"]);
// Upload our file
await Request.Content.ReadAsMultipartAsync(provider);
// Get our file
var file = provider.Contents.First();
var bytes = await file.ReadAsByteArrayAsync();
// Using a MemoryStream
using (var stream = new MemoryStream(bytes))
{
stream.Seek(0, SeekOrigin.Begin);
// Create the data
var data = "data:image/gif;base64," + Convert.ToBase64String(stream.ToArray());
// Return the data
return Ok(data);
}
}
But it isn't working.
When I get into the using block I get an error message:
"Error while copying content to a stream."
"Cannot access a closed file."
Does anyone know why?
The reason this is happening is the MultipartFormDataStreamProvider closes and disposes the uploaded files streams after it has written the uploaded data to the file location you provided when you passed this into the constructor: ConfigurationManager.AppSettings["UploadFolder"]
To access the files that have been uploaded you need to consult the file data on disk from the uploaded file location:
So in your example your code needs to use this:
// Read the first file from the file data collection:
var fileupload = provider.FileData.First;
// Get the temp name and path that MultipartFormDataStreamProvider used to save the file as:
var temppath = fileupload.LocalFileName;
// Now read the file's data from the temp location.
var bytes = File.ReadAllBytes(temppath);
Additionally if your using very small files you can instead use:
MultipartMemoryStreamProvider
This stores the file data in memory and should work as expected. Be warned though if you are using large files (25mb+) its wise to stream to disk first otherwise you might get out of memory exceptions as .net tries to hold the whole file in memory.
Related
I would like to take the contents of a file and rename the file while in memory to send with a different file name using an API.
The Goals:
Not alter the original file (file on disk) in any way.
Not create additional files (like a copy of the file with a new name). I'm trying to keep IO access as low as possible and do everything in memory.
Change the Name of a file object (in memory) to a different name.
Upload the file object to a WebAPI on another machine.
Have "FileA.txt" on source MachineA and have "FileB.txt" on destination MachineB.
I don't think it would matter but I have no plans to write the file back to the system (MachineA) with the new name, it will only be used to send the file object (in memory) to MachineB via a Web API.
I found a solution that uses reflection to accomplish this...
FileStream fs = new FileStream(#"C:\myfile.txt", FileMode.Open);
var myField = fs.GetType()
.GetField("_fileName", BindingFlags.Instance | BindingFlags.NonPublic)
myField.SetValue(fs, "my_new_filename.txt");
However, It's been a few years since that solution was given. Is there a better way to do this in 2021?
One other way would be defining the filename when you save it on MachineB.
You could pass this filename as a payload through the Web API and use it as the file name.
//buffer as byte[] and fileName as string would come from the request
using (FileStream fs = new FileStream(fileName, FileMode.Create))
{
fs.Write(buffer, 0, buffer.Length);
}
The best way I could come up with was using my old method from years ago. The following shows how I used it. I only do this to mask the original filename from the third-party WebAPI I'm sending it to.
// filePath: c:\test\my_secret_filename.txt
private byte[] GetBytesWithNewFileName(string filePath)
{
byte[] file = null;
using (var fs = new FileStream(filePath, FileMode.Open, FileAccess.Read))
{
// Change the name of the file in memory (does not affect the original file)
var fileNameField = fs.GetType().GetField(
"_fileName",
BindingFlags.Instance | BindingFlags.NonPublic
);
// If I leave out the next line, the file name field will have the full filePath
// string as its value in the resulting byte array. This will replace that with
// only the file name I wish to pass along "my_masked_filename.txt".
fileNameField.SetValue(fs, "my_masked_filename.txt");
// Get the filesize of the file and make sure it's compatible with
// the binaryreader object to be used
int fileSize;
try { fileSize = Convert.ToInt32(fs.Length); }
catch(OverflowException)
{ throw new Exception("The file is to big to convert using a binary reader."); }
// Get the file into a byte array
using (var br = new BinaryReader(fs)) { file = br.ReadBytes(fileSize); }
}
return file;
}
I am trying to get my MVC controller to return a PDF stored in an Azure Blob Container. The client's browser will download the PDF fine, but when they open it they will see "Error Failed to load PDF document." when opening it in Chrome (though the file does not open in other browsers either).
I was able to download the file on my machine and open it fine doing the following:
public static void DownloadFile()
{
CloudBlockBlob cloudBlockBlob =
CloudBlobContainer.GetBlockBlobReference("document.pdf");
AsyncCallback callback = new AsyncCallback(DownloadComplete);
cloudBlockBlob.BeginDownloadToFile(#"path\document.pdf", FileMode.Create,
callback, new object());
}
However I would rather not create any temp files on the server and return those. I would like to just create them in memory and return that to the client.
My Controller Code:
public async Task<FileStreamResult> Test()
{
MemoryStream stream = await BlobStorageUtils.DownloadFile();
return File(stream, "application/pdf", "document.pdf");
}
The code to retrieve the file from the Blob Container
public static async Task<MemoryStream> DownloadFile()
{
CloudBlockBlob cloudBlockBlob =
CloudBlobContainer.GetBlockBlobReference("document.pdf");
MemoryStream stream = new MemoryStream();
await cloudBlockBlob.DownloadToStreamAsync(stream);
return stream;
}
As I mentioned, my file downloads in the browser fine, but I receive the error when trying to open the file.
Eventually I would like this to work with any type of document, not just PDFs.
Edit: I should note that I have tried this with image files as well (PNG specifically) and had a similar issue where the image was corrupted or couldn't be opened. The error I received then was "It looks like we don't support this file format".
Update: See my solution below for how I got this to work.
The solution for this came from this link: Open a pdf file(using bytes) stored in Azure Blob storage
I ended up just dumping the byte stream from Azure into the response's output stream. However you need to make sure that the content type of the response is set to "application/pdf". My code ended up like this:
Controller Code:
public async Task<ActionResult> Test()
{
Response.Buffer = true;
Response.Clear();
Response.ContentType = "application/pdf";
await BlobStorageUtils.DownloadFile(Response.OutputStream);
return new EmptyResult();
}
Code to retrieve file from Blob Container
public static async Task DownloadFile(Stream outputStream)
{
CloudBlockBlob cloudBlockBlob =
CloudBlobContainer.GetBlockBlobReference("document.pdf");
await cloudBlockBlob.DownloadToStreamAsync(outputStream);
}
I have a function I use for aggregating streams from a zip archive.
private void ExtractMiscellaneousFiles()
{
foreach (var miscellaneousFileName in _fileData.MiscellaneousFileNames)
{
var fileEntry = _archive.GetEntry(miscellaneousFileName);
if (fileEntry == null)
{
throw new ZipArchiveMissingFileException("Couldn't find " + miscellaneousFileName);
}
var stream = fileEntry.Open();
OtherFileStreams.Add(miscellaneousFileName, (DeflateStream) stream);
}
}
This works well in most cases. However, if I have a zip within a zip, I get an excpetion on casting the stream to a DeflateStream:
System.InvalidCastException: Unable to cast object of type 'System.IO.Compression.SubReadStream' to type 'System.IO.Compression.DeflateStream'.
I am unable to find Microsoft documentation for a SubReadStream. I would like my zip within a zip as a DeflateStream. Is this possible? If so how?
UPDATE
Still no success. I attempted #Sunshine's suggestion of copying the stream using the following code:
private void ExtractMiscellaneousFiles()
{
_logger.Log("Extracting misc files...");
foreach (var miscellaneousFileName in _fileData.MiscellaneousFileNames)
{
_logger.Log($"Opening misc file stream for {miscellaneousFileName}");
var fileEntry = _archive.GetEntry(miscellaneousFileName);
if (fileEntry == null)
{
throw new ZipArchiveMissingFileException("Couldn't find " + miscellaneousFileName);
}
var openStream = fileEntry.Open();
var deflateStream = openStream;
if (!(deflateStream is DeflateStream))
{
var memoryStream = new MemoryStream();
deflateStream.CopyTo(memoryStream);
memoryStream.Position = 0;
deflateStream = new DeflateStream(memoryStream, CompressionLevel.NoCompression, true);
}
OtherFileStreams.Add(miscellaneousFileName, (DeflateStream)deflateStream);
}
}
But I get a
System.NotSupportedException: Stream does not support reading.
I inspected deflateStream.CanRead and it is true.
I've discovered this happens not just on zips, but on files that are in the zip but are not compressed (because too small, for example). Surely there's a way to deal with this; surely someone has encountered this before. I'm opening a bounty on this question.
Here's the .NET source for SubReadStream, thanks to #Quantic.
The return type of ZipArchiveEntry.Open() is Stream. An abstract type, in practice it can be a DeflateStream (you'd be happy), a SubReadStream (boo) or a WrappedStream (boo). Woe be you if they decide to improve the class some day and use a ZopfliStream (boo). The workaround is not good, you are trying to deflate data that is not compressed (boo).
Too many boos.
Only good solution is to change the type of your OtherFileStreams member. We can't see it, smells like a List<DeflateStream>. It needs to be a List<Stream>.
So it looks like the when storing a zip file inside another zip it doesn't deflate the zip but rather just inlines the content of the zip with the rest of the files with some information that these entries are part of a sub zip file. Which makes sense because applying compression to something that is already compressed is a waste of time.
This zip file is marked as CompressionMethodValues.Stored in the archive, which causes .NET to just return the original stream it read instead to wrapping it in a DeflateStream.
Source here: https://github.com/dotnet/corefx/blob/master/src/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs#L670
You could pass the stream into a ZipArchive, if it's not a DeflateStream (if you are interested in the file inside)
var stream = entry.Open();
if (!(stream is DeflateStream))
{
var subArchive = new ZipArchive(stream);
}
Or you can copy the stream to a FileStream (if you want to save it to disk)
var stream = entry.Open();
if (!(stream is DeflateStream))
{
var fs = File.Create(Path.GetTempFileName());
stream.CopyTo(fs);
fs.Close();
}
Or copy to any stream you are interested in using.
Note: This is also how .NET 4.6 behaves
I'm testing how to upload to AWS using SDK with a sample .txt file from a web app. The file uploads to the Bucket, but the downloaded file from the bucket is just an empty Notepad document without the text from the original uploaded file. I'm new to working with streams, so I'm not sure what could be wrong here. Does anyone see why the data wouldn't be sent in the transfer request? Thanks in advance!
using (var client = new AmazonS3Client(Amazon.RegionEndpoint.USWest1))
{
//Save File to Bucket
using (FileStream txtFileStream = (FileStream)UploadedHttpFileBase.InputStream)
{
try
{
TransferUtility fileTransferUtility = new TransferUtility();
fileTransferUtility.Upload(txtFileStream, bucketLocation,
UploadedHttpFileBase.FileName);
}
catch (Exception e)
{
e.Message.ToString();
}
}
}
EDIT:
Both TransferUtility and PutObjectRequest/PutObjectResponse/AmazonS3Client.PutObject saved a blank text file. Then, after having some trouble instantiating a new FileStream, a MemoryStream used after resetting the starting position to zero still saved a blank text file. Any ideas?
New Code:
using (var client = new AmazonS3Client(Amazon.RegionEndpoint.USWest1))
{
Stream saveableStream = new MemoryStream();
using (Stream source = (Stream)UploadedHttpFileBase.InputStream)
{
source.Position = 0;
source.CopyTo(saveableStream);
}
//Save File to Bucket
try
{
PutObjectRequest request = new PutObjectRequest
{
BucketName = bucketLocation,
Key = UploadedHttpFileBase.FileName,
InputStream = saveableStream
};
PutObjectResponse response = client.PutObject(request);
}
catch (Exception e)
{
e.Message.ToString();
}
}
Most probably that TransferUtility doesn't work good with temporary upload files. Try to copy your input stream somewhere (e.g. into other not-so-temporary file, or even MemoryStream if you're sure it would not give you OutOfMemory at some point). Another thing is to get rid of TransferUtility and use low-level AmazonS3Client.PutObject with which you get finer control over Stream lifetime (do not forget that you'll need to implement some retrying as S3 API is prone to returning random temporary errors).
The answer had something to do with nesting, which is still a little beyond my understanding, and not because the code posted here was inherently wrong. This code came after an initial StreamReader which checked the first line of the text file to determine whether or not to save the file. After moving the code out from the while loop doing the ReadLines, the upload worked. Everything works as it's supposed to now that the validation is reorganized so that there's no need for the nested Stream or MemoryStream.
I have the following code that uploads an image to Azure blob storage. I would like to encrypt the image data before uploading to the blob. I already have a helper class for encrypting and decrypting that I can use by calling AESEncryption.Encrypt("plainText", "key", salt");
I'm just trying to figure out how tom integrate my encryption method into the code. Also, I'm guessing that once it's encrypted instead of calling blob.UploadFromFile() I will be calling blob.UploadFromByteArray().
public override Task ExecutePostProcessingAsync()
{
try
{
// Upload the files to azure blob storage and remove them from local disk
foreach (var fileData in this.FileData)
{
var filename = BuildFilename(Path.GetExtension(fileData.Headers.ContentDisposition.FileName.Trim('"')));
// Retrieve reference to a blob
var blob = _container.GetBlockBlobReference(filename);
blob.Properties.ContentType = fileData.Headers.ContentType.MediaType;
blob.UploadFromFile(fileData.LocalFileName, FileMode.Open);
File.Delete(fileData.LocalFileName);
Files.Add(new FileDetails
{
ContentType = blob.Properties.ContentType,
Name = blob.Name,
Size = blob.Properties.Length,
Location = blob.Uri.AbsoluteUri
});
}
}
catch (Exception ex)
{
throw ex;
}
return base.ExecutePostProcessingAsync();
}
As I see it, you could do it 3 ways:
Encrypt the file beforehand and then upload that encrypted file.
As you mentioned, you could read the file in byte array and then encrypt that byte array and upload it using UploadFromByteArray method.
Similar to #2 but instead of uploading byte array, you could rely on streams and upload using UploadFromStream method.