Alright, so I would like to create a thumbnail storage with a couple of png image files in it.
I created a directory on the server side. Now in my controller I would like to process requests for images.
Currently the code of my controller looks like this:
public class ThumbnailController : ApiController
{
[Route("api/v1/thumbnail")]
[HttpGet]
public HttpResponseMessage GetThumbnail(string id)
{
var result = new HttpResponseMessage(HttpStatusCode.OK);
string filePath = $#"D:\server_data\Images\Thumbnails\{id}.png";
if (string.IsNullOrEmpty(filePath) || !File.Exists(filePath)) return Request.CreateErrorResponse(HttpStatusCode.NotFound, "Invalid file");
var image = Image.FromFile(filePath);
using (var memoryStream = new MemoryStream())
{
image.Save(memoryStream, ImageFormat.Jpeg);
result.Content = new ByteArrayContent(memoryStream.ToArray());
}
result.Content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
return result;
}
}
Now this one has a security issue: if I call something like http://my.address.com/api/v1/thumbnail?id=../securefile.png I would essentially get the securefile.png image file located in the parent directory. Similarly I could download any png image on the hard drive, which is obviously not desired.
For a reason the thumbnails directory must be detached from the server root, so I guess I can not use something like string filePath = HostingEnvironment.MapPath($"~/Images/{id}.png");
To avoid the problem I could trim starting dots or check the incoming id for special characters, but it does not feel like the right way to fight it. I wonder if better practices exist.
Trimming starting dots is not enough, consider an input similar to foo/bar/../../../outside.png.
You should validate the id parameter to only contain say letters, or even better, only numbers. You can achieve complex validation through validation attributes or model validation, both are basically the same. The point is that a user should not be able to enter any special characters, which would effectively mitigate the path injection threat.
Related
I try to save my images on my server, but I can't let my server save file and virus because of that I want to get image content as pixels of rgb and after that I create image by myself.
I can't use bitmap (or other type in C# like bitmapImage, ... etc) and I don't know how I can do this with sixlabors.ImageSharp.
I have some code that I tried but I can't implement the exact logic that I want (code shown here):
[HttpPost("[action]")]
public async Task<IActionResult> Get([FromForm] ImageFormat file)
{
await using var memoryStream = new MemoryStream();
await file.File.CopyToAsync(memoryStream);
IImageFormat format;
using (var image = Image.Load(memoryStream.ToArray(), out format))
{
using (var output = new MemoryStream())
{
image.Save(output, format);
var responseType = format.Name.ToLower();
return File(output.ToArray(), "application/octet-stream", file.File.FileName);
}
}
return null;
}
Can anybody help me with this problem?
i don't see a reason to convert image into image: there are several format zip-algorythms etc.wich you have to support in that case. example jpg is not bitmap, there is convertion issue - quality of image becomes less each conversion time. Image itself is not executable - it can be used only as container for virus body, can't harm your OSystem itself, another executable part should works somewhere.
But even if you would like to store images on disk, in other format - you can convert image to base64 text (one line of code, like example) - it less harmful and well known way to work with any file type. you can zip image by cszip, you can change file name and extension to hide file type.
I don't see a reasson to convert one image to another for this scenario/task.
I am saving zip files to an AWS S3 bucket. I am now trying to create a C# .NET API that will allow me to download a specified key from the bucket and save it to a HttpResponseMessage in the Content key.
I've referred to the following question to set up my response for zip files: How to send a zip file from Web API 2 HttpGet
I have modified the code in the previous question so that it instead reads from a TransferUtility stream.
Problem is I am coming into an error when trying to extract or view the file that looks like the following:
The response I am getting back from the API looks like:
The relevant code looks like:
[HttpGet, Route("GetFileFromS3Bucket")]
public HttpResponseMessage GetFileFromS3Bucket(string keyName)
{
HttpResponseMessage response = new HttpResponseMessage();
string bucketName = "myBucket";
RegionEndpoint bucketRegion = RegionEndpoint.ARegion;
IAmazonS3 s3Client;
s3Client = new AmazonS3Client(bucketRegion);
try
{
var fileTransferUtility = new TransferUtility(s3Client);
var stream = fileTransferUtility.OpenStream(bucketName, keyName);
response.Content = new StreamContent(stream);
response.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment");
response.Content.Headers.ContentDisposition.FileName = keyName + ".zip";
response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/zip");
response.StatusCode = HttpStatusCode.OK;
}
catch (Exception e)
{
response.Content = new StringContent("Something went wrong, error: " + e.Message);
response.StatusCode = HttpStatusCode.InternalServerError;
}
return response;
}
Results of troubleshooting:
The file from the Web API comes out with nearly double the expected size based on what is in S3. This is consistent across different files
Changing the bucket to be publicly accessible did not help (setting since reverted to not allowing public access)
Changing the file type to XML did not display a nicely formatted error (there was a suggestion that you may receive an XML response if an error was provided from S3)
Saving the S3 stream to a file and then saving directly to a file resulted in the correct file size. Seems safe to say the stream from S3 is not the problem
It appears that there ia a problem with the way the HTTPResponseMessage is handling the zip file. I'm unsure of whether it is actually on the server side, or whether it is on the client to parse the data and Swagger is simply incapable of doing that. Any help would be greatly appreciated.
Update 1
I do not believe this string is Base64 encoded as the result I got from converting the stream to a string is the following:
I've updated the code sample with the two lines showing the conversion from a stream to string.
Update 2
I've confirmed the issue is with how the response is handling the stream, or something in the response itself. Downloading the file stream from S3 and saving to a new file on the local computer resulted in a valid file that opened as expected.
Update 3
Link to GDrive folder with testing files: https://drive.google.com/drive/folders/1q_N3NTHz5E_nebtBQJHor3HfqUZWhGgd?usp=sharing
I unfortunately can't provide access to the original file as it contains sensitive data. The provided files are still causing the same problem however.
Interesting to note that the test file came out looking like:
The underscores on either side of the filename are quite strange.
I am running the following relevant packages:
Update 4
I've found the following UTF8 references in various files:
File: configuration91.svcinfo
I could not find anything that said anything about 'responseEncoding' anywhere in the project.
I am going to throw an answer up, because what's happening to you is unorthodox. I use S3 for many things and have done what you are doing with no problems in the past. To ensure that I am mimicking what you are doing, I duplicated your code:
[HttpGet, Route("GetFileFromS3Bucket/{keyName}")]
public HttpResponseMessage GetFileFromS3Bucket(string keyName)
{
string bucketName = "testzipfilesagain";
string awsAccessKey = "AKIAJ********A3QHOUA";
string awsSecretKey = "IYUJ9Gy2wFCQ************dCq5suFS";
IAmazonS3 client = new AmazonS3Client(awsAccessKey, awsSecretKey, RegionEndpoint.USEast1);
var fileTransferUtility = new TransferUtility(client);
var stream = fileTransferUtility.OpenStream(bucketName, "md5.zip");
var resp = new HttpResponseMessage();
resp.Content = new StreamContent(stream);
resp.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment");
resp.Content.Headers.ContentDisposition.FileName = keyName + ".zip";
resp.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/zip");
resp.StatusCode = HttpStatusCode.OK;
return resp;
}
These are the packages I have installed:
<ItemGroup>
<PackageReference Include="AWSSDK.S3" Version="3.3.111.37" />
<PackageReference Include="Microsoft.AspNetCore.Mvc.WebApiCompatShim" Version="2.2.0" />
<PackageReference Include="Swashbuckle.AspNetCore" Version="5.5.1" />
</ItemGroup>
Everything runs perfectly well.
Trying to troubleshoot your code is going to be fruitless because it works perfectly fine, but there is something wrong with your environment.
So this isn't an answer to your question, but a answer to how you can try to solve the issue at hand and get past this.
Make sure your nuget packages are up to date
Do you have any middleware injected in your pipeline? If so, what?
Post your startup.cs -- maybe something is out of order in your Configure routine.
Could you start a brand new project and try your code in that?
Can you try a small 5KB zip file and post the original and the corrupt so we can look?
I would love to get to the bottom of this as I really like to solve these types of problems.
EDIT 1
So I looked at the zip files and they have been run through a UTF8 encoding process. So, if you take your original zip file, and run this code on it:
var goodBytes = File.ReadAllBytes("Some test to upload to S3.zip");
var badBytes = File.ReadAllBytes("_Some test to upload to S3.zip.zip_");
File.WriteAllText("Some test to upload to S3.zip.utf8", Encoding.UTF8.GetString(goodBytes));
var utf8EncodedGoodBytes = File.ReadAllBytes("Some test to upload to S3.zip.utf8");
var identical = badBytes.SequenceEqual(utf8EncodedGoodBytes);
It the results are:
I am going to do some research and figure out what could be causing your stream to become UTF-8 encoded. Is there anything in your config that looks like this? Can you search your entire solution for anything that resembles "utf" or "utf8" or "utf-8"?
I am working on an Azure Function that should read two .csv files stored as Blobs in Azure Blob Storage and return a third, new, blob with lines that are different between both input blobs.
For example:
csv1:
12,aaa,bbb,ccc,ddd,eee,fff
13,aaa,bbb,ccc,ddd,eee,fff
csv2:
12,bbb,aaa,ccc,ddd,eee,fff
13,aaa,bbb,ccc,ddd,eee,fff
14,aaa,bbb,ccc,ddd,eee,fff
Output csv:
12,bbb,aaa,ccc,ddd,eee,fff
14,aaa,bbb,ccc,ddd,eee,fff
So far I have been able to read the Blob files but I have been unsuccessful in comparing them directly. I did manage to get it to work by reading in the Blobs and loading them into two different Datatables and perform the comparison between them. However, that method is far too slow and I am pretty sure there is a far more efficient way of handling it.
(Being more at home with Powershell, the Compare-Object function is kinda the exact thing I would love to create).
I can load in the Blobs using either the .DownloadText() or the .DownloadToStream() methods so getting the Blob contents is no problem.
blobA = container.GetBlockBlobReference("FileA");
blobB = container.GetBlockBlobReference("FileB");
string blobContentsA = blobA.DownloadText();
string blobContentsB = blobB.DownloadText();
or
string textA;
using (var memoryStream = new MemoryStream())
{
blobA.DownloadToStream(memoryStream);
textA = System.Text.Encoding.UTF8.GetString(memoryStream.ToArray());
}
string textB;
using (var memoryStream = new MemoryStream())
{
blobA.DownloadToStream(memoryStream);
textB = System.Text.Encoding.UTF8.GetString(memoryStream.ToArray());
}
I tried the code below but then I get a "cannot convert from 'System.Collections.Generic.IEnumerable' to 'string'" message so I guess I have to do something there, but what I have no clue to be honest.
I'm struggling a little with images on the Azure platform under dotnet core and I'm hoping someone can make a sensible suggestion.
Simple enough premise: user uploads image, saved in a database as base64 (about to move to Azure storage blob, but that's irrelevant to this). Later on, site owner comes along and clicks a button to get all these images down in a ZIP file. In the old days of .net framework this was easy. Now I seem to be hitting a set of 'yes, but' comments.
Yes, there's system.drawing.image but you can't use that because it's not in dotnet core (until recently).
Yes, you can use CoreCompat but it doesn't work on Azure because in Web Applications there's no support for GDI+.
Yes, even if I could, I'm developing on a Mac so it won't work locally as far as I can see.
I have tried beta4 of ImageSharp without a lot of success. It's random - sometimes it works, sometimes it just throws OutOfMemoryException.
I have tried SkiaSharp but similar results; sometimes it works, sometimes it spits out random error messages.
I'm not doing anything fancy in terms of processing, no resizing or anything. It should be a case of load file to byte array from Convert.FromBase64String, create Zip file entry, ultimately spit out zip file. The ZIP portion is fine, but I need something decent that can do the image work.
Here's a bit of code:
if(!string.IsNullOrEmpty(del.Headshot))
{
var output=SKImage.FromBitmap(SKBitmap.Decode(Convert.FromBase64String(del.Headshot)));
MemoryStream savedFile=new MemoryStream();
output.Encode(SKEncodedImageFormat.Jpeg, 100).SaveTo(savedFile);
string name=$"{del.FirstName} {del.LastName} - {del.Company}".Trim(Path.GetInvalidFileNameChars()) + "_Headshot.jpg";
ZipArchiveEntry entry=zip.CreateEntry(name);
using(Stream entryStream=entry.Open())
{
entryStream.Write(savedFile.ToArray(), 0, Convert.ToInt32(savedFile.Length));
}
output.Dispose();
savedFile.Dispose();
}
Can anyone give me a sensible suggestion for a library that can handle images, cross-platform and on Azure, before I pull out what little hair remains!
Thanks
EDIT: The first answer is technically correct, I don't need anything else. However, I might have been a bit wrong when I said I wasn't doing any image manipulation. Because it's all base64 without a filename being stored anywhere, I've no idea what sort of file it is. I'm therefore saving each one as JPEG to ensure that I can always output that file type and extension. Users I guess could be uploading JPG / PNG or even GIF.
Technically you do not need any of those other imaging (unless you are doing more that just zipping the content). Convert the base64 to byte array and pass that to the zip file. No need to save to disk just to read it back again for zipping.
//...
if(!string.IsNullOrEmpty(del.Headshot)) {
var imageBytes = Convert.FromBase64String(del.Headshot);
string name = $"{del.FirstName} {del.LastName} - {del.Company}".Trim(Path.GetInvalidFileNameChars()) + "_Headshot.jpg";
ZipArchiveEntry entry = zip.CreateEntry(name);
using(Stream entryStream = entry.Open()) {
entryStream.Write(imageBytes, 0, imageBytes.Length));
}
}
//...
Also using a minor hack for known image types when converted to base64
public static class ImagesUtility {
static IDictionary<string, string> mimeMap =
new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
{
{ "IVBOR", "png" },
{ "/9J/4", "jpg" },
//...add others
};
/// <summary>
/// Extract image file extension from base64 string.
/// </summary>
/// <param name="base64String">base64 string.</param>
/// <returns>file extension from string.</returns>
public static string GetFileExtension(string base64String) {
var data = base64String.Substring(0, 5);
var extension = mimeMap[data.ToUpper()];
return extension;
}
}
You could try to determine the file extension from its prefix
if(!string.IsNullOrEmpty(del.Headshot)) {
var imageBytes = Convert.FromBase64String(del.Headshot);
var ext = ImagesUtility.GetFileExtension(del.Headshot) ?? "jpg";
string name = $"{del.FirstName} {del.LastName} - {del.Company}".Trim(Path.GetInvalidFileNameChars()) + $"_Headshot.{ext}";
ZipArchiveEntry entry = zip.CreateEntry(name);
using(Stream entryStream = entry.Open()) {
entryStream.Write(imageBytes, 0, imageBytes.Length));
}
}
Now ideally, if you are able to control the type of image uploaded, then you should also validate and do what ever image processing when the data is being saved along with any needed metadata (ie content type). That way when extracting it from storage, you can be confident that it is the correct type and size. That saves you having to worry about that later on.
Aspose.Drawing and Aspose.Imaging can handle images and cross-platform running on .NET Core (I'm one of the developers).
I've implemented a file upload in ASP.Net MVC. I've created a view model that received the uploaded file as an HttpPostedFileWrapper and within the controller action I can save the file to disk.
However, I want to perform the actual save in service method which is in a class library that doesn't implement System.Web. I can't therefore pass the HttpPostedFileWrapper object to the service method.
Does anyone know how to achieve this, either by receiving the file as a different object or converting it to something else prior to passing it. The only way I can think of is to read the content of the file into a MemoryStream, and pass this along with the other parameters such as filename individually, but just wondered if there was a better way?
Thanks
The best approach would probably be retrieving the image data (as a byte[]) and the name of the image (as a string) and passing those along to your service, similar to the approach you mentioned :
public void UploadFile(HttpPostedFileWrapper file)
{
// Ensure a file is present
if(file != null)
{
// Store the file data
byte[] data = null;
// Read the file data into an array
using (var reader = new BinaryReader(file.InputStream))
{
data = reader.ReadBytes(file.ContentLength);
}
// Call your service here, passing along the data and file name
UploadFileViaService(file.FileName, data);
}
}
Since a byte[] and a string are very basic primatives, you should have no problem passing them to another service. A Stream is likely to work as well, but they can be prone to issues like being closed, whereas a byte[] will already have all of your content.