Develop Reference

ASP.NET Core 2 file download with Emojis in the filename creates malformed filename

For file downloading from an ASP.NET Core 2 application I’ve been using the PhysicalFileResult (an implementation of IActionResult) to return files from a controller GET action. For filenames with 'typical' characters everything works fine. This includes the accented and Kanji character sets that I've tested thus far.
However, if the filename contains an Emoji (something allowed by the Windows file system) the resulting filename is malformed or miss-encoded.
For example I'd expect the following code to deliver a file named 😀.png, instead it delivers a file named: ��.png.
public async Task<IActionResult> GetByIdAsync([FromRoute] long id)
{
return PhysicalFile("c:\\file.png", "image/png", "😀.png");
}
Is there a particular way that filenames containing Emojis need to be encoded in order for the filename to be preserved, or is this an issue with PhysicalFileResult?

The reality is that your issue is related to the mess that is the Content-Disposition header and more generally character encoding in HTTP headers. There is a very old Stack Overflow question: How to encode the filename parameter of Content-Disposition header in HTTP whose answers while dated cover a lot of what your issue is, but lets break it down.
The PhysicalFileResult inherits from FileResult and when being prepared for return to the client is processed by the FileResultExecutorBase and the code you are interested from that class is contained in the SetContentDispositionHeader method:
private static void SetContentDispositionHeader(ActionContext context, FileResult result)
{
if (!string.IsNullOrEmpty(result.FileDownloadName))
{
// From RFC 2183, Sec. 2.3:
// The sender may want to suggest a filename to be used if the entity is
// detached and stored in a separate file. If the receiving MUA writes
// the entity to a file, the suggested filename should be used as a
// basis for the actual filename, where possible.
var contentDisposition = new ContentDispositionHeaderValue("attachment");
contentDisposition.SetHttpFileName(result.FileDownloadName);
context.HttpContext.Response.Headers[HeaderNames.ContentDisposition] = contentDisposition.ToString();
}
}
Which ultimately lands you in the class ContentDispositionHeaderValue from the Microsoft.Net.Http.Headers namespace. There are currently 725 lines of code in this class most of which are there to ensure that the values returned are valid given the complicated nature of HTTP response headers.
RFC7230 specified that in the headers there is no useful encoding besides ASCII:
Historically, HTTP has allowed field content with text in the
ISO-8859-1 charset [ISO-8859-1], supporting other charsets only
through use of [RFC2047] encoding. In practice, most HTTP header field
values use only a subset of the US-ASCII charset [USASCII]. Newly
defined header fields SHOULD limit their field values to US-ASCII
octets. A recipient SHOULD treat other octets in field content
(obs-text) as opaque data.
Ultimately, some relief may be in sight as RFC8187 from September 2017 provides clarification which should alleviate some of this mess. Issue 2688 filed against HTTP Abstractions covers the need to incorporate this RFC into ASP.Net.
In the meantime you can try slugging an appropriately http encoded filename into the URI (http://example.com/filedownload/33/%F0%9F%98%80.png) rather than relying upon the flawed Content-Disposition header for filename suggestion. To be clear this is an incomplete fix and has the potential for not working on all clients.

How to convert a file into binary format for sending it inside a POST request payload

I'm trying to use this API, which converts files between different formats. As part of the request, it expects the file in Binary format. I'm trying to figure out exactly what that means when it comes to C#. Everything I've read points to using a byte array but that hasn't worked so far.
Essentially, I need to convert a document to binary and then use that raw data and plug it into another part of the system that's responsible for sending the actual POST request.
In an effort to understand the API better, I tested the endpoint and viewed the request payload. Chrome doesn't display the raw data that's sent as part of the File field:
Fiddler does, but it's gibberish (probably because it attempts to interpret the bytes as ASCII):
So I'm a bit confused. For the purposes for that form-data object in the request payload, what do I need to do to the file I have in C#? I tried this (I'm accessing the file from another system using that system's API):
DocumentInfo di = this.BoundEntryInfo as DocumentInfo;
string contentType = "application/msword";
string binaryValue;
using (DocstarReadStream filestream = di.ReadDoc(out contentType)) //open a stream to the word document in the other system
{
using (MemoryStream mstream = new MemoryStream())
{
filestream.CopyTo(mstream);
binaryValue = string.Join("", mstream.ToArray());
}
}
The resulting binaryValue string has a value like this:
807534200608000330223164210108901003250019082916711111011610111011695841211121011159346120109108321624240160020000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000018014820311019448166924714925015145183859823216217042213962150458223372412328625314619918825419019...
However, using this as the value of the File field in the POST request payload results in a Bad Request error (it says "conversion error" so I'm assuming the data is formatted wrong somehow).

Instead of converting the byte array to a string and sending that as the body, you should instead write the bytes to the request stream being sent through your HttpClient of choice. The content-type encoding is expecting you to send across the raw binary data as the payload, and has no idea what to do with the string interpretation.
This is why the data in fiddler looks like gibberish. It is, as you suspected, a text representation of binary data.

What characters are html encoded?

TL;DR Düsseldor is encoded to Düsseldor, Düsseldorf isn't encoded (by the default MVC encoding) why?
I'm building an api with soap access. When I get a request I html encode all the data.
Just before I end the current request I copy the Response.Stream and save it for logging.
I noticed that Düsseldorf is encoded by my api to "Düsseldorf". This is expected.
But in my logs it shows Düsseldorf encoded as "Düsseldorf" ( double encoded ) my logs saves after the MVC serializes writes to the response stream.
I think this is because soap encodes values before sending them over the wire.
So my next test was to turn off html encoding in my api.
So this time Düsseldorf was giving to the soap serializer.
But it was encoded as Düsseldorf?
EDIT: When I say I turn on / off the encoding what I mean is each field in the request I use HttpUtility.HtmlEncode("some field"). And save that to the database. Then I return the data with an MVC view like so:
public ActionResult CreateAccessCode ( CreateAccessCodeRequest request )
{
var response = new Response (request);
return new ContentResult (response);
}
My question is when the ContentResult method runs, it seems to do some encoding of its own and I was wondering what that was?

SOAP uses XML - HTML and XML require slightly different encoding, but it may or may not be a problem in your case.
Much more likely your manual construction of XML is not correct. Please do not construct XML with string concatenation - XDocument/XmlDocument are much more suitable for that.

How to View HTTPRequest information

I'm designing a web interface that is responsible for communicating with a piece of hardware. The hardware contacts the web page and sends an HTTP Request using POST that contains specific data in the body of http package using a "key=value" format.
Server-side I have code that does the following:
public override void ProcessRequest(HttpContext curContext)
{
if (curContext != null)
{
HttpResponse response = curContext.Response;
response.Clear();
response.ContentType = "text/plain";
response.BufferOutput = true;
response.StatusCode = 200; // HttpStatusCode.OK;
response.Write(response.StatusCode.ToString());
response.End();
}
}
But what I really need is the necessary code to review the Body and retrieve the data (which is in text/plain format). I'm fairly new to this type of web programming so I don't know exactly what code to write to look in the curContext to get this information, or if I even have my override method correct.
I was expecting to have something available like curContext.Request.Body but this isn't the case. How can I see the raw POST data in body of the Request? Can anyone point me in the right direction?
Thanks

You'll want to use the InputStream property on the HttpRequest instance returned by the Request property exposed by the HttpContext passed to your ProcessRequest method.
This will give you the contents of the POST request (which you can verify with a call to the HttpMethod property on the same HttpRequest).
Note that you'll have to use a StreamReader in order to convert the byte stream into strings which you'd then decode (since those key/value pairs should be url-encoded).
Fortunately, this is already done for you. On the HttpRequest instance, you can use the NameValueCollection returned by the Form property to check for the values passed as part of a POST request with url-encoded key/value pairs like so:
public override void ProcessRequest(HttpContext curContext)
{
if (curContext != null) return;
string value = curContext.Form["key"];
// Do other processing.
...
Note that you can also use the indexer on the HttpRequest, but that combines everything exposed by the QueryString, Form, Cookies and ServerVariables collections which I would say in this case is a bad idea, as you are specifically posting information and expecting that on the other side.

If you are putting the data in the body of the request in "key=value" format, it sounds like you are URL encoding form data (as long as you are delimiting the key/value pairs with an ampersand (&).
If that's the case, you should be able to use:
curContext.Request["key"]
And it will give you "value" in your example.

WCF Restful Service - Send File with Multiple fields in one requert

I am in the process of developing a WCF Restful Service. One of the requirements of the WCF is to allow a client to upload an image file and several input parameters that may contain multiple values.
I thought of several ways of sending a file with input parameters in one request. I am not sure what the best approach would be.
1) Accept a stream that contains a multipart form-data stream. A huge disadvantage with this approach is that I have to write a multipart parser. (aspNetCompatibilityEnabled="false")
[WebInvoke (UriTemplate = "Account",Method = "POST")]
public String Account(System.IO.Stream stream) {
MultiPartParser(stream);
}
2) Send the file as a stream and send other data in the QueryString. Only issue with this approach is that the values may be multi-line text data.
[WebInvoke (UriTemplate = "Account?input1={val1}&input2={val2}",Method = "POST")]
public String Account(System.IO.Stream stream) {
}
3) Convert the file as a Base64 string and encapsulate it in JSON or XML and send it with the other input parameters. Are there any limitations of this approach?
[WebInvoke (UriTemplate = "Account",Method = "POST")]
public String Account(String ImageFile, String input1, String input2) {
}
What is the best approach? Thank you for your time.

As always, the asnwer is "it depends". You mentioned the pros/cons of each approach, so it will really depend on your situation.
Pro: the file contents are not encoded (i.e., no size bloat). Con: you need to write the multipart parser on the server (and to package the request on the client), the multipart headers will add some (often non-significant) overhead to the requests
Pro: simple, operation is easy to write. Con: may need to URL-encode chars such as new line; size limitation of URIs may be an issue if additional data is large
Pro: No need for encoding, operation is easy to write. Con: size bloat due to base64-encoding (you can declare the parameter as byte[] so the decoding is done automatically for you), client will need to encode the file content, as well as wrap all the parameters from the request.
There's also a fourth option, which is to pass the additional parameters as HTTP headers.
Pro: no size limitation as in the URIs. Con: parameters are not explicitly declared, need to fetch them from the headers using the WebOperationContext, and still need to URI-encode chars outside the 0x20-0x7E range