Develop Reference

c# Multipart Form Data Encoding

I'm working on a C# application that submits some small files to a device via multipart form data. I've been unable to get the device to actually accept the data that I'm sending it and I have a feeling that it has to do with the actual encoding of the file as it's being transmitted somehow, but I'm not entirely sure. It returns a 200/OK, as a valid POST, but doesn't like the payload.
I've been using Chrome Dev Tools for the devices web interface to match as closely as possible. One thing I've noticed is that Dev Tools doesn't show me the actual payload, just the following as if it can't display it:
------WebKitFormBoundaryWwZOi6WAr7yb3yRE
Content-Disposition: form-data; name="[Redacted]"; filename="[AlsoRedacted]"
Content-Type: application/octet-stream
------WebKitFormBoundaryWwZOi6WAr7yb3yRE--
Alternatively, the MessageBox I've been using to help debug seem to have no issue displaying it.
The relevant portions of the code are below and would appreciate any suggestions.
using (FileStream fs = new FileStream(Filename, FileMode.Open, FileAccess.Read))
{
using (MultipartFormDataContent Content = new MultipartFormDataContent(String.Format("----------{0:N}", Guid.NewGuid())))
{
StreamContent ContentStream = new StreamContent(fs);
Content.Add(ContentStream, "filename", System.IO.Path.GetFileName(Filename));
ContentStream.Headers.ContentDisposition.Name = "UploadEdid";
ContentStream.Headers.ContentDisposition.FileNameStar = null;
ContentStream.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
String ContentString = Content.ReadAsStringAsync().Result;
System.Windows.MessageBox.Show(ContentString, ContentString.Length.ToString());
using (HttpResponseMessage Response = await HttpClient.PostAsync(RequestUri, Content))
{
if (Response.IsSuccessStatusCode)
{
HttpLoggedIn = true;
string ResponseString = await Response.Content.ReadAsStringAsync();
ParseHttpResponse(ResponseString);
}
else
{
HttpLoggedIn = false;
}
}
}
}

C# StreamContent and File.OpenRead() Not Producing HTTP'able multipart content

I have a .Net Core 2.0 application that is sending files to a Web API endpoint, using multipart content. Everywhere I've looked, such as C# HttpClient 4.5 multipart/form-data upload, makes it seem that it should be as easy as passing a FileStream to a StreamContent. However, when I make the post, it looks like the file is attaching as text, not bits.
Actual code:
var request = new HttpRequestMessage()
{
Method = HttpMethod.Post,
RequestUri = new Uri( "http://localhost:10442/filetest" )
};
var multiContent = new MultipartFormDataContent();
var filestream = File.OpenRead( path );
var filename = Path.GetFileName( path );
var streamContent = new StreamContent( filestream );
streamContent.Headers.Add( "Content-Type", "application/octet-stream" );
streamContent.Headers.Add( "Content-Disposition", $"form-data; name=\"file1\"; filename=\"{filename}\"" );
multiContent.Add( streamContent, "file", filename );
request.Content = multiContent;
var response = await new HttpClient().SendAsync( request );
The request looks like this which, as you may notice, is not all on one line (which I think is a/THE problem):
POST http://localhost:10442/filetest HTTP/1.1
Content-Type: multipart/form-data; boundary="c5295887-425d-4ec7-8638-20c6254f9e4b"
Content-Length: 88699
Host: localhost:10442
--c5295887-425d-4ec7-8638-20c6254f9e4b
Content-Type: application/octet-stream
Content-Disposition: form-data; name="file1"; filename="somepdf.pdf"
%PDF-1.7
%
1 0 obj
<</Type/Catalog/Version/1.7/Pages 3 0 R/Outlines 2 0 R/Names 8 0 R/Metadata 31 0 R>>
endobj
Fiddler shows the entire post all the way down to the end boundary, but await Request.Content.ReadAsStringAsync() in the endpoint only shows the first couple dozen bytes (it looks as if the stream wasn't finished, but if Fiddler got it all, shouldn't my endpoint have too?).
I was having similar trouble trying to hit a remote endpoint; I built this endpoint to test locally.
The exception I'm getting is:"Unexpected end of MIME multipart stream. MIME multipart message is not complete." To me, this makes sense both if I'm really only getting part of my stream, or if the line breaks are throwing something off.
I have also tried throwing some of the Idisposables into Usings but, as expected, that closes the streams and I get exceptions that way.
And for completeness's sake, here's the endpoint I'm calling:
public async void ReceiveFiles()
{
// exception happens here:
var mpData = await Request.Content.ReadAsMultipartAsync();
await Task.FromResult( 0 );
}

Try something like this:
static int Main(string[] args)
{
var request = new HttpRequestMessage()
{
Method = HttpMethod.Post,
RequestUri = new Uri("http://localhost:10442/filetest")
};
var path = "c:\\temp\\foo.bak";
using (var filestream = File.OpenRead(path))
{
var length = filestream.Length.ToString();
var streamContent = new StreamContent(filestream);
streamContent.Headers.Add("Content-Type", "application/octet-stream");
streamContent.Headers.Add("Content-Length", length);
request.Content = streamContent;
Console.WriteLine($"Sending {length} bytes");
var response = new HttpClient().SendAsync(request).Result;
Console.WriteLine(response.Content.ReadAsStringAsync().Result);
}
Console.WriteLine("Hit any key to exit");
Console.ReadKey();
return 0;
}
and
[HttpPost]
public async Task<IActionResult> Upload()
{
var buf = new byte[1024 * 64];
long totalBytes = 0;
using (var rs = Request.Body)
{
while (1 == 1)
{
int bytesRead = await rs.ReadAsync(buf, 0, buf.Length);
if (bytesRead == 0) break;
totalBytes += bytesRead;
}
}
var uploadedData = new
{
BytesRead = totalBytes
};
return new JsonResult(uploadedData) ;
}

I'm trying to solve a similar issue, and I'm not 100% to a solution yet, but maybe some of my research can help you.
It was helpful to me to read through the microsoft docs for .NET core file uploads, specifically for large files that use streams and multipart form data:
https://learn.microsoft.com/en-us/aspnet/core/mvc/models/file-uploads?view=aspnetcore-2.1#uploading-large-files-with-streaming
You already referenced it, but there's some relevant useful information in this answer:
C# HttpClient 4.5 multipart/form-data upload
This explains the details of the content-disposition header and how it is used with multipart form data requests: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition#As_a_header_for_a_multipart_body
As to your specific problem of the file being sent as text instead of bits, since http is text-based, it can only be sent as text, but that text can be encoded as you see fit. Perhaps your StreamContent needs a specific encoding to be used, like base64 encoding or similar? I do believe the newlines are significant in the multipart request, so hopefully setting the encoding for the file content as needed would be enough.
Another possibility: could it be that you need to set additional information on the file section's headers or in the definition of the StreamContent to indicate that it should expect to continue, or that the boundary information isn't put in correctly? See Multipart forms from C# client

I use this lib : https://github.com/jgiacomini/Tiny.RestClient
It's make easier to send multiplart file to send multipart files.
Here a sample :
await client.PostRequest("MultiPart/Test").
AsMultiPartFromDataRequest().
AddStream(stream1, "request", "request2.bin").
AddStream(stream2, "request", "request2.bin")
ExecuteAsync();

Http Response Contains MIME stream.. which has binary Images seperated by Boundary String.. need to get those Images and save using c#

when i make a HTTP request to the server it responses with the MIME Response Stream which has two or more images in it as Binary data which are separated by Boundary String
Now i need to extract those images only and save them individually into database
Stream Looks like this...
Header has
RETS-Version: RETS/1.0
MIME-Version: 1.0
Content-type: multipart/parallel;
boundary="simple boundary"
Http ResoinseStream Has
--simple boundary
Content-Type: image/jpeg
Content-ID: 123456
Object-ID: 1
<binary data>(Image1)
--simple boundary
Content-Type: image/jpeg
Content-ID: 123457
Object-ID: 1
<binary data>(Image2)
--simple boundary --
I need to extract image1 and image2 and save those in database as binary image.

If you use my MimeKit library, you can do this really easily:
static MimeEntity ParseHttpWebResponse (HttpWebResponse response)
{
var contentType = ContentType.Parse (response.ContentType);
return MimeEntity.Parse (contentType, response.GetResponseStream ());
}
static void SaveAllImages (HttpWebResponse response)
{
var entity = ParseHttpWebResponse (response);
var multipart = entity as Multipart;
if (multipart != null) {
foreach (var image in multipart.OfType<MimePart> ()) {
using (var memory = new MemoryStream ()) {
image.ContentObject.DecodeTo (memory);
var blob = memory.ToArray ();
// save it to your database
}
}
}
}

How to send an image to a browser

I am building a simplified web server, I was able to handle sending the HTML pages properly
but when I get a request for an image, my code doesn't give the browser the image
FileStream fstream = new FileStream(tempSplitArray[1],FileMode.Open,FileAccess.Read);
//The tempSplitArray //recieves the request from the browser
byte[] ar = new byte[(long)fstream.Length];
for (int i = 0; i < ar.Length; i++)
{
ar[i] = (byte)fstream.ReadByte();
}
string byteLine = "Content-Type: image/JPEG\n" + BitConverter.ToString(ar);
sw.WriteLine(byteLine);//This is the network stream writer
sw.Flush();
fstream.Close();
Excuse my ignorance and if there are any questions, or my question is not clear enough, let me know.

Basically you want your response to look like:
HTTP/1.1 200 OK
Content-Type: image/jpeg
Content-Length: *length of image*
Binary Image Data goes here
I'm assuming sw is a StreamWriter, but you need to write the raw bytes of the image.
So how about:
byte[] ar;
using(FileStream fstream = new FileStream(tempSplitArray[1],FileMode.Open,FileAccess.Read);)
{
//The tempSplitArray //recieves the request from the browser
ar = new byte[(long)fstream.Length];
fstream.read(ar, 0, fstream.Length);
}
sw.WriteLine("Content-Type: image/jpeg");
sw.WriteLine("Content-Length: {0}", ar.Length); //Let's
sw.WriteLine();
sw.BaseStream.Write(ar, 0, ar.Length);
It really helps to use a tool like fiddler to view the communications between browsers and a (real) webserver and try to replicate that.

How to get the file size from http headers

I want to get the size of an http:/.../file before I download it. The file can be a webpage, image, or a media file. Can this be done with HTTP headers? How do I download just the file HTTP header?

Yes, assuming the HTTP server you're talking to supports/allows this:
public long GetFileSize(string url)
{
long result = -1;
System.Net.WebRequest req = System.Net.WebRequest.Create(url);
req.Method = "HEAD";
using (System.Net.WebResponse resp = req.GetResponse())
{
if (long.TryParse(resp.Headers.Get("Content-Length"), out long ContentLength))
{
result = ContentLength;
}
}
return result;
}
If using the HEAD method is not allowed, or the Content-Length header is not present in the server reply, the only way to determine the size of the content on the server is to download it. Since this is not particularly reliable, most servers will include this information.

Can this be done with HTTP headers?
Yes, this is the way to go. If the information is provided, it's in the header as the Content-Length. Note, however, that this is not necessarily the case.
Downloading only the header can be done using a HEAD request instead of GET. Maybe the following code helps:
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://example.com/");
req.Method = "HEAD";
long len;
using(HttpWebResponse resp = (HttpWebResponse)(req.GetResponse()))
{
len = resp.ContentLength;
}
Notice the property for the content length on the HttpWebResponse object – no need to parse the Content-Length header manually.

Note that not every server accepts HTTP HEAD requests. One alternative approach to get the file size is to make an HTTP GET call to the server requesting only a portion of the file to keep the response small and retrieve the file size from the metadata that is returned as part of the response content header.
The standard System.Net.Http.HttpClient can be used to accomplish this. The partial content is requested by setting a byte range on the request message header as:
request.Headers.Range = new RangeHeaderValue(startByte, endByte)
The server responds with a message containing the requested range as well as the entire file size. This information is returned in the response content header (response.Content.Header) with the key "Content-Range".
Here's an example of the content range in the response message content header:
{
"Key": "Content-Range",
"Value": [
"bytes 0-15/2328372"
]
}
In this example the header value implies the response contains bytes 0 to 15 (i.e., 16 bytes total) and the file is 2,328,372 bytes in its entirety.
Here's a sample implementation of this method:
public static class HttpClientExtensions
{
public static async Task<long> GetContentSizeAsync(this System.Net.Http.HttpClient client, string url)
{
using (var request = new System.Net.Http.HttpRequestMessage(System.Net.Http.HttpMethod.Get, url))
{
// In order to keep the response as small as possible, set the requested byte range to [0,0] (i.e., only the first byte)
request.Headers.Range = new System.Net.Http.Headers.RangeHeaderValue(from: 0, to: 0);
using (var response = await client.SendAsync(request))
{
response.EnsureSuccessStatusCode();
if (response.StatusCode != System.Net.HttpStatusCode.PartialContent)
throw new System.Net.WebException($"expected partial content response ({System.Net.HttpStatusCode.PartialContent}), instead received: {response.StatusCode}");
var contentRange = response.Content.Headers.GetValues(#"Content-Range").Single();
var lengthString = System.Text.RegularExpressions.Regex.Match(contentRange, #"(?<=^bytes\s[0-9]+\-[0-9]+/)[0-9]+$").Value;
return long.Parse(lengthString);
}
}
}
}

WebClient webClient = new WebClient();
webClient.OpenRead("http://stackoverflow.com/robots.txt");
long totalSizeBytes= Convert.ToInt64(webClient.ResponseHeaders["Content-Length"]);
Console.WriteLine((totalSizeBytes));

HttpClient client = new HttpClient(
new HttpClientHandler() {
Proxy = null, UseProxy = false
} // removes the delay getting a response from the server, if you not use Proxy
);
public async Task<long?> GetContentSizeAsync(string url) {
using (HttpResponseMessage responce = await client.GetAsync(url))
return responce.Content.Headers.ContentLength;
}