I am saving a raw HTTP request to a text file and I need to read the multipart content within it. I know there are a number of 3rd party tools to do this but I need to do this with the .NET framework if possible. I have tried reading the txt file into a stream but I don't know how to extract the multipart content. Here is an example of the code I have been using and a sample file I am trying to read from :
var content = new MultipartFormDataContent();
var stream = new StreamContent(File.Open(#"C:\temp\test.txt", FileMode.Open));
content.Add(stream);
List<StreamContent> lstStreamContents = new List<StreamContent>();
if (content != null)
{
if (content.IsMimeMultipartContent())
await content.ReadAsMultipartAsync(StreamProvider);
foreach (MultipartContent multiPartContent in content)
{
foreach (var streamContent in multiPartContent).....
Text file :
Cache-Control: no-cache
Connection: Close
Content-Length: 5216
Content-Type: multipart/related;
boundary="cbsms-main-boundary";
start="<soap-envelope>", text/xml;
charset=utf-8
Accept: */*
Host: hostname
User-Agent: Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)
soapaction: ""
This is a multi-part message in MIME format.
--cbsms-main-boundary
Content-Type: text/xml;
charset="utf-8"
Content-ID: <soap-envelope>
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header>
<mm7:TransactionID
xmlns:mm7="http://www.3gpp.org/ftp/Specs/archive/23_series/23.140/schema/REL
-6-MM7-1-2">5947CCE35D5B4AEFB99DADDDF9472E67</mm7:TransactionID>
</SOAP-ENV:Header>
<SOAP-ENV:Body>
<mm7:DeliverReq
xmlns:mm7="http://www.3gpp.org/ftp/Specs/archive/23_series/23.140/schema/REL
-6-MM7-1-2">
<mm7:MM7Version>6.5.0</mm7:MM7Version>
<mm7:MMSRelayServerID>hostname</mm7:MMSRelayServerID>
<mm7:LinkedID>1-1754394156</mm7:LinkedID>
<mm7:SenderAddress>
<mm7:Number>46707630767</mm7:Number>
</mm7:SenderAddress>
<mm7:Recipients>
<mm7:To>
<mm7:Number>72401</mm7:Number>
</mm7:To>
</mm7:Recipients>
<mm7:TimeStamp>2008-05-08 11:16:39</mm7:TimeStamp> <mm7:Priority>Normal</mm7:Priority>
<mm7:Subject>Tube test</mm7:Subject>
</mm7:DeliverReq>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
--cbsms-main-boundary
Content-Type: multipart/mixed;
boundary="cbsms-sub-boundary"
Content-ID: <MM7-Media>
--cbsms-sub-boundary
content-type: application/smil;
Name=main.smil;Charset=utf-8
content-transfer-encoding: 7bit
content-id: <AAAA>
content-length: 483
<smil><head><layout><root-layout backgroundColor="#FFFFFF"
background-color="#FFFFFF" height="480px" width="640px"/> <region id="Image" top="0" left="0" height="50%" width="100%" fit="meet"/> <region id="Text" top="50%" left="0" height="50%" width="100%"
fit="scroll"/>
</layout>
</head>
<body><par dur="4000ms"><img src="smslogo.jpg" region="Image"></img> <text src="smil.txt" region="Text"><param name="foreground-color"
value="#000000"/>
</text>
</par>
</body>
</smil>
--cbsms-sub-boundary
content-type: image/jpeg;
Name=smslogo.jpg
content-transfer-encoding: Base64
content-location: smslogo.jpg
content-length: 2218
/9j/4AAQSkZJRgABAgAAZABkAAD/7AARRHVja3kAAQAEAAAAKgAA/+4ADkFkb2JlAGTAAAAA
Af/bAIQACwgICAgICwgICxAKCQoQEw4LCw4TFhERExERFhUREhISEhEVFRkaGxoZFSEhJCQh
ITAvLy8wNjY2NjY2NjY2NgEMCgoMDQwPDQ0PEw4ODhMUDg8PDhQaEhIUEhIaIhgVFRUVGCIe
IBsbGyAeJSUiIiUlLy8sLy82NjY2NjY2NjY2/8AAEQgAZgBmAwEiAAIRAQMRAf/EAJ4AAAEF
AQEAAAAAAAAAAAAAAAUAAQIEBgcDAQEBAQEBAAAAAAAAAAAAAAABAAIDBBAAAQMCAwQDCgkK
BwAAAAAAAQACAxEEIRIFMVETBkFhgXGRsSJCYrKT0xSh0TJScpIjMxbB4YKiJBUlNVUX8NJT
g6MmNhEBAAECBAQEBwAAAAAAAAAAAAERAiExcRKBkTKTQVGy0vAiQpKiAxP/2gAMAwEAAhED
EQA/ANDzDzHJYzughLpbx7i2KBhNTjhWnQvDTZ9bawy6jeOkkfiIG0DWV6KgVJ7VK5srePV7
y+pmuJXUzHyWgBuVvdopV3Iea+67dOMxSfBfglvrh+SOV2HynEmgCLxxNYwB73vd0uLnDwFU
rTLBC1o2nFx6yvbjpbtmfOVrJF5313fGmyR73fXd8ar8brS4oU1uWMke9313fGn4cW9313fG
q4lT8UKVVgRxed9d3xrwurWV7c1rM9jx5BcS09pSEoUhKpVnzAp7i/aHwunkhk2Zh8oHeM1Q
VnRzFrmj3ot9ZuHT2kx+xu2gNA6nBtO1ba/t23cdR9635B39SzV3bQXsD7W5ZnjfgQdoO8dY
Q5XTdE1rPNp4tQe7Qp7svwZlIfXySW41SQG30iT8CXWk8bHiNbxca8Pjtf6GFEkuu+fw3cU7
91L2f6ZVfOME+pyZdQuB55VXiKcbuqdRwTVANdqfjITDc1blO0bF7ibrU1EiHGqpcXcUPEqk
JutSqvibrT8ZDJbuOFoc8nxjRrQKucdzQNpUDqGT72CeMbzGSP1aqNRfjJxMhDdTtHGgmaDu
d4p/Wovds7XirXBw3g1UqiXHQe7IFzJTYTXvhWON01wQmW+t5JnVkAJPTh8KhdNWhtz/ANcu
j57fTYkoW7h+GLt1RTO3Ho+WxJTXsA9XkA1O5bUVD8R2Ki6bK0kYnYB1q9zQ1kE3vNAJHyuF
RtIFa1Qq3dx54GbczwT3G4qc7o+edRJthIQC+dwk6Q0CgO7EJ/dbtvybgHdnZ8SLwNozN0u6
VOlU0apAG736JpeQyRrcSGktdQbaAqUd017Q9hq04hFpuGyJ8jmijGlxw3Cqx9rO7EdCJE4N
DYkz6k09EETndryGrRNtAWgucakdGxZ7lxplfczHynsjHcaK/lWsomGrclCXTY5B44a8bnNB
8KD6tpsdhbG/t2NhfC5pOTAODiGkFq01CgHOE3D0psfTNK1vY2rlKcgeS94kTi3BoBPeC9LK
3bJaxxuANWhzqjpOKDtcRbP84UHatJZsDWU3Bo+BEM24yKW9pH+Gru1yDJnb4vR94xySsW/8
kuvpt9JqSXX2sjznL+1RRbnSO77qKlo4z3jPMY535AlzfLm1p0fzAR33Er05eZnkmfua1g7S
hyuxvnVqoxRje4nT0p2JqKbD9al4Ol3Dq0JblH6RosjauoCVoObJuHYRxDbLJj3GhZqJ+WM0
3Kc7s275XhpYwvO2Vz5T3K0HgWiIQ3RIOBawRf6cLQe7SqJkYph0iMIRWO55n+0s7cdAfIe0
5Qtkudc4T8bXXxg4QMZH20zHwqkXYQqRePwovnvaD31qrceITvKzGmt4l5COhgc897DwrVwD
7NvXihmzIUt/5JdfTb6TUlK3/k119Nvhakl29rnHNTq8xXY+a4eAItyuwGBz/ny07GgINzT/
AOkvvpj0Wo9ypJFJaNiY4GSNzy9nTjswQ4/XOrQkJlKiamNFNsdzlP8AtVtbjyGFx7rj+ZC7
JhnuILcD7yRje+RVS5muBNrkzQaiOkfeGKs8rxcfWrcbRGHSH9EfnT4OWdzployjXHeadgVh
Rtm5YW9eK9cqXeiFKlci1W5971m7m2h0rqdwGgXV7+YWljc3J2QxPf3m4fCuMRPzSOedpJNU
S5/saTQYszp5t2WNp+ErUBpDQNwog2gQFlnDUYyEyO7diO0QYikL1vX9z3W/O3wtSUbe4i/c
11JXxOM2OvncRsfhSU3X0ud80gfv+9PSXj0QgueaNwkheWPGwg0PwI7zS3+O3h88eiEFIS89
9d86ytQ8z65bYC4MjdzwHeFWzztqxicwsjDyKB4bQjr3IO4b1AsG5VBulAOklldLIS57jVzj
vKP8r6lZ6ZqBkvDkjkjMYkpXKSRiRuwQSlNiYioomg3Y1drtdU0u4Y0W95BJgKAPbXvE1VwZ
XYtIPcNfAuCcN4+SSFNs14zBkz2jqcQh2/q6fzzrFvZaTLYNeDdXVG8MGpaytS526q5pbAuo
OlxA768C2WR2aQlx3nFWWNytoqjF11ZdHsYhG0MbsjaGjsS1K9j06zkuJDRzQRG07XOOwBYV
mt6xDGIYrl7YxgKUrTu7VXfJd3j89xI6V295JQ1uwba3lk/t5dz1+04rX5vO95YUlG3dH/bi
78cZM7W5uivvDG07+CSm6+g/NMHLz9QmJuuFdVHGGWQ40G5pGxARZ6B5WoU/25P8iSSoN2c9
HFMWXLPlaifVy+zTiy5T8rUj6qX2aSSebPbSFjyf06mfVTezT+5cm/1N3qpvZpJK5jtF7lyd
/U3eqm9mn9y5O/qZ9VN7NJJXM9s3uPKNcNTPqpfZpjZcr9GpH1Uvs0klc120DZ8u+TqP/HJ7
NO200QVz35MVPHpHJWnVRle8kkg9tq44+UfwJMxk38Bw4klJq5uM2mFOJ97/AIokkkp0+3p+
OD//2Q==
--cbsms-sub-boundary
content-type: text/plain;
Name=smil.txt;Charset=utf-8
content-transfer-encoding: 7bit
content-location: smil.txt
content-length: 9
Tube test
--cbsms-sub-boundary--
--cbsms-main-boundary--
At the moment I am getting this error 'Unable to cast object of type 'System.Net.Http.StreamContent' to type 'System.Net.Http.MultipartContent'.'
EDIT :
foreach (var part in multipart.Contents)
{
if (part.Headers.ContentType.MediaType == "multipart/mixed")
{
if (part.IsMimeMultipartContent())
await part.ReadAsMultipartAsync(StreamProvider);
foreach (MultipartContent multiPartContent in StreamProvider.Contents)
{
This gives the error ''Unexpected end of MIME multipart stream. MIME multipart message is not complete'
You are adding a StreamContent as a part to the empty MultipartFormDataContent. Naturally, ReadAsMultipartAsync then does nothing. Instead, you want to read the StreamContent. Also, your file contains not just the content, but the headers as well. If you snip off the headers, this ought to work:
using (var stream = File.Open (#"C:\temp\test.txt", FileMode.Open))
{
// note: StreamContent has no Content-Type set by default
// set a suitable Content-Type for ReadAsMultipartAsync()
var content = new StreamContent (stream) ;
content.Headers.ContentType =
System.Net.Http.Headers.MediaTypeHeaderValue.Parse (
"multipart/related; boundary=cbsms-main-boundary") ;
// TODO: make this recursive if required...
var outerMultipart = await stream.ReadAsMultipartAsync () ;
foreach (var outerPart in outerMultipart.Contents)
{
if (outerPart.IsMimeMultipartContent())
{
var innerMultipart = await outerPart.ReadAsMultipartAsync () ;
foreach (var innerPart in innerMultipart.Contents) // do stuff
}
else // do other stuff
}
}
Use HttpContentMultipartExtensions class and ReadAsMultipartAsync method
http://msdn.microsoft.com/en-us/library/hh835439%28v=vs.108%29.aspx
Good example is here:
http://www.asp.net/web-api/overview/working-with-http/sending-html-form-data,-part-2
Related
I have a .Net Core 3.1 Web API that needs to interact with an external service that is providing a multipart/related response from a WCF Service with attachments in the context as the below block.
HTTP/1.1 200 OK
Cache-Control: private
Content-Length: 1790633
Content-Type: multipart/related; type="text/xml"; start="<8fb5b3efa6474add9032cb80870dc065>"; boundary="------=_Part_20200731120159.494997"
Server: Microsoft-IIS/10.0
Set-Cookie: ASP.NET_SessionId=jga2opbhoirgnhjhmmsuxi1k; path=/; HttpOnly; SameSite=Lax
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Set-Cookie: ARRAffinity=f7b4f7f5d8cbd8c8bc5d9a4bf49e8b5938412c4294f6e2c2b5c2d5cfc4b4d437;Path=/;HttpOnly;Domain=lparouterpoc.azurewebsites.net
Date: Fri, 31 Jul 2020 13:33:04 GMT
--------=_Part_20200731120159.494997
Content-Type: text/xml; charset=UTF-8
Content-Transfer-Encoding: binary
Content-Id: <8fb5b3efa6474add9032cb80870dc065>
<?xml version="1.0" encoding="iso-8859-1"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
...
</SOAP-ENV:Envelope>
--------=_Part_20200731120159.494997
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
Content-Id: 7b1fff220df5#xxxxx
%PDF-1.5
%����
...
Additional characters added in here
...
%%EOF
--------=_Part_20200731120159.494997
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
Content-Id: 844eb478dc05#xxxxx
%PDF-1.5
...
Additional characters added here
...
%%EOF
--------=_Part_20200731120159.494997--
Using SOAP UI, I see that there are attachments, but each attachment is a blank PDF.
The code in the .Net Core API is
using (var response = await Client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead))
{
var stream = await response.Content.ReadAsStreamAsync();
using var streamReader = new StreamReader(stream, Encoding.UTF8);
var message = await streamReader.ReadToEndAsync(); // This is where output is changed to add additional characters
var headers = response.Headers;
return new customResponse
{
ContentResult = new ContentResult
{
Content = message,
ContentType = response.Content.Headers.ContentType?.ToString(),
StatusCode = (int)response.StatusCode
},
ResponseHeaders = headers
};
}
The image below shows an example of where the additional characters are coming in causing the PDF's to be invalid \ corrupt \ blank.
An additional characteristic of this is the content length going from 1021690 to 1790633.
I have been able to debug it this far and test that the files before they get into the message (string) are correct as I am able to read the multipart content and write the files out to disk when checking that the files are indeed correct using
var content = await response.Content.ReadAsMultipartAsync();
for (var i = 0; i < content.Contents.Count; i++)
{
var item = content.Contents[i];
if (item.Headers.ContentType.ToString() == "application/octet-stream")
{
var bytes = await item.ReadAsByteArrayAsync();
File.WriteAllBytes(#$"c:\temp\{i}.pdf", bytes);
}
}
and then iterating through the content.Contents.
I would be grateful if anyone could tell me how \ why this is occurring and how I can stop this from occuring
Further investigation and hypothesis is could this be down to encoding? The WCF service for the attachments is TransferEncoding = "binary"
Left and middle of the image are the same, the middle written to disk direct from my API (not what I need) shows me that the file can be read and written.
The right is the direct output from the service, which comes back into SOAP UI and downloads as expected.
When looking at the response in the watch window the transfer encoding is empty so am a little confused on how to resolve this.
try this,
public async Task<HttpResponseMessage> Post(string requestBody, string action)
{
using (var request = new HttpRequestMessage(HttpMethod.Post, ""))
{
request.Headers.Add("SOAPAction", action);
request.Content = new StringContent(requestBody, Encoding.UTF8, "text/xml");
var response = await Client.SendAsync(request);
return response;
}
}
and then in your controller return it like this
var result = await _service.Post(requestBody, action);
var stream = await result.Content.ReadAsStreamAsync();
await stream.CopyToAsync(HttpContext.Response.Body);
formData.Add(sJobId,"job_id"); is sending "job_id\r\n" to the server
Here is my C# method:
public static async Task UploadAsync(string url, int job_id, string filename, string filePath) {
try {
// Submit the form using HttpClient and
// create form data as Multipart (enctype="multipart/form-data")
using (var fileStream = new StreamContent(System.IO.File.Open(filePath, FileMode.Open, FileAccess.Read)))
using (var formData = new MultipartFormDataContent()) {
StringContent sJobId = new StringContent(job_id.ToString());
StringContent sOthId = new StringContent(job_id.ToString());
// Try as I might C# adds a CrLf to the end of the job_id tag - so have to strip it in ruby
formData.Add(sOthId, "oth_id");
formData.Add(sJobId,"job_id");
fileStream.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
formData.Add(fileStream, "dt_file", filename);
HttpResponseMessage response = await HttpClient.PostAsync(url, formData);
// If the upload failed there is not a lot we can do
return;
}
} catch (Exception ex) {
// Not a lot we can do here - so just ignore it
System.Diagnostics.Debug.WriteLine($"Upload failed {ex.Message}");
}
}
This is what my Ruby puma server is receiving - see how oth_id and job_id have \r\n appended but "dt_file" does not.
Parameters: {"oth_id\r\n"=>"42157", "job_id\r\n"=>"42157", "dt_file"=>#<ActionDispatch::Http::UploadedFile:0x007f532817dc98 #tempfile=#<Tempfile:/tmp/RackMultipart20190715-37897-189ztb6.msg>, #original_filename="2019-07-15 164600.msg", #content_type="application/octet-stream", #headers="Content-Type: application/octet-stream\r\nContent-Disposition: form-data; name=dt_file; filename=\"2019-07-15 164600.msg\"; filename*=utf-8''2019-07-15%20164600.msg\r\n">}
How do I stop the formData.Add appending a \r\n to the name?
The raw message the application is sending to the server is
POST https://example.com/ HTTP/1.1
Host: example.com
Content-Type: multipart/form-data; boundary="93655e5a-b6b3-48d6-82c9-0d9aa99164cc"
Content-Length: 522
--93655e5a-b6b3-48d6-82c9-0d9aa99164cc
Content-Type: text/plain; charset=utf-8
Content-Disposition: form-data; name=oth_id
1234
--93655e5a-b6b3-48d6-82c9-0d9aa99164cc
Content-Type: text/plain; charset=utf-8
Content-Disposition: form-data; name=job_id
1234
--93655e5a-b6b3-48d6-82c9-0d9aa99164cc
Content-Type: application/octet-stream
Content-Disposition: form-data; name=dt_file; filename=myfile.txt; filename*=utf-8''myfile.txt
a,b,c,d
aa,"b,b","c
c",dd
aaa
--93655e5a-b6b3-48d6-82c9-0d9aa99164cc--
Have a look at the name values.
Looking at RFC 7578 I can see in every example, that the value for name is always quoted.
Content-Disposition: form-data; name="user"
I did not find any hint if it is mandantory or not to quote the values, so I cannot judge who is wrong here.
To get such quoted name values you only have to quote the values in code.
public static async Task UploadAsync(string url, int job_id, string filename, string filePath) {
try {
// Submit the form using HttpClient and
// create form data as Multipart (enctype="multipart/form-data")
using (var fileStream = new StreamContent(System.IO.File.Open(filePath, FileMode.Open, FileAccess.Read)))
using (var formData = new MultipartFormDataContent()) {
StringContent sJobId = new StringContent(job_id.ToString());
StringContent sOthId = new StringContent(job_id.ToString());
// Try as I might C# adds a CrLf to the end of the job_id tag - so have to strip it in ruby
formData.Add(sOthId, "\"oth_id\"");
formData.Add(sJobId,"\"job_id\"");
fileStream.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
formData.Add(fileStream, "\"dt_file\"", filename);
HttpResponseMessage response = await HttpClient.PostAsync(url, formData);
// If the upload failed there is not a lot we can do
return;
}
} catch (Exception ex) {
// Not a lot we can do here - so just ignore it
System.Diagnostics.Debug.WriteLine($"Upload failed {ex.Message}");
}
}
which will now post this
POST https://example.com/ HTTP/1.1
Host: example.com
Content-Type: multipart/form-data; boundary="c33cdc86-db44-40ef-8e6e-3e13a96218d1"
Content-Length: 528
--c33cdc86-db44-40ef-8e6e-3e13a96218d1
Content-Type: text/plain; charset=utf-8
Content-Disposition: form-data; name="oth_id"
1234
--c33cdc86-db44-40ef-8e6e-3e13a96218d1
Content-Type: text/plain; charset=utf-8
Content-Disposition: form-data; name="job_id"
1234
--c33cdc86-db44-40ef-8e6e-3e13a96218d1
Content-Type: application/octet-stream
Content-Disposition: form-data; name="dt_file"; filename=myfile.txt; filename*=utf-8''myfile.txt
a,b,c,d
aa,"b,b","c
c",dd
aaa
--c33cdc86-db44-40ef-8e6e-3e13a96218d1--
Just found a pull request for PowerShell
// .NET does not enclose field names in quotes, however, modern browsers and curl do.
contentDisposition.Name = $"\"{LanguagePrimitives.ConvertTo<String>(fieldName)}\"";
I'm trying to post to a web form defined as:
<form name="frmdata" method='post' enctype ='multipart/form-data' action ="http://www.rzp.cz/cgi-bin/aps_cacheWEB.sh">
<input type ="hidden" name ="VSS_SERV" value="ZVWSBJXML">
<input type="file" name="filename">
<input type ='submit' name ='x' value ='ODESLI'>
</form>
There is some additional documentation on the form here:
http://www.rzp.cz/docs/RZP02_XML_28.pdf
My latest try:
using (WebClient client = new WebClient())
{
NameValueCollection vals = new NameValueCollection();
vals.Add("VSS_SERV", "ZVWSBJXML");
string filecontent = #"<?xml version=""1.0"" encoding=""ISO-8859-2""?>";
filecontent = filecontent + #"
<VerejnyWebDotaz
elementFormDefault=""qualified""
targetNamespace=""urn:cz:isvs:rzp:schemas:VerejnaCast:v1""
xmlns=""urn:cz:isvs:rzp:schemas:VerejnaCast:v1"" version=""2.8"">";
filecontent = filecontent + #"
<Kriteria>
<IdentifikacniCislo>03358437</IdentifikacniCislo>
<PlatnostZaznamu>0</PlatnostZaznamu></Kriteria>";
filecontent = filecontent + #"</VerejnyWebDotaz>";
vals.Add("filename", filecontent);
client.Headers["ContentType"] = "multipart/form-data";
byte[] responseArray = client.UploadValues(#"http://www.rzp.cz/cgi-bin/aps_cacheWEB.sh", "POST", vals);
string str = Encoding.ASCII.GetString(responseArray);
}
But I can't get past this error:
<KodChyby>-1</KodChyby> (the xml filename does not contain xml defined by namespace)
How can I send this xml data to the form or rather there is a working form - http://stuff.petrovsky.cz/subdom/stuff/RZP/rzp-test-form.php - how to call and catch xml data? I would like to do the same request and get xml.
Using System.Net.Http I was able to construct the form request as a proof of concept using MultipartFormDataContent
Now initially when I tested it, I received 403 Forbidden response but I guessed that was to be expected given my location and that the endpoint might be region locked.
Raw Fiddler response
HTTP/1.1 403 Forbidden
Date: Sat, 27 Oct 2018 01:37:09 GMT
Server: IIS
Content-Length: 225
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /cgi-bin/aps_cacheWEB.sh
on this server.</p>
</body></html>
I was wrong and the forbidden appeared to be the default response for bad requests as you commented that you received the same forbidden error from within the region. So back to the drawing board I went.
I then copied the example HTML form locally and then proceeded to compare the requests from the form (which did actually work) and my code. Gradually making changes to match I was finally able to get a 200 OK response, but the body of the response was empty.
Apparently there was an issue with the server interpreting the boundary in the content type header if it is wrapped in quotes boundary="...".
After more adjustments it then started returning a message based on the content dispositions generated.
HTTP/1.1 200 OK
Date: Sat, 27 Oct 2018 19:55:11 GMT
Server: IIS
Serial: 10.145
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/plain; charset=ISO-8859-2
Content-Length: 169
Multiple definitions of VSS_SERV encountered in input.
If you're trying to do this intentionally (such as with select),
the variable must have a "List" suffix.
So it turns out that the XML API is expecting the request to be in a very specific format. Deviate from that and the request fails.
The MultipartFormDataContent was not generating the request correctly and this caused the server to not behave as expected. Other headers were being placed before the Content-Disposition headers of the parts and the Content-Disposition parameters were also not being enclosed in quotes. So by not including the content-type it in the parts and making sure the content-disposition headers were generated correctly eventually fixed the problem.
It is important to note the order of how the headers are added to the content so that the Content-Disposition header is set first for each part.
Working Code that generates the request in the desired format and gets the XML data.
[Test]
public async Task Post_Form() {
//Arrange
var stream = getXml();
var fileContent = new StreamContent(stream);
fileContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("form-data") {
Name = #"""filename""",
FileName = #"""req-details.xml""",
};
fileContent.Headers.ContentType = new MediaTypeHeaderValue("text/xml");
var stringContent = new ByteArrayContent(Encoding.UTF8.GetBytes("ZVWSBJXML"));
stringContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("form-data") {
Name = #"""VSS_SERV""",
};
//could have let system generate it but wanteed to rule it out as a problem
var boundary = "---------------------------" + DateTime.Now.Ticks.ToString("x", NumberFormatInfo.InvariantInfo);
var form = new MultipartFormDataContent(boundary);
//FIX: boundary quote issue
var contentType = form.Headers.ContentType.Parameters.First(o => o.Name == "boundary");
contentType.Value = contentType.Value.Replace("\"", String.Empty);
form.Add(stringContent);
form.Add(fileContent);
//var data = await form.ReadAsStringAsync(); //FOR TESTING PORPOSES ONLY!!
var client = createHttpClient("http://www.rzp.cz/");
//Act
var response = await client.PostAsync("cgi-bin/aps_cacheWEB.sh", form);
var body = await response.Content.ReadAsStringAsync();
//Assert
response.IsSuccessStatusCode.Should().BeTrue();
body.Should().NotBeEmpty();
var document = XDocument.Parse(body); //should be valid XML
document.Should().NotBeNull();
}
The code above generated the following request, which I extracted using fiddler (Pay close attention to the working format)
POST http://www.rzp.cz/cgi-bin/aps_cacheWEB.sh HTTP/1.1
User-Agent: System.Net.Http.HttpClient
Accept-Language: en-US, en; q=0.9
Accept: text/xml, application/xml
Cache-Control: max-age=0
Content-Type: multipart/form-data; boundary=---------------------------8d63c301f3e044f
Host: www.rzp.cz
Content-Length: 574
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
-----------------------------8d63c301f3e044f
Content-Disposition: form-data; name="VSS_SERV"
ZVWSBJXML
-----------------------------8d63c301f3e044f
Content-Disposition: form-data; name="filename"; filename="req-details.xml"
Content-Type: text/xml
<?xml version="1.0" encoding="iso-8859-2"?>
<VerejnyWebDotaz xmlns="urn:cz:isvs:rzp:schemas:VerejnaCast:v1" version="2.8">
<Kriteria>
<IdentifikacniCislo>75848899</IdentifikacniCislo>
<PlatnostZaznamu>0</PlatnostZaznamu>
</Kriteria>
</VerejnyWebDotaz>
-----------------------------8d63c301f3e044f--
Which was able to get the following response.
HTTP/1.1 200 OK
Date: Sat, 27 Oct 2018 21:17:50 GMT
Server: IIS
Serial: 10.145
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/xml;charset=ISO-8859-2
Content-Length: 931
<?xml version='1.0' encoding='iso-8859-2'?>
<VerejnyWebOdpoved xmlns="urn:cz:isvs:rzp:schemas:VerejnaCast:v1" version="2.8">
<Datum>27.10.2018</Datum>
<Kriteria>
<IdentifikacniCislo>75848899</IdentifikacniCislo>
<PlatnostZaznamu>0</PlatnostZaznamu>
</Kriteria>
<PodnikatelSeznam>
<PodnikatelID>212fbf8314e01506b0d7</PodnikatelID>
<ObchodniJmenoSeznam Popis="Jméno a příjmení:">Filip Zrůst</ObchodniJmenoSeznam>
<IdentifikacniCisloSeznam Popis="Identifikační číslo osoby:">75848899</IdentifikacniCisloSeznam>
<TypPodnikatele Popis="Typ podnikatele:">Fyzická osoba</TypPodnikatele>
<AdresaPodnikaniSeznam Popis="Adresa sídla:">Vlašská 358/7, 118 00, Praha 1 - Malá Strana</AdresaPodnikaniSeznam>
<RoleSubjektu Popis="Role subjektu:">podnikatel</RoleSubjektu>
<EvidujiciUrad Popis="Úřad příslušný podle §71 odst.2 živnostenského zákona:">Úřad městské části Praha 1</EvidujiciUrad>
</PodnikatelSeznam>
</VerejnyWebOdpoved>
From there it should be small work to parse the resulting XML as needed.
Supporting code
Generate or load the stream of the XML for the form
private static Stream getXml() {
var xml = #"<?xml version=""1.0"" encoding=""ISO-8859-2""?>
<VerejnyWebDotaz
xmlns=""urn:cz:isvs:rzp:schemas:VerejnaCast:v1""
version=""2.8"">
<Kriteria>
<IdentifikacniCislo>75848899</IdentifikacniCislo>
<PlatnostZaznamu>0</PlatnostZaznamu>
</Kriteria>
</VerejnyWebDotaz>";
var doc = XDocument.Parse(xml);//basically to validate XML
var stream = new MemoryStream();
doc.Save(stream);
stream.Position = 0;
return stream;
}
Was gradually able to whittle down the headers needed for a successful request after find the match that worked. Try removing others gradually to test if more can be removed safely to reduce the amount of unnecessary code needed.
private static HttpClient createHttpClient(string baseAddress) {
var handler = createHandler();
var client = new HttpClient(handler);
client.BaseAddress = new Uri(baseAddress);
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.TryAddWithoutValidation("User-Agent", "System.Net.Http.HttpClient");
client.DefaultRequestHeaders.TryAddWithoutValidation("Accept-Language", "en-US,en;q=0.9");
client.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "text/xml,application/xml");
client.DefaultRequestHeaders.ExpectContinue = false;
client.DefaultRequestHeaders.ConnectionClose = false;
client.DefaultRequestHeaders.CacheControl = new CacheControlHeaderValue() {
MaxAge = TimeSpan.FromSeconds(0)
};
return client;
}
private static HttpClientHandler createHandler() {
var handler = new HttpClientHandler();
// if the framework supports automatic decompression set automatic decompression
if (handler.SupportsAutomaticDecompression) {
handler.AutomaticDecompression = System.Net.DecompressionMethods.GZip |
System.Net.DecompressionMethods.Deflate;
}
return handler;
}
While I chose to use the asynchronous API of System.Net.Http, I found a similar question
Reference UploadFile with POST values by WebClient
With an answer that was done using WebClient that could be adapted to your question so that a request can be constructed similar to what was produced above.
I tried testing that one as well but got into the same forbidden error. Now that the correct format is know you should also be able to correctly craft a working request using WebClient/WebRequest
I am trying to flash a spark core from a C# application. I keep getting { error: Nothing to do? } response.
Below is my code
var url = string.Format("https://api.spark.io/v1/devices/{0}", sparkDeviceID);
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", accesstoken);
using (var formData = new MultipartFormDataContent())
{
HttpContent fileContent = new ByteArrayContent(Encoding.ASCII.GetBytes(rom));
//client.SendAsync()
formData.Add(fileContent, "file", "file");
var response = client.PutAsync(url, formData).Result;
if (!response.IsSuccessStatusCode)
throw new Exception("An error occurred during rom flash!");
var responseStream = response.Content.ReadAsStreamAsync().Result;
using (var reader = new StreamReader(responseStream, true))
{
var result = reader.ReadToEnd();
}
}
return true;
}
The documentation reads:
The API request should be encoded as multipart/form-data with a file field populated.
I believe the problem is the endpoint doesn't see the file. Any idea on how to resolve this?
Finally got it working.
The issue was the way .NET generated the content-disposition header for the file form data.
I used fiddler to compare the output of a successful put request to the put request that my code was generating:
Successful PUT request generated using CURL:
PUT http://127.0.0.1:8888/ HTTP/1.1
User-Agent: curl/7.33.0
Host: 127.0.0.1:8888
Accept: */*
Content-Length: 2861
Expect: 100-continue
Content-Type: multipart/form-data; boundary=------------------------5efcf64a370f13c8
--------------------------5efcf64a370f13c8
Content-Disposition: form-data; name="file"; filename="ms.ino"
Content-Type: application/octet-stream
...
My PUT request (unsuccessful):
PUT https://api.spark.io/v1/devices/{deviceid} HTTP/1.1
Authorization: Bearer {access_token}
Content-Type: multipart/form-data; boundary="135f5425-9342-4ffa-a645-99c04834026f"
Host: api.spark.io
Content-Length: 2878
Expect: 100-continue
--135f5425-9342-4ffa-a645-99c04834026f
Content-Type: application/octet-stream
Content-Disposition: form-data; name=file; filename=file.ino; filename*=utf-8''file.ino
...
If you'll notice the difference in the Content-Type for the actual file being sent:
Successful: Content-Disposition: form-data; name="file"; filename="ms.ino"
Unsuccessful: Content-Disposition: form-data; name=file; filename=file.ino; filename*=utf-8''file.ino
Most specifically, the resolution was to add quotes around the name attribute.
Resolution:
formData.Add(fileContent, "\"file\"", "file.ino");
I have the following code in a Windows Phone 8 project that uses RestSharp client:
public async Task<string> DoMultiPartPostRequest(String ext, JSonWriter jsonObject, ObservableCollection<Attachment> attachments)
{
var client = new RestClient(DefaultUri);
// client.Authenticator = new HttpBasicAuthenticator(username, password);
var request = new RestRequest(ext, Method.POST);
request.RequestFormat = DataFormat.Json;
request.AddParameter("json", jsonObject.ToString(), ParameterType.GetOrPost);
// add files to upload
foreach (var a in attachments)
request.AddFile("attachment", a.FileBody, "attachment.file", a.ContType);
var content = await client.GetResponseAsync(request);
if (content.StatusCode != HttpStatusCode.OK)
return "error";
return content.Content;
}
Fiddler shows the generated header:
POST http://192.168.1.101:9000/rayz/create HTTP/1.1
Content-Type: multipart/form-data; boundary=-----------------------------28947758029299
Content-Length: 71643
Accept-Encoding: identity
Accept: application/json, application/xml, text/json, text/x-json, text/javascript, text/xml
User-Agent: RestSharp 104.1.0.0
Host: 192.168.1.101:9000
Connection: Keep-Alive
Pragma: no-cache
-------------------------------28947758029299
Content-Disposition: form-data; name="json"
{
"userId": "2D73B43390041E868694A85A65E47A09D50F019C180E93BAACC454488F67A411",
"latitude": "35.09",
"longitude": "33.30",
"accuracy": "99",
"maxDistance": "dist",
"Message": "mooohv"
}
-------------------------------28947758029299
Content-Disposition: form-data; name="attachment"; filename="attachment.file"
Content-Type: image/jpeg
?????JFIF??`?`?????C? $" &0P40,,0bFJ:Ptfzxrfpn????????np????????|????????????C"$$0*0^44^?p??????????????????????????????????????????????????????`?"??????????????
-------------------------------28947758029299
The code above works fine on the Play2 API. However since the RestSharp does not seem to be stable I have decided to use the native HttpClient provided by Microsoft.
Hence I wrote another function that uses HttpClient to do the same job:
public async Task<string> DoMultiPartPostRequest2(String ext, JSonWriter jsonObject,
ObservableCollection<Attachment> attachments)
{
var client = new HttpClient();
var content = new MultipartFormDataContent();
var json = new StringContent(jsonObject.ToString());
content.Add(json, "json");
foreach (var a in attachments)
{
var fileContent = new StreamContent(new MemoryStream(a.FileBody));
fileContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("form-data")
{
Name = "attachment",
FileName = "attachment.file"
};
fileContent.Headers.ContentType = MediaTypeHeaderValue.Parse(a.ContType);
content.Add(fileContent);
}
var resp = await client.PostAsync(DefaultUri + ext, content);
if (resp.StatusCode != HttpStatusCode.OK)
return "error";
var reponse = await resp.Content.ReadAsStringAsync();
return reponse;
}
The header that is generated from that code is the following:
POST http://192.168.1.101:9000/rayz/create HTTP/1.1
Accept: */*
Content-Length: 6633
Accept-Encoding: identity
Content-Type: multipart/form-data; boundary="e01b2196-d24a-47a2-a99b-e82cc4a2f92e"
User-Agent: NativeHost
Host: 192.168.1.101:9000
Connection: Keep-Alive
Pragma: no-cache
--e01b2196-d24a-47a2-a99b-e82cc4a2f92e
Content-Type: text/plain; charset=utf-8
Content-Disposition: form-data; name=json
{
"userId": "2D73B43390041E868694A85A65E47A09D50F019C180E93BAACC454488F67A411",
"latitude": "35.09",
"longitude": "33.30",
"accuracy": "99",
"maxDistance": "dist",
"Message": "test"
}
--e01b2196-d24a-47a2-a99b-e82cc4a2f92e
Content-Disposition: form-data; name=attachment; filename=attachment.file
Content-Type: image/jpeg
?????JFIF??`?`?????C? $" &0P40,,0bFJ:Ptfzxrfpn????????np????????|????????????C"$$0*0^44^?p????????????????????????????????????????????????????????"??????????????
--e01b2196-d24a-47a2-a99b-e82cc4a2f92e--
So far so good. From my point of view the two headers seem to be identical.
However when I debug the Play 2 API after executing Http.MultipartFormData body = request().body().asMultipartFormData(); I noticed that the multipart data are not being parsed correctly.
More specifically the multipart filed in the body variable is as follows:
MultipartFormData(Map(),List(),List(BadPart(Map(ntent-type -> text/plain; charset=utf-8, content-disposition -> form-data; name=json)), BadPart(Map()), BadPart(Map()), BadPart(Map()), BadPart(Map())),List())
As you can notice it has several (actually 5 in this example) BadParts.
Example: BadPart(Map(ntent-type -> text/plain; charset=utf-8, content-disposition -> form-data; name=json))
Can anyone see what is going wrong here? Is the header generated by HttpClient wrong?
Here is the solution.. (hack)
There seems to be a problem with Play Framework when the boundary has quotes in it.
So i added the following code after multipart is created in order to remove them:
var content = new MultipartFormDataContent();
foreach (var param in content.Headers.ContentType.Parameters.Where(param => param.Name.Equals("boundary")))
param.Value = param.Value.Replace("\"", String.Empty);
Finally i had to add quotes "\"" manually to specific values on the header like the following:
Original: Content-Disposition: form-data; name=attachment; filename=attachment.file
Changed to: Content-Disposition: form-data; name="attachment"; filename="attachment.file"
and
Original: Content-Disposition: form-data; name=json
Changed to: Content-Disposition: form-data; name="json"
I don't think that its a mistake to have quotes or not anywhere in the header and maybe the parsing on play framework should be fixed accordingly.