I am trying to make a call from my Python client to my C#.NET api endpoint.
The following two code blocks work correctly:
# Python Client
endpoint = "http://localhost:12345/api/myController/hmacTestPut"
data = {"type": "my_type", "state": "my_state"}
headers = calc_hmac(...)
r = requests.put(url=endpoint, json=data, headers=headers)
// C# Endpoint
[HttpPut]
[Route("api/myController/hmacTestPut")]
//[HMACAuthentication]
public IHttpActionResult HMACTestPut(MyDTO obj)
{
return Ok(obj)
}
However, when I include the HMACAuthentication attribute on the C# endpoint, the MyDTO obj is null.
Since the endpoint is hit in both scenarios, I believe my HMACAuthenticationAttribute is correct.
Including [FromBody] does not seem to have an effect either way.
Is there a reason why adding HMACAuthentication to an endpoint would stop the body content from being received?
In case anybody else runs into this issue, this problem was (obviously) HMAC related.
In my HMACAuthenticationAttribute, in my ComputeHash() function, which is where I calculate the hash for the body of the request (as in POST or PUT body).
What was breaking is below, where reading the HttpContent to a MemoryStream left the HttpContent of the request empty for the actual endpoint to read nothing:
var ms = new MemoryStream();
await httpContent.copyToAsync(ms);
ms.Seek(0, SeekOrigin.Begin);
var content = ms.ToArray();
byte[] hash = md5.ComputeHash(content)
The solution was to change how we read in the content of the HttpRequest:
var content = await httpContent.ReadAsByteArrayAsync().ConfigureAwait(false);
byte[] hash = md5.ComputeHash(content)
Full simplified function below, in hopes that it helps somebody.
private static async Task<byte[]> ComputeHash(HttpContent httpContent)
{
using (MD5 md5 = MD5.Create())
{
byte[] hash = null;
var content = await httpContent.ReadAsByteArrayAsync().ConfigureAwait(false);
if (content.Length != 0)
{
hash = md5.ComputeHash(content);
}
return hash;
}
}
Related
I have a .NET Core 2.2 API Controller method that returns a FileContentResult like so:
[HttpGet]
public async Task<IActionResult> Get()
{
return File(_service.GetCsvFile(), "text/csv", "results.csv");
}
Also, I have an integration test that uses a WebHostFixture and httpClient to get the results. However, the content is always empty, whatever method I try. While there is a file with content being returned:
var response = await WebHost.Client.GetAsync(url, HttpCompletionOption.ResponseContentRead);
var result = await response.Content.ReadAsByteArrayAsync();
Also, when I do the following, no content is being read:
var response = await WebHost.Client.GetByteArrayAsync(url);
All other integration tests using this WebHostFixture run successful, so it seems this is the only endpoint that is not returning its data properly in my test environment. When I call the endpoint using Postman, a CSV file with content is being returned properly.
Also, in Postman, besides receiving the binary data in the body, the following headers are returned:
However, in my test method, after calling the GetAsync method, the
response.Content.Headers.ContentType is "text/csv", but the response.Content.Headers.ContentLength is 0.
What am I doing wrong and why is there no binary data being returned here?
Try this:
byte[] BytesResult = await httpClient.GetByteArrayAsync("YourRequestUri").ConfigureAwait(false);
Convert it to string and compare the result. It works great with approval tests.
Example:
var result = fileresult as FileContentResult;
byte[] buffer = result.FileContents;
string s = System.Text.Encoding.UTF8.GetString(buffer, 0, buffer.Length);
When reading a file you need to use GetStreamAsync
Example:
using (var csvStream = await WebHost.Client.GetStreamAsync(url))
{
// Do something with stream contents.
}
I have a PUT endpoint which receives a payload and coverts it to an object using [FromBody].
This works fine when running on localhost (using IIS Express).
But when I run it via the production server (IIS), it fails. The validation error is:
{
"": [
"A non-empty request body is required."
]
}
I can recreate this with both my client code and using Postman.
Thinking that maybe the payload was malformed, I used some middleware (posted elsewhere on Stack Overflow) to inspect the body before it reached the endpoint (see code below), but even at that stage the request.body is empty.
I can't see any other places where the request has already been read at this stage (as I appreciate that can clear it).
I've read countless Stack Overflow posts and other web pages, and nothing else seems to come close to this situation (where request.body is empty, even though you can see in the client the payload is sent).
Any help would be greatly appreciated. I'm happy to provide any further details.
For reference, here is the code.
Payload
{"currentlySaved":false,"type":"album"}
End point
// PUT api/<controller>/toggleSaveState
[HttpPut("toggleSaveState/{id}")]
public async Task<IActionResult> Put(string id, [FromBody] ToggleSaveStateRequest requestDetails)
{
...
}
The object representing the payload:
public class ToggleSaveStateRequest
{
public bool CurrentlySaved { get; set; }
public string Type { get; set; }
}
Middleware used to check the request.body
Note: the problem takes place whether I include this middleware or not. I've included it in a bid to diagnose the problem.
public class RequestResponseLoggingMiddleware
{
private readonly RequestDelegate _next;
public RequestResponseLoggingMiddleware(RequestDelegate next)
{
_next = next;
}
public async Task Invoke(HttpContext context)
{
//First, get the incoming request
var request = await FormatRequest(context.Request);
Console.WriteLine($"REQUEST: {request}");
//Copy a pointer to the original response body stream
var originalBodyStream = context.Response.Body;
//Create a new memory stream...
using (var responseBody = new MemoryStream())
{
//...and use that for the temporary response body
context.Response.Body = responseBody;
//Continue down the Middleware pipeline, eventually returning to this class
await _next(context);
//Format the response from the server
var response = await FormatResponse(context.Response);
//TODO: Save log to chosen datastore
Console.WriteLine($"RESPONSE: {response}");
//Copy the contents of the new memory stream (which contains the response) to the original stream, which is then returned to the client.
await responseBody.CopyToAsync(originalBodyStream);
}
}
private async Task<string> FormatRequest(HttpRequest request)
{
var body = request.Body;
//This line allows us to set the reader for the request back at the beginning of its stream.
request.EnableRewind();
//We now need to read the request stream. First, we create a new byte[] with the same length as the request stream...
var buffer = new byte[Convert.ToInt32(request.ContentLength)];
//...Then we copy the entire request stream into the new buffer.
await request.Body.ReadAsync(buffer, 0, buffer.Length);
//We convert the byte[] into a string using UTF8 encoding...
var bodyAsText = Encoding.UTF8.GetString(buffer);
//..and finally, assign the read body back to the request body, which is allowed because of EnableRewind()
request.Body = body;
return $"{request.Scheme} {request.Host}{request.Path} {request.QueryString} {bodyAsText}";
}
private async Task<string> FormatResponse(HttpResponse response)
{
//We need to read the response stream from the beginning...
response.Body.Seek(0, SeekOrigin.Begin);
//...and copy it into a string
string text = await new StreamReader(response.Body).ReadToEndAsync();
//We need to reset the reader for the response so that the client can read it.
response.Body.Seek(0, SeekOrigin.Begin);
//Return the string for the response, including the status code (e.g. 200, 404, 401, etc.)
return $"{response.StatusCode}: {text}";
}
}
Edit:
Version of endpoint without the [FromBody]:
The following is here to try and simplify my explanation.
// PUT api/<controller>/toggleSaveState
[HttpPut("toggleSaveState/{id}")]
public async Task<IActionResult> Put(string id)
{
using (var reader = new StreamReader(Request.Body))
{
var body = reader.ReadToEnd();
Console.WriteLine("body", body);
}
...
return Ok();
}
Given the above endpoint (with no middleware in place), when it's run locally, body has the value of:
{"currentlySaved":false,"type":"album"}
However, when it is run remotely, it is empty.
Problem with your code is here:
//Copy a pointer to the original response body stream
var originalBodyStream = context.Response.Body;
You are assigning a reference to the response stream, and then you are overwriting(that stream) it with an empty memory stream here:
//...and use that for the temporary response body
context.Response.Body = responseBody;
So the reference in originalBodyStream variable, is now pointing to that empty stream as well. Other words - at this point the content of context.Response.Body is lost.
If you want to copy the request body stream to the other stream, use below:
await context.Response.Body.CopyToAsync(originalBodyStream);
Remember that streams are reference type so this:
var bodyVar = request.Body;
assigns only a reference to the stream(not the value!) - other words any modifications on the request.Body will be reflected on the bodyVar as it points to the same place in memory.
I have a .Net Core 2.0 application that is sending files to a Web API endpoint, using multipart content. Everywhere I've looked, such as C# HttpClient 4.5 multipart/form-data upload, makes it seem that it should be as easy as passing a FileStream to a StreamContent. However, when I make the post, it looks like the file is attaching as text, not bits.
Actual code:
var request = new HttpRequestMessage()
{
Method = HttpMethod.Post,
RequestUri = new Uri( "http://localhost:10442/filetest" )
};
var multiContent = new MultipartFormDataContent();
var filestream = File.OpenRead( path );
var filename = Path.GetFileName( path );
var streamContent = new StreamContent( filestream );
streamContent.Headers.Add( "Content-Type", "application/octet-stream" );
streamContent.Headers.Add( "Content-Disposition", $"form-data; name=\"file1\"; filename=\"{filename}\"" );
multiContent.Add( streamContent, "file", filename );
request.Content = multiContent;
var response = await new HttpClient().SendAsync( request );
The request looks like this which, as you may notice, is not all on one line (which I think is a/THE problem):
POST http://localhost:10442/filetest HTTP/1.1
Content-Type: multipart/form-data; boundary="c5295887-425d-4ec7-8638-20c6254f9e4b"
Content-Length: 88699
Host: localhost:10442
--c5295887-425d-4ec7-8638-20c6254f9e4b
Content-Type: application/octet-stream
Content-Disposition: form-data; name="file1"; filename="somepdf.pdf"
%PDF-1.7
%
1 0 obj
<</Type/Catalog/Version/1.7/Pages 3 0 R/Outlines 2 0 R/Names 8 0 R/Metadata 31 0 R>>
endobj
Fiddler shows the entire post all the way down to the end boundary, but await Request.Content.ReadAsStringAsync() in the endpoint only shows the first couple dozen bytes (it looks as if the stream wasn't finished, but if Fiddler got it all, shouldn't my endpoint have too?).
I was having similar trouble trying to hit a remote endpoint; I built this endpoint to test locally.
The exception I'm getting is:"Unexpected end of MIME multipart stream. MIME multipart message is not complete." To me, this makes sense both if I'm really only getting part of my stream, or if the line breaks are throwing something off.
I have also tried throwing some of the Idisposables into Usings but, as expected, that closes the streams and I get exceptions that way.
And for completeness's sake, here's the endpoint I'm calling:
public async void ReceiveFiles()
{
// exception happens here:
var mpData = await Request.Content.ReadAsMultipartAsync();
await Task.FromResult( 0 );
}
Try something like this:
static int Main(string[] args)
{
var request = new HttpRequestMessage()
{
Method = HttpMethod.Post,
RequestUri = new Uri("http://localhost:10442/filetest")
};
var path = "c:\\temp\\foo.bak";
using (var filestream = File.OpenRead(path))
{
var length = filestream.Length.ToString();
var streamContent = new StreamContent(filestream);
streamContent.Headers.Add("Content-Type", "application/octet-stream");
streamContent.Headers.Add("Content-Length", length);
request.Content = streamContent;
Console.WriteLine($"Sending {length} bytes");
var response = new HttpClient().SendAsync(request).Result;
Console.WriteLine(response.Content.ReadAsStringAsync().Result);
}
Console.WriteLine("Hit any key to exit");
Console.ReadKey();
return 0;
}
and
[HttpPost]
public async Task<IActionResult> Upload()
{
var buf = new byte[1024 * 64];
long totalBytes = 0;
using (var rs = Request.Body)
{
while (1 == 1)
{
int bytesRead = await rs.ReadAsync(buf, 0, buf.Length);
if (bytesRead == 0) break;
totalBytes += bytesRead;
}
}
var uploadedData = new
{
BytesRead = totalBytes
};
return new JsonResult(uploadedData) ;
}
I'm trying to solve a similar issue, and I'm not 100% to a solution yet, but maybe some of my research can help you.
It was helpful to me to read through the microsoft docs for .NET core file uploads, specifically for large files that use streams and multipart form data:
https://learn.microsoft.com/en-us/aspnet/core/mvc/models/file-uploads?view=aspnetcore-2.1#uploading-large-files-with-streaming
You already referenced it, but there's some relevant useful information in this answer:
C# HttpClient 4.5 multipart/form-data upload
This explains the details of the content-disposition header and how it is used with multipart form data requests: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition#As_a_header_for_a_multipart_body
As to your specific problem of the file being sent as text instead of bits, since http is text-based, it can only be sent as text, but that text can be encoded as you see fit. Perhaps your StreamContent needs a specific encoding to be used, like base64 encoding or similar? I do believe the newlines are significant in the multipart request, so hopefully setting the encoding for the file content as needed would be enough.
Another possibility: could it be that you need to set additional information on the file section's headers or in the definition of the StreamContent to indicate that it should expect to continue, or that the boundary information isn't put in correctly? See Multipart forms from C# client
I use this lib : https://github.com/jgiacomini/Tiny.RestClient
It's make easier to send multiplart file to send multipart files.
Here a sample :
await client.PostRequest("MultiPart/Test").
AsMultiPartFromDataRequest().
AddStream(stream1, "request", "request2.bin").
AddStream(stream2, "request", "request2.bin")
ExecuteAsync();
Hello all am trying to do a login to my xamarin api using RestSharp, the API ought to return status code 200 OK if the authentication works and status code 415 if the authentication fails(wrong password) and other codes depending on what the case scenario, but instead i get a status code 0 on all other case asides when the authentication pass(status code 200 ok), the source code below is how i implement
//payload am sending to the api
RequestPayload res = new RequestPayload();
res.appid = appid;
res.data = data;
res.method = "Login";
//convert to json object
var MySerializedObject = JsonConvert.SerializeObject(res);
string APIUrl = ""http://142.168.20.15:8021/RouteTask";
//create client
RestClient client = new RestClient(APIUrl);
//create request
RestRequest request = new RestRequest(Method.POST);
// set request headeer
request.AddHeader("Content-Type", "application/x-www-form-urlencoded");
//request.AddJsonBody(MySerializedObject); --i have also tried this
request.AddParameter("application/json", MySerializedObject, ParameterType.RequestBody);
request.JsonSerializer.ContentType = "application/json; charset=utf-8";
request.AddParameter("RequestSource", "Web", "application/json", ParameterType.QueryString);
client.Timeout = 2000000;
var response = client.Execute(request); // where the issue appears
//RestResponse response = client.Execute(request); // i have tried this
//IRestResponse response = client.Execute(request); // i have tried this
if (response.IsSuccessful)
{
//use response data
}
on all scenerio it comes back with a StatusCode: 0, Content-Type: , Content-Length: 0) and errorMessage
"Error getting response stream (ReadAsync): ReceiveFailure Value
cannot be null. Parameter name: src"
screenshot below indicate when the api call fails
Response receieved when the authentication is valid
I was finally able to find a workaround for this. Bear with the long-winded response.
The tags mention Xamarin, which is what I am working in as well - specifically with iOS. I think it may actually be a bug with Mono, but I didn't take it that far to confirm.
The problem lies with the default way of copying the response buffer. In the RestSharp code, this is done by an extension method in MiscExtensions.cs called ReadAsBytes. It appears that with certain response buffers, the call to the Stream.Read method is failing. When this happens, the exception causes RestSharp to "shortcut" the rest of the processing on the response, hence the status code never gets filled in since it happens after the call to ReadAsBytes.
The good news is RestSharp does give a way to replace this call to ReadAsBytes with one of your own. This is done via the ResponseWriter property on the IRestRequest object. If it has a function defined, it will bypass the ReadAsBytes call and call the function you gave it instead. The problem is, this is defined as an Action and you don't get a copy of the full response object, so it's somewhat useless. Instead you have to use the AdvancedResponseWriter property. This one includes both the response object and the response stream. But you still have to set the ResponseWriter property or it won't bypass the default handler and you'll still get the error.
Ok, so how do you make this work? I ended up implementing it as a wrapper to RestClient so I wouldn't have to implement the code all over the place. Here's the basic setup:
public class MyRestClient : RestClient
{
public MyRestClient(string baseUrl) : base(baseUrl)
{ }
public override IRestResponse Execute(IRestRequest request)
{
request.ResponseWriter = s => { };
request.AdvancedResponseWriter = (input, response) => response.RawBytes = ReadAsBytes(input);
return base.Execute(request);
}
private static byte[] ReadAsBytes(Stream input)
{
var buffer = new byte[16 * 1024];
using (var ms = new MemoryStream())
{
int read;
try
{
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{ ms.Write(buffer, 0, read); }
return ms.ToArray();
}
catch (WebException ex)
{ return Encoding.UTF8.GetBytes(ex.Message); }
};
}
}
The ReadAsBytes method is actually just a copy/paste of the RestSharp ReadAsBytes method with the addition of a try/catch. If it fails, it returns the exception reason in to the response buffer. This may or may not be what you want, so modify as needed. You may also need to override other methods for Execute, but in my case this is the only one we're using so it was enough.
So far this seems to be doing the trick for me. Perhaps if someone got ambitious they could trace it all the way in to Mono to try and see what it doesn't like about the stream, but I don't have the time for it at the moment.
Good luck!
OK so after toying around with RestSharp for a bit, i realize just as #steve_In_Co mentioned earlier there were compatibility issues with MONO (we presume this is a bug) so i did it in a basic way using the .Net HTTP library and it works for me, so in case someone is still looking for a way out, find the working .net http implementation code below.
//payload am sending to the api
RequestPayload res = new RequestPayload();
res.appid = appid;
res.data = data;
res.method = "Login";
//convert to json object
var MySerializedObject = JsonConvert.SerializeObject(res);
string APIUrl = ""http://142.168.20.15:8021/RouteTask";
//create basic .net http client
HttpClient client = new HttpClient();
client.BaseAddress = new Uri(APIUrl);
// this was required in the header of my request,
// you may not need this, or you may need to adjust parameter
//("RequestSource","Web") or you own custom headers
client.DefaultRequestHeaders.Add("RequestSource", "Web");
// this class is custom, you can leave it out
connectionService = new ConnectionService();
//check for internet connection on users device before making the call
if (connectionService.IsConnected)
{
//make the call to the api
HttpResponseMessage response = await
client.PostAsJsonAsync(ApiConstants.APIDefault, res);
if (response.IsSuccessStatusCode)
{
string o = response.Content.ReadAsStringAsync().Result;
dynamic payload = JsonConvert.DeserializeObject(o);
string msg = payload["valMessage"];
resp.a = true;
resp.msg = payload["responseDescription"];
}
else
{
string o = response.Content.ReadAsStringAsync().Result;
dynamic payload = JsonConvert.DeserializeObject(o);
resp.a = false;
resp.msg = payload["response"];
}
}
I'm trying to implement HMAC security for an API. Everything works fine until I try to post a file.
The HMAC solution can be found here - https://github.com/gavinharriss/WebAPI.HMAC - it's a fork from the original to allow GET requests as well as POST requests.
The code to attach a file:
var requestContent = new MultipartFormDataContent();
var fileContent = new ByteArrayContent(file);
requestContent.Add(fileContent, "file", filename);
if I immediately call HttpContent.ReadAsByteArrayAsync() there is no issue, the byte array is available.
However, the HMAC HttpClient (HMACHttpClient) implements a DelegatingHandler (HMACDelegatingHandler) in order to attach the HMAC header to requests.
In the HMACDelegatingHandler the request is passed along as a HttpRequestMessage from which the HttpRequestMessage.Content property is used in a helper to build the HMAC signature.
When building the signature, the following code is called from a helper class:
private static async Task<byte[]> ComputeHash(HttpContent httpContent)
{
using (var md5 = MD5.Create())
{
byte[] hash = null;
if (httpContent != null)
{
var content = await httpContent.ReadAsByteArrayAsync(); // <-- Fails here
if (content.Length != 0)
{
hash = md5.ComputeHash(content);
}
}
return hash;
}
}
When stepping through the code the var content = await httpContent.ReadAsByteArrayAsync() line is hit, then nothing, no error. The requests just seems to go poof but everything is still running and the HttpClient request never gets sent.
Any ideas what's going on?
Having tested this with various sizes of file, I found the issue arose when files got around the 50,000 byte mark.
This post provided a solution: HttpContent.ReadAsStringAsync causes request to hang (or other strange behaviours).
If you replace erroring line in HMACHelper (line 66):
var content = await httpContent.ReadAsByteArrayAsync();
with this:
var ms = new MemoryStream();
await httpContent.CopyToAsync(ms);
ms.Seek(0, SeekOrigin.Begin);
var content = ms.ToArray();
It should stop hanging.