I have a PUT endpoint which receives a payload and coverts it to an object using [FromBody].
This works fine when running on localhost (using IIS Express).
But when I run it via the production server (IIS), it fails. The validation error is:
{
"": [
"A non-empty request body is required."
]
}
I can recreate this with both my client code and using Postman.
Thinking that maybe the payload was malformed, I used some middleware (posted elsewhere on Stack Overflow) to inspect the body before it reached the endpoint (see code below), but even at that stage the request.body is empty.
I can't see any other places where the request has already been read at this stage (as I appreciate that can clear it).
I've read countless Stack Overflow posts and other web pages, and nothing else seems to come close to this situation (where request.body is empty, even though you can see in the client the payload is sent).
Any help would be greatly appreciated. I'm happy to provide any further details.
For reference, here is the code.
Payload
{"currentlySaved":false,"type":"album"}
End point
// PUT api/<controller>/toggleSaveState
[HttpPut("toggleSaveState/{id}")]
public async Task<IActionResult> Put(string id, [FromBody] ToggleSaveStateRequest requestDetails)
{
...
}
The object representing the payload:
public class ToggleSaveStateRequest
{
public bool CurrentlySaved { get; set; }
public string Type { get; set; }
}
Middleware used to check the request.body
Note: the problem takes place whether I include this middleware or not. I've included it in a bid to diagnose the problem.
public class RequestResponseLoggingMiddleware
{
private readonly RequestDelegate _next;
public RequestResponseLoggingMiddleware(RequestDelegate next)
{
_next = next;
}
public async Task Invoke(HttpContext context)
{
//First, get the incoming request
var request = await FormatRequest(context.Request);
Console.WriteLine($"REQUEST: {request}");
//Copy a pointer to the original response body stream
var originalBodyStream = context.Response.Body;
//Create a new memory stream...
using (var responseBody = new MemoryStream())
{
//...and use that for the temporary response body
context.Response.Body = responseBody;
//Continue down the Middleware pipeline, eventually returning to this class
await _next(context);
//Format the response from the server
var response = await FormatResponse(context.Response);
//TODO: Save log to chosen datastore
Console.WriteLine($"RESPONSE: {response}");
//Copy the contents of the new memory stream (which contains the response) to the original stream, which is then returned to the client.
await responseBody.CopyToAsync(originalBodyStream);
}
}
private async Task<string> FormatRequest(HttpRequest request)
{
var body = request.Body;
//This line allows us to set the reader for the request back at the beginning of its stream.
request.EnableRewind();
//We now need to read the request stream. First, we create a new byte[] with the same length as the request stream...
var buffer = new byte[Convert.ToInt32(request.ContentLength)];
//...Then we copy the entire request stream into the new buffer.
await request.Body.ReadAsync(buffer, 0, buffer.Length);
//We convert the byte[] into a string using UTF8 encoding...
var bodyAsText = Encoding.UTF8.GetString(buffer);
//..and finally, assign the read body back to the request body, which is allowed because of EnableRewind()
request.Body = body;
return $"{request.Scheme} {request.Host}{request.Path} {request.QueryString} {bodyAsText}";
}
private async Task<string> FormatResponse(HttpResponse response)
{
//We need to read the response stream from the beginning...
response.Body.Seek(0, SeekOrigin.Begin);
//...and copy it into a string
string text = await new StreamReader(response.Body).ReadToEndAsync();
//We need to reset the reader for the response so that the client can read it.
response.Body.Seek(0, SeekOrigin.Begin);
//Return the string for the response, including the status code (e.g. 200, 404, 401, etc.)
return $"{response.StatusCode}: {text}";
}
}
Edit:
Version of endpoint without the [FromBody]:
The following is here to try and simplify my explanation.
// PUT api/<controller>/toggleSaveState
[HttpPut("toggleSaveState/{id}")]
public async Task<IActionResult> Put(string id)
{
using (var reader = new StreamReader(Request.Body))
{
var body = reader.ReadToEnd();
Console.WriteLine("body", body);
}
...
return Ok();
}
Given the above endpoint (with no middleware in place), when it's run locally, body has the value of:
{"currentlySaved":false,"type":"album"}
However, when it is run remotely, it is empty.
Problem with your code is here:
//Copy a pointer to the original response body stream
var originalBodyStream = context.Response.Body;
You are assigning a reference to the response stream, and then you are overwriting(that stream) it with an empty memory stream here:
//...and use that for the temporary response body
context.Response.Body = responseBody;
So the reference in originalBodyStream variable, is now pointing to that empty stream as well. Other words - at this point the content of context.Response.Body is lost.
If you want to copy the request body stream to the other stream, use below:
await context.Response.Body.CopyToAsync(originalBodyStream);
Remember that streams are reference type so this:
var bodyVar = request.Body;
assigns only a reference to the stream(not the value!) - other words any modifications on the request.Body will be reflected on the bodyVar as it points to the same place in memory.
Related
I've an ASP.NET Core server that, upon client request, start fetching a stream from an AXIS camera and returns it to the client for displaying. If the http request to server is made directly by the browser by means of img src attribute, it works fine. But if I use HttpClient (which I need to do since I need CancellationToken), the httpClient.GetStreamAsync instruction get stuck and I cannot parse the returned data.
Server side Controller (take from here):
[ApiController]
[Route("[controller]")]
public class CameraSystemController : ControllerBase
{
private string _contentTypeStreaming = "multipart/x-mixed-replace;boundary=myboundary";
private HttpClient _httpClient = new HttpClient(new HttpClientHandler { Credentials = ...});
[HttpGet("getStream")]
public async Task<IActionResult> GetStream(CancellationToken token)
{
Stream stream = await _httpClient.GetStreamAsync("http://.../axis-cgi/mjpg/video.cgi?resolution=800x600&compression=50", token);
if (stream != null) {
FileStreamResult result = new FileStreamResult(stream, _contentTypeStreaming) {
EnableRangeProcessing = true
};
return result;
} else {
return new StatusCodeResult((int)HttpStatusCode.ServiceUnavailable);
}
}
}
Now, as I said, as long as I make the browser perform the http request by means of
// LiveCamera.razor
<img src="CameraSystem/getStream" onerror="#OnImgLoadingError" alt="">
Data acquired in live mode by the camera is correctly displayed in browser.
If instead I make the request in the client this way:
//LiveCamera.razor.cs
[Inject] public HttpClient Http { get; private set; }
private CancellationTokenSource _cancellationTokenSource = new CancellationTokenSource();
private async void StartStreamingRequest(object sender, ElapsedEventArgs e) {
Console.WriteLine("I'm about to request live stream");
Stream responseStream = await Http.GetStreamAsync("CameraSystem/getStream", _cancellationTokenSource.Token);
Console.WriteLine("Header found!");
string boundary = "myboundary";
for (MultipartReader streamReader = new MultipartReader(boundary, responseStream); ;) {
MultipartSection nextFrame = await streamReader.ReadNextSectionAsync(_cancellationTokenSource.Token);
DisplayOneFrameCallback(nextFrame.Body);
}
}
private void DisplayOneFrameCallback(Stream body)
{
StreamReader reader = new StreamReader(body);
string frameData = reader.ReadToEnd();
_imgSrc = "data:image/jpeg;base64," + frameData;
StateHasChanged();
}
In this seconds case the request is performed correctly (server-side I can see the code executed and from task manager I can see the bandwidth usage increasing) but the client got stuck on await instruction, and subsequent code is never executed. Now, in Microsoft documentation under GetStreamAsync it is possible to read
This operation will not block. The returned Task object will complete after the response headers are read. This method does not read nor buffer the response body.
so I would expect it not to block. Thus I suppose there could be a problem in the header I'm producing server-side, even though browser request works just fine.
Just out of curiosity, I've captured with Wireshark the data between the camera and the server. The header and the initial part of the body are like this:
..-...HTTP/1.0 200 OK..Cache-Control: no-cache..Pragma: no-cache..Expires: Thu, 01 Dec 1994 16:00:00 GMT..Connection: close..Content-Type: multipart/x-mixed-replace;boundary=myboundary....--myboundary..Content-Type: image/jpeg..Content-Length: 30146......
I've double-checked with browser developer tools and I can confirm that sending the request from browser or from httpClient yield the same exact request. Furthermore right-clicking on the request url and issueing "open in new tab" opens a tab where I can see the camera live stream in both cases.
Could you please help me in this?
I am trying to make a call from my Python client to my C#.NET api endpoint.
The following two code blocks work correctly:
# Python Client
endpoint = "http://localhost:12345/api/myController/hmacTestPut"
data = {"type": "my_type", "state": "my_state"}
headers = calc_hmac(...)
r = requests.put(url=endpoint, json=data, headers=headers)
// C# Endpoint
[HttpPut]
[Route("api/myController/hmacTestPut")]
//[HMACAuthentication]
public IHttpActionResult HMACTestPut(MyDTO obj)
{
return Ok(obj)
}
However, when I include the HMACAuthentication attribute on the C# endpoint, the MyDTO obj is null.
Since the endpoint is hit in both scenarios, I believe my HMACAuthenticationAttribute is correct.
Including [FromBody] does not seem to have an effect either way.
Is there a reason why adding HMACAuthentication to an endpoint would stop the body content from being received?
In case anybody else runs into this issue, this problem was (obviously) HMAC related.
In my HMACAuthenticationAttribute, in my ComputeHash() function, which is where I calculate the hash for the body of the request (as in POST or PUT body).
What was breaking is below, where reading the HttpContent to a MemoryStream left the HttpContent of the request empty for the actual endpoint to read nothing:
var ms = new MemoryStream();
await httpContent.copyToAsync(ms);
ms.Seek(0, SeekOrigin.Begin);
var content = ms.ToArray();
byte[] hash = md5.ComputeHash(content)
The solution was to change how we read in the content of the HttpRequest:
var content = await httpContent.ReadAsByteArrayAsync().ConfigureAwait(false);
byte[] hash = md5.ComputeHash(content)
Full simplified function below, in hopes that it helps somebody.
private static async Task<byte[]> ComputeHash(HttpContent httpContent)
{
using (MD5 md5 = MD5.Create())
{
byte[] hash = null;
var content = await httpContent.ReadAsByteArrayAsync().ConfigureAwait(false);
if (content.Length != 0)
{
hash = md5.ComputeHash(content);
}
return hash;
}
}
I'm building a piece of middleware that automatically logs the information of inbound requests and their responses from an API. As part of that I would like to be able to log the body of the request, however this is a bit more difficult since I'm working with streams.
Currently, the middleware will read the body and log it, however since I'm reading the body using a StreamReader the contents of the body becomes essentially null after, since it's been read. Because of this the request continues and I'm getting "A non-empty request body is required." errors since there's no body present any longer in the request.
I want to be able to copy the stream over to a new variable so that I can read the second variable and log the body while keeping the original body of the request intact so it can proceed and be processed by my controller.
I've tried the below, but since this just creates a reference to the stream, the original is still affected.
public void AddInboundDetails(HttpRequest request)
{
var bodyStreamCopy = request.Body;
Add("requestBody", ReadStreamAsString(bodyStreamCopy));
}
private string ReadStreamAsString(Stream input)
{
StreamReader reader = new StreamReader(input, Encoding.UTF8);
return reader.ReadToEnd();
}
I've read about Stream.CopyTo in the docs, but since this is again reading the original stream it feels like this will just produce the same problem.
Update, same issue:
request.EnableBuffering();
var body = request.Body;
var buffer = new byte[Convert.ToInt32(request.ContentLength)];
await request.Body.ReadAsync(buffer, 0, buffer.Length);
var bodyAsText = Encoding.UTF8.GetString(buffer);
request.Body = body;
I'm trying to POST a MultipartFormDataContent in a real case scenario, a data content object could contain anything from a simple string to a video file I'm using a serialized object down there, just a proof of concept.
Also I would like to note that using JSON objects wont serve my real life scenarios
public class GzipMultipartContent : MultipartFormDataContent
{
public GzipMultipartContent()
{
Headers.ContentEncoding.Add("gzip");
}
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
return Task.Factory.StartNew(() =>
{
using (var gzip = new GZipStream(stream, CompressionMode.Compress, true))
base.SerializeToStreamAsync(gzip, context);
});
}
}
and here's how I call it
var gzipped = new GzipMultipartContent();
var test = new TestClass();
gzipped.Add(new StringContent(JsonConvert.SerializeObject(test)), "value");
var client = new HttpClient();
var result = client.PostAsync("http://localhost:60001/api/Home/", gzipped).Result;
and here's the post action in the controller
// POST: api/Home
[HttpPost]
public void Post([FromForm] object value)
{
}
I have added a break point at the server side and made sure it doesn't even reach the Post method, also I have tried with a normal POST request to make sure that it's not a server configuration problem or a URL mistyping
Client side
If the code in question is your real code, then there are at least two issues:
Did not wait on base.SerializeToStreamAsync
You created a new task, but you did not wait until the base class completed writing to the compressed stream in the task. So you could send unpredictable content to server.
Did not override Content-Length
MultipartFormDataContent calculates length of content based on data not compressed, since you have compressed data, you must re-compute length for the compressed data.
Frankly, I don't think you need to inherit from MultipartFormDataContent to make it compressed. Instead, you could compress the entire MultipartFormDataContent in a wrapper HttpContent:
public class GzipCompressedContent : HttpContent
{
private readonly HttpContent _content;
public GzipCompressedContent(HttpContent content)
{
// Copy original headers
foreach (KeyValuePair<string, IEnumerable<string>> header in content.Headers)
{
Headers.TryAddWithoutValidation(header.Key, header.Value);
}
Headers.ContentEncoding.Add("gzip");
_content = content;
}
protected override async Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
using (var gzip = new GZipStream(stream, CompressionMode.Compress, true))
{
// Compress the entire original content
await _content.CopyToAsync(gzip);
}
}
protected override bool TryComputeLength(out long length)
{
// Content-Lenght is optional, so set to -1
length = -1;
return false;
}
}
And use it:
var test = new TestClass();
using (var client = new HttpClient())
{
var form = new MultipartFormDataContent();
form.Add(new StringContent(JsonConvert.SerializeObject(test)), "value");
var compressed = new GzipCompressedContent(form);
var result = await client.PostAsync(..., compressed);
}
Server side
Your server needs to support compressed stream.
For example, by default, ASP.NET Core does not support compressed request, if you send GZip compressed request to an ASP.NET Core application, you will see exception:
System.IO.IOException: Unexpected end of Stream, the content may have already been read by another component.
at Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
The above exception happens in action invocation pipeline before any controller action takes place. So controller actions in this case could not be reached.
To fix such, you will need to enable server side request decompression support.
If you are using ASP.NET Core, check out this nuget package.
I am not sure if I understood the issue, but if it is that your request isn't getting to the server, while your "normal" POST requests are, then I think that I found your problem.
I think that the issue is that your server doesn't know what Content-Type is coming to it. I literally copy-pasted your code, but added
Headers.ContentType = new MediaTypeHeaderValue("application/x-gzip");
to GzipMultipartContent.cs ctor.
After I added the type, I was hitting my breakpoint in the localhost server.
Source: Content-Type
In requests, (such as POST or PUT), the client tells the server what type of data is actually sent.
I am trying to send a file to a server over a REST API. The file could potentially be of any type, though it can be limited in size and type to things that can be sent as email attachments.
I think my approach will be to send the file as a binary stream, and then save that back into a file when it arrives at the server. Is there a built in way to do this in .Net or will I need to manually turn the file contents into a data stream and send that?
For clarity, I have control over both the client and server code, so I am not restricted to any particular approach.
I'd recommend you look into RestSharp
http://restsharp.org/
The RestSharp library has methods for posting files to a REST service. (RestRequst.AddFile()). I believe on the server-side this will translate into an encoded string into the body, with the content-type in the header specifying the file type.
I've also seen it done by converting a stream to a base-64 string, and transferring that as one of the properties of the serialized json/xml object. Especially if you can set size limits and want to include file meta-data in the request as part of the same object, this works really well.
It really depends how large your files are though. If they are very large, you need to consider streaming, of which the nuances of that is covered in this SO post pretty thoroughly: How do streaming resources fit within the RESTful paradigm?
You could send it as a POST request to the server, passing file as a FormParam.
#POST
#Path("/upload")
//#Consumes(MediaType.MULTIPART_FORM_DATA)
#Consumes("application/x-www-form-urlencoded")
public Response uploadFile( #FormParam("uploadFile") String script, #HeaderParam("X-Auth-Token") String STtoken, #Context HttpHeaders hh) {
// local variables
String uploadFilePath = null;
InputStream fileInputStream = new ByteArrayInputStream(script.getBytes(StandardCharsets.UTF_8));
//System.out.println(script); //debugging
try {
uploadFilePath = writeToFileServer(fileInputStream, SCRIPT_FILENAME);
}
catch(IOException ioe){
ioe.printStackTrace();
}
return Response.ok("File successfully uploaded at " + uploadFilePath + "\n").build();
}
private String writeToFileServer(InputStream inputStream, String fileName) throws IOException {
OutputStream outputStream = null;
String qualifiedUploadFilePath = SIMULATION_RESULTS_PATH + fileName;
try {
outputStream = new FileOutputStream(new File(qualifiedUploadFilePath));
int read = 0;
byte[] bytes = new byte[1024];
while ((read = inputStream.read(bytes)) != -1) {
outputStream.write(bytes, 0, read);
}
outputStream.flush();
}
catch (IOException ioe) {
ioe.printStackTrace();
}
finally{
//release resource, if any
outputStream.close();
}
return qualifiedUploadFilePath;
}
Building on to #MutantNinjaCodeMonkey's suggestion of RestSharp. My use case was posting webform data from jquery's $.ajax method into a web api controller. The restful API service required the uploaded file to be added to the request Body. The default restsharp method of AddFile mentioned above caused an error of The request was aborted: The request was canceled. The following initialization worked:
// Stream comes from web api's HttpPostedFile.InputStream
(HttpContext.Current.Request.Files["fileUploadNameFromAjaxData"].InputStream)
using (var ms = new MemoryStream())
{
fileUploadStream.CopyTo(ms);
photoBytes = ms.ToArray();
}
var request = new RestRequest(Method.PUT)
{
AlwaysMultipartFormData = true,
Files = { FileParameter.Create("file", photoBytes, "file") }
};
Detect the file/s being transported with the request.
Decide on a path where the file will be uploaded (and make sure CHMOD 777 exists for this directory)
Accept the client connect
Use ready library for the actual upload
Review the following discussion:
REST file upload with HttpRequestMessage or Stream?
First, you should login to the server and get an access token.
Next, you should convert your file to stream and post the stream:
private void UploadFile(FileStream stream, string fileName)
{
string apiUrl = "http://example.com/api";
var formContent = new MultipartFormDataContent
{
{new StringContent(fileName),"FileName"},
{new StreamContent(stream),"formFile",fileName},
};
using HttpClient httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Add("Authorization", accessToken);
var response = httpClient.PostAsync(#$"{apiUrl}/FileUpload/save", formContent);
var result = response.Result.Content.ReadAsStringAsync().Result;
}
In this example, we upload the file to http://example.com/api/FileUpload/save and the controller has the following method in its FileUpload controller:
[HttpPost("Save")]
public ActionResult Save([FromForm] FileContent fileContent)
{
// ...
}
public class FileContent
{
public string FileName { get; set; }
public IFormFile formFile { get; set; }
}