ASP.NET web api 2 truncating json response - c#

I'm experiencing a very weird intermittent issue with my API calls. Sometimes the response gets truncated.
There isn't a pattern for how or when it gets truncated.
Response size is 200kb and configured response limit is 20mb.
Formatters configuration:
private static HttpConfiguration ConfigureFormatters(this HttpConfiguration config)
{
config.Formatters.Clear();
config.Formatters.Add(new JsonMediaTypeFormatter());
config.Formatters.JsonFormatter.SerializerSettings.ContractResolver = new CamelCasePropertyNamesContractResolver();
return config;
}
Simplified Api call:
[HttpGet, Route]
public IHttpActionResult Explore(int cityId)
{
var lists = exploreBuilderService.Build(cityId);
return Ok(lists);
}
lists type is List<SomeModel>. SomeModel is DTO with no circular references.
Sample response:
Response Headers
Content-Type: application/json; charset=utf-8
Server: Microsoft-IIS/10.0
X-Powered-By: ASP.NET
Date: Fri, 17 Jan 2020 12:02:37 GMT
Content-Length: 290248
Data
[..., {"id":47
... are other objects in the array.
It looks like json response string is cut in a random place. Our android application fails with MalformedJsonException when the response comes like this. It happens once on like 5-10 requests for the same data set.
Did you encounter a problem like this? Where should I look for potential problems causing this?

The issue was with one of our middlewares which did response interception.

Perhaps someone will face this problem now, like me.
But as it turned out, .Net 6
An ASP.NET Core web API endpoint could return a truncated JSON body with a 200 response code when an exception is thrown during serialization of the object returned by the action method if a sufficient portion of the response has already been successfully serialized beforehand, so that part of the response has already been sent to the client. Fortunately, the exception will still be logged properly. The only way to reliably prevent this is to ensure that the returned object can be successfully serialized.

Related

Receiving 500 error posting valid JSON to .NET Core 2 / .NET MVC Api. (post request is null and stating "System.Net.Http.NoWriteNoSeekStreamContent")

So quick background on what is currently going on in my solution. I'm writing pretty basic API web tests in c# using the XUnit framework and some json validation helpers I created to validate response messages and payloads received back from the API. I've been banging my head against a wall trying to figure out why my PostAsync() call is passing null once it reaches the API since I can see the request body is a valid json request when debugging my ApiTests solution. Below are some snippets of the test I'm currently debugging.
OrderControllerTests.cs
[Theory]
[MemberData(nameof(OrderControllerData.postCancelOrder), MemberType = typeof(OrderControllerData))]
public async Task PostValidCancelOrder(string jsonBody, string expectedResponseBody)
{
var postObject = "api/Order/cancel";
var request = new StringContent(jsonBody, Encoding.UTF8, "application/json");
var response = await _httpClient.PostAsync(postObject, request);
var content = await response.Content.ReadAsStringAsync();
ValidationHelper.ValidateNoErrors(response);
ValidationHelper.ValidateJson(content, expectedResponseBody);
}
A few things are happening in this test. All of which seem to be working fine and as intended (except for maybe #3 below).
Create my requestUri with postObject.
Create my HttpContent. XUnit is passing my jsonBody param by way of the MemberData attribute. Which looks into my data layer class for the postCancelOrder public static IEnumerable.
Posting the request to the API.
Reading the payload returned back in string format.
Validation Helper checks for HTTP response messages. The ValidateNoErrors method is simply just looking for a 200 response code. The second one doesn't matter for this question since my test blows up at the first ValidationHelper.
Right now this is currently what I am posting to the api.
"{\"orderId\": \"c701b60a-4fb1-4f79-9e17-0172a9a17bbc\",\"test1Fee\":\"5}\",\"test2Fee\":\"10\"}"
Here is the HttpPost within the Api/OrderController.
OrderController.cs
[HttpPost("cancel")]
public async Task<IActionResult> Cancel([FromBody]CancelOrderViewModel model)
{
var command = new CancelOrderCommand(model.OrderId, model.Test1Fee, model.Test2Fee);
var order = await _mediator.Send(command);
return Ok(order);
}
Essentially this HttpPost is passing a few parameters and then running a multiple methods and checks on those objects to determine if the order is eligible to be cancelled. What happens within the post doesn't matter right now. When debugging my test and it hits the entry point of the HttpPost method of the api, the 'command' and 'model' are both null. Naturally the Api will reject the request and responds with a 500 error.
{StatusCode: 500, ReasonPhrase: 'Internal Server Error', Version: 1.1, Content: System.Net.Http.NoWriteNoSeekStreamContent, Headers:{ Date: Mon, 22 Jan 2018 19:38:55 GMT Transfer-Encoding: chunked Server: Kestrel Request-Context: appId=cid-v1:93254013-d606-465f-8cee-12e422316f31 X-SourceFiles: =?UTF-8?B?RDpcUUFccWFWZW5kb3JcQWJzdHJhY3Rvck9tc0FwaVxzb3VyY2VcQXBpXGFwaVxPcmRlclxjYW5jZWw=?= X-Powered-By: ASP.NET Content-Type: application/json; charset=utf-8}}
Also the message is indicating that Object reference not set to an instance of an object since everything is null apparently.
So the problem has to be with how I'm constructing the request within my XUnit test. But I'm confused because I believe I'm serializing the json correctly. Even when the Api is running locally in my browser using swagger, the indicated example is
{
"orderId": "string",
"test1Fee": 0,
"test2Fee": 0
}
Could the escapes being created within my post object be the problem? If so, how do I go about fixing that? I tried using PostAsJsonAsync() instead of just PostAsync() but it continues to yield the same results.
What am I doing wrong here? I'm sure it's something pretty simple and I'm just missing it. Let me know if I need to provide anymore information. Any help would be greatly appreciated. Thanks.

403 when calling AirBnb api from C#

I've not used HttpClient before so apologies if it's an obvious one.
I'm poking about with the airbnb api http://airbnbapi.org/#view-listing-info
My understanding of the endpoint is that I don't need an auth token, as this is a public endpoint I'm trying to use. Unfortunately I'm getting a 403 no matter what I try to do and I'm not entirely sure why.
I've got the following code:
client.DefaultRequestHeaders.Add("client_id", "<My client Id>");
client.DefaultRequestHeaders.Add("locale", "en-gb");
client.DefaultRequestHeaders.Add("currency", "gbp");
var request = new HttpRequestMessage()
{
RequestUri = new Uri($"https://api.airbnb.com/v2/listings/{id}"),
Method = HttpMethod.Get,
};
var task = client.SendAsync(request)
.ContinueWith((taskwithmsg) =>
{
var response = taskwithmsg.Result;
//var jsonTask = response.Content.ReadAsAsync<JsonResult>();
//jsonTask.Wait();
//var jsonObject = jsonTask.Result;
return response.Content;
});
task.Wait();
return task.Result;
And I'm getting the following response:
- response {StatusCode: 403, ReasonPhrase: 'Forbidden', Version: 1.1, Content: System.Net.Http.StreamContent, Headers:
{
Connection: close
Date: Tue, 04 Apr 2017 19:44:40 GMT
Server: AkamaiGHost
Mime-Version: 1.0
Content-Length: 291
Content-Type: text/html
Expires: Tue, 04 Apr 2017 19:44:40 GMT
}} System.Net.Http.HttpResponseMessage
Any advice?
EDIT:
Macceturra wisely suggested I try to make the call with postman.
I've now established that I can make a call in postman and get a correct response back.
The request you're sending has the client_id as a HTTP header, when Airbnb is expecting it as a URL parameter.
Additionally, Airbnb requires the client to send an Accept (or User-Agent) header, or else it will still return "403 Forbidden" (probably should be "400 Bad Request").
Putting that together (and deleting the unnecessary headers):
var id = ...;
var clientId = ...;
var uri = new Uri($"https://api.airbnb.com/v2/listings/{id}?client_id={Uri.EscapeDataString(clientId)}&locale=en-gb&currency=GBP");
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Add("Accept", "application/json");
await client.GetAsync(uri);
I've not been able to mark a single answer as the solution as it came through in the comments.
Solution
Start by making calls in postman. This helped me realise that I should have been passing the client id as a url parameter and not as a header. Bradley Grainger went on to answer this too.
Try and make that exact call with the HttpClient. This still returned the 403, but now we have confidence in the url we're sending.
Change the content type to application/json. This beats the 403 and gets us a 200 with the requested data.
Big thanks to all who helped piece these steps together.
You may need to add a cookie to the request called _aat which contains the airbnb access token. This is generated during login. You'll be able to see it by looking at the headers in Fiddler or the Net section of the browser dev tools when browsing one of your own listings when logged in. Just copy and paste it from there into your code to test if you can get the request working properly and then later you can automate the process of getting the _aat cookie within your code. Make sure the http headers in your code also match what is being sent via the browser.

Twilio Error - 12300 - Invalid Content Type only sometimes

I have a C# / .NET WebApi endpoint tied to a number. When that number receives a text, it's forwarded to my API via webhook.
Sometimes (not all the time), I get an error in my debugger with the following:
Error - 12300
Invalid Content-Type
Twilio is unable to process the Content-Type of the provided URL.
Please see the Twilio Markup XML Documentation for more information on
valid Content-Types. You must return a Content-Type for all requests.
Requests without a Content-Type will appear in the App Monitor as a
502 Bad Gateway error.
In the response that triggered this, I see the following:
With the following headers:
Content-Type application/json; charset=utf-8
Pragma no-cache
Date Sat, 14 Jan 2017 02:57:45 GMT
X-AspNet-Version 4.0.30319
X-Powered-By ASP.NET
What might be causing this, and how do I address it?
After some research from TWIML MESSAGE: YOUR RESPONSE
this code seems to work
[HttpGet]
public HttpResponseMessage SmsAnswerCallBack(string id)
{
_smsAnswerCallBackCallIndex++;
var r = new SmsApiResult();
r.SetStatus(true, _smsSendCallIndex, _smsAnswerCallBackCallIndex);
r.DataList = _answers;
var res = Request.CreateResponse(HttpStatusCode.OK);
res.Content = new StringContent("<Response/>", Encoding.UTF8, "text/xml");
return res;
}
I too was sending a json response and getting this error. Using the answer by Frederic Torres got me on the right track. It looks like Twilio is looking for XML in TwiML format. But if you basically just return an empty "Response" element in text/xml format, that satisfies Twilio. So here is a simplified answer for anybody else that runs into this:
public ContentResult IncomingSMS(string To, string From, string Body)
{
//do stuff
//...
return Content("<Response/>", "text/xml");
}

How to get RestSharp to properly deflate compressed HTTP response?

I'm using RestSharp to make a call to REST service. My call looks something like this:
var request = new RestRequest("/foo", Method.POST);
request.JsonSerializer.ContentType = "application/json; charset=utf-8";
request.AddJsonBody(new string[] { "param1", "param2" });
var response = this._client.Execute<Foo>(request);
For most other calls this works fine. I'm running into issues when the response is compressed. The headers in the response look (mostly) like this:
HTTP/1.1 200 OK
Uncompressed-Size: 35000
Content-Length: 3019
Content-Encoding: deflate
Content-Type: application/json
The issue is when I call this method with RestSharp I keep getting the error:
Error: Block length does not match with its complement.
I've tried setting the Accept-Encoding header in the request but it still produces the error. I also tried using a custom deserializer but the error is occurring before deserialization. From what I can tell, RestSharp should automatically handle deflation if the Content-Encoding header says deflate (which it does).
How can I get RestSharp to handle the deflation properly?
UPDATE
In the end I was able to have the service changed to look for an Accept-Encoding header in the request with a value of identity. If found, the service was changed to return the data uncompressed.
This is unfortunately not really a solution to the original issue but it does resolve the problem for me. If a better solution is posted I will try it.
According to this post, you should be able to handle it if you won't pass charset=utf-8 in content type.
Please refer to this:
RestSharp compress request while making rest call to server

Web API 2 returning text/plain responses

Really struggling with something I hope people here can help with. I'm writing a RESTful API in Web API 2. Whenever I send a request to this service, the response is consistently being sent with a Content-Type of text/plain. Obviously this is no good, my response needs to be Content-Type of application/json. I've tried a few suggestions that I found from Google but I don't think I'm understanding the whole picture.
Is there something special I need to do in order to have my web service respond with application/json content? Note that I want this to work globally across the whole app, so I'm not after a way to modify a given response and set its content type - I want this to be a default behaviour for the whole web service: If a request contains an Accept header for application/json I want my web service to return that Content-Type instead of text/plain.
Edit to clarify:
My response contains an object called "responseData" that should be serialized into JSON and included in the body. I'm currently putting together my response like this:
HttpResponseMessage response = Request.CreateResponse(HttpStatusCode.OK, responseData);
return response;
responseData is a POCO. This get's correctly serialized as JSON and returned in the response - the only missing part is the Content-Type which is incorrectly set to "text/plain". I could manually change this on every single response I compose, but I'm wanting to configure this on a global level.
OK, assuming that your responseData is a string, the Content-type header will be text/plain when you create the HttpResponseMessage. Doesn't matter what the contents of the string are, since no attempt is made to determine this.
The solution is to create the appropriate Content object for your message, initialized with the media type you're returning:
HttpResponseMessage response = new HttpResponseMessage()
{
Content = new StringContent(
responseData,
Encoding.UTF8,
"application/json"
)
};
There are other methods that amount to returning a particular object type and letting the API libraries serialize to JSON or XML for you as required. I prefer to let the framework do the work for me where possible, but this is how you'd achieve it with a string you've created yourself.
For a strict JSON-only result, remove the XML formatter from the WebAPI configuration and return your POCO.
In App_Start\WebApiConfig.cs, add the following to the WebApiConfig.Register method:
config.Formatters.Remove(config.Formatters.XmlFormatter);
And for your API:
public class MyObject
{
public bool result;
public string reason;
}
public class TestController : ApiController
{
public MyObject GetData()
{
MyObject result = new MyObject { result = "true", reason = "Testing POCO return" };
return result;
}
}
I ran this up and requested /api/Test from Chrome, which doesn't even mention application/json in the Accept header. Here are the response headers up until it hits Content-Type:
HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/json; charset=utf-8
And the body:
{"result":true,"reason":"Testing POCO return"}
Since I disabled XML it defaulted to JSON.
Add the following to Global.asax file.
protected void Application_Start()
{
JsonSerializerSettings serializerSettings = new JsonSerializerSettings();
serializerSettings.Converters.Add(new IsoDateTimeConverter());
var jsonFormatter = new JsonNetFormatter(serializerSettings);
jsonFormatter.SupportedMediaTypes.Add(new MediaTypeHeaderValue("text/html"));
GlobalConfiguration.Configuration.Formatters.Insert(0, jsonFormatter);
}
Another possible source of the issue described is that there may be an authorization redirect in play as was the case for us when one of the engineers thought to re-use user authentication for an api.
This means incoming requests were being redirected to a login page which was the text/html response that couldn't be parsed by ReadAsync<>. A silly mistake to be sure, but not an easy one to spot.
The solution in that case was to remove the user authentication and implement HMAC based authentication for the api.

Categories

Resources