Develop Reference

Traverse and HtmlEncode strings in Json.net C#

I struggle with safely encoding html-like text in json. The text should be written into a <textarea>, transferred by ajax to the server (.net45 mvc) and stored in a database in a json-string.
When transferring to server, I get the famous "A potentially dangerous Request.Form value was detected" 500 server error. To avoid this message, I use the [AllowHtml] attribute on the model that are transferred. By doing so I open up for XSS-vulnerability, in case anyone paste in { "key1": "<script>alert(\"danger!\")</script>" }. As such, I would like to use something like
tableData.Json = AntiXssEncoder.HtmlEncode(json, true);
Problem is I cannot do this on the full json string, as it will render something like
{
"key1&quot: ...}
which of course is not what I want. It should be more like
{ "key1": "<script>alert("danger!")</script>" }
With this result the user can write whatever code they want, but I can avoid it to be rendered as html, and just display it as ordinary text. Does anyone know how to traverse json with C# (Newtonsoft Json.NET) such that strings can be encoded with AntiXssEncoder.HtmlEncode(... , ....);? Or am I on a wrong track here?
Edit:
The data is non-uniform, so deserialization into uniform objects is not an option.
The data will probably be opened to the public, so storing the data encoded would ease my soul.

If you already have the data as a JSON string, you could parse it into proper objects with something like Json.NET using JsonConvert.DeserializeObject() (or anything else, there are actually quite a few options to choose from). Once it's plain objects, you can go through them and apply any encoding you want, then serialize them again into a JSON string. You can also have a look at this question and its answers.
Another approach that you may take is just leave it alone until actually inserting stuff into the page DOM. You can store unencoded data in the database, you can even send it to the client without HTML encoding as JSON data (of course it needs to be encoded for JSON, but any serializer does that). You need to be careful not to generate it this way directly into the page source though, but as long as it's an AJAX response with text/json content type, it's fine. Then on the client, when you decide to insert it into the actual textarea, you need to make sure you insert it as text, and not html. Technically this could mean using jQuery's .text() instead of .html(), or your template engine's or client-side data binding solution's relevant method (text: instead of html: in Knockout, #: instead of #= in say Kendo UI, etc.)
The advantage of this is latter approach is that when sending the data, the server (something like an API) does not need to know or care about where or how a client will use the data, it's just data. The client may need different encoding for an HTML or a Javascript context, the server cannot necessarily choose the right one.
If you know it's just that text area though where this data is needed, you can of course take the first (your original) approach, encode it on the server, that's equally good (some may argue that's even better in that scenario).
The problem with answering this question is that details count a lot. In theory, there are a myriad of ways you could do it right, but sometimes a good solution differs from a vulnerable one in one single character.

So this is the solution I went for. I added the [AllowHtml] attribute in the ViewModel, so that I could send raw html from the textarea (through ajax).
With this attribute I avoid the System.Web.HttpRequestValidationException that MVC gives to protect against XSS dangers.
Then I traverse the json-string by parsing it as a JToken and encode the strings:
public class JsonUtils
{
public static string HtmlEncodeJTokenStrings(string jsonString)
{
var reconstruct = JToken.Parse(jsonString);
var stack = new Stack<JToken>();
stack.Push(reconstruct);
while (stack.Count > 0)
{
var item = stack.Pop();
if (item.Type == JTokenType.String)
{
var valueItem = item as JValue;
if(valueItem == null)
continue;
var value = valueItem.Value<string>();
valueItem.Value = AntiXssEncoder.HtmlEncode(value, true);
}
foreach (var child in item.Children())
{
stack.Push(child);
}
}
return reconstruct.ToString();
}
}
The resulting json-string will still be valid and I store it in DB. Now, when printing it in a View, I can use the strings directly from json in JS.
When opening it again in another <textarea> for editing, I have to decode the html entities. For that I "stole" some js-code (decodeHtmlEntities) from string.js; of course adding the licence and credit note.
Hope this helps anyone.

SignalR encoding in JSON

I have a Windows service that publishes data to a .NET client and a web client over SignalR. I've recently had some funny issues, but can't quite get a consistent behavior.
The problem lies in serializing the degrees sign, i.e. "°C". Most of the time it is serialized correctly, but I've had a few times where I see the following in my debugger:
See how the first time the "°" is serialized correctly, but the second time we see the question marks in the diamonds?
I've read this means it is an invalid UTF-8 character. But then why do all the other properties serialize correctly? This is a screenshot where you see one correct and one incorrect, but the entire JSON contains hundreds of these "°C" strings that look correct.
So why this one exception? It's not always the same position/property, and it doesn't always happen. This makes me think it must be a combination with preceding/succeeding characters, no?
Any ideas how to fix this or at least how to investigate this further?
Update
This is how I do serialization. I set it up on startup:
var serializerSettings = new JsonSerializerSettings();
serializerSettings.Converters.Add(new StructuredAmountJsonConverter());
var serializer = JsonSerializer.Create(serializerSettings);
GlobalHost.DependencyResolver = new AutofacDependencyResolver(_lifetimeScope);
GlobalHost.DependencyResolver.Register(typeof(JsonSerializer), () => serializer);
What's happening here is I'm telling SignalR to use Autofac to resolve dependencies. Then I register my JSON.NET serializer. The JSON.NET serializer has one custom converter, which converts my Amount class to the structure you see above (with a value and a unit property).
So you could think the problem lies in the converter, but then why is it working 95% of the time? Or should I specify the encoding in my converter?
Update 2
I've been using Fiddler to capture my network traffic and I can't see the wrong characters there. So I'm guessing the encoding problem is at the client side. I will investigate further.
Update 3
I've managed to capture the traffic in Fiddler and while it looks good in the Text view, when I select the HexView I do see something weird:
Notice how it says "°C" instead of "°C". So maybe it is sending it from the server in the wrong way?
Also, keep in mind my client is a .NET (WPF) client. This is my code to connect on the client side (simplified):
var url = "myUrl...";
_hubConnection = new HubConnection(url);
var hubProxy = _hubConnection.CreateHubProxy("MyHub");
hubProxy.On<object>("receive", OnDataReceived);
await _hubConnection.Start();
And when receiving data:
var message = JsonConvert.DeserializeObject<MyDataContract>(obj.ToString(), new StructuredAmountJsonConverter());
Update 4
This post makes me think this is happening:
The server/SignalR is sending my data as UTF-8, but the client is expecting latin-1 or Windows-1252 (probably the latter). So now I need to find out how I can make it use UTF-8.

May I know how you are serializing the objects? I think there is a way to specify the encoding types for serializing special characters like this. Here is a link I found- Special characters in object making JSON invalid (DataContractJsonSerializer)

JSOn object not deserializing properly in wcf webservice side

I am working on iOS project and sending the Json string to backend through wcf webservice, Its working successfully for many users but for some users backend getting incomplete json string.
Code for generating Json string
NSData *data = [NSJSONSerialization dataWithJSONObject:EmployeeDetails options:0 error:nil];
NSString *JsonString = [[NSString alloc] initWithData:data
encoding:NSUTF8StringEncoding];
And Attaching the json string with request by below code
[request setValue:#"application/x-www-form-urlencoded" forHTTPHeaderField:#"Content-Type"];
[request setHTTPBody:[jsonString dataUsingEncoding:NSUTF8StringEncoding]];
EmployeeDetails is NSMutableDictionary with employee details.
And
Here is successful Json string deserialized from BackEnd
{"FirstName":"Donald","LastCBPaymentType":6,"AcceptEmail":true,"CellPhone":"321-300-6874","Position":"3","CarrierNum":"4","EmpNum":"96874"}
And Incomplete Json string
{"FirstName":"roset","LastCBPaymentType":6,"AcceptEmail":true,"CellPhone":"321-300-6874","Position":"3","CarrierNum":"4","EmpNum":"98799
The incomplete json string occuring in backend for some user.
Any one can help to solve this issue ?

Well, I'd say TomSwift is on the right track... It's apparent that the broken JSON is simply missing "} from the end of the string. That does indeed look like something being sanitised by a regex...
You really should find the source of the problem; But for a quick and dirty fix while you figure it out, you could just run your own regex on the JSON you receive and if it doesn't end with a "}, just add it... First do something like: '/"[0-9]+$/' (to check if there is an open numeric string at the end, if there is, you close it by adding " ) then check if '/}$/' (if } is the last char, if it isn't: add it).
Remember, the above would only be a temporary fix (as it is somewhat heavy and not really that pretty), but I could imagine myself being forced to do something like this, simply because of time constraints. If the issue is actually coming from within the API, you could tell the creators about it and use something like that fix above while they fixed it.
PS. If anyone does choose to use a fix like the above, please please please, for the love of all that you hold holy, leave a comment in the code explaining why you did it (as opposed to telling us what you did...).

I had a similar problem in PHP/CodeIgniter recently and the problem was that the JSON was being "sanitized" via some overly ambitious regular expressions before it arrived to my handler. Perhaps there's some similar sanitization that happens in WCF?
Things I would try:
Using Charles Proxy to confirm that the request (POST?) is actually sending the full body as you expect.
Try Content-Type application/json instead.

Detecting The length of the string exceeds the value set on the maxJsonLength property at the server

First let me start by saying I know what this error means.
I know about <jsonSerialization maxJsonLength="50000000"/>
And I understand the (in my opinion, very good) reasons for having a limit.
So I am calling a webservice from javascript using the .NET ScriptManager to create all the client code for me.
The webservice gets's a bunch of data from SQL, turns it into an array of objects then returns this array.
Under the hood the .NET engine turns this array into a JSON array for sending to the client.
Sometimes however the resulting JSON string is too long and it throws an exception.
I would like some generic way of detecting this error and handling it gracefully (e.g. by sending some of the data with an indicator that another subsequent request should be made to get the rest)
What I do not want to do is set a sky high limit on the length of the json string as this could result in it taking a very long time before the client gets all the data. I believe the upper limit on this setting to be 2147483644 i.e. 2GB.
Chances are the client has got bored and wandered off by the time all that data arrives just in time to crash the javascript engine when it tries to parse it.
So I guess really what I am after is a way to "know" how long the resulting json string will be before returning it so that it can be truncated as appropriate, or some way to handle the resulting error, shorten the array and try again (and keep shortening and retrying until it works or the array is empty).
I have tried putting a try/catch around the return statement but this didn't work - I suspect because the webservice method is successfully returning the data to the framework but the framework then chokes on the data.
I would prefer a solution that doesn't involve returning a string, but if there is no other way I can put up with that.
[WebMethod]
[System.Web.Script.Services.ScriptMethod(ResponseFormat = System.Web.Script.Services.ResponseFormat.Json)]
public MyClass[] GetMyClasses()
{
List<MyClass> lst;
//code that populate lst with enough instances to break the JSON.
return lst.ToArray();
}

Instead of trying to figure out the resulting JSON length and then apply some sort of paging, maybe you should always page your data..?
You already know that the method can result in huge data, so why not change your method to GetMyClasses(int skip, int take) to enforce chunks of data of reasonable sizes.

Returning an array with PHP SOAP service

When I started this, I knew nothing about SOAP or how it worked. I learned a lot in the first few hours and the result was an operational test of the SOAP service (which I'm writing in PHP). I was able to create the SOAP client in both PHP and Visual Studio (which is what I 'm really aiming for) and return the information I wanted.
Since it was just a test and I wanted to return more (and better formatted) information, I started messing around with it. I added some types and wrote some schemas that I imported. When I finished adding everything I needed (this was all for one result) I saved it all and tested it. It ran without error, but didn't return anything.
I'm trying to return an array with 8 elements, with differing types. Here are the elements and types of that array:
Title - string
LinkId - int
Date - int
Author - string
Content - string
Id - int
Icon - int
Edited - boolean
The function GetNews will return 25 of those arrays (per page, the input is an integer for which page of results to get).
I cannot figure out how to serialize the array so that SOAP will return it properly.
Here's the URL to my WSDL file:
http://api.infectionist.com/soap.wsdl
PLEASE help me out, I am completely stuck. I can't see anything wrong with the code, and it's running without error in PHP and Visual Studio, it's just returning an empty result.

I'm guessing you are trying to return the entire array at once from within PHP? If so, you need to make sure that you return an XML string. Like:
<?php
return '
<data>
<title>Actual title</title>
<linkId>154278</linkId>
...
</data>
';
?>
Have you considered using a JSON string as an alternative to send the data from the SOAP server? This would allow you to have all the elements you need in one element and retain flexibility to add/remove elements later (you would need to inform your clients of this).