I've run into a confusing problem that i cannot seem to solve.
I'm using Json.Net and i've written a custom Json converter to handle a special case in my application.
The issue i've run into is in the deserialize or ReadJson method which is throwing an error when it tires to convert a JSON array into an array of strings:
The exact text of the error is: Unexpected token while deserializing object: PropertyName. Path 'RootPages', line 1, position 13.
As you can see from the inspector, the JProperty it is trying to deserialize (RootPages) has been parsed correctly and is valid JSON.
So i'm not entirely sure what is going on here, any enlightenment would be greatly appreciated..
If relevant, the original JSON string is as follows:
{
"RootPages": [
"TestItem1",
"TestItem2"
],
"Name": "root"
}
EDIT-CODE:
ReadJson Method:
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) {
if (PartialChildPageSerialization) {
var jsonObject = JObject.Load(reader);
var properties = jsonObject.Properties().ToList();
foreach (var property in objectType.GetProperties()) {
var type = property.PropertyType;
object value = null;
if (type.IsGenericType && type.GetGenericTypeDefinition() == typeof(ChildPageCollection<>)) {
var collection = Activator.CreateInstance(type);
var deserializedValue = properties.First(p => p.Name == property.Name).ToObject<string[]>();
type.GetMethod("PolulateFromSerializer").Invoke(collection, new object[] {deserializedValue});
value = collection;
}
else {
value = properties.First(p => p.Name == property.Name).ToObject(type);
}
property.SetValue(existingValue, value);
}
return existingValue;
}
return serializer.Deserialize(reader, objectType);
}
Snippet of the interesting part of ChildPageCollection:
public class ChildPageCollection<T> : IList<T> where T : DataPage
{
public string[] GetNames() => _internalNameList.ToArray();
internal void PolulateFromSerializer(string[] names) {
this.Clear();
_internalNameList.AddRange(names);
_hasFullList = false;
}
private void CheckFullList() {
if(!_hasFullList)
throw new InvalidOperationException("Collection has not been fully loaded, and full list is not avialable.");
}
private readonly List<T> _internalList = new List<T>();
private readonly List<string> _internalNameList = new List<string>();
private bool _hasFullList = true;
...
}
I expect this is because the first property is an object with a property called 'RootPages' which is a string[].
Unfortunately from the looks of your screenshots you are trying to turn an object into a string array.
This should work in the example you've given:
properties.First(p => p.Name == property.Name).Select(o => o.Children().Values<string>()).First().ToArray();
In place of:
properties.First(p => p.Name == property.Name).ToObject<string[]>();
Related
what I am trying to do is to convert complex xml document to json format, and I am using Newtonsoft.Json to achieve my goal. and I have came across small - big problem. So for example
I have a model that looks like:
public class assets
{
public UInt32 id {get; set;}
public String providerName {get; set;}
public String provider {get; set;}
public String realm {get; set;}
public ICollection<unit> unit {get; set;}
}
My intention is that user will stream xml content to method that will change that xml to json and i will post it to API.
To simplify User is pasting simple xml (normal xml is far more complex, but basically it would looks like many levels of example bellow)
<assets xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="assets.xsd" providerName="myProviderName" provider="myProvider" realm="myCatalog">
<unit idKey="newGeo_63119679"></unit>
<unit idKey="newGeo_63119179"></unit>
</assets>
Json result will look like:
{"#providerName":"myProviderName","#provider":"myProvider","#realm":"myCatalog","unit":[{"#idKey":"newGeo_63119679"},{"#idKey":"newGeo_63119577"}]}
So service that does all the magic looks like:
public async ValueTask<string> AddAsset(string body)
{
XmlDocument doc = new XmlDocument();
doc.LoadXml(body);
string json = JsonConvert.SerializeXmlNode(doc, Formatting.None, true);
HttpResponseMessage response = await this._httpClient.PostAsJsonAsync("/Asset/create_asset", json);
string responseJson = await response.Content.ReadAsStringAsync();
return responseJson;
}
Well OK this part works, but when I remove from xml one unit node (so only one unit node is left), my result is:
{"#providerName":"myProviderName","#provider":"myProvider","#realm":"myCatalog","unit":{"#idKey":"newGeo_63119679"}}
And now so needed array to deserialize it to model is gone. I know I could manipulate xml attributes to add json:Array='true'.
But I was wondering if there is more complex solution for example JsonConverter that can take search for property in given type and check it if its collection and if so assign it as json collection. How can I bite this problem?
And Also as I checked SerializeXmlNode has no converter parameter.
Well, I found my answer, by mixing information from #dbc and my happy innovation:
public class Converter<TEntity> : JsonConverter<TEntity> where TEntity : class
{
private readonly IEnumerable<Type> _entityTypes =
Assembly.GetExecutingAssembly().GetReferencedAssemblies()
.SelectMany(assembly => Assembly.Load(assembly).GetTypes().Where(t => t.Namespace == MigrationService.EntitiesNameSpace));
public override bool CanWrite => false;
public override void WriteJson(JsonWriter writer, TEntity value, JsonSerializer serialize) =>
throw new NotImplementedException($"Write option is turned off for {nameof(CollectionConverter)} custom json converter");
public override TEntity ReadJson(JsonReader reader, Type objectType, TEntity existingValue, bool hasExistingValue, JsonSerializer serializer)
{
void SetSimpleTypes(JToken token, TEntity instance, PropertyInfo property)
{
var value = token[property.Name];
if(value == null) return;
var cast = Convert.ChangeType(value, property.PropertyType);
property.SetValue(instance, cast);
}
void SetClassTypes(JToken token, TEntity instance, PropertyInfo property)
{
var constructedConverterType = typeof(CollectionConverter<>).MakeGenericType(property.PropertyType);
if (Activator.CreateInstance(constructedConverterType) is not JsonConverter converter) return;
var propertyInstance = JsonConvert.DeserializeObject(token[property.Name].ToString(), property.PropertyType, converter);
property.SetValue(instance, propertyInstance);
}
void SetCollectionTypes(JToken token, TEntity instance, PropertyInfo property)
{
var constructedCollectionType = typeof(List<>).MakeGenericType(property.PropertyType.GetGenericArguments().Single());
var collectionInstance = Activator.CreateInstance(constructedCollectionType) as IList;
var value = token[property.Name];
void HandleSingleCollectionType(string body)
{
var propertyCollectionType = property.PropertyType.GetGenericArguments().Single();
var constructedConverterType = typeof(CollectionConverter<>).MakeGenericType(propertyCollectionType);
if (Activator.CreateInstance(constructedConverterType) is not JsonConverter converter) return;
var convertedInstance = JsonConvert.DeserializeObject(body, propertyCollectionType, converter);
collectionInstance?.Add(convertedInstance);
}
switch (value)
{
case JArray array:
foreach (var body in array)
{
HandleSingleCollectionType(body.ToString());
}
break;
case JObject:
HandleSingleCollectionType(value.ToString());
break;
default:
Debug.WriteLine($"Unknown or empty json token value for property {property.Name}");
break;
}
property.SetValue(instance, collectionInstance);
}
JToken token = JToken.Load(reader);
var instance = (TEntity)Activator.CreateInstance(typeof(TEntity));
var properties = instance?.GetType().GetProperties(BindingFlags.Public | BindingFlags.DeclaredOnly | BindingFlags.Instance);
if (properties == null) return default;
foreach (PropertyInfo property in properties)
{
try
{
if (property.PropertyType.IsGenericType && property.PropertyType.GetGenericTypeDefinition() == typeof(ICollection<>))
{
SetCollectionTypes(token, instance, property);
}
else if (this._entityTypes.Any(type => type == property.PropertyType) && property.PropertyType.IsClass)
{
SetClassTypes(token, instance, property);
}
else
{
SetSimpleTypes(token, instance, property);
}
}
catch (NullReferenceException ex)
{
Debug.WriteLine($"Null value for entity class property {property.Name}, exception: {ex}");
}
catch (FormatException ex)
{
Debug.WriteLine($"Can not convert value property {property.Name} in {instance.GetType().Name}, exception: {ex}");
}
catch (Exception ex)
{
Debug.WriteLine($"Undefined exception for {property.Name} exception: {ex}");
}
}
return instance;
}
}
It converts very complex xml structures based on class structure so if class structure says something is collection it will build collection, if its an object (I mean class) it will create instance of a class and so on. Additionally it uses itself to convert any child xml has.
I have an issue with handling JSON data from an api that I use in my application. The problem is that the JSON contains some properties that are an object when there is an item, and become an array when there are more items. So that's a structure like this:
[
{
"MyObj": {
"Foo": "Bar"
}
},
{
"MyObj": [
{
"Foo": "Bar1"
},
{
"Foo": "Bar2"
}
]
}
]
I've tried several JSON to C# converters, some of them generate a property of type object, the quicktype.io converter generates this:
public class Example
{
[JsonProperty("MyObj")]
public MyObjUnion MyObj { get; set; }
}
public partial class MyObjElement
{
[JsonProperty("Foo")]
public string Foo { get; set; }
}
public struct MyObjUnion
{
public MyObjElement MyObjElement;
public MyObjElement[] MyObjElementArray;
public static implicit operator MyObjUnion(MyObjElement MyObjElement) => new MyObjUnion { MyObjElement = MyObjElement };
public static implicit operator MyObjUnion(MyObjElement[] MyObjElementArray) => new MyObjUnion { MyObjElementArray = MyObjElementArray };
}
internal class MyObjUnionConverter : JsonConverter
{
public override bool CanConvert(Type t) => t == typeof(MyObjUnion) || t == typeof(MyObjUnion?);
public override object ReadJson(JsonReader reader, Type t, object existingValue, JsonSerializer serializer)
{
switch (reader.TokenType)
{
case JsonToken.StartObject:
var objectValue = serializer.Deserialize<MyObjElement>(reader);
return new MyObjUnion { MyObjElement = objectValue };
case JsonToken.StartArray:
var arrayValue = serializer.Deserialize<MyObjElement[]>(reader);
return new MyObjUnion { MyObjElementArray = arrayValue };
}
throw new Exception("Cannot unmarshal type MyObjUnion");
}
public override void WriteJson(JsonWriter writer, object untypedValue, JsonSerializer serializer)
{
var value = (MyObjUnion)untypedValue;
if (value.MyObjElementArray != null)
{
serializer.Serialize(writer, value.MyObjElementArray);
return;
}
if (value.MyObjElement != null)
{
serializer.Serialize(writer, value.MyObjElement);
return;
}
throw new Exception("Cannot marshal type MyObjUnion");
}
public static readonly MyObjUnionConverter Singleton = new MyObjUnionConverter();
}
Although this does work correctly, it's still a bit cumbersome because to get the data you always need to check if it's in the MyObjElement or MyObjElementArray class.
So the question is if there are other, more elegant ways to solve this issue. (Other than changing the API output, which is not mine)
If you know the data structure before hand, I would create a custom data model for the Json file and then deserialise it like so:
CurrencyExchangeRates deserialisedData = JsonConvert.DeserializeObject<CurrencyExchangeRates>(savedData);
foreach (CurrentRate value in deserialisedData.ExchangeRates)
{
rate.ExchangeRates.Add(new CurrentRate { Rate = value.Rate, Timestamp = value.Timestamp });
}
This is how I did that in an application. Hope this helps a little.
You can check and cast each object in your JSON depending the type of object.
For example consider using Newtonsoft.Json library for JSON parsing, you can do the following:
// considering your JSON string
string jsonString = "[{'MyObj':{'Foo':'Bar'}},{'MyObj':[{'Foo':'Bar1'},{'Foo':'Bar2'}]}]";
// parse your JSON into JTokens
var tokens = JToken.Parse(jsonString);
// iterate through all the tokens
foreach (var token in tokens)
{
// your token is a grand child
JToken myObj = token.First.First;
// check if the grand child is array
if (myObj is JArray)
{
// cast the grand child token into MyObj list or array object
IEnumerable<MyObj> objList = myObj.ToObject<List<MyObj>>();
Console.WriteLine("converted to MyObj Array");
}
else if (myObj is JObject) // else if its a non array item
{
// cast the grand child token into MyObj object
MyObj obj = myObj.ToObject<MyObj>();
Console.WriteLine("converted to MyObj");
}
}
// your MyObj Type will look like this:
public class MyObj
{
public string Foo { get; set; }
}
I have my serialized JSON in this format:
string json = #"[{"Name": "std_id","Value": "111"}, {"Name": "cust_id","Value": "444"}]"
How do I deserialize it to a single anonymous object like this:
var paramObj = new {"std_id" = 111, "cust_id" = 444}
Since you said the values of the Name and Value properties in your JSON objects can vary, you will not be able to deserialize to an anonymous object. Anonymous types are defined at compile-time, which means you need to know the property names ahead of time to be able to define them. The only way to get around that is code generation, which I think is going to be overkill for this situation. Instead, I would suggest you deserialize into a JObject with a dynamic variable. This will get you pretty close to what you want. Here's how:
string json = #"[
{ ""Name"": ""std_id"", ""Value"": ""111"" },
{ ""Name"": ""cust_id"", ""Value"": ""444"" }
]";
dynamic obj = new JObject(JArray.Parse(json)
.Select(t => new JProperty((string)t["Name"], t["Value"])));
From there, you can access the properties like you would for an anonymous type (assuming you know what they are):
Console.WriteLine(obj.std_id);
Console.WriteLine(obj.cust_id);
If you don't know what the properties are, you can enumerate them like a dictionary:
foreach (var prop in obj)
{
Console.WriteLine(prop.Name + ": " + prop.Value);
}
Fiddle: https://dotnetfiddle.net/MRY2ny
Why anonymous object? you should deserialize to a type like below
public class RootObject
{
public string Name { get; set; }
public string Value { get; set; }
}
Then what you actually have is IEnumerable<RootObjct>. You can use use Linq and select First() from it like
RootObject = RootObjects.FirstOrDefault()
You could deserialize it into a dynamic. Like this:
var serializer = new JavaScriptSerializer();
var deserializedResult = serializer.Deserialize<dynamic>(json);
Reference:
JavaScriptSerializer Class
var definition = new { Name = "" };
string json1 = #"{'Name':'James'}";
var customer1 = JsonConvert.DeserializeAnonymousType(json1, definition);
Console.WriteLine(customer1.Name);
string json2 = #"{'Name':'Mike'}";
var customer2 = JsonConvert.DeserializeAnonymousType(json2, definition);
Console.WriteLine(customer2.Name);
source http://www.newtonsoft.com/json/help/html/DeserializeAnonymousType.htm
I know that this solution isn't perfect, but it works for the example provided by you and returns result that looks like paramObj in your example.
The idea is to create a custom Json converter.
First, let's create a DTO class to present a name-value item of incomming JSON.
public class NameValueJsonItem
{
public string Name { get; set; }
public string Value { get; set; }
}
Converter implementation:
public class DynamicJsonConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return true;
}
public override object ReadJson(JsonReader reader, Type objectType,
object existingValue, JsonSerializer serializer)
{
JToken token = JToken.Load(reader);
if (token == null || token.Type != JTokenType.Array)
{
return null;
}
List<NameValueJsonItem> parsedJson = token.ToObject<List<NameValueJsonItem>>();
ExpandoObject result = new ExpandoObject();
foreach (NameValueJsonItem item in parsedJson)
{
if (!String.IsNullOrEmpty(item.Name))
{
(result as IDictionary<string, object>)[item.Name] = item.Value;
}
}
return result;
}
public override void WriteJson(JsonWriter writer, object value,
JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
Sure, you can make it more safe by adding some exceptions handling etc inside the method if you want.
You can use this converter like that:
dynamic result = JsonConvert.DeserializeObject<dynamic>(json, new DynamicJsonConverter());
Hope it will help.
In my C# program, I am querying a webservice and getting a reply stream back in JSON that looks something like this:
{"val":12345.12},{"val":23456.23},{"val":34567.01},...
or, with possibly more than 1 value per reply object:
{"val1":12345.12,"val2":1},{"val1":23456.23,"val2":3},....
And I have the following code utilizing the Newtonsoft.Json library that parses the stream and performs some action on each parsed object, one at a time:
public void ParseReply<T>(StreamReader sr, Action<T> action)
{
using (var reader = new JsonTextReader(sr))
{
var ser = new JsonSerializer();
while (reader.Read())
{
if (reader.TokenType == JsonToken.EndArray)
break;
action(ser.Deserialize<T>(reader));
}
}
}
So, in the case of the second result, I have the following code:
public class MyObject
{
public double val1;
public double val2;
}
and then:
myJson.ParseReply<MyObject>(sr, obj => ...);
works perfectly.
But, in the case of the first reply (1 value per object), if I try and use my method in the following way:
myJson.ParseReply<double>(sr, dbl => ...);
I get an error saying:
Cannot deserialize the current JSON object (e.g. {"name":"value"}) into type 'System.Double' because the type requires a JSON primitive value (e.g. string, number, boolean, null) to deserialize correctly.
To fix this error either change the JSON to a JSON primitive value (e.g. string, number, boolean, null) or change the deserialized type so that it is a normal .NET type (e.g. not a primitive type like integer, not a collection type like an array or List<T>) that can be deserialized from a JSON object. JsonObjectAttribute can also be added to the type to force it to deserialize from a JSON object.
I'm really lost / curious how I could update my code to be able to parse both of these correctly and I'm a bit lost on this error message. Any help would REALLY be appreciated!!
If you change your static method to return an IEnumerable<T> rather than taking an Action<T>, you will be able to chain together Enumerable and LINQ to JSON methods in a very concise and natural way. Thus if you have:
public static class JsonExtensions
{
public static IEnumerable<T> ParseArray<T>(this TextReader textReader)
{
using (var reader = new JsonTextReader(textReader))
{
bool inArray = false;
var ser = JsonSerializer.CreateDefault();
while (reader.Read())
{
if (reader.TokenType == JsonToken.Comment)
continue;
if (reader.TokenType == JsonToken.StartArray && !inArray)
{
inArray = true;
continue;
}
if (reader.TokenType == JsonToken.EndArray)
break;
yield return ser.Deserialize<T>(reader);
}
}
}
public static IEnumerable<JToken> DescendantsAndSelf(this JToken node)
{
// This method should be present on JToken but is only present on JContainer.
if (node == null)
return Enumerable.Empty<JToken>();
var container = node as JContainer;
if (container != null)
return container.DescendantsAndSelf();
else
return new[] { node };
}
public static bool IsNumeric(this JTokenType type) { return type == JTokenType.Integer || type == JTokenType.Float; }
public static bool IsNumeric(this JToken token) { return token == null ? false : token.Type.IsNumeric(); }
}
You can do:
var json = #"[{""val"":12345.12},{""val"":23456.23},{""val"":34567.01}]";
// Select all properties named "val" and transform their values to doubles.
foreach (var val in new StringReader(json).ParseArray<JToken>()
.Select(t => (double)t.SelectToken("val")))
{
Debug.WriteLine(val);
}
// Select all primitive numeric values
foreach (var val in new StringReader(json).ParseArray<JToken>()
.SelectMany(t => t.DescendantsAndSelf())
.Where(t => t.IsNumeric())
.Select(t => (double)t))
{
Debug.WriteLine(val);
}
I asked a question yesterday here about reading properties from an anonymous object and writing them to private fields of a class. The problem solved. Here is the short story:
I have some data in json format. I deserialize them to ExpandoObject, and pass them as IDictionary<string, object> to method. It works fine, except Int32 properties. It seems they change to Int64, where? I don't know.
Here is the method again:
private Func<IDictionary<string, object>, dynamic> MakeCreator(
Type type, Expression ctor,
IEnumerable<PropertyToFieldMapper> maps) {
var list = new List<Expression>();
var vList = new List<ParameterExpression>();
// creating new target
var targetVariable = Expression.Variable(type, "targetVariable");
vList.Add(targetVariable);
list.Add(Expression.Assign(targetVariable, Expression.Convert(ctor, type)));
// accessing source
var sourceType = typeof(IDictionary<string, object>);
var sourceParameter = Expression.Parameter(sourceType, "sourceParameter");
// calling source ContainsKey(string) method
var containsKeyMethodInfo = sourceType.GetMethod("ContainsKey", new[] { typeof(string) });
var accessSourceIndexerProp = sourceType.GetProperty("Item");
var accessSourceIndexerInfo = accessSourceIndexerProp.GetGetMethod();
// itrate over writers and add their Call to block
var containsKeyMethodArgument = Expression.Variable(typeof(string), "containsKeyMethodArgument");
vList.Add(containsKeyMethodArgument);
foreach (var map in maps) {
list.Add(Expression.Assign(containsKeyMethodArgument, Expression.Constant(map.Property.Name)));
var containsKeyMethodCall = Expression.Call(sourceParameter, containsKeyMethodInfo,
new Expression[] { containsKeyMethodArgument });
// creating writer
var sourceValue = Expression.Call(sourceParameter, accessSourceIndexerInfo,
new Expression[] { containsKeyMethodArgument });
var setterInfo = map.Field.GetType().GetMethod("SetValue", new[] { typeof(object), typeof(object) });
var setterCall = Expression.Call(Expression.Constant(map.Field), setterInfo,
new Expression[] {
Expression.Convert(targetVariable, typeof(object)),
Expression.Convert(sourceValue, typeof(object))
});
Console.WriteLine(Expression.Lambda(setterCall));
list.Add(Expression.IfThen(containsKeyMethodCall, setterCall));
}
list.Add(targetVariable);
var block = Expression.Block(vList, list);
var lambda = Expression.Lambda<Func<IDictionary<string, object>, dynamic>>(
block, new[] { sourceParameter }
);
return lambda.Compile();
}
If we have this
public class Person {
public int Age { get; set; }
public string Name { get; set; }
}
class, and use this object
var data = new { Name = "Amiry", Age = 20 };
to initialize an instance of Person using above method, this error occurs:
Object of type 'System.Int64' cannot be converted to type 'System.Int32'.
But if we change Age property to:
public long Age { get; set; }
every thing looks fine and method works perfectly. I completely confused about why this happens. Do you have any idea?
The expression is correct. The problem is Json.NET. It converts all numeric values (in anonymous conversions) to Int64. So, I just need a custom convertor:
public class JsonIntegerConverter : JsonConverter {
public override bool CanConvert(Type objectType) {
return objectType == typeof(IDictionary<string, object>);
}
public override bool CanWrite {
get { return false; }
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) {
var result = new Dictionary<string, object>();
reader.Read();
while (reader.TokenType == JsonToken.PropertyName) {
var propertyName = (string)reader.Value;
reader.Read();
object value;
if (reader.TokenType == JsonToken.Integer) {
var temp = Convert.ToInt64(reader.Value);
if (temp <= Byte.MaxValue && temp >= Byte.MinValue)
value = Convert.ToByte(reader.Value);
else if (temp >= Int16.MinValue && temp <= Int16.MaxValue)
value = Convert.ToInt16(reader.Value);
else if (temp >= Int32.MinValue && temp <= Int32.MaxValue)
value = Convert.ToInt32(reader.Value);
else
value = temp;
} else
value = serializer.Deserialize(reader);
result.Add(propertyName, value);
reader.Read();
}
return result;
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) {
throw new NotSupportedException();
}
}
This is a concrete implementation, and absolutely can be implemented more extended and useful. But it just solve my current problem.
So your input Dictionary contains long (based on discussion in comments).
The easiest fix is to add Convert.ChangeType before SetValue.
(passing in sourceValue and Constant(map.Field.FieldType))
However, this may have an unintended consequence of allowing string -> int conversion.
Alternative is to add your own ConvertType method, where you decide how types are converted.