I am getting a big nested json object from an external api. I want to make sure all fields are populated in that json object. Is there any libray available to do that? Newtonsoft.Json has JToken class which checks if the Json's schema is valid or not but i don't find any method to check if all the fields inside json object are filled.
Scenario : I am building an api which gathers information about a person or entity. there are numerous sources of information. I need to continue searching for the data until the required object is full. So first call goes to api1, gets some data, checks if object is full. If object is not full then goes to api2 and so on. So the call returns after the object is full. One key point is that the required object schema is not static.
I could deserialize it to a POCO and loop through each nested object but i am looking for a better solution.
Any suggestion is highly appreciated.
If I understood you correctly, you have some complex JSON object, with a set of required and not-required properties.
I see here three solutions:
You've already mentioned JSON schema validation. Here is one of the implementations: Json.NET Schema.
Create a POCO object and mark all properties with corresponding Required/Not-Required attributes. In this case, you'll get exceptions if a json-string doesn't contain any mandatory data. For example, how it's done in terms of Json.NET's JsonPropertyAttribute:
[JsonObject]
public class PocoObject
{
[JsonProperty("$schema", Required = Required.Default, NullValueHandling = NullValueHandling.Ignore)]
public string Schema { get; set; }
[JsonProperty("name", Required = Required.Always)]
public string Name { get; set; }
[JsonProperty("properties", Required = Required.Always)]
public MorePropertiesObject Properties { get; set; }
}
As a bonus, you can add your custom attributes, methods, converters, etc
Deserialize raw json into a Dictionary-like structure and validate it with your own hands-written validator. Something like:
try
{
var jo = JObject.Parse(jsonString);
Contract.Assert(!string.IsNullOrEmpty(jo["root"]["prop1"].ToString()));
Contract.Assert(!string.IsNullOrEmpty(jo["root"]["prop2"].ToString()));
}
catch (JsonReaderException) { }
catch (JsonSerializationException) { }
Provided code samples are for mentioned by you Newtonsoft.Json
If you have a JSON string and simply want to check whether any property value or array item are null, you can parse to a JToken, recursively descend the JToken hierarchy with JContainer.DescendantsAndSelf(), and check each JValue for being null using the following extension method:
public static partial class JsonExtensions
{
public static bool AnyNull(this JToken rootNode)
{
if (rootNode == null)
return true;
// You might consider using some of the other checks from JsonExtensions.IsNullOrEmpty()
// from https://stackoverflow.com/questions/24066400/checking-for-empty-null-jtoken-in-a-jobject
return rootNode.DescendantsAndSelf()
.OfType<JValue>()
.Any(n => n.Type == JTokenType.Null);
}
public static IEnumerable<JToken> DescendantsAndSelf(this JToken rootNode)
{
if (rootNode == null)
return Enumerable.Empty<JToken>();
var container = rootNode as JContainer;
if (container != null)
return container.DescendantsAndSelf();
else
return new[] { rootNode };
}
}
And then do:
var root = JToken.Parse(jsonString);
var anyNull = root.AnyNull();
If you just want to check for null property values (i.e. null values in an array are OK) you can use the following extension method:
public static partial class JsonExtensions
{
public static bool AnyNullPropertyValues(this JToken rootNode)
{
if (rootNode == null)
return true;
return rootNode.DescendantsAndSelf()
.OfType<JProperty>()
.Any(p => p.Value == null || p.Value.Type == JTokenType.Null);
}
}
(It isn't entirely clear from your question which you want.)
Sample .Net fiddle.
Related
I use Manatee.Json.JsonValue a whole lot, for instance in a case I have now, where I have a property of type object. This object property can contain both literals or complex types.
When the property value contains a complex type of JsonValue, it is serialized as an empty object! Is it possible to avoid this, such that other property values are serialized, while object property values of type JsonValue are just copied?
Here is an example with Manatee.Json.13.0.2:
public class TestObject
{
public object Property { get; set; }
}
[TestClass]
public class TestClass
{
[TestMethod]
public void Test()
{
var testObject = new TestObject
{
Property = new JsonValue("testString")
};
JsonValue expected = new JsonValue(
new JsonObject
{
["Property"] = "testString",
}
);
var serializer = new Manatee.Json.Serialization.JsonSerializer();
JsonValue actual = serializer.Serialize(testObject);
var expectedString = expected.ToString();
/// expectedString "{\"Property\":\"testString\"}" string
var actualString = actual.ToString();
/// actualString "{\"Property\":{}}" string
Assert.AreNotEqual(expectedString, actualString);
}
}
Anyone have any idea how to do this?
So, I've come to find a solution that seems to cover my use case.
I have managed to create a simple JsonSerializer that copies values. Upon serialization, the goal is to go from the source object to a JsonValue target. If the source value is either a JsonValue, a JsonObject or a JsonArray instance, the serialization method returns a JsonValue. Other types throws an ArgumentOutOfRangeException.
Upon deserialization, the goal is to go from a JsonValue and back to the requested type. If the requested type is a JsonValue, the current JsonValue are returned. If the requested type is a JsonObject, either the current JsonValue.Object are returned, or a new JsonObject if JsonValue.Object is null. Same goes for JsonArray: JsonValue.Array ?? new JsonArray. Other requested types returns null;
The serializer must be registered with the SerializerFactory.AddSerializer(mySerializer) somewhere. For me it is typically in Global.asax.cs and in a MSTest [AssemblyInitialize()] method.
Here is my implementation at this point:
public class JsonValueSerializer : ISerializer
{
public bool ShouldMaintainReferences => false;
private Type[] HandledTypes = {
typeof(JsonValue),
typeof(JsonObject),
typeof(JsonArray)
};
public bool Handles(SerializationContextBase context)
{
return HandledTypes.Contains(context.InferredType);
}
public JsonValue Serialize(SerializationContext context)
{
if (context.Source is JsonValue jsonValue)
return jsonValue;
else if (context.Source is JsonObject jsonObject)
return new JsonValue(jsonObject);
else if (context.Source is JsonArray jsonArray)
return new JsonValue(jsonArray);
else
throw new ArgumentOutOfRangeException();
}
public object Deserialize(DeserializationContext context)
{
if (context.RequestedType == typeof(JsonValue))
{
return context.LocalValue;
}
else if (context.RequestedType == typeof(JsonObject))
{
return context.LocalValue.Object ?? new JsonObject();
}
else if (context.RequestedType == typeof(JsonArray))
{
return context.LocalValue.Array ?? new JsonArray();
}
return null!;
}
}
Weakness of this implementation:
Deserialization can be done with serializer.Deserialize<JsonObject>(myJsonValue). A problem with this implementation, is that when I have a complex class with an object property, and that property contains a JsonObject (or a JsonArray), it works to serialize the complex type into JsonValue, but upon deserialization, the previous JsonObject would be deserialized into a JsonValue of JsonValueType.Object. This is different from the original JsonObject, since JsonValue can contain a JsonObject, but JsonObject is a type in its own right.
Without additional mechanisms, it would be impossible to know if the JsonValue should be kept or if it should be turned back into a JsonObject or a JsonArray. One could decide that on deserialization, all JsonValue.Object would be constantly set to a JsonObject type, but that would be wrong when it initially was a JsonValue of JsonValueType.Object (same for JsonArray). Another approach could maybe be to let JsonObject get a "$type" parameter with the information, but that wouldn't work with JsonArray. I have no fix for this behaviour.
I'm trying to deserialize some Json objects using Json.NET. I'd like to be able to detect if I have a member in my class missing from the Json properties for that object. For instance, I have a class which looks like this :
public class MyClass
{
public int n;
public bool b;
public string s;
}
And a Json which looks like this
{"n":1,"b":true}
so it's missing the "s" property. When I try to deserialize that, members which are not in the Json will have default value. So "s" will be equal to null. Fair enough, but is it possible to detect that when I'm deserializing ?
In substance, I want to do pretty much the opposite of this Stackoverflow post
But in my case, the MissingMemberHandling setting seems to do nothing, sadly.
Json.Net provides a way to achieve that.
You can set an attribute on the property in your Model class. and if
that property is not available in JSON it'll throw an exception.
Here is the Example
Model
public class Videogame
{
[JsonProperty(Required = Required.Always)]
public string Name { get; set; }
[JsonProperty(Required = Required.AllowNull)]
public DateTime? ReleaseDate { get; set; }
}
Test Code
string json = #"{
'Name': 'Starcraft III'
}";
Videogame starcraft = JsonConvert.DeserializeObject<Videogame>(json);
You can read more about this here
This is what i can think of at the moment.
public void CheckData()
{
if(n == null)
{
Console.Write("n is null");
}
if (b == null)
{
Console.Write("b is null");
}
if (s == null)
{
Console.Write("s is null");
}
}
Run a method like that after json maps to the object to see if there were any still null.
Or you can try something like this
var fields = typeof(MyClass).GetFields();
foreach (var field in fields)
{
if(!json.ContainsKey(field.Name))
{
Console.Write($"{field.Name} is missing");
}
}
I have a Serializer/Deserializer that use a references PreserveReferencesHandling = PreserveReferencesHandling.All.
The issue, is that I have circular references.
Here is a very simple example.
class Node
{
public Node(object value)
{
Value = value;
}
public object Value { get; set; }
public Node Left { get; set; }
public Node Right { get; set; }
}
My test scenario is:
var obj = new Node("o")
{
Left = new Node("oL"),
Right = new Node("oR")
};
obj.Right.Right = obj; // Circular reference!
When I deserialize, i have the following IReferenceResolver
private class InternalReferenceResolver : IReferenceResolver
{
private readonly Deserializer _par;
public InternalReferenceResolver(Deserializer par)
{
_par = par;
}
public object ResolveReference(object context, string reference)
{
object refObject;
if (!_par._processed.TryGetValue(reference, out refObject))
{
refObject = _par.DeserializeObject(reference);
}
return refObject;
}
public string GetReference(object context, object value)
{
throw new NotSupportedException("Only for Serialization");
}
public bool IsReferenced(object context, object value)
{
return false;
}
public void AddReference(object context, string reference, object value)
{
_par._processed.Add(reference, value);
}
}
As you can see, when JSON.NET inform me of a new ref->object (via AddReference()) I add it to a dictionary.
When JSON.NET request an object for a specific reference (via ResolveReference()) I recurse, and deserialize that reference.
The issue is, JSON.NET calls ResolveReference() for each of the object references, before it calls it's AddReference().
I would expect the flow of Deserialization to be:
Identify Object Type
Construct the object
AddReference(id, newObj)
Resolve References + Populate Properties
What I see happens is:
Identify Object Type
Resolve References
Construct the object
AddReference(id, newObj)
Populate Properties
My Questions:
Why is it made the latter, am i missing something with my suggested flow?
how can I overcome this issue, having a "Bare" object just for referencing and only then actually resolve the references ?
The Two-Phase deserialization you are seeing is arises because your Node class only has a parameterized constructor. As explained in Issue with serializing/deserializing object graph with self-referencing objects in combination with JSON constructor. #715:
JamesNK commented on Nov 28, 2015
Non-default constructors and preserving references don't work well together because the child values of a type have to be deserialized before the parent is created, so the reference resolves to null.
Thus, since Value is a mutable property anyway, you should add a parameterless constructor to Node:
class Node
{
public Node() : this(null) { }
public Node(object value)
{
Value = value;
}
// Remainder unchanged.
}
It can be non-public if you mark it with [JsonConstructor] or deserialize using the setting ConstructorHandling.AllowNonPublicDefaultConstructor. And if Value were immutable, you would need to make it privately settable and mark it with [JsonProperty]
[JsonProperty]
public object Value { get; private set; }
(Data contract attributes can be used in place of Json.NET attributes if you prefer.)
Notes:
Since your question lacks a complete and verifiable example, your code might encounter other problems that aren't fixed by adding a parameterless constructor.
For a related question, see Usage of non-default constructor breaks order of deserialization in Json.net.
For a related issue, see PreserveReferencesHandling.Objects deserialize does not work with non-default constructor #678.
Another thing, can I tell Json.NET to only handle the constructor parameters and leave the rest... ?
Not according to Issue 715. Since a JSON object is an unordered set of name/value pairs, Json.NET needs to parse the entire object to ensure it has loaded all the constructor parameters. As it is a single-pass serializer the non-constructor parameters will get loaded into... something... before the object can be constructed. Json.NET has chosen to deserialize them in one step to the final target member type rather than to an intermediate JToken and later the final member type. This can be see in JsonSerializerInternalReader.ResolvePropertyAndCreatorValues().
Well, I Found a solution for my problem:
The first Deserialization i perform, i use a custom IContractResolver that exclude all properties that are not relevant to the constructor...
at the second pass, I use Populate and use the default IContractResolver
private class InternalOnlyCtorContractResolver : IContractResolver
{
private readonly IContractResolver _base;
public InternalOnlyCtorContractResolver(IContractResolver _base)
{
this._base = _base;
}
public JsonContract ResolveContract(Type type)
{
var contract = _base.ResolveContract(type);
var objectContract = contract as JsonObjectContract;
if (objectContract != null)
{
var creatorParameters = new HashSet<string>(objectContract.CreatorParameters.Select(p => p.PropertyName));
var irrelevantProperties = objectContract.Properties
.Where(p => !creatorParameters.Contains(p.PropertyName))
.ToArray();
foreach (var irrelevantProperty in irrelevantProperties)
{
objectContract.Properties.Remove(irrelevantProperty);
}
//TODO Can be optimized better
}
return contract;
}
}
If for some reason, the Circular Reference is needed by the Constructor,
It still cause a loop, but it is kinda impossible to create without having a second constructor anyway.
When serializing an object, I have a List<T> property where T is an abstract type which I'd like to serialize naturally. However, when deserializing the object, I want/need to manually deserialize this abstract list. I'm doing the latter part with a custom JsonConverter which looks at a property on each item
in the list and then deserializes it to the correct type and puts it in the list.
But I also need to deserialize the rest of the object, without doing it property-by-property. Pseudo-code follows.
MyObject
class MyObject {
public Guid Id;
public string Name;
public List<MyAbstractType> Things;
}
MyObjectConverter
class MyObjectConverter : JsonConverter {
public override object ReadJson(reader, ..., serializer) {
// Line below will fail because the serializer will attempt to deserlalize the list property.
var instance = serializer.Deserialize(reader);
var rawJson = JObject.Load(reader); // Ignore reading from an already-read reader.
// Now loop-over and deserialize each thing in the list/array.
foreach (var item in rawJson.Value<JArray>(nameof(MyObject.Things))) {
var type = item.Value<string>(nameof(MyAbstractType.Type));
MyAbstractType thing;
// Create the proper type.
if (type == ThingTypes.A) {
thing = item.ToObject(typeof(ConcreteTypeA));
}
else if (type == ThingTypes.B) {
thing = item.ToObject(typeof(ConcreteTypeB));
}
// Add the thing.
instance.Things.Add(thing);
}
return instance;
}
}
To recap, I want to manually handle the deserialization of Things, but allow them to naturally serialize.
So I'm using json.net to deserialize some data from the interwebs. And the bumblebrains providing the data are making it hard for me. The data type of one of the properties of the object I'm trying to deserialize varies depending on the contents of another property in the object, like so:
{
"typeindicator": "A",
"object_I_Need": { ...formatted for object type A ... }
}
Or
{
"typeindicator": "B",
"object_I_Need": { ...formatted for object type B ... }
}
I need the "object_I_Need" regardless of whether it's type A or B and I have the schema for both types.
I don't need to solve this completely in the deserializer. I just JsonExtensionDataAttribute to check for unknown fields so I was wondering if I could just let the object_I_Need fall into that and then deserialize that seperately... is there a way to do that?
What are my options here? I guess I could deserialize the whole thing into a dynamic object, decide what type the object_I_Need is and serialize it again? I hope there's a better way, though. Suggestions appreciated.
Thanks,
Crash
You can use Newtonsoft to deserialize into a dynamic object and then build you object from there.
var dyn = JsonConvert.DeserializeObject<dynamic>(rawJson);
var myObject = new O();
if (dyn.typeindicator.Value == "A") {
myObject.PropA = dyn.object_I_Need.AAAA.Value;
}
else
{
myObject.PropA = dyn.object_I_Need.anotherA.Value;
}
Here's what works for me here:
public class DeCrash
{
[JsonProperty("object_I_Need")]
protected JObject ObjectINeed;
[JSonProperty("typeindicator")]
public String TypeIndicator { get; set; }
[JsonIgnore]
public TypeA ObjectA
{
get
{
return TypeIndicator == "A" ? ObjectINeed.ToObject<TypeA> : null;
}
}
[JsonIgnore]
public TypeB ObjectB
{
get
{
return TypeIndicator == "B" ? ObjectINeed.ToObject<TypeB> : null;
}
}
}