Develop Reference

Can dapper deserialize json stored as text?

public class MyType
{
public int Id { get; set;}
public int[] MyArray { get; set; }
}
var sql = "SELECT id, MyArrayAsJson as MyArray";
var x = await connection.QueryAsync<MyType>(sql);
I have a string stored in the database which looks like json: [1,2,3,4,5]
When I query the db with Dapper, I would like dapper to deserialize to an object, MyType. Dapper wants MyArrayAsJson to be a string because it is, but I want it to deserialize to an int array. Is this possible?

public class JsonTypeHandler : SqlMapper.ITypeHandler
{
public void SetValue(IDbDataParameter parameter, object value)
{
parameter.Value = JsonConvert.SerializeObject(value);
}
public object Parse(Type destinationType, object value)
{
return JsonConvert.DeserializeObject(value as string, destinationType);
}
}
SqlMapper.AddTypeHandler(typeof(int[]), new JsonTypeHandler());

Dapper wants nothing to do with your fancy serialization shenanigans :) Basically, no: read it from the database as a string, then deserialize.
Adding an API that provided more direct / raw access to the incoming data as a BLOB/CLOB sequence would be nice, but it doesn't exist in Dapper today.

Dapper cannot do that, but you can do that!
It is just a little work using Dapper Multi Mapping
Select your [JsonColumn] as a string, and while mapping the results deserialize the JSON column, for example, if you have a Foo class and you want to deserialize the [JsonColumn] to it, do that:
using (var db = _context.Connect())
{
return await db.QueryAsync<Foo, string, Foo>(query, (foo, jsonColumnString) =>
{
return JsonSerializer.Deserialize<Foo>(jsonColumnString);
},
splitOn: "JsonColumn",
param: parameters);
}

How to serialize only not null properties in derived class in Json.Net

Entity objects needs converted to string to store them in log file for human reading.
Properties may added to derived entities at runtime. Serialize is defined in entity base class.
I triesd code below but it returns properties with null values also.
Since many properties have null values, it makes logs difficult to read.
Hoq to serialize only not-null properties ?
I tried also
var jsonSettings = new JsonSerializerSettings()
{
DefaultValueHandling = DefaultValueHandling.Ignore,
Formatting = Formatting.Indented,
TypeNameHandling= TypeNameHandling.All
};
return JsonConvert.SerializeObject(this, GetType(), jsonSettings);
this serializes only EntityBase class properties as confirmed in newtonsoft json does not serialize derived class properties
public class EntityBase {
public string Serialize()
{
var s = new System.Web.Script.Serialization.JavaScriptSerializer();
s.Serialize(this);
}
}
public class Customer: EntityBase {
public string Name, Address;
}
testcase:
var cust= new Customer() { Name="Test"};
Debug.Writeline( cust.Serialize());
Observed: result contains "Address": null
Expected: result shoud not contain Address property.
ASP.NET/Mono MVC4, .NET 4.6 are used

You can create your own serializer to ignore null properties.
JsonConvert.SerializeObject(
object,
Newtonsoft.Json.Formatting.None,
new JsonSerializerSettings
{
NullValueHandling = NullValueHandling.Ignore
});

Newtonsoft JSON schema compare containing object type

I have a model like the one described below.
public class QueryValueDataModel
{
public object Value { get; set; }
public QueryValueType ValueType { get; set; }
}
I have a JSON serialization of QueryValueDataModel type:
{
"Value": 100,
"ValueType": 0
}
When I compare the schema of the JSON object with the QueryValueDataModel schema
var obj = JToken.Load(reader);
IList<ValidationError> errorMessages = new List<ValidationError>();
SchemaExtensions.IsValid(obj, schema, out errorMessages)
I get an error and the schema don't match:
Invalid type. Expected Object, Null but got Integer
How can I deal with this situation and make the schema comparison work? Or alternately, is there some workaround for it?

You can use JSchemaGenerator to create JSchema from any object and compare it with JObject.IsValid() method
JSchemaGenerator generator = new JSchemaGenerator();
JSchema schema = generator.Generate(typeof(QueryValueDataModel));
JObject o = JObject.Parse(jsonstr);
if(o.IsValid(schema))
{
QueryValueDataModel deserialized = o.ToObject(typeof(QueryValueDataModel));
}
else
{
throw new Exception("schema dosn't match");
}
Please note! JSchemaGenerator has been moved to Newtonsoft.Json.Schema nuget package.

How to handle null/empty values in JsonConvert.DeserializeObject

I have the following code:
return (DataTable)JsonConvert.DeserializeObject(_data, (typeof(DataTable)));
Then, I tried:
var jsonSettings = new JsonSerializerSettings
{
NullValueHandling = NullValueHandling.Ignore
};
return (DataTable)JsonConvert.DeserializeObject<DataTable>(_data, jsonSettings);
The return line is throwing the error:
{"Error converting value \"\" to type 'System.Double'."}
Lots of solutions online suggesting creating custom Class with nullable types but this won't work for me. I can't expect the json to be in a certain format. I have no control over the column count, column type, or column names.

You can supply settings to JsonConvert.DeserializeObject to tell it how to handle null values, in this case, and much more:
var settings = new JsonSerializerSettings
{
NullValueHandling = NullValueHandling.Ignore,
MissingMemberHandling = MissingMemberHandling.Ignore
};
var jsonModel = JsonConvert.DeserializeObject<Customer>(jsonString, settings);

An alternative solution for Thomas Hagström, which is my prefered, is to use the property attribute on the member variables.
For example when we invoke an API, it may or may not return the error message, so we can set the NullValueHandling property for ErrorMessage:
public class Response
{
public string Status;
public string ErrorCode;
[JsonProperty(NullValueHandling = NullValueHandling.Ignore)]
public string ErrorMessage;
}
var response = JsonConvert.DeserializeObject<Response>(data);
The benefit of this is to isolate the data definition (what) and deserialization (use), the deserilazation needn’t to care about the data property, so that two persons can work together, and the deserialize statement will be clean and simple.

You can subscribe to the 'Error' event and ignore the serialization error(s) as required.
static void Main(string[] args)
{
var a = JsonConvert.DeserializeObject<DataTable>("-- JSON STRING --", new JsonSerializerSettings
{
Error = HandleDeserializationError
});
}
public static void HandleDeserializationError(object sender, ErrorEventArgs errorArgs)
{
var currentError = errorArgs.ErrorContext.Error.Message;
errorArgs.ErrorContext.Handled = true;
}

ASP.NET CORE:
The accepted answer works perfectly. But in order to make the answer apply globally, in startup.cs file inside ConfigureServices method write the following:
services.AddControllers().AddNewtonsoftJson(options =>
{
options.SerializerSettings.NullValueHandling = NullValueHandling.Ignore;
});
The answer has been tested in a .Net Core 3.1 project.

Newtonsoft JSON.NET Deserialization error

I am have a bunch of long json output in individual files. I need to read these files and deserialize them into the entities that originally generated the json (I have access to the original entities). Each file has the json output that was generated by serializing an object of type IEnumerable<Response<MyEntity>>.
I am getting an exception when attempting to deserialize, but I don't understand why. Here is my attempt to deserialize:
List<string> jsonOutputStrings;
// Read json from files and populate jsonOutputStrings list
List<Response<MyEntity>> apiResponses = new List<Response<MyEntity>>();
foreach (string json in jsonOutputStrings)
{
apiResponses.AddRange(JsonConvert.DeserializeObject<List<Response<MyEntity>>>(json, new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All }).ToList());
}
I also tried deserializing to an IEnumerable instead of a list:
apiResponses.AddRange(JsonConvert.DeserializeObject<IEnumerable<Response<MyEntity>>>(json, new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All }).ToList());
I get the following exception:
A first chance exception of type
'Newtonsoft.Json.JsonSerializationException' occurred in
Newtonsoft.Json.dll
Additional information: Cannot create and populate list type
System.Linq.Enumerable+WhereSelectListIterator`2[Entities.Requirement,Entities.RequirementEntity].
Path
'$values[0].ReturnedEntity.Summaries.$values[0].Requirements.$values',
line 1, position 715.
The entire json is too long to post (and also contains some confidential data) but I did find a place in the json that has this:
"Requirements":{"$id":"7","$type":"System.Linq.Enumerable+WhereSelectListIterator`2[[Entities.Requirement, Entities],[Entities.RequirementEntity, API.Entities]], System.Core","$values":[...]}
In the entity that I'm trying to deserialize into (same one that was originally serialized), Requirements is of type IEnumerable<Entities.RequirementEntity>.
It doesn't make sense to me how serialization from MyEntity works but deserializing to the same type doesn't. How do I solve this problem?

You must browse through Response<> and MyEntity and see how collections are initialized. This tells us that one of the collections in some class is created using Where method from linq. You can reproduce this error by executing this code:
class MyEntity
{
public MyEntity()
{
Data = new List<string>().Where(x => true);
}
public IEnumerable<string> Data { get; set; }
}
class Program
{
static void Main(string[] args)
{
string data = #"[{""Data"":[""a"",""b""]}]";
var j = JsonConvert.DeserializeObject<IEnumerable<MyEntity>>(data);
}
}
Another possibility is to have metadata in json. Then you have 2 solutions:
Set TypeNameHandling to TypeNameHandling.None (as somebody mentioned in comment);
Replace not working types in string with working ones
Using TypeNameHandling.None may lead to wrong deserialization for exmaple when you have IEnumerable<BaseType> and that list contains subtype of BaseType.
In that case you should go for second option. Basically you should replace any type that is not deserializing and replace it for example with List.
Sample code:
class MyEntity
{
public IEnumerable<string> Data { get; set; }
}
class Program
{
static void Main(string[] args)
{
IList<MyEntity> entities = new MyEntity[] {
new MyEntity { Data = new [] { "1", "2" }.Where(x => x != string.Empty) },
new MyEntity { Data = new [] { "A", "B" }.AsQueryable().Where(x => x != string.Empty) },
new MyEntity { Data = new List<string> { "A", "B" } },
};
string data = JsonConvert.SerializeObject(entities, Formatting.Indented, new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All });
data = Regex.Replace(data, "\"\\$type\":\\s+\"System.Linq.Enumerable\\+WhereArrayIterator(.+?), System.Core\",", "\"$type\": \"System.Collections.Generic.List$1, mscorlib\",", RegexOptions.Singleline);
var j = JsonConvert.DeserializeObject<IEnumerable<MyEntity>>(data, new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All });
}
}

Is there a Linq result in the serialized entity? Maybe defined as an IEnumerable and filled with a Linq Where result? It seems that's where your problem lies.
You should convert it to a List or Array before serializing.