Serialize and Deserialize CSV file as Stream - c#

I'm trying to send a Stream of data to an API through my own API. The third-party API takes an object and the object has an object property who has a property value of Stream. In my code that consumes the my API, I need to read a CSV file and then serialize the DTO object which containts the Stream property to send it to my API. My API will then just pass the DTO object to the third party API.
My issue is that I am currently serializing the Stream property as a base 64 string, but am unable to de-serialize it back into the DTO object.
DTO Object:
public class BulkLeadRequest
{
public Format3 FileFormat { get; set; }
public FileParameter FileParameter { get; set; }
public string LookupField { get; set; }
public string PartitionName { get; set; }
public int? ListId { get; set; }
public int? BatchId { get; set; }
}
Format3 is an enum:
public enum Format3
{
[System.Runtime.Serialization.EnumMember(Value = #"csv")]
Csv = 0,
[System.Runtime.Serialization.EnumMember(Value = #"tsv")]
Tsv = 1,
[System.Runtime.Serialization.EnumMember(Value = #"ssv")]
Ssv = 2,
}
FileParamter object with Stream property:
public partial class FileParameter
{
public FileParameter(System.IO.Stream data)
: this(data, null)
{
}
public FileParameter(System.IO.Stream data, string fileName)
: this(data, fileName, null)
{
}
public FileParameter(System.IO.Stream data, string fileName, string contentType)
{
Data = data;
FileName = fileName;
ContentType = contentType;
}
public System.IO.Stream Data { get; private set; }
public string FileName { get; private set; }
public string ContentType { get; private set; }
}
In order to test the serialization I have this small unit test.
public void TestSerializationDeserialization()
{
BulkLeadRequest bulkLeadRequest = new BulkLeadRequest();
bulkLeadRequest.FileFormat = Format3.Csv;
bulkLeadRequest.FileParameter = new FileParameter(new StreamReader("C:\temp\\SmallFile.csv").BaseStream);
string serializedObject = JsonConvert.SerializeObject(bulkLeadRequest, new StreamStringConverter());
var obj = JsonConvert.DeserializeObject<BulkLeadRequest>(serializedObject,
new JsonSerializerSettings {Converters = new List<JsonConverter> {new StreamStringConverter()}}); // Issue here
Assert.Equal(bulkLeadRequest, obj);
}
The StreamStringConverter:
public class StreamStringConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return typeof(Stream).IsAssignableFrom(objectType);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
string objectContents = (string)reader.Value;
byte[] base64Decoded = Convert.FromBase64String(objectContents);
MemoryStream memoryStream = new MemoryStream(base64Decoded);
return memoryStream;
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
FileStream valueStream = (FileStream)value;
byte[] fileBytes = new byte[valueStream.Length];
valueStream.Read(fileBytes, 0, (int)valueStream.Length);
string bytesAsString = Convert.ToBase64String(fileBytes);
writer.WriteValue(bytesAsString);
}
}
In my TestSerializationDeserialization unit test, I get the following error:
An exception of type 'Newtonsoft.Json.JsonSerializationException' occurred in Newtonsoft.Json.dll but was not handled in user code
Unable to find a constructor to use for type Cocc.MarketoSvcs.Business.FileParameter. A class should either have a default constructor, one constructor with arguments or a constructor marked with the JsonConstructor attribute. Path 'FileParameter.Data', line 1, position 40.
If I add a default constructor to FileParameter then the Data property will be empty when deserialized. If I remove the implicit conversion in the JsonConvert.DeserializeObject<BulkLeadRequest>(..); call to JsonConvert.DeserializeObject(...); I will get an object, but not the BulkLeadRequest and with the base64 string still and not the Stream object I'd expect.
Serialized:
{"FileFormat":0,"FileParameter":{"Data":"RklSU1ROQU1FLE1ETElOSVQsTEFTVE5BTUUNCkNyaXN0aW5hLE0sRGlGYWJpbw0KTmVsbHksLFBhbGFjaW9zDQpNYXR0aGV3LEEsTmV2ZXJz","FileName":null,"ContentType":null},"LookupField":null,"PartitionName":null,"ListId":null,"BatchId":null}
What am I doing wrong?

You are getting this error because the serializer does not know how to instantiate your FileParameter class. It prefers to use a default constructor if possible. It can use a parameterized constructor in some cases, if you give it a hint as to which one by marking it with a [JsonConstructor] attribute and all the constructor parameters match up to properties in the JSON object. However, that won't work in your case because all of your constructors expect a Stream and you require special handling for that.
To solve this you need a way for the serializer to instantiate your class without the stream and then use the converter to create the stream. You said you tried adding a default constructor, but then the stream was null. The reason it did not work is because all of your setters are private. With the default constructor, the serializer was able to instantiate the FileParameter, but then it could not populate it.
There are a couple of ways to make it work.
Solution 1 - modify the FileParameter class
Add a default constructor to the FileParameter class as you did before. It can be private if you also mark it with [JsonConstructor].
Add [JsonProperty] attributes to all of the private-set properties that need to be populated from the JSON. This will allow the serializer to write to the properties even though they are private.
So your class would look like this:
public partial class FileParameter
{
public FileParameter(System.IO.Stream data)
: this(data, null)
{
}
public FileParameter(System.IO.Stream data, string fileName)
: this(data, fileName, null)
{
}
public FileParameter(System.IO.Stream data, string fileName, string contentType)
{
Data = data;
FileName = fileName;
ContentType = contentType;
}
[JsonConstructor]
private FileParameter()
{
}
[JsonProperty]
public System.IO.Stream Data { get; private set; }
[JsonProperty]
public string FileName { get; private set; }
[JsonProperty]
public string ContentType { get; private set; }
}
Here is a working demo: https://dotnetfiddle.net/371ggK
Solution 2 - use a ContractResolver
If you cannot easily modify the FileParameter class because it is generated or you don't own the code, you can use a custom ContractResolver to do the same thing programmatically. Below is the code you would need for that.
public class CustomContractResolver : DefaultContractResolver
{
protected override JsonObjectContract CreateObjectContract(Type objectType)
{
JsonObjectContract contract = base.CreateObjectContract(objectType);
if (objectType == typeof(FileParameter))
{
// inject a "constructor" to use when creating FileParameter instances
contract.DefaultCreator = () => new FileParameter(null);
// make the private properties writable
string[] propNames = new string[]
{
nameof(FileParameter.Data),
nameof(FileParameter.FileName),
nameof(FileParameter.ContentType),
};
foreach (JsonProperty prop in contract.Properties.Where(p => propNames.Contains(p.UnderlyingName)))
{
prop.Writable = true;
}
}
return contract;
}
}
To use the resolver, add it to the JsonSerializerSettings along with the StreamStringConverter:
var obj = JsonConvert.DeserializeObject<BulkLeadRequest>(serializedObject,
new JsonSerializerSettings
{
ContractResolver = new CustomContractResolver(),
Converters = new List<JsonConverter> { new StreamStringConverter() }
});
Working demo: https://dotnetfiddle.net/VHf359
By the way, in the WriteJson method of your StreamStringConverter, I noticed you are casting the value to a FileStream even though the CanConvert method says it can handle any Stream. You can fix it by changing this line:
FileStream valueStream = (FileStream)value;
to this:
Stream valueStream = (Stream)value;

Related

Manipulating a Json string from within a custom Json Converter

In order to ask my question, I will be referring to #Brian Rogers 's answer here.
In this answer ReadJson is doing something relatively simple. How could I however add a layer in there in order to manipulate the incoming Json string before deserialising it into an object and then returning it?
Here is the type of things I would like to do (modified version of Brian's WrappedObjectConvert class):
class WrappedObjectConverter : JsonConverter
{
private string CustomParsing(string jsonString)
{
string modifiedJsonString;
// Some renaming
modifiedJsonString= Regex.Replace(modifiedJsonString, $#"(?<="")CarName(?="":\s)", "Myname", RegexOptions.IgnoreCase);
modifiedJsonString= Regex.Replace(modifiedJsonString, $#"(?<="")CustName(?="":\s)", "Myname", RegexOptions.IgnoreCase);
modifiedJsonString= Regex.Replace(modifiedJsonString, $#"(?<="")MyName(?="":\s)", "Myname", RegexOptions.IgnoreCase);
modifiedJsonString= Regex.Replace(modifiedJsonString, $#"(?<="")SomeAddr(?="":\s)", "AddressLine1 ", RegexOptions.IgnoreCase);
// Renaming IsPublic true/false to IsPrivate false/true
modifiedJsonString= Regex.Replace(modifiedJsonString, "\"IsPublic\": true,", "\"IsPrivate\": false,", RegexOptions.IgnoreCase);
modifiedJsonString = Regex.Replace(modifiedJsonString, "\"IsPublic\": false,", "\"IsPrivate\": true,", RegexOptions.IgnoreCase);
}
public override bool CanConvert(Type objectType)
{
return true;
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JToken token = JToken.Load(reader);
string modifiedJsonString = CustomParsing(token.ToString());
return ????; // How to return the object
// I could do something of the sort, but not sure it's got its place here:
// return JsonConvert.DeserializeObject<RootObject>(modifiedJsonString );
}
public override bool CanWrite
{
get { return false; }
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
The client class has also been slightly modified by adding the field "IsPrivate":
public class Client
{
[JsonConverter(typeof(WrappedObjectConverter))]
public List<Product> ProductList { get; set; }
[JsonConverter(typeof(WrappedObjectConverter))]
public string Name { get; set; }
[JsonConverter(typeof(WrappedObjectConverter))]
public bool IsPrivate { get; set; }
[JsonConverter(typeof(WrappedObjectConverter))]
public string AddressLine1 { get; set; }
}
And the demo with a modified Json (some labels have been changed from Brian's example, which need to be parsed and modified):
class Program
{
static void Main(string[] args)
{
string json = #"
{
""Result"": {
""Client"": {
""ProductList"": {
""Product"": [
{
""MyName"": {
""CarName"": ""Car polish"",
""IsPublic"": ""True""
}
}
]
},
""MyName"": {
""CustName"": ""Mr. Clouseau""
},
""AddressLine1"": {
""SomeAddr"": ""Hightstreet 13""
}
}
}
}";
RootObject obj = JsonConvert.DeserializeObject<RootObject>(json);
Client client = obj.Result.Client;
foreach (Product product in client.ProductList)
{
Console.WriteLine(product.Name);
}
Console.WriteLine(client.Name);
Console.WriteLine(client.AddressLine1);
}
}
As you can see, the way the parsing is being done is a bit hacky, so my questions are:
Can I incorporate this parsing to the classes themselves without making a mess of my classes?
If my approach is the way to go, how do I recreate the object so that ReadJson() can return it (see
question marks in code above)
Taking it to the next level: If the client class had a constructor taking in arguments (passed to a base class), how would you do 2. (as the extra level of nesting would complicate things I believe)
If this is the wrong way to go, I am open to suggestions
From your question and comments it sounds like you have some complex JSON and your goal is to flatten it down into a simpler class structure. But you have some additional constraints:
The target classes for deserialization don't have default (parameterless) constructors.
You want to map the keys in the JSON to differently-named properties in the classes.
For some properties you want to translate the values as well (e.g. convert true to false and vice versa).
You want to keep the mapping logic all in one place, separate from the classes.
You can do all this with a custom JsonConverter. The key is to load the JSON data into a JObject inside the converter. From there you can use SelectToken to specify paths to retrieve specific pieces of data from the JObject. You can then use these pieces to construct your objects via their non-default constructors. At the same time you can translate any values that require it.
For example, let's say you are starting with the JSON in your question, and the classes you really want to deserialize to look like this:
public class Client
{
public Client(string name, string addressLine1, List<Product> productList)
{
Name = name;
AddressLine1 = addressLine1;
ProductList = productList;
}
public List<Product> ProductList { get; set; }
public string Name { get; set; }
public string AddressLine1 { get; set; }
}
public class Product
{
public Product(string name, bool isPrivate)
{
Name = name;
IsPrivate = isPrivate;
}
public string Name { get; set; }
public bool IsPrivate { get; set; }
}
Here is a custom converter that will handle the deserialization:
class CustomConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return objectType == typeof(Client);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JObject obj = JObject.Load(reader);
string name = (string)obj.SelectToken("Result.Client.MyName.CustName");
string addressLine1 = (string)obj.SelectToken("Result.Client.AddressLine1.SomeAddr");
List<Product> productList = obj.SelectToken("Result.Client.ProductList.Product")
.Select(jt =>
{
string prodName = (string)jt.SelectToken("MyName.CarName");
bool isPublic = string.Equals((string)jt.SelectToken("MyName.IsPublic"), "True", StringComparison.OrdinalIgnoreCase);
return new Product(prodName, !isPublic);
})
.ToList();
Client client = new Client(name, addressLine1, productList);
return client;
}
public override bool CanWrite => false;
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
To use the converter, you can either add a [JsonConverter] attribute to the Client class like this:
[JsonConverter(typeof(CustomConverter))]
public class Client
{
...
}
Or you can pass the converter as a parameter to JsonConvert.DeserializeObject() like this:
Client client = JsonConvert.DeserializeObject<Client>(json, new CustomConverter());
Here is a working demo: https://dotnetfiddle.net/EwtQHh

WCF: Data contract serializer with multiple modules

In one of my C# projects I use a WCF data contract serializer for serialization to XML. The framework however consists of multiple extension modules that may be loaded or not, dependent on some startup configuration (I use MEF in case it matters). In the future the list of modules may potentially grow and I fear that this situation may someday pose problems with module-specific data. As I understand I can implement a data contract resolver to bidirectionally help the serializer locate types, but what happens if the project contains data it cannot interpret because the associated module is not loaded?
I am looking for a solution that allows me to preserve existing serialized data in cases where not the full set of modules is loaded (or even available). I think of this as a way to tell the de-serializer "if you don't understand what you get, then don't try to serialize it, but please keep the data somewhere so that you can put it back when serializing the next time". I think my problem is related to round-tripping, but I wasn't very successful (yet) in finding a hint on how to deal with such a case where complex types may be added or removed between serialization actions.
Minimal example:
Suppose I start my application with the optional modules A, B and C and produce the following XML (AData, BData and CData are in a collection and may be all derived from a common base class):
<Project xmlns="http://schemas.datacontract.org/2004/07/TestApplication" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<Data>
<ModuleData i:type="AData">
<A>A</A>
</ModuleData>
<ModuleData i:type="BData">
<B>B</B>
</ModuleData>
<ModuleData i:type="CData">
<C>C</C>
</ModuleData>
</Data>
</Project>
In case I skip module C (containing the definition of CData) and load the same project, then the serializer fails because it has no idea how to deal with CData. If I can somehow manage to convince the framework to keep the data and leave it untouched until someone opens the project again with module C, then I win. Of course I could implement dynamic data structures for storing extension data, e.g., key-value trees, but it would be neat to use the existing serialization framework also in extension modules. Any hint on how to achieve this is highly appreciated!
The example code to produce the above output is as follows:
using System;
using System.IO;
using System.Collections.Generic;
using System.Runtime.Serialization;
namespace TestApplication
{
// common base class
[DataContract]
public class ModuleData : IExtensibleDataObject
{
public virtual ExtensionDataObject ExtensionData { get; set; }
}
[DataContract]
public class AData : ModuleData
{
[DataMember]
public string A { get; set; }
}
[DataContract]
public class BData : ModuleData
{
[DataMember]
public string B { get; set; }
}
[DataContract]
public class CData : ModuleData
{
[DataMember]
public string C { get; set; }
}
[DataContract]
[KnownType(typeof(AData))]
[KnownType(typeof(BData))]
public class Project
{
[DataMember]
public List<ModuleData> Data { get; set; }
}
class Program
{
static void Main(string[] args)
{
// new project object
var project1 = new Project()
{
Data = new List<ModuleData>()
{
new AData() { A = "A" },
new BData() { B = "B" },
new CData() { C = "C" }
}
};
// serialization; make CData explicitly known to simulate presence of "module C"
var stream = new MemoryStream();
var serializer1 = new DataContractSerializer(typeof(Project), new[] { typeof(CData) });
serializer1.WriteObject(stream, project1);
stream.Position = 0;
var reader = new StreamReader(stream);
Console.WriteLine(reader.ReadToEnd());
// deserialization; skip "module C"
stream.Position = 0;
var serializer2 = new DataContractSerializer(typeof(Project));
var project2 = serializer2.ReadObject(stream) as Project;
}
}
}
I also uploaded a VS2015 solution here.
Your problem is that you have a polymorphic known type hierarchy, and you would like to use the round-tripping mechanism of DataContractSerializer to read and save "unknown" known types, specifically XML elements with an xsi:type type hint referring to a type not currently loaded into your app domain.
Unfortunately, this use case simply isn't implemented by the round-tripping mechanism. That mechanism is designed to cache unknown data members inside an ExtensionData object, provided that the data contract object itself can be successfully deserialized and implements IExtensibleDataObject. Unfortunately, in your situation the data contract object cannot be constructed precisely because the polymorphic subtype is unrecognized; instead the following exception gets thrown:
System.Runtime.Serialization.SerializationException occurred
Message="Error in line 4 position 6. Element
'http://www.Question45412824.com:ModuleData' contains data of the
'http://www.Question45412824.com:CData' data contract. The
deserializer has no knowledge of any type that maps to this contract.
Add the type corresponding to 'CData' to the list of known types - for
example, by using the KnownTypeAttribute attribute or by adding it to
the list of known types passed to DataContractSerializer."
Even if I try to create a custom generic collection marked with [CollectionDataContract] that implements IExtensibleDataObject to cache items with unrecognized contracts, the same exception gets thrown.
One solution is to take advantage of the fact that your problem is slightly less difficult than the round-tripping problem. You (the software architect) actually know all possible polymorphic subtypes. Your software does not, because it isn't always loading the assemblies that contain them. Thus what you can do is load lightweight dummy types instead of the real types when the real types aren't needed. As long as the dummy types implement IExtensibleDataObject and have the same data contract namespace and name and the real types, their data contracts will be interchangeable with the "real" data contracts in polymorphic collections.
Thus, if you define your types as follows, adding a Dummies.CData dummy placeholder:
public static class Namespaces
{
// The data contract namespace for your project.
public const string ProjectNamespace = "http://www.Question45412824.com";
}
// common base class
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class ModuleData : IExtensibleDataObject
{
public ExtensionDataObject ExtensionData { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class AData : ModuleData
{
[DataMember]
public string A { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class BData : ModuleData
{
[DataMember]
public string B { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
[KnownType(typeof(AData))]
[KnownType(typeof(BData))]
public class Project
{
[DataMember]
public List<ModuleData> Data { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class CData : ModuleData
{
[DataMember]
public string C { get; set; }
}
namespace Dummies
{
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class CData : ModuleData
{
}
}
You will be able to deserialize your Project object using either the "real" CData or the "dummy" version, as shown with the test below:
class Program
{
static void Main(string[] args)
{
new TestClass().Test();
}
}
class TestClass
{
public virtual void Test()
{
// new project object
var project1 = new Project()
{
Data = new List<ModuleData>()
{
new AData() { A = "A" },
new BData() { B = "B" },
new CData() { C = "C" }
}
};
// serialization; make CData explicitly known to simulate presence of "module C"
var extraTypes = new[] { typeof(CData) };
var extraTypesDummy = new[] { typeof(Dummies.CData) };
var xml = project1.SerializeXml(extraTypes);
ConsoleAndDebug.WriteLine(xml);
// Demonstrate that the XML can be deserialized with the dummy CData type.
TestDeserialize(project1, xml, extraTypesDummy);
// Demonstrate that the XML can be deserialized with the real CData type.
TestDeserialize(project1, xml, extraTypes);
try
{
// Demonstrate that the XML cannot be deserialized without either the dummy or real type.
TestDeserialize(project1, xml, new Type[0]);
Assert.IsTrue(false);
}
catch (AssertionFailedException ex)
{
Console.WriteLine("Caught unexpected exception: ");
Console.WriteLine(ex);
throw;
}
catch (Exception ex)
{
ConsoleAndDebug.WriteLine(string.Format("Caught expected exception: {0}", ex.Message));
}
}
public void TestDeserialize<TProject>(TProject project, string xml, Type[] extraTypes)
{
TestDeserialize<TProject>(xml, extraTypes);
}
public void TestDeserialize<TProject>(string xml, Type[] extraTypes)
{
var project2 = xml.DeserializeXml<TProject>(extraTypes);
var xml2 = project2.SerializeXml(extraTypes);
ConsoleAndDebug.WriteLine(xml2);
// Assert that the incoming and re-serialized XML are equivalent (no data was lost).
Assert.IsTrue(XNode.DeepEquals(XElement.Parse(xml), XElement.Parse(xml2)));
}
}
public static partial class DataContractSerializerHelper
{
public static string SerializeXml<T>(this T obj, Type [] extraTypes)
{
return obj.SerializeXml(new DataContractSerializer(obj == null ? typeof(T) : obj.GetType(), extraTypes));
}
public static string SerializeXml<T>(this T obj, DataContractSerializer serializer)
{
serializer = serializer ?? new DataContractSerializer(obj == null ? typeof(T) : obj.GetType());
using (var textWriter = new StringWriter())
{
var settings = new XmlWriterSettings { Indent = true };
using (var xmlWriter = XmlWriter.Create(textWriter, settings))
{
serializer.WriteObject(xmlWriter, obj);
}
return textWriter.ToString();
}
}
public static T DeserializeXml<T>(this string xml, Type[] extraTypes)
{
return xml.DeserializeXml<T>(new DataContractSerializer(typeof(T), extraTypes));
}
public static T DeserializeXml<T>(this string xml, DataContractSerializer serializer)
{
using (var textReader = new StringReader(xml ?? ""))
using (var xmlReader = XmlReader.Create(textReader))
{
return (T)(serializer ?? new DataContractSerializer(typeof(T))).ReadObject(xmlReader);
}
}
}
public static class ConsoleAndDebug
{
public static void WriteLine(object s)
{
Console.WriteLine(s);
Debug.WriteLine(s);
}
}
public class AssertionFailedException : System.Exception
{
public AssertionFailedException() : base() { }
public AssertionFailedException(string s) : base(s) { }
}
public static class Assert
{
public static void IsTrue(bool value)
{
if (value == false)
throw new AssertionFailedException("failed");
}
}
Another solution would be to replace your List<ModuleData> with a custom collection that implements IXmlSerializable and handles the polymorphic serialization entirely manually, caching the XML for unknown polymorphic subtypes in a list of unknown elements. I wouldn't recommend that however since even straightforward implementations of IXmlSerializable can be quite complex, as shown here and, e.g., here.
Following dbc's wonderful suggestion of using dummies to exploit the roundtripping mechanism to do the job, I made the solution more generic by generating the dummy types on the fly as needed.
The core of this solution is the following simple function that internally invokes the C# compiler:
private Type CreateDummyType(string typeName, string typeNamespace)
{
var className = $"DummyClass_{random_.Next()}";
var code = $"[System.Runtime.Serialization.DataContract(Name=\"{typeName}\", Namespace=\"{typeNamespace}\")] public class {className} : ModuleData {{}}";
using (var provider = new CSharpCodeProvider())
{
var parameters = new CompilerParameters();
parameters.ReferencedAssemblies.Add("System.Runtime.Serialization.dll");
parameters.ReferencedAssemblies.Add(GetType().Assembly.Location); // this assembly (for ModuleData)
var results = provider.CompileAssemblyFromSource(parameters, code);
return results.CompiledAssembly.GetType(className);
}
}
I combined this with a DataContractResolver that takes care of any unknown types and generates dummies as needed to preserve their data during subsequent (de)serializations.
For completeness I put the recent iteration of the sample code here:
using System;
using System.IO;
using System.Collections.Generic;
using System.Runtime.Serialization;
using System.Diagnostics;
using System.Xml;
using System.Xml.Linq;
using Microsoft.CSharp;
using System.CodeDom.Compiler;
public static class Namespaces
{
public const string BaseNamespace = "http://www.Question45412824.com";
public const string ProjectNamespace = BaseNamespace + "/Project";
public const string ExtensionNamespace = BaseNamespace + "/Extension";
}
// common base class
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class ModuleData : IExtensibleDataObject
{
public ExtensionDataObject ExtensionData { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class AData : ModuleData
{
[DataMember]
public string A { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
public class BData : ModuleData
{
[DataMember]
public string B { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
[KnownType(typeof(AData))]
[KnownType(typeof(BData))]
public class Project
{
[DataMember]
public List<ModuleData> Data { get; set; }
}
[DataContract(Namespace = Namespaces.ProjectNamespace)]
internal class CSubData : ModuleData
{
[DataMember]
public string Name { get; set; }
}
[DataContract(Namespace = Namespaces.ExtensionNamespace)]
public class CData : ModuleData
{
[DataMember]
public ModuleData C { get; set; }
}
class Program
{
static void Main(string[] args)
{
new TestClass().Test();
}
}
class TestClass
{
public virtual void Test()
{
// new project object
var project1 = new Project()
{
Data = new List<ModuleData>()
{
new AData() { A = "A" },
new BData() { B = "B" },
new CData() { C = new CSubData() { Name = "C" } }
}
};
// serialization; make CData explicitly known to simulate presence of "module C"
var extraTypes = new[] { typeof(CData), typeof(CSubData) };
ConsoleAndDebug.WriteLine("\n== Serialization with all types known ==");
var xml = project1.SerializeXml(extraTypes);
ConsoleAndDebug.WriteLine(xml);
ConsoleAndDebug.WriteLine("\n== Deserialization and subsequent serialization WITH generic resolver and unknown types ==");
TestDeserialize(project1, xml, new GenericDataContractResolver());
ConsoleAndDebug.WriteLine("\n== Deserialization and subsequent serialization WITHOUT generic resolver and unknown types ==");
try
{
// Demonstrate that the XML cannot be deserialized without the generic resolver.
TestDeserialize(project1, xml, new Type[0]);
Assert.IsTrue(false);
}
catch (AssertionFailedException ex)
{
Console.WriteLine("Caught unexpected exception: ");
Console.WriteLine(ex);
throw;
}
catch (Exception ex)
{
ConsoleAndDebug.WriteLine(string.Format("Caught expected exception: {0}", ex.Message));
}
}
public void TestDeserialize<TProject>(TProject project, string xml, Type[] extraTypes)
{
TestDeserialize<TProject>(xml, extraTypes);
}
public void TestDeserialize<TProject>(string xml, Type[] extraTypes)
{
var project2 = xml.DeserializeXml<TProject>(extraTypes);
var xml2 = project2.SerializeXml(extraTypes);
ConsoleAndDebug.WriteLine(xml2);
// Assert that the incoming and re-serialized XML are equivalent (no data was lost).
Assert.IsTrue(XNode.DeepEquals(XElement.Parse(xml), XElement.Parse(xml2)));
}
public void TestDeserialize<TProject>(TProject project, string xml, DataContractResolver resolver)
{
TestDeserialize<TProject>(xml, resolver);
}
public void TestDeserialize<TProject>(string xml, DataContractResolver resolver)
{
var project2 = xml.DeserializeXml<TProject>(resolver);
var xml2 = project2.SerializeXml(resolver);
ConsoleAndDebug.WriteLine(xml2);
// Assert that the incoming and re-serialized XML are equivalent (no data was lost).
Assert.IsTrue(XNode.DeepEquals(XElement.Parse(xml), XElement.Parse(xml2)));
}
}
public static partial class DataContractSerializerHelper
{
public static string SerializeXml<T>(this T obj, Type[] extraTypes)
{
return obj.SerializeXml(new DataContractSerializer(obj == null ? typeof(T) : obj.GetType(), extraTypes));
}
public static string SerializeXml<T>(this T obj, DataContractResolver resolver)
{
return obj.SerializeXml(new DataContractSerializer(obj == null ? typeof(T) : obj.GetType(), null, int.MaxValue, false, false, null, resolver));
}
public static string SerializeXml<T>(this T obj, DataContractSerializer serializer)
{
serializer = serializer ?? new DataContractSerializer(obj == null ? typeof(T) : obj.GetType());
using (var textWriter = new StringWriter())
{
var settings = new XmlWriterSettings { Indent = true };
using (var xmlWriter = XmlWriter.Create(textWriter, settings))
{
serializer.WriteObject(xmlWriter, obj);
}
return textWriter.ToString();
}
}
public static T DeserializeXml<T>(this string xml, DataContractResolver resolver)
{
return xml.DeserializeXml<T>(new DataContractSerializer(typeof(T), null, int.MaxValue, false, false, null, resolver));
}
public static T DeserializeXml<T>(this string xml, Type[] extraTypes)
{
return xml.DeserializeXml<T>(new DataContractSerializer(typeof(T), extraTypes));
}
public static T DeserializeXml<T>(this string xml, DataContractSerializer serializer)
{
using (var textReader = new StringReader(xml ?? ""))
using (var xmlReader = XmlReader.Create(textReader))
{
return (T)(serializer ?? new DataContractSerializer(typeof(T))).ReadObject(xmlReader);
}
}
}
public static class ConsoleAndDebug
{
public static void WriteLine(object s)
{
Console.WriteLine(s);
Debug.WriteLine(s);
}
}
public class AssertionFailedException : System.Exception
{
public AssertionFailedException() : base() { }
public AssertionFailedException(string s) : base(s) { }
}
public static class Assert
{
public static void IsTrue(bool value)
{
if (value == false)
throw new AssertionFailedException("failed");
}
}
class GenericDataContractResolver : DataContractResolver
{
private static readonly Random random_ = new Random();
private static readonly Dictionary<Tuple<string, string>, Type> toType_ = new Dictionary<Tuple<string, string>, Type>();
private static readonly Dictionary<Type, Tuple<string, string>> fromType_ = new Dictionary<Type, Tuple<string, string>>();
private Type CreateDummyType(string typeName, string typeNamespace)
{
var className = $"DummyClass_{random_.Next()}";
var code = $"[System.Runtime.Serialization.DataContract(Name=\"{typeName}\", Namespace=\"{typeNamespace}\")] public class {className} : ModuleData {{}}";
using (var provider = new CSharpCodeProvider())
{
var parameters = new CompilerParameters();
parameters.ReferencedAssemblies.Add("System.Runtime.Serialization.dll");
parameters.ReferencedAssemblies.Add(GetType().Assembly.Location); // this assembly (for ModuleData)
var results = provider.CompileAssemblyFromSource(parameters, code);
return results.CompiledAssembly.GetType(className);
}
}
// Used at deserialization; allows users to map xsi:type name to any Type
public override Type ResolveName(string typeName, string typeNamespace, Type declaredType, DataContractResolver knownTypeResolver)
{
var type = knownTypeResolver.ResolveName(typeName, typeNamespace, declaredType, null);
// resolve all unknown extension datasets; all other should be explicitly known.
if (type == null && declaredType == typeof(ModuleData) && typeNamespace == Namespaces.ExtensionNamespace)
{
// if we already have this type cached, then return the cached one
var typeNameAndNamespace = new Tuple<string, string>(typeName, typeNamespace);
if (toType_.TryGetValue(typeNameAndNamespace, out type))
return type;
// else compile the dummy type and remember it in the cache
type = CreateDummyType(typeName, typeNamespace);
toType_.Add(typeNameAndNamespace, type);
fromType_.Add(type, typeNameAndNamespace);
}
return type;
}
// Used at serialization; maps any Type to a new xsi:type representation
public override bool TryResolveType(Type type, Type declaredType, DataContractResolver knownTypeResolver, out XmlDictionaryString typeName, out XmlDictionaryString typeNamespace)
{
if (knownTypeResolver.TryResolveType(type, declaredType, null, out typeName, out typeNamespace))
return true; // known type
// is the type one of our cached dummies?
var typeNameAndNamespace = default(Tuple<string, string>);
if (declaredType == typeof(ModuleData) && fromType_.TryGetValue(type, out typeNameAndNamespace))
{
typeName = new XmlDictionaryString(XmlDictionary.Empty, typeNameAndNamespace.Item1, 0);
typeNamespace = new XmlDictionaryString(XmlDictionary.Empty, typeNameAndNamespace.Item2, 0);
return true; // dummy type
}
return false; // unknown type
}
}

Dotnet core -> Newtonsoft DeserializeObject<T> with Dynamic Child [duplicate]

Given following json result:
The default json result has a known set of fields:
{
"id": "7908",
"name": "product name"
}
But can be extended with additional fields (in this example _unknown_field_name_1 and _unknown_field_name_2) of which the names are not known when requesting the result.
{
"id": "7908",
"name": "product name",
"_unknown_field_name_1": "some value",
"_unknown_field_name_2": "some value"
}
I would like the json result to be serialized and deserialized to and from a class with properties for the known fields and map the unknown fields (for which there are no properties) to a property (or multiple properties) like a dictionary so they can be accessed and modified.
public class Product
{
public string id { get; set; }
public string name { get; set; }
public Dictionary<string, string> fields { get; set; }
}
I think I need a way to plug into a json serializer and do the mapping for the missing members myself (both for serialize and deserialize).
I have been looking at various possibilities:
json.net and custom contract resolvers (can't figure out how to do it)
datacontract serializer (can only override onserialized, onserializing)
serialize to dynamic and do custom mapping (this might work, but seems a lot of work)
let product inheriting from DynamicObject (serializers work with reflection and do not invoke the trygetmember and trysetmember methods)
I'm using restsharp, but any serializer can be plugged in.
Oh, and I cannot change the json result, and this or this didn't help me either.
Update:
This looks more like it: http://geekswithblogs.net/DavidHoerster/archive/2011/07/26/json.net-custom-convertersndasha-quick-tour.aspx
An even easier option to tackling this problem would be to use the JsonExtensionDataAttribute from JSON .NET
public class MyClass
{
// known field
public decimal TaxRate { get; set; }
// extra fields
[JsonExtensionData]
private IDictionary<string, JToken> _extraStuff;
}
There's a sample of this on the project blog here
UPDATE Please note this requires JSON .NET v5 release 5 and above
See https://gist.github.com/LodewijkSioen/5101814
What you were looking for was a custom JsonConverter
This is a way you could solve it, although I don't like it that much. I solved it using Newton/JSON.Net. I suppose you could use the JsonConverter for deserialization aswell.
private const string Json = "{\"id\":7908,\"name\":\"product name\",\"_unknown_field_name_1\":\"some value\",\"_unknown_field_name_2\":\"some value\"}";
[TestMethod]
public void TestDeserializeUnknownMembers()
{
var #object = JObject.Parse(Json);
var serializer = new Newtonsoft.Json.JsonSerializer();
serializer.MissingMemberHandling = Newtonsoft.Json.MissingMemberHandling.Error;
serializer.Error += (sender, eventArgs) =>
{
var contract = eventArgs.CurrentObject as Contract ?? new Contract();
contract.UnknownValues.Add(eventArgs.ErrorContext.Member.ToString(), #object[eventArgs.ErrorContext.Member.ToString()].Value<string>());
eventArgs.ErrorContext.Handled = true;
};
using (MemoryStream memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(Json)))
using (StreamReader streamReader = new StreamReader(memoryStream))
using (JsonReader jsonReader = new JsonTextReader(streamReader))
{
var result = serializer.Deserialize<Contract>(jsonReader);
Assert.IsTrue(result.UnknownValues.ContainsKey("_unknown_field_name_1"));
Assert.IsTrue(result.UnknownValues.ContainsKey("_unknown_field_name_2"));
}
}
[TestMethod]
public void TestSerializeUnknownMembers()
{
var deserializedObject = new Contract
{
id = 7908,
name = "product name",
UnknownValues = new Dictionary<string, string>
{
{"_unknown_field_name_1", "some value"},
{"_unknown_field_name_2", "some value"}
}
};
var json = JsonConvert.SerializeObject(deserializedObject, new DictionaryConverter());
Console.WriteLine(Json);
Console.WriteLine(json);
Assert.AreEqual(Json, json);
}
}
class DictionaryConverter : JsonConverter
{
public DictionaryConverter()
{
}
public override bool CanConvert(Type objectType)
{
return objectType == typeof(Contract);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
throw new NotImplementedException();
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
var contract = value as Contract;
var json = JsonConvert.SerializeObject(value);
var dictArray = String.Join(",", contract.UnknownValues.Select(pair => "\"" + pair.Key + "\":\"" + pair.Value + "\""));
json = json.Substring(0, json.Length - 1) + "," + dictArray + "}";
writer.WriteRaw(json);
}
}
class Contract
{
public Contract()
{
UnknownValues = new Dictionary<string, string>();
}
public int id { get; set; }
public string name { get; set; }
[JsonIgnore]
public Dictionary<string, string> UnknownValues { get; set; }
}
}
I thought I'd throw my hat in the ring since I had a similar problem recently. Here's an example of the JSON I wanted to deserialize:
{
"agencyId": "agency1",
"overrides": {
"assumption.discount.rates": "value: 0.07",
".plan": {
"plan1": {
"assumption.payroll.growth": "value: 0.03",
"provision.eeContrib.rate": "value: 0.35"
},
"plan2": {
".classAndTier": {
"misc:tier1": {
"provision.eeContrib.rate": "value: 0.4"
},
"misc:tier2": {
"provision.eeContrib.rate": "value: 0.375"
}
}
}
}
}
}
This is for a system where overrides apply at different levels and are inherited down the tree. In any case, the data model I wanted was something that would allow me to have a property bag with these special inheritance rules also supplied.
What I ended up with was the following:
public class TestDataModel
{
public string AgencyId;
public int Years;
public PropertyBagModel Overrides;
}
public class ParticipantFilterModel
{
public string[] ClassAndTier;
public string[] BargainingUnit;
public string[] Department;
}
public class PropertyBagModel
{
[JsonExtensionData]
private readonly Dictionary<string, JToken> _extensionData = new Dictionary<string, JToken>();
[JsonIgnore]
public readonly Dictionary<string, string> Values = new Dictionary<string, string>();
[JsonProperty(".plan", NullValueHandling = NullValueHandling.Ignore)]
public Dictionary<string, PropertyBagModel> ByPlan;
[JsonProperty(".classAndTier", NullValueHandling = NullValueHandling.Ignore)]
public Dictionary<string, PropertyBagModel> ByClassAndTier;
[JsonProperty(".bargainingUnit", NullValueHandling = NullValueHandling.Ignore)]
public Dictionary<string, PropertyBagModel> ByBarginingUnit;
[OnSerializing]
private void OnSerializing(StreamingContext context)
{
foreach (var kvp in Values)
_extensionData.Add(kvp.Key, kvp.Value);
}
[OnSerialized]
private void OnSerialized(StreamingContext context)
{
_extensionData.Clear();
}
[OnDeserialized]
private void OnDeserialized(StreamingContext context)
{
Values.Clear();
foreach (var kvp in _extensionData.Where(x => x.Value.Type == JTokenType.String))
Values.Add(kvp.Key, kvp.Value.Value<string>());
_extensionData.Clear();
}
}
The basic idea is this:
The PropertyBagModel on deserialization by JSON.NET has the ByPlan, ByClassAndTier, etc. fields populated and also has the private _extensionData field populated.
Then JSON.NET calls the private OnDeserialized() method and that will move the data from _extensionData to Values as appropriate (or drop it on the floor otherwise - presumably you could log this if it was something you wanted to know). We then remove the extra gunk from _extensionData so it doesn't consume memory.
On serialization, the OnSerializing method gets calls where we move stuff into _extensionData so it gets saved.
When serialization has finished, OnSerialized gets called and we remove the extra stuff from _extensionData.
We could further delete and recreate the _extensionData Dictionary when needed but I didn't see a real value in this as I'm not using tons of these objects. To do this we'd just create on OnSerializing and delete on OnSerialized. On OnDeserializing, instead of clearing, we could free it.
I was looking into a similar issue and found this post.
Here is a way to do it using reflection.
To make it more generic, one should check the type of the property instead of simply using ToString() in propertyInfo.SetValue, unless OFC all the actual properties are strings.
Also, lowercase property names is not standard in C# but given that GetProperty is case sensitive there are few other options.
public class Product
{
private Type _type;
public Product()
{
fields = new Dictionary<string, object>();
_type = GetType();
}
public string id { get; set; }
public string name { get; set; }
public Dictionary<string, object> fields { get; set; }
public void SetProperty(string key, object value)
{
var propertyInfo = _type.GetProperty(key);
if (null == propertyInfo)
{
fields.Add(key,value);
return;
}
propertyInfo.SetValue(this, value.ToString());
}
}
...
private const string JsonTest = "{\"id\":7908,\"name\":\"product name\",\"_unknown_field_name_1\":\"some value\",\"_unknown_field_name_2\":\"some value\"}";
var product = new Product();
var data = JObject.Parse(JsonTest);
foreach (var item in data)
{
product.SetProperty(item.Key, item.Value);
}

JSON .NET Custom Name Resolver for Sub-Properties

I have an API that returns a JSON object from a MongoDB in which one of the properties is an "open-ended" document, meaning it can be any valid JSON for that property. I don't know what the names of the properties are ahead of time, they can be any string. I only know that this particular property needs to be serialized exactly how it is stored in the database. So if the property name that was originally stored was "Someproperty", the serialized response in JSON needs to be "Someproperty", NOT "someProperty".
We have this configuration:
ContractResolver = new CamelCasePropertyNamesContractResolver();
in our CustomJsonSerializer, but it is messing with the formatting of the response when returning the "open ended" JSON. It is camel-casing all of these properties when in fact we want the response to be exactly how they are stored in MongoDB (BSON). I know the values are maintaining their case when storing/retrieving via the database, so that is not the issue.
How can I tell JSON.net to essentially bypass the CamelCasePropertyNameResolver for all of the child properties of a particular data point?
EDIT:
Just to give a bit more info, and share what I have already tried:
I thought about overriding the PropertyNameResolver like so:
protected override string ResolvePropertyName(string propertyName)
{
if (propertyName.ToLower().Equals("somedocument"))
{
return propertyName;
}
else return base.ResolvePropertyName(propertyName);
}
However, if I have a JSON structure like this:
{
"Name" : "MyObject",
"DateCreated" : "11/14/2016",
"SomeDocument" :
{
"MyFirstProperty" : "foo",
"mysecondPROPERTY" : "bar",
"another_random_subdoc" :
{
"evenmoredata" : "morestuff"
}
}
}
then I would need all of the properties any child properties' names to remain exactly as is. The above override I posted would (I believe) only ignore on an exact match to "somedocument", and would still camelcase all of the child properties.
What you can do is, for the property in question, create a custom JsonConverter that serializes the property value in question using a different JsonSerializer created with a different contract resolver, like so:
public class AlternateContractResolverConverter : JsonConverter
{
[ThreadStatic]
static Stack<Type> contractResolverTypeStack;
static Stack<Type> ContractResolverTypeStack { get { return contractResolverTypeStack = (contractResolverTypeStack ?? new Stack<Type>()); } }
readonly IContractResolver resolver;
JsonSerializerSettings ExtractAndOverrideSettings(JsonSerializer serializer)
{
var settings = serializer.ExtractSettings();
settings.ContractResolver = resolver;
settings.CheckAdditionalContent = false;
if (settings.PreserveReferencesHandling != PreserveReferencesHandling.None)
{
// Log an error throw an exception?
Debug.WriteLine(string.Format("PreserveReferencesHandling.{0} not supported", serializer.PreserveReferencesHandling));
}
return settings;
}
public AlternateContractResolverConverter(Type resolverType)
{
if (resolverType == null)
throw new ArgumentNullException("resolverType");
resolver = (IContractResolver)Activator.CreateInstance(resolverType);
if (resolver == null)
throw new ArgumentNullException(string.Format("Resolver type {0} not found", resolverType));
}
public override bool CanRead { get { return ContractResolverTypeStack.Count == 0 || ContractResolverTypeStack.Peek() != resolver.GetType(); } }
public override bool CanWrite { get { return ContractResolverTypeStack.Count == 0 || ContractResolverTypeStack.Peek() != resolver.GetType(); } }
public override bool CanConvert(Type objectType)
{
throw new NotImplementedException("This contract resolver is intended to be applied directly with [JsonConverter(typeof(AlternateContractResolverConverter), typeof(SomeContractResolver))] or [JsonProperty(ItemConverterType = typeof(AlternateContractResolverConverter), ItemConverterParameters = ...)]");
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
using (ContractResolverTypeStack.PushUsing(resolver.GetType()))
return JsonSerializer.CreateDefault(ExtractAndOverrideSettings(serializer)).Deserialize(reader, objectType);
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
using (ContractResolverTypeStack.PushUsing(resolver.GetType()))
JsonSerializer.CreateDefault(ExtractAndOverrideSettings(serializer)).Serialize(writer, value);
}
}
internal static class JsonSerializerExtensions
{
public static JsonSerializerSettings ExtractSettings(this JsonSerializer serializer)
{
// There is no built-in API to extract the settings from a JsonSerializer back into JsonSerializerSettings,
// so we have to fake it here.
if (serializer == null)
throw new ArgumentNullException("serializer");
var settings = new JsonSerializerSettings
{
CheckAdditionalContent = serializer.CheckAdditionalContent,
ConstructorHandling = serializer.ConstructorHandling,
ContractResolver = serializer.ContractResolver,
Converters = serializer.Converters,
Context = serializer.Context,
Culture = serializer.Culture,
DateFormatHandling = serializer.DateFormatHandling,
DateFormatString = serializer.DateFormatString,
DateParseHandling = serializer.DateParseHandling,
DateTimeZoneHandling = serializer.DateTimeZoneHandling,
DefaultValueHandling = serializer.DefaultValueHandling,
EqualityComparer = serializer.EqualityComparer,
// No Get access to the error event, so it cannot be copied.
// Error = += serializer.Error
FloatFormatHandling = serializer.FloatFormatHandling,
FloatParseHandling = serializer.FloatParseHandling,
Formatting = serializer.Formatting,
MaxDepth = serializer.MaxDepth,
MetadataPropertyHandling = serializer.MetadataPropertyHandling,
MissingMemberHandling = serializer.MissingMemberHandling,
NullValueHandling = serializer.NullValueHandling,
ObjectCreationHandling = serializer.ObjectCreationHandling,
ReferenceLoopHandling = serializer.ReferenceLoopHandling,
// Copying the reference resolver doesn't work in the default case, since the
// actual BidirectionalDictionary<string, object> mappings are held in the
// JsonSerializerInternalBase.
// See https://github.com/JamesNK/Newtonsoft.Json/blob/master/Src/Newtonsoft.Json/Serialization/DefaultReferenceResolver.cs
ReferenceResolverProvider = () => serializer.ReferenceResolver,
PreserveReferencesHandling = serializer.PreserveReferencesHandling,
StringEscapeHandling = serializer.StringEscapeHandling,
TraceWriter = serializer.TraceWriter,
TypeNameHandling = serializer.TypeNameHandling,
// Changes in Json.NET 10.0.1
//TypeNameAssemblyFormat was obsoleted and replaced with TypeNameAssemblyFormatHandling in Json.NET 10.0.1
//TypeNameAssemblyFormat = serializer.TypeNameAssemblyFormat,
TypeNameAssemblyFormatHandling = serializer.TypeNameAssemblyFormatHandling,
//Binder was obsoleted and replaced with SerializationBinder in Json.NET 10.0.1
//Binder = serializer.Binder,
SerializationBinder = serializer.SerializationBinder,
};
return settings;
}
}
public static class StackExtensions
{
public struct PushValue<T> : IDisposable
{
readonly Stack<T> stack;
public PushValue(T value, Stack<T> stack)
{
this.stack = stack;
stack.Push(value);
}
// By using a disposable struct we avoid the overhead of allocating and freeing an instance of a finalizable class.
public void Dispose()
{
if (stack != null)
stack.Pop();
}
}
public static PushValue<T> PushUsing<T>(this Stack<T> stack, T value)
{
if (stack == null)
throw new ArgumentNullException();
return new PushValue<T>(value, stack);
}
}
Then use it like so:
public class RootObject
{
public string Name { get; set; }
public DateTime DateCreated { get; set; }
[JsonProperty(NamingStrategyType = typeof(DefaultNamingStrategy))]
[JsonConverter(typeof(AlternateContractResolverConverter), typeof(DefaultContractResolver))]
public SomeDocument SomeDocument { get; set; }
}
public class SomeDocument
{
public string MyFirstProperty { get; set; }
public string mysecondPROPERTY { get; set; }
public AnotherRandomSubdoc another_random_subdoc { get; set; }
}
public class AnotherRandomSubdoc
{
public string evenmoredata { get; set; }
public DateTime DateCreated { get; set; }
}
(Here I am assuming you want the "SomeDocument" property name to be serialized verbatim, even though it wasn't entirely clear from your question. To do that, I'm using JsonPropertyAttribute.NamingStrategyType from Json.NET 9.0.1. If you're using an earlier version, you'll need to set the property name explicitly.)
Then the resulting JSON will be:
{
"name": "Question 40597532",
"dateCreated": "2016-11-14T05:00:00Z",
"SomeDocument": {
"MyFirstProperty": "my first property",
"mysecondPROPERTY": "my second property",
"another_random_subdoc": {
"evenmoredata": "even more data",
"DateCreated": "2016-11-14T05:00:00Z"
}
}
}
Note that this solution does NOT work well with preserving object references. If you need them to work together, you may need to consider a stack-based approach similar to the one from Json.NET serialize by depth and attribute
Demo fiddle here.
Incidentally, have you considered storing this JSON as a raw string literal, as in the answer to this question?
I think you should look at this backwards.
Instead of trying to NOT touch the properties you don't know, let that be the default behavior and touch the ones you DO know.
In other words, don't use the CamelCasePropertyNamesContractResolver. Deal with the properties you know appropriately and let the other ones pass through transparently.

Custom JSON Derivative Format

I would like to have a serialization format that is nearly identical to JSON, except that key-values are represented as <key>="<value>" instead of "<key>":"<value>".
With Newtonsoft I made a custom JsonConverter called TsonConverter that works fairly well, except that it can't "see" an embedded dictionary. Given the following type:
public class TraceyData
{
[Safe]
public string Application { get; set; }
[Safe]
public string SessionID { get; set; }
[Safe]
public string TraceID { get; set; }
[Safe]
public string Workflow { get; set; }
[Safe]
public Dictionary<string, string> Tags {get; set; }
[Safe]
public string[] Stuff {get; set;}
}
And the following code:
TsonConverter weird = new TsonConverter();
JsonSerializerSettings settings = new JsonSerializerSettings();
settings.NullValueHandling = NullValueHandling.Ignore;
settings.Converters.Add(weird);
var tracey = new TraceyData();
tracey.TraceID = Guid.NewGuid().ToString();
tracey.SessionID = "5";
tracey.Tags["Referrer"] = "http://www.sky.net/deals";
tracey.Stuff = new string[] { "Alpha", "Bravo", "Charlie" };
tracey.Application = "Responsive";
string stuff = JsonConvert.SerializeObject(tracey, settings);
I get this:
[Application="Responsive" SessionID="5" TraceID="082ef853-92f8-4ce8-9f32-8e4f792fb022" Tags={"Referrer":"http://www.sky.net/deals"} Stuff=["Alpha","Bravo","Charlie"]]
Obviously I have also overridden the StartObject/EndObject notation, replacing { } with [ ]. Otherwise the results are not bad.
However, there is still the problem of the internal dictionary. In order
to convert the dictionary as well to use my <key>="<value>" format, it looks like I must make a deep dictionary converter.
I'm wondering if there is an easier way to do this.
Perhaps the Newtonsoft tool has a "property generator" and "key-value" generator property that I can set that globally handles this for me?
Any suggestions?
And while we're here, I wonder if there is a StartObject/EndObject formatter property override I can set, which would handle the other customization I've shown above. It would be nice to "skip" making JsonConverter tools for these kinds of simple alterations.
Incidentally:
My custom JsonConverter is choosing properties to serialize based on the [Safe] attribute shown in my sample. This is another nice-to-have. It would be wonderful if the JSon settings could expose an "attribute handler" property that lets me override the usual JSon attributes in favor of my own.
I have no need to de-serialize this format. It is intended as a one-way operation. If someone wishes also to explain how to de-serialize my custom format as well that is an interesting bonus, but definitely not necessary to answer this question.
Appendix
Below is the TraceConverter I had made. It references a FieldMetaData class that simply holds property info.
public class TsonConverter : JsonConverter
{
public override bool CanRead
{
get
{
return false;
}
}
public override bool CanConvert(Type ObjectType)
{
return DataClassifier.TestForUserType(ObjectType);
}
public override void WriteJson(
JsonWriter writer, object value, JsonSerializer serializer)
{
Type objType = value.GetType();
var props = objType.GetProperties(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic);
var propMap = from p in props
from a in p.GetCustomAttributes(typeof(ProfileAttribute), false)
select new FieldMetaData(p, (ProfileAttribute)a);
//writer.WriteStartObject();
writer.WriteStartArray();
bool loopStarted = true;
foreach(var prop in propMap){
object rawValue = prop.GetValue(value);
if (rawValue != null || serializer.NullValueHandling == NullValueHandling.Include)
{
string jsonValue = JsonConvert.SerializeObject(prop.GetValue(value), this);
if (loopStarted)
{
loopStarted = false;
writer.WriteRaw(String.Format("{0}={1}", prop.Name, jsonValue));
}
else
{
writer.WriteRaw(String.Format(" {0}={1}", prop.Name, jsonValue));
}
}
//writer.WriteRaw(String.Format("{0}={1}", prop.Name, prop.GetValue(value)));
//writer.WritePropertyName(prop.Name, false);
//writer.WriteValue(prop.GetValue(value));
}
writer.WriteEndArray();
}
public override object ReadJson(
JsonReader reader, Type objectType,
object existingValue, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
Rather than creating your own converter, you're going to need to create your own subclass of JsonWriter that writes to your custom file format. (This is how Json.NET implements its BsonWriter.) In your case, your file format is close enough to JSON that you can inherit from JsonTextWriter:
public class TsonTextWriter : JsonTextWriter
{
TextWriter _writer;
public TsonTextWriter(TextWriter textWriter)
: base(textWriter)
{
if (textWriter == null)
throw new ArgumentNullException("textWriter");
QuoteName = false;
_writer = textWriter;
}
public override void WriteStartObject()
{
SetWriteState(JsonToken.StartObject, null);
_writer.Write('[');
}
protected override void WriteEnd(JsonToken token)
{
switch (token)
{
case JsonToken.EndObject:
_writer.Write(']');
break;
default:
base.WriteEnd(token);
break;
}
}
public override void WritePropertyName(string name)
{
WritePropertyName(name, true);
}
public override void WritePropertyName(string name, bool escape)
{
SetWriteState(JsonToken.PropertyName, name);
var escaped = name;
if (escape)
{
escaped = JsonConvert.ToString(name, '"', StringEscapeHandling);
escaped = escaped.Substring(1, escaped.Length - 2);
}
// Maybe also escape the space character if it appears in a name?
_writer.Write(escaped.Replace("=", #"\u003d"));// Replace "=" with unicode escape sequence.
_writer.Write('=');
}
/// <summary>
/// Writes the JSON value delimiter. (Remove this override if you want to retain the comma separator.)
/// </summary>
protected override void WriteValueDelimiter()
{
_writer.Write(' ');
}
/// <summary>
/// Writes an indent space.
/// </summary>
protected override void WriteIndentSpace()
{
// Do nothing.
}
}
Having done this, now all classes will be serialized to your custom format when you use this writer, for instance:
var tracey = new TraceyData();
tracey.TraceID = Guid.NewGuid().ToString();
tracey.SessionID = "5";
tracey.Tags["Referrer"] = "http://www.sky.net/deals";
tracey.Stuff = new string[] { "Alpha", "Bravo", "Charlie" };
tracey.Application = "Responsive";
JsonSerializerSettings settings = new JsonSerializerSettings();
settings.NullValueHandling = NullValueHandling.Ignore;
using (var sw = new StringWriter())
{
using (var jsonWriter = new TsonTextWriter(sw))
{
JsonSerializer.CreateDefault(settings).Serialize(jsonWriter, tracey);
}
Debug.WriteLine(sw.ToString());
}
Produces the output
[Application="Responsive" SessionID="5" TraceID="2437fe67-9788-47ba-91ce-2e5b670c2a34" Tags=[Referrer="http://www.sky.net/deals"] Stuff=["Alpha" "Bravo" "Charlie"]]
As far as deciding whether to serialize properties based on the presence of a [Safe] attribute, that's sort of a second question. You will need to create your own ContractResolver and override CreateProperty, for instance as shown here: Using JSON.net, how do I prevent serializing properties of a derived class, when used in a base class context?
Update
If you want to retain the comma separator for arrays but not objects, modify WriteValueDelimiter as follows:
/// <summary>
/// Writes the JSON value delimiter. (Remove this override if you want to retain the comma separator.)
/// </summary>
protected override void WriteValueDelimiter()
{
if (WriteState == WriteState.Array)
_writer.Write(',');
else
_writer.Write(' ');
}

Categories

Resources