I am converting all of our projects to use JSON.NET rather than DataContractJsonSerializer. How do I use JSON.NET to write to an XmlDictionaryWriter?
Current Implementation (using DataContractJsonSerializer):
public class ErrorBodyWriter : BodyWriter
{
protected override void OnWriteBodyContents(XmlDictionaryWriter writer)
{
if (Format == WebContentFormat.Json)
{
// How do I use json.net below?
var serializer =
new DataContractJsonSerializer(typeof(ErrorMessage));
serializer.WriteObject(writer, Error);
}
else { // xml }
}
public ErrorBodyWriter() : base(true){}
public ErrorMessage Error { get; set; }
public WebContentFormat Format { get; set; }
}
You can't do that directly. WCF is primarily XML-based, and to support JSON it was defined a JSON-to-XML mapping which if one wrote XML in a very specific format, and the underlying XML writer could produce JSON, then the proper JSON would be output.
The default WCF JSON serializer (DataContractJsonSerializer) knows how to write JSON to a XML writer using that mapping. JSON.NET doesn't. So one option is to write to a memory stream using JSON.NET, then read it into XML using the WCF JSON/XML reader, then use that to write to the XmlDictionaryWriter. The code would look somehow like the snippet below (written on notepad, some kinks may need to be addressed):
public class ErrorBodyWriter : BodyWriter
{
protected override void OnWriteBodyContents(XmlDictionaryWriter writer)
{
if (Format == WebContentFormat.Json)
{
var json = JsonConvert.SerializeObject(Error);
var jsonBytes = Encoding.UTF8.GetBytes(json);
using (var reader = JsonReaderWriterFactory.CreateJsonReader(jsonBytes, XmlDictionaryReaderQuotas.Max)) {
writer.WriteNode(reader, false);
}
}
else { // xml }
}
public ErrorBodyWriter() : base(true){}
public ErrorMessage Error { get; set; }
public WebContentFormat Format { get; set; }
}
Related
I'm trying to deserialize some xml into a C# object. The trick is for the most part, I know what this object will look like. However, this is one child that has dynamic elements.
(here is an example)
<measurement>
<Time>2021-02-02</Time>
<ID>1</ID>
<LeftWheel>
<ValuesRead>
<DynamicValue>12.3</DynamicValue>
<DynamicValue2>2.3</DynamicValue2>
<DynamicValue4>1.3</DynamicValue4>
<DynamicValue3>10.3</DynamicValue3>
</ValuesRead>
</LeftWheel>
<RightWheel>
<ValuesRead>
<DynamicValue>12.3</DynamicValue>
<DynamicValue2>2.3</DynamicValue2>
<DynamicValue6>1.3</DynamicValue6>
<DynamicValue10>10.3</DynamicValue10>
</ValuesRead>
</RightWheel>
</measurement>
In this XML, Measurement, Time, and ID are always going to in the object.
The LeftWheel and RightWheel elements are always going to be there with ValuesRead, but the ValuesRead children are dynamic and can be anything.
I have tried making a C# object to reflect most the structure, and then using the XmlSerializer.UnknownElement to pick up the unknown elements in the ValuesRead element, but I cannot link it to the parent above to know if it is on the LeftWheel or RightWheel.
XmlSerializer serializer = new XmlSerializer(typeof(FVISSiteEvent));
serializer.UnknownElement += UnknownElementFound;
Is there a way I can define the LeftWheel and RightWheel classes to be dynamic for the serialization, while having the other classes not dynamic?
You should be able to use the UnknownElementFound event to manually handle these aspects of serialization. See: Serialize XML array of unknown element name
Other options could be to specify the types you expect to see as XmlElementAtrribute decorated properties and they will just be null if they aren’t deserialized.
There’s also the nuclear option of implementing IXmlSerializable in your class and taking full control of the deserialization.
Uses Custom Serializer :
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Serialization;
using System.Xml.Linq;
using System.Xml.Schema;
namespace ConsoleApplication16
{
class Program
{
const string INPUT_FILENAME = #"c:\temp\test.xml";
const string OUTPUT_FILENAME = #"c:\temp\test1.xml";
static void Main(string[] args)
{
XmlReader reader = XmlReader.Create(INPUT_FILENAME);
XmlSerializer serializer = new XmlSerializer(typeof(Measurement));
Measurement measurement = (Measurement)serializer.Deserialize(reader);
XmlWriterSettings settings = new XmlWriterSettings();
settings.Indent = true;
XmlWriter writer = XmlWriter.Create(OUTPUT_FILENAME,settings);
serializer.Serialize(writer, measurement);
}
}
[XmlRoot("measurement")]
public class Measurement
{
public DateTime Time { get; set; }
public int ID { get; set; }
[XmlArray("LeftWheel")]
[XmlArrayItem("ValuesRead")]
public List<Wheel> leftWheel { get; set; }
[XmlArray("RightWheel")]
[XmlArrayItem("ValuesRead")]
public List<Wheel> rightWheel { get; set; }
}
public class Wheel : IXmlSerializable
{
List<decimal> values { get; set; }
// Xml Serialization Infrastructure
public void WriteXml(XmlWriter writer)
{
int count = 0;
XElement valuesRead = new XElement("ValuesRead");
for (int i = 0; i < values.Count; i++ )
{
valuesRead.Add(new XElement("ValuesRead" + (i == 0? "" : i.ToString()), values[i]));
}
writer.WriteRaw(valuesRead.ToString());
}
public void ReadXml(XmlReader reader)
{
XElement values = (XElement)XElement.ReadFrom(reader);
this.values = values.Elements().Where(x => x.Name.LocalName.StartsWith("DynamicValue")).Select(x => (decimal)x).ToList();
}
public XmlSchema GetSchema()
{
return (null);
}
}
}
Besides using custom Xml serialization to deserialize your xml file, here is one another approach using Cinchoo ETL - an open source library to handle it simple way (those open to try it!)
Define POCO Class
public class Measurement
{
public DateTime Time { get; set; }
public int ID { get; set; }
[ChoXPath("LeftWheel/ValuesRead/*")]
public double[] LeftWheel { get; set; }
[ChoXPath("RightWheel/ValuesRead")]
public dynamic RightWheel { get; set; }
}
Deserialize using ChoETL
using (var r = ChoXmlReader<Measurement>.LoadText(xml)
.WithXPath("/")
)
{
foreach (var rec in r)
rec.Print();
}
Sample fiddle: https://dotnetfiddle.net/KtNvra
Disclaimer: I'm author of this library.
I've managed to resolve this using a dynamic type when deserializing.
When I deserialize ValuesRead, it is a defined as a dynamic type.
When deserialized, it turns into an XmlNode and from there I iterate over the node use the Name and InnerText values to read all the data.
I'm trying to send a Stream of data to an API through my own API. The third-party API takes an object and the object has an object property who has a property value of Stream. In my code that consumes the my API, I need to read a CSV file and then serialize the DTO object which containts the Stream property to send it to my API. My API will then just pass the DTO object to the third party API.
My issue is that I am currently serializing the Stream property as a base 64 string, but am unable to de-serialize it back into the DTO object.
DTO Object:
public class BulkLeadRequest
{
public Format3 FileFormat { get; set; }
public FileParameter FileParameter { get; set; }
public string LookupField { get; set; }
public string PartitionName { get; set; }
public int? ListId { get; set; }
public int? BatchId { get; set; }
}
Format3 is an enum:
public enum Format3
{
[System.Runtime.Serialization.EnumMember(Value = #"csv")]
Csv = 0,
[System.Runtime.Serialization.EnumMember(Value = #"tsv")]
Tsv = 1,
[System.Runtime.Serialization.EnumMember(Value = #"ssv")]
Ssv = 2,
}
FileParamter object with Stream property:
public partial class FileParameter
{
public FileParameter(System.IO.Stream data)
: this(data, null)
{
}
public FileParameter(System.IO.Stream data, string fileName)
: this(data, fileName, null)
{
}
public FileParameter(System.IO.Stream data, string fileName, string contentType)
{
Data = data;
FileName = fileName;
ContentType = contentType;
}
public System.IO.Stream Data { get; private set; }
public string FileName { get; private set; }
public string ContentType { get; private set; }
}
In order to test the serialization I have this small unit test.
public void TestSerializationDeserialization()
{
BulkLeadRequest bulkLeadRequest = new BulkLeadRequest();
bulkLeadRequest.FileFormat = Format3.Csv;
bulkLeadRequest.FileParameter = new FileParameter(new StreamReader("C:\temp\\SmallFile.csv").BaseStream);
string serializedObject = JsonConvert.SerializeObject(bulkLeadRequest, new StreamStringConverter());
var obj = JsonConvert.DeserializeObject<BulkLeadRequest>(serializedObject,
new JsonSerializerSettings {Converters = new List<JsonConverter> {new StreamStringConverter()}}); // Issue here
Assert.Equal(bulkLeadRequest, obj);
}
The StreamStringConverter:
public class StreamStringConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return typeof(Stream).IsAssignableFrom(objectType);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
string objectContents = (string)reader.Value;
byte[] base64Decoded = Convert.FromBase64String(objectContents);
MemoryStream memoryStream = new MemoryStream(base64Decoded);
return memoryStream;
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
FileStream valueStream = (FileStream)value;
byte[] fileBytes = new byte[valueStream.Length];
valueStream.Read(fileBytes, 0, (int)valueStream.Length);
string bytesAsString = Convert.ToBase64String(fileBytes);
writer.WriteValue(bytesAsString);
}
}
In my TestSerializationDeserialization unit test, I get the following error:
An exception of type 'Newtonsoft.Json.JsonSerializationException' occurred in Newtonsoft.Json.dll but was not handled in user code
Unable to find a constructor to use for type Cocc.MarketoSvcs.Business.FileParameter. A class should either have a default constructor, one constructor with arguments or a constructor marked with the JsonConstructor attribute. Path 'FileParameter.Data', line 1, position 40.
If I add a default constructor to FileParameter then the Data property will be empty when deserialized. If I remove the implicit conversion in the JsonConvert.DeserializeObject<BulkLeadRequest>(..); call to JsonConvert.DeserializeObject(...); I will get an object, but not the BulkLeadRequest and with the base64 string still and not the Stream object I'd expect.
Serialized:
{"FileFormat":0,"FileParameter":{"Data":"RklSU1ROQU1FLE1ETElOSVQsTEFTVE5BTUUNCkNyaXN0aW5hLE0sRGlGYWJpbw0KTmVsbHksLFBhbGFjaW9zDQpNYXR0aGV3LEEsTmV2ZXJz","FileName":null,"ContentType":null},"LookupField":null,"PartitionName":null,"ListId":null,"BatchId":null}
What am I doing wrong?
You are getting this error because the serializer does not know how to instantiate your FileParameter class. It prefers to use a default constructor if possible. It can use a parameterized constructor in some cases, if you give it a hint as to which one by marking it with a [JsonConstructor] attribute and all the constructor parameters match up to properties in the JSON object. However, that won't work in your case because all of your constructors expect a Stream and you require special handling for that.
To solve this you need a way for the serializer to instantiate your class without the stream and then use the converter to create the stream. You said you tried adding a default constructor, but then the stream was null. The reason it did not work is because all of your setters are private. With the default constructor, the serializer was able to instantiate the FileParameter, but then it could not populate it.
There are a couple of ways to make it work.
Solution 1 - modify the FileParameter class
Add a default constructor to the FileParameter class as you did before. It can be private if you also mark it with [JsonConstructor].
Add [JsonProperty] attributes to all of the private-set properties that need to be populated from the JSON. This will allow the serializer to write to the properties even though they are private.
So your class would look like this:
public partial class FileParameter
{
public FileParameter(System.IO.Stream data)
: this(data, null)
{
}
public FileParameter(System.IO.Stream data, string fileName)
: this(data, fileName, null)
{
}
public FileParameter(System.IO.Stream data, string fileName, string contentType)
{
Data = data;
FileName = fileName;
ContentType = contentType;
}
[JsonConstructor]
private FileParameter()
{
}
[JsonProperty]
public System.IO.Stream Data { get; private set; }
[JsonProperty]
public string FileName { get; private set; }
[JsonProperty]
public string ContentType { get; private set; }
}
Here is a working demo: https://dotnetfiddle.net/371ggK
Solution 2 - use a ContractResolver
If you cannot easily modify the FileParameter class because it is generated or you don't own the code, you can use a custom ContractResolver to do the same thing programmatically. Below is the code you would need for that.
public class CustomContractResolver : DefaultContractResolver
{
protected override JsonObjectContract CreateObjectContract(Type objectType)
{
JsonObjectContract contract = base.CreateObjectContract(objectType);
if (objectType == typeof(FileParameter))
{
// inject a "constructor" to use when creating FileParameter instances
contract.DefaultCreator = () => new FileParameter(null);
// make the private properties writable
string[] propNames = new string[]
{
nameof(FileParameter.Data),
nameof(FileParameter.FileName),
nameof(FileParameter.ContentType),
};
foreach (JsonProperty prop in contract.Properties.Where(p => propNames.Contains(p.UnderlyingName)))
{
prop.Writable = true;
}
}
return contract;
}
}
To use the resolver, add it to the JsonSerializerSettings along with the StreamStringConverter:
var obj = JsonConvert.DeserializeObject<BulkLeadRequest>(serializedObject,
new JsonSerializerSettings
{
ContractResolver = new CustomContractResolver(),
Converters = new List<JsonConverter> { new StreamStringConverter() }
});
Working demo: https://dotnetfiddle.net/VHf359
By the way, in the WriteJson method of your StreamStringConverter, I noticed you are casting the value to a FileStream even though the CanConvert method says it can handle any Stream. You can fix it by changing this line:
FileStream valueStream = (FileStream)value;
to this:
Stream valueStream = (Stream)value;
I an using DataContractSerializer and i want to separate data of same object into multiple files.
[DataContract]
public class TestObj
{
[DataMember]
protected double field1 = 0.0;
[DataMember]
protected double field2 = 0.0;
}
Specifically I want to save field1 in one XML file and field2 in a different XML file. Is there any way to do that using data contract serialization?
This is how I am serializing currently:
DataContractSerializer serializaer = new DataContractSerializer(GetType(), null,
0x7FFFFFFF /*maxItemsInObjectGraph*/,
false /*ignoreExtensionDataObject*/,
true /*preserveObjectReferences : this is most important to me */,
null /*dataContractSurrogate*/);
var fs = File.Open(fName, FileMode.Create);
serializaer.WriteObject(fs, this);
fs.Dispose();
return true;
I can suggest using Custom Xml Writer paired with serializer.
public class CustomWriter : XmlTextWriter
{
public CustomWriter(TextWriter writer) : base(writer) { }
public CustomWriter(Stream stream, Encoding encoding) : base(stream, encoding) { }
public CustomWriter(string filename, Encoding encoding) : base(filename, encoding) { }
public List<string> FieldList { get; set; }
private string _localName;
public override void WriteStartElement(string prefix, string localName, string ns)
{
if (!FieldList.Contains(localName))
base.WriteStartElement(prefix, localName, ns);
else
_localName = localName;
}
public override void WriteString(string text)
{
if (!FieldList.Contains(_localName))
base.WriteString(text);
}
public override void WriteEndElement()
{
if (!FieldList.Contains(_localName))
base.WriteEndElement();
else
_localName = null;
}
}
Use it as follows:
var data = new TestObj();
var serializer = new DataContractSerializer(
typeof(TestObj), null, 0x7FFFFFFF, false, true, null);
using (var writer = new CustomWriter(Console.Out)) // Specify filename or stream instead
{
writer.Formatting = Formatting.Indented;
writer.FieldList = new List<string> { "field1", "field3" }; // Specify fields to ignore
serializer.WriteObject(writer, data);
}
Just specify a list of fields that should be ignored in the FieldList property.
Of course, with this way, the DataContractSerializer will intermediate create xml with all the elements contained in the class. And only after that our custom writer will filter them. But it will happen on the fly, very quickly and effectively.
Once you serialise the whole object you can then split it using Linq to XML.
Oh, I even found an example how to do it at How to Split an XML file into multiple XML Files.
I'm working with some third party XML that has no formally defined schema, only example XML. I have several thousand XML files from this third party. There is no guarantee that every possible element lies within one or more of these files. The third party service could send me a new file with a new element!
I can view these files and reverse engineer types relatively easily.
For example:
<MyObject>
<MyProperty>Some value</MyProperty>
</MyObject>
Could deserialize to
public class MyObject
{
public string MyProperty { get; set; }
}
No problems so far.
But what if I attempt to deserialize this:
<MyObject>
<MyProperty>Some value</MyProperty>
<MyOtherProperty>Some value</MyOtherProperty>
</MyObject>
into my class above? I want it to throw an exception, so I can be notified that my class does not accommodate MyOtherProperty.
Is there a way to do this?
I'd like to share the code I wrote using the accepted answer. Using the below utility method, I can deserialize without checks for unknown stuff, and with checks, by setting strict=true.
I hope readers find this useful!
public static T XmlDeserialize<T>(string xml, bool strict = false)
{
using (var stringReader = new StringReader(xml))
{
using (var xmlTextReader = new XmlTextReader(stringReader))
{
var xmlSerializer = new XmlSerializer(typeof(T));
if (strict)
{
var options = new XmlDeserializationEvents();
options.OnUnknownElement += (sender, args) =>
{
throw new XmlDeserializationException(xml, $"Unexpected Element {args.Element.LocalName} on line {args.LineNumber}.");
};
options.OnUnknownAttribute += (sender, args) =>
{
throw new XmlDeserializationException(xml, $"Unexpected Element: {args.Attr.LocalName} on line {args.LineNumber}.");
};
options.OnUnknownNode += (sender, args) =>
{
throw new XmlDeserializationException(xml, $"Unexpected Element: {args.LocalName} on line {args.LineNumber}.");
};
return (T)xmlSerializer.Deserialize(xmlTextReader, options);
}
return (T)xmlSerializer.Deserialize(xmlTextReader);
}
}
}
And that exception class I'm throwing looks like this:
public class XmlDeserializationException : Exception
{
public string Xml { get; private set; }
public XmlDeserializationException(
string xml, string message) : base (message)
{
Xml = xml;
}
}
I can check my logs and look up the line number in the actual xml. Works perfectly. Thanks, pfx.
The XmlSerializer has a Deserialize overload allowing to pass in an options element by which to hook to some events; eg. OnUnknownElement.
XmlDeserializationEvents options = new XmlDeserializationEvents();
options.OnUnknownElement += (sender, args) => {
XmlElement unknownElement = args.Element;
// throw an Exception with this info.
} ;
var o = serializer.Deserialize(xml, options) as MyObject;
The OnUnknowElement takes nested elements into account.
With the classes below
public class MyObject
{
public string MyProperty { get; set; }
public MyOtherObject Other { get; set; }
}
public class MyOtherObject
{
public string SomeProperty { get; set; }
}
and the following xml
<MyObject>
<MyProperty>Some value</MyProperty>
<Other>
<SomeProperty>...</SomeProperty>
<UFO>...</UFO>
</Other>
</MyObject>
The OnUnknowElement handler will trigger for the UFO element.
One way of doing so would be to use your current object model and create an XSD out of it. You can then check new files against that XSD and throw if it doesn't validate.
Extend your class with an XmlAnyElement container.
Any unknown elements will end up in that array.
After deserialization check whether that array is empty.
public class MyObject
{
[XmlAnyElement]
public XmlElement[] UnknownElements;
public string MyProperty { get; set; }
}
I have a json response displayed as below. I am using datacontractserializer to serialize.
If I need only "text" and "created time" from this Json response...how should be my DataContract looks like?
Do I need to have all these six properties in my data contract ? and use "IgnoreDataMember" as attribute?
Also, do I need to give same name for my properties in datacontract (Ex : screenName, text as property name ?)
"abcDetails":[
{
"screenName":"my name",
"text":"tweet desc",
"createdTime":1423494304000,
"entities":{ },
"name":"abc",
"id":"123"
}]
To answer your questions:
You can omit properties that you do not need and DataContractJsonSerializer will skip over them.
You class's property names can differ from the JSON property names as long as you set the DataMemberAttribute.Name value to be the same as the name appearing in the JSON.
Your JSON is invalid, it is missing outer braces. I assume that's just a copy/paste error in your question.
Thus your classes could look like:
[DataContract]
public class Detail
{
[DataMember(Name="text")]
public string Text { get; set; }
[DataMember(Name="createdTime")]
public long CreatedTimeStamp { get; set; }
}
[DataContract]
public class RootObject
{
[DataMember(Name="abcDetails")]
public List<Detail> Details { get; set; }
}
And to test:
string json = #"
{
""abcDetails"":[
{
""screenName"":""my name"",
""text"":""tweet desc"",
""createdTime"":1423494304000,
""entities"":{ },
""name"":""abc"",
""id"":""123""
}]
}
";
var response = DataContractJsonSerializerHelper.GetObject<RootObject>(json);
foreach (var detail in response.Details)
{
Console.WriteLine(string.Format("Created Time: {0}; Text: \"{1}\"", detail.CreatedTimeStamp, detail.Text));
}
produces the output
Created Time: 1423494304000; Text: "tweet desc"
using the helper class:
public static class DataContractJsonSerializerHelper
{
public static T GetObject<T>(string json) where T : class
{
DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(T));
return GetObject<T>(json, serializer);
}
public static T GetObject<T>(string json, DataContractJsonSerializer serializer)
{
using (var stream = GenerateStreamFromString(json))
{
var obj = serializer.ReadObject(stream);
return (T)obj;
}
}
private static MemoryStream GenerateStreamFromString(string value)
{
return new MemoryStream(Encoding.Unicode.GetBytes(value ?? ""));
}
}