I an using DataContractSerializer and i want to separate data of same object into multiple files.
[DataContract]
public class TestObj
{
[DataMember]
protected double field1 = 0.0;
[DataMember]
protected double field2 = 0.0;
}
Specifically I want to save field1 in one XML file and field2 in a different XML file. Is there any way to do that using data contract serialization?
This is how I am serializing currently:
DataContractSerializer serializaer = new DataContractSerializer(GetType(), null,
0x7FFFFFFF /*maxItemsInObjectGraph*/,
false /*ignoreExtensionDataObject*/,
true /*preserveObjectReferences : this is most important to me */,
null /*dataContractSurrogate*/);
var fs = File.Open(fName, FileMode.Create);
serializaer.WriteObject(fs, this);
fs.Dispose();
return true;
I can suggest using Custom Xml Writer paired with serializer.
public class CustomWriter : XmlTextWriter
{
public CustomWriter(TextWriter writer) : base(writer) { }
public CustomWriter(Stream stream, Encoding encoding) : base(stream, encoding) { }
public CustomWriter(string filename, Encoding encoding) : base(filename, encoding) { }
public List<string> FieldList { get; set; }
private string _localName;
public override void WriteStartElement(string prefix, string localName, string ns)
{
if (!FieldList.Contains(localName))
base.WriteStartElement(prefix, localName, ns);
else
_localName = localName;
}
public override void WriteString(string text)
{
if (!FieldList.Contains(_localName))
base.WriteString(text);
}
public override void WriteEndElement()
{
if (!FieldList.Contains(_localName))
base.WriteEndElement();
else
_localName = null;
}
}
Use it as follows:
var data = new TestObj();
var serializer = new DataContractSerializer(
typeof(TestObj), null, 0x7FFFFFFF, false, true, null);
using (var writer = new CustomWriter(Console.Out)) // Specify filename or stream instead
{
writer.Formatting = Formatting.Indented;
writer.FieldList = new List<string> { "field1", "field3" }; // Specify fields to ignore
serializer.WriteObject(writer, data);
}
Just specify a list of fields that should be ignored in the FieldList property.
Of course, with this way, the DataContractSerializer will intermediate create xml with all the elements contained in the class. And only after that our custom writer will filter them. But it will happen on the fly, very quickly and effectively.
Once you serialise the whole object you can then split it using Linq to XML.
Oh, I even found an example how to do it at How to Split an XML file into multiple XML Files.
Related
I'm trying to send a Stream of data to an API through my own API. The third-party API takes an object and the object has an object property who has a property value of Stream. In my code that consumes the my API, I need to read a CSV file and then serialize the DTO object which containts the Stream property to send it to my API. My API will then just pass the DTO object to the third party API.
My issue is that I am currently serializing the Stream property as a base 64 string, but am unable to de-serialize it back into the DTO object.
DTO Object:
public class BulkLeadRequest
{
public Format3 FileFormat { get; set; }
public FileParameter FileParameter { get; set; }
public string LookupField { get; set; }
public string PartitionName { get; set; }
public int? ListId { get; set; }
public int? BatchId { get; set; }
}
Format3 is an enum:
public enum Format3
{
[System.Runtime.Serialization.EnumMember(Value = #"csv")]
Csv = 0,
[System.Runtime.Serialization.EnumMember(Value = #"tsv")]
Tsv = 1,
[System.Runtime.Serialization.EnumMember(Value = #"ssv")]
Ssv = 2,
}
FileParamter object with Stream property:
public partial class FileParameter
{
public FileParameter(System.IO.Stream data)
: this(data, null)
{
}
public FileParameter(System.IO.Stream data, string fileName)
: this(data, fileName, null)
{
}
public FileParameter(System.IO.Stream data, string fileName, string contentType)
{
Data = data;
FileName = fileName;
ContentType = contentType;
}
public System.IO.Stream Data { get; private set; }
public string FileName { get; private set; }
public string ContentType { get; private set; }
}
In order to test the serialization I have this small unit test.
public void TestSerializationDeserialization()
{
BulkLeadRequest bulkLeadRequest = new BulkLeadRequest();
bulkLeadRequest.FileFormat = Format3.Csv;
bulkLeadRequest.FileParameter = new FileParameter(new StreamReader("C:\temp\\SmallFile.csv").BaseStream);
string serializedObject = JsonConvert.SerializeObject(bulkLeadRequest, new StreamStringConverter());
var obj = JsonConvert.DeserializeObject<BulkLeadRequest>(serializedObject,
new JsonSerializerSettings {Converters = new List<JsonConverter> {new StreamStringConverter()}}); // Issue here
Assert.Equal(bulkLeadRequest, obj);
}
The StreamStringConverter:
public class StreamStringConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return typeof(Stream).IsAssignableFrom(objectType);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
string objectContents = (string)reader.Value;
byte[] base64Decoded = Convert.FromBase64String(objectContents);
MemoryStream memoryStream = new MemoryStream(base64Decoded);
return memoryStream;
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
FileStream valueStream = (FileStream)value;
byte[] fileBytes = new byte[valueStream.Length];
valueStream.Read(fileBytes, 0, (int)valueStream.Length);
string bytesAsString = Convert.ToBase64String(fileBytes);
writer.WriteValue(bytesAsString);
}
}
In my TestSerializationDeserialization unit test, I get the following error:
An exception of type 'Newtonsoft.Json.JsonSerializationException' occurred in Newtonsoft.Json.dll but was not handled in user code
Unable to find a constructor to use for type Cocc.MarketoSvcs.Business.FileParameter. A class should either have a default constructor, one constructor with arguments or a constructor marked with the JsonConstructor attribute. Path 'FileParameter.Data', line 1, position 40.
If I add a default constructor to FileParameter then the Data property will be empty when deserialized. If I remove the implicit conversion in the JsonConvert.DeserializeObject<BulkLeadRequest>(..); call to JsonConvert.DeserializeObject(...); I will get an object, but not the BulkLeadRequest and with the base64 string still and not the Stream object I'd expect.
Serialized:
{"FileFormat":0,"FileParameter":{"Data":"RklSU1ROQU1FLE1ETElOSVQsTEFTVE5BTUUNCkNyaXN0aW5hLE0sRGlGYWJpbw0KTmVsbHksLFBhbGFjaW9zDQpNYXR0aGV3LEEsTmV2ZXJz","FileName":null,"ContentType":null},"LookupField":null,"PartitionName":null,"ListId":null,"BatchId":null}
What am I doing wrong?
You are getting this error because the serializer does not know how to instantiate your FileParameter class. It prefers to use a default constructor if possible. It can use a parameterized constructor in some cases, if you give it a hint as to which one by marking it with a [JsonConstructor] attribute and all the constructor parameters match up to properties in the JSON object. However, that won't work in your case because all of your constructors expect a Stream and you require special handling for that.
To solve this you need a way for the serializer to instantiate your class without the stream and then use the converter to create the stream. You said you tried adding a default constructor, but then the stream was null. The reason it did not work is because all of your setters are private. With the default constructor, the serializer was able to instantiate the FileParameter, but then it could not populate it.
There are a couple of ways to make it work.
Solution 1 - modify the FileParameter class
Add a default constructor to the FileParameter class as you did before. It can be private if you also mark it with [JsonConstructor].
Add [JsonProperty] attributes to all of the private-set properties that need to be populated from the JSON. This will allow the serializer to write to the properties even though they are private.
So your class would look like this:
public partial class FileParameter
{
public FileParameter(System.IO.Stream data)
: this(data, null)
{
}
public FileParameter(System.IO.Stream data, string fileName)
: this(data, fileName, null)
{
}
public FileParameter(System.IO.Stream data, string fileName, string contentType)
{
Data = data;
FileName = fileName;
ContentType = contentType;
}
[JsonConstructor]
private FileParameter()
{
}
[JsonProperty]
public System.IO.Stream Data { get; private set; }
[JsonProperty]
public string FileName { get; private set; }
[JsonProperty]
public string ContentType { get; private set; }
}
Here is a working demo: https://dotnetfiddle.net/371ggK
Solution 2 - use a ContractResolver
If you cannot easily modify the FileParameter class because it is generated or you don't own the code, you can use a custom ContractResolver to do the same thing programmatically. Below is the code you would need for that.
public class CustomContractResolver : DefaultContractResolver
{
protected override JsonObjectContract CreateObjectContract(Type objectType)
{
JsonObjectContract contract = base.CreateObjectContract(objectType);
if (objectType == typeof(FileParameter))
{
// inject a "constructor" to use when creating FileParameter instances
contract.DefaultCreator = () => new FileParameter(null);
// make the private properties writable
string[] propNames = new string[]
{
nameof(FileParameter.Data),
nameof(FileParameter.FileName),
nameof(FileParameter.ContentType),
};
foreach (JsonProperty prop in contract.Properties.Where(p => propNames.Contains(p.UnderlyingName)))
{
prop.Writable = true;
}
}
return contract;
}
}
To use the resolver, add it to the JsonSerializerSettings along with the StreamStringConverter:
var obj = JsonConvert.DeserializeObject<BulkLeadRequest>(serializedObject,
new JsonSerializerSettings
{
ContractResolver = new CustomContractResolver(),
Converters = new List<JsonConverter> { new StreamStringConverter() }
});
Working demo: https://dotnetfiddle.net/VHf359
By the way, in the WriteJson method of your StreamStringConverter, I noticed you are casting the value to a FileStream even though the CanConvert method says it can handle any Stream. You can fix it by changing this line:
FileStream valueStream = (FileStream)value;
to this:
Stream valueStream = (Stream)value;
I need to deserialize a raw xml to a particular object. However I am having issues when it comes to boolean and enum types, as case sensitivity is intact.
public MyObjectTypeDeserializeMethod(string rawXML)
{
DataContractSerializer serializer = new DataContractSerializer(typeof(MyObjectType));
MyObjectType tempMyObject = null;
try
{
// Use Memory Stream
using (MemoryStream memoryStream = new MemoryStream())
{
// Use Stream Writer
using (StreamWriter streamWriter = new StreamWriter(memoryStream))
{
// Write and Flush
streamWriter.Write(rawXML);
streamWriter.Flush();
// Read
memoryStream.Position = 0;
tempMyObject = (MyObjectType)serializer.ReadObject(memoryStream);
}
}
}
catch (Exception e)
{
throw e;
}
return tempMyObject;
}
public class MyObjectType
{
public bool boolValue {get; set;}
public MyEnumType enumValue {get; set;}
}
If the raw XML contains
<boolValue>true</boolValue>
it works fine. However it throws an exception whenever the value is different to the the previous, such as
<boolValue>True</boolValue>
How can this issue be solved in order to allow case insensitive boolean and enum values to be passed from the raw XML?
The xml specification defines xml to be case sensitive, and defines booleans to be the (note case) literals true and false. DataContractSerializer is doing it right. If the value is True, then it isn't an xml boolean, and should be treated as a string.
There are many way to resolve this. An simple way is this approach:
public class MyObjectType
{
[XmlIgnore] public bool BoolValue; // this is not mapping directly from the xml
[XmlElement("boolValue")]
public string BoolInternalValue // this is mapping directly from the xml and assign the value to the BoolValue property
{
get { return BoolValue.ToString(); }
set
{
bool.TryParse(value, out BoolValue);
}
}
...
and I use XmlSerializer for Deserialize the xml:
public static T Deserialize<T>(string xmlContent)
{
T result;
var xmlSerializer = new XmlSerializer(typeof(T));
using (TextReader textReader = new StringReader(xmlContent))
{
result = ((T)xmlSerializer.Deserialize(textReader));
}
return result;
}
I am converting all of our projects to use JSON.NET rather than DataContractJsonSerializer. How do I use JSON.NET to write to an XmlDictionaryWriter?
Current Implementation (using DataContractJsonSerializer):
public class ErrorBodyWriter : BodyWriter
{
protected override void OnWriteBodyContents(XmlDictionaryWriter writer)
{
if (Format == WebContentFormat.Json)
{
// How do I use json.net below?
var serializer =
new DataContractJsonSerializer(typeof(ErrorMessage));
serializer.WriteObject(writer, Error);
}
else { // xml }
}
public ErrorBodyWriter() : base(true){}
public ErrorMessage Error { get; set; }
public WebContentFormat Format { get; set; }
}
You can't do that directly. WCF is primarily XML-based, and to support JSON it was defined a JSON-to-XML mapping which if one wrote XML in a very specific format, and the underlying XML writer could produce JSON, then the proper JSON would be output.
The default WCF JSON serializer (DataContractJsonSerializer) knows how to write JSON to a XML writer using that mapping. JSON.NET doesn't. So one option is to write to a memory stream using JSON.NET, then read it into XML using the WCF JSON/XML reader, then use that to write to the XmlDictionaryWriter. The code would look somehow like the snippet below (written on notepad, some kinks may need to be addressed):
public class ErrorBodyWriter : BodyWriter
{
protected override void OnWriteBodyContents(XmlDictionaryWriter writer)
{
if (Format == WebContentFormat.Json)
{
var json = JsonConvert.SerializeObject(Error);
var jsonBytes = Encoding.UTF8.GetBytes(json);
using (var reader = JsonReaderWriterFactory.CreateJsonReader(jsonBytes, XmlDictionaryReaderQuotas.Max)) {
writer.WriteNode(reader, false);
}
}
else { // xml }
}
public ErrorBodyWriter() : base(true){}
public ErrorMessage Error { get; set; }
public WebContentFormat Format { get; set; }
}
If you run this code:
public class Program
{
public class MyClass
{
public string Text { get; set; }
}
static MyClass myClass = new MyClass();
static string Desktop = "C:\\Users\\Juan Luis\\Desktop\\";
static void Main(string[] args)
{
myClass.Text = "\r\nhello";
Console.WriteLine((int)myClass.Text[0]);
Save(Desktop + "file.xml", myClass);
myClass = Load(Desktop + "file.xml");
Console.WriteLine((int)myClass.Text[0]);
Console.Read();
}
private static MyClass Load(string fileName)
{
MyClass result = null;
using (Stream stream = File.Open(fileName, FileMode.Open))
{
XmlSerializer xmlFormatter = new XmlSerializer(typeof(MyClass));
result = (MyClass)xmlFormatter.Deserialize(stream);
}
return result;
}
private static void Save(string fileName, MyClass obj)
{
using (Stream tempFileStream = File.Create(fileName))
{
XmlSerializer xmlFormatter = new XmlSerializer(typeof(MyClass));
xmlFormatter.Serialize(tempFileStream, obj);
}
}
}
The output will be 13, 10. The XmlSerializer is removing the carriage return. This is a problem in my case because I need to compare strings for equality in a class that gets serialized and deserialized, and this is causing two strings that are equal before serializing to be unequal after serialized. What would be the best work around?
Edit: After reading answers, this was my solution, in case it will help anyone:
public class SafeXmlSerializer : XmlSerializer
{
public SafeXmlSerializer(Type type) : base(type) { }
public new void Serialize(Stream stream, object o)
{
XmlWriterSettings ws = new XmlWriterSettings();
ws.NewLineHandling = NewLineHandling.Entitize;
using (XmlWriter xmlWriter = XmlWriter.Create(stream, ws))
{
base.Serialize(xmlWriter, o);
}
}
}
I wouldn't call it unreliable exactly: the XmlSerializer strips white space around text inside elements. If it didn't do this then the meaning of XML documents would change according to how you formatted them in the IDE.
You could consider putting the text in a CDATA section, which will preserve the contents exactly. For example, How do you serialize a string as CDATA using XmlSerializer?
Edit: This looks to have a better explanation of where the problem lies, along with a simpler solution - How to keep XmlSerializer from killing NewLines in Strings?
Can I make XmlSerializer ignore the namespace (xmlns attribute) on deserialization so that it doesn't matter if the attribute is added or not or even if the attribute is bogus? I know that the source will always be trusted so I don't care about the xmlns attribute.
Yes, you can tell the XmlSerializer to ignore namespaces during de-serialization.
Define an XmlTextReader that ignores namespaces. Like so:
// helper class to ignore namespaces when de-serializing
public class NamespaceIgnorantXmlTextReader : XmlTextReader
{
public NamespaceIgnorantXmlTextReader(System.IO.TextReader reader): base(reader) { }
public override string NamespaceURI
{
get { return ""; }
}
}
// helper class to omit XML decl at start of document when serializing
public class XTWFND : XmlTextWriter {
public XTWFND (System.IO.TextWriter w) : base(w) { Formatting= System.Xml.Formatting.Indented;}
public override void WriteStartDocument () { }
}
Here's an example of how you would de-serialize using that TextReader:
public class MyType1
{
public string Label
{
set { _Label= value; }
get { return _Label; }
}
private int _Epoch;
public int Epoch
{
set { _Epoch= value; }
get { return _Epoch; }
}
}
String RawXml_WithNamespaces = #"
<MyType1 xmlns='urn:booboo-dee-doo'>
<Label>This document has namespaces on its elements</Label>
<Epoch xmlns='urn:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'>0</Epoch>
</MyType1>";
System.IO.StringReader sr;
sr= new System.IO.StringReader(RawXml_WithNamespaces);
var s1 = new XmlSerializer(typeof(MyType1));
var o1= (MyType1) s1.Deserialize(new NamespaceIgnorantXmlTextReader(sr));
System.Console.WriteLine("\n\nDe-serialized, then serialized again:\n");
XmlSerializerNamespaces ns = new XmlSerializerNamespaces();
ns.Add("urn", "booboo-dee-doo");
s1.Serialize(new XTWFND(System.Console.Out), o1, ns);
Console.WriteLine("\n\n");
The result is like so:
<MyType1>
<Label>This document has namespaces on its elements</Label>
<Epoch>0</Epoch>
</MyType1>
If you expect no namespace, but the input has namespaces, then you can set
Namespaces = false
on your XmlTextReader.
Exdended Wolfgang Grinfeld answer (w/o exception handling):
public static Message Convert(XmlDocument doc)
{
Message obj;
using (TextReader textReader = new StringReader(doc.OuterXml))
{
using (XmlTextReader reader = new XmlTextReader(textReader))
{
reader.Namespaces = false;
XmlSerializer serializer = new XmlSerializer(typeof(Message));
obj = (Message)serializer.Deserialize(reader);
}
}
return obj;
}
Solved this by using XmlSerializer Deserialize to read from xml instead from stream. This way before xml is Deserialized, using Regex to remove xsi:type from the xml. Was doing this is Portable Class Library for Cross Platform, so did not had many other options :(. After this the deserialization seems to work fine.
Following code can help,
public static TClass Deserialize<TClass>(string xml) where TClass : class, new()
{
var tClass = new TClass();
xml = RemoveTypeTagFromXml(xml);
var xmlSerializer = new XmlSerializer(typeof(TClass));
using (TextReader textReader = new StringReader(xml))
{
tClass = (TClass)xmlSerializer.Deserialize(textReader);
}
return tClass;
}
public static string RemoveTypeTagFromXml(string xml)
{
if (!string.IsNullOrEmpty(xml) && xml.Contains("xsi:type"))
{
xml = Regex.Replace(xml, #"\s+xsi:type=""\w+""", "");
}
return xml;
}
Why try to make the XmlSerializer forget how XML works? It's a fact of XML that two elements with the same name but different namespaces are different elements.
If you want to process XML that has no namespaces, then you should pre-process it to remove the namespaces, and then pass it to the serializer.