I am toying around with serializing and deserializing CSV files and I am wondering if there is an existing library, similar in concept to the XmlSerializer, which can declaratively define objects and (de)serialize them to/from a file or stream. I have looked around a bit but have not found anything focused on serialization. I already have pretty solid code for parsing CSV documents per RFC 4180, but what would be really helpful is the serialization part. What I am not looking for is just a parser, advice to use String.Split(), etc.
Is there an existing project out there, or should I build one?
Bonus etiquette question: if I do end up rolling my own serializer, is it appropriate to answer this question with a link to the codeplex project?
I would highly recommend servicestack.text for this purpose. Avialable on nuget:
Install-package servicestack.text
It suports serialization to many data formats and unlike the built in XmlSerializer, you don't need to decorate all your properties with attributes. Here's an example to serialize to CSV.
using ServiceStack.Text;
...
var csv = CsvSerializer.SerializeToCsv(new[]{
new Dog () {
Bark = "Woof!",
Male = true,
Size = 10
}});
I've used this project (CsvHelper) in the past, and it works similar to the build in .NET serializer classes in the sense that you use attributes to craft the input/output.
There's really no need to roll your own, since there are tons out there. If you do end up rolling your own, feel free to post it. Most users, when answering a question with something they've written themselves (or are affiliated in some way) usually give a disclaimer saying so as a courtesy.
You should take a look into FileHelpers Library.
Some sample code from their site:
using FileHelpers;
// First declare the record class
[DelimitedRecord(",")]
public class SampleType
{
public string Field1;
public int Field2;
}
public void WriteExample()
{
FileHelperEngine engine = new FileHelperEngine(typeof(SampleType));
SampleType[] records = new SampleType[1];
records[0] = new SampleType();
records[0].Field1 = "Hello World";
records[0].Field2 = 12;
engine.WriteFile("destination.txt", records);
// Now the file contains the created record in this format:
//
// Hello World,12
}
Related
I have a requirement to take a library of C# classes that implement protobuf-net, and convert them into .proto files, which need to be converted using protoc into .py files. I understand that the .ToProto() function does this just fine, but I came up against an issue involving collections and generics when converting from .proto to .py files. When trying to serialize a list of DateTimes, for example I get the following error X.proto:64:13. "List_TimeSpan" is not defined. As this had not caused an issue upon serialization into a protobuf file, I wasn't aware of this situation at the time.
I am currently using proto-buf.net 2.3.2 for this project; it's the version some of my other work has been done with and I am aware that this could just be solved with a version upgrade. I'm just not sure if that is the answer with the digging I've done so far. If there's something else that I'm missing, I would truly appreciate any help that can be thrown my way.
If we consider:
[ProtoContract]
public class Foo {
[ProtoMember(12)]
public List<DateTime> Times { get; } = new List<DateTime>();
}
then GetProto<T>() in both v2.3.2 (the version mentioned in the question) and v2.4.4 (the current default version) generate:
syntax = "proto2";
import "protobuf-net/bcl.proto"; // schema for protobuf-net's handling of core .NET types
message Foo {
repeated .bcl.DateTime Times = 12;
}
So on the surface of it, it should already be just fine. If you're doing something more exotic (perhaps using a list in a dictionary value?), I'd be happy to help, but I'm going to need more of a clue as to what you're doing. Posting some C# that shows the thing you're seeing would be a great place to start.
Note that when protobuf-net first came around, there was no agreed transmission format for date/time-like values, so protobuf-net made something up, but it turns out to not be a convenient fit for cross-platform work; the following is a hard breaking change (it is not data compatible), but if possible, I would strongly recommend the well-known format that Google added later:
[ProtoContract]
public class Foo {
[ProtoMember(12, DataFormat = DataFormat.WellKnown)]
public List<DateTime> Times { get; } = new List<DateTime>();
}
which generates:
syntax = "proto2";
import "google/protobuf/timestamp.proto";
message Foo {
repeated .google.protobuf.Timestamp Times = 12;
}
I've got c# objects that I need references to in javascript for default population. Currently I'm maintaining 2 different objects which is not that maintainable.
For example ( simplified for demo purposes ):
C#
public class Text
{
public string Name {get;set;}
}
JSON
{
'text': {
name: undefined
}
}
I know there is a number of ways to accomplish this but wondering if anyone has a recommended solution. Thanks!
I personally recommend json.NET. Getting the json of any object is as simple as;
using Newtonsoft.Json;
string json = JsonConvert.SerializeObject(new Text { Name = "test" });
There are a lot of other options but I've been using it since before there was json serilization support in .NET and I strongly prefer it over what is there now. In fact I think it's better in every way, if you want a big robust data layer I like it more and it's vastly superior for one off serilizations.
If you are using .NET 4.0 or above, you can use DataContractJsonSerializer class.
I recommend you to look at this benchmark http://theburningmonk.com/2013/09/binary-and-json-serializer-benchmarks-updated/
I have a XML file, with a structure like
<items>
<item>
<someDetail>
A value here
</someDetail>
</item>
<item>
<someDetail>
Another value here
</someDetail>
</item>
</items>
With multiple items in it.
I want to deserialize the XML on session start ideally, to turn the XML data to objects based on a class (c# asp.net 4)
I have tried several ways with either no success, or a solution which seems clunky and inelegant.
What would people suggest?
I have tried using the xsd.exe tool, and have tried with the xml reader class, as well as usin XElement class to loop through the xml and then create new someObject(props).
These maybe the best and/or only way, but with it being so easy for database sources using the entities framework, I wondered if there was a similar way to do the same but from a xml source.
The best way to deserialize XML it to create a class that corresponds to the XML structure into which the XML data will deserialize.
The latest serialization technology uses Data Contracts and the DataContractSerializer.
You decorate the class I mentioned above with DataMember and DataItem attributes and user the serializer to deserialize.
I'd use directly the .NET XML serialization - classes declarations:
public class Item {
[XmlElement("someDetail")]
public string SomeDetail;
} // class Item
[XmlRoot("items")]
public class MyData {
[XmlElement("item")]
public List<Item> Items;
public static MyData Deserialize(Stream source)
{
XmlSerializer serializer = new XmlSerializer(typeof(MyData));
return serializer.Deserialize(source) as MyData;
} // Deserialize
} // class MyData
and then to read the XML:
using (FileStream fs = new FileStream(#"c:\temp\items.xml", FileMode.Open, FileAccess.Read)) {
MyData myData = MyData.Deserialize(fs);
}
I've concluded is there is not simple unified mechanism (probably due to the inherent complexities involved with non trivial cases - this question always crops up in the context of simple scenarios like your example xml).
Xml serialization is pretty easy to use. For your example, you would just have to create a class to contain a items and another class for the actual item. You might have to apply some attributes to get everything to work correctly, but the coding will not be much. Then it's as easy as -
var serializer = new XmlSerializer(typeof(ItemsContainer));
var items = serializer.Deserialize(...) as ItemsContainer;
Datasets are sometimes considered "yesterday tech" but I use them when they solve the problem well, and you can leverage the designer. The generated code is not pretty but the bottom line is you can persist to a database via the auto generated adapters and to XML using a method right on the data set. You can read it in this way as well.
XSD.exe isn't that bad once you get used to it. I printed the help to a text file and included it in my solutions for a while. When you use the /c option to create classes, you get clean code that can be used with the XmlSerialzier.
Visual Studio 2010 (maybe other versions too) has an XML menu which appears when you have an Xml file open and from that you can also generate an XSD from sample Xml. So in a couple of steps you could take your example xml and generate the XSD, then run it through XSD.exe and use the generated classes with a couple of lines XmlSerializer code... it feels like a lot of machinations but you get used to it.
I have a Java application for Android that will have some static data, which will be loaded at start up. This data is generated by a C# app as a bunch of classes with lists and Dictionaries, something like this:
public class DataContainer
{
public List<A> As = new List<A>();
public Dictionary<int, B> Bs = new Dictionary<int, B>();
// ...
}
public class A
{
public int IdA;
public int IdSomething;
public DateTime dt;
//...
}
public class B
{
public int IdB;
public string Name;
List<C> Cs = new List<C>();
//...
}
public class C
{
//...
}
(The DataContainer class is the root class that contains all the data to be transferred. I then have a similar class structure in the Java project into which I want the data to be imported.)
My issue is that I can't seem to find a combination of a .NET and Java tools/libraries that would be able to write and read the same format.
I've tried JSON with DataContractJsonSerializer and then loading with GSON but that doesn't work with Dictionaries/Maps (works fine with Lists but I need the dictionaries). The .NET exporter uses format like
"Bs":[{"Key":1,"Value":{...}}, ...]
while it looks like GSON uses
{key:value,key:value,...}
I've also looked at some XML but since there isn't a standard formatting all the libraries use something different by default and I really don't want to write all the writing/parsing rules by hand. Also, JSON has much less overhead compared to XML.
So, is there some pair of libraries for .NET export and Java import that can handle nested classes with Lists and Dictionaries? The things I've managed to find were either in the other direction or did just simple one class serialization.
BTW: I'm also using Tuples for Dictionary keys, but if nothing can handle that I'll just use nested Dictionaries, that is not a big deal.
EDIT: Since I can't answer my own question due to low rep here is the solution that worked for me:
So, I've tried a few more combinations and serializing using JSon.NET on the C#/.NET side and then loading using GSON on the Java side works great, except Tuples. For example, using Tuple as a key for dictionary will get serialized as:
{ "(False, 3)" : { ... }, "(True, 1)" : { ... }, ... }
So I have to use String keys in Java for now but that is not an issue. Possible solution is to write a custom class for the Tuple in both .NET and Java.
Example code:
string serializeObject = JsonConvert.SerializeObject(dataContainer, new IsoDateTimeConverter());
File.WriteAllText("test.json", serializeObject);
and Java:
InputStream is = getResources().openRawResource(R.raw.test); // open from /res/raw/test.json
Gson gson = new Gson();
InputStreamReader r = new InputStreamReader(is);
DataContainer dc = gson.fromJson(r, DataContainer.class);
Log.i("jsondebug", dc.Packers.get(0).Something);
You can have a look at Google Protocol Buffers. Protocol buffers are Google's lingua franca for data. Below is an excerpt from Protocol Buffers Developer Guide:
Protocol buffers are a flexible, efficient, automated mechanism for
serializing structured data – think XML, but smaller, faster, and
simpler. You define how you want your data to be structured once, then
you can use special generated source code to easily write and read
your structured data to and from a variety of data streams and using a
variety of languages. You can even update your data structure without
breaking deployed programs that are compiled against the "old" format.
Protocol buffers have many advantages over XML for serializing
structured data. Protocol buffers:
are simpler
are 3 to 10 times smaller
are 20 to 100 times faster
are less ambiguous
generate data access classes that are easier to use programmatically
An XML mapping would work just fine. Serialize as XML and send it over. You'll need marshallers and unmarshallers on either end.
Java has its JAXB standard for binding XML to objects. Perhaps you can manage it if you follow that standard.
I think you need to approach this from a different perspective. Think first about a JSON or XML format that will work well with both the C# and Java side. Think of that as an API or contract between the C# and Android apps. Instead of using libraries that marshall/unmarshall according to their own formats, make the code on each side read/write to the contract you defined.
Alternatively, you might want to look into using a binary format like Apache Thrift or Protocol Buffers.
I have to send information too a third party in an XML format they have specified, a very common task I'm sure.
I have set of XSD files and, using XSD.exe, I have created a set of types. To generate the XML I map the values from the types within my domain to the 3rd party types:
public ExternalBar Map(InternalFoo foo) {
var bar = new ExternalBar;
bar.GivenName = foo.FirstName;
bar.FamilyName = foo.LastName;
return bar;
}
I will then use the XMLSerializer to generate the files, probably checking them against the XSD before releasing them.
This method is very manual though and I wonder if there is a better way using the Framework or external tools to map the data and create the files.
LINQ to XML works quite well for this... e.g.
XElement results = new XElement("ExternalFoos",
from f in internalFoos
select new XElement("ExternalFoo", new XAttribute[] {
new XAttribute("GivenName", f.FirstName),
new XAttribute("FamilyName", f.LastName) } ));
Firstly, I'm assuming that the object properties in your existing domain map to the 3rd party types without much manipulation, except for the repetitive property assignments.
So I'd recommend just using standard XML serialization of your domain tree (generate an outbound schema for your classes using XSD), then post-processing the result via a set of XSLT stylesheets. Then after post-processing, validate the resulting XML documents against the 3rd party schemas.
It'll probably be more complicated than that, because it really depends on the complexity of the mapping between the object domains, but this is a method that I've used successfully in the past.
As far as GUI tools are concerned I've heard (but not used myself) that Stylus Studio is pretty good for schema-to-schema mappings (screenshot here).