Develop Reference

Most painless way to write structured data in python and read in C#

I have a bunch of python objects with fields containing arrays of various dimensions and data types (ints or floats) and I want to write this to a file which I can read into C# objects elsewhere. I'm probably just going to write an XML file, but I thought there might be a quicker/easier way saving and reading it. Also, the resulting XML file will be rather large, which I would like to avoid if it is not too much hassle.
Is there a tried and tested file format that is compatible (and simple to use) with both languages?

What you are trying to do is called serialization. JSON is an excellent option for doing this with support in both languages.

Because you are working with floats etc.. I would consider looking at a format like BSON - "BSON is a binary format in which zero or more key/value pairs are stored as a single entity." It allows you to specify types, sizes etc...
http://bsonspec.org/#/specification
There are libraries for python, C# etc....
There are a heap of other compact easier to use than xml formats like bson out there. I only suggested this particular one as it was the first I remembered.

IS the output of binary java serialization is equals to c# serialization

I am working on a project which is made up on c# and there are some of data are serialized And now I need the same values to be serialized in java.
So, can I get the java serialized out put which should equivalent to the c# serialized out put. Because c# code is already been done I cant change the reader format. I need to send the same data by java which is currently in c#.
So, is the serialized out put of both the language are same.

So, is the serialized out put of both the language are same.
Certainly not if you use the default binary serialization mechanisms of each platform. It would be almost inconceivable that they could be compatible.
You should pick a platform-neutral serialization format, such as Protocol Buffers, Thrift, YAML, JSON, XML (with custom serializers) etc.

The binary serialization is almost always not the same, because of different serialization implementations. However thats why we have XML or Json and other inter-compatible formats, so we can use them regardless of the technology we use.

Deserializing a java serialized file in C#

I have a have a java project that serializes some objects and ints to a file with functions like
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeInt(RANK_SIZE);
oos.writeObject(_firstArray);
oos.writeObject(_level[3]);
oos.writeObject(_level[4]);
...
Now I have trouble deserializing that file with C# (tried the BinaryFormatter) as it apparently can only deserialize the whole file into a single object (or arrays, but I have different objects with different lenghts).
I tried first to port the generation of these files to C#, but failed miserably. These files are small and I don't must to generate them myself.
Do I need to alter the way those files are generated in Java or can I deserialize it in any way?

There is nothing in the standard .NET framework which knows how to deserialize Java objects. You could, in theory, use the Java serialization spec and write your own deserialization code in C#. But it would be a large and complex project, and I don't think you'd find many customers interested in using it.
Far, far easier would be to change the way you serialize the data to use a portable format: JSON, or some form of XML. There are both Java and C# libraries to deal with such formats and the total effort would be orders of magnitude less. I would vastly prefer this second approach.

Maybe an at first sight counterintuitive solution might be to serialize them to some commonly understandable format like JSON? I'm not even sure the binary format for serialized objects in Java is guaranteed to remain unchanged between Java versions....
Jackson is my personal favorite when it comes to Java JSON libraries.

http://www.ikvm.net/ IKVM can do it perfectly.

my C# client app need to deserialize complex JSON sent by Java app

If my C# client app need to deserialize complex JSON from Java server app, what is the best option I have?
Here are two conditions need to consider:
1) speed is the most important
2) Json format could include information about the Java data type, C# client app. need to recognize it and convert it to C# corespondent type. for exmaple,
...,"Variable1" : [ "java.math.BigDecimal", 0E-8 ],
"Variable2" : [ "com.xmlasia.x5.refdata.instrument.model.MarginGroup"],...
IMO because of point 2, the only way is to build my own deserializer, am I right?
Regard to point 1, if I use Json.net to deserialize the Json, and then convert to arraylist, with it have significant impact on the speed? Is there an other better way?
The disadvantage of the arraylist approach is that the extractJson method get really messy, and I think arraylist is slow.

I think that the easiest would be to build some bridge that will translate this JSON to something more interoperable.

It is unlikely that “speed is the most important”, otherwise the data would need to be send in a binary format. However all the main json parsers are also fast, so this is unlikely to be an issue.
If a parser just ignored the java data type and map fields based on names to the fields in your .net objects you may be ok. Otherwise you need a json parser that will give you back dictionary of the fields so you can process them yourself. There should be no need to write you own string processing code to decode the json, that is a solved problem.
There are lots of json libs for .net, as it is a long time since I looked at them, I can’t recommend the best one for you use.

C# - XML Deserialization vs Binary Deserialization vs Binary+Deflate

I'm saving a huge list of objects to a file, and later deserializing them. The resulting xml file can be about 3 gigs in size.
I want the Deserialization to be super fast, so i tried all three approaches (xml,binary,compressed)
Obviously, deserialization a compressed file takes much longer than an XML one. But i saw binary deserialization also taking a lot more time vs the xml deserialisation. Is that normal? Shoudn't both xml and binary take pretty much the same time to deserialize the object?
Also what do you think will be the best option to use in terms of a good balance between file size and deserialization speeds?

In this performance comparison between all sorts of serialization methods that come with .NET (BinaryFormatter, XmlSerializer, DataContractSerializer, etc.) and protobuf, the protobuf serializer seems to way ahead of the serializers that come with .NET. The resulting size appaears to be smaller as well. If the protobuf format is an option for you, I strongly recommend you have a look at it. :-)
Another option: if deserializing is slow, only deserialize the parts you really need. Create an index file that tells you the offsets of the objects you write to the data file, so you can quickly deserialize the objects you need in a random-access fashion.

Customise your serialisation, either fully or by implementing ISerializable and then using binary (though custom XML may also be worth experimenting with). Don't serialise memoised fields, but just the key field their value is based upon. Look for other areas where you can reduce size by serialising enough information to build part of the graph, rather than the full representation of the graph.
Then use deflate with that.