Develop Reference

Serialization of objects (C#)

I am trying to understand serialization and the way data is passed between the client and server, but I am still not clear.
Is converting objects to JSON strings (or XML/something else) what serialization actually stands for? If not, what is the link between the two?
(the language I am using to learn is c#, if it matters)
Thanks!

I would define serialization more broadly; as the means to which an object (usually only represented by memory) is translated into an easily transferrable format. Those formats include:
JSON
XML
Binary
FlatBuffer
ProtoBuf
And many, many others.
Deserialization is then the means by which that format is translated back into an object in memory.

Most painless way to write structured data in python and read in C#

I have a bunch of python objects with fields containing arrays of various dimensions and data types (ints or floats) and I want to write this to a file which I can read into C# objects elsewhere. I'm probably just going to write an XML file, but I thought there might be a quicker/easier way saving and reading it. Also, the resulting XML file will be rather large, which I would like to avoid if it is not too much hassle.
Is there a tried and tested file format that is compatible (and simple to use) with both languages?

What you are trying to do is called serialization. JSON is an excellent option for doing this with support in both languages.

Because you are working with floats etc.. I would consider looking at a format like BSON - "BSON is a binary format in which zero or more key/value pairs are stored as a single entity." It allows you to specify types, sizes etc...
http://bsonspec.org/#/specification
There are libraries for python, C# etc....
There are a heap of other compact easier to use than xml formats like bson out there. I only suggested this particular one as it was the first I remembered.

Deserializing C# Binary in Java

I have a system where a serialized file is created with a C# program and then deserialized in another C# program. I'm wondering if it's possible to do binary deserialization of a C# file in Java?
Thanks

You can try using some serializator that has implementations for both platforms and outputs data in a platform-independet format, like Protobuf.
Or if you need a full RPC over network between Java and C# application, you can go for Apache Thrift.

I assume you are speaking of an object serialized with BinaryFormatter. The answer then is a qualified "yes," since Java implements a Turing machine. However, this is will not be straightforward.
In this case the data will be in a format most suitable for consumption by a .NET runtime, and will contain information about .NET types and assemblies. You would have to implement your own reader for this format, and then have some way to map between .NET and Java types. (The Mono project implements a BinaryFormatter compatible with .NET's, so you could use their reader implementation as a reference.)
As an alternative, consider using another format for data serialization, such as JSON. This will give you instant portability to a wide array of languages, as well as the possibility for easy human inspection of the data.

Deserializing an object in Java which was serialized with C#'s built-in binary serialization would you'd to implement C#'s deserialization logic in java. That's a pretty involved process, so let's compare some options:
Use a third party library for serialization which works for C# and Java.
Write a routine to serialize each object. One in C#, one in Java. This will be tedious, and hard to maintain.
Implement C#'s serialization logic in Java, or vice versa. This will be difficult, time consuming, and you likely won't get it right the first time.
I recommend option 1, use a third-party library. Here's two third-party libraries I've used and highly suggest.
Google ProtoBufs
Apache Thrift

You can use any cross-platform binary format. Your options include, among others:
Protobuf
BSON (Binary JSON)
GZIP
JSON and XML (herrrrp) are also options, albeit text-based ones.
One other option would be to base64-encode the data, and decode it on the other side; albeit you may get a huge payload because it's binary (probably not a good idea).

How I can deserialize python pickles in C#?

I have some python data, serialized to pickles and need to use it in C# program. So is there any way to deserialize python pickles in C#? I can't change data format to JSON or etc.

You say you can't change the program that generates the pickle. But surely you can write a separate Python program to read the pickle and write it out again as JSON?
import json, pickle
with open("data.pickle", "rb") as fpick:
with open("data.json", "w") as fjson:
json.dump(pickle.load(fpick), fjson)

Quote from the documentation:
The data format used by pickle is Python-specific. This has the
advantage that there are no restrictions imposed by external standards
such as XDR (which can’t represent pointer sharing); however it means
that non-Python programs may not be able to reconstruct pickled Python
objects.
So the answer to your question is no, you cannot deserialize it in C#. You will have to use an interoperable format such as XML or JSON if you need to communicate with other platforms.

You can try embedding IronPython and unpickling from there, then making the unpickled object available to the C# application.
Note that pickles are designed to serialize Python objects, so this approach only works if you have very simple objects with clear mappings to C# equivalents. It also requires that your IronPython environment have access to all modules defining the classes of all objects contained in the pickle (same as in CPython).
You should try to serialize your data some other more interoperable way (such as JSON or XML) if possible.

Pyrolite has an Unpickler class that will turn a pickle into an object.

There is now a NuGet Razorvine.Pickle, for serializing and deserializing pickle files in .NET.

Deserializing a java serialized file in C#

I have a have a java project that serializes some objects and ints to a file with functions like
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeInt(RANK_SIZE);
oos.writeObject(_firstArray);
oos.writeObject(_level[3]);
oos.writeObject(_level[4]);
...
Now I have trouble deserializing that file with C# (tried the BinaryFormatter) as it apparently can only deserialize the whole file into a single object (or arrays, but I have different objects with different lenghts).
I tried first to port the generation of these files to C#, but failed miserably. These files are small and I don't must to generate them myself.
Do I need to alter the way those files are generated in Java or can I deserialize it in any way?

There is nothing in the standard .NET framework which knows how to deserialize Java objects. You could, in theory, use the Java serialization spec and write your own deserialization code in C#. But it would be a large and complex project, and I don't think you'd find many customers interested in using it.
Far, far easier would be to change the way you serialize the data to use a portable format: JSON, or some form of XML. There are both Java and C# libraries to deal with such formats and the total effort would be orders of magnitude less. I would vastly prefer this second approach.

Maybe an at first sight counterintuitive solution might be to serialize them to some commonly understandable format like JSON? I'm not even sure the binary format for serialized objects in Java is guaranteed to remain unchanged between Java versions....
Jackson is my personal favorite when it comes to Java JSON libraries.

http://www.ikvm.net/ IKVM can do it perfectly.