I have some pieces of data that are byte arrays byte[], and I need to render them as base64 in JSON, but as raw byte arrays in BSON.
How can I easily do this in JSON.Net?
So, far I have something like so:
class Data
{
public byte[] Bytes{get;set;}
}
Converting to BSON is fine, but when converting to JSON, it is of course not base64 encoded and treated as a string
Hmm, using the following code with Json.Net 6.0.1, it appears to work just as you want with no special treatment: byte arrays are converted to base-64 strings and vice versa. Are you serializing your objects in a different way, or using an old version? If not, can you provide some code that demonstrates the problem?
string s = "Foo Bar Baz Quux";
Data data = new Data
{
Bytes = Encoding.UTF8.GetBytes(s)
};
string json = JsonConvert.SerializeObject(data);
Console.WriteLine(json);
data = JsonConvert.DeserializeObject<Data>(json);
Console.WriteLine(Encoding.UTF8.GetString(data.Bytes));
Output:
{"Bytes":"Rm9vIEJhciBCYXogUXV1eA=="}
Foo Bar Baz Quux
Related
This question already has an answer here:
Newtonsoft JSON serialization for byte[] property [duplicate]
(1 answer)
Closed 2 years ago.
I have created the following test vector class:
public class TestVector
{
public UInt16 MaxBlockSize { get; }
public byte[] Payload { get; set; }
public TestVector(ushort maxBlockSize, byte[] payload)
{
MaxBlockSize = maxBlockSize;
Payload = payload;
}
}
In my test, I am populating a list of vectors defined as per:
private static HashSet<TestVector> myVectors = new HashSet<TestVector>();
And then serialising "myVectors" using JsonConvert and write the result to a file as per:
var jsonOuput = JsonConvert.SerializeObject(myVectors , new JsonSerializerSettings{ObjectCreationHandling = ObjectCreationHandling.Replace})
File.WriteAllText(#"e:\MyJson.json", jsonOuput);
Here is a Json typical output (with a list/Hashset composed of 2 vectors):
[
{
"MaxBlockSize": 256,
"Payload": "bjQSAAAAAABvNBIAAAAA..."
},
{
"MaxBlockSize": 256,
"Payload": "VjQSVzQSWDQS...."
},
]
Now what I do not get is why "Payload" is serialised as a string and not as an array.
My questions are:
What is this string format (ASCII code maybe?) and why is it used instead of a byte[] type of representation?
Is there a way to get the "Payload" byte[] to be printed in a more readable way?
What is this string format (ASCII code maybe?) and why is it used instead of a byte[] type of representation?
See json.Net documentation for primitive types:
Byte[] String (base 64 encoded)
So the format is base64. This is probably used since it is a reasonably efficient encoding of binary data, encoding 6 bits per character. Encoding values as an array would use much more space.
It is somewhat common to encode images or similar chunks of data as byte arrays. Since these can be large it is useful to keep the size down as much as possible.
Is there a way to get the "Payload" byte[] to be printed in a more readable way?
There are various base64 converters online that can convert it to hex, oct, string, or whatever format you prefer to view your binary data in. But for many applications it is not very useful since the binary data often represents something that is already serialized in some way.
I receive some JSON from a Java third-party system that contains Avro schemas in JSON format. An example looks like this:
{"type":"record", "name":"AvroRecord", "namespace":"Parent.Namespace", "fields": [{"name":"AvroField", "type":"bytes", "default":"\u00FF"}]}
I parse this JSON to do some C# code generation. The result would look like this:
public partial class AvroRecord
{
[AvroField(Name = "AvroField", Type = "bytes", DefaultValueText = "ÿ")]
public byte[] AvroField { get; set; }
public AvroRecord() { this.AvroField = new byte[] { 255 }; }
}
Eventually, from the C# representation of the schema, I need to infer back the original schema. Once I get that inferred schema, it will be sent over to the original system for comparison. That is why I want to keep the original string value for the default value, since I don't know if:
{"type":"record", "name":"AvroRecord", "namespace":"Parent.Namespace", "fields": [{"name":"AvroField", "type":"bytes", "default":"\u00FF"}]}
and
{"type":"record", "name":"AvroRecord", "namespace":"Parent.Namespace", "fields": [{"name":"AvroField", "type":"bytes", "default":"ÿ"}]}
will result in an exact match or it will have a problem.
I use JSON.NET to convert from the raw schema as a string to something more useful that I can work with:
JToken token = JToken.Parse(schema);
Is there a way in JSON.NET or any other JSON parsing library to control the parsing and copy a value without being parsed? Basically, a way to avoid "\u00FF" becoming "ÿ"
Say I have a string representing an array of objects in JSON form:
string s = "[{\"name\":\"Person1\"},{\"name\":\"Person2\"}]";
What I want is an array of strings, each string being the string representation of a JSON object - NOT the object itself. It should look something like this:
string[] s = new string[]
{
"{\"name\":\"Person1\"}",
"{\"name\":\"Person2\"}"
};
1) Almost every search I attempt pulls up millions of results on how to simply deserialize a JSON string using (eg) Json.NET. This is not what I want to do.
2) I have tried building a class representing the objects to temporarily loop through a deserialize/serialize mapping each to a string in an array, but the schema for the objects is variable (hence why I only need a string representation).
3) I have attempted a few regex to try and do this, but my JSON string can contain fields that contain JSON strings as their value (icky, but out of my control) and so nested character escaping etc drove me partially mad before I decided to beg for help here.
Surely this should be simple? Anybody got any pointers?
You'll need to deserialize it, and then serialize each object independently.
For example (using Newtonsoft.Json):
string json = "[{\"name\":\"Person1\"},{\"name\":\"Person2\"}]";
var objects = JsonConvert.DeserializeObject<List<object>>(json);
var result = objects.Select(obj => JsonConvert.SerializeObject(obj)).ToArray();
Yields (as a string[]):
{"name":"Person1"}
{"name":"Person2"}
If you try to avoid deserializing and serializing, you're almost certain to run into an edge case that will break your code.
string s = "[{\"name\":\"Person1\"},{\"name\":\"Person2\"}]";
var Json = JsonConvert.DeserializeObject<List<object>>(s);
string[] Jsonn = Json.Select(x => x.ToString()).ToArray();
[] Jsonn returns string array instead of object array with JObject formatted.
Hope this one help you.
Why don't you just use this
string s = "[{\"name\":\"Person1\"},{\"name\":\"Person2\"}]";
string[] t = s.Split(',');
I tried it. It simply gives you string array as you want it....
I am converting from XML to JSON using SerializeXmlNode. Looks the expected behavior is to convert all XML values to strings, but I'd like to emit true numeric values where appropriate.
// Input: <Type>1</Type>
string json = JsonConvert.SerializeXmlNode(node, Newtonsoft.Json.Formatting.Indented, true);
// Output: "Type": "1"
// Desired: "Type": 1
Do I need to write a custom converter to do this, or is there a way to hook into the serialization process at the appropriate points, through delegates perhaps? Or, must I write my own custom JsonConverter class to manage the transition?
Regex Hack
Given the complexity of a proper solution, here is another (which I'm not entirely proud of, but it works...).
// Convert to JSON, and remove quotes around numbers
string json = JsonConvert.SerializeXmlNode(node, Newtonsoft.Json.Formatting.Indented, true);
// HACK to force integers as numbers, not strings.
Regex rgx = new Regex("\"(\\d+)\"");
json = rgx.Replace(json, "$1");
XML does not have a way to differentiate primitive types like JSON does. Therefore, when converting XML directly to JSON, Json.Net does not know what types the values should be, short of guessing. If it always assumed that values consisting only of digits were ordinal numbers, then things like postal codes and phone numbers with leading zeros would get mangled in the conversion. It is not surprising, then, that Json.Net takes the safe road and treats all values as string.
One way to work around this issue is to deserialize your XML to an intermediate object, then serialize that to JSON. Since the intermediate object has strongly typed properties, Json.Net knows what to output. Here is an example:
class Program
{
static void Main(string[] args)
{
string xml = #"<root><ordinal>1</ordinal><postal>02345</postal></root>";
XmlSerializer xs = new XmlSerializer(typeof(Intermediary));
using (TextReader reader = new StringReader(xml))
{
Intermediary obj = (Intermediary)xs.Deserialize(reader);
string json = JsonConvert.SerializeObject(obj , Formatting.Indented);
Console.WriteLine(json);
}
}
}
[XmlRoot("root")]
public class Intermediary
{
public int ordinal { get; set; }
public string postal { get; set; }
}
Output of the above:
{
"ordinal": 1,
"postal": "02345"
}
To make a more generic solution, yes, you'd have to write your own converter. In fact, the XML-to-JSON conversion that takes place when calling SerializeXmlNode is done using an XmlNodeConverter that ships with Json.Net. This converter itself does not appear to be very extensible, but you could always use its source code as a starting point to creating your own.
I need to express a collection of about 10-15 short strings (and maybe some ints) as a fairly compact alphanumeric string - one which I can send as a parameter in a get request.
Basically, I'm thinking that my collection will be a hashtable, and I'd like to serialize it so it looks sort of like a viewstate string (only hopefully not so long!).
eg.
testpage.aspx?code=rO0ABXNyAAlTb21lQ2xhc3PSHbLk6OgfswIAA0kAAWl
and then testpage.aspx can deserialize this back to the original collection.
Is this possible?
One option here is to pick a delimiter, for example ¤; join the strings, encode them (perhaps UTF8), and pack the bytes as base-64...
string[] data = {"abc","123", "def"};
string s = string.Join("¤", data);
byte[] raw = Encoding.UTF8.GetBytes(s);
string alphaNumeric = Convert.ToBase64String(raw); // send this
(you may need to handle the few non-alphanumeric characters that base-64 uses).
And to reverse it:
raw = Convert.FromBase64String(alphaNumeric);
s = Encoding.UTF8.GetString(raw);
data = s.Split('¤');
If you want to send key/value pairs... well, the obvious choice would be query-string parameters themselves, since they are designed for this. But if you need it as a byte-stream:
var data = new DbConnectionStringBuilder();
data["foo"] = "abc";
data["bar"] = "123 + ;la";
string s = data.ConnectionString;
byte[] raw = Encoding.UTF8.GetBytes(s);
string alphaNumeric = Convert.ToBase64String(raw); // send this
raw = Convert.FromBase64String(alphaNumeric);
s = Encoding.UTF8.GetString(raw);
data.ConnectionString = s;
foreach (string key in data.Keys)
{
Console.WriteLine(key + "=" + data[key]);
}
Why don't you just serialize the data using protobuf-net and pass it through the Session? Or, if it has to be a string, just use XmlSerializer?
Personally, passing serialized data through the URL seems really bad to me!
you can serialize your dictionary/hashtable to JSON,
and then change it to base64 (just to make it a tad less visible and resolve possible usage of URL characters etc).
or you can just URLEncode it.
You can use standard .Net serialization and serialize your object to a MemoryStream. You can then read out the contents of the MemoryStream as a byte array and use Convert.ToBase64String on the array to get a string representation of it.
To deserialize it you can do the opposite.
If you are worried that the serialized object is too large, you can wrap the MemoryStream in a System.IO.Compression.DeflateStream to compress it.
Thank you all, putting it together I wrote a test console app. I was a bit disappointed that the resulting string was so long.
My example implements a DeflateStream, which for my datasize probably introduces more overhead than it saves with compression. But even without the compression it was still pretty big.
What I was hoping to achieve was to make something slightly more compact (obfuscation for the user was a plus, but not critical) - I suspect that it'd actually be better for me to use a plain old parameterized string. Maybe JSON might be ok, but I'm using ASP.net 2.0, and I don't think that I get a readybaked json serializer there.
Nonetheless, I learnt something new and interesting, so thanks for that!
static void Main(string[] args)
{
Hashtable ht1 = new Hashtable(1);
ht1.Add("name", "bob");
Console.WriteLine(ToCompactString(ht1));
Console.WriteLine();
string str = "name:bob";
Console.WriteLine(ToCompactString(str));
Console.ReadLine();
}
private static string ToCompactString(object obj)
{
var ms = new MemoryStream();
var ds = new DeflateStream(ms, CompressionMode.Compress);
var bf = new BinaryFormatter();
bf.Serialize(ds, obj);
byte[] bytes = ms.ToArray();
ds.Close();
ms.Close();
string result = Convert.ToBase64String(bytes);
return result;
}