Trying to implement avro serialization and deserialization in .net for kafka messages. Message model as follows.
public class SampleMessage
{
public string Hash{ get; set; }
public string FileName { get; set; }
public string Data { get; set; }
public Dictionary<string, string> Content { get; set; }
public string LineDetails { get; set; }
}
So, Is it possible to create Avro schema for Dictionary in c#?
Yes. For that, you should use the complex type "map" (https://avro.apache.org/docs/current/spec.html#Maps
Maps use the type name "map" and support one attribute:
values: the schema of the map's values.
Map keys are assumed to be strings.
Given the example, your .avsc would look something like this:
{
"type": "record",
"name": "SampleMessage",
"namespace": "samplemessage",
"fields": [
{
"name": "hash",
"type": "string"
},
{
"name": "fileName",
"type": "string"
},
{
"name": "data",
"type": "string"
},
{
"name": "content",
"type": {
"type": "map",
"values": "string"
}
},
{
"name": "lineDetails",
"type": "string"
}
]
}
Related
I have this JSON string that should be posted from JavaScript to the API:
"model": "kpi.availability",
"typeId": "kpi.availability",
"name": "Availability",
"description": "",
"version": "1.0.0",
"properties": {
"X": {
"dataType": "string",
"value": ""
},
"Y": {
"dataType": "number",
"value": 0,
"isMandatory": true
},
"Z": {
"dataType": "number",
"value": 0,
"isMandatory": true
}
}
here we have 3 properties, just for instance, but it can be more than 3 with different names.
And have this C# model which doesn't work
public class KPIType{
public string model { get; set; }
public string typeId { get; set; }
public string name { get; set; }
public string description { get; set; }
public int version { get; set; }
public IDictionary<string, PropertyItem>[] properties { get; set; }
//public List<IDictionary<string, PropertyItem>> properties { get; set; } //Didn't work
}
public class PropertyItem {
public string dataType { get; set; }
public string value { get; set; }
public bool isMandatory { get; set; }
}
But when trying to send it to the backend, it fails at the client side and I'm getting this error:
"The JSON value could not be converted to System.Collections.Generic.IDictionary`2[System.String,ABB.Advanced.Services.Management.Controllers.PropertyItem][]. Path: $.kpiType.properties | LineNumber: 0 | BytePositionInLine: 294."
This JSON is incorrect in array section. It should be like this:
{
"model": "kpi.availability",
"typeId": "kpi.availability",
"name": "Availability",
"description": "",
"version": "1.0.0",
"properties": [
{
"dataType": "string",
"value": "",
"isMandatory": true
},
{
"dataType": "number",
"value": 0,
"isMandatory": true
},
{
"dataType": "number",
"value": 0,
"isMandatory": true
}
]
}
If you want to pass objects with names into property array then you need to add "name" property to the object and then find it in your service by the name.
Exempli gratia:
"properties": [
{
"name": "A",
"dataType": "string",
"value": "",
"isMandatory": true
},
{
"name": "B",
"dataType": "number",
"value": 0,
"isMandatory": true
},
{
"name": "C",
"dataType": "number",
"value": 0,
"isMandatory": true
}
]
Two things are obvious mismatch:
Properties in C# is array, but in JSON file it is an object. Not sure what's the best to fix it, maybe best change the JSON file itself, if possible
isMandatory is bool, that is, required. However, in X it is missing. If it is optional - change it to bool?
Just remove the "[]" in front of the field "properties":
I mean use this:
public IDictionary<string, PropertyItem>[] properties { get; set; }
Replace it with:
public IDictionary<string, PropertyItem> Properties { get; set; }
I highly recommend you use a better naming convention on your class fields according to the Microsoft official
docs
NOTE: If your JSON deserialization is case sensitive you can use the [JsonProperty] attribute on your props like this:
[JsonProperty("properties")
public IDictionary<string, PropertyItem> Properties { get; set; }
public class PropertyItem
{
[JsonProperty("dataType")]
public string DataType { get; set; }
[JsonProperty("value")]
public string Value { get; set; }
[JsonProperty("isMandatory")]
public bool IsMandatory { get; set; }
}
I have this JSON:
[
{
"Attributes": [
{
"Key": "Name",
"Value": {
"Value": "Acc 1",
"Values": [
"Acc 1"
]
}
},
{
"Key": "Id",
"Value": {
"Value": "1",
"Values": [
"1"
]
}
}
],
"Name": "account",
"Id": "1"
},
{
"Attributes": [
{
"Key": "Name",
"Value": {
"Value": "Acc 2",
"Values": [
"Acc 2"
]
}
},
{
"Key": "Id",
"Value": {
"Value": "2",
"Values": [
"2"
]
}
}
],
"Name": "account",
"Id": "2"
},
{
"Attributes": [
{
"Key": "Name",
"Value": {
"Value": "Acc 3",
"Values": [
"Acc 3"
]
}
},
{
"Key": "Id",
"Value": {
"Value": "3",
"Values": [
"3"
]
}
}
],
"Name": "account",
"Id": "2"
}
]
And I have these classes:
public class RetrieveMultipleResponse
{
public List<Attribute> Attributes { get; set; }
public string Name { get; set; }
public string Id { get; set; }
}
public class Value
{
[JsonProperty("Value")]
public string value { get; set; }
public List<string> Values { get; set; }
}
public class Attribute
{
public string Key { get; set; }
public Value Value { get; set; }
}
I am trying to deserialize the above JSON using the code below:
var objResponse1 = JsonConvert.DeserializeObject<RetrieveMultipleResponse>(JsonStr);
but I am getting this error:
Cannot deserialize the current JSON array (e.g. [1,2,3]) into type
'test.Model.RetrieveMultipleResponse' because the type requires a JSON
object (e.g. {"name":"value"}) to deserialize correctly. To fix this
error either change the JSON to a JSON object (e.g. {"name":"value"})
or change the deserialized type to an array or a type that implements
a collection interface (e.g. ICollection, IList) like List that can
be deserialized from a JSON array. JsonArrayAttribute can also be
added to the type to force it to deserialize from a JSON array. Path
'', line 1, position 1.
Your json string is wrapped within square brackets ([]), hence it is interpreted as array instead of single RetrieveMultipleResponse object. Therefore, you need to deserialize it to type collection of RetrieveMultipleResponse, for example :
var objResponse1 =
JsonConvert.DeserializeObject<List<RetrieveMultipleResponse>>(JsonStr);
If one wants to support Generics (in an extension method) this is the pattern...
public static List<T> Deserialize<T>(this string SerializedJSONString)
{
var stuff = JsonConvert.DeserializeObject<List<T>>(SerializedJSONString);
return stuff;
}
It is used like this:
var rc = new MyHttpClient(URL);
//This response is the JSON Array (see posts above)
var response = rc.SendRequest();
var data = response.Deserialize<MyClassType>();
MyClassType looks like this (must match name value pairs of JSON array)
[JsonObject(MemberSerialization = MemberSerialization.OptIn)]
public class MyClassType
{
[JsonProperty(PropertyName = "Id")]
public string Id { get; set; }
[JsonProperty(PropertyName = "Name")]
public string Name { get; set; }
[JsonProperty(PropertyName = "Description")]
public string Description { get; set; }
[JsonProperty(PropertyName = "Manager")]
public string Manager { get; set; }
[JsonProperty(PropertyName = "LastUpdate")]
public DateTime LastUpdate { get; set; }
}
Use NUGET to download Newtonsoft.Json add a reference where needed...
using Newtonsoft.Json;
Can't add a comment to the solution but that didn't work for me. The solution that worked for me was to use:
var des = (MyClass)Newtonsoft.Json.JsonConvert.DeserializeObject(response, typeof(MyClass));
return des.data.Count.ToString();
Deserializing JSON array into strongly typed .NET object
Use this, FrontData is JSON string:
var objResponse1 = JsonConvert.DeserializeObject<List<DataTransfer>>(FrontData);
and extract list:
var a = objResponse1[0];
var b = a.CustomerData;
To extract the first element (Key) try this method and it will be the same for the others :
using (var httpClient = new HttpClient())
{
using (var response = await httpClient.GetAsync("Your URL"))
{
var apiResponse = await response.Content.ReadAsStringAsync();
var list = JObject.Parse(apiResponse)["Attributes"].Select(el => new { Key= (string)el["Key"] }).ToList();
var Keys= list.Select(p => p.Key).ToList();
}
}
var objResponse1 =
JsonConvert.DeserializeObject<List<RetrieveMultipleResponse>>(JsonStr);
worked!
I got a JSON-File, that contains a various number of addresses.
Now I need a JSON-Schema to validate the complete JSON-File:
My JSON-File is looking similar to this:
[
{
"ID": "FIRST",
"Name" : "FirstName",
"ZIP" : "0815"
},
{
"ID": "SECOND",
"Name" : "SecondName",
"ZIP" : "4711"
},
{
"ID": "THIRD",
"Name" : "ThirdName",
"ZIP" : "123"
}
]
These are only three addresses, but the JSON-File can contain much more of them.
How has the Schema to look like?
I want to use the Schema-Generator to create the JSON-Schema!!!
Thanx for your help in advance.
Using newtonsoft Json.net.
You can generate the Schema from an object. doc
Your address class is simply :
public class Address
{
public string ID { get; set; }
public string Name { get; set; }
public string ZIP { get; set; }
}
The convertion will be :
using Newtonsoft.Json;
using Newtonsoft.Json.Schema;
JsonSchemaGenerator generator = new JsonSchemaGenerator();
JsonSchema schema = generator.Generate(typeof(Address[]));
Live Demo
You can use NJsonSchema Library (accessible via Nuget Packages) for generating schema from Json directly. For example,
var schema = JsonSchema.FromSampleJson(json);
var result = schema.ToJson();
Sample Output
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "array",
"items": {
"$ref": "#/definitions/Anonymous"
},
"definitions": {
"Anonymous": {
"type": "object",
"properties": {
"ID": {
"type": "string"
},
"Name": {
"type": "string"
},
"ZIP": {
"type": "string"
}
}
}
}
}
I am trying to parse a bunch of XML files into a single JSON file, which is already working.
The final JSON file looks like the following:
{
"items": [
{
"subItems": [
{
"Name": "Name",
"Value": "Value",
"Type": "text"
},
{
"Name": "Name",
"Value": "Value",
"Type": "text"
}
]
},
{
"subItems": [
{
"Name": "Name",
"Value": "Value",
"Type": "text"
},
{
"Name": "Name",
"Value": "Value",
"Type": "text"
},
...
Instead, I want to achieve the following structure:
{
"items": [
[
{
"Name": "Name",
"Value": "Value",
"Type": "text"
},
{
"Name": "Name",
"Value": "Value",
"Type": "text"
}
],
[
{
"Name": "Name",
"Value": "Value",
"Type": "text"
},
{
"Name": "Name",
"Value": "Value",
"Type": "text"
}
]
]
}
But I don't know how to define my objects in order to do so, my current structure is as follows:
public class Items
{
public List<Item> items;
}
public class Item
{
public List<SubItem> subItems;
}
public class SubItem
{
public string Name { get; set; }
public string Value { get; set; }
public string Type { get; set; }
}
How should I do this?
The answer is simple: turn your objects into lists: This will remove the prop names (and object notation in json).
public class Items
{
public List<Item> items; //list with prop name 'items'
}
public class Item : List<SubItem> // list in list like json notation
{
}
public class SubItem // Object in the list in list
{
public string Name { get; set; }
public string Value { get; set; }
public string Type { get; set; }
}
As #FlilipCordas noted Inheriting from list is bad practice (for good reason)
you are better of this way:
public class Items
{
public List<List<SubItem>> items; //list with list with prop name 'items'
}
public class SubItem // Object in the list in list
{
public string Name { get; set; }
public string Value { get; set; }
public string Type { get; set; }
}
Copy your json then go in Visual Studio.
Click on "Edit" > "Paste Special" > "Paste JSON as Classes"
All the classes are automatically created. I hope this tip could help you.
I have controller that recieves object of SearchProfilesModel. It consists of array of SearchProfiles and several properties in base classes.
Method:
[HttpGet]
[ResponseType(typeof(List<UserSearchResult>))]
[Route("SearchWithParams")]
public async Task<HttpResponseMessage>
SearchWithParams([FromUri] SearchProfilesModel model)
{
// Some logic
return Request.CreateResponse(HttpStatusCode.OK, result);
}
Model class:
public class SearchProfilesModel : LoginRequiredModel
{
[Required]
[JsonProperty("search_profiles")]
public List<SearchProfileViewModel> ProfilesList { get; set; }
}
List item class:
public class SearchProfileViewModel
{
[JsonProperty("name")]
public string Name { get; set; }
[JsonProperty("surname")]
public string Surname { get; set; }
[JsonProperty("facebook_id")]
public string FacebookId { get; set; }
[JsonProperty("email")]
public string Email { get; set; }
}
I`m using Swagger ui (swashbuckle lib) to test my controllers, but when it generates help page for this method it shows following:
Help page
My question is - can i somehow show structure of list item (from SearchProfileViewModel) in parameters? And if i can - how?
P.S. - Sorry for my English, it`s not my native language.
The structure of your object is in the generated doc, take a look at mine:
http://swashbuckletest.azurewebsites.net/swagger/docs/v1
{
"swagger": "2.0",
"info": {...},
"host": "swashbuckletest.azurewebsites.net",
"schemes": [...],
"paths": {...},
"definitions": {
"Data": {
"type": "object",
"properties": {
"integ": {
"format": "int32",
"type": "integer",
"example": 123
},
"doub": {
"format": "double",
"type": "number",
"example": 9858.216
},
"boolea": {
"type": "boolean"
},
"guid": {
"format": "uuid",
"type": "string",
"example": "f5849915-43c8-434c-92a7-7383d1acb631"
},
"date": {
"format": "date-time",
"type": "string"
}
}
},
...