I need to write a dynamically typed system in a statically typed language using the System.Text.Json lib in .net core 3.1. What I need is to deserialize files into a Python like dict. I have to use C# for this because we can't get python to authenticate against our systems correctly. And C# is our default lang, so...
In the real world, these documents are in CosmosDB. This program takes in a configuration file that points to specific fields in document structures and it has to inspect those fields, maybe update them, then write the result back to the CosmosDB. Currently, I'm building a system that points to the specific field. Since the system has no clue what the structure of the doc is that it's getting back from the cosmos, it can't use formal models. I am currently working on the system that inspects or updates the fields using static files in the test suite. I'll worry about cosmos DB when this is finished.
Take this file for example:
{
"azureSql": {
"databaseName": "ordersdb",
"tables": [
{
"tableName": "mytable",
"columnNames": [
"column1",
"column2"
]
}
]
},
"cosmosDb": {
"databaseName": "CosmosDb",
"collections": [
{
"collectionName": "TestCollection",
"fieldNames": [
"MyStatus.json.path.to.field"
]
}
]
}
}
files like this get read in. The input can be virtually any type of legit json. I need to put that into a Dictionary that I can access similar to Python mydict["cosmosDb"]["collections"][0]["fieldNames"]
The line:
_entities = JsonSerializer.Deserialize<Dictionary<string, object>>(jsonString);
doesn't work because it only serializes the first level. The rest are json entry types.
I found this C# way to mimic Python Dictionary Syntax
and its close, but it only supports string and doesn't work in the Deserialize method.
Any ideas on how to solve this?
I suggest you to use dynamic
var entities = JsonSerializer.Deserialize<dynamic>(jsonString)
Would the following classes structure work?
public class Rootobject
{
public Azuresql azureSql { get; set; }
public Cosmosdb cosmosDb { get; set; }
}
public class Azuresql
{
public string databaseName { get; set; }
public List<Table> tables { get; set; }
}
public class Table
{
public string tableName { get; set; }
public List<string> columnNames { get; set; }
}
public class Cosmosdb
{
public string databaseName { get; set; }
public List<Collection> collections { get; set; }
}
public class Collection
{
public string collectionName { get; set; }
public List<string> fieldNames { get; set; }
}
Edit:
This works:
using Newtonsoft.Json.Linq;
var result = JObject.Parse(jsonFile);
var detail = result["cosmosDb"]["collections"][0]["collectionName"];
Related
I am building chunk of code(generic) which will return list of property and values from any structure of object (either list or single object or list within single object or single object within list or list within list or single object within object) in json based on input request.
Let me explain you the use case- I am getting list of properties in accordance with class name and I need to send back its values in json in list format.
For e.g. I am getting input request for following properties SPTeam.Name, SPRole.Name, SPTeam.AltName1, Client.Name and I want to process data retrieved from DB and send its values in list format in json like,
{
"SPTeam":[
{
"Name": "AUBERT & DUVAL Development",
"AltName1": "AUBERT & DUVAL Development",
"SPRole" : [
{
"Name": "Charter Full Stack Developer II"
},
{
"Name": "General Consulting Engineer"
}]
},
{
"Name": "SmartHands_Dedicated_Team",
"AltName1": "SmartHands_Dedicated_TeamAltName1",
"SPRole" : [
{
"Name": "Accountant"
},
{
"Name": "Senior accountant"
}]
}],
"Client" : [
{
"Name": "Davita Medical Group"
}
]}
The data model or the structure from which I need to get properties and values is-
Public class SPTeam
{
public int id { get; set; }
public string Name { get; set; }
public string AltName1 { get; set; }
public string AltName2 { get; set; }
public List<SPRole> SPRole { get; set; }
}
Public class SPRole
{
public int id { get; set; }
public string Name { get; set; }
public int AvgCostHr { get; set; }
public int SellPriceHr { get; set; }
public int GPPct { get; set; }
}
Public class Client
{
public int id { get; set; }
public string Name { get; set; }
}
The data model or structure and its relationship with other classes is dynamic (not any fixed class) and I need to retrieve data only for specific properties maintaining association/relationships between them.
Any help would be appreciated...
Thanks,
Well, it seems that a core portion of your question may be resolved by the following:
How to create json structure from list of data?
However, in your case I'm not sure what level of generic capability you are requiring. You mention that "The data model or structure and its relationship with other classes is DYNAMIC (not any fixed class)" on the one hand, then you mention: "I need to retrieve data only for SPECIFIC PROPERTIES maintaining association/relationships between them".
What is the level of generic capability between, what seems to me, to be conflicting requirements? If you only want to retrieve specific properties, then where are you tracking the properties (and associated class structures) you want to retrieve?
I believe the answer you're looking for will depend on how you answer this.
I am trying to store json object from my API project to CosmosDB using Microsoft.Azure.Cosmos.Table.
My Json looks like this:
{
"age": 0,
"type": "network-error",
"url": "https://www.example.com/",
"body": {
"sampling_fraction": 0.5,
"referrer": "http://example.com/",
"server_ip": "2001:DB8:0:0:0:0:0:42",
"protocol": "h2",
"method": "GET",
"request_headers": {},
"response_headers": {},
"status_code": 200,
"elapsed_time": 823,
"phase": "application",
"type": "http.protocol.error"
}
}
And my Cosmos is of type Table API hence I need to flatten it but couldn't find out a way to do it in a most generic way, MSDN documentation doesn't have much about implementation.
Thanks much, appreciate your time and help.
Create a .Net class for your model and pass that to TableEntity.Flatten. You can then use the returned Dictionary of flattened key and EntityProperty value pairs to create a DynamicTableEntity class by specifying the partition and row keys. And you can insert that to cosmos db table storage.
Usage should be similar to how it was used in Table Storage api. Have a look at the unit tests here:
https://github.com/Azure/azure-storage-net/commit/daff940a506b27650901b9b9feb59131ffacce6d
ShapeEntity shapeEntity = new ShapeEntity(Guid.NewGuid().ToString(), Guid.NewGuid().ToString(), "square", 4, 4);
OperationContext operationContext = new OperationContext();
DynamicTableEntity dynamicTableEntity = new DynamicTableEntity(shapeEntity.PartitionKey, shapeEntity.RowKey)
{
Properties = TableEntity.Flatten(shapeEntity, operationContext)
};
and you can write this DynamicTableEntity into Cosmos DB using the SDK.
To read the flattened DynamicTableEntity back into the complex object, when you read it, you need to use TableEntity.ConvertBack api.
Based on the document you posted above, it will be more appropriate to use the SQL API/Mongo API to store the data in cosmosdb as the document looks sort of JSON.
As you mentioned in the comment if other tables are in relational way , i would recommend you to go through data modelling and design your tables
if you still want to store it in table storage, convert your json to a model and serialize it before inserting using Flatten
public class RequestHeaders
{
}
public class ResponseHeaders
{
}
public class Body
{
public double sampling_fraction { get; set; }
public string referrer { get; set; }
public string server_ip { get; set; }
public string protocol { get; set; }
public string method { get; set; }
public RequestHeaders request_headers { get; set; }
public ResponseHeaders response_headers { get; set; }
public int status_code { get; set; }
public int elapsed_time { get; set; }
public string phase { get; set; }
public string type { get; set; }
}
public class RootObject
{
public int age { get; set; }
public string type { get; set; }
public string url { get; set; }
public Body body { get; set; }
}
I'm creating a little system to let me see who is in my chat on Twitch using their JSON API. However, while I successfully got the information, I can't figure out how to parse it correctly.
This is the string that is being produced:
{
"_links": {},
"chatter_count": 1,
"chatters": {
"moderators": [
"teonnyn"
],
"staff": [],
"admins": [],
"global_mods": [],
"viewers": []
}
}
This is the object I created to deserialize it to, but I have no idea for sure if it's exactly correct:
public class users
{
public string[] links;
public int chatter_count;
public string[] moderators;
public string[] staff;
public string[] admins;
public string[] global_mods;
public string[] viewers;
}
I'm using Newtonsoft.JSON to parse it - which would be the correct way to push the string to the "users" object?
No, the C# class you have doesn't really correlate correctly to the JSON:
Your links member doesn't match the name _links in JSON.
_links is defined as an array, but should be an object - it's {} in JSON, not [].
Likewise chatters, which should be a custom class as well.
Starting with Visual Studio 2013 Update 2, you can generate a C# class from a JSON sample. This is what it generated for your JSON:
public class Rootobject
{
public _Links _links { get; set; }
public int chatter_count { get; set; }
public Chatters chatters { get; set; }
}
public class _Links
{
}
public class Chatters
{
public string[] moderators { get; set; }
public object[] staff { get; set; }
public object[] admins { get; set; }
public object[] global_mods { get; set; }
public object[] viewers { get; set; }
}
As you can see, it maps moderators properly to a string[] but gets a bit confused and uses object[] for the rest, because the snippet contains to data for it to base the type on.
If you can get a JSON sample with more data - ideally, with every field being present and having representative data - you'll get the best mapping.
Also, you should change Rootobject to your own class name, of course. User or TwitchUser should do it.
Once you have a class that corresponds correctly to your JSON, using JSON.NET to parse it is very simple:
Rootobject yourData = JsonConvert.DeserializeObject<Rootobject>(inputJsonString);
And you're done.
How to list all minecraft profiles by launcher_profiles.json file?
I tried to use the site json2csharp.com, but unfortunately when it generated the class ready code he has returned all the profiles as if it were also a class.
for example:
I used this simple code minecraft profile file ...
{
"profiles": {
"1.7.10": {
"name": "1.7.10",
"lastVersionId": "1.7.10"
}
},
"selectedProfile": "1.7.10"
}
But when I send the site to convert C# it returns this:
public class __invalid_type__1710
{
public string name { get; set; }
public string lastVersionId { get; set; }
}
public class Profiles
{
public __invalid_type__1710 __invalid_name__1.7.10 { get; set; }
}
public class RootObject
{
public Profiles profiles { get; set; }
public string selectedProfile { get; set; }
}
See for yourself: Json2CSharp
Have you any way I can read the launcher_profiles.json file minecraft using Newtonsoft.Json.Linq?
While useful in many cases, json2csharp.com is not foolproof. As you've seen, it does not handle cases where key names are dynamic or otherwise cannot be converted into valid C# identifiers. In these cases you will need to make manual adjustments to the generated classes. For example, you can use a Dictionary<string, Profile> in place of a static class to handle the dynamic keys of the profiles object.
Define your classes like this:
public class RootObject
{
public Dictionary<string, Profile> profiles { get; set; }
public string selectedProfile { get; set; }
}
public class Profile
{
public string name { get; set; }
public string lastVersionId { get; set; }
}
You can then deserialize into the RootObject class using either JavaScriptSerializer or Json.Net, whichever you prefer.
Here is a fiddle using Json.Net: https://dotnetfiddle.net/ZlEK63
So the problem may be that the launcher_profiles.json is not really kosher JSON.
Put this into Json2CSharp to see what I mean:
{
"profiles": [
{
"name": "1.7.10",
"lastVersionId": "1.7.10"
}
],
"selectedProfile": "1.7.10"
}
The difference here is that I've redefined the profiles node to correctly represent a collection (array) that's mapped to a generic list in C#.
You may need to manually parse that file as JSON.Net or other options will not be able to work with the invalid json format.
I generally don't work with the Linq versions of the Json.Net library, but I've come up with a simple example of how to get all of the names of the profiles (you can't serialize to a class with the given format).
class Program
{
//Add another "profile" to show this works with more than one
private static String json = "{ \"profiles\": { \"1.7.10\": { \"name\": \"1.7.10\", \"lastVersionId\": \"1.7.10\" }, \"1.7.11\": { \"name\": \"1.7.11\", \"lastVersionId\": \"1.7.11\" } }, \"selectedProfile\": \"1.7.10\" }";
static void Main(string[] args)
{
//Parse to JObject
var obj = Newtonsoft.Json.Linq.JObject.Parse(json);
foreach (var profile in obj["profiles"])
{
foreach (var child in profile.Children())
{
Console.WriteLine(child["name"]);
}
}
}
}
This is a JSON message I get from server (which I can't change). There might be many more objects (time / value) returned, but in this case there is only one. The format stays the same regardless.
{
"data": [
{
"time": "2014-12-12T13:52:43",
"value": 255.0
}
]
}
I'm trying to deserialize the JSON to a very simple C# object.
public class Dataentry {
public float Value { get; set; }
public DateTime Time { get; set; }
}
I've tried deserialization with Newtonsoft's JSON.Net and RestSharp libraries with no success. The following code doesn't work, but neither does anything else I've tried :-) I get no error -- just an empty object with default initial values.
var myObject = JsonConvert.DeserializeObject<Dataentry> (jsonString);
Since those libraries are not very intuitive or well documented in this kind of case, I'm lost. Is this kind of JSON impossible to deserialize? I really would like to use a ready-made library, so any help would be appreciated.
This is not working because your JSON is specifying a collection and you are trying to deseralize into one object. There are plenty of json to c# class generators you can paste json into to get an appropriate class definition(s) one such generator is located here
A more appropriate definition would be
public class Datum
{
public string time { get; set; }
public double value { get; set; }
}
public class RootObject
{
public List<Datum> data { get; set; }
}
Then deseralize as
var myObject = JsonConvert.DeserializeObject<RootObject> (jsonString);
I'd like add some extra explanetion to your question...
You write I'm trying to deserialize the JSON to a very simple C# object. - unfortunatelly this is not the complete truth. What you are trying is to deserialize a collection of a very simple C# objects. The indicator for this are the square brackets in your json:
{
"data": [
{
"time": "2014-12-12T13:52:43",
"value": 255.0
}
]
}
It means that there is a class with a property named data (it can ba mapped to some other name but for the sake of simplicity let's stick to this name) and that this property is a collection type. It can be one of any types that support the IEnumerable interface.
public class DataCollection
{
public DataItem[] data { get; set; }
//public List<DataItem> data { get; set; } // This would also work.
//public HashSet<DataItem> data { get; set; } // This would work too.
}
public class DataItem
{
public float value { get; set; }
public DateTime time { get; set; } // This would work because the time is in an ISO format I believe so json.net can parse it into DateTime.
}
The next step is to tell Json.Net how to deserialize it. Now when you know it's a complex data type you can use the type that describes the json structure for deserialization:
var dataCollection = JsonConvert.DeserializeObject<DataCollection>(jsonString);
If you didn't have the data property in you json string but something like this:
[
{
"time": "2014-12-12T13:52:43",
"value": 255.0
},
{
"time": "2016-12-12T13:52:43",
"value": 25.0
},
]
you could directly deserialize it as a collection:
var dataItems = JsonConvert.DeserializeObject<List<DataItem>>(jsonString);
or
var dataItems = JsonConvert.DeserializeObject<DataItem[]>(jsonString);
change your DateEntry binding Definition
public class ArrayData{
public DataEntry data {set; get;}
}
public class DataEntry {
public float Value { get; set; }
public DateTime Time { get; set; }
}
in your method now you can received an ArraData Object
be careful with datetime string values sent for correct binding