Saving deserialized JSON objects to database with duplicate child entities - c#

I am retrieving some JSON from an API call and deserializing it into it's component objects. Everything works absolutely fine, until I come to saving to the database. The reason is, there are child objects with duplicate keys (which is absolutely correct as far as the data is concerned) but when I save the top level object, it throws a primary key violation error on the child object.
Here is a sample of my JSON (I know it is not complete);
{
"count": 149,
"filters": {},
"competitions": [
{
"id": 2006,
"area": {
"id": 2001,
"name": "Africa",
"countryCode": "AFR",
"ensignUrl": null
},
"name": "WC Qualification",
"code": null,
"emblemUrl": null,
"plan": "TIER_FOUR",
"currentSeason": {
"id": 555,
"startDate": "2019-09-04",
"endDate": "2021-11-16",
"currentMatchday": null,
"winner": null
},
"numberOfAvailableSeasons": 2,
"lastUpdated": "2018-06-04T23:54:04Z"
},
{
"id": 2025,
"area": {
"id": 2011,
"name": "Argentina",
"countryCode": "ARG",
"ensignUrl": null
},
"name": "Supercopa Argentina",
"code": null,
"emblemUrl": null,
"plan": "TIER_FOUR",
"currentSeason": {
"id": 430,
"startDate": "2019-04-04",
"endDate": "2019-04-04",
"currentMatchday": null,
"winner": null
},
"numberOfAvailableSeasons": 2,
"lastUpdated": "2019-05-03T05:08:18Z"
},
{
"id": 2023,
"area": {
"id": 2011,
"name": "Argentina",
"countryCode": "ARG",
"ensignUrl": null
},
"name": "Primera B Nacional",
"code": null,
"emblemUrl": null,
"plan": "TIER_FOUR",
"currentSeason": {
"id": 547,
"startDate": "2019-08-16",
"endDate": "2020-06-14",
"currentMatchday": 30,
"winner": null
},
"numberOfAvailableSeasons": 3,
"lastUpdated": "2020-05-15T00:00:02Z"
},
Currently I am just saving the top level object, and I expect/want all of the child objects to save too. If I turn off the primary keys on the child objects (make them identidy columns rather than their actual values), this all works fine and all of the child objects save perfectly. As you can see from the JSON, "area" 2011 is a duplicate, there are two competitions that have the same area, so data wise it is correct, but with the proper primary keys of the "area" turned on, it trips out as it is trying to insert a duplicate record.
So I understand perfectly what is going on and why it is erroring, what I want to know is, is there a simple way of just telling EF to ignore duplicate key errors. I can't add a try catch to saving the top level object as it just saves nothing when it hits the error.
I have tried saving the individual child objects, testing for their existence prior to the save, but when it tries to save the parent level object, it tries to save the child object too, leaving me with the same problem.
Here is my code for saving the top level object (cut down for simplicity);
public class Area
{
[Key, DatabaseGenerated(DatabaseGeneratedOption.None)]
public int id { get; set; }
public string name { get; set; }
public string countryCode { get; set; }
public string ensignUrl { get; set; }
}
public class Winner
{
[Key, DatabaseGenerated(DatabaseGeneratedOption.None)]
public int id { get; set; }
public string name { get; set; }
public string shortName { get; set; }
public string tla { get; set; }
public string crestUrl { get; set; }
}
public class CurrentSeason
{
[Key, DatabaseGenerated(DatabaseGeneratedOption.None)]
public int id { get; set; }
public string startDate { get; set; }
public string endDate { get; set; }
public int? currentMatchday { get; set; }
public Winner winner { get; set; }
}
public class Competition
{
[Key, DatabaseGenerated(DatabaseGeneratedOption.None)]
public int id { get; set; }
public Area area { get; set; }
public string name { get; set; }
public string code { get; set; }
public string emblemUrl { get; set; }
public string plan { get; set; }
public CurrentSeason currentSeason { get; set; }
public int numberOfAvailableSeasons { get; set; }
public DateTime lastUpdated { get; set; }
}
public class Example
{
[Key, DatabaseGenerated(DatabaseGeneratedOption.None)]
public int count { get; set; }
public IList<Competition> competitions { get; set; }
}
static void Main(string[] args)
{
string json = GET(#"http://my.url.com/api/stuff");
Example example = JsonConvert.DeserializeObject<Example>(json);
using(var db = new ExampleContext())
{
db.Examples.Add(example);
db.SaveChanges();
}
}
Thanks in anticipation.

Unfortunately there isn't any straight way to overcome your problem.
EF Change Tracker tracks entities by their reference and the only way to solve your problem is to create same object for all identical areas.
For this you have two choices:
1- Loop over example after this line
Example example = JsonConvert.DeserializeObject<Example>(json);
and find all identical areas and replace all with one of them.
2- Use PreserveReferencesHandling feature of NewtonSoft. But it needs to apply on both Serialize and Deserialize sides:
Server(Api) side:
string json = JsonConvert.SerializeObject(data, Formatting.Indented,
new JsonSerializerSettings { PreserveReferencesHandling = PreserveReferencesHandling.Objects });
Client side:
var example = JsonConvert.DeserializeObject<Example>(json,
new JsonSerializerSettings { PreserveReferencesHandling = PreserveReferencesHandling.Objects });

Related

Create a custom resolver to map properties into separate models [duplicate]

This question already has answers here:
Json.NET deserialize or serialize json string and map properties to different property names defined at runtime
(4 answers)
Closed last month.
I am writing a custom extension to DefaultContractResolver to map names between a JSON string and models to be deserialized.
The naming in these models is not consistent and I was specifically asked to not change them (renaming, adding attributes, etc).
The json looks like this
"parameter": {
"alarms": [
{
"id": 1,
"name": "alarm1",
"type": 5,
"min": 0,
"max": 2
}
],
"settings": [
{
"id": 1,
"name": "setting1",
"description": "something here"
"type": 1,
"value": 2
}
]
}
The Parameter class (output) has models for Alarms and Settings.
For example, these models look like this:
public class Alarm
{
public int AlarmId { get; set; }
public string AlarmName { get; set; }
public AlarmType RbcType { get; set; }
public int MinimumTolerated { get; set; }
public int MaximumTolerated { get; set; }
}
public class Setting
{
public int SettId { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public SettingType Type { get; set; }
public int IntValue { get; set; }
}
As an example the value of "id" in the json can relate to AlarmId or SettId, so I cannot have just one resolver to perform the ResolvePropertyName(string propertyName)
And I don't know how to get about with this.
I don't think that you need any mapper, I would use this code
var jObj = JObject.Parse(json)["parameter"];
var parameters = new
{
alarms = jObj["alarms"].Select(x => new Alarm { AlarmId = (int)x["id"], AlarmName = (string)x["name"] }).ToList(),
settings = jObj["settings"].Select(x => new Setting { SettId = (int)x["id"] }).ToList()
};

deserialize then serialize json with dynamic field name

I have a JSON file that looks something like this.
{
"id": "top1",
"components": [
{
"type": "resistor",
"id": "res1",
"resistance": {
"default": 100,
"min": 10,
"max": 1000
},
"netlist": {
"t1": "vdd",
"t2": "n1"
}
},
{
"type": "nmos",
"id": "m1",
"m(l)": {
"default": 1.5,
"min": 1,
"max": 2
},
"netlist": {
"drain": "n1",
"gate": "vin",
"source": "vss"
}
}
]
}
I want to make an API using oop to work with that JSON file,
I made the following classes.
public class Topology
{
[Required]
public string id { get; set; }
[Required]
public List<TopologyComponents> components { get; set; }
}
public class TopologyComponents
{
[Required]
public string type { get; set; }
[Required]
public string id { get; set; }
[Required]
public Values ???????? {get; set; }
[Required]
public Dictionary<string, string> netlist { get; set; }
}
public class Values
{
[Required]
public double #default { get; set; }
[Required]
public double min { get; set; }
[Required]
public double max { get; set; }
}
my question is those question marks????????
the field name is dynamic resistance, m(l), .....
how can I handle those cases?
I tried Jackson annotations, expandobject, and dictionaries. but none of them worked as I want.
From the looks of it, your Topology class will need to have Dictionary<string, dynamic> data type since the keys of components will be arbitrary. Even though type and id will be the same across all components, netlist and one other property will be dynamic.
Change your list of components to Dictionary<string, dynamic> and then get the data you need by first checking which property actually exists in the component.
public class Topology
{
[Required]
public string id { get; set; }
[Required]
public List<Dictionary<string, dynamic>> components { get; set; }
}
this will give you a list of dictionary with string as the keys and dynamic objects as the values. You can iterate through the keys using foreach loop on components.Keys and perhaps a switch statement to see if the key you expect exists when iterating through each component.
Example on how you can create your own list of components... not sure how you are going to use the data since that will drive the way you deserialize this,
var obj = JsonConvert.DeserializeObject<Topology>(jsonText);
List<dynamic> allComps = new List<dynamic>();
foreach(var component in obj.components)
{
var comp = new ExpandoObject() as IDictionary<string, object>;
foreach(var key in component.Keys)
{
switch (key)
{
case "id":
case "type":
comp.Add(key, component[key].ToString());
break;
case "netlist":
comp.Add(key, component[key].ToObject<Dictionary<string, string>>());
break;
default:
comp.Add(key, component[key].ToObject<Values>());
break;
}
}
allComps.Add(comp);
}

The returned data is partially missing in ASP.NET Web API GET method

I have some trouble with my ASP.NET Web API application with returned data.
I have 2 models, Province-to-District as One-to-Many relationships. I use Eager Loading following a tutorial from Microsoft Website
Here is my Province.cs model
[Table("province")]
public class Province
{
[Key]
[Column("id")]
public int Id { get; set; }
[Column("displayName")]
public string DisplayName { get; set; }
[Column("code")]
[StringLength(10)]
public string Code { get; set; }
// FKs
public virtual ICollection<District> Districts { get; set; }
}
My District.cs model
[Table("district")]
public class District
{
[Key]
[Column("id")]
public int Id { get; set; }
[Column("displayName")]
public string DisplayName { get; set; }
[Column("prefix")]
public string Prefix { get; set; }
[Column("provinceId")]
public int ProvinceId { get; set; }
// FKs
public virtual Province Province { get; set; }
}
At my LocationController.cs
public async Task<ActionResult<IEnumerable<Province>>> GetProvince([FromQuery] Province _province)
{
// Eager Loading
return await _context.province.Include(p => p.Districts).Where(p => p.Id == _province.Id).ToListAsync();
}
When I call https://localhost:44346/api/province?id=1 or with any id existed in my database. Both Postman and all my browsers (MS Edge, Chrome...) return only one part of a JSONArray. Look likes this:
[
{
"id": 1,
"displayName": "Hồ Chí Minh",
"code": "SG",
"districts": [
{
"id": 1,
"displayName": "Bình Chánh",
"prefix": "Huyện",
"provinceId": 1
That's it. Only 11 lines of data. It doesn't even match the JSON format.
My expected return must be:
[
{
"id": 1,
"displayName": "Hồ Chí Minh",
"code": "SG",
"districts": [
{
"id": 1,
"displayName": "Bình Chánh",
"prefix": "Huyện",
"provinceId": 1
},
{
...
},...
},...
]
Or at least, the right format of JSON.
My previous 2 APIs get all data of each table work fine.
What I'm missing here? Thanks.
You should create a model and return it instead of your entities.
This is happening because of self reference loop.
If you are using Newtonsoft.Json you can ignore it in startup.cs
services.AddMvc()
.AddJsonOptions(options =>
{
options.SerializerSettings.ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore;
})

Deserialize dynamic JSON data to C# object [duplicate]

This question already has answers here:
How can I parse a JSON string that would cause illegal C# identifiers?
(3 answers)
Using JSON.NET to read a dynamic property name
(1 answer)
Create a strongly typed c# object from json object with ID as the name
(1 answer)
Closed 4 years ago.
I have a JSON string like this:
{
"ProfileChange": 5,
"profileId": "anId",
"profileChanges": [
{
"changeType": "fullProfileUpdate",
"profile": {
"_id": "g3fwerfgscsdadad",
"rvn": 7,
"wipeNumber": 5,
"accountId": "gw4gefsdfswfwfwdqe",
"profileId": "anId",
"version": "latestUpdate",
"items": {
"abc-123": {
"templateId": "default",
"attributes": {
"max": 0,
"bonus": 1,
"seen": 1,
"xp": 0,
"variants": [],
"favorite": false
},
"quantity": 1
},
"zxc-456": {
"templateId": "default",
"attributes": {
"max": 0,
"bonus": 1,
"seen": 1,
"xp": 0,
"variants": [],
"favorite": false
},
"quantity": 1
}
}
}
}
]
}
I want to deserialize it to a C# object, and here is my code:
class Model
{
[JsonProperty("ProfileChange")] public long ProfileChange { get; set; }
[JsonProperty("profileId")] public string ProfileId { get; set; }
[JsonProperty("profileChanges")] public List<ProfileChange> ProfileChanges { get; set; }
}
public class ProfileChange
{
[JsonProperty("changeType")] public string ChangeType { get; set; }
[JsonProperty("profile")] public Profile Profile { get; set; }
}
public class Profile
{
[JsonProperty("_id")] public string Id { get; set; }
[JsonProperty("rvn")] public long Rvn { get; set; }
[JsonProperty("wipeNumber")] public long WipeNumber { get; set; }
[JsonProperty("accountId")] public string AccountId { get; set; }
[JsonProperty("profileId")] public string ProfileId { get; set; }
[JsonProperty("version")] public string Version { get; set; }
[JsonProperty("items")] public Items Items { get; set; }
}
public class Items
{
public List<IDictionary<string, ItemDetails>> Item { get; set; }
}
public class ItemDetails
{
[JsonProperty("templateId")] public string TemplateId { get; set; }
}
And then I implement it in my main implementation like this:
var obj = JsonConvert.DeserializeObject<Model>(json);
And it does deserialize most of the data, however, the part I am struggling to deserialize is the Items one. Since they are generated dynamically, I want to create a dictionary of the items, with string for the entry and ItemDetails for the details of it.
I have used Dictionary and also made it a list, since there are multiple items, but it still is null. I also tried non-list, didn't return anything. Only null. All other details are fine, except the dictionary and dynamically generated JSON data.
I need help to fix the issue with deserializer returning null for the Items Dictionary.

Complex json deserialization

I have to deserialize the following json response (the Result list has variable length):
{
"ResultSet": {
"Query": "volkswagen",
"Result": [
{
"symbol": "VLKAY",
"name": "Volkswagen AG",
"exch": "PNK",
"type": "S",
"exchDisp": "OTC Markets",
"typeDisp": "Equity"
},
{
"symbol": "VOW3.DE",
"name": "Volkswagen AG",
"exch": "GER",
"type": "S",
"exchDisp": "XETRA",
"typeDisp": "Equity"
},
{
"symbol": "VOW.DE",
"name": "Volkswagen AG",
"exch": "GER",
"type": "S",
"exchDisp": "XETRA",
"typeDisp": "Equity"
}
]
}
}
What I got:
JavaScriptSerializer js = new JavaScriptSerializer();
string jsonString = "...String is here...";
SearchObj obj = js.Deserialize<SearchObj>(jsonString);
I understand that I usually have to create a fitting obj. e.g. SearchObj which will get filled but in this case I'm not entirely sure how this object is supposed to look like. I came up with:
class Data
{
public string symbol { get; set; }
public string name { get; set; }
public string exch { get; set; }
public string type { get; set; }
public string exchDisp { get; set; }
public string typeDisp { get; set; }
}
class Container
{
public string Query { get; set; }
public List<Data> Result { get; set; }
}
class SearchObj
{
public Container ResultSet { get; set; }
}
But guess what, it's not working, I only get ResultSet = null.
Try to change your class Container as
class Container
{
public string Query { get; set; }
public Data[] Result { get; set; }
}
I have not tested it, based on my observation
I always feel bad when I answer my own question but here it goes.
Basically my idea was correct, I only made one mistake which is that I don't need the
class SearchObj
{
public Container ResultSet { get; set; }
}
Using
Container obj = js.Deserialize<Container>(jsonString);
instead of
SearchObj obj = js.Deserialize<SearchObj>(jsonString);
made the trick. Both Data[] and List<Data> in Container work btw.
Edit:
From giammins comment it seems that it is working on some machines without that change but I guess that's a case for undefined behavior.
You can use http://www.json2charp.com to create your classes.

Categories

Resources