I have a datatable with 15 columns out of which 2 columns contain inner datatables containing 5 columns each. Is it possible to break hierarchy and add inner datatable columns to main datatable columns?
Or can I seperate those 2 columns from datatable and load them into different sql server table?
Inner datatables are formed by reading JSONArray object with multiple values.
I use following code part to Deserialize JSON into Datatable
var dtset = JsonConvert.DeserializeObject<DataTable>(JSON);
My JSON looks like below,
[{
"cpty": "SG, LP",
"internallegalentity": "ABC, LN",
"createdate": null,
"paymentcurrency": "USD",
"id": 123,
"version": 1,
"revisiondate": "2020-01-31 12:33:39.714",
"amount": -4.25,
"duedate": "2020-01-29",
"asofdate": "2020-02-25",
"isdeleted": false,
"source": "XYZ",
"isprovisional": null,
"isproforma": null,
"items": [{
"id": 123,
"amount": -4.25,
"currency": "USD",
"itemid": 15505,
"quantity": 1600.0,
"unit": "MWh",
"enddate": "2020-02-29",
"startdate": "2020-02-01",
"transactiondate": "2020-01-29",
"description": "Cash",
"price": 1.0
}],
"payments": [{
"createdate": "2020-02-12T00:00:00Z",
"transactiondate": "2020-02-11T12:00:00Z",
"paymentmethod": "Check",
"amount": 100.0,
"paymentcurrency": "USD",
"id": 123
},
{
"createdate": "2020-02-12T00:00:00Z",
"transactiondate": "2020-02-12T11:30:00Z",
"paymentmethod": "Check",
"amount": 100.0,
"paymentcurrency": "USD",
"id": 123
}
]
},
{
"counterparty": "AU",
"internallegalentity": "xyc, LP",
"createdate": null,
"paymentcurrency": "USD",
"id": 125,
"version": 1,
"revisiondate": "2020-01-31 12:33:39.922",
"amount": -5120.0,
"duedate": "2020-01-30",
"asofdate": "2020-02-25",
"isdeleted": false,
"source": "XYZ",
"isprovisional": null,
"isproforma": null,
"items": [{
"id": 125,
"amount": -1280.0,
"currency": "USD",
"itemid": 15508,
"quantity": 64000.0,
"unit": "MWh",
"enddate": "2021-02-28",
"startdate": "2021-01-01",
"transactiondate": "2020-01-30",
"description": "Cash",
"price": 1.0
},
{
"id": 125,
"amount": -1280.0,
"currency": "USD",
"itemid": 15507,
"quantity": 64000.0,
"unit": "MWh",
"enddate": "2021-02-28",
"startdate": "2021-01-01",
"transactiondate": "2020-01-30",
"description": "Cash",
"price": 1.0
}
]
}
]
Is there any way to read above JSON in more approprate way?
I'm assuming the real struggle you're having is not so much about the bulk copy aspect per se, but about flattening out the datatables. So aside from some simple code at the end, I'll really focus on that problem.
By the way, this is how you represent json in c# (fill in the elipsies of course):
static string json = #"
[
{
""cpty"": ""SG, LP"",
""internallegalentity"": ""ABC, LN"",
""createdate"": null,
""paymentcurrency"": ""USD"",
""id"": 123,
""version"": 1,
...
},
...
]";
First, create three DataTable variables. The first is your parsed json data, the other two are empty ones to receive your items and payments data:
DataTable jsonTable = JsonConvert.DeserializeObject<DataTable>(json);
DataTable items = null;
DataTable payments = null;
Next, loop through your parsed json data and extract your nested datatables into their variable counterparts. On the first pass, you'll want to copy the tables into their respective variables. On subsequent passes, you'll want to merge the tables into the variables. And be careful for the DBNull result in your payments field.
foreach (DataRow row in jsonTable.Rows) {
// get the inner "items"
var jsonItems = (DataTable)row["items"];
if (items == null)
items = jsonItems.Copy(); // first pass
else
items.Merge(jsonItems); // subsequent passes
// there is a null payments record, ignore it
if (row["payments"] == DBNull.Value)
continue;
// get the inner payments
var jsonPayments = (DataTable)(row["payments"]);
if (payments == null)
payments = jsonPayments.Copy(); // first not null pass
else
payments.Merge(jsonPayments); // subsequent passes
}
Now that you have the items and payments tables, delete those fields from your originally parsed table:
jsonTable.Columns.Remove("items");
jsonTable.Columns.Remove("payments");
You are now ready to bulk copy as appropriate. The following untested code assumes you already have existing tables with columns in the same order as your datatable representations.
using (var bulkcopy = new SqlBulkCopy("connection string")) {
bulkcopy.DestinationTableName = "main";
bulkcopy.WriteToServer(jsonTable);
bulkcopy.DestinationTableName = "items";
bulkcopy.WriteToServer(items);
bulkcopy.DestinationTableName = "payments";
bulkcopy.WriteToServer(payments);
}
Related
Delete a specific node from all the available documents in a bucket
Is it possible to delete an element from the children array based on the Id from the below sample document using .NET SDK for Couchbase in C#?
i.e. I need to delete all the details of Id=100( delete the entire element in the children array).
I can achieve it using N1QL. N1QL performance is slow when there are thousands of document involved in couchbase
{
"results": [{
"tutorial": {
"type": "contact",
"title": "Mr.",
"fname": "Ian",
"lname": "Taylor",
"age": 56,
"email": "ian#gmail.com",
"children": [{
"Id": "100",
"Details": [{
"fname": "Abama",
"age": 17,
"gender": "F"
}]
},
{
"Id": "101",
"Details": [{
"fname": "Alex",
"age": 17,
"gender": "M"
}]
}
],
"hobbies": [
"golf",
"surfing"
],
"relation": "cousin"
}
}]
}
I am new in MongoDB and I am developing a software by C# and MongoDB. My data structure is like this
{
"Id": 1,
"Title": "myTitle",
"Geners": [ "Drama", "Action" ],
"Category": 1,
"Casts": [
{
"Id": 1,
"Name": "myName",
"Gender": "Male",
"Age": 35
},
{
"Id": 2,
"Name": "herName",
"Gender": "Female",
"Age": 30
},
{
"Id": 3,
"Name": "hisName",
"Gender": "Male",
"Age": 45
}
]
}
This is just one document and I have about 5 million documents. I want to run a query like below to count the records based on Category and shows me how many movie do I have in each category and I want to put Casts field in result.
db.getCollection('myCollection').aggregate([
{
$group:{"_id":"$Category", "count": {$sum:1},
"Casts":{$push:"$Casts"}}
}
])
this is close to something I want but the problem is, it puts Casts data in second level of array like {"Id":1, ... , "Casts":[[{},{},...]]} but I need it like this {"Id":1, ... , "Casts":[{},{},...]}
How can I show the data like that?
If duplicates are acceptable, then the following aggregation will suffice:
db.getCollection('myCollection').aggregate([
{ $unwind:"$Casts"},
{
$group:{"_id":"$Category", "count": {$sum:1},
"Casts":{$push:"$Casts"}}
}
])
Update:
Since you need the count to be valid, there's a few more hoops to jump through.
db.getCollection('myCollection').aggregate([
{ $group:{"_id":"$Category", "count": {$sum:1}, "Casts":{$addToSet:"$Casts"}}},
{$unwind:"$Casts"},
{$unwind:"$Casts"},
{ $group:{"_id":"$_id", "count": {$first:"$count"}, "Casts":{$addToSet:"$Casts"}}},
])
Let me know if that helps
I have a valid json (any json string) string and trying to convert it to Dataset but Newtonsoft.Json failed to do so.
Json text:
{"root": {
"Item": [
{
"Name": "Super Mario Bros",
"Count": "14",
"Price": "29,99",
"Comment": "-No Comment-",
"Artist": "N/A",
"Publisher": "Nintendo",
"Genre": "Video Games",
"Year": "1985",
"ProductID": "001"
},
{
"Name": "The Legend of Zelda",
"Count": "12",
"Price": "34,99",
"Comment": "-No Comment-",
"Artist": "N/A",
"Publisher": "Nintendo",
"Genre": "Video Games",
"Year": "1986",
"ProductID": "002"
}
]
}
}
Code:
var table = JsonConvert.DeserializeObject<DataSet>(jsonText);
Error:
Unexpected JSON token when reading DataTable. Expected StartArray, got StartObject. Path 'root', line 1, position 9.
Edit 1:
user can pass any type of json and i need to convert it to DataSet
for the above example "root" element can contain any other property
like "child1":"val1", "child2":"val2" and so forth. so, the output
dataset will contain 2 tables namse root(should have one rows of
properties 1 and 2) and item(should have 2 rows of type name,count,price
etc).
It is not working because the JSON object representing the DataSet is not at the root level of the JSON. In your JSON, it is inside a property called root, which is inside another wrapper object. So you will need to take that outer object into account when you deserialize. You can either define a wrapper class and deserialize into that:
public class Wrapper
{
[JsonProperty("root")]
public DataSet DataSet { get; set; }
}
Then:
DataSet ds = JsonConvert.DeserializeObject<Wrapper>(json).DataSet;
(Fiddle)
Or, if you don't want to make a class, you can instead deserialize into a JObject, navigate down to the root property and then materialize it to a DataSet from there:
DataSet ds = JObject.Parse(json)["root"].ToObject<DataSet>();
(Fiddle)
The Json you showed is invalid.
It should look like this, to be load to the DataSet:
{
"Item": [
{
"Name": "Super Mario Bros",
"Count": "14",
"Price": "29,99",
"Comment": "-No Comment-",
"Artist": "N/A",
"Publisher": "Nintendo",
"Genre": "Video Games",
"Year": "1985",
"ProductID": "001"
},
{
"Name": "The Legend of Zelda",
"Count": "12",
"Price": "34,99",
"Comment": "-No Comment-",
"Artist": "N/A",
"Publisher": "Nintendo",
"Genre": "Video Games",
"Year": "1986",
"ProductID": "002"
}
]
}
Code:
var dataSet = JsonConvert.DeserializeObject<DataSet>(jsonText);
var table = dataSet.Tables[0];
just add [] while stringifying the object:
JSON.stringify([this.YourModel])
I have JSON data with nested arrays (see example below). What I am trying to accomplish is to deserialize this data into a DataSet where each nested array gets inserted into a corresponding datatable.
Example:
[
{
"Id": "1",
"LastName": "John",
"FirstName": "Doe",
"MiddleInitial": "I",
"DateOfBirth": "2000-10-05",
"Gender": "M",
"LastModifiedDate": "2017-03-13 14:36:53",
"Classes": [
{
"ClassNumber": "21",
"TeacherID": "15"
},
{
"ClassNumber": "12",
"TeacherID": "10"
}
]
},
{
"Id": "2",
"LastName": "Jane",
"FirstName": "Doe",
"MiddleInitial": "K",
"DateOfBirth": "2000-10-05",
"Gender": "F",
"LastModifiedDate": "2017-03-13 14:36:53",
"Classes": [
{
"ClassNumber": "11",
"TeacherID": "8"
},
{
"ClassNumber": "4",
"TeacherID": "26"
}
]
}]
So the dataset would contain 2 datatables. One with all of the records from the main array and the second with all of the records from the "Classes" array.
You kind of have to create the DataSet and its DataTables manually and fill them manually. There's no automatic way of doing it, if that's what you were hoping for. If you need to maintain the relationships between the objects you have to add a foreign key to the Class anyway, otherwise there's no way to know which person a Class belongs to.
The table columns could be generated from the JSON properties, of course, if you're careful enough with how you write the code or if the JSON structure is always the same without exception.
I have a JSON string that I'm getting from Facebook API, in which I have a node whose name changes according to its content, for example some time it is 45, or 58 etc.
It could be any number.
I want its value. How to get it?
Example:
{
"data": [
{
"id": "1492292372_10201810786059989",
"created_time": "2014-04-05T09:00:54+0000"
},
{
"id": "1492292372_10201804679827337",
"created_time": "2014-04-04T07:29:07+0000"
},
{
"id": "1492292372_10201804649306574",
"created_time": "2014-04-04T07:10:33+0000"
},
{
"id": "1492292372_10201801316823264",
"created_time": "2014-04-03T18:31:50+0000"
},
{
"id": "1492292372_10201798962284402",
"created_time": "2014-04-03T06:24:47+0000"
},
{
"message_tags": {
"0": [
{
"id": "1492292372",
"name": "Yawar Sohail",
"type": "user",
"offset": 0,
"length": 12
}
],
"15": [
{
"id": "1489845168",
"name": "Zeeshan Anjum",
"type": "user",
"offset": 15,
"length": 13
}
]
},
"id": "1492292372_10201796274777216",
"created_time": "2014-04-02T17:57:05+0000"
},
{
"id": "1492292372_10201794080482360",
"created_time": "2014-04-02T07:26:23+0000"
},
Inside message_tags there are two nodes [0 and 15] they dynamically changes according to their offset values. I want names, type and ids inside these nodes.
You can deserialize your JSON into an ExpandoObject:
var converter = new ExpandoObjectConverter();
dynamic obj = JsonConvert.DeserializeObject<ExpandoObject>(json, converter);
Which dynamically adds members to your object at runtime, and allows you to iterate over them as described in this answer:
foreach (var prop in obj.GetType().GetProperties(BindingFlags.Instance | BindingFlags.Public))
{
Console.WriteLine("Name: {0}, Value: {1}",prop.Name, prop.GetValue(obj,null));
}
That way you can iterate over obj.message_tags to get the individual messages, and obtain all their details respectively.