Develop Reference

LiteDB Insert list with BsonRef

Hi and thanks in advance everyone!
I have a collection of the following objects:
public class ItemsModel
{
public List<int> IdCollection { get; set; }
public string Name { get; set; }
public int Weight { get; set; }
}
List<ItemsModel> col = ...;
I want to optimally store this with LiteDb and be able to modify the records.
Each ItemsModel has a unique Name+Weight set.
In the entire col, the elements of the IdCollection are also unique.
Body example:
List<ItemsModel>:
[{
IdCollection: [1,3,5,6,...],
Name: "first name",
Weight: 10
},
{
IdCollection: [2,4,...],
Name: "second name",
Weight: 5
}]
I want to index by Id
I want to expand into two tables for easy storage in LiteDb:
[{
_id: 1,
NameAndWeight: {&ref: "names"}
},
{
_id: 2,
NameAndWeight: {&ref: "names"}
},
{
_id: 3,
NameAndWeight: {&ref: "names"}
},
...
]
[{
Name: "first name",
Weight: 10
},
{
Name: "second name",
Weight: 5
}]
For this I have to make new storage classes:
public class ItemsModel
{
[BsonId]
public int Id { get; set; }
[BsonRef("names")]
public NamesModel NameAndWeight { get; set; }
}
public class NamesModel
{
[BsonId(true)]
public ObjectId Id { get; set; }
public string Name { get; set; }
public int Weight { get; set; }
}
But next step I'm having trouble...
Tell me, can I somehow save data using Insert array and Include in one operation?
Or should I use foreach to first write the NamesModel in "names" DB, get the generated _id, then write the ItemsModel with a link to the NamesModel already written to the database?
using (var db = new LiteDatabase(_strConnection))
{
var itemsDb = db.GetCollection<ItemsModel>("items");
var namesDb = db.GetCollection<NamesModel>("names");
itemsDb.EnsureIndex(x => x.Id, true);
foreach (var group in col)
{
var name = new NamesModel(group.Name, group.Weight);
namesDb.Insert(name);
var itemDb = group.IdCollection.Select(el => new ItemsModel(el, name));
var h = itemsDb.Insert(itemDb);
}
}
it is too long(

Now I did like this:
using (var db = new LiteDatabase(_strConnection))
{
var itemsDb = db.GetCollection<ItemsModel>("items");
var namesDb = db.GetCollection<NamesModel>("names");
itemsDb.EnsureIndex(x => x.Id, true);
namesDb.EnsureIndex(x => x.Name);
var temp = col.Select(el => (el.IdCollection, new NamesModel(el.Name, el.Weight))).ToList();
namesDb.Insert(temp.Select(el => el.Item2));
var temp2 = temp.SelectMany(gr => gr.IdCollection.Select(el => new ItemsModel(el, gr.Item2)));
eventsIdDB.Insert(temp2);
}
Performed basic operations in linq to reduce the number of hits in liteDb

Sorting select with cast

Is it possible to do special formatted order in c#?
I have *.json file with data like
{
"RECORDS": [
{
"ROWW": "279166",
"ALBUMID": "3",
"LINK": "https://...1"
},
{
"ROWW": "279165",
"ALBUMID": "1",
"LINK": "https://...2"
},
{
"ROWW": "279164",
"ALBUMID": "2",
"LINK": "https://...3"
}]
}
... a lot of records. And I need to get DataRows ordered by Roww casted like number.
That's How I trying to do this:
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
//...
namespace WindowsFormsApp1
{
public partial class Form1 : Form
{
public class RECORD_PHOTO
{
public string ROWW { get; set; }
public string ALBUMID { get; set; }
public string LINK { get; set; }
}
public class PhotoObject
{
public List<RECORD_PHOTO> RECORDS { get; set; }
}
public List<string[]> listForJson = new List<string[]>();
public List<String> PHOTO_Rows = new List<String>();
public List<String> PHOTO_AlbumIds = new List<String>();
public List<String> PHOTO_Links = new List<String>();
string JsonFileName = #"C:\temp_work\test.json"
public DataTable dtFromJson;
private void Starting()
{
string jsonn = File.OpenText(JsonFileName).ReadToEnd();
var result = JsonConvert.DeserializeObject<PhotoObject>(JsonFileName);
PHOTO_Rows = result.RECORDS.Select(p => p.ROWW).ToList();
PHOTO_AlbumIds = result.RECORDS.Select(p => p.ALBUMID).ToList();
PHOTO_Links = result.RECORDS.Select(p => p.LINK).ToList();
for (int i = 0; i < PHOTO_Rows.Count; i++)
{
listForJson.Add(new string[] { PHOTO_Rows[i], PHOTO_AlbumIds[i], PHOTO_Links[i]});
}
dtFromJson = ConvertListToDataTable(VKPH);
dtFromJson.Columns[0].ColumnName = "ROWW";
dtFromJson.Columns[1].ColumnName = "ALBUMID";
dtFromJson.Columns[2].ColumnName = "LINK";
//
DataRow[] rows = dtFromJson.Select("", "ROWW ASC");
}
}
}
But Sorting in Select by "ROWW" worked like String so I need to add some cast like:
DataRow[] rows = dtFromJson.Select("", "TO_NUMBER(ROWW) ASC");
But this is wrong

The DataColumn.Expression property can be used to add another column to the table that maintains an int version of your ROWW column:
dtFromJson.Columns.Add("ROWWint", typeof(int)).Expression = "CONVERT([ROWW], 'System.Int32')";
You can then orderby this column in your Select
DataRow[] rows = dtFromJson.Select("", "ROWWint");
See it in action: https://dotnetfiddle.net/Cuh2np
--
You could also query the row collection using LINQ:
dtFromJson = dtFromJson.Cast<DataRow>().OrderBy(x => Convert.ToInt32((string)x["ROWW"])).ToArray();

if you want to sort by ROWW just like number then try below piece of code:
string jsonn = File.OpenText(JsonFileName).ReadToEnd();
//Your code modified. Initially it was reading file name .
//Now reading json string from file.
var result = JsonConvert.DeserializeObject<PhotoObject>(jsonn);
//Sort by ROWW after converting to int.
var sortedByRowwResult = result.RECORDS.OrderBy(x => Convert.ToInt32(x.ROWW)).ToList();
After this you can put into data table. But please check your requirement. Is it really required to put json into datatable?

How do i insert Adds many objects to the the List In c# MongoDB.Driver

How do I insert Adds many objects to the List In c# MongoDB.Driver?
my c# Entity
/// <summary>LogTest</summary>
public class VisitLog
{
/// <summary>MongoDB特有的字段</summary>
[MongoDB.Bson.Serialization.Attributes.BsonElement("_id")]
[JsonConverter(typeof(ObjectIdConverter))]
public MongoDB.Bson.ObjectId MongoId { get; set; }
/// <summary>YMD datetime</summary>
public int Yymmdd { get; set; }
/// <summary>Visitor</summary>
public string Visitor { get; set; }
/// <summary>VisitInfos</summary>
public List<VisitInfo> VisitInfos { get; set; }
}
In the MongoDBCode Like the code
// 1
{
"_id": ObjectId("5f506eb02000a9b52d72a600"),
"Yymmdd": NumberInt("20200903"),
"Visitor": "360spider",
"VisitInfos": [ ]
}
i will add objects to the "VisitInfos": [ ]
How do I insert Adds many objects to the List In c# MongoDB.Driver?
Way 1: insert only one object. my test code is:
var filter = Builders<VisitLog>.Filter.Eq("_id", item.MongoId);
var update = Builders<VisitLog>.Update.Push("VisitInfos", new VisitInfo { Visitor = Visitor, Browser = "IE", Ip = "192.168.1.1", Createtime = DateTime.Now.ToUnixTimeLocalIslong() });
var result = BB.UpdateOne(filter, update);
The Way 2: i want to insert InsertManyAsync
var items = BB.Find(x => x.Yymmdd.Equals(Yymmdd) && x.Visitor.Equals(Visitor)).Project<VisitLog>(fields).ToList();
if (items.Count > 0)
{
var item = items[0];
var VisitInfos = new List<VisitInfo>();
for (int j = 0; j < 10000; j++)
{
VisitInfos.Add(new VisitInfo { Visitor = Visitor, Browser = "IE", Ip = "192.168.1.1", Createtime = DateTime.Now.ToUnixTimeLocalIslong() });
}
var filter = Builders<VisitLog>.Filter.Eq("_id", item.MongoId);
var update = Builders<VisitLog>.Update.Push("VisitInfos", VisitInfos);
var result = BB.UpdateOne(filter, update);
}
the way 2 is failed.
please help me.
this very much.....

On the Builders<Order>.Update there's a PushEach which accepts an IEnumerable. This is equivalent to doing:
{ $push: { scores: { $each: [ 90, 92, 85 ] } } }
https://docs.mongodb.com/manual/reference/operator/update/push/#append-multiple-values-to-an-array
For simplicity here's an example of an order and order items.
In MongoDB we'll have:
> db.orders.find()
{ "_id" : ObjectId("5f50aef4d7d9f967d0322932"), "Items" : [ ] }
Then we'll execute the following C# Code.
var client = new MongoClient();
var db = client.GetDatabase("test");
var items = db.GetCollection<Order>("orders");
var filter = Builders<Order>.Filter.Empty;
var update = Builders<Order>.Update.PushEach(x => x.Items, new[]
{
new OrderItem{Name = "Order 1", Price = 10.1M},
new OrderItem{Name = "Order 2", Price = 20.2M}
});
await items.UpdateOneAsync(filter, update);
public class Order
{
public ObjectId Id { get; set; }
public List<OrderItem> Items { get; set; }
= new List<OrderItem>();
}
public class OrderItem
{
public string Name { get; set; }
public decimal Price { get; set; }
}
Now if we take a look at our document in MongoDB we'll have the 2 items added to our array
db.orders.find().pretty()
{
"_id" : ObjectId("5f50aef4d7d9f967d0322932"),
"Items" : [
{
"Name" : "Order 1",
"Price" : "10.1"
},
{
"Name" : "Order 2",
"Price" : "20.2"
}
]
}

Convert CSV (nested objects) to JSON

I need to convert a CSV data (with one line of header, and one line of data) in a JSON object.
The CSV contains nested columns, there is an example:
id,name,category/id,category/name,category/subcategory/id,category/subcategory/name,description
0,Test123,15,Cat123,10,SubCat123,Desc123
And I want a JSON to look like this:
{
"id": 0,
"name": "Test123",
"category": {
"id": 15,
"name": "Cat123",
"subcategory": {
"id": 10,
"name": "SubCat123",
}
},
"description": "Desc123"
}
I've tried CsvHelper and ChoETL libs, without success, because these libs, as far as I know, require that I have a class as model, but I don't have these classes, because the data is totally dynamic.
The site http://www.convertcsv.com/csv-to-json.htm is a good example to make this with success.
Just paste the JSON that I have created above, go to Step 3 and check the option 'Recreate nested objects and arrays', then click 'CSV to JSON' in step 5.
But I need to make this in my application, without using external frameworks.
How can I make this to work?

If you do not have it, add the newtonsoft library(dll) and then add the folowing references
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
Add the following classes
public class Rootobject
{
public int id { get; set; }
public string name { get; set; }
public Category category { get; set; }
public string description { get; set; }
}
public class Category
{
public int id { get; set; }
public string name { get; set; }
public Subcategory subcategory { get; set; }
}
public class Subcategory
{
public int id { get; set; }
public string name { get; set; }
}
then use this code
DataTable CSVData = new DataTable(); // your csv rows
HashSet<Rootobject> MyObjectsList = new HashSet<Rootobject>(); //create hashset to handle your classes
foreach(DataRow row in CSVData.Rows)
{
//change the indices in ItemArray with the right indices
MyObjectsList.Add(new Rootobject() {
id = (int)row.ItemArray[0], name = (string)row.ItemArray[0], category = new Category() {
id = (int)row.ItemArray[0], name = (string)row.ItemArray[0], subcategory = new Subcategory() {
id = (int)row.ItemArray[0], name = (string)row.ItemArray[0]
}
}
});
}
string _ResultObj = JsonConvert.SerializeObject(MyObjectsList); //here get your json string

With the latest ChoETL.JSON 1.0.1.6, you can convert them easily. This will work only if the headers are simple texts, no spaces or special characters allowed.
using (var json = new ChoJSONWriter("nested.json")
.SingleElement()
.Configure(c => c.SupportMultipleContent = true)
.Configure(c => c.DefaultArrayHandling = false)
)
{
using (var csv = new ChoCSVReader("nested.csv").WithFirstLineHeader())
json.Write(csv.Select(i => i.ConvertToNestedObject('/')));
}
Output JSON will look at below
[
{
"id":0,
"name":"Test123",
"category": {
"id": 15,
"name": "Cat123",
"subcategory": {
"id": 10,
"name": "SubCat123"
}
},
"description":"Desc123"
}
]
Sample fiddle: https://dotnetfiddle.net/vttMIB
UPDATE:
Cinchoo ETL now supports native nested object support by just setting 'NestedColumnSeparator' configuration parameter to '/'. Sample below shows how to
using (var json = new ChoJSONWriter("nested.json")
.SingleElement()
.Configure(c => c.SupportMultipleContent = true)
.Configure(c => c.DefaultArrayHandling = false)
)
{
using (var csv = new ChoCSVReader("nested.csv")
.WithFirstLineHeader()
.Configure(c => c.NestedColumnSeparator = '/')
)
json.Write(csv);
}
Sample fiddle: https://dotnetfiddle.net/xFlQso

Is there any way to obtain following json result?

I have a datatable
classname Division Subject
I A English
I A Maths
I B English
II A English
i need the output as
[
{
"className":I,
Division:[
{
name:A,
subject:[
{
name:english
},
{
name:maths
}
]
},
{
name:B,
subject:[
{
name:English
}
]
}
]
},
{
"ClassName":II,
Division:[
{
name:A,
subject:[
{
name:english
}
]
}
]
}
]

Assuming you are starting with a DataTable like this:
DataTable table = new DataTable();
table.Columns.Add("className");
table.Columns.Add("Division");
table.Columns.Add("Subject");
table.Rows.Add("I", "A", "English");
table.Rows.Add("I", "A", "Maths");
table.Rows.Add("I", "B", "English");
table.Rows.Add("II", "A", "English");
You can get the output you want like this:
List<DataRow> rows = new List<DataRow>();
foreach (DataRow row in table.Rows)
rows.Add(row);
var result = rows.GroupBy(r => r["className"])
.Select(g => new
{
className = g.Key,
division = g.GroupBy(r => r["Division"])
.Select(g1 => new
{
name = g1.Key,
subject = g1.Select(r => new
{
name = r["Subject"]
})
})
});
string json = JsonConvert.SerializeObject(result, Formatting.Indented);
Console.WriteLine(json);
Working demo: https://dotnetfiddle.net/67mjiu

You got two options:
Create the JSON by hand (using a StringBuilder or JSON.NETs built in object types)
Create DTO (Data Transfer Objects) which looks like the wanted structure, fill them with data, and then serialize them.
DTOs:
public class Clazz
{
public string className { get; set; }
public Division[] Division { get; set; }
}
public class Division
{
public string name { get; set; }
public Subject[] subject { get; set; }
}
public class Subject
{
public string name { get; set; }
}
Your JSON is not valid. All names and string values must be escaped with quotes.