Develop Reference

C# MongoDB - Filtering nested array data

i am new on MongoDB and i am trying to use it in C# context. Let´s say, i have documents like this:
[
{
"Number": "2140007529",
"Name": "ABC",
"IsInactive": true,
"EntryList": [
{
"Timestamp": "2022-06-01T14:00:00.000+00:00",
"Value": 21564.0
},
{
"Timestamp": "2022-07-01T21:31:00.000+00:00",
"Value": 21568.0
},
{
"Timestamp": "2022-08-02T21:21:00.000+00:00",
"Value": 21581.642
},
{
"Timestamp": "2022-09-02T15:42:00.000+00:00",
"Value": 21593.551
},
{
"Timestamp": "2022-09-26T13:00:00.000+00:00",
"Value": 21603
}
]
},
{
"Number": "2220000784",
"Name": "XYZ",
"IsInactive": false,
"EntryList": [
{
"Timestamp": "2022-09-26T13:00:00.000+00:00",
"Value": 0.0
},
{
"Timestamp": "2022-10-01T08:49:00.000+00:00",
"Value": 5.274
},
{
"Timestamp": "2022-11-01T09:56:00.000+00:00",
"Value": 76.753
},
{
"Timestamp": "2022-12-01T19:43:00.000+00:00",
"Value": 244.877
},
{
"Timestamp": "2023-01-01T11:54:00.000+00:00",
"Value": 528.56
},
{
"Timestamp": "2023-02-01T17:21:00.000+00:00",
"Value": 802.264
}
]
}
]
I want to get the document where the IsInactive flag is false. But for the EntryList there should be returned entries greater than Timestamp 2022-12-31 only.I should look like this:
{
"Number": "2220000784",
"Name": "XYZ",
"IsInactive": false,
"EntryList": [
{
"Timestamp": "2023-01-01T11:54:00.000+00:00",
"Value": 528.56
},
{
"Timestamp": "2023-02-01T17:21:00.000+00:00",
"Value": 802.264
}
]
}
So, here is my question. How can i filter nested arrays in return value with C#. Thanks for help!
I tried to get the result with aggregate function of MongoDB in MongoDB Compass. I got it work with but not in C#.

I think you are looking for a query similar to this one.
So you can try something like this code:
var desiredTimestamp = new DateTime(2022, 12, 31);
var results = collection.AsQueryable()
.Where(x => x.IsInactive == false && x.EntryList.Any(e => e.Timestamp >= desiredTimestamp))
.Select(obj => new
{
Number = obj.Number,
Name = obj.Name,
IsInactive = obj.IsInactive,
EntryList = obj.EntryList
.Where(e => e.Timestamp >= desiredTimestamp)
.ToList()
}).ToList()
Note that I'm assuming your Timestamp is a date type, otherwise you can't compare date and string.

Calculate the cumulative sum of a grouped collection

This is the expected result: The accumulatedMaxPax should be equal to the previous value + the current maxPax. It should reset (=0) on every destinations change.
[
{
"date": "2022-05-06",
"destinations": [
{
"id": 1,
"description": "PAXOS - ANTIPAXOS",
"abbreviation": "PA",
"ports": [
{
"id": 1,
"description": "CORFU PORT",
"maxPax": 200,
"accumulatedMaxPax": 200 // Initial (accumulatedMaxPax = maxPax )
},
{
"id": 4,
"description": "BENITSES PORT",
"maxPax": 200,
"accumulatedMaxPax": 400 // 200 (previous) + 200 (current) = 400
},
{
"id": 17,
"description": "BOUKARI PORT",
"maxPax": 100,
"accumulatedMaxPax": 500 // 400 (previous) + 100 (current) = 500
},
{
"id": 2,
"description": "LEFKIMMI PORT",
"maxPax": 400,
"accumulatedMaxPax": 900 // 500 (previous) + 400 (current) = 900
}
]
},
{
"id": 3,
"description": "BLUE LAGOON",
"abbreviation": "BL",
"ports": [
{
"id": 1,
"description": "CORFU PORT",
"maxPax": 215,
"accumulatedMaxPax": 215
},
{
"id": 4,
"description": "BENITSES PORT",
"maxPax": 215,
"accumulatedMaxPax": 430
}
]
},
{
"id": 7,
"description": "PARTY",
"abbreviation": "PRT",
"ports": [
{
"id": 1,
"description": "CORFU PORT",
"maxPax": 200,
"accumulatedMaxPax": 200
}
]
}
]
}
]
This is the code which works according to the specs, with the missing part of the puzzle as question marks!
public IEnumerable<AvailabilityCalendarGroupVM> GetForCalendar(string fromDate, string toDate) {
return context.Schedules
.Where(x => x.Date >= Convert.ToDateTime(fromDate) && x.Date <= Convert.ToDateTime(toDate))
.GroupBy(x => x.Date)
.Select(x => new AvailabilityCalendarGroupVM {
Date = DateHelpers.DateTimeToISOString(x.Key.Date),
Destinations = x.GroupBy(x => new { x.Date, x.Destination.Id, x.Destination.Description, x.Destination.Abbreviation }).Select(x => new DestinationCalendarVM {
Id = x.Key.Id,
Description = x.Key.Description,
Abbreviation = x.Key.Abbreviation,
Ports = x.GroupBy(x => new { x.PortId, x.Port.Description, x.Port.Abbreviation, x.MaxPax, x.Port.StopOrder }).OrderBy(x => x.Key.StopOrder).Select(x => new PortCalendarVM {
Id = x.Key.PortId,
Description = x.Key.Description,
MaxPax = x.Key.MaxPax,
AccumulatedMaxPax = NOW WHAT ???
})
})
}).ToList();
}

How can I filter Mongodb specific document's subdocument

Consider you have Order and OrderDetails. I'm trying to filter order details for the specific order.
For instance if you have document something like this:
{
"orders": [
{
"id": 1,
"date": "02.04.2020 ...",
"user": {
"name": "xx",
"surname": "yy"
},
"orderDetails": [
{
"id": 1,
"productId": 5,
"quantity": 1,
"state": 3
},
{
"id": 2,
"productId": 3,
"quantity": 4,
"state": 3
},
{
"id": 3,
"productId": 4,
"quantity": 12,
"state": 2
},
{
"id": 4,
"productId": 7,
"quantity": 8,
"state": 2
},
{
"id": 5,
"productId": 12,
"quantity": 9,
"state": 3
}
]
},
{
"id": 2,
"date": "01.04.2020 ...",
"user": {
"name": "xx",
"surname": "yy"
},
"orderDetails": [
{
"id": 6,
"productId": 5,
"quantity": 1,
"state": 3
},
{
"id": 7,
"productId": 3,
"quantity": 4,
"state": 3
},
{
"id": 8,
"productId": 4,
"quantity": 12,
"state": 2
}
]
}
}
What I'm trying to do is first filtering by order and then state of an order detail. I have a code like this but it always brings correct order with all orderDetails. Seems like it doesn't care about equal filter for orderDetails.
Actually it's working but not filtering. Because I only have 3 types of state(enum) and int values are 1,2,3. Query brings nothing if I give 4.
var builder = Builders<Order>.Filter;
var filter = builderk.And(builder.Eq("_id", ObjectId.Parse(elementid)), builder.Eq("orderDetails.state", 3));
var result = _mongoRepository.FindByFilter(filter).ToList();
I also tried AnyEq and something like that filters but didn't work.
I will be very happy if anyone can help me.
Thanks.

You can use aggregation operation for operations such as filter sorting in detail records.
If we continue through the sample, you must first create a filter for your master data.
var builderMaster = Builders<Order>.Filter;
var filterMaster = builderMaster.Eq("_id", ObjectId.Parse(elementid));
Then you need to create a different filter for the details.
Important: You must use the BsonDocument type when creating detail filters. Because you cannot give a specific type when you filter the details.
var builderDetail = Builders<BsonDocument>.Filter;
var filterDetail = builderDetail.Eq("orderDetails.state", 3);
Then you can start typing the query.
var list = _mongoRepository.Aggregate()
.Match(filterMaster)
.Unwind("orderDetails")// Name the details.
.Match(filterDetail)
.Sort(sort)// optionally
.Skip(skip) // optionally
.Limit(limit) // optionally
.ToList();
It will give you a list of BsonDocument with the given parameters. After that, you have to map with your own detail class.
var resultList = new List<OrderDetails>();
foreach (var master in list)
{
var masterObj = BsonSerializer.Deserialize<dynamic>(master);
foreach (var item in masterObj)
{
if (item.Key == "orderDetails")
{
var mapper = new MapperConfiguration(cfg => { }).CreateMapper();
var retItem = mapper.Map<OrderDetails>(item.Value);
resultList.Add(retItem);
}
}
}

Generic method to convert a flat JSON array to nested JSON

I have a JSON object as below
[
{
"Id": 7,
"Name": "Colocation Folder",
"ParentId": 1,
"depth": 0
},
{
"Id": 8,
"Name": "CoLo Real Estate",
"ParentId": 7,
"depth": 1
},
{
"Id": 10,
"Name": "CoLo: Burst",
"ParentId": 7,
"depth": 1
},
{
"Id": 34,
"Name": "CoLo Dedicated Bandwidth",
"ParentId": 7,
"depth": 1
},
{
"Id": 10035,
"Name": "Infrastructure as a Service",
"ParentId": 7,
"depth": 1
},
{
"Id": 10037,
"Name": "Software as a Service",
"ParentId": 7,
"depth": 1
},
{
"Id": 10038,
"Name": "IaaS Component Upgrade",
"ParentId": 7,
"depth": 1
},
{
"Id": 668,
"Name": "CoLo Misc Folder",
"ParentId": 7,
"depth": 1
},
{
"Id": 758,
"Name": "CoLo: Conduit Fee",
"ParentId": 668,
"depth": 2
},
{
"Id": 765,
"Name": "CoLo: Private VLAN",
"ParentId": 668,
"depth": 2
}
]
The Id and ParentId fields show the relation between the items. I need to make it as a nested JSON using C#.
Since there will be many such models, I don't want to create individual classes for each model. Is there a generic approach in C# that will take a flat JSON array, take the ID and ParentId fields as input and then return me a nested JSON with all other fields in the array as well? For example, I am looking for an output of nested JSON as below:
[
{
"Id": 7,
"Name": "Colocation Folder",
"items": [
{
"Id": 8,
"Name": "CoLo Real Estate",
"ParentId": 7
},
{
"Id": 10,
"Name": "CoLo: Burst",
"ParentId": 7
},
{
"Id": 34,
"Name": "CoLo Dedicated Bandwidth",
"ParentId": 7
},
{
"Id": 10035,
"Name": "Infrastructure as a Service",
"ParentId": 7
},
{
"Id": 10037,
"Name": "Software as a Service",
"ParentId": 7
},
{
"Id": 10038,
"Name": "IaaS Component Upgrade",
"ParentId": 7
},
{
"Id": 668,
"Name": "CoLo Misc Folder",
"ParentId": 7,
"items": [
{
"Id": 758,
"Name": "CoLo: Conduit Fee",
"ParentId": 668
},
{
"Id": 765,
"Name": "CoLo: Private VLAN",
"ParentId": 668
}
]
}
]
}
]

If you use Json.Net, you can do this conversion in a generic way using the LINQ-to-JSON API (JObjects). The idea is to parse the JSON array and add all the individual items to a dictionary keyed by Id. Then, loop over the dictionary items, and for each one, try to look up the parent. If the parent is found, add the item to the parent's items array (creating it if needed). Otherwise, add the item to the root array. Along the way, remove the depth property from each item, since you don't seem to want that in the output. Lastly, just dump the root array to string to get the final result.
var dict = JArray.Parse(json)
.Children<JObject>()
.ToDictionary(jo => (string)jo["Id"], jo => new JObject(jo));
var root = new JArray();
foreach (JObject obj in dict.Values)
{
JObject parent;
string parentId = (string)obj["ParentId"];
if (parentId != null && dict.TryGetValue(parentId, out parent))
{
JArray items = (JArray)parent["items"];
if (items == null)
{
items = new JArray();
parent.Add("items", items);
}
items.Add(obj);
}
else
{
root.Add(obj);
}
JProperty depth = obj.Property("depth");
if (depth != null) depth.Remove();
}
Console.WriteLine(root.ToString());
Fiddle: https://dotnetfiddle.net/Buza6T

You can use a dynamic object with JSON.Net like so to detect your properties dynamically then you could build a new json object with the desired nesting:
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
dynamic d = JArray.Parse(stringy);
foreach(var ob in d)
{
if(ob.ParentID != ob.Id)
{
string debug = "oh snapple, it's a child object";
}
}

Share my working code for you at jsFiddle full source code
recursive function is:
function getNestedChildren(arr, parent) {
var out = []
for(var i in arr) {
if(arr[i].parent == parent) {
var children = getNestedChildren(arr, arr[i].id)
if(children.length) {
arr[i].children = children
}
out.push(arr[i])
}
}
return out
}
full source code:
function getNestedChildren(arr, parent) {
var out = []
for(var i in arr) {
if(arr[i].ParentId == parent) {
var items = getNestedChildren(arr, arr[i].Id)
if(items.length) {
arr[i].items = items
}
out.push(arr[i])
}
}
return out
}
var flat_array = [
{
"Id": 7,
"Name": "Colocation Folder",
"ParentId": 1,
"depth": 0
},
{
"Id": 8,
"Name": "CoLo Real Estate",
"ParentId": 7,
"depth": 1
},
{
"Id": 10,
"Name": "CoLo: Burst",
"ParentId": 7,
"depth": 1
},
{
"Id": 34,
"Name": "CoLo Dedicated Bandwidth",
"ParentId": 7,
"depth": 1
},
{
"Id": 10035,
"Name": "Infrastructure as a Service",
"ParentId": 7,
"depth": 1
},
{
"Id": 10037,
"Name": "Software as a Service",
"ParentId": 7,
"depth": 1
},
{
"Id": 10038,
"Name": "IaaS Component Upgrade",
"ParentId": 7,
"depth": 1
},
{
"Id": 668,
"Name": "CoLo Misc Folder",
"ParentId": 7,
"depth": 1
},
{
"Id": 758,
"Name": "CoLo: Conduit Fee",
"ParentId": 668,
"depth": 2
},
{
"Id": 765,
"Name": "CoLo: Private VLAN",
"ParentId": 668,
"depth": 2
}
]
var nested = getNestedChildren(flat_array, 1)
console.log(nested)

C# Backend Pro Is needed ! Data weirdness

Here's my Code
public Tournament Read(int id)
{
using (var context = new DragonLairContext())
{
Tournament tournament = context.Tournaments
.Include(a => a.Game)
.Include(g => g.Game.Genre)
.Include(b => b.Groups.Select(g => g.Teams))
.Include(k => k.Groups.Select(y => y.Teams.Select(l => l.Players)))
.Include(c => c.TournamentType)
.FirstOrDefault(d => d.Id == id);
return tournament;
}
}
I'm using API and due to serialization issues, I need to convert my entities to DTOobject.
Here's a snippet from the Dto Converter
public override DTOTournament Convert(Tournament t)
{
if (t.Game == null || t.TournamentType == null || t.Groups == null) throw new ArgumentException("Missing some data");
DTOTournament dtoTournament = new DTOTournament();
List<DTOGroup> dtoGroups = new List<DTOGroup>();
DTOTournamentType dtoTournamentType = new DTOTournamentType() { Id = t.TournamentType.Id, Type = t.TournamentType.Type };
foreach (var group in t.Groups)
{
if (group.Teams == null) return dtoTournament;
List<DTOTeam> dtoTeams = new List<DTOTeam>();
foreach (var team in group.Teams)
{
List<DTOPlayer> dtoPlayers = new List<DTOPlayer>();
if (team.Players == null) return dtoTournament;
foreach (var player in team.Players)
{
dtoPlayers.Add(new DTOPlayer() { Id = player.Id, Name = player.Name });
}
dtoTeams.Add(new DTOTeam()
{
Id = team.Id,
Name = team.Name,
Win = team.Win,
Loss = team.Loss,
Draw = team.Draw,
DtoPlayers = dtoPlayers
});
}
dtoGroups.Add(new DTOGroup()
{
Id = group.Id,
Name = group.Name,
DtoTeams = dtoTeams,
DtoTournament = dtoTournament
});
}
dtoTournament.DTOTournamentType = dtoTournamentType;
dtoTournament.Id = t.Id;
dtoTournament.Name = t.Name;
dtoTournament.StartDate = t.StartDate;
dtoTournament.DtoGroups = dtoGroups;
dtoTournament.DtoGame = new DTOGame() { Id = t.Game.Id, Name = t.Game.Name, DtoGenre = new DTOGenre() { Id = t.Game.Genre.Id, Name = t.Game.Genre.Name } };
return dtoTournament;
}
Here's the Json
{
"$id": "1",
"Id": 1,
"Name": "I'm a Tournament",
"StartDate": "2015-12-08T00:00:00",
"DTOTournamentType": {
"$id": "2",
"Id": 1,
"Type": "I'm type 2vs2",
"DtoTournaments": null
},
"DtoGroups": [
{
"$id": "3",
"Id": 1,
"Name": "I'm a Group",
"DtoTournament": {
"$ref": "1"
},
"DtoTeams": [
{
"$id": "4",
"Id": 1,
"Draw": 0,
"Loss": 0,
"Win": 0,
"Name": "I'm a Team",
"DtoPlayers": [
{
"$id": "5",
"Id": 1,
"Name": "I'm a Group",
"DtoTeams": null
}
],
"DtoGroups": null
},
{
"$id": "6",
"Id": 2,
"Draw": 0,
"Loss": 0,
"Win": 0,
"Name": "I'm a Team",
"DtoPlayers": [
{
"$id": "7",
"Id": 1,
"Name": "I'm a Group",
"DtoTeams": null
}
],
"DtoGroups": null
}
]
}
],
"DtoGame": {
"$id": "8",
"Id": 1,
"Name": "Im a Game - Wars",
"DtoGenre": {
"$id": "9",
"Id": 1,
"Name": "I'm Genre Roleplaying",
"DtoGames": null
},
"DtoTournaments": null
}
}
My DB contains 3 players - note converted to json
[
{
"$id": "1",
"Id": 1,
"Name": "I'm player Søren",
"DtoTeams": [
{
"$id": "2",
"Id": 1,
"Draw": 0,
"Loss": 0,
"Win": 0,
"Name": "I'm a Team",
"DtoPlayers": null,
"DtoGroups": null
}
]
},
{
"$id": "3",
"Id": 2,
"Name": "I'm player Mark",
"DtoTeams": [
{
"$id": "4",
"Id": 1,
"Draw": 0,
"Loss": 0,
"Win": 0,
"Name": "I'm a Team",
"DtoPlayers": null,
"DtoGroups": null
}
]
},
{
"$id": "5",
"Id": 3,
"Name": "I'm player René",
"DtoTeams": [
{
"$id": "6",
"Id": 2,
"Draw": 0,
"Loss": 0,
"Win": 0,
"Name": "I'm a Team",
"DtoPlayers": null,
"DtoGroups": null
}
]
}
]
So my problem which I really can't figure out. How come my DtoPlayer.Name has the same name as DtoGroup.Name. Have a look at my includes.

This exception is thrown because you are trying to use a property that was not retrieved from the database after you have disposed the database context. You can see the data while debugging because the variable is defined inside the using scope. You should also fetch the Teams on Group class.
using (var context = new DragonLairContext())
{
Tournament tournament = context.Tournaments
.Include(a => a.Game)
.Include(g => g.Game.Genre)
.Include(b => b.Groups.Select(g => g.Teams))
.Include(c => c.TournamentType)
.FirstOrDefault(d => d.Id == id);
return tournament;
}

Since you said that you could see data in debug mode, try this method.
Tournament tournament = context.Tournaments
.Include(a => a.Game)
.Include(g => g.Game.Genre)
.Include(b => b.Groups)
.Include(c => c.TournamentType).ToList()
.FirstOrDefault(d => d.Id == id);