Develop Reference

How to write this MongoDB (Aggregate) query into C#

I need to perform group-by with the $max operator in MongoDB. I figure out the query which is working in the MongoDB database but was not able to write the same using C#.
db.getCollection('Employee').aggregate(
[
{$unwind : "$Projects"},
{
"$group" : {
"_id" : "$EmpId",
"LastUpdated" : {"$max" : "$Projects.LastUpdated"}
}
}
]);
Below C# code is giving an error:
"Projects" is not valid property.
_db.GetCollection<BsonDocument>(collection).Aggregate().Unwind(i=>i.Projects)

Assume this is your sample data:
[{
"EmpId": 1,
"Projects": [
{
"LastUpdated": {
"$date": "2021-10-22T16:00:00Z"
}
},
{
"LastUpdated": {
"$date": "2021-11-07T16:00:00Z"
}
},
{
"LastUpdated": {
"$date": "2022-01-22T16:00:00Z"
}
}
]
}]
and this is your model class:
public class Employee
{
public int EmpId { get; set; }
public List<EmployeeProject> Projects { get; set; }
}
public class EmployeeProject
{
public DateTime UpdatedDate { get; set; }
}
To use the Projects property, you need to specify your collection as Project type as:
_db.GetCollection<Employee>("Employee")
Solution 1: Mix use of AggregateFluent and BsonDocument
var result = _db.GetCollection<Employee>("Employee")
.Aggregate()
.Unwind(i => i.Projects)
.Group(new BsonDocument
{
{ "_id", "$EmpId" },
{ "LastUpdated", new BsonDocument("$max", "$Projects.LastUpdated") }
})
.ToList();
Solution 2: Full use of AggregateFluent
Pre-requisites:
Need to create a model for unwinded Project.
public class UnwindEmployeeProject
{
public int EmpId { get; set; }
public EmployeeProject Projects { get; set; }
}
var result = _db.GetCollection<Employee>("Employee")
.Aggregate()
.Unwind<Employee, UnwindEmployeeProject>(i => i.Projects)
.Group(
k => k.EmpId,
g => new
{
EmpId = g.Key,
LastUpdated = g.Max(x => x.Projects.LastUpdated)
})
.ToList();
Solution 3: Full use of BsonDocument
With Mongo Compass, you can export your query to C#.
PipelineDefinition<Employee, BsonDocument> pipeline = new BsonDocument[]
{
new BsonDocument("$unwind", "$Projects"),
new BsonDocument("$group",
new BsonDocument
{
{ "_id", "$EmpId" },
{ "LastUpdated",
new BsonDocument("$max", "$Projects.LastUpdated") }
})
};
var result = _db.GetCollection<Employee>("Employee")
.Aggregate(pipeline)
.ToList();
Output

Parsing json file in c# to get into all sections and properties

==========Update
So in trying to phrase the issue properly, I have misled the responders in not giving me the answer I need. Apologies, let me try to clarify.
I need to be able to cycle through a json file that is not as structured as I indicated in the OP, it is much more random. The OP file did not convey that very well.
Let me try to describe what is closer to the file I'll be getting. The first two levels will be fixed, I'll call them LevelA and LevelB. But the properties in LevelB can be any pair of random data. This is a better json file example:
{
"LevelA": {
"LevelB": [
{
"EmpName": "John",
"EmpGender": "Male",
"Age": "25"
},
{
"FavoriteFood": "Beer",
"BaseballTeam": "Kansas City Royals"
},
{
"Red": "10",
"Blue": "40",
"White: "True"
}
]
}
}
Say I need to write out the following to console:
A LevelB entry has these properties:
Property: EmpName, Value: John
Property: EmpGender, Value: Male
Property: Age, Value: 25
A LevelB entry has these properties:
Property: FavoriteFood, Value: Beer
Property: BaseballTeam, Value: Kansas City Royals
A LevelB entry has these properties:
Property: Red, Value: 10
Property: Blue, Value: 40
Property: White, Value: True
It may not make sense but, I need to find a way to do that, I need that knowledge. I appreciate the answers about using a model, but I don't see a way to use a model without complicating what I think should be a simple task. Although not simple enough for me to figure out apparently. :)
==========
C#, VS 2019, a .NET framework console test app. I'm looking to do something simple but I can't find the right syntax.
I have a json file with the following structure. I want to loop through each Employee below and get at its properties (name, gender, etc...):
{
"Company": {
"Employees": [
{
"EmpName": "John",
"EmpGender": "Male",
"Age": "25"
},
{
"EmpName": "Mary",
"EmpGender": "Female"
},
{
"EmpName": "Bill",
"Age": "30"
}
]
}
}
First question is which package will do the job? I've installed Microsoft.Extensions.Configuration.Json and System.Configuration.ConfigurationManager but am having difficulties getting the syntax correct. Are these the right packages to use? I decided to use those because I thought I could load the file via ConfigurationBuilder and that the GetSection and/or GetChildren methods would help me. I can load the file but I can't figure out how to use those methods to give me what I want.
Second I don't want to build a model for this, I just want at the data. If I can get each employee into a dictionary, I can analyze that, and that would get me going. Thanks for any advice.

You can use the built-in JSON library in the .net core
using System.Text.Json;
add the following model definition
public class Rootobject
{
public Company Company { get; set; }
}
public class Company
{
public Employee[] Employees { get; set; }
}
public class Employee
{
public string EmpName { get; set; }
public string EmpGender { get; set; }
public string Age { get; set; }
}
deserialize your object like the following
string jsonData = File.ReadAllText("data.json");
Rootobject ob = JsonSerializer.Deserialize<Rootobject>(jsonData);
now you have ob in you c# represent your JSON as C# object
I don't want to build a model for this
if you use Visual Studio you can auto generate your model classes required for your JSON as described here the above models are auto generated by Visual Studio

If you don't want to create a model,and get a List<Dictionary<string,string>>,you can use custom model binding,here is a demo:
CustomBinder:
public class CustomBinder:IModelBinder
{
public Task BindModelAsync(ModelBindingContext bindingContext)
{
if (bindingContext == null)
{
throw new ArgumentNullException(nameof(bindingContext));
}
var model = new List<Dictionary<string, string>>();
using (var reader = new StreamReader(bindingContext.HttpContext.Request.Body))
{
var body = reader.ReadToEndAsync();
var mydata = JsonConvert.DeserializeObject<JObject>(body.Result);
var s = mydata["Company"]["Employees"].ToString();
var list=JsonConvert.DeserializeObject<List<JObject>>(s);
foreach (var jobj in list) {
Dictionary<string, string> d = new Dictionary<string, string>();
foreach (var x in jobj)
{
string name = x.Key;
string value = x.Value.ToString();
d.Add(x.Key, x.Value.ToString());
}
model.Add(d);
}
}
bindingContext.Result = ModelBindingResult.Success(model);
return Task.CompletedTask;
}
}
Action:
public void TestJson1([ModelBinder(BinderType = typeof(CustomBinder))] List<Dictionary<string, string>> jsondata)
{
}
result:

Why is it a requirement not to build models? In my opinoin that is the easiest way and if you need to expand due to JSON changes, you can easily adapt it in the code. I paste a sample code here what is using System.Text.Json namespace (no other packages are also required due to it is built-in).
using System;
using System.Collections.Generic;
using System.Text.Json;
using System.Text.Json.Serialization;
namespace jsontest
{
class Program
{
static void Main(string[] args)
{
string input = "{ \"Company\": { \"Employees\": [{ \"EmpName\": \"John\", \"EmpGender\": \"Male\", \"Age\": \"25\" }, { \"EmpName\": \"Mary\", \"EmpGender\": \"Female\" }, { \"EmpName\": \"Bill\", \"Age\": \"30\" }]}}";
var processedInput = JsonSerializer.Deserialize<Company>(input);
foreach (var item in processedInput.Peoples.PeopleList)
{
Console.WriteLine($"{item.Name} - {item.Gender} - {item.Age}");
}
}
}
public class Employee
{
[JsonPropertyName("EmpName")]
public string Name { get; set; }
[JsonPropertyName("EmpGender")]
public string Gender { get; set; }
[JsonPropertyName("Age")]
public string Age { get; set; }
}
public class Employees
{
[JsonPropertyName("Employees")]
public List<Employee> PeopleList { get; set; }
}
public class Company
{
[JsonPropertyName("Company")]
public Employees Peoples { get; set; }
}
}
Update after your update:
Strange why you store data in JSON on this way. I wrote a quick code, it is using regex and some built-in function to parse the text. At the end result is similar what you would prefer. Code is a bit long, but only because I put some comment to make understanding easier.
using System;
using System.Collections.Generic;
using System.Text.Json;
using System.Text.Json.Serialization;
namespace jsontest
{
class Program
{
static void Main(string[] args)
{
List<List<ListItem>> result = new List<List<ListItem>>();
string input = "{ \"Company\": { \"Employees\": [{ \"EmpName\": \"John\", \"EmpGender\": \"Male\", \"Age\": \"25\" }, { \"EmpName\": \"Mary\", \"EmpGender\": \"Female\" }, { \"EmpName\": \"Bill\", \"Age\": \"30\" }]}}";
// Remove new lines, so input will become one single line
input = input.Replace(Environment.NewLine, " ");
// 1. group is the group name (in this case Employees)
// 2. group is the content after group name
string pattern1 = #"[{].+[{](.+)[\[](.+)[\]]";
foreach (System.Text.RegularExpressions.Match m in System.Text.RegularExpressions.Regex.Matches(input, pattern1))
{
// groupName -> "Employees":
string groupName = m.Groups[1].Value;
// groupName -> Employees
groupName = groupName.Substring(0, groupName.LastIndexOf("\""));
groupName = groupName.Substring(groupName.IndexOf("\"") + 1);
// contentList -> { "EmpName": "John", "EmpGender": "Male", "Age": "25" }, { "EmpName": "Mary", "EmpGender": "Female" }, { "EmpName": "Bill", "Age": "30" }
string contentList = m.Groups[2].Value;
// Split the line onto more lines where "}," characters
// { "EmpName": "John", "EmpGender": "Male", "Age": "25"
// { "EmpName": "Mary", "EmpGender": "Female"
// { "EmpName": "Bill", "Age": "30" }
string[] contentItems = contentList.Split("},");
foreach (var item in contentItems)
{
// Check every group and store them in separate list
result.Add(new List<ListItem>());
string[] keys = item.Split(",");
foreach (var key in keys)
{
// Check every Key-Value pair and store their value in the current list
string pattern2 = "[\"](.+)[:].[\"](.+)[\"]";
foreach (System.Text.RegularExpressions.Match m2 in System.Text.RegularExpressions.Regex.Matches(key, pattern2))
{
result[result.Count - 1].Add(new ListItem() { Property = groupName, Key = m2.Groups[1].Value.Substring(0, m2.Groups[1].Value.Length - 1), Value = m2.Groups[2].Value });
}
}
}
}
for (int i = 0; i < result.Count; i++)
{
for (int j = 0; j < result[i].Count; j++)
{
if (j == 0)
Console.WriteLine($"A {result[i][j].Property} entry has these properties:");
Console.WriteLine($"Proprty: {result[i][j].Key}, Value: {result[i][j].Value}");
}
}
}
}
class ListItem
{
public string Property { get; set; }
public string Key { get; set; }
public string Value { get; set; }
}
}
Output of this code is:
A Employees entry has these properties:
Proprty: EmpName, Value: John
Proprty: EmpGender, Value: Male
Proprty: Age, Value: 25
A Employees entry has these properties:
Proprty: EmpName, Value: Mary
Proprty: EmpGender, Value: Female
A Employees entry has these properties:
Proprty: EmpName, Value: Bill
Proprty: Age, Value: 30

MongoDB updateMany set a boolean field using equals

I have an IsDefault field in my model, and I want to set it to true for one document only and false for the others. Is this possible in a single query? I tried
{ $set: { IsDefault: {$eq: [_id, id ]}}}
{ $set: { IsDefault: { $cond: { if: { $eq: [$_id, id] }, then: true, else: false } } } }
and it didn't work. I also tried passing a lambda expression but it won't build either. Any idea if this is possible or do I need to use two seperate updateOne statements to the the current IsDefault to false, and the new one to true?

you can achieve it with an aggregation pipeline update like this:
db.collection.updateMany(
{},
[
{
$set: {
IsDefault: {
$eq: ["$_id", ObjectId("5f65c2fcaf29d00898173d03")]
}
}
}
])
here's a full example using mongodb.entities:
using MongoDB.Entities;
using System.Threading.Tasks;
namespace StackOverflow
{
public class Book : Entity
{
public string Title { get; set; }
public bool IsDefault { get; set; }
}
static class Program
{
private static async Task Main()
{
await DB.InitAsync("test", "localhost");
var books = new[] {
new Book { Title = "book 1"},
new Book { Title = "book 2"},
new Book { Title = "book 3"}
};
await books.SaveAsync();
var stage = new Template<Book>(#"
{
$set: {
IsDefault: {
$eq: ['$_id', ObjectId('<idvalue>')]
}
}
}")
.Tag("idvalue", books[1].ID);
await DB.Update<Book>()
.Match(_ => true)
.WithPipelineStage(stage)
.ExecutePipelineAsync();
}
}
}

How to put multiple values in Dictionary instances using LINQ?

My Json Response is Following below:
{"d":
{"RowData":
[{"GenreId":11,"GenreName":"Musical","subjecturl":"subjecturl_1","logourl":"logourl_1"},
{"GenreId":12,"GenreName":"kids","subjecturl":"subjecturl_2","logourl":"logourl_2"},
{"GenreId":13,"GenreName":"other","subjecturl":"subjecturl_3","logourl":"logourl_3"},
{"GenreId":14,"GenreName":"Musical","subjecturl":"subjecturl_4","logourl":"logourl_4"},
{"GenreId":15,"GenreName":"Music","subjecturl":"subjecturl_5","logourl":"logourl_5"},
{"GenreId":16,"GenreName":"Faimaly","subjecturl":"subjecturl_6","logourl":"logourl_6"},
{"GenreId":17,"GenreName":"other","subjecturl":"subjecturl_7","logourl":"logourl_7"},
{"GenreId":18,"GenreName":"other","subjecturl":"subjecturl_8","logourl":"logourl_8"},
{"GenreId":19,"GenreName":"kids","subjecturl":"subjecturl_9","logourl":"logourl_9"},
{"GenreId":20,"GenreName":"Musical","subjecturl":"subjecturl_10","logourl":"logourl_10"},
{"GenreId":21,"GenreName":"other","subjecturl":"subjecturl_11","logourl":"logourl_11"}]}}
Using the above Response I tried to make like below Response :
{"rows": [{
"title": "Musical",
"items": [{"hdsubjecturl": "subjecturl_1"},{"hdsubjecturl": "subjecturl_4"},{"hdsubjecturl": "subjecturl_10"}]
},{
"title": "kids",
"items": [{"hdsubjecturl": "subjecturl_2"},{"hdsubjecturl": "subjecturl_9"}]
},{
"title": "Music",
"items": [{"hdsubjecturl": "subjecturl_5"}]
},{
"title": "other",
"items": [{"hdsubjecturl": "subjecturl_3"},{"hdsubjecturl": "subjecturl_7"},{"hdsubjecturl": "subjecturl_8"},{"hdsubjecturl": "subjecturl_11"}]
},{
"title": "Faimaly",
"items": [{"hdsubjecturl": "subjecturl_6"}]
}]
}
My Code is below :
JObject Root = JObject.Parse(jsonData["d"].ToString());
var unique = Root["RowData"].GroupBy(x => x["GenreName"]).Select(x => x.First()).ToList(); // here fetch 5 record
foreach (var un in unique)
{
var GenreName = new
{
title = un["GenreName"],
items = new
{
hdsubjecturl = "logourl"
}
};
var GenreNamereq = JsonConvert.SerializeObject(GenreName, Newtonsoft.Json.Formatting.Indented);
genstr.Append(GenreNamereq, 0, GenreNamereq.Length);
genstr.Append(",");
using (System.IO.StreamWriter file = new System.IO.StreamWriter(subdir + "\\GenreName.json"))
{
string st = genstr.ToString().Substring(0, (genstr.Length - 1));
file.WriteLine("{\n\"rows\": [\n" + st + "\n}"); //seasion number 21 terminate
file.Close();
}
}
Using above code my output is below for First Field :
{"rows":
[{
"title": "Musical",
"items": {
"hdsubjecturl": "logourl"
}
}]
}
Using below code I tried to fetch Multiple values using specific Field :
List<string> CategoryList = new List<string>();
var unique = Root["RowData"].GroupBy(x => x["GenreName"]).Select(x => x.First()).ToList(); // here fetch 8 record
foreach (var cat in unique)
{
CategoryList.Add(cat["GenreName"].ToString());
}
List<List<string>> myList = new List<List<string>>();
for (int i=0;i<CategoryList.Count();i++)
{
var results = from x in Root["RowData"]
where x["GenreName"].Value<string>() == CategoryList[i]
select x;
foreach (var token in results)
{
Console.WriteLine(token["logourl"]);
}
// myList.Add(results);
}
In the First code, I used JObject for fetching a Root node. But, using the above query it takes by default JTocken. So, I used foreach loop here.
I used Dictionary instances for JSON Creation. Using this code I successfully fetched hdsubjecturl in for loop. But, I don't know how to put multiple values in Dictionary instances. Because I get title fields only single times using a unique query and items fields inside a hdsubjetcurl is more than one. Does anyone know how it's possible?

You can group your RowData by GenreName token, then use ToDictionary method to get a result dictionary and map it to desired structure (with title and hdsubjecturl). Finally create a result object using JObject.FromObject
var json = JObject.Parse(jsonString);
var data = json["d"]?["RowData"]
.GroupBy(x => x["GenreName"], x => x["subjecturl"])
.ToDictionary(g => g.Key, g => g.ToList())
.Select(kvp => new { title = kvp.Key, items = kvp.Value.Select(x => new { hdsubjecturl = x }) });
var result = JObject.FromObject(new { rows = data });
Console.WriteLine(result);
It gives you the expected result.
Edit: according to comments, GroupBy and Select expressions should be updated to map more then one property in result title item
var data = json["d"]?["RowData"]
.GroupBy(x => x["GenreName"])
.ToDictionary(g => g.Key, g => g.ToList())
.Select(kvp => new
{
title = kvp.Key,
items = kvp.Value.Select(x => new { hdsubjecturl = x["subjecturl"], url = x["logourl"] })
});
var result = JObject.FromObject(new { rows = data });

Consider trying this code, (using Newtonsoft Json deserializer);
public partial class Root
{
public D D { get; set; }
}
public partial class D
{
public RowDatum[] RowData { get; set; }
}
public partial class RowDatum
{
public long GenreId { get; set; }
public string GenreName { get; set; }
public string Subjecturl { get; set; }
public string Logourl { get; set; }
}
public partial class Response
{
public Row[] Rows { get; set; }
}
public partial class Row
{
public string Title { get; set; }
public Item[] Items { get; set; }
}
public partial class Item
{
public string Hdsubjecturl { get; set; }
}
public class Program
{
public static void Main(string[] args)
{
var json =
#"{""d"":{""RowData"":[{""GenreId"":11,""GenreName"":""Musical"",""subjecturl"":""subjecturl_1"",""logourl"":""logourl_1""},{""GenreId"":12,""GenreName"":""kids"",""subjecturl"":""subjecturl_2"",""logourl"":""logourl_2""},{""GenreId"":13,""GenreName"":""other"",""subjecturl"":""subjecturl_3"",""logourl"":""logourl_3""},{""GenreId"":14,""GenreName"":""Musical"",""subjecturl"":""subjecturl_4"",""logourl"":""logourl_4""},{""GenreId"":15,""GenreName"":""Music"",""subjecturl"":""subjecturl_5"",""logourl"":""logourl_5""},{""GenreId"":16,""GenreName"":""Faimaly"",""subjecturl"":""subjecturl_6"",""logourl"":""logourl_6""},{""GenreId"":17,""GenreName"":""other"",""subjecturl"":""subjecturl_7"",""logourl"":""logourl_7""},{""GenreId"":18,""GenreName"":""other"",""subjecturl"":""subjecturl_8"",""logourl"":""logourl_8""},{""GenreId"":19,""GenreName"":""kids"",""subjecturl"":""subjecturl_9"",""logourl"":""logourl_9""},{""GenreId"":20,""GenreName"":""Musical"",""subjecturl"":""subjecturl_10"",""logourl"":""logourl_10""},{""GenreId"":21,""GenreName"":""other"",""subjecturl"":""subjecturl_11"",""logourl"":""logourl_11""}]}}";
var root = JsonConvert.DeserializeObject<Root>(json);
var rows = root.D.RowData.ToLookup(d => d.GenreName)
.Select(g => new Row()
{
Title = g.Key,
Items = g.ToList().Select(rd => new Item() {Hdsubjecturl = rd.Logourl}).ToArray()
}).ToArray();
var response = new Response()
{
Rows = rows
}; // reponse is the type of Json Response you wanted to achieve
Console.WriteLine();
}
}

Extract a particular record from MongoDB where the records use Object GUID in C#

One of the Mongo objects I use (referred to as 'Admin' here) uses GUID as the primary key and has a list of 'DailyActivities' which contains datevalue and another list called 'Subactivities'. The Admin object looks like something below. I am struggling to find resources in C# that will help extract DailyActivities that only corresponds to a particular date with Subactivities that has a category of 'Power Consumption'.
{
"_id" : ObjectId("5a2b7b887df7ce464404dc7d"),
"DailyActivities" : [
{
"datetime" : ISODate("2017-12-09T16:29:00.916Z"),
"Subactivities" : [
{
"entryDate" : ISODate("2017-12-09T06:30:26.658Z"),
"category" : "Power Consumption"
},
{
"entryDate" : ISODate("2017-12-09T06:30:26.658Z"),
"category" : "Machinery"
}
]
},
{
"datetime" : ISODate("2017-12-13T00:00:00.916Z"),
"Subactivities" : [
{
"entryDate" : ISODate("2017-12-13T06:30:26.658Z"),
"category" : "Lamination"
}
]
}
]
}
The result I would like to receive should be:
{
"_id" : ObjectId("5c7044f07ef75175b2b8efd6"),
"entryDate" : ISODate("2017-12-09T06:30:26.658Z"),
"category" : "Power Consumption"
}

Right now, I don't have the time to complete this exercise but here is something to get you going. I shall edit and improve this (as in convert it into some c# a lot of which won't be possible using a typed approach...) next week.
db.collection.aggregate([{
$project: {
"DailyActivities": {
$filter: {
input: "$DailyActivities",
cond: {
$eq: [ "$$this.datetime", ISODate("2017-12-09T16:29:00.916Z") ]
}
}
}
}
}, {
$unwind: "$DailyActivities"
}, {
$unwind: "$DailyActivities.Subactivities"
}, {
$replaceRoot: {
"newRoot": "$DailyActivities.Subactivities"
}
}, {
$match: {
"category": "Power Consumption"
}
}])

let me offer you a solution which uses MongoDAL as the data access layer. it's a wrapper around the c# driver so you get all the features of the driver plus a highly typed api.
using System;
using System.Linq;
using MongoDAL;
namespace AdminActs
{
class Admin : Entity
{
public DailyActivity[] DailyActivities { get; set; }
}
class DailyActivity
{
public DateTime Time { get; set; }
public SubActivity[] SubActivities { get; set; }
}
class SubActivity
{
public DateTime EntryDate { get; set; }
public string Category { get; set; }
}
class Program
{
static void Main(string[] args)
{
new DB("activities");
var now = DateTime.Now;
var admin = new Admin
{
DailyActivities = new DailyActivity[]
{
new DailyActivity{
Time = now,
SubActivities = new SubActivity[]
{
new SubActivity{
Category ="Power Consumption",
EntryDate = DateTime.Now}
}
}
}
};
admin.Save();
var subActivities = admin.Collection()
.SelectMany(a => a.DailyActivities)
.Where(da => da.Time == now)
.SelectMany(da => da.SubActivities)
.Where(sa => sa.Category == "Power Consumption");
var res = subActivities.ToArray();
Console.ReadKey();
}
}
}