I need to perform group-by with the $max operator in MongoDB. I figure out the query which is working in the MongoDB database but was not able to write the same using C#.
db.getCollection('Employee').aggregate(
[
{$unwind : "$Projects"},
{
"$group" : {
"_id" : "$EmpId",
"LastUpdated" : {"$max" : "$Projects.LastUpdated"}
}
}
]);
Below C# code is giving an error:
"Projects" is not valid property.
_db.GetCollection<BsonDocument>(collection).Aggregate().Unwind(i=>i.Projects)
Assume this is your sample data:
[{
"EmpId": 1,
"Projects": [
{
"LastUpdated": {
"$date": "2021-10-22T16:00:00Z"
}
},
{
"LastUpdated": {
"$date": "2021-11-07T16:00:00Z"
}
},
{
"LastUpdated": {
"$date": "2022-01-22T16:00:00Z"
}
}
]
}]
and this is your model class:
public class Employee
{
public int EmpId { get; set; }
public List<EmployeeProject> Projects { get; set; }
}
public class EmployeeProject
{
public DateTime UpdatedDate { get; set; }
}
To use the Projects property, you need to specify your collection as Project type as:
_db.GetCollection<Employee>("Employee")
Solution 1: Mix use of AggregateFluent and BsonDocument
var result = _db.GetCollection<Employee>("Employee")
.Aggregate()
.Unwind(i => i.Projects)
.Group(new BsonDocument
{
{ "_id", "$EmpId" },
{ "LastUpdated", new BsonDocument("$max", "$Projects.LastUpdated") }
})
.ToList();
Solution 2: Full use of AggregateFluent
Pre-requisites:
Need to create a model for unwinded Project.
public class UnwindEmployeeProject
{
public int EmpId { get; set; }
public EmployeeProject Projects { get; set; }
}
var result = _db.GetCollection<Employee>("Employee")
.Aggregate()
.Unwind<Employee, UnwindEmployeeProject>(i => i.Projects)
.Group(
k => k.EmpId,
g => new
{
EmpId = g.Key,
LastUpdated = g.Max(x => x.Projects.LastUpdated)
})
.ToList();
Solution 3: Full use of BsonDocument
With Mongo Compass, you can export your query to C#.
PipelineDefinition<Employee, BsonDocument> pipeline = new BsonDocument[]
{
new BsonDocument("$unwind", "$Projects"),
new BsonDocument("$group",
new BsonDocument
{
{ "_id", "$EmpId" },
{ "LastUpdated",
new BsonDocument("$max", "$Projects.LastUpdated") }
})
};
var result = _db.GetCollection<Employee>("Employee")
.Aggregate(pipeline)
.ToList();
Output
Related
I am using .net core 5 web api project. I have following classes:
public class GroupedShowbackSummaryListDto
{
public int RuleId { get; set; }
public string RuleName { get; set; }
public decimal TotalPrice { get; set; }
public List<GroupedProjectForShowbackSummary> Projects { get; set; }
}
public class GroupedProjectForShowbackSummary
{
public int? Id { get; set; }
public string Name { get; set; }
public decimal TotalPrice { get; set; }
}
I would like group by rule id and show list of projects as response, I am trying:
var queryable = _context.ShowbackSummaries.Where(x => x.ProjectId != null).AsQueryable();
var summary = queryable.ToList();
var grouped = summary.GroupBy(x => x.ShowbackRuleId).Select(c => new GroupedShowbackSummaryListDto
{
RuleId = c.Key,
RuleName = c.Select(f => f.ShowbackRule.Name).First(),
TotalPrice = c.Select(f => f.Price).Sum(),
Projects = new List<GroupedProjectForShowbackSummary>
{
new()
{
Id = c.Select(f => f.ProjectId).First(),
Name = c.Select(f => f.Project.Name).First(),
TotalPrice = c.Select(f => f.Price).First()
}
}
}).ToList();
return grouped;
I know I am using first and it returns only first project but I would like to return all, if I switch to list int id for ex, it will show me:
[
100,
101,
...
]
I would like multiple result for my current response:
[
{
"ruleId": 1,
"ruleName": "rule-1",
"totalPrice": 400,
"projects": [
{
"id": 1169,
"name": "lubos-cncf",
"totalPrice": 200
}
]
},
{
"ruleId": 2,
"ruleName": "rule-2",
"totalPrice": 300,
"projects": [
{
"id": 1169,
"name": "lubos-cncf",
"totalPrice": 300
}
]
}
]
P.S. Get all projects like this.
This should work (inside your GroupBy-Expression):
Projects = c.Select(f =>
new GroupedProjectForShowbackSummary()
{
Id = f.ProjectId,
Name = f.Project.Name,
TotalPrice = f.Price
}).ToList()
I have the following model of the collection "evaluations":
[BsonCollection("evaluations")]
public class Evaluation : Document
{
[BsonRepresentation(BsonType.ObjectId)]
public string AlertId { get; set; }
public string EvaluationStatus { get; set; }
public DateTime EvaluatedAt { get; set; }
}
What I would like to do is to retrieve a list with the last evaluation of each Alert object, which would mean grouping evaluations by AlertId then getting the latest EvaluatedAt of the group, and then return those evaluations.
With the result being a list of 'Evaluation' objects, (one evaluation per AlertId)
How would I construct that query using MongoDB driver with Aggregation?
Thanks for the help!
With a little research came up with this solution:
var alertIds = alertss.Select(x => x.Id.ToString()).ToList();
var sortByLatest = Builders<Evaluation>.Sort.Descending(x => x.EvaluatedAt);
var groupByAlertId = new BsonDocument
{
{ "_id", "$AlertId" },
{ "EvaluationId", new BsonDocument { { "$first", "$_id" } } },
{ "EvaluationStatus", new BsonDocument { { "$first", "$EvaluationStatus" } } },
{ "EvaluatedAt", new BsonDocument { { "$first", "$EvaluatedAt" } } },
};
ProjectionDefinition<BsonDocument> projection = new BsonDocument
{
{"_id", "$EvaluationId"},
{"AlertId", "$_id"},
{"EvaluationStatus", "$EvaluationStatus"},
{"EvaluatedAt", "$EvaluatedAt"},
};
var evaluations = evaluationsRepository.GetCollection().Aggregate()
.Match(x => alertIds.Contains(x.AlertId))
.Sort(sortByLatest)
.Group(groupByAlertId)
.Project(projection)
.As<Evaluation>().ToList();
return evaluations;
I am using c#,along with MongoDB.
I have a class that can be resembled by this.
Its a sample, that represents something, please dont comment on the class design
[CollectionName("Venues")]
public class Venue
{
public string Name { get; set; }
public dictionary<string,object> Properties { get; set; }
}
var venue = new Venue
{
Name = "Venue 1",
Properties = new Dictionary<string,object>
{
{ "Chairs", "18" },
{ "Tables", "4" },
{ "HasWaterfall", true }
}
}
Assuming I have an object in a collection, that looks like that.
I would like to find out of it is possible to do two things.
1: Load from the database, only a single item from the dictionary,
currently I can only see how this can be done, by loading the entire
record from the database and then manually getting the value by key.
2: Determine the average of a single item within the database.
For example, across all records I would like to work out the average
chairs, again without loading all records and then doing it in memory with
linq etc....
Basically your sample document gets stored as a below JSON:
{
"_id" : ObjectId("..."),
"Name" : "Venue 1",
"Properties" : {
"Chairs" : "18",
"Tables" : "4",
"HasWaterfall" : true
}
}
This gives you a possibility to define a projection using dot notation:
var filter = Builders<Venue>.Filter.Eq(f => f.Name, "Venue 1");
var projection = Builders<Venue>.Projection.Include("Properties.Chairs");
List<BsonDocument> data = Col.Find(filter).Project(projection).ToList();
which returns below following BsonDocument:
{ "_id" : ObjectId("..."), "Properties" : { "Chairs" : "18" } }
To get the average you need to use $toInt operator introduced in MongoDB 4.0 to convert your values from string to int. Try:
var project = new BsonDocument()
{
{ "chairs", new BsonDocument() { { "$toInt", "$Properties.Chairs" } } }
};
var group = new BsonDocument()
{
{ "_id", "null" },
{ "avg", new BsonDocument() { { "$avg", "$chairs" } } }
};
var avg = Col.Aggregate().Project(project).Group(group).First();
here's an alternative way of doing it using MongoDB.Entities convenience library.
using System.Collections.Generic;
using System.Linq;
using MongoDB.Entities;
namespace StackOverflow
{
class Program
{
[Name("Venues")]
public class Venue : Entity
{
public string Name { get; set; }
public Dictionary<string, object> Properties { get; set; }
}
static void Main(string[] args)
{
new DB("test");
var venue1 = new Venue
{
Name = "Venue 1",
Properties = new Dictionary<string, object> {
{ "Chairs", 28 },
{ "Tables", 4 },
{ "HasWaterfall", true }
}
};
venue1.Save();
var venue2 = new Venue
{
Name = "Venue 2",
Properties = new Dictionary<string, object> {
{ "Chairs", 38 },
{ "Tables", 4 },
{ "HasWaterfall", true }
}
};
venue2.Save();
var chairs = DB.Find<Venue, object>()
.Match(v => v.Name == "Venue 1")
.Project(v => new { ChairCount = v.Properties["Chairs"] })
.Execute();
var avgChairs = DB.Collection<Venue>()
.Average(v => (int)v.Properties["Chairs"]);
}
}
}
results in the following queries being made to the database:
getting chairs in venue 1:
db.runCommand({
"find": "Venues",
"filter": {
"Name": "Venue 1"
},
"projection": {
"Properties.Chairs": NumberInt("1"),
"_id": NumberInt("0")
},
"$db": "test"
})
getting average chair count across all venues:
db.Venues.aggregate([
{
"$group": {
"_id": NumberInt("1"),
"__result": {
"$avg": "$Properties.Chairs"
}
}
}
])
One of the Mongo objects I use (referred to as 'Admin' here) uses GUID as the primary key and has a list of 'DailyActivities' which contains datevalue and another list called 'Subactivities'. The Admin object looks like something below. I am struggling to find resources in C# that will help extract DailyActivities that only corresponds to a particular date with Subactivities that has a category of 'Power Consumption'.
{
"_id" : ObjectId("5a2b7b887df7ce464404dc7d"),
"DailyActivities" : [
{
"datetime" : ISODate("2017-12-09T16:29:00.916Z"),
"Subactivities" : [
{
"entryDate" : ISODate("2017-12-09T06:30:26.658Z"),
"category" : "Power Consumption"
},
{
"entryDate" : ISODate("2017-12-09T06:30:26.658Z"),
"category" : "Machinery"
}
]
},
{
"datetime" : ISODate("2017-12-13T00:00:00.916Z"),
"Subactivities" : [
{
"entryDate" : ISODate("2017-12-13T06:30:26.658Z"),
"category" : "Lamination"
}
]
}
]
}
The result I would like to receive should be:
{
"_id" : ObjectId("5c7044f07ef75175b2b8efd6"),
"entryDate" : ISODate("2017-12-09T06:30:26.658Z"),
"category" : "Power Consumption"
}
Right now, I don't have the time to complete this exercise but here is something to get you going. I shall edit and improve this (as in convert it into some c# a lot of which won't be possible using a typed approach...) next week.
db.collection.aggregate([{
$project: {
"DailyActivities": {
$filter: {
input: "$DailyActivities",
cond: {
$eq: [ "$$this.datetime", ISODate("2017-12-09T16:29:00.916Z") ]
}
}
}
}
}, {
$unwind: "$DailyActivities"
}, {
$unwind: "$DailyActivities.Subactivities"
}, {
$replaceRoot: {
"newRoot": "$DailyActivities.Subactivities"
}
}, {
$match: {
"category": "Power Consumption"
}
}])
let me offer you a solution which uses MongoDAL as the data access layer. it's a wrapper around the c# driver so you get all the features of the driver plus a highly typed api.
using System;
using System.Linq;
using MongoDAL;
namespace AdminActs
{
class Admin : Entity
{
public DailyActivity[] DailyActivities { get; set; }
}
class DailyActivity
{
public DateTime Time { get; set; }
public SubActivity[] SubActivities { get; set; }
}
class SubActivity
{
public DateTime EntryDate { get; set; }
public string Category { get; set; }
}
class Program
{
static void Main(string[] args)
{
new DB("activities");
var now = DateTime.Now;
var admin = new Admin
{
DailyActivities = new DailyActivity[]
{
new DailyActivity{
Time = now,
SubActivities = new SubActivity[]
{
new SubActivity{
Category ="Power Consumption",
EntryDate = DateTime.Now}
}
}
}
};
admin.Save();
var subActivities = admin.Collection()
.SelectMany(a => a.DailyActivities)
.Where(da => da.Time == now)
.SelectMany(da => da.SubActivities)
.Where(sa => sa.Category == "Power Consumption");
var res = subActivities.ToArray();
Console.ReadKey();
}
}
}
First of all, I am using NEST 5.5.0.
I have the following use of a remote elasticsearch-index:
var node = new Uri("http://distribution.virk.dk/cvr-permanent");
var settings = new
ConnectionSettings(node).DefaultIndex("virksomhed");
settings.BasicAuthentication("username", "password");
var client = new ElasticClient(settings);
var searchResponse = client.Search<Company>(s => s
.AllTypes().Query(q => q
.Match(m => m
.Field(f => f.cvrNumber)
.Query("35954716")
)
)
);
The mappings in the index (without a bunch of other properties besides cvrNummer) are as follows:
{
"cvr-permanent-prod-20170205" : {
"mappings" : {
"virksomhed" : {
"_size" : {
"enabled" : true
},
"properties" : {
"Vrvirksomhed" : {
"properties" : {
"type" : "long"
},
"cvrNummer" : {
"type" : "string"
},
}
}
},
}
}
}
}
}
I also have the following class which the result is supposed to be mapped to:
[ElasticsearchType(Name = "virksomhed")]
public class Company
{
[Text(Name = "Vrvirksomhed.cvrNummer")]
public string cvrNumber { get; set; }
}
Now, the search (searchResponse) holds the expected results (1 result), where the part concerning cvrNummer looks as follows:
"hits": {
"total": 1,
"max_score": 17.34601,
"hits": [
{
"_index": "cvr-permanent-prod-20170205",
"_type": "virksomhed",
"_id": "4000333383",
"_score": 17.34601,
"_source": {
"Vrvirksomhed": {
"cvrNummer": 35954716,
"regNummer": [
{
"regnummer": "A/S35855",
"periode": {
"gyldigFra": "1956-06-01",
"gyldigTil": "1999-10-18"
},
"sidstOpdateret": "2015-02-10T00:00:00.000+01:00"
}
],
"brancheAnsvarskode": null,
"reklamebeskyttet": false,
"navne": [
...
However, when i look in searchResponse.Documents, I have the correct type (Company), but the value of cvrNumber is null.
Any ideas what I'm doing wrong, since the value of cvrNummer is not mapped into cvrNumber on the instance of Company in searchResponse.Documents?
Thanks in advance for your input!
UPDATE
I tried the following without success, still got the expected result, but cvrNumber is still null (in searchResponse.Documents):
[ElasticsearchType(Name = "virksomhed")]
public class Company
{
[Object(Name = "Vrvirksomhed")]
public Vrvirksomhed Vrvirksomhed { get; set; }
}
public class Vrvirksomhed
{
[Text(Name = "cvrNummer")]
public string cvrNumber { get; set; }
}
With the query:
var searchResponse = client.Search<Vrvirksomhed>(s => s
.AllTypes().Query(q => q
.Match(m => m
.Field(f => f.cvrNumber)
.Query("35954716")
)
)
);
UPDATE
It works with the following modifications to the query:
var searchResponse = client.Search<Company>(s => s
.AllTypes().Query(q => q
.Match(m => m
.Field(f => f.Vrvirksomhed.cvrNumber)
.Query("35954716")
)
)
);
[ElasticsearchType(Name = "virksomhed")]
public class Company
{
[Text(Name = "Vrvirksomhed.cvrNummer")]
public string cvrNumber { get; set; }
}
Vrvirksomhed looks like it should be a POCO property on Company mapped either as an object datatype or nested datatype (take a look at nested objects in the Definitive Guide for the differences), where that POCO has a property called cvrNumber, similar to
[ElasticsearchType(Name = "virksomhed")]
public class Company
{
[Object(Name = "Vrvirksomhed")]
public Vrvirksomhed Vrvirksomhed { get; set; }
}
public class Vrvirksomhed
{
[Text(Name = "cvrNummer")]
public string cvrNumber { get; set; }
}