I have a list of parent elements that needs to be filtered based on the structure inside a child list.
Here is the data structure
"items": [
{
"id": "6691e62b-90as-43b2-k1l3-2fbf039295b5",
"details": {
"zone": {
"id": "cc07de83-3m21-1pp1-a123-a98bd8fb5fb8",
"name": "TestName",
}
},
}
]
var findresult = collection(f => f.items.Any(fb => fb.details.zone.name == "TestName")).ToList();
Expected that only items with the matching zone name is returned but I just returns everything in the db
to get parent documents that contain matching sub document, you need to use $elemMatch operator:
db.Parent.aggregate([
{
"$match": {
"items": {
"$elemMatch": {
"details.zone.name": "second zone"
}
}
}
}
])
if you want to get only the matching sub documents (items), you need $unwind and $project like so:
db.Parent.aggregate([
{
"$unwind": "$items"
},
{
"$project": {
"items": "$items",
"_id": NumberInt("0")
}
},
{
"$match": {
"items.details.zone.name": "second zone"
}
}
])
here's the c# code that generated above queries. code uses MongoDB.Entities for brevity. the querying part is the same for the official driver. just use collection.AsQueryable() in place of DB.Queryable<Parent>()
using MongoDB.Driver.Linq;
using MongoDB.Entities;
using System;
using System.Linq;
namespace StackOverflow
{
public class Program
{
public class Parent : Entity
{
public Item[] items { get; set; }
}
public class Item
{
public string id { get; set; }
public Detail details { get; set; }
}
public class Detail
{
public string id { get; set; }
public Zone zone { get; set; }
}
public class Zone
{
public string id { get; set; }
public string name { get; set; }
}
private static void Main(string[] args)
{
new DB("test");
(new[] {
new Parent {
items = new[]
{
new Item
{
id = Guid.NewGuid().ToString(),
details = new Detail
{
id = Guid.NewGuid().ToString(),
zone = new Zone
{
id = Guid.NewGuid().ToString(),
name = "first zone"
}
}
}
}
},
new Parent {
items = new[]
{
new Item
{
id = Guid.NewGuid().ToString(),
details = new Detail
{
id = Guid.NewGuid().ToString(),
zone = new Zone
{
id = Guid.NewGuid().ToString(),
name = "second zone"
}
}
}
}
}
}).Save();
//get all Parent entities that have 'second zone'
var result = DB.Queryable<Parent>()
.Where(p => p.items.Any(i => i.details.zone.name == "second zone"))
.ToArray();
//get only items that have 'second zone'
var items = DB.Queryable<Parent>()
.SelectMany(p => p.items)
.Where(i => i.details.zone.name == "second zone")
.ToArray();
}
}
}
update after comments:
in order to get parent and filter out non-matching child items, you need a projection stage with $elemMatch like so:
db.Parent.find({
"items": {
"$elemMatch": {
"details.zone.name": "second zone"
}
}
}, {
"_id": NumberInt("1"),
"items": {
"$elemMatch": {
"details.zone.name": "second zone"
}
}
})
c# MongoDB.Entities:
var result = DB.Find<Parent>()
.Match(p => p.items.Any(i => i.details.zone.name == "second zone"))
.Project(b =>
b.Include(p => p.ID)
.ElemMatch(p => p.items, i => i.details.zone.name == "second zone"))
.Execute();
c# official driver:
var projection = Builders<Parent>.Projection
.Include(p => p.ID)
.ElemMatch(p => p.items, i => i.details.zone.name == "second zone");
var result = collection.Find(p => p.items.Any(i => i.details.zone.name == "second zone"))
.Project(projection)
.ToList();
Related
Having this strange-looking json response:
{
"invoices":{
"0":{
"invoice":{
"id":"420",
"invoicecontents":{
"0":{
"invoicecontent":{
"name":"Here's the name of the content 0"
}
},
"1":{
"invoicecontent":{
"name":"Here's the name of the content 1"
}
}
}
}
},
"1":{
"invoice":{
"id":"420",
"invoicecontents":{
"0":{
"invoicecontent":{
"name":"Here's the name of the content 0"
}
}
}
}
},
"parameters":{
"limit":"3",
"page":"1",
"total":"420"
}
},
"status":{
"code":"OK"
}
}
How do I change the structure into this easy-to-deserialize one?
{
"invoices":[
{
"id":"420",
"invoicecontents":[
{
"name":"Here's the name of the content 0"
},
{
"name":"Here's the name of the content 1"
}
]
},
{
"id":"420",
"invoicecontents":[
{
"name":"Here's the name of the content 0"
}
]
}
]
}
I'd like to deserialize into List of Invoices as below
class Invoice {
public string Id { get; set; }
[JsonProperty("invoicecontents")]
public InvoiceContent[] Contents { get; set; }
class InvoiceContent {
public string Name { get; set; }
}
}
There's no problem with getting the status code or parameters, I simply do this:
var parsed = JObject.Parse(jsonInvoices);
var statusCode = parsed?["status"]?["code"]?.ToString();
var parameters = parsed?["invoices"]?["parameters"]?;
The real problem starts when I'm trying to achieve easy-to-deserialize json structure I've mentioned before. I've tried something like this:
var testInvoices = parsed?["invoices"]?
.SkipLast(1)
.Select(x => x.First?["invoice"]);
But I can't manage to "repair" invoicecontents/invoicecontent parts.
My goal is to deserialize and store the data.
This isn't JSON.
JavaScript Object Notation literally describes Objects. What you have here is the punchline of a joke that starts out with: "A SQL JOIN, a StringBuilder, and a couple for loops walk into a bar..."
As others have demonstrated, JObject is great for working with JSON that would be impractical to define as classes. You can use Linq to navigate through it, but JSONPath Expressions can be much simpler.
Example 1: Let's get the status code.
var status = parsed.SelectToken(".status.code").Value<string>();
Example 2: Let's 'deserialize' our invoices.
public string Invoice
{
public string Id { get; set; }
public List<string> Content { get; set; }
}
public List<Invoice> GetInvoices(string badJson)
{
var invoices = JObject.Parse(badJson).SelectTokens(".invoices.*.invoice");
var results = new List<Invoice>();
foreach (var invoice in invoices)
{
results.Add(new Invoice()
{
Id = invoice.Value<string>("id"),
Contents = invoice.SelectTokens(".invoicecontents.*.invoicecontent.name")
.Values<string>().ToList()
// Note: JToken.Value<T> & .Values<T>() return nullable types
});
}
return results;
}
try this
var jsonParsed = JObject.Parse(json);
var invoices = ((JObject) jsonParsed["invoices"]).Properties()
.Select(x => (JObject) x.Value["invoice"] ).Where(x => x != null)
.Select(s => new Invoice
{
Id = s.Properties().First(x => x.Name == "id").Value.ToString(),
Contents = ((JObject)s.Properties().First(x => x.Name == "invoicecontents").Value).Properties()
.Select(x => (string) x.Value["invoicecontent"]["name"]).ToList()
}).ToList();
and I simplified your class too, I don't see any sense to keep array of classes with one string, I think it should be just array of strings
public class Invoice
{
public string Id { get; set; }
public List<string> Contents { get; set; }
}
result
[
{
"Id": "420",
"Contents": [
"Here's the name of the content 0",
"Here's the name of the content 1"
]
},
{
"Id": "420",
"Contents": [
"Here's the name of the content 0"
]
}
]
I need to perform group-by with the $max operator in MongoDB. I figure out the query which is working in the MongoDB database but was not able to write the same using C#.
db.getCollection('Employee').aggregate(
[
{$unwind : "$Projects"},
{
"$group" : {
"_id" : "$EmpId",
"LastUpdated" : {"$max" : "$Projects.LastUpdated"}
}
}
]);
Below C# code is giving an error:
"Projects" is not valid property.
_db.GetCollection<BsonDocument>(collection).Aggregate().Unwind(i=>i.Projects)
Assume this is your sample data:
[{
"EmpId": 1,
"Projects": [
{
"LastUpdated": {
"$date": "2021-10-22T16:00:00Z"
}
},
{
"LastUpdated": {
"$date": "2021-11-07T16:00:00Z"
}
},
{
"LastUpdated": {
"$date": "2022-01-22T16:00:00Z"
}
}
]
}]
and this is your model class:
public class Employee
{
public int EmpId { get; set; }
public List<EmployeeProject> Projects { get; set; }
}
public class EmployeeProject
{
public DateTime UpdatedDate { get; set; }
}
To use the Projects property, you need to specify your collection as Project type as:
_db.GetCollection<Employee>("Employee")
Solution 1: Mix use of AggregateFluent and BsonDocument
var result = _db.GetCollection<Employee>("Employee")
.Aggregate()
.Unwind(i => i.Projects)
.Group(new BsonDocument
{
{ "_id", "$EmpId" },
{ "LastUpdated", new BsonDocument("$max", "$Projects.LastUpdated") }
})
.ToList();
Solution 2: Full use of AggregateFluent
Pre-requisites:
Need to create a model for unwinded Project.
public class UnwindEmployeeProject
{
public int EmpId { get; set; }
public EmployeeProject Projects { get; set; }
}
var result = _db.GetCollection<Employee>("Employee")
.Aggregate()
.Unwind<Employee, UnwindEmployeeProject>(i => i.Projects)
.Group(
k => k.EmpId,
g => new
{
EmpId = g.Key,
LastUpdated = g.Max(x => x.Projects.LastUpdated)
})
.ToList();
Solution 3: Full use of BsonDocument
With Mongo Compass, you can export your query to C#.
PipelineDefinition<Employee, BsonDocument> pipeline = new BsonDocument[]
{
new BsonDocument("$unwind", "$Projects"),
new BsonDocument("$group",
new BsonDocument
{
{ "_id", "$EmpId" },
{ "LastUpdated",
new BsonDocument("$max", "$Projects.LastUpdated") }
})
};
var result = _db.GetCollection<Employee>("Employee")
.Aggregate(pipeline)
.ToList();
Output
My Json Response is Following below:
{"d":
{"RowData":
[{"GenreId":11,"GenreName":"Musical","subjecturl":"subjecturl_1","logourl":"logourl_1"},
{"GenreId":12,"GenreName":"kids","subjecturl":"subjecturl_2","logourl":"logourl_2"},
{"GenreId":13,"GenreName":"other","subjecturl":"subjecturl_3","logourl":"logourl_3"},
{"GenreId":14,"GenreName":"Musical","subjecturl":"subjecturl_4","logourl":"logourl_4"},
{"GenreId":15,"GenreName":"Music","subjecturl":"subjecturl_5","logourl":"logourl_5"},
{"GenreId":16,"GenreName":"Faimaly","subjecturl":"subjecturl_6","logourl":"logourl_6"},
{"GenreId":17,"GenreName":"other","subjecturl":"subjecturl_7","logourl":"logourl_7"},
{"GenreId":18,"GenreName":"other","subjecturl":"subjecturl_8","logourl":"logourl_8"},
{"GenreId":19,"GenreName":"kids","subjecturl":"subjecturl_9","logourl":"logourl_9"},
{"GenreId":20,"GenreName":"Musical","subjecturl":"subjecturl_10","logourl":"logourl_10"},
{"GenreId":21,"GenreName":"other","subjecturl":"subjecturl_11","logourl":"logourl_11"}]}}
Using the above Response I tried to make like below Response :
{"rows": [{
"title": "Musical",
"items": [{"hdsubjecturl": "subjecturl_1"},{"hdsubjecturl": "subjecturl_4"},{"hdsubjecturl": "subjecturl_10"}]
},{
"title": "kids",
"items": [{"hdsubjecturl": "subjecturl_2"},{"hdsubjecturl": "subjecturl_9"}]
},{
"title": "Music",
"items": [{"hdsubjecturl": "subjecturl_5"}]
},{
"title": "other",
"items": [{"hdsubjecturl": "subjecturl_3"},{"hdsubjecturl": "subjecturl_7"},{"hdsubjecturl": "subjecturl_8"},{"hdsubjecturl": "subjecturl_11"}]
},{
"title": "Faimaly",
"items": [{"hdsubjecturl": "subjecturl_6"}]
}]
}
My Code is below :
JObject Root = JObject.Parse(jsonData["d"].ToString());
var unique = Root["RowData"].GroupBy(x => x["GenreName"]).Select(x => x.First()).ToList(); // here fetch 5 record
foreach (var un in unique)
{
var GenreName = new
{
title = un["GenreName"],
items = new
{
hdsubjecturl = "logourl"
}
};
var GenreNamereq = JsonConvert.SerializeObject(GenreName, Newtonsoft.Json.Formatting.Indented);
genstr.Append(GenreNamereq, 0, GenreNamereq.Length);
genstr.Append(",");
using (System.IO.StreamWriter file = new System.IO.StreamWriter(subdir + "\\GenreName.json"))
{
string st = genstr.ToString().Substring(0, (genstr.Length - 1));
file.WriteLine("{\n\"rows\": [\n" + st + "\n}"); //seasion number 21 terminate
file.Close();
}
}
Using above code my output is below for First Field :
{"rows":
[{
"title": "Musical",
"items": {
"hdsubjecturl": "logourl"
}
}]
}
Using below code I tried to fetch Multiple values using specific Field :
List<string> CategoryList = new List<string>();
var unique = Root["RowData"].GroupBy(x => x["GenreName"]).Select(x => x.First()).ToList(); // here fetch 8 record
foreach (var cat in unique)
{
CategoryList.Add(cat["GenreName"].ToString());
}
List<List<string>> myList = new List<List<string>>();
for (int i=0;i<CategoryList.Count();i++)
{
var results = from x in Root["RowData"]
where x["GenreName"].Value<string>() == CategoryList[i]
select x;
foreach (var token in results)
{
Console.WriteLine(token["logourl"]);
}
// myList.Add(results);
}
In the First code, I used JObject for fetching a Root node. But, using the above query it takes by default JTocken. So, I used foreach loop here.
I used Dictionary instances for JSON Creation. Using this code I successfully fetched hdsubjecturl in for loop. But, I don't know how to put multiple values in Dictionary instances. Because I get title fields only single times using a unique query and items fields inside a hdsubjetcurl is more than one. Does anyone know how it's possible?
You can group your RowData by GenreName token, then use ToDictionary method to get a result dictionary and map it to desired structure (with title and hdsubjecturl). Finally create a result object using JObject.FromObject
var json = JObject.Parse(jsonString);
var data = json["d"]?["RowData"]
.GroupBy(x => x["GenreName"], x => x["subjecturl"])
.ToDictionary(g => g.Key, g => g.ToList())
.Select(kvp => new { title = kvp.Key, items = kvp.Value.Select(x => new { hdsubjecturl = x }) });
var result = JObject.FromObject(new { rows = data });
Console.WriteLine(result);
It gives you the expected result.
Edit: according to comments, GroupBy and Select expressions should be updated to map more then one property in result title item
var data = json["d"]?["RowData"]
.GroupBy(x => x["GenreName"])
.ToDictionary(g => g.Key, g => g.ToList())
.Select(kvp => new
{
title = kvp.Key,
items = kvp.Value.Select(x => new { hdsubjecturl = x["subjecturl"], url = x["logourl"] })
});
var result = JObject.FromObject(new { rows = data });
Consider trying this code, (using Newtonsoft Json deserializer);
public partial class Root
{
public D D { get; set; }
}
public partial class D
{
public RowDatum[] RowData { get; set; }
}
public partial class RowDatum
{
public long GenreId { get; set; }
public string GenreName { get; set; }
public string Subjecturl { get; set; }
public string Logourl { get; set; }
}
public partial class Response
{
public Row[] Rows { get; set; }
}
public partial class Row
{
public string Title { get; set; }
public Item[] Items { get; set; }
}
public partial class Item
{
public string Hdsubjecturl { get; set; }
}
public class Program
{
public static void Main(string[] args)
{
var json =
#"{""d"":{""RowData"":[{""GenreId"":11,""GenreName"":""Musical"",""subjecturl"":""subjecturl_1"",""logourl"":""logourl_1""},{""GenreId"":12,""GenreName"":""kids"",""subjecturl"":""subjecturl_2"",""logourl"":""logourl_2""},{""GenreId"":13,""GenreName"":""other"",""subjecturl"":""subjecturl_3"",""logourl"":""logourl_3""},{""GenreId"":14,""GenreName"":""Musical"",""subjecturl"":""subjecturl_4"",""logourl"":""logourl_4""},{""GenreId"":15,""GenreName"":""Music"",""subjecturl"":""subjecturl_5"",""logourl"":""logourl_5""},{""GenreId"":16,""GenreName"":""Faimaly"",""subjecturl"":""subjecturl_6"",""logourl"":""logourl_6""},{""GenreId"":17,""GenreName"":""other"",""subjecturl"":""subjecturl_7"",""logourl"":""logourl_7""},{""GenreId"":18,""GenreName"":""other"",""subjecturl"":""subjecturl_8"",""logourl"":""logourl_8""},{""GenreId"":19,""GenreName"":""kids"",""subjecturl"":""subjecturl_9"",""logourl"":""logourl_9""},{""GenreId"":20,""GenreName"":""Musical"",""subjecturl"":""subjecturl_10"",""logourl"":""logourl_10""},{""GenreId"":21,""GenreName"":""other"",""subjecturl"":""subjecturl_11"",""logourl"":""logourl_11""}]}}";
var root = JsonConvert.DeserializeObject<Root>(json);
var rows = root.D.RowData.ToLookup(d => d.GenreName)
.Select(g => new Row()
{
Title = g.Key,
Items = g.ToList().Select(rd => new Item() {Hdsubjecturl = rd.Logourl}).ToArray()
}).ToArray();
var response = new Response()
{
Rows = rows
}; // reponse is the type of Json Response you wanted to achieve
Console.WriteLine();
}
}
I am using c#,along with MongoDB.
I have a class that can be resembled by this.
Its a sample, that represents something, please dont comment on the class design
[CollectionName("Venues")]
public class Venue
{
public string Name { get; set; }
public dictionary<string,object> Properties { get; set; }
}
var venue = new Venue
{
Name = "Venue 1",
Properties = new Dictionary<string,object>
{
{ "Chairs", "18" },
{ "Tables", "4" },
{ "HasWaterfall", true }
}
}
Assuming I have an object in a collection, that looks like that.
I would like to find out of it is possible to do two things.
1: Load from the database, only a single item from the dictionary,
currently I can only see how this can be done, by loading the entire
record from the database and then manually getting the value by key.
2: Determine the average of a single item within the database.
For example, across all records I would like to work out the average
chairs, again without loading all records and then doing it in memory with
linq etc....
Basically your sample document gets stored as a below JSON:
{
"_id" : ObjectId("..."),
"Name" : "Venue 1",
"Properties" : {
"Chairs" : "18",
"Tables" : "4",
"HasWaterfall" : true
}
}
This gives you a possibility to define a projection using dot notation:
var filter = Builders<Venue>.Filter.Eq(f => f.Name, "Venue 1");
var projection = Builders<Venue>.Projection.Include("Properties.Chairs");
List<BsonDocument> data = Col.Find(filter).Project(projection).ToList();
which returns below following BsonDocument:
{ "_id" : ObjectId("..."), "Properties" : { "Chairs" : "18" } }
To get the average you need to use $toInt operator introduced in MongoDB 4.0 to convert your values from string to int. Try:
var project = new BsonDocument()
{
{ "chairs", new BsonDocument() { { "$toInt", "$Properties.Chairs" } } }
};
var group = new BsonDocument()
{
{ "_id", "null" },
{ "avg", new BsonDocument() { { "$avg", "$chairs" } } }
};
var avg = Col.Aggregate().Project(project).Group(group).First();
here's an alternative way of doing it using MongoDB.Entities convenience library.
using System.Collections.Generic;
using System.Linq;
using MongoDB.Entities;
namespace StackOverflow
{
class Program
{
[Name("Venues")]
public class Venue : Entity
{
public string Name { get; set; }
public Dictionary<string, object> Properties { get; set; }
}
static void Main(string[] args)
{
new DB("test");
var venue1 = new Venue
{
Name = "Venue 1",
Properties = new Dictionary<string, object> {
{ "Chairs", 28 },
{ "Tables", 4 },
{ "HasWaterfall", true }
}
};
venue1.Save();
var venue2 = new Venue
{
Name = "Venue 2",
Properties = new Dictionary<string, object> {
{ "Chairs", 38 },
{ "Tables", 4 },
{ "HasWaterfall", true }
}
};
venue2.Save();
var chairs = DB.Find<Venue, object>()
.Match(v => v.Name == "Venue 1")
.Project(v => new { ChairCount = v.Properties["Chairs"] })
.Execute();
var avgChairs = DB.Collection<Venue>()
.Average(v => (int)v.Properties["Chairs"]);
}
}
}
results in the following queries being made to the database:
getting chairs in venue 1:
db.runCommand({
"find": "Venues",
"filter": {
"Name": "Venue 1"
},
"projection": {
"Properties.Chairs": NumberInt("1"),
"_id": NumberInt("0")
},
"$db": "test"
})
getting average chair count across all venues:
db.Venues.aggregate([
{
"$group": {
"_id": NumberInt("1"),
"__result": {
"$avg": "$Properties.Chairs"
}
}
}
])
I'm having difficulties querying a Mongo-DB collection.
The document I'm using
public class Customer
{
public ObjectId Id { get; set; }
public int CustomerId { get; set; }
public List<Address> Addresses { get; set; }
}
public class Address
{
public string Name { get; set; }
}
Some sample-Data
{
CustomerId: 2,
Addresses: [
{
Name: "Daniel"
},
{
Name: "Eric"
},
{
Name: "Dan"
}
]
}
I now want to query the documents by the CustomerId and filter some of the Addresses-values to return all Addresses with a Name like %Dan%. As the address-collection can be huge I want to already filter those addresses at query-time
var mdb = mongoClient.GetDatabase("test");
var collection = mdb.GetCollection<WebApi>("Customer");
var builder = Builders<Customer>.Filter;
var searchTerm = "Dan";
When querying like that it works but it holds all addresses:
var filter = builder.And(builder.Eq("InstallationId", 2),
builder.Regex("Addresses.Name", new BsonRegularExpression(".*" + searchTerm + ".*", "i")))
var result = collection.Find(filter).FirstOrDefault();
What I'd like to get is:
[
{
Name: "Daniel"
},
{
Name: "Dan"
}
]
or
{
CustomerId: 2,
Addresses: [
{
Name: "Daniel"
},
{
Name: "Dan"
}
]
}
I also tried several approaches with Aggregate/Match/Project but can't get it to work. "AsQueryable" also does not work as IndexOf is not implemented on the driver.
collection.AsQueryable().Where(x => x.CustomId == 2).SelectMany(x =>
x.Addresses.Where(a => a.Name.IndexOf(searchTerm, StringComparison.InvariantCultureIgnoreCase) >= 0).ToList();
Used Versions:
MongoDB.Driver v 2.6.1 (also Bson)
.Net Framework 4.5.2
This should get you going:
var result = collection
.Aggregate()
.Match(c => c.CustomerId == 2)
.Project(c => new
{
c.CustomerId,
Addresses = c.Addresses.Where(a => a.Name.IndexOf(searchTerm) != -1)
})
.ToList();
The driver will translate this into:
db.Customer.aggregate([{
$match: { CustomerId: 2 }
}, {
$project: {
CustomerId: "$CustomerId",
Addresses: {
$filter: {
input: "$Addresses",
as: "a",
cond: {
$ne: [ { $indexOfBytes: [ "$$a.Name", "Dan" ] }, -1 ]
}
}
},
_id: 0
}
}])
For the case-insensitive version, there will be a the option to use $regex at some stage in the future (see https://jira.mongodb.org/browse/SERVER-11947). However, what you can do just now is use ToUpper():
var result = collection
.Aggregate()
.Match(c => c.CustomerId == 2)
.Project(c => new
{
c.CustomerId,
Addresses = c.Addresses.Where(a => a.Name.ToUpper().IndexOf(searchTerm.ToUpper()) != -1)
})
.ToList();