I'm trying to build aggregate pipeline with value search according to the value with the C# mongo driver.
The first stage would to filter where KeyA is "A" and then the second stage would be to do the search where I have a certain list of keys that I need to go over and compare their value with a regex, according to documentation the regex only checks strings and I need to check any type that the value can hold.
I'm adding a model and code examples for what I have done so far but couldn't get to the expected end result, any help would be appriciated.
Model for example:
{
"_id" : ObjectId(),
"KeyA" : "A",
"KeyB" : NumberInt(5),
"KeyNestedObj" : [
{
"KeyC" : "C"
},
{
"KeyD" : "D"
},
{
"KeyE" : NumberInt(5)
}
]
}
Mongo shell query:
db.MongoTests.aggregate([{ "$match": { "KeyA": "A" }},
{ "$match": { "$or": [{ "KeyNestedObj": { "$elemMatch": { "KeyC": /^.*?Regex.*?$/ } } }, { "KeyNestedObj": { "$elemMatch": { "KeyE": /^.*?Regex.*?$/ }}}]}}])
C# query build:
var searchableFields = new List<SearchableField>
{
new SearchableField{ Field = "KeyC" },
new SearchableField{ Field = "KeyE" }
};
var searchableFieldsFilter = new List<FilterDefinition<Model>>();
var commonSearchBuilder = Builders<Model>.Filter;
var nestedObjSearchBuilder = Builders<KeyNestedObj>.Filter;
var searchValueRegex = new BsonRegularExpression($"^.*?{searchValue}.*?$");
foreach (var searchableField in searchableFields)
{
var searchNestedField =
commonSearchBuilder.ElemMatch(x => x.KeyNestedObj,
nestedObjSearchBuilder.Regex(searchableField.Field, searchValueRegex));
searchableFieldsFilter.Add(searchNestedField);
}
return Builders<ActivityDataModel>.Filter.And(searchableFieldsFilter);
Code aggregation query:
var renderedSearchQuery = searchStage.Render(mongoContext.Data.DocumentSerializer,
mongoContext.Data.Settings.SerializerRegistry);
var results= mongoContext.Data.Aggregate().Match(filterQuery).AppendStage<Model>(renderedSearchQuery).ToListAsync();
Related
Lets say I am having two collections named CollectionA and CollectionB, Both collection have different fields.
CollectionA will have multiple documents with Same field,
CollectionB contain only one document
Example
CollectionA
{
"UniqeId" :1,
"Hobbies" : "Eating"
},
{
"UniqeId" :2,
"Hobbies" : "Sleeping"
},
{
"UniqeId" :3,
"Hobbies" : "Walking"
}
CollectionB
{
"UserName" :"Sukuna",
"UserType" : "Villan"
}
I want output like this
{
"UniqeId" :1,
"Hobbies" : "Eating",
"UserName" :"Sukuna",
"UserType" : "Villan"
}
Consider All the documents in a CollectionA will contain same fields
And you can see there is no Unique fields between the two collection, and you can see we need to apply filter in CollectionA
ie) UniqeId=1
I am using C#, and I can able do two DB request to get those collection details (One req for CollectionA output and another one for CollectionB output) and manage to combine both in API level to get the desired output, but I want to do in DB level itself,
I don't want two DB calls, that is eating the API performance, so is there anyway to achieve this in a single DB call or by using any aggregate pipeline?.
Thanks in Advance
lookup with localField 1 and foreignField 1
db.a.aggregate([
{
$lookup: {
from: "b",
localField: "1",
foreignField: "1",
as: "docs"
}
},
{
$replaceRoot: {
newRoot: {
$mergeObjects: [
"$$ROOT",
{ $first: "$docs" }
]
}
}
},
{
$unset: [ "docs", "_id" ]
},
{
$group: {
_id: "$UserName",
doc: { $first: "$$ROOT" }
}
},
{
$replaceWith: "$doc"
}
])
mongoplayground
Finally after lots of trail and error and playing with pipeline, I can able to do that in aggregate pipeline, and I am using unionWith and group.
Here is the C# code
var pipeline1= new BsonDocument("$unionWith",
new BsonDocument
{
{ "coll", "CollectionB" },
{ "pipeline",
new BsonArray
{
new BsonDocument("$match",
new BsonDocument("UniqeId", 1))
} }
});
var pipeline2 = new BsonDocument("$group",
new BsonDocument
{
{ "_id", 0 },
{ "merged",
new BsonDocument("$push", "$$ROOT") }
});
var pipeline3 = new BsonDocument("$replaceRoot",
new BsonDocument("newRoot",
new BsonDocument("$mergeObjects", "$merged")));
var pipeline4 = new BsonDocument("$project",
new BsonDocument("_id", 0));
BsonDocument[] pipeline = new BsonDocument[] { pipeline1, pipeline2, pipeline3, pipeline4 };
var dbResponse = await collection.Aggregate<BsonDocument>(pipeline).ToListAsync();
Is there any way to use Filters in C# and translate this mongo query?
{'EmailData.Attachments.Files': {$all: [{Name: 'a.txt'},{Name: 'b.txt'},{Name:'c.txt'}], $size: 3}}
my data model is like:
{
"_id": ObjectId("5f0a9c07b001406068c073c1"),
"EmailData" : [
{
"Attachments" : {
"Files" : [
{
"Name" : "a.txt"
},
{
"Name" : "b.txt"
},
{
"Name" : "c.txt"
}
]
}
}
]
}
I have something like this in my mind:
var Filter =
Builders<EmailEntity>.Filter.All(s => s.EmailData????);
or something like:
var Filter =
Builders<EmailEntity>.Filter.ElemMatch(s => s.EmailData???)
I was wondering is there any way in the above filter to use All inside ElemMatch?
The difficulty here is that EmailData.Attachments.Files is an array within another array so C# compiler will get lost when you try to use Expression Trees.
Thankfully there's another approach when you need to define a field using MongoDB .NET driver. You can take advantage of StringFieldDefinition<T> class.
Try:
var files = new[] { new FileData(){ Name = "a.txt"}, new FileData() { Name = "b.txt" }, new FileData() { Name = "c.txt" } };
FieldDefinition<EmailEntity> fieldDef = new StringFieldDefinition<EmailEntity>("EmailData.Attachments.Files");
var filter = Builders<EmailEntity>.Filter.And(
Builders<EmailEntity>.Filter.All(fieldDef, files),
Builders<EmailEntity>.Filter.Size(fieldDef, 3));
var result= collection.Find(filter).ToList();
I need to run the following query in MongoDB in my C# code. I use MongoDB.Driver 2.7.2 and MongoDB 4.0.
I would like to use $lookup rather than $project / $unwind in order to prevent naming each one of the collection's possible fields.
db.getCollection('Collection1').aggregate([
{ "$match" : { "Id" : 1 } },
{ "$lookup": {
"from": "Collection2",
"let": { collection1Id : "$Id" },
"pipeline": [
{ $match:
{ $expr:
{ $and:
[
{ $eq : [ "$Collection1Id", "$$collection2Id" ] },
{ $in : [ "$_id" , [1, 2, 3]] }
]
}
}
}
],
"as": "item"
}
}
])
I expect the output of Collection1 joined with Collection2 under multiple join conditions.
As you didn't specify a particular class mapping and example documents, below answer will use BsonDocument type and example data from the manual $lookup: specify multiple join conditions with lookup
BsonArray subpipeline = new BsonArray();
subpipeline.Add(
new BsonDocument("$match",new BsonDocument(
"$expr", new BsonDocument(
"$and", new BsonArray {
new BsonDocument("$eq", new BsonArray{"$stock_item", "$$order_item"} ),
new BsonDocument("$gte", new BsonArray{"$instock", "$$order_qty"} )
}
)
))
);
var lookup = new BsonDocument("$lookup",
new BsonDocument("from", "warehouses")
.Add("let",
new BsonDocument("order_item", "$item")
.Add("order_qty", "$ordered"))
.Add("pipeline", subpipeline)
.Add("as", "stockdata")
);
var results = collection.Aggregate()
.Match(new BsonDocument("_id", 1))
.AppendStage<BsonDocument>(lookup).ToEnumerable();
foreach (var x in results)
{
Console.WriteLine(x.ToJson());
}
Please note that support for more expressive $lookup using PipelineDefinitionBuilder is coming for version 2.8.x. For more information see CSHARP-2013
I am trying to create a console application which will read through a Mongo db and filter out some data based on some business logic and display the results on console screen.
Till now I am able to write the following query and execute it successfully on Mongo shell. Here is the query:
{db.Collection.aggregate([
{$unwind: { path: "$Pages"}},
{$group : {
_id :{UrlPath: "$Pages.Url.Path", I_Id : "$_id", Date: { $dateToString: { format: "%Y-%m-%d",date: "$Pages.DateTime"}}, CId: "$CId",
x: {$sum:1},
y : {$sum:"$V"}
}},
{$group : {
_id : {Date: "$_id.Date",CId: "$_id.CId", PageUrl: "$_id.UrlPath"},
p: {$sum:1},
q : {$sum:"$x"},
TotalCount: {$sum:"$y"}
}},
{$sort:{p:-1}},
],{allowDiskUse: true}).pretty();
};
The problem I am facing is replicating the same query logic in C# code. Till now I able to connect to mongo db collections and able to basic CRUD operations.
But replicating this aggregate logic is really blowing my mind. I have tried using PipelineDefinition option but can't manage to get correct output out of it.
Can some one please guide me in the correct direction here.
Thanks in advance
Finally I was able to achieve it using collection.Aggregate() method.
Code looks like this:
collection.Aggregate(new AggregateOptions { AllowDiskUse = true})
.Unwind(i => i.Pages)
.Group(new BsonDocument
{
{
"_id", new BsonDocument
{
{ "Date", new BsonDocument("$dateToString", new BsonDocument {{"format", "%Y-%m-%d"}, {"date", "$Pages.DateTime"}})},
{"CId","$_id"}
}
},
{
"x", new BsonDocument
{
{"$sum", "$Value"}
}
}
}
)
.Group(new BsonDocument
{
{
"_id", new BsonDocument
{
{"Date", "$_id.Date"},
{"CId", "$_id.CId"}
}
},
{
"p", new BsonDocument
{
{"$sum", 1}
}
},
{
"q", new BsonDocument
{
{"$sum", "$x"}
}
}
}
)
.Sort(new BsonDocument("q", -1))
.ToList();
I am doing a NearSphere query with the C# MongoDB driver 2.0 and it works fine.
The results are ordered by distance automatically but I would like to get that distance back for each of the search results to be able to display it back.
I found this post that says how to do it for the old version of the driver Retrieving the Distance "dis" result from a Near query but didn't manage to find how to do it with the new drivers.
This is my code:
var collection = database.GetCollection<MyType>("myTypes");
var locFilter = Builders<MyType>.Filter.NearSphere(x => x.GeoLocation, criteria.Long, criteria.Lat, criteria.RadiusInMiles/3963.2);
var results = await collection.Find(locFilter).ToListAsync();
I guess I have to do something before calling ToList on the IFindFluent result ?
Any help ?
Thanks a lot
The C# MongoDB Driver lacks of some operations but the reality there's not any restriction querying the database. The methods Project, Match and so forth are just helpers to build a PipeLine that's exactly a BsonDocument like any other.
I had to query the database from C# using a similar approach to yours:
db.mycoll.aggregate(
[ { $geoNear :
{ near : { type : "Point", coordinates : [-34.5460069,-58.48894900000001] },
distanceField : "dist.calculated", maxDistance : 100,
includeLocs : "dist.location",
num : 5, spherical : true }
} ,
{ $project : {_id: 1, place_id:1, name:1, dist:1} }
] ).pretty()
As you know there's not a geoNear method to build it in the driver, so you can just create the BsonDocument.
To show you that everything can be built in that way find below a sample query from C# not using the project clausule either, I built it from the BsonDocument. You can push BsonDocument's in the pipeline as you wish.
var coll = _database.GetCollection<UrbanEntity>("mycoll");
var geoNearOptions = new BsonDocument {
{ "near", new BsonDocument {
{ "type", "Point" },
{ "coordinates", new BsonArray {-34.5460069,-58.48894900000001} },
} },
{ "distanceField", "dist.calculated" },
{ "maxDistance", 100 },
{ "includeLocs", "dist.location" },
{ "num", 5 },
{ "spherical" , true }
};
var projectOptions = new BsonDocument {
{ "_id" , 1 },
{ "place_id", 1 },
{ "name" , 1 },
{ "dist", 1}
};
var pipeline = new List<BsonDocument>();
pipeline.Add( new BsonDocument { {"$geoNear", geoNearOptions} });
pipeline.Add( new BsonDocument { {"$project", projectOptions} });
using(var cursor = await coll.AggregateAsync<BsonDocument>(pipeline)) {
while(await cursor.MoveNextAsync()) {
foreach (var doc in cursor.Current) {
// Here you have the documents ready to read
}
}
}
I hope it helps.