I'm getting tired that I can't figure out(nor alone nor through SO) how to pull out deeply nested data within BsonDocument or custom classes directly. I know that I should use for sure a filter and a projection to get out an array/list of Guids nested within another array.
Following is the structure(simplified):
//Thread
{
Id: "B2",
Answers: [
{
Id: "A1",
Likes: [ "GUID1", "GUID2", "ETC" ] //<- this array, and only this.
}
]
}
I have both the Thread.Id and Answer.Id as filtering data but then I tried with:
var f = Builders<BsonDocument>.Filter;
var filter = f.And(f.Eq("Id", ids.ThreadId), f.Eq("Answers.$[].Id", ids.AnswerId));
var projection = Builders<BsonDocument>.Projection.Include("Answers.Likes.$");
var likes = await dbClient.GetCollection<BsonDocument>(nameof(Thread))
.Find(filter)
.Project(projection)
.FirstOrDefaultAsync();
But this query always returns null, what I'm doing wrong from this POV?
It is not possible to project the individual fields from array in projection using regular queries.
You can at best project the matching element using regular queries and then map the likes.
Something like
var f = Builders<BsonDocument>.Filter;
var filter = f.And(f.Eq("Id", ids.ThreadId), f.Eq("Answers.Id", ids.AnswerId));
var projection = Builders<BsonDocument>.Projection.Include("Answers.$");
var answer = await dbClient.GetCollection<BsonDocument>(nameof(Thread))
.Find(filter)
.Project(projection)
.FirstOrDefaultAsync();
Alternatively you can use filters with map using aggregation to match the answer element by id followed by projection to map the like field.
Something like
var f = Builders<BsonDocument>.Filter;
var match = f.And(f.Eq("Id", ids.ThreadId), f.Eq("Answers.Id", ids.AnswerId));
var project = new BsonDocument("newRoot",
new BsonDocument("$arrayElemAt", new BsonArray {
new BsonDocument("$map",
new BsonDocument
{
{ "input",
new BsonDocument("$filter", new BsonDocument
{
{ "input", "$Answers"},
{"cond", new BsonDocument("$eq", new BsonArray { "$$this.Id", ids.AnswerId})}
})
},
{ "in", new BsonDocument("Likes", "$$this.UserLikes") }
}),
0}));
var pipeline = collection.Aggregate()
.Match(match)
.AppendStage<BsonDocument, BsonDocument, BsonDocument>(new BsonDocument("$replaceRoot", project));
var list = pipeline.ToList();
Working example here - https://mongoplayground.net/p/wM1z6q92_mV
I wasn't able to fetch Likes with single filtering and projection. However, I was able to achieve it by using aggregation pipeline.
private async Task<BsonArray> GetLikes(string docId, string answerId)
{
var client = new MongoClient();
var idFilter = Builders<BsonDocument>.Filter.Eq("ID", docId);
var answerIdFilter = Builders<BsonDocument>.Filter.Eq("Answers.ID", answerId);
var projection = Builders<BsonDocument>.Projection.Exclude("_id").Include("Answers.Likes");
var likes = await client.GetDatabase("test").GetCollection<BsonDocument>("items")
.Aggregate(new AggregateOptions())
.Match(idFilter)
.Unwind("Answers")
.Match(answerIdFilter)
.Project(projection)
.FirstOrDefaultAsync();
return likes == null ? null
: (likes.GetElement("Answers").Value as BsonDocument).GetElement("Likes").Value as BsonArray;
}
For some reason result included document in original structure as opposed to including just a document with Likes property so I had to do some post processing afterwards.
Related
{
"#search.score": 1,
"id": "1",
"FullName": "Adam",
"UserName": "Adam1903",
"IsVerified": true,
"PartitionKey": "32fbq",
"IsSearchableUser": true
}
Thats is my document on azure search index... I have collection List followedUsers = new List(){1,3};... what I want to achieve in azure search is something like this...
Select * from User where Id is in(1,2,3)
You can retrieve all the search result into List<> and use the following code to filter by id in array list.
var s = new List<Search>();
var exceptions = new int[] {1,2,3};
var filtered = s.Where(x=> exceptions.Contains(x.id));
Update:
If you want to use SearchParameters.SearchFields Property to query parameter pegs search to specific fields, you could use the following code to query specific Id field.
parameters = new SearchParameters()
{
SearchFields = new[] { "id" }
};
results = indexClient.Documents.Search<Hotel>("1", parameters);
Is there any way to use Filters in C# and translate this mongo query?
{'EmailData.Attachments.Files': {$all: [{Name: 'a.txt'},{Name: 'b.txt'},{Name:'c.txt'}], $size: 3}}
my data model is like:
{
"_id": ObjectId("5f0a9c07b001406068c073c1"),
"EmailData" : [
{
"Attachments" : {
"Files" : [
{
"Name" : "a.txt"
},
{
"Name" : "b.txt"
},
{
"Name" : "c.txt"
}
]
}
}
]
}
I have something like this in my mind:
var Filter =
Builders<EmailEntity>.Filter.All(s => s.EmailData????);
or something like:
var Filter =
Builders<EmailEntity>.Filter.ElemMatch(s => s.EmailData???)
I was wondering is there any way in the above filter to use All inside ElemMatch?
The difficulty here is that EmailData.Attachments.Files is an array within another array so C# compiler will get lost when you try to use Expression Trees.
Thankfully there's another approach when you need to define a field using MongoDB .NET driver. You can take advantage of StringFieldDefinition<T> class.
Try:
var files = new[] { new FileData(){ Name = "a.txt"}, new FileData() { Name = "b.txt" }, new FileData() { Name = "c.txt" } };
FieldDefinition<EmailEntity> fieldDef = new StringFieldDefinition<EmailEntity>("EmailData.Attachments.Files");
var filter = Builders<EmailEntity>.Filter.And(
Builders<EmailEntity>.Filter.All(fieldDef, files),
Builders<EmailEntity>.Filter.Size(fieldDef, 3));
var result= collection.Find(filter).ToList();
I've got a json file containing
{
"Accounts": null,
"AccountTypes": null,
"Actions": null,
"Photos": [
{
"Instance": "...",
"Key": "..."
},
....
]
}
Now I want to get all the Instance properties from the Photo objects. I've got the following code:
var photos = new List<Photo>();
string json = File.ReadAllText(file);
dynamic jsonObj = Newtonsoft.Json.JsonConvert.DeserializeObject(json, typeof(object));
var jsonPhotos = jsonObj.Photos as IEnumerable<dynamic>;
var instances = jsonPhotos.Select(x => x.Instance);
foreach (var instance in instances)
photos.Add(new Photo
{
Document = Convert.FromBase64String(instance)
});
However, jsonPhotos.Select(x => x.Instance); isn't returning anything...
I am able to get things working by using
var instances = new List<string>();
foreach (var photo in jsonPhotos)
instances.Add(photo.Instance.Value);
But can I solve this in a LINQ way?
Why just don't use Json.Linq for that? Parse JSON to JObject instance, then map every token from Photos array to Photo instance (I've omitted Convert.FromBase64String because OP sample doesn't have a valid base64 data, but converting Instance value can be easily added)
var json = JObject.Parse(jsonString);
var photos = json["Photos"]
.Select(token => new Photo
{
Document = token["Instance"]?.Value<string>()
})
.ToList();
The .Select(x => x.Instance) indeed returns ... on .NET Core 3.1. Can you verify that the contents of the json variable are actually what you expect?
Specifically
jsonPhotos.Select(x => x.Instance);
works as expected, while
jsonPhotos.Select(x => x.SomeNonExistingProperty);
enumerates nothing / empty values.
For example, this code prints Instance A, then Instance B, then nothing twice:
var json = #"
{
""Photos"": [
{
""Instance"": ""Instance A"",
""Key"": ""...""
},
{
""Instance"": ""Instance B"",
""Key"": ""...""
}]
}";
var jsonObj = JsonConvert.DeserializeObject<dynamic>(json);
var jsonPhotos = jsonObj.Photos as IEnumerable<dynamic>;
var instances = jsonPhotos.Select(x => x.Instance);
foreach (var instance in instances)
{
Console.WriteLine(instance);
}
// In contrast, this one will print empty lines.
instances = jsonPhotos.Select(x => x.SomeNonExistingProperty);
foreach (string instance in instances)
{
Console.WriteLine(instance);
}
I took the liberty to change the deserialization to dynamic directly, but it also works with the original code from the question.
I have a list of BsonDocument:
var list = db.GetCollection<BsonDocument>(collectionName);
var myIds = list.Find(_ => true)
.Project(Builders<BsonDocument>.Projection.Include("_id"))
.ToList();
that contains:
myIds = "{
{ "_id" : "cc9d9282-c9d2-4cba-a776-ffddsds274d5" },
{ "_id" : "2c1ddd82-c9d2-4dda-afr6-d79ff1274d56" },
{ "_id" : "ss969281-c9d2-4cba-a776-d79ffds274d5" }
}"
And want to query like this:
var deleted =list.DeleteMany(Builders<MessageExchange>.Filter.In("_id", myIds));
I also have tried the following:
var filter = new BsonDocument("_id", new BsonDocument("$in", new BsonArray(myIds)));
var deleted = list.DeleteMany(filter);
Returns the attribute DeletedCount = 0
Could somebody point what seems to be wrong about the filter?
You'll have to extract the _id from the BsonDocument like this:
var extractedIds = myIds.Select(x => x["_id"].ToString()).ToList();
After which you can use it in the filter.
list.DeleteMany(Builders<MessageExchange>.Filter.In("_id", extractedIds));
Make sure that the _id part of the filter matches that of the MessageExchange class
Another way to do so is by making it strong typed:
list.DeleteMany(Builders<MessageExchange>.Filter.In(x => x.Id, extractedIds));
This works as well (based on Skami's answer):
var filter = new BsonDocument("_id", new BsonDocument("$in", new BsonArray(extractedIds)));
list.DeleteMany(filter);
therefore is not tied to the MessageExchange class.
The data:
The collection contains a list of audit records and I want to return the last modified items from the collection.
For example:
So the query needs to return Audit 1235 and 1237 Only.
The following statement works in Mongo Shell and returns the data sub-millisecond, I just need to also figure out how to return the entire Collection item instead of just the Id.
db.Forms.aggregate(
{ $group: { _id: "$Id", lastModifiedId: { $last: "$_id" } } }
)
However, I need to convert this to the C# Driver's syntax.
I have the following at the moment but it's not working and returns (for lack of a better term) weird data (see screencap under the statement).
var results = collection.Aggregate()
.Group(new BsonDocument { { "_id", "$Id" }, { "lastModifiedId", new BsonDocument("$last", "_id") } })
.ToListAsync().Result.ToList();
My current solution gets the full collection back and then runs it through an extension method to get the latest records (where list is the full collection):
var lastModifiedOnlyList =
from listItem in list.OrderByDescending(_ => _.AuditId)
group listItem by listItem.Id into grp
select grp.OrderByDescending(listItem => listItem.AuditId)
.FirstOrDefault();
While this code works, it is EXTREMELY slow because of the sheer amount of data that is being returned from the collection, so I need to do the grouping on the list as part of the collection get/find.
Please let me know if I can provide any additional information.
Update: With Axel's help I managed to get it resolved:
var pipeline = new[] { new BsonDocument { { "$group", new BsonDocument { { "_id", "$Id" }, { "LastAuditId", new BsonDocument { { "$last", "$_id" } } } } } } };
var lastAuditIds = collection.Aggregate<Audit>(pipeline).ToListAsync().Result.ToList().Select(_=>_.LastAuditId);
I moved that to it's own method and then use the IDs to get the collection items back, with my projection working as well:
var forLastAuditIds = ForLastAuditIds(collection);
var limitedList = (
projection != null
? collection.Find(forLastAuditIds & filter, new FindOptions()).Project(projection)
: collection.Find(forLastAuditIds & filter, new FindOptions())
).ToListAsync().Result.ToList();
"filter" in this case is either an Expression or a BsonDocument. The performance is great as well - sub-second for the whole thing. Thanks for the help, Axel!
I think you're doing an extra OrderBy, this should do:
var lastModifiedOnlyList =
from listItem in list
group listItem by listItem.Id into grp
select grp.OrderByDescending(listItem => listItem.AuditId)
.FirstOrDefault();
EDIT:
To gain performance in the query, you could use the Aggregate function differently:
var match = new BsonDocument
{
{
"$group",
new BsonDocument
{
{ "_id", "$Id" },
{ "lastModifiedId", new BsonDocument
{
{
"$last", "$_id"
}
}}
}
}
};
var pipeline = new[] { match };
var result = collection.Aggregate(pipeline);
That should be the equivalent of your Mongo Shell query.