Develop Reference

Mongo error when trying to update nested arrays: No array filter found for identifier

I am trying to update a document in Mongo that represents a community with the following scenario.
A community has a collection of blocks
A block has a collection of floors
A floor has a collection of doors
A door has a collection of label names
Given a document Id and information about the labels that must be placed into each door, I want to use the MongoDb C# driver v2.10.4 and mongo:latest to update nested lists (several levels).
I've reading the documentation, about array filters, but I can't have it working.
I've created a repository from scratch to reproduce the problem, with instructions on the Readme on how to run the integration test and a local MongoDB with docker.
But as a summary, my method groupds the labels so that I can bulk place names on the desired door and then it iterates over these groups and updates on Mongo the specific document setting the desired value inside some levels deep nested object. I couldn't think of a more efficient way.
All the code in the above repo.
The DB document:
public class Community
{
public Guid Id { get; set; }
public IEnumerable<Block> Blocks { get; set; } = Enumerable.Empty<Block>();
}
public class Block
{
public string Name { get; set; } = string.Empty;
public IEnumerable<Floor> Floors { get; set; } = Enumerable.Empty<Floor>();
}
public class Floor
{
public string Name { get; set; } = string.Empty;
public IEnumerable<Door> Doors { get; set; } = Enumerable.Empty<Door>();
}
public class Door
{
public string Name { get; set; } = string.Empty;
public IEnumerable<string> LabelNames = Enumerable.Empty<string>();
}
The problematic method with array filters:
public async Task UpdateDoorNames(Guid id, IEnumerable<Label> labels)
{
var labelsGroupedByHouse =
labels
.ToList()
.GroupBy(x => new { x.BlockId, x.FloorId, x.DoorId })
.ToList();
var filter =
Builders<Community>
.Filter
.Where(x => x.Id == id);
foreach (var house in labelsGroupedByHouse)
{
var houseBlockName = house.Key.BlockId;
var houseFloorName = house.Key.FloorId;
var houseDoorName = house.Key.DoorId;
var names = house.Select(x => x.Name).ToList();
var update =
Builders<Community>
.Update
.Set($"Blocks.$[{houseBlockName}].Floors.$[{houseFloorName}].Doors.$[{houseDoorName}].LabelNames", names);
await _communities.UpdateOneAsync(filter, update);
}
}
The exception is
MongoDB.Driver.MongoWriteException with the message "A write operation resulted in an error.
No array filter found for identifier 'Block 1' in path 'Blocks.$[Block 1].Floors.$[Ground Floor].Doors.$[A].LabelNames'"
Here's a more visual sample on how the nested structure looks like in the database. Notice the value I want to update is the LabelNames, which is an array of string.
I appreciate any help to have this working and suggestions on whether it's the right approach assuming that I cannot change the repository's method signature.
SOLUTION RESULT:
Thanks for the quick answer #mickl, it works perfectly.
Result at this repo's specific point of history exactly as suggested.

The $[{houseBlockName}] expects an identifier which acts as a placeholder and has a corresponding filter defined within arrayfilters (positional filtered). It seems like you're trying to pass the filter value directly which is incorrect.
Your C# code can look like this:
var houseBlockName = house.Key.BlockId;
var houseFloorName = house.Key.FloorId;
var houseDoorName = house.Key.DoorId;
var names = house.Select(x => x.Name).ToList();
var update = Builders<Community>.Update.Set("Blocks.$[block].Floors.$[floor].Doors.$[door].LabelNames", names);
var arrayFilters = new List<ArrayFilterDefinition>();
ArrayFilterDefinition<BsonDocument> blockFilter = new BsonDocument("block.Name", new BsonDocument("$eq", houseBlockName));
ArrayFilterDefinition<BsonDocument> floorFilter = new BsonDocument("floor.Name", new BsonDocument("$eq", houseFloorName));
ArrayFilterDefinition<BsonDocument> doorFilter = new BsonDocument("door.Name", new BsonDocument("$eq", houseDoorName));
arrayFilters.Add(blockFilter);
arrayFilters.Add(floorFilter);
arrayFilters.Add(doorFilter);
var updateOptions = new UpdateOptions { ArrayFilters = arrayFilters };
var result = _communities.UpdateOne(filter, update, updateOptions);

{
var filterCompany = Builders<CompanyInfo>.Filter.Eq(x => x.Id, Timekeepping.CompanyID);
var update = Builders<CompanyInfo>.Update.Set("LstPersonnel.$[i].Timekeeping.$[j].CheckOutDate", DateTime.UtcNow);
var arrayFilters = new List<ArrayFilterDefinition>
{
new BsonDocumentArrayFilterDefinition<BsonDocument>(new BsonDocument("i.MacAddress",new BsonDocument("$eq", Timekeepping.MacAddress) )),
new BsonDocumentArrayFilterDefinition<BsonDocument>(new BsonDocument("j.Id", new BsonDocument("$eq", timeKeeping.Id)))
};
var updateOptions = new UpdateOptions { ArrayFilters = arrayFilters};
var updateResult = await _companys.UpdateOneAsync(filterCompany, update, updateOptions);
return updateResult.ModifiedCount != 0;
}

How to read boolean value from Cosmos DB from C#

I have some data in documentdb with following structure:
{
id:1,
key:"USA",
states:["New York","New Jersey", "Ohio", "Florida" ]
}
I need to check if "California" is available in the above document with a query from C# using CreateDocumentQuery which returns true/false. How can I read the boolean value from CreateDocumentQuery?

Assuming you have a DTO looking something like this:
class Item
{
[JsonProperty("id")]
public string Id { get; set; }
[JsonProperty("key")]
public string Key { get; set; }
[JsonProperty("states")]
public List<string> States { get; set; }
}
You can use the Contains method on an array to check if the value exists or not. The LINQ to SQL converter will turn this into a sql query for you. All you have to do is to add any other predicates in the where clause and run it. I am intentionally only selecting the Id of the document to make save some RUs and make it run faster.
If you add the partition key in the expression present in the where clause, or if you know what the partition key value is, please set it in the feed options to enhance performance. Cross partition queries are not really recommended as part of your day to day workflow.
You can use the CountAsync() extension method of the DocumentQueryable to get the count of doucments matching the predicate and then do a > 0 to see if it exists.
Here is the code for that:
public async Task<bool> ArrayContainsAsync()
{
var documentClient = new DocumentClient(new Uri("https://localhost:8081"), "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==");
var collectionUri = UriFactory.CreateDocumentCollectionUri("dbname", "collectionName");
return (await documentClient
.CreateDocumentQuery<Item>(collectionUri, new FeedOptions { EnableCrossPartitionQuery = true })
.Where(x => x.States.Contains("California")).CountAsync()) > 0;
}
However, the code below will take control of the query and exit on first match which will be way more efficient than the code above.
public async Task<bool> ArrayContainsAsync()
{
var documentClient = new DocumentClient(new Uri("https://localhost:8081"), "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==");
var collectionUri = UriFactory.CreateDocumentCollectionUri("dbname", "collectionName");
var query = documentClient
.CreateDocumentQuery<Item>(collectionUri, new FeedOptions { EnableCrossPartitionQuery = true })
.Where(x => x.States.Contains("California")).Select(x=> x.Id).AsDocumentQuery();
while (query.HasMoreResults)
{
var results = await query.ExecuteNextAsync();
if (results.Any())
return true;
}
return false;
}

C# MongoDB LINQ: Cannot query nested list

I'm trying to query a MongoDB collection using official C# driver. Here's the object structure I've created:
IMongoDatabase db = mongoClient.GetDatabase("appdb");
IMongoCollection<MusicFile> musicfiles = db.GetCollection<MusicFile>("files");
public class MusicFile
{
public ObjectId Id { get; set; }
public string Name { get; set; }
public IList<Comment> Comments { get; set; }
}
public class Comment
{
public string Text { get; set; }
}
This is the query I'm trying to get any MusicFile objects that contains a Comment object with property Text = "Comment1":
musicfiles.AsQueryable().Where(f => f.Comments != null && f.Comments.Any(c => c.Text == "Comment1")).ToList();
I can't get this query to work, it always returns an empty list. I also tried this, which too didn't work:
musicfiles.Find(f => f.Comments.Any(c => c.Text == "Comment1")).ToList()
But, if I get the complete collection is memory, the query works:
musicfiles.Find(FilterDefinition<MusicFile>.Empty).ToList().Where(f => f.Comments != null && f.Comments.Any(c => c.Text == "Comment1")).ToList();
This seems like a very inefficient way to query. Any suggestions?

OK. I'm back at home. Try this:
var musicFilter = Builders<MusicFile>.Filter;
var commentFilter = Builders<Comment>.Filter;
var files = musicfiles
.Find(
musicFilter.NE(m => m.Comments, null)
& musicFilter.ElemMatch(m => m.Comments, commentFilter.Eq(c => c.Text, "Comment1"))
)
.ToEnumerable()
.ToList();
Note I call .ToList() because, otherwise, if you iterate through files multiple times, you'll get multiple calls to the database for the same objects.

RavenDB - stream index query results in exception

We're currently trying to use the Task<IAsyncEnumerator<StreamResult<T>>> StreamAsync<T>(IQueryable<T> query, CancellationToken token = null), running into some issues.
Our document look something like:
public class Entity
{
public string Id { get; set; }
public DateTime Created { get; set; }
public Geolocation Geolocation { get; set; }
public string Description { get; set; }
public IList<string> SubEntities { get; set; }
public Entity()
{
this.Id = Guid.NewGuid().ToString();
this.Created = DateTime.UtcNow;
}
}
In combination we've a view model, which is also the model were indexing:
public class EntityViewModel
{
public string Id { get; set; }
public DateTime Created { get; set; }
public Geolocation Geolocation { get; set; }
public string Description { get; set; }
public IList<SubEntity> SubEntities { get; set; }
}
And ofcourse, the index, with the resulttype inheriting from the viewmodel, to enable that SubEntities are mapped and output correctly, while enabling the addition of searchfeatures such as fulltext etc.:
public class EntityWithSubentitiesIndex : AbstractIndexCreationTask<Entity, EntityWithSubentitiesIndex.Result>
{
public class Result : EntityViewModel
{
public string Fulltext { get; set; }
}
public EntityWithSubentitiesIndex ()
{
Map = entities => from entity in entities
select new
{
Id = entity.Id,
Created = entity.Created,
Geolocation = entity.Geolocation,
SubEntities = entity.SubEntities.Select(x => LoadDocument<SubEntity>(x)),
Fulltext = new[]
{
entity.Description
}.Concat(entity.SubEntities.Select(x => LoadDocument<SubEntity>(x).Name)),
__ = SpatialGenerate("__geolokation", entity.Geolocation.Lat, entity.Geolocation.Lon)
};
Index(x => x.Created.Date, FieldIndexing.Analyzed);
Index(x => x.Fulltext, FieldIndexing.Analyzed);
Spatial("__geolokation", x => x.Cartesian.BoundingBoxIndex());
}
}
Finally we're querying like this:
var query = _ravenSession.Query<EntityWithSubentitiesIndex.Result, EntityWithSubentitiesIndex>()
.Customize(c =>
{
if (filter.Boundary == null) return;
var wkt = filter.Boundary.GenerateWkt().Result;
if (!string.IsNullOrWhiteSpace(wkt))
{
c.RelatesToShape("__geolokation", wkt, SpatialRelation.Within);
}
})
.AsQueryable();
// (...) and several other filters here, removed for clarity
var enumerator = await _ravenSession.Advanced.StreamAsync(query);
var list = new List<EntityViewModel>();
while (await enumerator.MoveNextAsync())
{
list.Add(enumerator.Current.Document);
}
When doing so we're getting the following exception:
System.InvalidOperationException: The query results type is 'Entity'
but you expected to get results of type 'Result'. If you want to
return a projection, you should use
.ProjectFromIndexFieldsInto() (for Query) or
.SelectFields() (for DocumentQuery) before calling to
.ToList().
According to the documentation, the Streaming API should support streaming via an index, and querying via an IQueryable at once.
How can this be fixed, while still using an index, and the streaming API, to:
Prevent having to page through the normal query, to work around the default pagesize
Prevent having to load the subentities one at a time when querying
Thanks in advance!

Try to use:
.As<Entity>()
(or .OfType<Entity>()) in your query. That should work in the regular stream.
This is a simple streaming query using "TestIndex" that is an index over an entity Test and I'm using a TestIndex.Result to look like your query. Note that this is actually not what the query will return, it's only there so you can write typed queries (ie. .Where(x => x.SomethingMapped == something))
var queryable = session.Query<TestIndex.Result, TestIndex>()
.Customize(c =>
{
//do stuff
})
.As<Test>();
var enumerator = session.Advanced.Stream(queryable);
while (enumerator.MoveNext())
{
var entity = enumerator.Current.Document;
}
If you instead want to retrieve the values from the index and not the actual entity being indexed you have to store those as fields and then project them into a "view model" that matches your mapped properties. This can be done by using .ProjectFromIndexFieldsInto<T>() in your query. All the stored fields from the index will be mapped to the model you specify.
Hope this helps (and makes sense)!
Edit: Updated with a, for me, working example of the Streaming API used with ProjectFromIndexFieldsInto<T>() that returns more than 128 records.
using (var session = store.OpenAsyncSession())
{
var queryable = session.Query<Customers_ByName.QueryModel, Customers_ByName>()
.Customize(c =>
{
//just to do some customization to look more like OP's query
c.RandomOrdering();
})
.ProjectFromIndexFieldsInto<CustomerViewModel>();
var enumerator = await session.Advanced.StreamAsync(queryable);
var customerViewModels = new List<CustomerViewModel>();
while (await enumerator.MoveNextAsync())
{
customerViewModels.Add(enumerator.Current.Document);
}
Console.WriteLine(customerViewModels.Count); //in my case 504
}
The above code works great for me. The index has one property mapped (name) and that property is stored. This is running the latest stable build (3.0.3800).

As #nicolai-heilbuth stated in the comments to #jens-pettersson's answer, it seems to be a bug in the RavenDB client libraries from version 3 onwards.
Bug report filed here: http://issues.hibernatingrhinos.com/issue/RavenDB-3916

How do I dynamically apply a conditional operator to a field using the official MongoDB-CSharp-Driver?

I'm trying to generate a query that finds all large, red things with a cost greater than 3.
This query seems to be what I'm after:
{ "color" : "red", "size" : "large", "cost" : { "$gt" : 3.0 } }
But, I am unable to find an elegant way to create the cost condition using the official MongoDB CSharp Driver. This is one hack which seems to create the query:
QueryConditionList gt = Query.GT("cost", BsonDouble.Create(3));
QueryDocument query = new QueryDocument();
query.Add("color", "red");
query.Add("size", "large");
query.Add(gt.ToBsonDocument().Elements);
List<BsonDocument> results = events.Find(query).ToList();
Another way to do it which seems to work is like this:
QueryDocument query = new QueryDocument();
query.Add("color", "red");
query.Add("size", "large");
query.Add("cost", new BsonDocument("$gt", BsonDouble.Create(3)));
List<BsonDocument> results = events.Find(query).ToList();
Are either of these approaches a good way to accomplish this? Is there another?
I need to use techniques which allow me to dynamically build the query and add fields that will be involved in the query. I was hoping to find a way to add a condition via query.Add( ) but I don't know if that is possible.
Any help is appreciated.

You can use the Query builder throughout, like so:
var query = Query.And(
Query.EQ("color", "red"),
Query.EQ("size", "large"),
Query.GT("cost", 3)
);
update Sorry, I see what you're asking, now.
You could do something like this, also:
int i = 0;
var qc = QueryComplete[3];
qc[i++] = Query.EQ("color", "red");
qc[i++] = Query.EQ("size", "large");
qc[i++] = Query.GT("cost", 3);
var query = Query.And(qc);
This way, you can still use the builder methods and have it be dynamic.

You can data drive it in a brute force manner, just build a tree of "QueryElement" and call BuildQuery to recursively build it as in this example class:
public class QueryElement
{
public enum eOperator
{
AND, OR, EQ, NE, GT, GTE, LT, LTE //etc.
};
public eOperator Operator { get; set; }
public string Field { get; set; }
public BsonValue Value { get; set; }
public List<QueryElement> Children { get; set; }
public IMongoQuery BuildQuery()
{
int i = 0;
var qc = new IMongoQuery[(Children!=null)?Children.Count:0];
if (Children != null)
{
foreach (var child in Children)
{
qc[i++] = child.BuildQuery();
}
}
switch (Operator)
{
// multiple element operators
case eOperator.AND:
return Query.And(qc);
case eOperator.OR:
return Query.And(qc);
// single element operators
case eOperator.EQ:
return Query.EQ(Field, Value);
case eOperator.NE:
return Query.NE(Field, Value);
case eOperator.GT:
return Query.GT(Field, Value);
case eOperator.GTE:
return Query.GTE(Field, Value);
case eOperator.LT:
return Query.LT(Field, Value);
case eOperator.LTE:
return Query.LTE(Field, Value);
}
return null;
}
}