Develop Reference

Make an Array of unique elements in c# mongodb driver

How can I create an array of unique elements in MongoDB c# driver,
I don't want to check every time if this element already in the array or not.
suppose :
list=[1,2,3,4]
then I shouldn't be able to add duplicate element (such as 3 )

You can use the AddToSet or AddToSetEach method, every time you create or update the array, as mentioned in the comments:
var update = Builders<Entity>.Update.AddToSetEach(e => e.Items, new [] {1, 2});
collection.UpdateOne(new BsonDocument(), update, new UpdateOptions { IsUpsert = true });
And you can define a schema validation when creating the collection, to ensure that duplicate items will never be allowed (an error would be thrown on insert/update, “Document failed validation”).
You can define the schema in the MongoDB shell, or here is how to do it in C#:
var options = new CreateCollectionOptions<Entity>
{
ValidationAction = DocumentValidationAction.Error,
ValidationLevel = DocumentValidationLevel.Strict,
Validator = new FilterDefinitionBuilder<Entity>().JsonSchema(new BsonDocument
{
{ "bsonType", "object" },
{ "properties", new BsonDocument("Items", new BsonDocument
{
{ "type" , "array" },
{ "uniqueItems", true }
})
}
})
};
database.CreateCollection("entities", options, CancellationToken.None);
where Entity is an example class like this:
public class Entity
{
public ObjectId Id { get; set; }
public int[] Items { get; set; }
}
Here are the API docs for CreateCollectionOptions and in the unit tests you can see examples of usage - e.g. JsonSchema(). Unfortunately I don't see anything in the reference docs with more thorough explanations.

Push an item to a deeply nested array in MongoDb

Is it possible (preferably using the C# Builders) to add a new item to a deeply nested array I.e. an array within an array within an array.
My data model looks something like :
public class Company
{
public string Id { get; set; }
public string Name { get; set; }
public IEnumerable<Department> Departments { get; set; }
}
public class Department
{
public string Id { get; set; }
public string Name { get; set; }
public IEnumerable<Managers> Managers { get; set; }
}
public class Manager
{
public string Id { get; set; }
public string Name { get; set; }
public IEnumerable<Employee> Employees { get; set; }
}
public class Employee
{
public string Id { get; set; }
public string Name { get; set; }
}
Which translates to:
{
"Id": 12345,
"Name": "Company Ltd",
"Departments": [
{
"Id": 1,
"Name": "Development",
"Managers" : [
{
"Id" : 5555,
"Name" : "The Boss",
"Employees": [
{
"Id" : 123,
"Name" : "Developer 1"
},
{
"Id" : 124,
"Name" : "Developer 2"
}
]
}
]
}
]
}
If I wanted to add another employee under a specific manager how would I go about doing that?

In order to push to a nested array, you must make use of the positional operator $ in order to specify a matching outer array element to apply the operation to. For example:
db.collection.update(
{"my_array._id": myTargetId},
{$push: {"my_array.$.my_inner_array": myArrayElem}}
);
This breaks down, however, for traversing nested arrays--that is, you can only use the positional operator on the single array, not any nested ones. This is a well-defined problem as noted in the MongoDB documentation.
If you absolutely need to perform these kinds of nested array operations, then you have a couple of options available to you:
The first, and preferred, is to update your document structure and avoid nesting arrays more than one level deep. This will avoid the issue altogether, but will require any existing data to be migrated to the new structure and additional efforts to be made to structure the data in the way you need on the fly on retrieval. Separate client and server representations of your data will end up being required.
The second is to perform a series of less-reliable steps:
1. Retrieve the original document.
2. Locate the indexes for each array where your target element is located manually.
3. Attempt an update on the specific index chain and attempt to match that index chain as well.
4. Check the result of the update attempt--if it fails, then it's possible that the document was changed while the indexes were being calculated.
For example, if you wanted to update manager with ID 5555 to have the additional employee, you'd perform the following query after retrieving the indexes:
// Index chain found to be Departments.0 and Managers.0
db.collection.update(
{
"Id": 12345,
"Departments.0.Managers.0.Id": 5555 // Specify index chain 0,0 and ensure that our target still has Id 5555.
},
{ $push: {
"Departments.0.Managers.0.Employees": myNewEmployee // Push to index chain 0,0
}}
);

Use positional operator for each of the arrays except the one you want to push to.
Use array filters in update options to specify the department and manager ids. The letter used in the array filters should match the letter used as the positional operator in the update definition. So "d.Id" -> "Departments.$[d]"
If you want to match on more than one property you can use a dictionary in the array filter.
private IMongoCollection<Company> _collection;
public async Task AddEmployee()
{
var filter = Builders<Company>.Filter.Where(d => d.Id == "companyId");
var update = Builders<Company>.Update
.Push("Departments.$[d].Managers.$[m].Employees", new Employee { Id = "employeeId", Name = "employeeName" });
var updateOptions = new UpdateOptions
{
ArrayFilters = new List<ArrayFilterDefinition>
{
new BsonDocumentArrayFilterDefinition<BsonDocument>(new BsonDocument("d.Id", "departmentId")),
new BsonDocumentArrayFilterDefinition<BsonDocument>(new BsonDocument("m.Id", "managerId")),
}
};
await _collection.UpdateOneAsync(filter, update, updateOptions);
}
The problem here is that you need to use strings in the update definition and filters but not sure how to manage it without strings.
To remove an employee from the array is similar but you will have to specify an extra filter for the employee that you want to remove.
public async Task FireEmployee()
{
var filter = Builders<Company>.Filter.Where(d => d.Id == "companyId");
var employeeFilter = Builders<Employee>.Filter.Where(e => e.Id == "employeeId");
var update = Builders<Company>.Update
.PullFilter("Departments.$[d].Managers.$[m].Employees", employeeFilter);
var updateOptions = new UpdateOptions
{
ArrayFilters = new List<ArrayFilterDefinition>
{
new BsonDocumentArrayFilterDefinition<BsonDocument>(new BsonDocument("d.Id", "departmentId")),
new BsonDocumentArrayFilterDefinition<BsonDocument>(new BsonDocument("m.Id", "managerId")),
}
};
await _collection.UpdateOneAsync(filter, update, updateOptions);
}

How can I delete a field value from an Elasticsearch document using NEST?

I am using Nest to insert/update Documents in Elasticsearch. Here's a sample POCO class that I use to map the Document...
public class MyClass
{
public string Id { get; set; }
public decimal? MyField { get; set; }
}
This works as expected...When I add the Document, if the nullable field MyField has a value, it is returned in the JSON. If the nullable field doesn't have a value, it's not returned in the _source JSON because null values aren't stored.
However, there might be times where I need to update a single document and remove the value from a single field. That is, when I first insert the document, MyField has a value and is returned in the Elasticsearch JSON result. Then later, for whatever reason, I need to remove that value.
I am using partial document updates, and if possible, would prefer to keep it like that. (The full Document model I'm using will have 100+ fields, and my index will eventually have 100M+ records.) So, I'm looking for the most efficient way possible to partially update the documents.
Thanks in advance!

Think how would you do it just using api? there are several ways. for example like this:
// kibana version
POST my_index/_update_by_query
{
"query": {
"ids": {
"values": [
"12345"
]
}
},
"script": {
"source": "ctx._source.myField = null"
}
}
// nest version
var response = client.UpdateByQuery<CustomerDto>(u => u
.Query(q => q
.Ids(i => i.Values(12345))
)
.Script("ctx._source.flag = null")
//.Conflicts(Conflicts.Proceed)
.Refresh(true)
);
or this:
// kibana version
POST my_index/my_type/12345/_update
{
"script": {
"source": "ctx._source.remove('myField')"
}
}
// nest version
var response = client.Update<CustomerDto>(
new UpdateDescriptor<CustomerDto, CustomerDto>("index_name", "type", 12345)
.Script(s => s.Source("ctx._source.remove('middleName')")));

Insert a document while auto incrementing a sequence field in MongoDB

Problem statement:
I have a collection in MongoDB that has a field with the type Int32. I would like to add a document to this collection. I need to increment the value by 1 for each insert as that field is indexed and must be unique.
Options:
[preferable] Increment the value on the DB side. That is, not specifying a new (higher) value. Just instruct MongoDB to auto increment upon insert.
Reading first. Executing a find query against the DB to find the current (before insert) highest value first, incrementing in memory, and inserting the new doc. This might fail due to racing conditions (the operation is not atomic).
keeping an index counter in memory. Not an option for me as there are multiple apps writing to the same collection (legacy limitation).
Other Ideas?
Example:
{
_id: ....
index: 123,
open: true
}
await collection.InsertOneAsync(record.ToBsonDocument());
The new doc inserted should have index value of 124
Language:
C#
Questions:
Can you provide a sample code (C#) to achieve the first option?
Extra info:
I do not have access to the code of the other app (which keeps its own index number). So having another collection and adding an sequence resolver function will not work as this will trigger a change to the legacy app.

MongoDB has a default tutorial on how to achieve that here
1 - Create a counters collections and insert the id there:
db.counters.insert(
{
_id: "userid",
seq: 0
}
)
2 - Create a custom function to retrieve the next value:
function getNextSequence(name) {
var ret = db.counters.findAndModify(
{
query: { _id: name },
update: { $inc: { seq: 1 } },
new: true
}
);
return ret.seq;
}
Use the getNextSequence to retrieve the next value:
db.users.insert(
{
_id: getNextSequence("userid"),
name: "Sarah C."
}
)
db.users.insert(
{
_id: getNextSequence("userid"),
name: "Bob D."
}
)

I had to do this in a project using MongoDB C# Driver.
Here's what I did: I created a separated collection called Sequence, with the name and value of it and I also created a repository for it.
Here is the code of class Sequence:
public class Sequence
{
[BsonId]
[BsonRepresentation(BsonType.ObjectId)]
[BsonElement("_id")]
public string Id { get; set; }
public string SequenceName { get; set; }
public int SequenceValue { get; set; }
}
And now the code of the method to generate the sequence value:
public class SequenceRepository
{
protected readonly IMongoDatabase _database;
protected readonly IMongoCollection<Sequence> _collection;
public SequenceRepository(IMongoDatabase database)
{
_database = database;
_colecao = _database.GetCollection<Sequence>(typeof(Sequence).Name);
}
public int GetSequenceValue(string sequenceName)
{
var filter = Builders<Sequence>.Filter.Eq(s => s.SequenceName, sequenceName);
var update = Builders<Sequence>.Update.Inc(s => s.SequenceValue , 1);
var result = _colecao.FindOneAndUpdate(filter, update, new FindOneAndUpdateOptions<Sequence, Sequence> { IsUpsert = true, ReturnDocument = ReturnDocument.After });
return result.SequenceValue;
}
}
Finally I called this method before insert some document:
public void Inserir(Order order)
{
order.Code = new SequenceRepository(_database).GetSequenceValue("orderSequence");
_collection.InsertOne(order);
}

You can create a Mongo Sequence in a separate collection counter
db.counter.insert({ _id: "mySeq", seq: 0 })
You can encapsulate sequence logic in a simple function like this
function getNextMySeq(name) {
var ret = db.counter.findAndModify({
query: { _id: name },
update: { $inc: { seq: 1 } },
new: true
});
return ret.seq;
}
Now simply use the function call during the insert
db.collection.insert({
index: getNextMySeq("mySeq")
})

ElasticSearch: Querying a field that's an array of objects

I've got data indexed using ElasticSearch, and I'm having trouble querying a particular field. A snippet of the JSON is as follows:
{
"_index": "indexName",
"_type": "type",
"_id": "00001",
"color": "red",
"place": "london",
"person": [
{
"name": "john",
"friends": [
"mary",
"jane"
]
}
{
"name": "jack",
"friends": [
"lisa",
"alex"
]
}
]
}
I need to query the index and pick out all records where one of the names inside person is "john".
I'm using Client.Search to do this, and I've had no trouble querying the fields that aren't nested (like color) by using:
var searchResults = client.Search<People>(s => s
.Index("indexName")
.Type("type")
.Query(q => q
.Bool(b => b
.Must(
x => x.Match(m => m.OnField(p => p.color).Query("red")),
x => x.Match(m => m.OnField(p => p.place).Query("london"))))));
I've got People defined as follows:
public class People
{
public string color {get; set; }
public string place {get; set; }
public List<Person> person {get; set; }
}
public class Person
{
public string name {get; set; }
// "friends" isn't here as I don't pull data from it
}
I'm unsure as to how to query on name as it's "inside" people - any help is greatly appreciated.

You need to wrap query in nested_query to have access to nested fields.
{
"nested" : {
"path" : "person",
"query" : {
"match" : {"person.name" : "john"}
}
}
}
Exceprt from documentation:
The query is executed against the nested objects / docs as if they
were indexed as separate docs (they are, internally) and resulting in
the root parent doc (or parent nested mapping).
Basically internally nested fields are stored as separate documents nearby (so they are quick to fetch) the original document. By default elastic doesn't load them, so you need to explicitly tell him that you want to access it. You could say nested fields are lazy ;)
Sorry It's been a long time since I worked on .Net and Linq. Don't know the API. But you need to create something like that.
Edit.
From github source and your code I think you need to:
var s = new SearchDescriptor<People>()
.Query(ff=>ff
.Nested(n=>n
.Path(f=>f.person[0])
.Query(q=>q.Term(f=>f.person[0].name,"john"))
)
);
Edit2.
Did you try direct curl to server? Or try query in head plugin? Something like:
curl -XPOST 'http://localhost:9202/indexName' -d '
{
"query": {
"nested": {
"path": "person",
"query": {
"query_string": {
"query": "person.name: john"
}
}
}
}
}'
This works on my cluster (with changed column names).

After a long while, I finally figured out that my data wasn't actually indexed as nested in the first place, and so simply adding
.Term("person.name", "john")
to my query worked perfectly.