Develop Reference

How to use formula (or Expression) to update value using Mongo DB C# Drivers 4.2?

I am quite new to MongoDb (and MongoDb C# Drivers) and lately, we are trying to implement an update wherein we use the value of a field and a variable (method parameter) to update several fields.
Basically, our doc is something like this
public class Inventory {
public string _id;
public decimal Quantity;
public decimal TotalCost;
}
What we want to do is to update the quantity and totalCost based on a passed value (qty).
(1) TotalCost -= (qty * (TotalCost / Quantity))
(2) Quantity -= Qty
The logic behind this is that we want to retain the average cost of our item.
Take note: the value of quantity field in step (1) should use the original value and not the result of step (2).
We can implement this using 2 queries but in our case, we need to execute this logic in one call only as there are different threads that will update a single item.
I have read the docs about Aggregate and Projection (and using Expressions) but I cannot seem to figure out how to use or combine the result of projection into the aggregate update.
Tried this projection to return the value that should be deducted from totalCost
Builders.Projection.Expression(e => (e.totalCost / e.quantity) *
-qty);
Thank you and hope you guys can point us in the right direction.
Here is an equivalent of what we are trying to achieve in mongo shell, provided that qty = 500.
db.inventory.updateOne(
{ _id: "1" },
[
{ "$set": { "TotalCost": { "$add": ["$TotalCost", { "$multiply": [-500, { "$divide": ["$TotalCost", "$Quantity"] }] }] } }
}
] )

There's currently no type-safe helper methods for creating update pipelines. However, you can just use the same json-ish syntax like you do with the console. Here's bit of example C# code.
// Some id to filter on
var id = ObjectId.GenerateNewId();
var db = client.GetDatabase("test");
var inventory = db.GetCollection<Inventory>("inventory");
var filter = Builders<Inventory>.Filter
.In(x => x.Id, id);
var update = Builders<Inventory>.Update.Pipeline(
new PipelineStagePipelineDefinition<Inventory, Inventory>(
new PipelineStageDefinition<Inventory, Inventory>[]
{
#"{ ""$set"": { ""TotalCost"": { ""$add"": [""$TotalCost"", { ""$multiply"": [-500, { ""$divide"": [""$TotalCost"", ""$Quantity""] }] }] } } }",
}));
await inventory.UpdateOneAsync(filter, update)
.ConfigureAwait(false);

How to avoid posting duplicates into elasticsearch using Nest .NET 6.x?

When data from a device goes into the elastic there are duplicates. I like to avoid this duplicates. I'm using a object of IElasticClient, .NET and NEST to put data.
I searched for a method like ElasticClient.SetDocumentId(), but cant find.
_doc doc = (_doc)obj;
HashObject hashObject = new HashObject { DataRecordId = doc.DataRecordId, TimeStamp = doc.Timestamp };
// hashId should be the document ID.
int hashId = hashObject.GetHashCode();
ElasticClient.IndexDocumentAsync(doc);
I would like to update the data set inside the Elastic instead of adding one more same object right now.

Assuming the following set up
var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
var settings = new ConnectionSettings(pool)
.DefaultIndex("example")
.DefaultTypeName("_doc");
var client = new ElasticClient(settings);
public class HashObject
{
public int DataRecordId { get; set; }
public DateTime TimeStamp { get; set; }
}
If you want to set the Id for a document explicitly on the request, you can do so with
Fluent syntax
var indexResponse = client.Index(new HashObject(), i => i.Id("your_id"));
Object initializer syntax
var indexRequest = new IndexRequest<HashObject>(new HashObject(), id: "your_id");
var indexResponse = client.Index(indexRequest);
both result in a request
PUT http://localhost:9200/example/_doc/your_id
{
"dataRecordId": 0,
"timeStamp": "0001-01-01T00:00:00"
}
As Rob pointed out in the question comments, NEST has a convention whereby it can infer the Id from the document itself, by looking for a property on the CLR POCO named Id. If it finds one, it will use that as the Id for the document. This does mean that an Id value ends up being stored in _source (and indexed, but you can disable this in the mappings), but it is useful because the Id value is automatically associated with the document and used when needed.
If HashObject is updated to have an Id value, now we can just do
Fluent syntax
var indexResponse = client.IndexDocument(new HashObject { Id = 1 });
Object initializer syntax
var indexRequest = new IndexRequest<HashObject>(new HashObject { Id = 1});
var indexResponse = client.Index(indexRequest);
which will send the request
PUT http://localhost:9200/example/_doc/1
{
"id": 1,
"dataRecordId": 0,
"timeStamp": "0001-01-01T00:00:00"
}
If your documents do not have an id field in the _source, you'll need to handle the _id values from the hits metadata from each hit yourself. For example
var searchResponse = client.Search<HashObject>(s => s
.MatchAll()
);
foreach (var hit in searchResponse.Hits)
{
var id = hit.Id;
var document = hit.Source;
// do something with them
}

Thank you very much Russ for this detailed and easy to understand description! :-)
The HashObject should be just a helper to get a unique ID from my real _doc object. Now I add a Id property to my _doc class and the rest I will show with my code below. I get now duplicates any more into the Elastic.
public void Create(object obj)
{
_doc doc = (_doc)obj;
string idAsString = doc.DataRecordId.ToString() + doc.Timestamp.ToString();
int hashId = idAsString.GetHashCode();
doc.Id = hashId;
ElasticClient.IndexDocumentAsync(doc);
}

Comparing two fields of mongo collection using c# driver in mono

Am completely new to Mongodb and C# driver.
Development is being done using Monodevelop on Ubuntu 14.04 and Mongodb's version is 3.2.10 :
Currently my code has a POCO as below:
public class User
{
public String Name { get; set;}
public DateTime LastModifiedAt { get; set;}
public DateTime LastSyncedAt { get; set;}
public User ()
{
}
}
Have been able to create a collection and also to add users.
How do I find users, whose LastModifiedAt timestamp is greater than LastSyncedAt timestamp ? Did some searching, but haven't been able to find the answer.
Any suggestions would be of immense help
Thanks

Actually, it is not very simple. This should be possible with querysuch as :
var users = collection.Find(user => user.LastModifiedAt > user.LastSyncedAt).ToList();
But unfortunetly MongoDriver could not translate this expression.
You could either query all Users and filter on the client side:
var users = collection.Find(Builders<User>.Filter.Empty)
.ToEnumerable()
.Where(user => user.LastModifiedAt > user.LastSyncedAt)
.ToList();
Or send json query, because MongoDb itself is able to do it:
var jsonFliter = "{\"$where\" : \"this.LastModifiedAt>this.LastSyncedAt\"}";
var users = collection.Find(new JsonFilterDefinition<User>(jsonFliter))
.ToList();
And, yes, you need an Id - Property for your model class, i haven't mentioned it first, because i thought you do have one, just not posted in the question.

There is another way to do it. First lets declare collection:
var collection = Database.GetCollection<BsonDocument>("CollectionName");
Now lets add our project:
var pro = new BsonDocument {
{"gt1", new BsonDocument {
{ "$gt", new BsonArray(){ "$LastModifiedAt", "$LastSyncedAt" }
}
} },
{"Name", true },
{"LastModifiedAt", true },
{"LastSyncedAt", true }
};
Now lets add our filter:
var filter = Builders<BsonDocument>.Filter.Eq("gt1", true);
We'll aggregate our query:
var aggregate = collection.Aggregate(new AggregateOptions { AllowDiskUse = true })
.Project(pro)
.Match(filter)
Now our query is ready. We can check our query as follow:
var query=aggregate.ToString();
Lets run our query as follow:
var query=aggregate.ToList();
This with return the required data in list of bson documents.
This solution will work mongo c# driver 3.6 or above. Please comment in case of any confusion. Hopefully i'll able to explain this.

ElasticSearch accent insensitive query with NEST C# client

I´m trying to make a query in ElasticSearch with the NEST c# client a query without accent, my data has portuguese latin word with accent. See the code bellow:
var result = client.Search<Book>(s => s
.From(0)
.Size(20)
.Fields(f => f.Title)
.FacetTerm(f => f.OnField(of => of.Genre))
.Query(q => q.QueryString(qs => qs.Query("sao")))
);
This search did not find anything. My data on this index contains many titles like: "São Cristóvan", "São Gonçalo".
var settings = new IndexSettings();
settings.NumberOfReplicas = 1;
settings.NumberOfShards = 5;
settings.Analysis.Analyzers.Add("snowball", new Nest.SnowballAnalyzer { Language = "Portuguese" });
var idx5 = client.CreateIndex("idx5", settings);
How I can make query "sao" and find "são" using ElasticSearch?
I think have to create index with right properties, but I already tried many settings like.
or in Raw Mode:
{
"idx" : {
"settings" : {
"index.analysis.filter.jus_stemmer.name" : "brazilian",
"index.analysis.filter.jus_stop._lang_" : "brazilian"
}
}
}
How can I make the search and ignore accents?
Thanks Friends,

See the solution:
Connect on elasticsearch search with putty execute:
curl -XPOST 'localhost:9200/idx30/_close'
curl -XPUT 'localhost:9200/idx30/_settings' -d '{
"index.analysis.analyzer.default.filter.0": "standard",
"index.analysis.analyzer.default.tokenizer": "standard",
"index.analysis.analyzer.default.filter.1": "lowercase",
"index.analysis.analyzer.default.filter.2": "stop",
"index.analysis.analyzer.default.filter.3": "asciifolding",
"index.number_of_replicas": "1"
}'
curl -XPOST 'localhost:9200/idx30/_open'
Replace "idx30" with name of your index
Done!

I stumbled upon this thread since I got the same problem.
Here's the NEST code to create an index with an AsciiFolding Analyzer:
// Create the Client
string indexName = "testindex";
var uri = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(uri).SetDefaultIndex(indexName);
var client = new ElasticClient(settings);
// Create new Index Settings
IndexSettings set = new IndexSettings();
// Create a Custom Analyzer ...
var an = new CustomAnalyzer();
// ... based on the standard Tokenizer
an.Tokenizer = "standard";
// ... with Filters from the StandardAnalyzer
an.Filter = new List<string>();
an.Filter.Add("standard");
an.Filter.Add("lowercase");
an.Filter.Add("stop");
// ... just adding the additional AsciiFoldingFilter at the end
an.Filter.Add("asciifolding");
// Add the Analyzer with a name
set.Analysis.Analyzers.Add("nospecialchars", an);
// Create the Index
client.CreateIndex(indexName, set);
Now you can Map your Entity to this index (it's important to do this after you created the Index)
client.MapFromAttributes<TestEntity>();
And here's how such an entity could look like:
[ElasticType(Name = "TestEntity", DisableAllField = true)]
public class TestEntity
{
public TestEntity(int id, string desc)
{
ID = id;
Description = desc;
}
public int ID { get; set; }
[ElasticProperty(Analyzer = "nospecialchars")]
public string Description { get; set; }
}
There you go, the Description-Field is now inserted into the index without accents.
You can test this if you check the Mapping of your index:
http://localhost:9200/testindex/_mapping
Which then should look something like:
{
testindex: {
TestEntity: {
_all: {
enabled: false
},
properties: {
description: {
type: "string",
analyzer: "nospecialchars"
},
iD: {
type: "integer"
}
}
}
}
}
Hope this will help someone.

You'll want to incorporate an ACSII Folding filter into your analyzer to accomplish this. That will mean constructing the snowballanalyzer form tokenizers and filters (unless nest allows you to add filters to non-custom analyzers. ElasticSearch doesn't, though, as far as I know).
A SnowballAnalyzer incorporates:
StandardTokenizer
StandardFilter
(Add the ASCIIFolding Filter here)
LowercaseFilter
StopFilter (with the appropriate stopword set)
SnowballFilter (with the appropriate language)
(Or maybe here)
I would probably try to add the ASCIIFoldingFilter just before LowercaseFilter, although it might be better to add it as the very las step (after SnowballFilter). Try it both ways, see which works better. I don't know enough about either the Protuguese stemmer to say which would be best for sure.

Mongo db / C# - How to do bounding box queries?

Per title - I am using the official mongodb driver and I am looking to get all POIs within the given bounding box.
So far I have:
MongoCollection<BsonDocument> collection = _MongoDatabase.GetCollection("pois");
BsonArray lowerLeftDoc = new BsonArray(new[] { lowerLeft.Lon, lowerLeft.Lat});
BsonArray upperRightDoc = new BsonArray(new[] { upperRight.Lon, upperRight.Lat});
BsonDocument locDoc = new BsonDocument
{
{ "$within", new BsonArray(new[] { lowerLeftDoc, upperRightDoc})}
};
BsonDocument queryDoc = new BsonDocument { { "loc", locDoc }};
IList<TrafficUpdate> updates = new List<TrafficUpdate>();
var results = collection.Find(new QueryDocument(queryDoc)).SetLimit(limit);
foreach (BsonDocument t in results)
{
}
Unfortunatelly this doesnt work. I get:
QueryFailure flag was unknown $within type: 0 (response was { "$err" :
"unknown $within type: 0", "code" : 13058 }).

The problem in your code is that you didnt specify which geo operation you wanted to use. You only specified $within but missed where. you must specify $within along with $box (bounding box) , $polygon , $center or $centerSphere / $nearSphere.
This is the correct mongo syntax to run $box queries
> box = [[40.73083, -73.99756], [40.741404, -73.988135]]
> db.places.find({"loc" : {"$within" : {"$box" : box}}})
i am not sure about the c# mongodb syntax. But if you include '$box', it ll work

You could also use the Query builder for this query:
var query = Query.Within("loc", lowerLeft.Lon, lowerLeft.Lat,
upperRight.Lon, upperRight.Lat);
and let the query builder worry about the nitty gritty details of creating a properly formed query.