Develop Reference

C# MongoDB Update / Upsert List<Object> to Collection

I was searching for a way to Update / Upsert in MongoDB a List of items to a MongoDB collection.
Is there any way to do it or I have to use a loop to update the items one by one?
P.S: The problem is not making a method that would do the Job (one by one) but I want to avoid too much iterations with the MongoDB database.
Here's the method that I'm currently using:
public static void UpdateAll()
{
var client = new MongoClient("mongodb://server_ip:27017");
var db = client.GetDatabase("M_PROJECT");
var collection = db.GetCollection<Product>("products");
//Config.Products is a List<Product> that was previously retrieved from the same collection in MongoDB
foreach(Product product in Config.Products)
{
var filter = Builders<Product>.Filter.Eq(p => p.ID, product.ID);
var update = Builders<Product>.Update.Set("Name", product.Name).Set("Price", 20);
collection.UpdateOne(filter, update, new UpdateOptions() { IsUpsert = true });
}
}
And maybe without specifying every Field/Property that I want to update, but just applying the class instance.

try a bulk replace like so:
var models = new List<WriteModel<Product>>();
foreach (var product in Config.Products)
{
if (product.ID == null) product.ID = ObjectId.GenerateNewId();
var upsert = new ReplaceOneModel<Product>(
filter: Builders<Product>.Filter.Eq(p => p.ID, product.ID),
replacement: product)
{ IsUpsert = true };
models.Add(upsert);
}
collection.BulkWrite(models);
the thing to note here is that it will completely overwrite the data stored in the db with the data from your product class instances. but i think you'd be okay cause you said the products are retrieved from the same collection.
this is what my library MongoDB.Entities does internally to achieve products.Save()

Yes, you can use UpdateMany() instead of UpdateOne(). See https://www.mongodb.com/blog/post/quick-start-csharp-and-mongodb--update-operation and https://mongodb.github.io/mongo-csharp-driver/2.9/apidocs/html/M_MongoDB_Driver_IMongoCollectionExtensions_UpdateMany__1.htm for more details.

How to use Addfields in MongoDB C# Aggregation Pipeline

Mongo DB's Aggregation pipeline has an "AddFields" stage that allows you to project new fields to the pipeline's output document without knowing what fields already existed.
It seems this has not been included in the C# driver for Mongo DB (using version 2.7).
Does anyone know if there are any alternatives to this? Maybe a flag on the "Project" stage?

I'm not sure all the BsonDocument usage is required. Certainly not in this example where I append the textScore of a text search to the search result.
private IAggregateFluent<ProductTypeSearchResult> CreateSearchQuery(string query)
{
FilterDefinition<ProductType> filter = Builders<ProductType>.Filter.Text(query);
return _collection
.Aggregate()
.Match(filter)
.AppendStage<ProductType>("{$addFields: {score: {$meta:'textScore'}}}")
.Sort(Sort)
.Project(pt => new ProductTypeSearchResult
{
Description = pt.ExternalProductTypeDescription,
Id = pt.Id,
Name = pt.Name,
ProductFamilyId = pt.ProductFamilyId,
Url = !string.IsNullOrEmpty(pt.ShopUrl) ? pt.ShopUrl : pt.TypeUrl,
Score = pt.Score
});
}
Note that ProductType does have a Score property defined as
[BsonIgnoreIfNull]
public double Score { get; set; }
It's unfortunate that $addFields is not directly supported and we have to resort to "magic strings"

As discussed here Using $addFields in MongoDB Driver for C# you can build the aggregation stage yourself with a BsonDocument.
To use the example from https://docs.mongodb.com/manual/reference/operator/aggregation/addFields/
{
$addFields: {
totalHomework: { $sum: "$homework" } ,
totalQuiz: { $sum: "$quiz" }
}
}
would look something like this:
BsonDocument expression = new BsonDocument(new List<BsonElement>() {
new BsonElement("totalHomeWork", new BsonDocument(new BsonElement("$sum", "$homework"))),
new BsonElement("totalQuiz", new BsonDocument(new BsonElement("$sum", "$quiz")))
});
BsonDocument addFieldsStage = new BsonDocument(new BsonElement("$addFields", expression));
IAggregateFluent<BsonDocument> aggregate = col.Aggregate().AppendStage(addFieldsStage);
expression being the BsonDocument representing
{
totalHomework: { $sum: "$homework" } ,
totalQuiz: { $sum: "$quiz" }
}
You can append additional stages onto the IAggregateFluent Object as normal
IAggregateFluent<BsonDocument> aggregate = col.Aggregate()
.Match(filterDefintion)
.AppendStage(addFieldsStage)
.Project(projectionDefintion);

Update collection from DbSet object via Linq

i know it is not complicated but i struggle with it.
I have IList<Material> collection
public class Material
{
public string Number { get; set; }
public decimal? Value { get; set; }
}
materials = new List<Material>();
materials.Add(new Material { Number = 111 });
materials.Add(new Material { Number = 222 });
And i have DbSet<Material> collection
with columns Number and ValueColumn
I need to update IList<Material> Value property based on DbSet<Material> collection but with following conditions
Only one query request into database
The returned data from database has to be limited by Number identifier (do not load whole database table into memory)
I tried following (based on my previous question)
Working solution 1, but download whole table into memory (monitored in sql server profiler).
var result = (
from db_m in db.Material
join m in model.Materials
on db_m.Number.ToString() equals m.Number
select new
{
db_m.Number,
db_m.Value
}
).ToList();
model.Materials.ToList().ForEach(m => m.Value= result.SingleOrDefault(db_m => db_m.Number.ToString() == m.Number).Value);
Working solution 2, but it execute query for each item in the collection.
model.Materials.ToList().ForEach(m => m.Value= db.Material.FirstOrDefault(db_m => db_m.Number.ToString() == m.Number).Value);
Incompletely solution, where i tried to use contains method
// I am trying to get new filtered collection from database, which i will iterate after.
var result = db.Material
.Where(x=>
// here is the reasonable error: cannot convert int into Material class, but i do not know how to solve this.
model.Materials.Contains(x.Number)
)
.Select(material => new Material { Number = material.Number.ToString(), Value = material.Value});
Any idea ? For me it is much easier to execute stored procedure with comma separated id values as a parameter and get the data directly, but i want to master linq too.

I'd do something like this without trying to get too cute :
var numbersToFilterby = model.Materials.Select(m => m.Number).ToArray();
...
var result = from db_m in db.Material where numbersToFilterBy.Contains(db_m.Number) select new { ... }

ElasticSearch accent insensitive query with NEST C# client

I´m trying to make a query in ElasticSearch with the NEST c# client a query without accent, my data has portuguese latin word with accent. See the code bellow:
var result = client.Search<Book>(s => s
.From(0)
.Size(20)
.Fields(f => f.Title)
.FacetTerm(f => f.OnField(of => of.Genre))
.Query(q => q.QueryString(qs => qs.Query("sao")))
);
This search did not find anything. My data on this index contains many titles like: "São Cristóvan", "São Gonçalo".
var settings = new IndexSettings();
settings.NumberOfReplicas = 1;
settings.NumberOfShards = 5;
settings.Analysis.Analyzers.Add("snowball", new Nest.SnowballAnalyzer { Language = "Portuguese" });
var idx5 = client.CreateIndex("idx5", settings);
How I can make query "sao" and find "são" using ElasticSearch?
I think have to create index with right properties, but I already tried many settings like.
or in Raw Mode:
{
"idx" : {
"settings" : {
"index.analysis.filter.jus_stemmer.name" : "brazilian",
"index.analysis.filter.jus_stop._lang_" : "brazilian"
}
}
}
How can I make the search and ignore accents?
Thanks Friends,

See the solution:
Connect on elasticsearch search with putty execute:
curl -XPOST 'localhost:9200/idx30/_close'
curl -XPUT 'localhost:9200/idx30/_settings' -d '{
"index.analysis.analyzer.default.filter.0": "standard",
"index.analysis.analyzer.default.tokenizer": "standard",
"index.analysis.analyzer.default.filter.1": "lowercase",
"index.analysis.analyzer.default.filter.2": "stop",
"index.analysis.analyzer.default.filter.3": "asciifolding",
"index.number_of_replicas": "1"
}'
curl -XPOST 'localhost:9200/idx30/_open'
Replace "idx30" with name of your index
Done!

I stumbled upon this thread since I got the same problem.
Here's the NEST code to create an index with an AsciiFolding Analyzer:
// Create the Client
string indexName = "testindex";
var uri = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(uri).SetDefaultIndex(indexName);
var client = new ElasticClient(settings);
// Create new Index Settings
IndexSettings set = new IndexSettings();
// Create a Custom Analyzer ...
var an = new CustomAnalyzer();
// ... based on the standard Tokenizer
an.Tokenizer = "standard";
// ... with Filters from the StandardAnalyzer
an.Filter = new List<string>();
an.Filter.Add("standard");
an.Filter.Add("lowercase");
an.Filter.Add("stop");
// ... just adding the additional AsciiFoldingFilter at the end
an.Filter.Add("asciifolding");
// Add the Analyzer with a name
set.Analysis.Analyzers.Add("nospecialchars", an);
// Create the Index
client.CreateIndex(indexName, set);
Now you can Map your Entity to this index (it's important to do this after you created the Index)
client.MapFromAttributes<TestEntity>();
And here's how such an entity could look like:
[ElasticType(Name = "TestEntity", DisableAllField = true)]
public class TestEntity
{
public TestEntity(int id, string desc)
{
ID = id;
Description = desc;
}
public int ID { get; set; }
[ElasticProperty(Analyzer = "nospecialchars")]
public string Description { get; set; }
}
There you go, the Description-Field is now inserted into the index without accents.
You can test this if you check the Mapping of your index:
http://localhost:9200/testindex/_mapping
Which then should look something like:
{
testindex: {
TestEntity: {
_all: {
enabled: false
},
properties: {
description: {
type: "string",
analyzer: "nospecialchars"
},
iD: {
type: "integer"
}
}
}
}
}
Hope this will help someone.

You'll want to incorporate an ACSII Folding filter into your analyzer to accomplish this. That will mean constructing the snowballanalyzer form tokenizers and filters (unless nest allows you to add filters to non-custom analyzers. ElasticSearch doesn't, though, as far as I know).
A SnowballAnalyzer incorporates:
StandardTokenizer
StandardFilter
(Add the ASCIIFolding Filter here)
LowercaseFilter
StopFilter (with the appropriate stopword set)
SnowballFilter (with the appropriate language)
(Or maybe here)
I would probably try to add the ASCIIFoldingFilter just before LowercaseFilter, although it might be better to add it as the very las step (after SnowballFilter). Try it both ways, see which works better. I don't know enough about either the Protuguese stemmer to say which would be best for sure.

Update List<string> in mongoDB

I have a list of strings I want to update in MongoDB using C# driver. How do I do this?
List<string> Images = someList;
var update = Update.Set("Images", Images);
collection.Update(query, update, UpdateFlags.Upsert);
this will give me an error saying that 'Images' is not BsonValue.. How do I convert string list to the bsonvalue? Thanks

It looks like Update.Set is wanting a BsonValue and you can't implicitly convert from List to BsonValue.
You look like you are doing Upserts anyway, could you use Save instead?
One way to solve this issue using Serialization and Save would be:
public class SomeListClass
{
public ObjectId id { get; set; }
public List<string> Images { get; set; }
}
SomeListClass slc = new SomeListClass();
slc.Images = someList;
collection.Save(slc);

That's what I did to solve it: I converted that list to BsonArray:
List<string> Images = someList;
var update = Update.Set("Images", new BsonArray(Images));
collection.Update(query, update, UpdateFlags.Upsert);

If you are using the latest 1.5 version of the C# driver you can also use the new typed Update builder and let it figure out the correct element name and how to serialize the new value.
List<string> images = someList;
var update = Update<SomeListClass>.Set(x => x.Images, images);