I'm using collection.FindOneAndDeleteAsync, but this uses a ton of cpu when used to get many documents. What's the best way to go about finding multiple documents(anywhere from 100 to 50k) and delete, using the c# mongo driver?
Thanks
You need to Find the docs you want to delete, and then delete them using DeleteMany with a filter of _id: {$in: ids}, where ids is an enumerable of the _id values of those documents.
C# example:
public class Entity
{
public ObjectId id { get; set; }
public string name { get; set; }
}
// Find the documents to delete
var test = db.GetCollection<Entity>("test");
var filter = new BsonDocument();
var docs = test.Find(filter).ToList();
// Get the _id values of the found documents
var ids = docs.Select(d => d.id);
// Create an $in filter for those ids
var idsFilter = Builders<Entity>.Filter.In(d => d.id, ids);
// Delete the documents using the $in filter
var result = test.DeleteMany(idsFilter);
Related
I got an .net core application that is pretty straight forward it is using REST to add and download objects to and from mongo db. Adding items works really well. Getting a list that contains all items aswell, but when I try to access one using id then everytime I get null. What should i change to make this piece of code work. It means get a Tool object from database using it unique ID when there's one matching in database.
Here's a object in database
Here's my repository class
private IMongoCollection<Tool> Tools => _database.GetCollection<Tool>("Tools");
public async Task<Tool> GetAsync(Guid id) =>
await Tools.AsQueryable().FirstOrDefaultAsync(tool => tool.Id == id);
Argument looks like that when I check it out in debugger "{ee1aa9fa-5d17-464c-a8ba-f685203b911f}"
Edit
Tool Class Properties
public Guid Id { get; protected set; }
public string Model { get; protected set; }
public string Brand { get; protected set; }
public string Type { get; protected set; }
public uint Box { get; protected set; }
Fixed check comments
Project on github
The easiest way to do this in C# MongoDB Driver is to set a global GuidRepresentation setting which can be found on the BsonDefaults object. This is a global setting and will effect all serialization/deserialization of GUIDs in to Bson Binary Objects.
BsonDefaults.GuidRepresentation = GuidRepresentation.PythonLegacy;
var collection = new MongoClient().GetDatabase("test").GetCollection<ClassA>("test");
var item = collection.Find(x => x.MyId == new Guid("ee1aa9fa-5d17-464c-a8ba-f685203b911f"))
.FirstOrDefault();
The second option is to convert the GUID manually from a LUUID to a CSUUID, for this there is a helper class within the MongoDB driver of GuidConverter, with this it converts the GUID in to byte[] which is normally used for storage but we can use it for our query.
BsonDefaults.GuidRepresentation = GuidRepresentation.CSharpLegacy;
var collection = new MongoClient().GetDatabase("test").GetCollection<ClassA>("test");
var luuid = new Guid("0798f048-d8bb-7048-bb92-7518ea4272cb");
var bytes = GuidConverter.ToBytes(luuid, GuidRepresentation.PythonLegacy);
var csuuid = new Guid(bytes);
var item = collection.Find(x => x.MyId == csuuid)
.FirstOrDefault();
I also noticed that you're using Robo 3T (formerly Robomongo), within this application you can set how GUIDs are displayed by going to Options -> Legacy UUID Encodings
I'm trying to create multiple unique indexes using c# MongoDB driver connecting to Azure DocumentDB instance, but I'm receiving the following exception when trying to create the second unique index:
MongoDB.Driver.MongoCommandException: 'Command createIndexes failed: Message: {"Errors":["The number of unique keys cannot be greater than 1."]}
I can't seem to find any documentation regarding the number of unique keys for Azure DocumentDB collection. Note that this exception does not occur when using actual MongoDB instance.
var keys = Builders<ProductEntity>.IndexKeys.Ascending(p => p.UPC);
var options = new CreateIndexOptions<ProductEntity>() { Name = "UX_UPC", Unique = true, Sparse = true };
var result = await _collection.Indexes.CreateOneAsync(keys, options);
keys = Builders<ProductEntity>.IndexKeys.Ascending(p => p.Manufacturer).Ascending(p => p.MPN);
options = new CreateIndexOptions<ProductEntity>() { Name = "UX_Manufacturer_MPN", Unique = true, Sparse = true };
result = await _collection.Indexes.CreateOneAsync(keys, options);
public class ProductEntity
{
public Guid Id { get; set; }
public string UPC { get; set; }
public string MPN { get; set; }
public string Manufacturer { get; set; }
}
From this blog, we could find it does not support for Unique indexes currently.
General availability is just the beginning for all the features and improvements we have in stored for DocumentDB: API for MongoDB. In the near future, we will be releasing support for Unique indexes and a couple major performance improvements.
This thread discussed Cannot create index in Azure DocumentDb with Mongodb protocol, you could refer to it.
We are looking to switch from a relational database to elastic search and I am trying to get some basic code up and running with Nest. We have existing objects which use guids for ids that I would like to save into an elastic search index.
I don't want to add any specific attributes as the class is used in different applications and I don't want to add unnecessary dependencies to Nest.
Right now my code looks like this:
var node = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(node)
settings.DefaultIndex = "test";
var client = new ElasticClient(settings);
var testItem = new TestType { Id = Guid.NewGuid(), Name = "Test", Value = "10" };
var response = client.Index(testItem);
With TestType as:
public class TestType
{
public Guid Id { get; set; }
public string Name { get; set; }
public decimal Value { get; set; }
}
However I get an error like:
ServerError: 400Type: mapper_parsing_exception Reason: "failed to
parse [id]" CausedBy: "Type: number_format_exception Reason: "For
input string: "c9c0ed42-86cd-4a94-bc86-a6112f4c9188""
I think I need to specify a mapping that tells the server the Id is a string, but I can't find any examples or documentation on how I do this without using the attributes.
Assuming you're using Elasticsearch 2.x and NEST 2.x (e.g. latest of both at time of writing is Elasticsearch 2.3.5 and NEST 2.4.3), then NEST will automatically infer the id of a POCO by default from the Id property. In the case of a GUID id, this will be saved as a string in Elasticsearch.
Here's an example to get you going
void Main()
{
var node = new Uri("http://localhost:9200");
var settings = new ConnectionSettings(node)
// default index to use if one is not specified on the request
// or is not set up to be inferred from the POCO type
.DefaultIndex("tests");
var client = new ElasticClient(settings);
// create the index, and explicitly provide a mapping for TestType
client.CreateIndex("tests", c => c
.Mappings(m => m
.Map<TestType>(t => t
.AutoMap()
.Properties(p => p
// don't analyze ids when indexing,
// so they are indexed verbatim
.String(s => s
.Name(n => n.Id)
.NotAnalyzed()
)
)
)
)
);
var testItem = new TestType { Id = Guid.NewGuid(), Name = "Test", Value = "10" };
// now index our TestType instance
var response = client.Index(testItem);
}
public class TestType
{
public Guid Id { get; set; }
public string Name { get; set; }
public decimal Value { get; set; }
}
Take a look at the Automapping documentation for more examples of how to explicitly map a POCO for controlling norms, analyzers, multi_fields, etc.
What I normally do is to have a separate class that is only specific to Elasticsearch. And use Automapper to map that into a DTO or ViewModel, or Model into the Elasticsearch Document.
That way, you won't have to expose an object that have a dependency in NEST and attributes that might be specific only to Elasticsearch.
Another good reason is that normally, documents in ES are flat, so you would normally flatten your objects before you index them to ES.
I am starting off with azure document db. I was trying to update an existing document. When I use the following query everything works:
dynamic Team2Doc = client.CreateDocumentQuery<Document>(documentCollection.DocumentsLink).Where(d => d.Id == "t002").AsEnumerable().FirstOrDefault();
Team2Doc.TeamName = "UPDATED_TEAM_2";
await client.ReplaceDocumentAsync(Team2Doc);
but if use the below code:
dynamic Team2Doc = client.CreateDocumentQuery<Document>(documentCollection.DocumentsLink).Where(d => d.TeamName== "team1").AsEnumerable().FirstOrDefault();
Team2Doc.TeamName = "UPDATED_TEAM_2";
await client.ReplaceDocumentAsync(Team2Doc);
I get this error:
"The best overloaded method match for
'Microsoft.Azure.Documents.Client.DocumentClient.ReplaceDocumentAsync(Microsoft.Azure.Documents.Document,
Microsoft.Azure.Documents.Client.RequestOptions)' has some invalid
arguments"
Is there anyway to retrieve a document by one of the properties and update the document?
The where clause is trying to query the property TeamName which does not exist in Document class.
Changing the type of the queryable to your data model should fix it.
For example, say you have the following data model:
public class EmployeeDocument : Document
{
// Other properties that you may have similarly defined ....
public class string TeamName
{
get
{
return this.GetValue<string>("TeamName");
}
set
{
this.SetValue("TeamName", value);
}
}
}
Then you can modify your query like this:
var team2Doc = client.CreateDocumentQuery<EmployeeDocument>(documentCollection.DocumentsLink).Where(d => d.TeamName== "team1").AsEnumerable().FirstOrDefault();
team2Doc.TeamName = "UPDATED_TEAM_2";
await client.ReplaceDocumentAsync(team2Doc);
Note that you have to use the EmployeeDocument, instead of the Document class, while creating the document queryable. That will let you query on EmployeeDocument properties.
SQL Version
Creating a document model for each of your existing data models may not be feasible if you have a large number of data models. In that case you may want to try out the SQL query syntax.
Refer to Aravind's answer in this post. The example he uses is for deleting documents, but it can be easily modified to update them too.
You can also create a model with Id:
public class Employee
{
[JsonPropery("id")]
public class string Id { get; set; }
public class string TeamName { get; set; }
}
And then replace the document using it's Id:
var employee= client
.CreateDocumentQuery<Employee>(documentCollection.DocumentsLink)
.Where(d => d.TeamName== "team1")
.AsEnumerable()
.FirstOrDefault();
employee.TeamName = "team2";
var documentUri = UriFactory.CreateDocumentUri(databaseName, collectionName, employee.Id);
await client.ReplaceDocumentAsync(employee);
I am having trouble with expressing this query in C# MongoDB, I want it to return all the results of an objectID where it does not equal to "000000000000000000000000" which works in MongoVue; But I can't get it work in my program.
{"ProfilePictureId" : {$ne: new ObjectId ("000000000000000000000000")}}
I am using official C# driver:
var query = new QueryDocument();
foreach (BsonDocument book in col.Find(query))
{
...
}
You can build your query as follows:
var query = Query.NE("ProfilePictureId", ObjectId.Empty);
ObjectId.Empty returns an ObjectId composed of all zeroes.
Assuming that you are querying for documents of a class looking something like:
public class Profile {
public ObjectId ProfilePictureId { get; set; }
//... other attributes, construcotrs, methods etc...
}
You can also write your query using expression lambdas like this:
var query = Query<Profile>.NE(s => s.ProfilePictureId, ObjectId.Empty);