Develop Reference

Execute raw query that could handle all numerous of documents mongo driver c#

I have the situation that filtering condition on collection is quite long. And I prefer to write it by hand and using the mongoDb RunCommandAsync method in C# driver to get the Bson Document as result then Deserialize it back to object.
As the doc said it will return a BsonDocument, instead of a Cursor to iterate with.
I made a small test from a collection with some dummy filter condition as below, and i got ~215k docs from robo 3T, but when it comes to RunCommandAsync in c#, it default return 101 document. I attempted to increase the batchSize to 200.000, but the execution can only return ~22k docs(which i guess it's the limitation of 4MB per batch operation).
var cmdDoc = #"{
'find': 'NotificationHistories',
'filter': {'CreatedTime': {$exists: true}}
'batchSize': 200000,
'singleBatch': true
}";
var cmd = new JsonCommand<BsonDocument>(cmdDoc);
var res = await database.RunCommandAsync(cmd, ReadPreference.PrimaryPreferred, CancellationToken.None);
So my final question is:
Is there anyway we can execute raw query that could handle all the numerous of documents ?
The result can be any of Cursor, BsonDocument, Array of BsonDocument,...
Or there's just no possible way to workaround this but use the normal Find or Aggregate method of Collection ?
Any clue would be appriciate, since i felt its quite long and cumbersome writing those especially with Unwind, Group, Lookup,...

101 records is a number of records in the firstBatch of your cursor, iterating of next records(batch by batch) should be done via getMore command (but I think it's a wrong that you think in this way, I see no reason to avoid a regular way).

MongoDB and returning collections efficiently

I am very new to Mongo (this is actually day 1) and using the C# driver that is available for it. One thing that I want to know (as I am not sure how to word it in Google) is how does mongo handle executing queries when I want to grab a part of the collection.
What I mean by this is that I know that with NHibernate and EF Core, the query is first built and it will only fire when you cast it. So say like an IQueryable to IEnnumerable, .ToList(), etc.
Ex:
//Query is fired when I call .ToList, until that point it is just building it
context.GetLinqQuery<MyObject>().Where(x => x.a == 'blah').ToList();
However, with Mongo's examples it appears to me that if I want to grab a filtered result I will first need to get the collection, and then filter it down.
Ex:
var collection = _database.GetCollection<MyObject>("MyObject");
//Empty filter for ease of typing for example purposes
var filter = Builders<MyObject>.Filter.Empty;
var collection.Find(filter).ToList();
Am I missing something here, I do not think I saw any overload in the GetCollection method that will accept a filter. Does this mean that it will first load the whole collection into memory, then filter it? Or will it still be building the query and only execute it once I call either .Find or .ToList on it?
I ask this because at work we have had situations where improper positioning of .ToList() would result is seriously weak performance. Apologies if this is not the right place to ask.
References:
https://docs.mongodb.com/guides/server/read_queries/

The equivalent to your context.GetLinqQuery<MyObject>() would be to use AsQueryable:
collection.AsQueryable().Where(x => x.a == "blah").ToList();
The above query will be executed server side* and is equivalent to:
collection.Find(Builders<MyObject>.Filter.Eq(x => x.a, "blah")).ToEnumerable().ToList();
* The docs state that:
Only LINQ queries that can be translated to an equivalent MongoDB query are supported. If you write a LINQ query that can’t be translated you will get a runtime exception and the error message will indicate which part of the query wasn’t supported.

Broken Skip method in let subquery

In my application there is an issue part that allows questions and responses. Something like (Pseudo code, as this is actually generated from Entity Framework):
class Response
{
string Author;
string Comment;
DateTime Date;
}
class Issue
{
IEnumerable<Response> Responses;
}
We have a summary page where we just want to show the last two responses. I tried a linq query like this:
from issue in db.Issue
let responses = from response in issue.Responses orderby response.Date
select new
{
Issue = issue,
Question = responses.FirstOrDefault(),
Answer = responses.Skip(1).FirstOrDefault()
}
But this gives me the error that Skip can only be used on ordered collections. I checked responses and it was an IOrderedEnummerable. I thought maybe the problem was that it was Enumerable instead of IOrderedQueryable, and saw that this happened because issue.Response is a collection so I switched the let statement to be:
let response = from response in db.Responses where response.IssueId = issue.ID // etc.
but this did not resolve the issue (but response did become an IOrderedQueryable) so I'm not really sure why entity won't accept the skip here. If I put the skip in the let statement, it works without problem (but then I can't get the first response). The issue seams to only occur by trying to put a portion of this statement in a variable before using it.

The problem here is: how will/should EF translate your query into a SQL statement? There is no straightforward SQL equivalent of Skip(1). Just try to write your query in SQL and you should see what I mean.
If you want an "easy" solution, then just get all responses from the DB and identify the ones you need in code.
If you want to minimize the data being read from the DB, the solutions might range from creating a view to writing a stored procedure to changing your tables so that your tables better reflect the data model in the application.

I'm not quite sure what's going on here, but wouldn't this be maybe a little simpler:
var theResponse = db.Issue.Select(i => new {
Issue = i,
Question = i.Responses.FirstOrDefault(),
Answer = i.Responses.OrderBy(r => r.Date).Skip(1).FirstOrDefault()
});
But this is also weird, because you are getting a full Issue object with all of its properties and whatnot, including the Reponse objects and stuffing it back into an Issue property of your dynamic type beside all of the Response objects...

MongoDB performance problems in Unity Game

We are decided to use mongo db in our game for real time database but the performance of the search result is not acceptable. These are the test result with 15.000 documents and 17 fields(strings, int,float)
// 14000 ms
MongoUrl url = new MongoUrl("url-adress");
MongoClient client = new MongoClient(url);
var server = client.GetServer();
var db = server.GetDatabase("myDatabase");
var collection = db.GetCollection<PlayerFields>("Player");
var ranks = collection.FindAll().AsQueryable().OrderByDescending(p=>p.Score).ToList().FindIndex(FindPlayer).Count();
This one is the worst. //.ToList() is for testing purposes. Don't use in production code.
Second test
//9000 ms
var ranks = collection.FindAll().AsQueryable().Where(p=>p.Score < PlayerInfos.Score).Count();
Third test
//2000 ms
var qq = Query. GT("Kupa", player.Score);
var ranks = collection.Find( qq ).Where(pa=>(pa.Win + pa.Lose + pa.Draw) != 0 );
Is there any other way to make fast searches in mongo with C# .Net 2.0. We want to get player's rank according to users score and rank them.

To caveat this, I've not been a .NET dev for a few years now, so if there is a problem with the c# driver then I can't really comment, but I've got a good knowledge of Mongo so hopefully I'll help...
Indexes
Indexes will help you out a lot here. As you are ordering and filtering on fields which aren't indexed, this will only cause you problems as the database gets larger.
Indexes are direction specific (ascending/descending). Meaning that your "Score" field should be indexed descending:
db.player.ensureIndex({'Score': -1}) // -1 indicating descending
Queries
Also, Mongo is really awesome (in my opinion) and it doesn't look like you're using it to be best of it's abilities.
Your first call:
var ranks = collection.FindAll().AsQueryable().OrderByDescending(p=>p.Score).ToList().FindIndex(FindPlayer).Count();
It appears (this is where my .NET knowledge may be letting me down) that you're retrieving the entire collection ToList(), then filtering it in memory (FindPlayer predicate) in order to retrieve a subset of data. I believe that this will be evaluating the entire curser (15.000 documents) into the memory of your application.
You should update your query so that Mongo is doing the work rather than your application.
Given your other queries are filtering on Score, adding the index as described above should drastically increase the performance of these other queries
Profiling
If the call that you're expecting to make when run from the mongo cli is behaving as expected, it could be that the driver is making slightly different queries.
In the mongo CLI, you will first need to set the profiling:
db.setProfilingLevel(2)
You can then query the profile collection to see what queries are actually being made:
db.system.profile.find().limit(5).sort({ts: -1}).pretty()
This will show you the 5 most recent calls.

Query projection with MongoDB 10gen driver

Recently i was playing with mongodb official driver.
The problem that i've encountered was how to make query projection.
Example if i have a persisted object
class A{
id
PropA
PropB
List<LargeObjects>
}
How can i only retrieve id, PropA and PropB instead of retrieving the whole object ?
How can be done with the mongodb official c# driver ?

Query projection is available through:
MongoCollection<>.Find().SetFields(include/exclude);

As of v1.8 of the official 10gen MongoDB C# driver, (and as Zambonilli eluded to in a previous answer) the Select linq operator will always be performed on the client side, not on the db server.
Documentation (also provided by Sunil Raj in a previous answer): http://docs.mongodb.org/ecosystem/tutorial/use-linq-queries-with-csharp-driver/
Almost half way down the page, under the "Select" linq query operator, is a large red box that reads:
Warning: Select does not result in fewer fields being returned from the server. The entire document is pulled back and passed to the native Select method. Therefore, the projection is performed client side.

You can use the below linq query for this:
//NB: Not tested
MongoCollection<BsonDocument> Acollection = _db.database.GetCollection<BsonDocument>("A");
var resultlist = (from k in Acollection.AsQueryable<A>()
select k.id,k.PropA,k.PropB);
More information on the linq queries can be found here:
http://www.mongodb.org/display/DOCS/CSharp+Driver+LINQ+Tutorial#CSharpDriverLINQTutorial-SupportedLINQqueryoperators

Take a look at FluentMongo:
https://github.com/craiggwilson/fluent-mongo/wiki/Linq
It is available on Nuget aswell, search for "fluentmongo"

Use FindAs<> () with a type that includes only the fields you want. See docs.
Add the [BsonIgnoreExtraElements] or the [BsonExtraElements] attribute on that class. See docs.

Using the Mongo profiler I was able to determine that sometimes the Linq results are projected on the client. Therefore it depends on what your client needs are. If you want to return a resulting document with partial data from the Mongo server then you'll want to use Marjan or Ian's answers. Otherwise, if you want to read the record and project it to a different data type then use Linq.