Develop Reference

Mongo.net - BypassDocumentValidation is not working

I'm trying to insert a json like this (fieldname with a "."), in a Net Core Console Project
{"name.field" : "MongoDB", "type" : "Database"}
Using the C# code belove:
-with InsertManyOptions with BypassDocumentValidation in true
var options = new InsertManyOptions
{
BypassDocumentValidation = true,
IsOrdered = false
};
await _collection.InsertManyAsync(items, options);
But I have this exception:
Element name 'name.field' is not valid
I´m using :
C# Mongo Driver 2.5
Net Core Project
MongoDB version 4.0.3
Any idea? Thanks!

The BypassDocumentValidation can be used to bypass the JSON Schema validation. The issue you are facing, however, is due to the C# driver which explicitly prevents the use of the dot symbol . as part of a field name.
This used to be required up until MongoDB v3.6 which officially added support for fields with ".".
Looking into the internals of the C# driver you can see that the BsonWriter.WriteName method calls contains this code which throws the Exception you're seeing:
if (!_elementNameValidator.IsValidElementName(name))
{
var message = string.Format("Element name '{0}' is not valid'.", name);
throw new BsonSerializationException(message);
}
The _elementNameValidator is something that is managed internally by the driver which in fact comes with a NoOpElementNameValidator that doesn't do any validations. The driver, however, won't use this validator for "normal" collections.
All that said, I would strongly advise against the use of field names with "unusual" characters anyway because this is likely to set you up for unexpected behaviour and all sorts of other issues down the road.
In order to get around this you can do one of the following things:
a) Write your own custom serializer which is an option that I would personally steer clear off if possible - it adds complexity that most of the time shouldn't be required.
b) Use the below helper extension (copied from one of the unit testing projects inside the driver) to convert the BsonDocument into a RawBsonDocument which can then successfully written to the server:
public static class RawBsonDocumentHelper
{
public static RawBsonDocument FromBsonDocument(BsonDocument document)
{
using (var memoryStream = new MemoryStream())
{
using (var bsonWriter = new BsonBinaryWriter(memoryStream, BsonBinaryWriterSettings.Defaults))
{
var context = BsonSerializationContext.CreateRoot(bsonWriter);
BsonDocumentSerializer.Instance.Serialize(context, document);
}
return new RawBsonDocument(memoryStream.ToArray());
}
}
public static RawBsonDocument FromJson(string json)
{
return FromBsonDocument(BsonDocument.Parse(json));
}
}
And then simply write the RawBsonDocument to the server:
RawBsonDocument rawDoc = RawBsonDocumentHelper.FromJson("{\"name.field\" : \"MongoDB\", \"type\" : \"Database\"}");
collection.InsertOne(rawDoc);

How to set MongoDB Change Stream 'OperationType' in the C# driver?

When running the new MongDB Server, version 3.6, and trying to add a Change Stream watch to a collection to get notifications of new inserts and updates of documents, I only receive notifications for updates, not for inserts.
This is the default way I have tried to add the watch:
IMongoDatabase mongoDatabase = mongoClient.GetDatabase("Sandbox");
IMongoCollection<BsonDocument> collection = mongoDatabase.GetCollection<BsonDocument>("TestCollection");
var changeStream = collection.Watch().ToEnumerable().GetEnumerator();
changeStream.MoveNext();
var next = changeStream.Current;
Then I downloaded the C# source code from MongoDB to see how they did this. Looking at their test code for change stream watches, they create a new document(Insert) and then change that document right away(Update) and THEN set up the Change Stream watch to receive an 'update' notification.
No example is given on how to watch for 'insert' notifications.
I have looked at the Java and NodeJS examples, both on MongoDB website and SO, which seems to be straight forward and defines a way to see both Inserts and Updates:
var changeStream = collection.watch({ '$match': { $or: [ { 'operationType': 'insert' }, { 'operationType': 'update' } ] } });
The API for the C# driver is vastly different, I would have assumed they would have kept the same API for C# as Java and NodeJS. I found no or very few examples for C# to do the same thing.
The closest I have come is with the following attempt but still fails and the documentation for the C# version is very limited (or I have not found the right location). Setup is as follows:
String json = "{ '$match': { 'operationType': { '$in': ['insert', 'update'] } } }";
var options = new ChangeStreamOptions { FullDocument = ChangeStreamFullDocumentOption.UpdateLookup };
PipelineDefinition<ChangeStreamDocument<BsonDocument>, ChangeStreamDocument<BsonDocument>> pipeline = new EmptyPipelineDefinition<ChangeStreamDocument<BsonDocument>>().Match(Builders<ChangeStreamDocument<BsonDocument>>.Filter.Text(json,"json"));
Then running the statement below throws an Exception:
{"Command aggregate failed: $match with $text is only allowed as the
first pipeline stage."}
No other Filter options has worked either, and I have not found a way to just enter the JSON as a string to set the 'operationType'.
var changeStream = collection.Watch(pipeline, options).ToEnumerable().GetEnumerator();
changeStream.MoveNext();
var next = changeStream.Current;
My only goal here is to be able to set the 'operationType' using the C# driver. Does anyone know what I am doing wrong or have tried this using the C# driver and had success?
After reading though a large number of webpages, with very little info on the C# version of the MongoDB driver, I am very stuck!
Any help would be much appreciated.

Here is a sample of code I've used to update the collection Watch to retrieve "events" other than just document updates.
IMongoDatabase sandboxDB = mongoClient.GetDatabase("Sandbox");
IMongoCollection<BsonDocument> collection = sandboxDB.GetCollection<BsonDocument>("TestCollection");
//Get the whole document instead of just the changed portion
ChangeStreamOptions options = new ChangeStreamOptions() { FullDocument = ChangeStreamFullDocumentOption.UpdateLookup };
//The operationType can be one of the following: insert, update, replace, delete, invalidate
var pipeline = new EmptyPipelineDefinition<ChangeStreamDocument<BsonDocument>>().Match("{ operationType: { $in: [ 'replace', 'insert', 'update' ] } }");
var changeStream = collection.Watch(pipeline, options).ToEnumerable().GetEnumerator();
changeStream.MoveNext(); //Blocks until a document is replaced, inserted or updated in the TestCollection
ChangeStreamDocument<BsonDocument> next = changeStream.Current;
enumerator.Dispose();
The EmptyPiplineDefinition...Match() argument could also be:
"{ $or: [ {operationType: 'replace' }, { operationType: 'insert' }, { operationType: 'update' } ] }"
If you wanted to use the $or command, or
"{ operationType: /^[^d]/ }"
to throw a little regex in there. This last one is saying, I want all operationTypes unless they start with the letter 'd'.

Prepared statement caching issue in Cassandra Csharp driver

I believe I have found a bug with the logic of how a prepared statement is cached in the StatementFactory in the Cassandra csharp driver (version 2.7.3). Here is the use case.
Guid key = Guid.NewGuid(); // your key
ISession session_foo = new Session("foo"); //This is pseudo code
ISession session_bar = new Session("bar");
var foo_mapper = new Mapper(session_foo); //table foo_bar
var bar_mapper = new Mapper(session_bar); //table foo_bar
await Task.WhenAll(
foo_mapper.DeleteAsync<Foo>("WHERE id = ?", key),
bar_mapper.DeleteAsync<Bar>("WHERE id = ?", key));
We have found that after running this deletes, only the first request is succeeding. After diving in the the source code of StatementFactory
public Task<Statement> GetStatementAsync(ISession session, Cql cql)
{
if (cql.QueryOptions.NoPrepare)
{
// Use a SimpleStatement if we're not supposed to prepare
Statement statement = new SimpleStatement(cql.Statement, cql.Arguments);
SetStatementProperties(statement, cql);
return TaskHelper.ToTask(statement);
}
return _statementCache
.GetOrAdd(cql.Statement, session.PrepareAsync)
.Continue(t =>
{
if (_statementCache.Count > MaxPreparedStatementsThreshold)
{
Logger.Warning(String.Format("The prepared statement cache contains {0} queries. Use parameter markers for queries. You can configure this warning threshold using MappingConfiguration.SetMaxStatementPreparedThreshold() method.", _statementCache.Count));
}
Statement boundStatement = t.Result.Bind(cql.Arguments);
SetStatementProperties(boundStatement, cql);
return boundStatement;
});
}
You can see that the cache only uses the cql statement. In our case, we have the same table names in different keyspaces (aka sessions). Our cql statement in both queries look the same. ie DELETE FROM foo_bar WHERE id =?.
If I had to guess, I would say that a simple fix would be to combine the cql statement and keyspace together as the cache key.
Has anyone else run into this issue before?

As a simple workaround, I am skipping the cache by using the DoNotPrepare
await _mapper.DeleteAsync<Foo>(Cql.New("WHERE id = ?", key).WithOptions(opt => opt.DoNotPrepare()));
I also found an open issue with Datastax

There is an open ticket to fix this behaviour.
As a workaround, you can use a different MappingConfiguration instances when creating the Mapper:
ISession session1 = cluster.Connect("ks1");
ISession session2 = cluster.Connect("ks2");
IMapper mapper1 = new Mapper(session1, new MappingConfiguration());
IMapper mapper2 = new Mapper(session2, new MappingConfiguration());
Or, you can reuse a single ISession instance and fully-qualify your queries to include the keyspace.
MappingConfiguration.Global.Define(
new Map<Foo>()
.TableName("foo")
.KeyspaceName("ks1"),
new Map<Bar>()
.TableName("bar")
.KeyspaceName("ks2"));
ISession session = cluster.Connect();
IMapper mapper = new Mapper(session);

MongoDb Connect to Replica Set Primary Issue C#

How do I get back the name of the primary database? Lets say database3 was primary
Thanks
var connString = "mongodb://database1,database2,database3/?replicaSet=repl";
var client = new MongoClient(connString);
var server = client.GetServer().Instances.FirstOrDefault(server => server.IsPrimary);
var address = server.Address;

Having looked at the source code of the MongoDB driver, there is no straightforward way to get the name of the primary server from the driver itself. However, you can query server name in MOngoDB by executing {isMaster:1} using RunCommand. You can then parse the primary server from the returned JSON document. This approach works regardless if you are connected to the primary or secondary server.
var mongoClient = new MongoClient(clientSettings);
var testDB = mongoClient.GetDatabase("test");
var cmd = new BsonDocument("isMaster", "1");
var result = testDB.RunCommand<BsonDocument>(cmd);
var primaryServer = result.Where(x => x.Name == "primary").FirstOrDefault().Value.ToString();

Thanks to Jaco for putting me on the right track I solved my problem by doing the following
public static string GetPrimaryDatabase()
{
var mongoClient = new MongoClient(clientSettings);
var server = mongoClient.GetServer();
var database = server.GetDatabase("test");
var cmd = new CommandDocument("isMaster", "1");
var result = database.RunCommand(cmd);
return result.Response.FirstOrDefault(
response => response.ToString().Contains("primary")).Value.ToString();
}

You really should not handle connecting to the right server yourself. The MongoDB driver handles that for you.
Just specify all of your servers in the connections string and the driver will connect to one of them and get replica set's current state on its own. The driver will then direct write operations to the current primary. Read operations may be directed to the primary, or any other server, depending on the read preference you specify.
You can read about forming replica set connection strings here: https://docs.mongodb.org/v3.0/reference/connection-string/

LinqPad connect two azure databases on same server

Per the FAQ (1), I can add additional databases to my existing connection in a number of ways. I have tried them all, and none work for SQL Azure.
In fact, SQL Azure as a provider, doesn't even include the option to "Include additional databases."
Can someone please tell me a workaround for LinqPad to connect two databases? I am trying to create a migration linqpad script to sync data from one database to another.
http://www.linqpad.net/FAQ.aspx#cross-database

This fails because SQL Azure does not let you create linked servers. See
Can linked server providers be installed on a SQL Azure Database instance?
If you simply want to copy data from one database to another, and the schemas are the same, a workaround is to create a separate connection using the same TypedDataContext class:
void Main()
{
CopyFrom<Customer>("<source connection string>");
}
void CopyFrom<TTable> (string sourceCxString) where TTable : class
{
// Create another typed data context for the source. Note that it must have compatible schema:
using (var sourceContext = new TypedDataContext (sourceCxString) { ObjectTrackingEnabled = false })
{
// Delete the rows currently in our table:
ExecuteCommand ("delete " + Mapping.GetTable (typeof (TTable)).TableName);
// Insert the rows from the source table into the target table and submit changes:
GetTable<TTable>().InsertAllOnSubmit (sourceContext.GetTable<TTable>());
SubmitChanges();
}
}
Simple Select Example:
void Main()
{
SimpleSelect("<your conn string>");
}
void SimpleSelect (string sourceCxString)
{
// Create another typed data context for the source. Note that it must have compatible schema:
using (var sourceContext = new TypedDataContext (sourceCxString) { ObjectTrackingEnabled = false })
{
sourceContext.Assignee.OrderByDescending(a => a.CreateTimeStamp).Take(10).Dump();
Assignee.OrderByDescending(a => a.CreateTimeStamp).Take(10).Dump();
}
}