Are failed commands (inserts, updated, deletes etc.) logged anywhere by Mongo DB?
I'm using the C# driver and some commands fail (e.g. inserts) due to duplicate unique key (enforced by an index), so I want to see in retrospect which documents were being inserted.
I would like to see the raw documents that failed, after the driver serialized them.
By the way, as I understand the Mongo oplog only contains successful commands.
Are failed commands (inserts, updated, deletes etc.) logged anywhere by Mongo DB?
I don't think they are, but maybe I haven't tried hard enough to find them yet.
However, you can log them in the application by setting the ClusterConfigurator on the MongoClientSettings like this
//Build the initial settings from the MongoConnectionString
MongoClientSettings settings = MongoClientSettings.FromConnectionString("MongoConnectionString");
//Subscribe on the following events
settings.ClusterConfigurator += cb =>
{
cb.Subscribe(delegate (CommandStartedEvent startedEvent) { Console.WriteLine($"Started: {startedEvent.Command} with OpId: {startedEvent.OperationId}"); });
cb.Subscribe(delegate (CommandSucceededEvent succeededEvent) { Console.WriteLine($"Succeeded OpId: {succeededEvent.OperationId}"); });
cb.Subscribe(delegate (CommandFailedEvent failedEvent) { Console.WriteLine($"Failed OpId: {failedEvent.OperationId}"); });
};
//Builld a MongoClient with the new Settings
var client = new MongoClient(settings);
This example will only write the commands that are being exected and if which OperationId failed or succeeded.
But from here on you can extend it by keeping track of which command got started and what OperationId it runs on.
For a completed example you can see this Gist (as it seems like too much code to post here).
Which can be called as this:
var settings = MongoClientSettings.FromConnectionString("MongoConnectionString");
new MongoTracking().ConfigureTracking(settings);
var client = new MongoClient(settings);
For the record, this does the logging in the application and not the database.
Related
So I have a few different parse servers setup.
One server is just to capture error logs from various applications (I have a LOT out there) in nice uniformed database.
So I might have a specific standalone data migration tool that if it encounters an error, will write out the exception into this Error_log parse table/class. No problem there.
But, if I have an app that uses a Parse Database for itself, I have not been able to figure out how to let it work on it's own parse server configuration for it's own stuff, but write out error logs to this other Parse server instance.
Yes... I could go through the trouble of writing out something via the REST api just for writing out logs,but I am I trying to avoid that and stick with native parse APIs for the particular platform I am on because of the benefits that the APIs give over REST (like save eventually for the none .NET stuff).
EDIT
Some clarification was requested so here I go...
On the app side of things (c# for this example but the same holds true for iOS etc)… I do the usual initialization of the Parse client as such …
ParseClient.Initialize(new ParseClient.Configuration
{
ApplicationId = "MyAppID",
WindowsKey = "MyDotNetKey",
Server = "www.myparseserver.com/app1"
});
So for all calls to save a parse object go through that parse client connection
But what I need to do would be something like this ….
//Main App cloud database
ParseClient1.Initialize(new ParseClient.Configuration
{
ApplicationId = "MyAppID",
WindowsKey = "MyDotNetKey",
Server = "www.myparseserver.com/app1"
});
ParseClient2.Initialize(new ParseClient.Configuration
{
ApplicationId = "MyAppID",
WindowsKey = "MyDotNetKey",
Server = "www.myparseserver.com/errorcollection"
});
try{
ParseConfig config = null;
config = await ParseConfig.GetAsync().ParseClient1;
} catch (Exception ex){
ParseObject MyError = new ParseObject("Error_Log");
MyError["Application"] = "My First App-App2";
MyError["Error"] = ex.message;
await MyError.Save().ParseClient2;
}
Yes - this is all fake code... my point is I want to be able to have multiple ParseClient instances in one app.
Now... I can simply write a routine that writes out errors that resets the ParseClient.Initialization to the error parse server instance and then redo it back to the original (primary app data) instance when it's done... but that is just asking for trouble in a multi threaded environment and will cause conflicts if some other thread in the app goes to write out parse data right at the moment the error method resets the init.
If ParseClient were IDisposable I could probably do that using :
ParseClient ParseErrorServer = new ParseClient();
ParseErrorServer.ApplicationId = "hmmm";
ParseErrorServer.WindwosKey= "hmmm";
ParseErrorServer.Server= "www.hmmm.com/errorcollection";
using ParseErrorServer {
//Do The Work
}
Is that clear as mud yet? ;P
Without alteration I believe none of the Parse SDKs have the ability to initialise multiple instances.
In the iOS SDK for example, is possible to make a new instance (say with a different server url) upon restarting the app but you cannot have multiple. There has also been discussion on the iOS SDK about being able to change the configuration without restart but no one has implemented this yet.
We would happily review a PR for this, however it would require a major and complex overhaul as you would have to manage cache, users etc across multiple instances.
Not sure what I'm doing wrong here, but I'm trying to insert a document into my-documents in the storage database my-db but this isn't happening.
I create a document client like this-
_endpointUri = new Uri(Properties.Settings.Default.DocumentDBEndpoint);
_privateKey = Properties.Settings.Default.DocumentDBKey;
_databaseName = Properties.Settings.Default.DocumentDBDatabase;
_collectionName = collectionName;
_collectionUri = UriFactory.CreateDocumentCollectionUri(_databaseName, _collectionName);
_documentClient = new DocumentClient(_endpointUri, _privateKey);
I then try to insert a document into the collection.
public async Task Set(T document)
{
await _documentClient.CreateDocumentAsync(_collectionUri.ToString(), document);
}
I tried passing the _collectionUri as well as the toString and they both do the same thing. If I step through the debugger the toString of that collection uri looks like-
dbs/my-db/colls/my-documents
I've tried set .ConfigureAwait(false) on the call, as well as not awaiting it at all. Of course when it's not awaited it steps over the line fine, but no document ever gets added to the my-documents collection.
I'm not sure if i'm using CreateDocumentAsync (also tried UpsertDocumentAsync) because not errors are thrown.
Can anyone see from the information given what mistake I'm making?
Edit In my testing with Azure I did opt to use DocumentDB with MongoDB Api. That may be related. Since I can easily blow it all away I will recreate it using the DocumentDB Api and see if that fixes the problem.
This turned out to be two different problems I assumed were the same.
Document Not Creating-
So I set this up originally to use the MongoDB driver which dumped my databases/collections in Azure Storage. The issue of the documents never getting created was because DocumentDB doesn't store itself in the Azure Storage resource. When you create a new DocumentDB look under Settings > Collections > Browse to see that no databases exist. Unfortunately it doesn't appear DocumentDB has a desktop db management application and this needs to be done on the azure portal.
So the issue was, plainly, that the database/collection I was pointing to didn't exist for the above reason.
Await Never Returning-
Apparently if you're going to await something you need to await all the way through the call stack.
(async) POST:Api (awaits) -> (async) Service:CreateDocument -> (async) Repository:Set (awaits) -> (async) DocumentClient:CreateDocumentAsync
The problem was that even though my service was async, it was not awaiting the call to Repository:Set. Awaiting everything up the chain allowed me to step-through and receive a response from the server.
I have a Console Application project written in C# which I've added Application Insights to with the following NuGet packages.
Microsoft.ApplicationInsights
Microsoft.ApplicationInsights.Agent.Intercept
Microsoft.ApplicationInsights.DependencyCollector
Microsoft.ApplicationInsights.NLogTarget
Microsoft.ApplicationInsights.PerfCounterCollector
Microsoft.ApplicationInsights.Web
Microsoft.ApplicationInsights.WindowsServer
Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel
I've configured my InstrumentationKey in the config file and I'm firing up a TelemetryClient on startup using the with the following code:
var telemetryClient = new TelemetryClient();
telemetryClient.Context.User.Id = Environment.UserName;
telemetryClient.Context.Session.Id = Guid.NewGuid().ToString();
telemetryClient.Context.Device.OperatingSystem = Environment.OSVersion.ToString();
Everything is working well except AI is not capturing any requests that get sent to Mongo, I can see requests going off to SQL server in the 'Application map' but no sign of any other external requests. Is there any way that I can see telemetry of requests made to Mongo?
EDIT - Thanks to Peter Bons I ended up with pretty much the following which works like a charm and allows me to distinguish between success and failure:
var telemetryClient = new TelemetryClient();
var connectionString = connectionStringSettings.ConnectionString;
var mongoUrl = new MongoUrl(connectionString);
var mongoClientSettings = MongoClientSettings.FromUrl(mongoUrl);
mongoClientSettings.ClusterConfigurator = clusterConfigurator =>
{
clusterConfigurator.Subscribe<CommandSucceededEvent>(e =>
{
telemetryClient.TrackDependency("MongoDB", e.CommandName, DateTime.Now.Subtract(e.Duration), e.Duration, true);
});
clusterConfigurator.Subscribe<CommandFailedEvent>(e =>
{
telemetryClient.TrackDependency("MongoDB", $"{e.CommandName} - {e.ToString()}", DateTime.Now.Subtract(e.Duration), e.Duration, false);
});
};
var mongoClient = new MongoClient(mongoClientSettings);
I am not familiar with MongoDB but as far as I can tell there is no default support for it when it comes to Application Insights. But that does not mean you cannot do this, it will just involve some more code.
Again, I am not familiar with MongoDB but according to http://www.mattburkedev.com/logging-queries-from-mongodb-c-number-driver/ there is built-in support for logging the generated queries. Now, we only need to hook this up to Application Insights.
Since you already know how to use the TelemetryClient we can use the custom tracking methods provided by that class. See https://learn.microsoft.com/nl-nl/azure/application-insights/app-insights-api-custom-events-metrics for the available custom tracking methods.
All you need to do is to insert some code like this:
telemetryClient.TrackDependency(
"MongoDB", // The name of the dependency
query, // Text of the query
DateTime.Now, // Time that query is executed
TimeSpan.FromSeconds(0), // Time taken to execute query
true); // Indicates success
The class telemetryClient is thread-safe so you can reuse it.
Now, according to the referenced blogpost you should be able to do something like this:
var client = new MongoClient(new MongoClientSettings()
{
Server = new MongoServerAddress("localhost"),
ClusterConfigurator = cb =>
{
cb.Subscribe<CommandStartedEvent>(e =>
{
telemetryClient.TrackDependency(
"MongoDB", // The name of the dependency
e.Command.ToJson() // Text of the query
DateTime.Now, // Time that query is executed
TimeSpan.FromSeconds(0), // Time taken to execute query
true); // Indicates success
});
}
});
Again, I am not familiar with MongoDB but I hope this is a starting point for your imagination on how to adapt it to your needs using your knowledge of MongoDB.
EDIT:
If there is also a CommandCompletedEvent or similar event as opposed to the CommandStartedEvent event you should probably track the dependency there because you should then be able to calculate (or simpel read) the time spent and maybe get the actual value for the success indicator.
GetServer is gone for good. How do i check if the server is connected or even exists?
Example code:
// This server exists
var exists = new MongoClient("mongodb://192.168.2.109:27017");
// This server does not exist
var doesNotExist = new MongoClient("mongodb://194.168.200.129:27017");
// Both states return "Discennected"
var connStateExisting = exists.Cluster.Description.State;
var connStateNotExisting = doesNotExist.Cluster.Description.State;
// GetDatabase("name") works for both without errors.
How can i check if a server can be connected?
The Cluster.Description.State does not update immediately. When i checked, it was updated after roughly 100+ milliseconds. The driver contains a connection pool and it seems to do quite a lot asynchronous.
However, the Cluster-property has a "DescriptionChanged"-event that is fired once the connection is done.
If someone else has any knowledge about connections and timeouts, please share it.
I connect with the following code to a mongo database.
Then i iterated through the server descriptions of the cluster but the state is always "disconnected" but when i look with tools like monogchef i can see that all server are connected and there is one primary and all others are secondaries
var client = new MongoClient(conString);
var db = client.GetDatabase("admin");
foreach (var server in client.Cluster.Description.Servers)
{
Console.WriteLine(server.State); // Always returns disconnected.
}
How can I read who is the primary and when has been the last election?
Just found it.
Have to make a dummy request then i find the data under
server.ReplicaSetConfig.Primary