Develop Reference

RavenDB and XUnit: Multiple Tests per TestClass

I currently try to setup some integration testing in order to verify that my RavenDB queries return what i expect.
I followed the documentation and have a single test up and running.
The problem starts when i add an additional test method to the same class. I get the following exception:
System.InvalidOperationException : Cannot configure server after it was started. Please call 'ConfigureServer' method before any 'GetDocumentStore' is called.
So basically it tells me, the server is already running. But this confuses me. Since XUnit creates an new instance of the test class for every discovered test method in that class. Additionally it calls Dispose() on any instance, that implements IDisposable. Which is indirectly implemented by the base class RavenTestDriver.
So what i think is happening:
XUnit creates a new instance of my test class
XUnit calls my test method
My test method calls ConfigureServer, the RavenDB Embedded Server is started
My test method finishes
XUnit calls Dispose on my instance of the test class, the RavenDB Embedded Server is stopped
Rince and repeat for the next test method
But i looks like, my assumption on #5 is wrong. The RavenDB Embedded Server seems to be never stopped. Additionally i cannot find a way to manually stop it. I was trying to manually dispoose it with EmbeddedServer.Instance.Dispose(). But that doesn't change anything. (The .Instance gives a clue, that the EmbeddedServer is probably a singleton, which may be part of the problem here).
I also tried to move the ConfigureServer call into the constructor of the test class. Since XUnit class the constructore for every test method (just like the setup-method from JUnit). But then i get the same result.
But the interesting part is: Calling ConfigureServer in two different classes just works fine.
I have created a small reproducer repository.
So has anybody an idea on how to setup RavenDB in a Unit/Integration-Test environment where you want to run multiple tests against it?

Remove the ConfigureServer method from all tests & constructor. Calling GetDocumentStore() will create the embedded server.
https://github.com/ravendb/ravendb/blob/e8f08f191e1b085421ca5b4db191b199e7a8fc69/src/Raven.TestDriver/RavenTestDriver.cs#L272
If you want to configure the server then you should set it in a static constructor:
static MovieTests_ConfigureInConstructor()
{
ConfigureServer(new TestServerOptions() {
CommandLineArgs = new System.Collections.Generic.List<string> { "--RunInMemory=true", },
FrameworkVersion = null,
});
}

NSubstitue Received() does not work when I run multible test in row, but if I run the test individually they work

I have multiple unit tests, where I do check if a method was called.
I use the NSubstitute mocking libraries to check rather a method was called with the help of the "Received()" method, just like that:
MessageHandling.Received().Submit(Messages.DATA_EXPORT_SUCCESS);
The tests are working fine when I run them individually, but when run them all, some of them fail for no apparent reason. When I debug the code, I see that the method which should get called was called, but the Received() method from NSubstitute says there was no call at all.
I do also call the ClearReceivedCalls() in my TearDown methode
MessageHandling.ClearReceivedCalls();
But this does not seem to help.
Is there something else I should take care of, when using the Received() method?
My test functions are a bit more complicated then just the to check for a call, but that is the only reason why my tests fail.

I assume MessageHandling is initialized as a single instance property, that is used in every test? Try to make your test class stateless, by initializing a new mocked instance in every test.

Why is [TestClass]'s constructor called multiple times for each [TestMethod]?

In case of you have multiple test method on a test class. Class's constructor is going to run multiple times. How we can explain this overload?

From my understanding of MSTest, the test class gets instantiated for each [TestMethod]. I'm guessing you are attempting to run configuration code before any of the tests are ran. If that's the case I'd recommend you:
A) update your question to explain what it is exactly you would like to accomplish
B) make use of the [ClassInitialize] attribute to mark a method to be ran once and only once before any of the class's tests are ran
What Does ClassInitialize Do
ClassInitialize is one of the many attributes available when using MSTest to write unit tests in C#. The more common ones include TestClass, TestMethod, and TestInitialize. This one indicates that the method should be ran once before running any of the methods marked with TestMethod. There's another attribute that goes hand-in-hand with it called ClassCleanup which gets ran after all of the test methods get ran.
You can read more details about these and more attributes at learn.microsoft.com

Inheriting test methods across projects

I'm trying to have a test class for the VS test framework, that is derived from another test class and inherits all its test methods (so that each appears twice). This was suggested in a comment on another ticket.
When this is done within the same project, it works fine. When done across projects, everything compiles fine but the new tests don't show up.
In this minimal solution, the problem is demonstrated. Two projects, in each we derive from the same base class in the first project. But the new (duplicate) tests only show up for the same-project child class.
I suspect something is optimized away in someway, but don't know what or how top debug this. I tried adding a dummy TestMethod to the second-project test class, just so that something must happen with the class - then only this test method shows, without the derived methods. Suggestions would be appreciated.

Moq - Tests run ok when executed independently, but fail when executed in a row

We are using Moq to perform some unit tests, and there's some strange behaviour, perhaps it's some configuration issue that i'm missing.
basically, i have 2 tests (which call a windows workflow, which calls custom activities that call a method using Invoke. i don't know if this helps but i want to give as much info as i can). The tests run OK when executed alone, but if I execute them in a same run, the first one passes and the second fails (doesn't matter if i change the order of them, the 2nd one always fails)
The mock is recreated every time, loaded using Unity. ex :
MockProcessConfigurator = MockFactory.Create<IProcessConfigurator>();
MockProcessConfigurator.Setup(x => x.MyMethod(It.IsAny<Order>()));
[...]
InversionOfControl.Instance.Register<IProcessConfigurator>(MockProcessConfigurator .Object)
The invoked call (WF custom activity) is
var invoker = new WorkflowInvoker(new MyWorkflow());
invoker.Invoke(inputParameter);
The call (Invoked call) is
MyModuleService.ProcessConfigurator.MyMethod(inputOrder);
when debugging, i see that the ProcessConfigurator is always mocked.
The call that fails in the test is something as simple as this :
MockEnvironment.MockProcessConfigurator.Verify(x => x.MyMethod(It.IsAny<Order>()), Times.Exactly(1));
When debugging, the method is actually called everytime, so i suspect that there's something worng with the mock instance. I'm a bit lost here, because things seem to be implemented correctly, but for some reason when they're run one after the other, there's a problem

This type of error commonly occurs when the two tests share something.
For example, you set up your mock with an expectation that a method will be called 1 time in your test setup, and then two tests each call that method 1 time - your expectation will fail because it has now been called 2 times.
This suggests that you should be moving the set up of expectations into each test.

A generic troubleshooting for this type of problem is to try to isolate the dependency between the two test.
Move any setup-code to inside the tests.
Move any tear-down code to inside the tests.
Move any field initializers to inside the tests. (Those are only run once per fixture!)
This should make both your test pass when run together. When you got the green-lights, you can start moving out duplicated stuff again to initializers/setup one piece at the time, running the tests after each change you make.
You should be able to learn what is causing this coupling between the tests. Good luck!

Thought I'd add an additional situation and solution I just came across:
If you are running tests from two separate projects at the same time and each project is using a different version of Moq, this same problem can happen.
For me, I had TestProjectA using Moq 4.2.14 and TestProjectB using Moq 4.2.15 on accident (courtesy of Nuget). When running a test from A and a test from B simultaneously, the first test succeeded and the second silently failed.
Adjusting both projects to use the same version solved the issue.

To expand on the answer Sohnee gave, I would check my setup/teardown methods to make sure you're tidying everything up properly. I've had similar issues when running tests in bulk and not having tidied up my mock objects.

I don't know if this is relevant, but MockFactory.Create seems odd. I normally create mocks as follows:
var mockProcessConfigurator = new Mock<IProcessConfigurator>();
When using a MockFactory (which I never have needed), you would normally create an instance of it. From the moq QuickStart:
var factory = new MockFactory(MockBehavior.Strict) { DefaultValue = DefaultValue.Mock };
Calling a static method on MockFactory seems to defeat the purpose. If you have a nonstandard naming convention where MockFactory is actually a variable of type MockFactory, that's probably not your issue (but will be a constant source of confusion). If MockFactory is a property of your test class, insure that it is recreated on SetUp.
If possible I would eliminate the factory, as it is a potential source of shared state.
EDIT: As an aside, WorkflowInvoker.Invoke takes an Activity as a parameter. Rather than creating an entire workflow to test a custom activity, you can just pass an instance of the custom activity. If that's what you want to test, it keeps your unit test more focused.