Develop Reference

NSubsitute - Receive() fails when more than one method of the same service is invoked

I'm switching from Moq to NSubstitute and have a weird problem - I have a simple Interface which I subistitute:
public interface ISampleService
{
string GenerateCode(SampleData data);
BitmapImage GetCode(string code);
}
Above interface is called in this method:
public void GenerateCode()
{
SampleCode = _sampleService.GenerateCode(Label);
Code = _sampleService.GetCode(SampleCode);
}
And try to verify that both method were called but below test fails:
[TestMethod]
public void SomethingChanged_WhenIsChanged_SampleCodeShouldBeGenerated()
{
Sut.GenerateCode();
_barcodeService.Received().GenerateCode(Arg.Any<SampleData>());
_barcodeService.Received().GetCode(Arg.Any<string>());
}
Only last substituted method was verified as invoked _barcodeService.Received().GetCode(Arg.Any<string>()); but _barcodeService.Received().GenerateCode(Arg.Any<SampleData>()); throws ReceivedCallsException
When I change the order of invoking methods in Sut.GenerateCode() method - situation is opposite: _barcodeService.Received().GenerateCode(Arg.Any<SampleData>()); is verified as infoked but _barcodeService.Received().GetCode(Arg.Any<string>()); throws exception.
Interesting thing is that when run this test:
[TestMethod]
public void SomethingChanged_WhenIsChanged_SampleCodeShouldBeGenerated()
{
_sampleService.GenerateCode(Arg.Any<SampleData>()).Returns("111");
Sut.GenerateCode();
_barcodeService.Received().GenerateCode(Arg.Any<SampleData>());
}
It throws ReceivedCallsException but Sut.SampleCode = "111" -> It seems for me as only last call of substituted service is notified as invoked...
I cannot also invoke Received.InOrder method because it throws CallSequenceNotFoundException
Could you please let me now what I did wrong?
Thanks in advance for your help!
PS. When I change testing method like below (double the line: SampleCode = _sampleService.GenerateCode(Label); all tests are green
public void GenerateCode()
{
SampleCode = _sampleService.GenerateCode(Label);
SampleCode = _sampleService.GenerateCode(Label);
Code = _sampleService.GetCode(SampleCode);
}
Did anyone has similar situation?

Selenium - disable test and do not show in test explorer [duplicate]

Is it possible to skip all tests from a specific class like in NUnit
[TestFixture]
[Ignore("Reason")]
public class TestClass {
}

No - there is no such facility at present, and the last time it was requested it was considered too low value to add,
One quick way of achieving the effect in xUnit is to comment out the public - private classes are not reflected over (obviously it won't appear on the skip list that way though).
UPDATE: Another way is to put a TraitAttribute on the class and then (assuming you're using the xunit.console runner) filter it out by running with /-trait traitName. (e.g. you can achieve ExplicitAttribute, some aspects of the BDD frameworky technique of Pending tests and similar semantics that way - of course the big problem is they don't show up in any reports when using any of these filtering techniques)
UPDATE 2: You can do
const string skip = "Class X disabled";
[Fact(Skip=skip)]
void Test() {}
Then you can change to to const string skip = null to undo the skip. The (dis)advantage of this is that the test is still shown as a Skipped test in the test list, generally with a reason included in the test run report (vs making it private which makes it likely to be forgotten)

Here is my hack to avoid error xUnit1000: Test classes must be public (checked on single Fact, I think Theories can be hacked this way, too).
// Uncomment to enable tests
//public class FactSwitch : FactAttribute { } // public! ahh, a bug!
// Uncomment to disable tests
internal class FactSwitch : Attribute { }
public class MyTests
{
[FactSwitch]
public void MyTest1()
{
"it".ShouldBe("it");
}
}
(3 years later)
While searching for the same solution I found there are better ways to do the same.
Let's rewrite the example above in a way Ruben Bartelink suggested (continuation of his idea).
public class MyTests
{
//const string SkipOrNot = null; // Run all tests
const string SkipOrNot = "reason"; // Skip all tests
[Fact(Skip = SkipOrNot)]
public void MyTest1()
{
"it".ShouldBe("it");
}
}
Nathan Cooper suggested a good improvement for my idea:
public class MyTests
{
// Uncomment to disable tests
//private class FactAttribute : Attribute { }
[Fact]
public void MyTest1()
{
"it".ShouldBe("it");
}
}
So I like both ideas from Ruben and Nathan. There is a subtle difference between using Skip="something" (Ruben) and not using Skip at all. Using "Skip" will put all your tests in a "Skipped tests" warning zone, while "FactAttribute : Attribute" will hide them.

I've found yet another way of temporary disabling entire class without compiler warning.
Disabled:
[System.Diagnostics.CodeAnalysis.SuppressMessage("Usage", "xUnit1000:Test classes must be public", Justification = "Disabled")]//*/
/*
public /**/class DatabaseTests
{
}
to enable move the /* one line up (i.e. using alt+up):
/*
[System.Diagnostics.CodeAnalysis.SuppressMessage("Usage", "xUnit1000:Test classes must be public", Justification = "Disabled")]//*/
public /**/class DatabaseTests
{
}
Note that using full namespace path for SupressMessage does not mess up with your usings.

You need to set the your class access level as as internal and surpress message as #Miq did:
[System.Diagnostics.CodeAnalysis.SuppressMessage("Usage", "xUnit1000:Test classes must be public", Justification = "Disabled")]
internal class MyClassThatIsNotATestClass
{ ... }

You can create LocalOnlyFactAttribute
public class LocalOnlyFactAttribute : FactAttribute
{
//uncomment to run on local
//const string skip = null;
//keep this to avoid slow running tests on other env
const string skip = "Disabled slow running tests.";
public override string Skip { get => skip; set => base.Skip = value; }
}

As far as I know, the simplest way to dynamically skip a whole xUnit test class at runtime is to use the TestFrameworkAttribute at the assembly level, to point to a class that implements the ITestFramework interface (or inherits from XunitTestFramework, which is simpler) and which overrides the CreateDiscoverer() method to return another class, that implements the ITestFrameworkDiscoverer interface (or inherits from XunitTestFrameworkDiscoverer, which is simpler), where you can finally override the IsValidTestClass() method, to decide whether a class should be skipped or not.
Here is some sample code:
[assembly: TestFramework("MyNamespace.Xunit.MyTestFramework", "MyAssembly")]
namespace MyNamespace.Xunit
{
public class MyTestFramework : XunitTestFramework
{
public MyTestFramework(IMessageSink messageSink)
: base(messageSink)
{
}
protected override ITestFrameworkDiscoverer CreateDiscoverer(
IAssemblyInfo assemblyInfo)
=> new MyTestFrameworkDiscoverer(
assemblyInfo,
SourceInformationProvider,
DiagnosticMessageSink);
}
public class MyTestFrameworkDiscoverer : XunitTestFrameworkDiscoverer
{
public MyTestFrameworkDiscoverer(
IAssemblyInfo assemblyInfo,
ISourceInformationProvider sourceProvider,
IMessageSink diagnosticMessageSink,
IXunitTestCollectionFactory collectionFactory = null)
: base(
assemblyInfo,
sourceProvider,
diagnosticMessageSink,
collectionFactory)
{
}
protected override bool IsValidTestClass(ITypeInfo type)
=> base.IsValidTestClass(type) &&
FilterType(type);
protected virtual bool FilterType(ITypeInfo type)
{
// Insert your custom filter conditions here.
return true;
}
}
}
Tested with xUnit 2.4.1.
We are using it in Pomelo.EntityFrameworkCore.MySql (see AssemblyInfo.cs and MySqlXunitTestFrameworkDiscoverer.cs) (a bit more complex than the sample code here).

You could achieve this through a custom ITestClassCommand.
See http://mariangemarcano.blogspot.be/2010/12/xunitnet-running-tests-testcategory.html

Here's another hack that requires minimal changes to code
using FactAttribute = System.Runtime.CompilerServices.CompilerGeneratedAttribute;
using TheoryAttribute = System.Runtime.CompilerServices.CompilerGeneratedAttribute;
Any compatible attribute can be used for the replacement.
If you also use the InlineDataAttribute then you'll need to define a replacement as I don't think there's an existing compatible attribute.
using InlineDataAttribute = DummyDataAttribute;
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
internal class DummyDataAttribute : Attribute
{
public DummyDataAttribute(params object[] data)
{
}
}

Adding a reason almost after one year after the initial question. I have a set of tests which are calling real server apis, and I would like to run then on demand. With nUnit, it has Ignore attribute : with that set, test runner will skip those tests, but I can still manually run it.
xUnit has no such feature. The nearest one is setting such a class level attribute, and comment it out when I want to run it.

Consider creating LocalOnlyFactAttribute, which can be reused across multiple test files.
public class LocalOnlyFactAttribute : FactAttribute
{
//uncomment to run on local
//const string skip = null;
//keep this to avoid slow running tests on other env
const string skip = "Disabled slow running tests.";
public override string Skip { get => skip; set => this.Skip = value; }
}

Share variable value between tests in Xunit test [duplicate]

I have written the xUnit test cases in C#. That test class contains so many methods. I need to run the whole test cases in a sequence. How can I set the test case sequence in xUnit?

In xUnit 2.* this can be achieved using the TestCaseOrderer attribute to designate an ordering strategy, which can be used to reference an attribute that is annotated on each test to denote an order.
For example:
Ordering Strategy
[assembly: CollectionBehavior(DisableTestParallelization = true)]
public class PriorityOrderer : ITestCaseOrderer
{
public IEnumerable<TTestCase> OrderTestCases<TTestCase>(IEnumerable<TTestCase> testCases) where TTestCase : ITestCase
{
var sortedMethods = new SortedDictionary<int, List<TTestCase>>();
foreach (TTestCase testCase in testCases)
{
int priority = 0;
foreach (IAttributeInfo attr in testCase.TestMethod.Method.GetCustomAttributes((typeof(TestPriorityAttribute).AssemblyQualifiedName)))
priority = attr.GetNamedArgument<int>("Priority");
GetOrCreate(sortedMethods, priority).Add(testCase);
}
foreach (var list in sortedMethods.Keys.Select(priority => sortedMethods[priority]))
{
list.Sort((x, y) => StringComparer.OrdinalIgnoreCase.Compare(x.TestMethod.Method.Name, y.TestMethod.Method.Name));
foreach (TTestCase testCase in list)
yield return testCase;
}
}
static TValue GetOrCreate<TKey, TValue>(IDictionary<TKey, TValue> dictionary, TKey key) where TValue : new()
{
TValue result;
if (dictionary.TryGetValue(key, out result)) return result;
result = new TValue();
dictionary[key] = result;
return result;
}
}
Attribute
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
public class TestPriorityAttribute : Attribute
{
public TestPriorityAttribute(int priority)
{
Priority = priority;
}
public int Priority { get; private set; }
}
Test Cases
[TestCaseOrderer("FullNameOfOrderStrategyHere", "OrderStrategyAssemblyName")]
public class PriorityOrderExamples
{
[Fact, TestPriority(5)]
public void Test3()
{
// called third
}
[Fact, TestPriority(0)]
public void Test2()
{
// called second
}
[Fact, TestPriority(-5)]
public void Test1()
{
// called first
}
}
xUnit 2.* ordering samples here

Testpriority: at the bottom of this page.
[PrioritizedFixture]
public class MyTests
{
[Fact, TestPriority(1)]
public void FirstTest()
{
// Test code here is always run first
}
[Fact, TestPriority(2)]
public void SeccondTest()
{
// Test code here is run second
}
}
BTW, I have the same problem right now. And yes, it is not the clean art.. but QA wanted a manual test.. so an automated test with a specific order already is a big leap for them.. (cough) and yes, it is not really unit testing..

If you really have the need to prioritize your tests (probably not your unit tests) you can use Xunit.Priority.
I have used it for some integration testing and works really well and simple without the overhead of having to write your prioritization classes, for simple case scenarios

For some reason, XUnit.Priority didn't work for me. In my test cases, it wasn't running the tests in the priority order specified.
So I tried XUnitPriorityOrderer, which is similar to use but was working (To quickly test it, save the following code in a text editor as OrderedXUnitTests.linq, then open it with LinqPad 6 and execute it. Alternatively, you can also copy the TestClass to Visual Studio and add XUnit, XUnit.Runner.VisualStudio and XUnitPriorityOrderer):
<Query Kind="Program">
<NuGetReference>XUnitPriorityOrderer</NuGetReference>
<Namespace>Xunit</Namespace>
<Namespace>XUnitPriorityOrderer</Namespace>
</Query>
#load "xunit"
// using XUnitPriorityOrderer
// see: https://github.com/frederic-prusse/XUnitPriorityOrderer
void Main()
{
RunTests(); // Call RunTests() or press Alt+Shift+T to initiate testing.
}
#region private::Tests
[TestCaseOrderer(CasePriorityOrderer.TypeName, CasePriorityOrderer.AssembyName)]
public class TestClass
{
static List<string> Order { get; set; }
public TestClass()
{
Order = Order ?? new List<string>();
}
[Fact, Order(2)]
void Test_Xunit_AnotherTest()
{
Order.Add("Test_Xunit_AnotherTest");
Assert.True(3 + 1 == 4);
}
[Fact, Order(1)]
void Test_Xunit()
{
Order.Add("Test_XUnit");
Assert.True(1 + 1 == 2);
}
[Fact, Order(99)]
void Print_Order()
{
Order.Add("Print_Order");
var strOrder = string.Join(", ", Order.ToArray());
strOrder.Dump("Execution Order");
Assert.True(true);
}
}
#endregion
This will run the tests in given order (Order(1), Order(2) and then Order(99)) and will dump the executed tests finally (test method Print_Order()).

You can't, by design. It's deliberately random in order to prevent anyone getting one of those either by desire or by accident.
The randomness is only for a given Test class, so you may be able to achieve your goals by wrapping items you want to control the order of inside a nested class - but in that case, you'll still end up with random order whenever you have more than two Test Methods in a class.
If you're trying to manage the building up of fixtures or context, the built-in IUseFixture<T> mechanism may be appropriate. See the xUnit Cheat Sheet for examples.
But you really need to tell us more about what you're trying to do or we'll just have to get speculative.

Roslyn - CodeFixProvider is being fired but DiagnosticAnalyzer not for class destructor

Pseudo-code for my analyser
My test class:
public class TestClass
{
~TestClass()
{
}
}
My diagnostic analyzer class with analyze method:
public class TestClassAnalyzer : DiagnosticAnalyzer
{
public const string AnalyzerId = "TestId";
...
private static void AnalyzeSymbol(SymbolAnalysisContext context)
{
var methodDeclaration = (IMethodSymbol)context.Symbol;
if (methodDeclaration.MethodKind == MethodKind.Destructor)
{
return;
}
context.ReportDiagnostic(Diagnostic.Create(...));
}
}
My code fix provider with fix method:
public class TestClassCodeFixProvider : CodeFixProvider
{
public sealed override ImmutableArray<string> FixableDiagnosticIds =>
ImmutableArray.Create(TestClassAnalyzer.AnalyzerId);
...
private async Task<Solution> PerformFixAsync(Document document, ...)
{
...
return await Renamer.RenameSymbolAsync(...)
}
}
If I put a breakpoint after the line with check for destructor inside my TestClassAnalyzer class, then my code never stops/breaks which makes sense to me because I jump out of the method with return keyword.
Nevertheless, my code fix is being fired (I can put breakpoint inside the PerformFixAnync method and code stops/breaks there) and I can see red squiggly.
Does anybody have any idea why the code fix is being fired although it shouldn't?

I guess your analyzer returns early for descructors as the condition is wrong:
if (methodDeclaration.MethodKind == MethodKind.Destructor)
{
return;
}
context.ReportDiagnostic(Diagnostic.Create(...));
negate the condition or report the condition in the then-block:
if (methodDeclaration.MethodKind != MethodKind.Destructor)
{
return;
}
context.ReportDiagnostic(Diagnostic.Create(...));

It turned out that my solution contained another class buried somewhere between folders with some old code without destructor check condition and this was causing the troubles. This class was not event part of the source code in TFS ...

Strange execution of get accessor in c#?

I set up a simple program just to test how the code inside a get accessor executes (since I had been having some issues in another project), and found something quite strange:
class Program {
static void Main(string[] args) {
var test = new TestClass();
var testBool = test.TestBool;
}
}
public class TestClass {
private bool _testBool = true;
public bool TestBool {
get {
if (_testBool) {
Console.WriteLine("true!");
} else {
Console.WriteLine("false! WTF!");
}
_testBool = false;
return _testBool;
}
}
}
I expected the output to be
true!
But what I got instead was
true!
false! WTF!
Just what is going on here?

If I had to guess, I'd say that the debugger ran it once to show the members of a local variable in the IDE.
If you have side effects in properties (which you shouldn't), don't run it in the IDE :)
Try it at the console; it should behave itself there.

No repro.
And don't write Getters with side effects.