Thanks in advance!
We have some Automation tests using the selenium web driver which are great and provide a really good regression pack.
The problem is now we have feature toggles in our code. So I need to say ignore these tests unless that feature toggle is turned On/ Off. I can't find anything really searching Google.
Ideally I don't want a 'if' statement at the top of the Feature tests but it looks like it's going to be the main way. My initial thoughts where to create a custom attribute
public class IsFeatureFlagTurnedOn : Attribute
{
public IsFeatureFlagTurnedOn(string featureToggleName)
{
FeatureToggleName = featureToggleName;
}
public string FeatureToggleName {get;}
}
public class MyTests
{
[TestMethod]
[IsFeatureFlagTurnedOn("MyFeature1")]
public void ItShould()
{
// only run if MyFeature1 is turned on
}
}
I some how need to hook into the MSTest pipeline and say if this attribute is present and the logic for MyFeature1 is turned off then don't run this test - Looked at dynamically adding the [Ignore] but with no luck.
This is running through VSTS and I could use [TestCategories] but I'd have to keep updating the pipeline to which feature is turned on/off which I don't want to do.
Any help or suggestions would be great!
MSTest v2 now has a lot of extensibility points, and you can achieve this by extending the TestMethodAttribute. First we add two attribute arguments, a string for a property name and a Type that has the property. Then we override the Execute method and invoke the property via reflection. If the result is true, we'll execute the test as normal, otherwise we return an 'inconclusive` test result.
public class TestMethodWithConditionAttribute : TestMethodAttribute
{
public Type ConditionParentType { get; set; }
public string ConditionPropertyName { get; set; }
public TestMethodWithConditionAttribute(string conditionPropertyName, Type conditionParentType)
{
ConditionPropertyName = conditionPropertyName;
ConditionParentType = conditionParentType;
}
public override TestResult[] Execute(ITestMethod testMethod)
{
if (ConditionParentType.GetProperty(ConditionPropertyName, BindingFlags.Static | BindingFlags.Public)?.GetValue(null) is bool condiiton && condiiton)
{
return base.Execute(testMethod);
}
else
{
return new TestResult[] { new TestResult { Outcome = UnitTestOutcome.Inconclusive } };
}
}
}
Now we can use our new attribute like this:
[TestClass]
public class MyTests
{
[TestMethodWithCondition(nameof(Configuration.IsMyFeature1Enabled), typeof(Configuration))]
public void MyTest()
{
//...
}
}
public static class Configuration
{
public static bool IsMyFeature1Enabled => false;
}
The above is a very generic solution. You could also customize it a little more to your particular use case to perhaps avoid quite so much verbosity in the attribute declaration:
public class TestMethodForConfigAttribute : TestMethodAttribute
{
public string Name { get; set; }
public TestMethodForConfigAttribute(string name)
{
Name = name;
}
public override TestResult[] Execute(ITestMethod testMethod)
{
if (IsConfigEnabled(Name))
{
return base.Execute(testMethod);
}
else
{
return new TestResult[] { new TestResult { Outcome = UnitTestOutcome.Inconclusive } };
}
}
public static bool IsConfigEnabled(string name)
{
//...
return false;
}
}
And use it like:
[TestClass]
public class MyTests
{
[TestMethodForConfig("MyFeature1")]
public void MyTest()
{
//...
}
}
Based on my reading of this, you may need to use Assert.Inconclusive
Related
I'm trying to customize some of nUnits behaviour, however I'm constantly hitting a brick wall
because of nUnits heavy use of code reflection. Test methods (and also setup methods etc) are passed all the way down, deep into the framework, and are converted into a delegate at the latest step possible.
The classes I'm interested in are called TestCommands and only there the framework becomes functional.
For reference here is a snippet I found in nUnits source of the TestMethodCommand class which propably is the bread and butter test execution delegate.
public class TestMethodCommand : TestCommand
{
private readonly TestMethod testMethod;
private readonly object[] arguments;
public TestMethodCommand(TestMethod testMethod) : base(testMethod)
{
this.testMethod = testMethod;
this.arguments = testMethod.Arguments;
}
public override TestResult Execute(TestExecutionContext context)
{
object result = InvokeTestMethod(context); // missing a branch deciding about sync vs. async
// missing some code that checks object against "expected result"
return context.CurrentResult;
}
private object InvokeTestMethod(TestExecutionContext context)
{
return testMethod.Method.Invoke(context.TestObject, arguments);
}
}
I'm puzzled why nUnit couldn't wrap the test method into an Func<object> way way sooner and just pass the context along. As it stands for now if I don't have a MethodInfo nUnit can't run it.
In case you wonder, here is an example of a thing I want to do but I ran into the same problem in other instances as well.
[Scenario(When: "Device Registration reads out PCB Type",
Then: "Device Type might change")]
public void Identifier_Changes_Are_Recognized()
{
var changedType = reference.ChangeType(DeviceType.Terminal);
var changedID = reference.ChangeID(123456);
Assert.Multiple(() =>
{
AssertIsSameDevice(reference, changedType);
AssertIsDifferentDevice(reference, changedID);
});
}
This scenario attribute is supposed to print a small description like so.
public void RunBeforeTest()
{
var text = new MultiLineText
("Scenario:",
"\tGiven:\t" + When,
"\tThen:\t" + Then,
"-------------\n"
);
Console.WriteLine(text);
}
I reaaallly want to tell nUnit "Look, here is an action, please run it" but for the time beeing this seems very hard to achieve. Did anyone else here run in these kinds of problems?
Are there possibly ways to achieve what I'm trying to do? Maybe create my own TestCommand, but as I mentioned, these objects only get created very deep into the framework.
OP here (Writing from my home account)
I looked more into this and actually found a working solution:
public class ArbitraryCodeExecutionWrapper : DelegatingTestCommand
{
public ArbitraryCodeExecutionWrapper(TestCommand innerCommand) : base(innerCommand)
{
}
public Action<TestExecutionContext> BeforeTest { get; init; } = _ => { };
public Action<Test, TestResult> AfterTest { get; init; } = (_,_) => { };
public override TestResult Execute(TestExecutionContext context)
{
BeforeTest(context);
var result = innerCommand.Execute(context);
AfterTest(context.CurrentTest, result);
return result;
}
}
public class NUnitTestCommandWrapperAttribute : Attribute, IWrapTestMethod
{
protected virtual void BeforeTest(TestExecutionContext context)
{
}
protected virtual void AfterTest(Test test, TestResult result)
{
}
public TestCommand Wrap(TestCommand command)
=> new ArbitraryCodeExecutionWrapper(command)
{
BeforeTest = BeforeTest,
AfterTest = AfterTest
};
}
public class ScenarioAttribute : NUnitTestCommandWrapperAttribute
{
public string When { get; init; } = "";
public string Then { get; init; } = "";
protected override void BeforeTest(TestExecutionContext context)
{
var text = new MultiLineText
("Scenario:",
"\tGiven:\t" + When,
"\tThen:\t" + Then
);
Console.WriteLine(text);
}
protected override void AfterTest(Test test, TestResult result)
{
Console.WriteLine("After Test");
}
}
[TestFixture]
public class TestCodeExecution
{
[Test]
[Scenario(When = "nUnit Comes here",
Then = "Print Hello World")]
public void Try_Out_Code_Execution()
{
Console.WriteLine("Hello World");
}
}
public class MultiLineText
{
private List<string> items = new();
public static implicit operator string(MultiLineText text) => text.ToString();
public MultiLineText(params string[] lines)
{
items = lines.ToList();
}
public override string ToString() => string.Join("\n", items);
}
I am trying to create a custom attribute in console application but it is not working. My custom attribute never gets called. I found a good example here Custom Attribute not being hit
but not happy with its implementation.
I am wondering how data annotations works in MVC. we don't have to call it separately.
Is MVC calling those data annotations attribute behind the scene?
I wish to create custom attribute that I can use it on any class property same like data annotations attribute. But calling it separately like in above link is not what i am looking.
Here is what I have tried:
using System;
namespace AttributePractice
{
[AttributeUsage(AttributeTargets.Property)]
public class CustomMessageAttribute : Attribute
{
public static readonly CustomMessageAttribute Default = new CustomMessageAttribute();
protected string Message { get; set; }
public CustomMessageAttribute() : this(string.Empty)
{
Console.WriteLine("Default message is empty");
}
public CustomMessageAttribute(string message)
{
Message = message;
}
public string MyMessage =>
Message;
public override bool Equals(object obj)
{
if (obj == this)
return true;
if (obj is CustomMessageAttribute customMessageAttribute)
return customMessageAttribute.Message == MyMessage;
return false;
}
public override int GetHashCode()
{
return MyMessage.GetHashCode();
}
public override bool IsDefaultAttribute()
{
return Equals(Default);
}
}
public class Person
{
//This never works
// I am looking to use this attribute anywhere without calling it
// separately , same like data annotations
[CustomMessage("Hello world")]
public string Name { get; set; }
public int Age { get; set; }
public void DisplayPerson()
{
Console.WriteLine(Name);
Console.WriteLine(Age);
}
}
internal static class Program
{
private static void Main(string[] args)
{
var personObj = new Person
{
Name = "Tom",
Age = 28
};
personObj.DisplayPerson();
}
}
}
Can anybody tell me how to make my custom attribute works like data annotation way?
yes, if you need 10 custom attributes, you should create 10 separate.
Is there in C# some kind of equivalent of ExpectedSystemExit in Java? I have an exit in my code and would really like to be able to test it. The only thing I found in C# is a not really nice workaround.
Example Code
public void CheckRights()
{
if(!service.UserHasRights())
{
Environment.Exit(1);
}
}
Test Code
[TestMethod]
public void TestCheckRightsWithoutRights()
{
MyService service = ...
service.UserHasRights().Returns(false);
???
}
I am using the VS framework for testing (+ NSubstitute for mocking) but it is not a problem to switch to nunit or whatever for this test.
You should use dependency injection to supply to the class being tested an interface that provides an environmental exit.
For example:
public interface IEnvironment
{
void Exit(int code);
}
Let's also assume that you have an interface for calling UserHasRights():
public interface IRightsService
{
bool UserHasRights();
}
Now suppose your class to be tested looks like this:
public sealed class RightsChecker
{
readonly IRightsService service;
readonly IEnvironment environment;
public RightsChecker(IRightsService service, IEnvironment environment)
{
this.service = service;
this.environment = environment;
}
public void CheckRights()
{
if (!service.UserHasRights())
{
environment.Exit(1);
}
}
}
Now you can use a mocking framework to check that IEnvironment .Exit() is called under the right conditions. For example, using Moq it might look a bit like this:
[TestMethod]
public static void CheckRights_exits_program_when_user_has_no_rights()
{
var rightsService = new Mock<IRightsService>();
rightsService.Setup(foo => foo.UserHasRights()).Returns(false);
var enviromnent = new Mock<IEnvironment>();
var rightsChecker = new RightsChecker(rightsService.Object, enviromnent.Object);
rightsChecker.CheckRights();
enviromnent.Verify(foo => foo.Exit(1));
}
Ambient contexts and cross-cutting concerns
A method such as Environment.Exit() could be considered to be a cross-cutting concern, and you might well want to avoid passing around an interface for it because you can end up with an explosion of additional constructor parameters. (Note: The canonical example of a cross cutting concern is DateTime.Now.)
To address this issue, you can introduce an "Ambient context" - a pattern which allows you to use a static method while still retaining the ability to unit test calls to it. Of course, such things should be used sparingly and only for true cross-cutting concerns.
For example, you could introduce an ambient context for Environment like so:
public abstract class EnvironmentControl
{
public static EnvironmentControl Current
{
get
{
return _current;
}
set
{
if (value == null)
throw new ArgumentNullException(nameof(value));
_current = value;
}
}
public abstract void Exit(int value);
public static void ResetToDefault()
{
_current = DefaultEnvironmentControl.Instance;
}
static EnvironmentControl _current = DefaultEnvironmentControl.Instance;
}
public class DefaultEnvironmentControl : EnvironmentControl
{
public override void Exit(int value)
{
Environment.Exit(value);
}
public static DefaultEnvironmentControl Instance => _instance.Value;
static readonly Lazy<DefaultEnvironmentControl> _instance = new Lazy<DefaultEnvironmentControl>(() => new DefaultEnvironmentControl());
}
Normal code just calls EnvironmentControl.Current.Exit(). With this change, the IEnvironment parameter disappears from the RightsChecker class:
public sealed class RightsChecker
{
readonly IRightsService service;
public RightsChecker(IRightsService service)
{
this.service = service;
}
public void CheckRights()
{
if (!service.UserHasRights())
{
EnvironmentControl.Current.Exit(1);
}
}
}
But we still retain the ability to unit-test that it has been called:
public static void CheckRights_exits_program_when_user_has_no_rights()
{
var rightsService = new Mock<IRightsService>();
rightsService.Setup(foo => foo.UserHasRights()).Returns(false);
var enviromnent = new Mock<EnvironmentControl>();
EnvironmentControl.Current = enviromnent.Object;
try
{
var rightsChecker = new RightsChecker(rightsService.Object);
rightsChecker.CheckRights();
enviromnent.Verify(foo => foo.Exit(1));
}
finally
{
EnvironmentControl.ResetToDefault();
}
}
For more information about ambient contexts, see here.
I ended up creating a new method which I can then mock in my tests.
Code
public void CheckRights()
{
if(!service.UserHasRights())
{
Environment.Exit(1);
}
}
internal virtual void Exit()
{
Environment.Exit(1);
}
Unit test
[TestMethod]
public void TestCheckRightsWithoutRights()
{
MyService service = ...
service.When(svc => svc.Exit()).DoNotCallBase();
...
service.CheckRights();
service.Received(1).Exit();
}
If your goal is to avoid extra classes/interfaces just to support tests, how do you feel about Environment.Exit action via Property Injection?
class RightsChecker
{
public Action AccessDeniedAction { get; set; }
public RightsChecker(...)
{
...
AccessDeniedAction = () => Environment.Exit();
}
}
[Test]
public TestCheckRightsWithoutRights()
{
...
bool wasAccessDeniedActionExecuted = false;
rightsChecker.AccessDeniedAction = () => { wasAccessDeniedActionExecuted = true; }
...
Assert.That(wasAccessDeniedActionExecuted , Is.True);
}
i have a simple Setup method in my tests thats creates two instances of an object (make by and Id and a Description properties) and i have done it with autofixture:
MyObject o1 = fixture.Create<MyObject>();
MyObject o2 = fixture.Create<MyObject>();
next i try to save objects to db but i get error of duplicate key, i debug the setup and see that o1 and o2 have the same Id
According to Wiki, number should be generate progressivly:
Autogenerated Number
int autoGeneratedNumber = fixture.Create<int>();
Sample Result
int: 1, followed by 2, then by 3, etc.
but seem id does not work in this way with object property so now i use this simple workaround:
MyObject o1= fixture.Build<MyObject>().With(x => x.Id, 1).Create();
MyObject o2= fixture.Build<MyObject>().With(x => x.Id, 2).Create();
but don't like it very much
is here a way to use ISpecimenBuilder for setting up Autofixture to make it generate progressive id?
Some more info:
this is my base test class:
public class BaseDBTest
{
public BaseDBTest()
{ }
public Ploeh.AutoFixture.Fixture fixture { get { return new Fixture(); } }
}
and test setup:
[TestFixture]
public class MyObjectTests : BaseDBTest
{
MyObject o1;
MyObject o2;
[TestFixtureSetUp]
public void CreaDati()
{
o1= fixture.Create<MyObject >();
o2= fixture.Create<MyObject >();
}
}
strange things is:
if I debug of the specific test objects are created with different id and random, but if I debug of all the tests of my project (with visual studio 2013 using Nunit runner) id's are created equal
EDIT2
MyObject definition, quite complex, sorry:
public class MyObject: LookUpObject<MyObject, int>
{
}
public abstract class LookUpObject<TObject, TKeyType> : EquatableObject<TObject>, IKeyedEntity<TKeyType>
where TObject : class
where TKeyType : struct
{
private TKeyType id;
private string description;
private bool isValid;
public virtual TKeyType Id
{
get { return id; }
set { id = value; }
}
public virtual string Description
{
get { return description; }
set { description= value; }
}
public virtual bool IsValid
{
get { return isValid; }
set { isValid= value; }
}
protected LookUpObject()
{
}
}
EDIT 3
image of the strange things make with Nunit (I was afraid it might depend on Visual Studio),
single test run link
project test run link
This is because in your base class property to get the Fixture, you are returning a new Fixture object every time. The auto generation of Ids can only be guaranteed per Fixture instance.
Change this:
public class BaseDBTest
{
public BaseDBTest()
{ }
public Ploeh.AutoFixture.Fixture fixture { get { return new Fixture(); } }
}
to this:
public class BaseDBTest
{
private Fixture _fixture = new Fixture();
public BaseDBTest()
{ }
public Ploeh.AutoFixture.Fixture fixture { get { return _fixture; } }
}
I'm using the System.Composition namespace from the MEF for web and Windows Store apps NuGet package in a new ASP.NET MVC4 project.
I've read that in MEF2 you no longer use Lazy<IExtension, IExtensionMetadata>, but now you must provide a concrete type for the metadata view (and possibly use ExportFactory<> instead of Lazy<> ?).
However, I can't find any examples of how this should all work - just a few mentions of using a concrete type instead of an interface.
I've tried a few things, but keep getting the following error - "Export metadata for 'AccountID' is missing and no default value was supplied".
My code...
Creating the container (in Global.asax or App_Start folder):
// Get assemblies that will be providing imports and exports
var assemblies = GetAssemblies();
// Get conventions that will be used to find imports and exports
var conventions = GetConventions();
var container = new ContainerConfiguration().WithAssemblies(assemblies, conventions).CreateContainer();
// Create and apply a MefControllerFactory so controllers can be composed
ControllerBuilder.Current.SetControllerFactory(new MefControllerFactory(container));
GetConventions() method:
private static ConventionBuilder GetConventions()
{
var conventionBuilder = new ConventionBuilder();
conventionBuilder.ForTypesDerivedFrom<IController>().Export();
conventionBuilder.ForTypesDerivedFrom<IExtension>().Export<IExtension>();
conventionBuilder.ForTypesMatching(t => t.Namespace != null && t.Namespace.EndsWith(".Parts")).Export().ExportInterfaces();
return conventionBuilder;
}
IExtension.cs:
public interface IExtension
{
void DoWork();
}
ExtensionMetadata.cs:
public class ExtensionMetadata
{
public int AccountID { get; set; }
}
ExtensionA.cs (same as ExtensionB.cs):
public void DoWork()
{
System.Diagnostics.Debug.WriteLine("ExtensionA doing work..");
}
ExtensionManager.cs:
public class ExtensionManager
{
private IEnumerable<ExportFactory<IExtension, ExtensionMetadata>> _extensions;
public ExtensionManager(IEnumerable<ExportFactory<IExtension, ExtensionMetadata>> extensions)
{
_extensions = extensions;
}
public void DoWork(int accountID)
{
foreach (var extension in _extensions)
{
if (extension.Metadata.AccountID == accountID)
{
extension.DoWork();
}
}
}
}
I think I'm missing something quite major here. Basically I want to lazily import all Extensions, check their metadata and if a condition is fulfilled have that extension do something.
Would really appreciate your feedback or any links to sample code / tutorials that cover my scenario.
Many thanks!
I think I've worked it out after reading this SO question.
I created a Metadata Attribute:
[MetadataAttribute]
public class ExtensionMetadataAttribute : ExportAttribute, IExtensionMetadata
{
public int AccountID { get; set; }
public ExtensionMetadataAttribute(int accountID) : base(typeof (IExtension))
{
AccountID = accountID;
}
}
Then modified ExtensionA.cs:
[ExtensionMetadata(1)]
public class ExtensionA : IExtension
{
public void DoWork()
{
System.Diagnostics.Debug.WriteLine("ExtensionA doing work..");
}
}
And now ExtensionManager.cs looks like this:
public class ExtensionManager : IExtensionManager
{
private readonly IEnumerable<ExportFactory<IExtension, ExtensionMetadata>> _extensions;
public ExtensionManager(IEnumerable<ExportFactory<IExtension, ExtensionMetadata>> extensions)
{
_extensions = extensions;
}
public void DoWork(int accountID)
{
foreach (var extension in _extensions)
{
if (extension.Metadata.AccountID == accountID)
{
using (var foo = extension.CreateExport())
{
foo.Value.DoWork();
}
}
}
}
}
This seems to do the trick, but I would still be interested in any feedback re best practices, performance issues etc.
Thanks!