i've got class as below and i need to test RunMethod. Problem is, that ExternalClass needs some application run on server side (app providing external dll's). My dev environment has that app but my dev environment for test hasn't. My question is, how to shim/fake that ExternalClass in UnitTests to not check if app exist (test always faild in env without that external app)? This class is not important in tests but run automatically if i execute RunMethod.
public class MyExampleClass : ISomeInterface
{
private static ExternalClass = new ExternalClass(string someParam);
public object RunMethod()
{
/* Actuall code hear, doesn't matter */
/* few unimportant (from the view point of tester) operation in ExternalClass (object loggin etc.) */
return someVar;
}
}
MsFakes generate a property named AllInstances to the shim class, through this property you can override the behavior of any instance method:
[TestMethod]
public void TestMethod1()
{
using (ShimsContext.Create())
{
ShimExternalClass.AllInstances.ToString01 = () =>
{
return String.Empty();
};
Assert.IsNull(new ExternalClass().ToString());
}
}
Related
We have some UI integration tests that can't be run on the build server as launching test GUI app requires running the build agent as an user (instead of a service which is how it's currently setup).
This causes the build pipeline to get stuck. So I'd like to run these tests locally but not on the build server.
Is there a way to achieve this using xUnit or MSTests and Azure DevOps build pipeline?
You sure can.
Setup an Environment variable to indicate if it's running on the build server in your build.yml file.
variables:
- name: IsRunningOnBuildServer
value: true
Answer 1: Using xUnit
Now create a custom fact attribute to use this:
// This is taken from this SO answer: https://stackoverflow.com/a/4421941/8644294
public class IgnoreOnBuildServerFactAttribute : FactAttribute
{
public IgnoreOnBuildServerFactAttribute()
{
if (IsRunningOnBuildServer())
{
Skip = "This integration test is skipped running in the build server as it involves launching an UI which requires build agents to be run as non-service. Run it locally!";
}
}
/// <summary>
/// Determine if the test is running on build server
/// </summary>
/// <returns>True if being executed in Build server, false otherwise.</returns>
public static bool IsRunningOnBuildServer()
{
return bool.TryParse(Environment.GetEnvironmentVariable("IsRunningOnBuildServer"), out var buildServerFlag) ? buildServerFlag : false;
}
}
Now use this FactAttribute on your test methods that you want to skip running on build server. For eg:
[IgnoreOnBuildServerFact]
public async Task Can_Identify_Some_Behavior_Async()
{
// Your test code...
}
Answer 2: Using MSTests
Create a custom test method attribute to override the Execute method:
public class SkipTestOnBuildServerAttribute : TestMethodAttribute
{
public override TestResult[] Execute(ITestMethod testMethod)
{
if (!IsRunningOnBuildServer())
{
return base.Execute(testMethod);
}
else
{
return new TestResult[] { new TestResult { Outcome = UnitTestOutcome.Inconclusive } };
}
}
public static bool IsRunningOnBuildServer()
{
return bool.TryParse(Environment.GetEnvironmentVariable("IsRunningOnBuildServer"), out var buildServerFlag) ? buildServerFlag : false;
}
}
Now use this TestMethodAttribute on your test methods that you want to skip running on build server. For eg:
[SkipTestOnBuildServer]
public async Task Can_Identify_Some_Behavior_Async()
{
// Your test code...
}
You can filter out tests based on namespace etc.
dotnet test --filter FullyQualifiedName!~IntegrationTests
This will run all tests NOT containing "IntetegrationTests" in it's namespace.
You can read more about test filtering here: https://learn.microsoft.com/en-us/dotnet/core/testing/selective-unit-tests
I have some api controllers and generated a swagger client with tests for them and filled it out. Here an example of filled test class:
[TestFixture]
public class CategoriesApiTests
{
private CategoriesApi _instance;
[SetUp]
public void Init()
{
_instance = new CategoriesApi("http://localhost:5000/");
}
[Test]
public void InstanceTest()
{
Assert.IsInstanceOf<CategoriesApi>(_instance, "instance is a CategoriesApi");
}
}
But my tests only pass if I run my application.
How can I emulate running an application for testing?
What you may want to do is to separate all business logic into a separate DLL and then use unit testing.
We have an open source project for running remote XUnit.Net tests. The feature is that the body of the test is designed to run on a remote process and the results are serialized back to visual studio. The project can be found here.
https://github.com/Weingartner/XUnitRemote
For example a test might be like
[SampleProcessFact]
public void OutOfProcess()
{
_Output.WriteLine("Process name: " + Process.GetCurrentProcess().ProcessName);
Assert.Equal(1,1);
}
SampleProcessFact is a custom attribute declared so.
[AttributeUsage(AttributeTargets.Method)]
[XunitTestCaseDiscoverer("XUnitRemote.Test.SampleProcessFactDiscoverer", "XUnitRemote.Test")]
public class SampleProcessFactAttribute : FactAttribute { }
[AttributeUsage(AttributeTargets.Method)]
[XunitTestCaseDiscoverer("XUnitRemote.Test.ScheduledSampleProcessFactDiscoverer", "XUnitRemote.Test")]
public class ScheduledSampleProcessFactAttribute : FactAttribute { }
[AttributeUsage(AttributeTargets.Method)]
[XunitTestCaseDiscoverer("XUnitRemote.Test.SampleProcessTheoryDiscoverer", "XUnitRemote.Test")]
public class SampleProcessTheoryAttribute : TheoryAttribute { }
See https://github.com/Weingartner/XUnitRemote/blob/master/XUnitRemote.Test/XUnit.cs#L26 for source code.
However we would like an option so that if I tag my test case as SampleProcessFact then the test runner will only run the tests sequentially.
I am aware that I can tag all my test cases with TestCollection(string id) and it should prevent sequential running but this should be encapsulated in the SampleProcessFact tag if possible. This should work across all test cases in all assemblies.
My question only relates to the firing off of tests from visual studio. The remote part works fine but visual studio is calling our remote test case engine in parallel for all the tests.
Any ideas?
We made a checkin to our project that solved the problem.
https://github.com/Weingartner/XUnitRemote/commit/565e1dfb55f65ff0612afa40bc5d076c69bb739c
The general solution was to intercept the ITestMethod instance within XUnitRemoteTestCaseDiscoverer::Discover and rewrite the UniqueId of it's test collection.
public IEnumerable<IXunitTestCase> Discover(ITestFrameworkDiscoveryOptions discoveryOptions, ITestMethod testMethod, IAttributeInfo factAttribute)
{
if(CollectionId.HasValue)
testMethod = WrapTestMethod(testMethod, CollectionId.Value);
if (!Process.GetCurrentProcess().ProcessName.Equals(Path.GetFileNameWithoutExtension(_ExePath), StringComparison.OrdinalIgnoreCase))
{
yield return new XUnitRemoteTestCase(_DiagnosticMessageSink, discoveryOptions.MethodDisplayOrDefault(), testMethod, _Id, _ExePath);
}
else
{
foreach (var testCase in _DefaultTestCaseDiscoverer.Discover(discoveryOptions, testMethod, factAttribute))
{
yield return _TestCaseConverter(testCase);
}
}
}
and
private ITestMethod WrapTestMethod(ITestMethod testMethod, Guid uniqueId)
{
var testClass = testMethod.TestClass;
var testCollection = testClass.TestCollection;
testCollection = new TestCollection
(testCollection.TestAssembly, testCollection.CollectionDefinition, testCollection.DisplayName)
{
UniqueID = uniqueId
};
testClass = new TestClass(testCollection, testClass.Class);
testMethod = new TestMethod(testClass, testMethod.Method);
return testMethod;
}
I'm trying to test business logic in queries in services. So I don't want my tests to have real access to the database, because they are unit tests, not integration tests.
So I've made a simple example of my context and how I'm trying to shim it.
I have an entity
public class SomeEntity
{
public int Id { get; set; }
public string Name { get; set; }
}
and a service
public class Service
{
public int CountSomeEntites()
{
using (var ctx = new Realcontext())
{
int result = ctx.SomeEntities.Count();
return result;
}
}
}
And this is the real context
public partial class Realcontext : DbContext
{
public virtual DbSet<SomeEntity> SomeEntities { get; set; }
public Realcontext() : base("name=Realcontext")
{
InitializeContext();
}
partial void InitializeContext();
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
throw new UnintentionalCodeFirstException();
}
}
So I've tried to create a fake context and I detourned the constructor of the real context in my test method
This is the fake context
public class FakeContext : DbContext
{
public DbSet<SomeEntity> SomeEntities { get; set; }
public FakeContext()
{
}
}
And finally the test class
[TestClass]
public class ServiceTests
{
[TestMethod]
public void CountEmployee_ShoulReturnCorrectResult()
{
using (ShimsContext.Create())
{
ShimRealcontext.Constructor = context => GenerateFakeContext();
ShimDbContext.AllInstances.Dispose = () => DummyDispose();
Service service = new Service();
int result = service.CountSomeEntites();
Assert.AreEqual(result, 2);
}
}
private FakeContext GenerateFakeContext()
{
FakeContext fakeContext = new FakeContext();
fakeContext.SomeEntities.AddRange(new[]
{
new SomeEntity {Id = 1, Name = "entity1"},
new SomeEntity {Id = 2, Name = "entity2"}
});
return fakeContext;
}
}
When I run the test, the RealContext constructor is returned properly, a FakeContext is built in the GenerateFakeContext() method, it contains 2 SomeEntities and it is returned, but right after, in the service, the property SomeEntities of the variable ctx equals to null.
Is it because my variable ctx is declared as a new RealContext()? But calling the constructor of RealContext returns a FakeContext(), so isn't the variable supposed to be of type FakeContext?
Am I doing something wrong? Or is there any other way to test the service without accessing the real database?
I had the simlair situation and I solved it with build configuration and conditional compilation. It's not the best solution, but it worked for me and solved the problem. Here is the receipt:
1. Create DataContext interface
First you need to create an interface which will be implemented by both context classe you going to use. Let it be named just 'IMyDataContext'. Inside it you need to describe all DbSets you need to have access to.
public interaface IMyDataContext
{
DbSet<SomeEntity> SomeEntities { get; set; }
}
And both your context classes need to impelemt it:
public partial class RealDataContext : DataContext, IMyDataContext
{
DbSet<SomeEntity> SomeEntities { get; set; }
/* Contructor, Initialization code, etc... */
}
public class FakeDataContext : DataContext, IMyDataContext
{
DbSet<SomeEntity> SomeEntities { get; set; }
/* Mocking, test doubles, etc... */
}
By the way you can even make properies read-only at interface level.
2. Add 'Test' build configuration
Here you can find how to add new build configuration. I named my configuratin 'Test'. After new configuration is created, go to your DAL project properties, Build section on the left pane. In the 'Configuration' drop-down select the configuration you've just created and in input 'Conditional compilation symbols' type 'TEST'.
3. Incapsulate context injection
To be clear, my approach is still method/property based DI solution =)
So now we need to implement some injection code. For simplicity you can add it right into your service or extract into another class if you need more abstraction. The main idea is to use conditional compilation direcitves instead of IoC framework.
public class Service
{
// Injector method
private IMyDataContext GetContext() {
// Here is the main code
#if TEST // <-- In 'Test' configuration
// we will use fake context
return new FakeDataContext();
#else
// in any other case
// we will use real context
return new RealDataContext();
#endif
}
public int CountSomeEntites()
{
// the service works with interface and does know nothing
// about the implementation
using (IMyDataContext ctx = GetContext())
{
int result = ctx.SomeEntities.Count();
return result;
}
}
}
Limitations
The described approach solves the the problem you described, but it has a limitation: as IoC allows you switch contexts dynamically at runtime, conditional complation requires you to recompile the solution.
In my case it's not a problem - my code isn't covered by tests for 100% and I don't run them on each build. Usually I run tests only before commiting the code, so it's very easy to switch the build configuration in VS, run tests, make sure that nothing was broke and then return to debug mode. In release mode you don't need to run test either. Even if you need - you can craete "Release build test mode" configuration and continue to use the same solution.
Another problem is if you have continuos integration - you need to make additional setup to your build server. Here you have two ways:
Setup two build definitions: one for release and one for tests. If your server is setup to automatic release you need to be careful because test fails will be shown in the second one while the first is deployed.
Set complex build definition which builds your code in Test configuration for the first time, runs test and if they are OK - then recompiles the code in target configuration and prepare to deploy.
Thus, as any solution this one is yet another compromise between simplisity and flexibility.
UPDATE
After some time I understand that the way I described above is very heavy. I mean - build configurations. In case of only two IDataContext implementations: 'Core' and 'Fake' you can simply use bool paramenter and simple if/else branches instead of compilation directives #if/#else/#endif and all the head ache configuring your build server.
If you have more than two implementations - you can use enum and switch block. A probem here is to define what you will return in default case or if value is out of enum's range.
But the main benefit of such approach is that you can be no longer bound to compilation time. Injector parameter could be changed at any time, for example using web.config and ConfigurationManager. Using it you could event switch your data context at run time.
I am working on a legacy application and my current goal is to set up some integration test which also take use of structure map. So far I had no problem with using SM for unit testing and production code for almost a year. But unfortunately we are forced to use setter injection cos our BOs are real dependency beast (a lot of these objects have circular dependencies on others)
Our workout for this was to use a two step approach: Were we first created all of these objects without inject properties (which cause this circular dependencies) and after wards build up those objects (inject dependency objects through their properties) when they were already created. (See example class A, B and Logic which have circular dependencies)
This worked fine till now ->
When we run lets say 3 integration tests after each other than only the first "turns green" the other test are throwing execptions when structuremap tries to create the instances (in the GetManagerInstances) methods.
I am guessing SM holds its property injection config static.
Definitions
internal interface IM { }
internal interface ILogic { }
internal interface IMA : IM { }
internal interface IMB : IM { }
internal class A : IMA
{
public A() { Console.WriteLine("A"); }
public IMB BInstance { get; set; }
}
internal class B : IMB
{
public B() { Console.WriteLine("B"); }
public IMA AInstance { get; set; }
public ILogic Logic { get; set; }
}
internal class Logic : ILogic
{
public Logic() { Console.WriteLine("Logic"); }
public IMA AInstance { get; set; }
}
Initialization
private static IContainer _container;
while (true)
{
Initialize();
}
internal static void Initialize()
{
_container = new Container(c =>
{
c.For<ILogic>().Singleton().Use<Logic>();
c.For<IMB>().Singleton().Use<B>();
c.For<IMA>().Singleton().Use<A>();
});
// All managers are created and afterwards properties are set to avoid a circular build stack.
List<object> managerInstances = GetManagerInstances();
// After the manager instances are created the dependencies are injected in the properties.
BuildUpManagersUsingPropertyInjection(managerInstances);
}
private static List<object> GetManagerInstances()
{
List<object> managerInstances = new List<object>();
foreach (Type pluginType in new List<Type> { typeof(IMA), typeof(IMB) })
{
try
{
managerInstances.Add(_container.GetInstance(pluginType));
}
catch (Exception ex)
{
// exception on second test run -> see below
}
}
return managerInstances;
}
private static void BuildUpManagersUsingPropertyInjection(List<object> managerInstances)
{
_container.Configure(x => x.SetAllProperties(c =>
{
// configure the property injection for all managers
c.Matching(prop => typeof(IM).IsAssignableFrom(prop.PropertyType));
// client logic is also used by managers
c.TypeMatches(type => type.Equals(typeof(ILogic)));
}));
_container.BuildUp(_container.GetInstance<ILogic>());
managerInstances.ForEach(x => _container.BuildUp(x));
}
Result
StructureMap Exception Code: 295
Bidirectional Dependency Problem
detected with RequestedType:
StructureMapTests.IMA, Name:
237b6fb1-7dee-4f09-b663-c33bb793afc6,
ConcreteType: StructureMapTests.A.
The BuildStack is:
1.) RequestedType: StructureMapTests.IMA, Name:
237b6fb1-7dee-4f09-b663-c33bb793afc6,
ConcreteType: StructureMapTests.A
2.) RequestedType: StructureMapTests.IMB, Name:
ad692281-cd27-493c-8271-3dcf86bacd41,
ConcreteType: StructureMapTests.B
Exception when building
*
Analysis
The output for the First test run is:
A
B
Logic --> Everything OK
The output for the Second test run is:
A
B
Exception when building instances
B
A
Exception when buildinginstances
Logic
A
B --> REPEATING ENDLESS
Please help :(
EDIT
I'm targeting 4.0 .Net framework and using StructureMap release 2.6.2
I have confirmed this is a bug in the current version of StructureMap (as of January 2011). The setter injection policies are in fact being incorrectly shared between containers. The issue has been reported to the StructureMap mailing list.
this does solve the problem with the circular dependency (no exception is thrown when initializing) but the properties are not filled correctly.
A.BInstance = null
B.AInstance = null
Logic.AInstance = null
the properties should be set to the correct instance!