Develop Reference

Skip unit tests while running on the build server

We have some UI integration tests that can't be run on the build server as launching test GUI app requires running the build agent as an user (instead of a service which is how it's currently setup).
This causes the build pipeline to get stuck. So I'd like to run these tests locally but not on the build server.
Is there a way to achieve this using xUnit or MSTests and Azure DevOps build pipeline?

You sure can.
Setup an Environment variable to indicate if it's running on the build server in your build.yml file.
variables:
- name: IsRunningOnBuildServer
value: true
Answer 1: Using xUnit
Now create a custom fact attribute to use this:
// This is taken from this SO answer: https://stackoverflow.com/a/4421941/8644294
public class IgnoreOnBuildServerFactAttribute : FactAttribute
{
public IgnoreOnBuildServerFactAttribute()
{
if (IsRunningOnBuildServer())
{
Skip = "This integration test is skipped running in the build server as it involves launching an UI which requires build agents to be run as non-service. Run it locally!";
}
}
/// <summary>
/// Determine if the test is running on build server
/// </summary>
/// <returns>True if being executed in Build server, false otherwise.</returns>
public static bool IsRunningOnBuildServer()
{
return bool.TryParse(Environment.GetEnvironmentVariable("IsRunningOnBuildServer"), out var buildServerFlag) ? buildServerFlag : false;
}
}
Now use this FactAttribute on your test methods that you want to skip running on build server. For eg:
[IgnoreOnBuildServerFact]
public async Task Can_Identify_Some_Behavior_Async()
{
// Your test code...
}
Answer 2: Using MSTests
Create a custom test method attribute to override the Execute method:
public class SkipTestOnBuildServerAttribute : TestMethodAttribute
{
public override TestResult[] Execute(ITestMethod testMethod)
{
if (!IsRunningOnBuildServer())
{
return base.Execute(testMethod);
}
else
{
return new TestResult[] { new TestResult { Outcome = UnitTestOutcome.Inconclusive } };
}
}
public static bool IsRunningOnBuildServer()
{
return bool.TryParse(Environment.GetEnvironmentVariable("IsRunningOnBuildServer"), out var buildServerFlag) ? buildServerFlag : false;
}
}
Now use this TestMethodAttribute on your test methods that you want to skip running on build server. For eg:
[SkipTestOnBuildServer]
public async Task Can_Identify_Some_Behavior_Async()
{
// Your test code...
}

You can filter out tests based on namespace etc.
dotnet test --filter FullyQualifiedName!~IntegrationTests
This will run all tests NOT containing "IntetegrationTests" in it's namespace.
You can read more about test filtering here: https://learn.microsoft.com/en-us/dotnet/core/testing/selective-unit-tests

TestContext class not found in .NET-core

I am moving and refactoring a code from .NET-Framework to .NET-Core in C#. When I run a simple test on a method which is supposed to sort a List, I get this error:
"System.MissingMethodException: Method not found: 'System.Collections.IDictionary Microsoft.VisualStudio.TestTools.UnitTesting.TestContext.get_Properties()'."
I have checked references to other namespaces that are necessary. I searched for the error online and I realized that the TestContext Class has not been provided in .NET Core yet! Is there another way or a replacement library I can use instead? Thank you.
[TestMethod]
public void TestMethod()
{
// Arrange
// Grab an arbitrary start time.
Time startTime = Time.Now;
List<TimeValue> values = new List<TimeValue>
{
// Make sure that second timestamp comes before the first
timestamp
new TimeValue(startTime.PlusMinutes(1)),
new TimeValue(startTime)
};
// Ensure that this is sorted in ascending order of time.
List<TimeValue> expectedValues = values.OrderBy(value =>
value.Timestamp).ToList();
CollectionAssert.AreNotEqual(expectedValues, values);
// Act
SortArray myObj = new SortArray(values);
// Assert
CollectionAssert.AreEqual(expectedValues, SortArray.Values);
}
I expect the TestMethod to run, but it does not run and gives me the following error:
'System.Collections.IDictionary Microsoft.VisualStudio.TestTools.UnitTesting.TestContext.get_Properties()'.

A possible alternate you could use is xUnit. Its a open sourced tool and you can use it with .NET Core.
Microsoft offers a tutorial on how to xUnit with the .NET Core.
Another possibility is "dotnet test" which is a unit testing tool that Microsoft made compatible for .NET Core.

Try adding to your test class the following property:
public TestContext TestContext { get; set; }
In general, MSTest does not seem to be actively developed. As it's shipped with Visual Studio, Microsoft keeps it working on .NET (and somewhat even on .NET Core) but they seem to use xUnit themselves internally, so it makes sense to consider switching your tests to xUnit either.

It works when you provide a field of type TestContext. It doesn't work when you make it a property. Following works with .NET Core 3.1 as described here.
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace TimeLogger.Tests
{
[TestClass]
public class YourTestClass
{
private static TestContext Context;
[ClassInitialize]
public static void InitClass(TestContext testContext)
{
Context = testContext;
}
[TestMethod]
public void Test_1()
{
Assert.IsTrue(true);
}
[TestMethod]
public void Test_2()
{
Assert.IsTrue(true);
}
}
}
But after changing
private static TestContext Context;
into
private static TestContext Context { get; set; }
causes tests don't run anymore.

Visual Studio 2017 not discovering NUnit tests when tests class inherits from a class that implements NHibernate

I'm having this issue that can't resolve, I'm upgrading from NUnit 2.6.4 to 3.9.0, my test project have multiple test class, and when I change NUnit version, some of my tests weren't discovered by test explorer, after some research, all tests missing inherits or somehow implement NHibernate and Spring NUnit testing nuget package. When I remove inheritance, tests are discover. No solution works for this.
Nuget packages version:
Spring.Testing.NUnit
2.0.1 NUnit 3.9.0
NHibernate 3.3.3.4
NUnitTestAdapter 3.9
This is my NHibernate class:
using System;
using NHibernate;
using Spring.Data.NHibernate.Generic;
using Spring.Data.NHibernate.Support;
using Spring.Testing.NUnit;
namespace Testproject{
public class NHibernateTestClass : AbstractTransactionalSpringContextTests
{
//Some methods here
}
}
This is my test class:
using System;
using System.Collections.Generic;
using System.Linq;
using NUnit.Framework;
namespace Testproject{
public class TestClass: NhibernateTestClass{
//Some test methods here
}
}
I have tried referencing NUnit framework in my NHibernateTestClass but, with no result.
Edit:
Forgot to add that my Hibernate test class was inheriting from that spring test class.

Just install NUnit3TestAdapter via nuget.
And add NUnit attributes([TestFixture],[Test]) before class and method declaring.

Well i have simmilar problem but with visual studio 2015 (community)
I have 2 test classes one is:
namespace WarehouseTemplate.Tests
{
[TestFixture]
public class Test1
{
[SetUp]
public void Init()
{
}
[Test()]
public void Can_generate_schema()
{
var cfg = new Configuration();
cfg.Configure();
new SchemaExport(cfg).Execute(true, true, false);
}
}
}
With can be found in test explorer, and then i have this one
namespace WarehouseTemplate.Tests
{
[TestFixture]
public class TestDao : AbstractDaoIntegrationTests
{
private IProductDao productDao;
private ISessionFactory sessionFactory;
// These properties will be injected based on type
public IProductDao ProductDao
{
set { productDao = value; }
}
public ISessionFactory SessionFactory
{
set { sessionFactory = value; }
}
[SetUp]
public void Init()
{
}
[Test()]
public void CustomerDaoTests()
{//logic here
}
}
}
where AbstractDaoIntegrationTests looks
namespace WarehouseTemplate.Tests
{
[TestFixture]
public class AbstractDaoIntegrationTests : AbstractTransactionalDbProviderSpringContextTests
{
protected override string[] ConfigLocations
{
get
{
return new string[]
{
"referenceString"
};
}
}
}
}
But i cant find this test only first one:
NUnit Adapter 3.9.0.0: Test execution started
Running all tests in E:\Zabava\C# programy\WarehouseTemplate\WarehouseTemplate\bin\Debug\WarehouseTemplate.exe
NUnit3TestExecutor converted 1 of 1 NUnit test cases
NUnit Adapter 3.9.0.0: Test execution complete
So far any tip for possible problem is that spring NET has different NUNIT version which needs scpeficy reference or NUnit Adapter

I found Answer and solution
I just instale NUnit 2.6.3 )i manually choose older one, and propriet Adapter version now i can see my tests

Nunit. Take screenshot on test failure using ITestListener

I want to take screenshot of a failed test case. But I don't know how to force Nunit to use my listener.
I was trying to use IAddins, but Nunit doesn't have NUnit.Core.Extensibility lib.
My code:
using System;
using OpenQA.Selenium;
using NUnit.Framework.Interfaces;
using AT_MentoringPortal.Driver;
using System.Drawing.Imaging;
namespace AT_MentoringPortal.listeners
{
public class ScreenshotListener : ITestListener
{
private readonly string path = ".//screens//";
public void TestFinished(ITestResult result)
{
if (result.ResultState.Status == TestStatus.Failed)
{
IWebDriver driver = DriverFactory.GetDriver();
this.MakeScreenshot(driver, result.Name);
}
}
public void MakeScreenshot(IWebDriver driver, string testName)
{
string timestamp = DateTime.Now.ToString("yyyy-MM-dd-hhmm-ss");
var screenshot = ((ITakesScreenshot)driver).GetScreenshot();
screenshot.SaveAsFile($"{this.path}{timestamp} {testName}", ImageFormat.Jpeg);
}
public void TestOutput(TestOutput output)
{
// throw new NotImplementedException();
}
public void TestStarted(ITest test)
{
// throw new NotImplementedException();
}
}
}
Please, show me how to start my listener in test class.

ITestListener is an internal interface used by NUnit itself in running tests. There was a similar interface in NUnit V2 (TestListener) and you were able to create addins that used it. NUnit 3 doesn't have addins in the way that NUnit 2 did, although it can be extended in other ways.
Did you want to save a screen shot for only certain tests? Or for each test in a certain fixture? Or more generally?
To do it within a fixture, you could use a OneTimeTearDown method.

NUnit 2.6.2 TestContext.CurrentContext always null

I'm trying to use the TestContext.CurrentContext of NUnit 2.6.2 but it's always null.
What I would like is to have an output with the result of tests, but if I run the following code I always get a NullReferenceException in the TearDown method.
All the properties inside Test and Result are throwing the exception.
[TestFixture]
public class UtilitiesTests
{
[TearDown]
public void TearDown()
{
//using console here just for sake of simplicity.
Console.WriteLine(String.Format("{0}: {1}", TestContext.CurrentContext.Test.FullName, TestContext.CurrentContext.Result.Status));
}
[Test]
public void CleanFileName()
{
var cleanName = Utilities.CleanFileName("my &file%123$99\\|/?\"*:<>.jpg");
Assert.AreEqual("my-efile12399---.jpg", cleanName);
}
}
What I'm possibly doing wrong?

According to this discussion you have to make sure you execute with the correct version of the NUnit testrunner. The version has to be NUnit 2.6.2.
Try to run your tests with nunit-console with the correct version.
Update: I did set up a new project in VS2012 and added NUnit 2.6.2 and NUnit.Runners 2.6.2 using NuGet. With the Resharper Testrunner I did get no error but also no Console output, so I did run NUnit.exe from <project-folder>\packages\NUnit.Runners.2.6.2\tools\
This is the output I recieved:
The result looks ok.
I ran your example code above.
However, I had to modify your code so I could run it:
using System;
using NUnit.Framework;
[TestFixture]
public class UtilitiesTests
{
[TearDown]
public void TearDown()
{
//using console here just for sake of simplicity.
Console.WriteLine(String.Format("{0}: {1}", TestContext.CurrentContext.Test.FullName, TestContext.CurrentContext.Result.Status));
}
[Test]
public void CleanFileName()
{
var cleanName = "my &file%123$99\\|/?\"*:<>.jpg";
Assert.AreEqual("my &file%123$99\\|/?\"*:<>.jpg", cleanName);
}
}
You should try to run your tests using NUnit.exe again, but before verify that you have the correct verison in Help -> About NUnit ...
Mine looks like this: