Develop Reference

Skip unit tests while running on the build server

We have some UI integration tests that can't be run on the build server as launching test GUI app requires running the build agent as an user (instead of a service which is how it's currently setup).
This causes the build pipeline to get stuck. So I'd like to run these tests locally but not on the build server.
Is there a way to achieve this using xUnit or MSTests and Azure DevOps build pipeline?

You sure can.
Setup an Environment variable to indicate if it's running on the build server in your build.yml file.
variables:
- name: IsRunningOnBuildServer
value: true
Answer 1: Using xUnit
Now create a custom fact attribute to use this:
// This is taken from this SO answer: https://stackoverflow.com/a/4421941/8644294
public class IgnoreOnBuildServerFactAttribute : FactAttribute
{
public IgnoreOnBuildServerFactAttribute()
{
if (IsRunningOnBuildServer())
{
Skip = "This integration test is skipped running in the build server as it involves launching an UI which requires build agents to be run as non-service. Run it locally!";
}
}
/// <summary>
/// Determine if the test is running on build server
/// </summary>
/// <returns>True if being executed in Build server, false otherwise.</returns>
public static bool IsRunningOnBuildServer()
{
return bool.TryParse(Environment.GetEnvironmentVariable("IsRunningOnBuildServer"), out var buildServerFlag) ? buildServerFlag : false;
}
}
Now use this FactAttribute on your test methods that you want to skip running on build server. For eg:
[IgnoreOnBuildServerFact]
public async Task Can_Identify_Some_Behavior_Async()
{
// Your test code...
}
Answer 2: Using MSTests
Create a custom test method attribute to override the Execute method:
public class SkipTestOnBuildServerAttribute : TestMethodAttribute
{
public override TestResult[] Execute(ITestMethod testMethod)
{
if (!IsRunningOnBuildServer())
{
return base.Execute(testMethod);
}
else
{
return new TestResult[] { new TestResult { Outcome = UnitTestOutcome.Inconclusive } };
}
}
public static bool IsRunningOnBuildServer()
{
return bool.TryParse(Environment.GetEnvironmentVariable("IsRunningOnBuildServer"), out var buildServerFlag) ? buildServerFlag : false;
}
}
Now use this TestMethodAttribute on your test methods that you want to skip running on build server. For eg:
[SkipTestOnBuildServer]
public async Task Can_Identify_Some_Behavior_Async()
{
// Your test code...
}

You can filter out tests based on namespace etc.
dotnet test --filter FullyQualifiedName!~IntegrationTests
This will run all tests NOT containing "IntetegrationTests" in it's namespace.
You can read more about test filtering here: https://learn.microsoft.com/en-us/dotnet/core/testing/selective-unit-tests

How to generate tests based on data in nunit framework using C#

So i have this code:
[TestFixture]
[Category("MyTestSet")]
public class MyTests
{
[Test]
public void TestCase12()
{
ExecuteTestCase(12);
}
[Test]
public void TestCase13()
{
ExecuteTestCase(13);
}
[Test]
public void TestCase14()
{
ExecuteTestCase(14);
}
}
The ExecuteTestCase gets test parameters from my web server and executes the test case with these settings.
Each time i add a new test case parameters on my web server i need to add a new test in my C# code and pass the ID of test case parameters i have in my web server database and compile my code.
Is there any way to do it automatically? Like say, C# gets from my server ID's of all test case parameters and creates tests for them on the fly?
What is important, test cases change frequently. I was thinking about running all test cases in one test case on a loop, but than i'd be unable to run my test cases separately for example in Nunit IDE.
So my question is: how to run multiple test cases depending on data i receive on run time.

You can use TestCaseSourceattribute in order to get parameters from web service and have your test cases auto generated
[TestFixture]
[Category("MyTestSet")]
public class MyTests
{
[Test, TestCaseSource(nameof(GetTestParameters))]
public void TestCase(int parameter)
{
ExecuteTestCase(parameter);
}
static int[] GetTestParameters()
{
//call web service and get parameters
return new[] { 1, 2, 3 };
}
}
documentation

How to distinguish between testsuite and testcase on the report

Using Selenium C# web driver with NUnit for automation. I am generating Allure report using command line and my report gets fantastically created but I need help on the following issue:
I have the following structure using Page object model (2 Test and 1 Page). Now when I see the report it shows at the top Test run (2 testsuites, 2 testcases) and each testcase is a testsuite. I want it to say 1 testsuites, 2 testcases. How do I do that?
namespace ApplicationName.TestCases
{
[TestFixture]
class VerifyCreateOrder
{
IWebDriver driver;
[SetUp]
public void Initialize()
{
driver = new FirefoxDriver();
}
[TestCase]
public void doCreateOrder()
{
LoginPage loginPage = new LoginPage();
//some Assertion
}
}
}
namespace ApplicationName.TestCases
{
[TestFixture]
class SearchOrder
{
IWebDriver driver;
[SetUp]
public void Initialize()
{
driver = new FirefoxDriver();
}
[TestCase]
public void doSearchOrder()
{
LoginPage loginPage = new LoginPage();
//some Assertion
}
}
}
The below is my LoginPage Page object:
namespace ApplicationName.Pages
{
class LoginPage
{
public void doLogin(IWebDriver driver, String username, String password)
{
driver.Navigate().GoToUrl("http://www.myxyzsite.com");
driver.FindElement(By.Id("xyz")).SendKeys(username);
driver.FindElement(By.Id("xyz")).SendKeys(password);
driver.FindElement(By.Id("xyz")).Click();
}
}
}
I read about the NUnit suite attribute at http://www.nunit.org/index.php?p=suite&r=2.5.5 and created a c# class with enumerator as described but how do i call it/wire it? What changes do I need to make for my test classes?
namespace NUnit.Tests
{
public class MyTestSuite
{
[Suite]
public static IEnumerable Suite
{
get
{
ArrayList suite = new ArrayList();
suite.Add(new VerifyCreateOrder());
suite.Add(new SearchOrder());
return suite;
}
}
}
}

I want it to say 1 testsuites, 2 testcases. How do I do that?
Without adding a Suite or similar, you could put both Test cases into the same TestFixture, since that's what the testsuite output is built from. You may be able to do that using a partial class, or you can simply conflate the two classes. However, your Suite solution is a better choice.
What changes do I need to make for my test classes?
Call NUnit with the option /fixture:NUnit.Tests.MyTestSuite.
Note that all of this has changed with NUnit 3 and the Suite attribute is gone. I can't see any way to do what you want in NUnit 3 short of reorganizing your test cases.
If it's very important to merge tests into suites, you can use XSLT. The NUnit test result schema is quite straightforward and easy to manipulate using XSLT.

Selective Ignore of NUnit tests

I want to ignore certain tests based on data I pulled from a configuration file during the TestFixtureSetUp. Is there a way to ignore running a test based on parameters?
[TestFixture]
public class MessagesTests
{
private bool isPaidAccount;
[TestFixtureSetUp]
public void Init () {
isPaidAccount = ConfigurationManager.AppSettings["IsPaidAccount"] == "True";
}
[Test]
//this test should run only if `isPaidAccount` is true
public void Message_Without_Template_Is_Sent()
{
//this tests an actual web api call.
}
}
If account we are testing with is a paid account, the test should run fine, if not, the method will throw an exception.
Would there be an extension of the attribute [Ignore(ReallyIgnore = isPaidAccount )]? Or should I write this inside the method and run 2 separate test cases for eg.
public void Message_Without_Template_Is_Sent()
{
if(isPaidAccount)
{
//test for return value here
}
else
{
//test for exception here
}
}

You can use Assert.Ignore() like Matthew states. You could also use Assert.Inconclusive() if you want to categorize the result differently.
This Question/Answer is slightly similar: Programmatically skip an nunit test

NUnit, TestDriven.NET, WatiN and Specflow

I am trying to do some BDD testing using Specflow, NUnit and WatiN. I am using TestDriven.NEt to run the test. Here is my first test:
[Binding]
[TestFixture, RequiresSTA]
public class RegisterUserSteps
{
private IE _ie = new IE();
[When(#"the user visits the registration page")]
public void WhenTheUserVisitsTheRegistrationPage()
{
_ie.GoTo("http://localhost:1064/Register/");
}
[When(#"enter the following information")]
public void WhenEnterTheFollowingInformation(Table table)
{
foreach(var tableRow in table.Rows)
{
var field = _ie.TextField(Find.ByName(tableRow["Field"]));
if(!field.Exists)
{
Assert.Fail("Field does not exists!");
}
field.TypeText(tableRow["Value"]);
}
}
[When(#"click the ""Register"" button")]
public void WhenClickTheRegisterButton()
{
ScenarioContext.Current.Pending();
}
[Then(#"the user should be registered")]
public void ThenTheUserShouldBeRegistered()
{
ScenarioContext.Current.Pending();
}
}
The problem is that it never goes to the
[When(#"enter the following information")]
public void WhenEnterTheFollowingInformation(Table table)
It just launches the browser and perform the first step. Am I missing something?

Without looking at the test, it seems you are missing an important step (Given). Usually it is like this:
Given I go to some page
And all the set up data are available - optional
When I enter the following info
And I click "Register" button
Then I see something
Basically the steps are GWT (Given, When, Then). It's Gherkin language, so if you google for it you'll see more info. When you have multiple things for a given step, you have to use And, example, When ...... And......., not When...... When........