Develop Reference

How to Soak Test Selenium Scripts

Recently I have been struggling with flaky UI tests in Selenium. After doing a lot of research on ways to make tests a lot more robust (explicit waits, etc.) I came across some people here on stack overflow that were suggesting soak testing the selenium scripts to try and work out all the small, seemingly random, failures that I was getting. I seems like a great idea but there doesn't appear to be a whole lot of resources on how to accomplish this.
My Current thought process is to find some way to write unit tests in MSTest that take the Selenium tests I have, run them multiple times and log the results. But so far I've had no luck with that approach. Specifically, I've tried something like this:
[TestClass]
public abstract class MSTestClass
{
public TestContext TestContext { get; set; }
[AssemblyInitialize]
public static void SetupAssembly(TestContext testContext)
{
}
[TestInitialize]
public void SetupTest()
{
//pull the browser type variable out of the test.runsettings file via the TestContext object
string browserType = TestContext.Properties["Browser"].ToString();
//Start a new Browser for the current test
Browser.Start(browserType);
//Resize the Browser to your application's target width
Browser.ResizeTo(1440, 900);
//pull the site URl from the AppSettings file
string siteURL = System.Configuration.ConfigurationManager.AppSettings["URL"];
Browser.Visit(siteURL);
}
[TestMethod]
public void Test()
{
}
[TestCleanup]
public void CleanupTest()
{
Browser.Shutdown();
}
}
[TestClass]
public class ExampleSoakTests
{
[TestMethod]
public void ActualSoakTest()
{
ExampleTest testClass = new ExampleTest();
for(int i = 0; i<10; i++)
{
try
{
testClass.SetupTest();
testClass.ActualTest();
testClass.CleanupTest();
Log.Success();
}
catch(Exception e)
{
Log.Exception(e);
}
}
}
}
The issue that I have been having with the above code is that the TestContext property doesn't get initialized. I'm assuming that this is because the test method wasn't called by the Visual Studio Test Runner?
In conclusion any help with my current approach or any ideas on other ways to go about Soak Testing my Selenium Scripts would be a great help.

SpecFlow find the steps definitions but it does not execute them

I am building a test framework using NUnit + SpecFlow + Selenium. I have a solution with two projects(so far). at the top level I have the Suite Framework so: PageFactory, DriverFactory, CommonPages, etc. The other project has the actual tests(cucumber), test steps and test pages.
Both projects have the same NuGet packages installed and the second has references to the Suite Framework.
Everything seems to be fine: I have [BeforeTestRun], [BeforeScenario] and [AfterTestRun] on the framework and when I run the test it is able to find them and execute them but when the code gets to the Cucumber features it just skips them, I mean it highlights them but it does not drill into them.
I checked the steps definitions and they are there (I can go to definition and it does find them no matter in which project they are) and the binding seems to be right.
Going to definition
Finding the definition
So far this is an example of my code:
feature:In this file the background refers to file placed on the framework project and the scenario to the feature steps inside the same project.
Feature: Reports
As an admin
I want to access to the Reports
So I can see the information related to my product
Background:
When I navigate to the 'Reports' page
And I navigate to my product reports
#Regression
Scenario: I can open the report
When I click on the 'overall' tile
Then the report is displayed
And the data matches the database
Backgorund steps:
using UI.Suite.CommonPages.Pages;
using OpenQA.Selenium;
using TechTalk.SpecFlow;
namespace UI.Suite.CommonPages.Steps
{
[Binding]
public class SideBarNavigation
{
private readonly SideMenuComponent sideMenuComponent;
public SideBarNavigation(IWebDriver driver)
{
sideMenuComponent = new SideMenuComponent(driver);
}
[When(#"I navigate to the '(.*)' page")]
public void NavigateTo(string page)
{
sideMenuComponent.SideMenuNavigation(page);
}
}
}
Scenarios steps:
using NUnit.Framework;
using TechTalk.SpecFlow;
using UI.Products.Tests.Pages;
using OpenQA.Selenium;
namespace UI.Products.Tests.Steps
{
[Binding]
public class ReportsSteps
{
private readonly ReportsPage _reports;
public ReportsSteps(IWebDriver driver)
{
_reports = new ReportsPage(driver);
}
[When(#"I navigate to my product reports")]
public void WhenINavigateToMyProducts()
{
_reports.SelectMyProduct();
}
[When(#"I click on the '(.*)' tile")]
public void WhenIClickOnAGivenReport(string report)
{
_reports.SelectReportTileAndConfig(report);
}
[Then(#"the report is displayed")]
public void TheReportDisplays()
{
Assert.IsTrue(_reports.HasReportLoaded(), "The report has not loaded correctly");
}
[Then(#"the data matches the database")]
public void TheDataDisplays()
{
Assert.IsTrue(_reports.DoesUIMatchDB(), "The data on the database does not match the UI");
}
}
}
Thanks for your help.

How to run a test project twice using Selenium C# with NUnit

I have a test project which is developed using selenium c# with Nunit. It is integrated with the Build process (CI).
As per the business rule, I have to test the application with two profiles
1. Admin
2. Agent
Which means I have to execute all the selenium tests once with Admin role and re-run all those tests again with Agent role.
class TestCases
{
string user = string.Empty;
ChromeDriver d;
[SetUp]
public void setup()
{
d = new ChromeDriver();
d.Url = "https://www.example.com/";
user = d.FindElement(By.Id("Role")).Text; //Admin or Agent
}
[Test]
public void TestCaseID_1()
{
if (user.Equals("Admin") || user.Equals("Agent"))
{
// code
}
}
[Test]
public void TestCaseID_2()
{
if (user.Equals("Agent"))
{
// code
}
}
[Test]
public void TestCaseID_3()
{
if (user.Equals("Admin"))
{
//change user role from Admin to Agent
}
}
[TearDown]
public void tearrdown()
{
d.Dispose();
d.Quit();
}
}
I am looking for something like this:
1. All the tests should first run as an Admin.
2. In the last test, Admin should degrade his role from Admin to Agent.
3. Now, Re-run all the tests once again for an Agent.
Currently, I am able to test the application either as an Admin or as an Agent.
I am kind of confused on how to re-run those scripts for a new profile (Agent). Kindly help me out.
Thank you for your time and effort.

You can use Test case attribute available in Nunit refer this link here
Then your testmethod can be take a parameter as userRole. Now the testmethod will be run twice with parameter both these roles.
[TestCase("Admin")]
[TestCase("Agent")]
public void UnitTestName(string userRole)
{
//Arrange
//Act
//Assert
}

How to distinguish between testsuite and testcase on the report

Using Selenium C# web driver with NUnit for automation. I am generating Allure report using command line and my report gets fantastically created but I need help on the following issue:
I have the following structure using Page object model (2 Test and 1 Page). Now when I see the report it shows at the top Test run (2 testsuites, 2 testcases) and each testcase is a testsuite. I want it to say 1 testsuites, 2 testcases. How do I do that?
namespace ApplicationName.TestCases
{
[TestFixture]
class VerifyCreateOrder
{
IWebDriver driver;
[SetUp]
public void Initialize()
{
driver = new FirefoxDriver();
}
[TestCase]
public void doCreateOrder()
{
LoginPage loginPage = new LoginPage();
//some Assertion
}
}
}
namespace ApplicationName.TestCases
{
[TestFixture]
class SearchOrder
{
IWebDriver driver;
[SetUp]
public void Initialize()
{
driver = new FirefoxDriver();
}
[TestCase]
public void doSearchOrder()
{
LoginPage loginPage = new LoginPage();
//some Assertion
}
}
}
The below is my LoginPage Page object:
namespace ApplicationName.Pages
{
class LoginPage
{
public void doLogin(IWebDriver driver, String username, String password)
{
driver.Navigate().GoToUrl("http://www.myxyzsite.com");
driver.FindElement(By.Id("xyz")).SendKeys(username);
driver.FindElement(By.Id("xyz")).SendKeys(password);
driver.FindElement(By.Id("xyz")).Click();
}
}
}
I read about the NUnit suite attribute at http://www.nunit.org/index.php?p=suite&r=2.5.5 and created a c# class with enumerator as described but how do i call it/wire it? What changes do I need to make for my test classes?
namespace NUnit.Tests
{
public class MyTestSuite
{
[Suite]
public static IEnumerable Suite
{
get
{
ArrayList suite = new ArrayList();
suite.Add(new VerifyCreateOrder());
suite.Add(new SearchOrder());
return suite;
}
}
}
}

I want it to say 1 testsuites, 2 testcases. How do I do that?
Without adding a Suite or similar, you could put both Test cases into the same TestFixture, since that's what the testsuite output is built from. You may be able to do that using a partial class, or you can simply conflate the two classes. However, your Suite solution is a better choice.
What changes do I need to make for my test classes?
Call NUnit with the option /fixture:NUnit.Tests.MyTestSuite.
Note that all of this has changed with NUnit 3 and the Suite attribute is gone. I can't see any way to do what you want in NUnit 3 short of reorganizing your test cases.
If it's very important to merge tests into suites, you can use XSLT. The NUnit test result schema is quite straightforward and easy to manipulate using XSLT.

Selective Ignore of NUnit tests

I want to ignore certain tests based on data I pulled from a configuration file during the TestFixtureSetUp. Is there a way to ignore running a test based on parameters?
[TestFixture]
public class MessagesTests
{
private bool isPaidAccount;
[TestFixtureSetUp]
public void Init () {
isPaidAccount = ConfigurationManager.AppSettings["IsPaidAccount"] == "True";
}
[Test]
//this test should run only if `isPaidAccount` is true
public void Message_Without_Template_Is_Sent()
{
//this tests an actual web api call.
}
}
If account we are testing with is a paid account, the test should run fine, if not, the method will throw an exception.
Would there be an extension of the attribute [Ignore(ReallyIgnore = isPaidAccount )]? Or should I write this inside the method and run 2 separate test cases for eg.
public void Message_Without_Template_Is_Sent()
{
if(isPaidAccount)
{
//test for return value here
}
else
{
//test for exception here
}
}

You can use Assert.Ignore() like Matthew states. You could also use Assert.Inconclusive() if you want to categorize the result differently.
This Question/Answer is slightly similar: Programmatically skip an nunit test