I'm writing integration tests using the MSTest framework. The tests and code under test all have significant logging built into them.
I'm trying to figure out a way to hook into the Assert's output so I can write it to the log files along with the rest of log.
For example, if I have a test method like
[TestMethod]
SomeRandomIntegrationTest()
{
//Code to actually run the test, and includes logging.
Assert.AreEqual(true, false, "This error message should also appear in the log");
}
I would get
Message: Assert.AreEqual failed. Expected true. Got false. This error message should also appear in the log.
in my log file.
I tried doing
private StringBuilder testOutputBuilder;
private StringWriter testOutputWriter;
private TextWriter originalWriter;
[TestInitialize]
public virtual void Initialize()
{
//Redirect the test output into the log files
testOutputBuilder = new StringBuilder();
testOutputWriter = new StringWriter(testOutputBuilder);
originalWriter = Console.Out;
Console.SetOut(testOutputWriter);
}
[TestCleanup]
public virtual void TestCleanup()
{
if (TestContext.CurrentTestOutcome != UnitTestOutcome.Passed)
{
//File logging happens here using the testOutputBuilder
}
Console.SetOut(originalWriter);
testOutputWriter.Dispose();
}
but the testOutputBuilder returns an empty string.
How can I grab the string outputs from the assert methods in MSTest?
I wrote a workaround using separate function:
public string WriteErrorToFile(TextWriter textwriter, string errorMessage)
{
textwriter.WriteLine(errorMessage);
return errorMessage;
}
and code inside test should be modified like:
Assert.AreEqual(true, false, WriteErrorToFile(originalWriter, "This error message should also appear in the log"));
If you have only one log file then you can remove first parameter to the function.
Hope this helps
I did this:
public void OutputAssert(Action func)
{
try
{
func();
}
catch (Exception ex)
{
OutputToFile(ex.Message);
throw ex;
}
}
And then in the test:
[TestMethod]
public void TestAssertOutput()
{
OutputAssert(() => Assert.AreEqual(false, true, "test message"));
}
The output being:
Assert.AreEqual failed. Expected:<False>. Actual:<True>. test message
Related
I am still learning to use MSTest and Moq for automated unit testing in my application. I have successfully mocked the code and run it. It is showing that the tests are passed , but the code coverage is 0%. This is my code below.What needs to be changed so that code coverage becomes 100%.
I know this question has been asked a couple of times before, but nothing seems to help me.So can anyone suggest me what am I doing wrong.
Any help is highly appreciated.Thanks.
PS: I'm using Sonarcube for knowing the code coverage.
using Moq;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Threading.Tasks;
using System.Diagnostics.CodeAnalysis;
namespace MyNameSpace
{
[TestClass]
public class ApplicationTest
{
readonly Helper moqHelper = new Helper();
[TestMethod()]
public void GetDataFromDataBaseMoq()
{
Task<bool> returnValue;
Mock<Application> mockType = new Mock<Application>();
mockType.CallBase = true;
mockType.Setup(x => x.GetDataFromDataBase()).Returns(returnValue = moqHelper.GetDataFromDataBaseMoq());
if (returnValue.Result)
{
Assert.IsTrue(true);
}
else
{
Assert.Fail();
}
}
}
[ExcludeFromCodeCoverage]
class Helper
{
internal async Task<bool> GetDataFromDataBaseMoq()
{
bool returnValue = true;
return returnValue;
}
}
public class Application : IApplication
{
public virtual async Task<bool> GetDataFromDataBase()
{
//if data retrive successfull, return true, else false
return true;
}
}
public interface IApplication
{
Task<bool> GetDataFromDataBase();
}
}
You're not testing your application code, you're testing your mock. You could've seen this by setting a breakpoint in Application.GetDataFromDataBase() and debugging your test; you'd see it won't be hit.
You need to only mock dependencies, if any. So rewrite your test to actually call into your code:
[TestMethod]
public async Task GetDataFromDataBase_Returns_True()
{
// Arrange
IApplication classUnderTest = new Application();
// Act
var result = await classUnderTest.GetDataFromDataBase();
// Assert
Assert.IsTrue(result);
}
And you'll see the need for all the mocks and helpers goes away.
Quite straighforward. This is generic code from some sandbox experiments. ExcludeFromCodeCoverage seems to be working properly for synchronhous methods. No runsettings or any configuration related to the coverage are in place.
[ExcludeFromCodeCoverage] // Still included in coverage report.
private async Task ExecuteRetryable(Func<Task> function)
{
try
{
await retryPolicyAsync.ExecuteAsync(function);
}
catch (Exception exception)
{
LogException(exception);
}
}
[ExcludeFromCodeCoverage] // Not included in coverage report as expected.
private void LogException(Exception exception)
{
if (TelemetryClient == null)
return;
var telemetry = new ExceptionTelemetry(exception);
telemetry.Properties.Add("typeCode", "ExceptionFromTheSandbox");
TelemetryClient.TrackException(telemetry);
}
}
Could this be a bug or am I missing something?
[Test]
public void AssertMultipleTest()
{
try
{
Assert.Multiple(() =>
{
Assert.That(true, Is.False);
Assert.That(7, Is.Zero);
});
}
catch (Exception e)
{
Log.Save(e.ToString());
throw;
}
}
The test function generates the following report:
One or more failures in Multiple Assert block:
1) Expected: False
But was: True
2) Expected: 0
But was: 7
How can I get the same report in the catch-block? Now I only get the following:
Tests.AssertMultipleTest - NUnit.Framework.MultipleAssertException: One or more failures in Multiple Assert block:
at NUnit.Framework.Assert.Multiple(TestDelegate testDelegate)
at AnonymizeDataTests.Tests.AssertMultipleTest() in C:\Development\TestClass.cs:line 76
Maybe I found a solution to my problem. Instead of logging the error in the test itself I skip the entire try-catch-block and log the result in TearDown instead. Something like this:
[TearDown]
public void TearDown()
{
if (TestContext.CurrentContext.Result.Outcome.Status != TestStatus.Passed)
{
Log.Save(TestContext.CurrentContext.Result.Message);
}
}
[Test]
public void AssertMultipleTest()
{
Assert.Multiple(() =>
{
Assert.That(true, Is.False);
Assert.That(7, Is.Zero);
});
}
When I implement cross browser testing with nunit using TestFixture my tests fails when run together, passed when run individually. Exception was throwed when SendKeys method was called because argument was null, but this is not the cause, because when i run this test again test will passed. Ofcourse im tried to debug this issue but i dont find solution. Simple OpenHomePage Test works fine. Here is my code:
[TestFixture(typeof(ChromeDriver))]
[TestFixture(typeof(FirefoxDriver))]
public class TestClass<TWebDriver> where TWebDriver : IWebDriver, new()
{
[OneTimeSetUp]
public void CreateDriver()
{
try
{
PropertiesCollection.driver = new TWebDriver();
Console.WriteLine("Opened browser");
PropertiesCollection.driver.Url = "http://localhost:81/";
Console.WriteLine("Opened URL");
PropertiesCollection.driver.Manage().Window.Maximize();
//initialize test data from excel sheet
ExcelLib.PopulateInCollection(#"c:\users\bolec\documents\visual studio 2015\Projects\RowingSectionTests\RowingSectionTests\TestData.xlsx");
}
catch (Exception msg)
{
Console.WriteLine(msg.ToString());
}
}
[OneTimeTearDown]
public void FixtureTearDown()
{
HomePageObjects homeObj = new HomePageObjects();
homeObj.Logoff();
if (PropertiesCollection.driver != null) PropertiesCollection.driver.Quit();
}
[TearDown]
public void TearDown()
{
//Take screen on failure
if (TestContext.CurrentContext.Result.Outcome.Status.Equals(TestStatus.Failed))
{
string fileName = Regex.Replace(TestContext.CurrentContext.Test.FullName + "_" + DateTime.Now.ToString(), "[^a-z0-9\\-_]+", "_", RegexOptions.IgnoreCase);
((ITakesScreenshot)PropertiesCollection.driver).GetScreenshot().SaveAsFile(#"c:\users\bolec\documents\visual studio 2015\Projects\RowingSectionTests\RowingSectionTests\Screenshots\" + fileName + ".png", System.Drawing.Imaging.ImageFormat.Png);
}
}
//will always passed
[Test]
public void OpenHomePage()
{
HomePageObjects homeObj = new HomePageObjects();
}
//login with correct credentials will login to acc
[Test]
public void Login()
{
HomePageObjects homeObj = new HomePageObjects();
LoginPageObjects loginObj = homeObj.ToLoginPage();
loginObj.Login(ExcelLib.ReadData(1, "UserName"), ExcelLib.ReadData(1, "Password"));
//checking is URL correct after loggin
Assert.AreEqual("http://localhost:81/", PropertiesCollection.driver.Url.ToString());
//checking is login is correct on navbar
Assert.AreEqual(homeObj.GetUserLoginStringInButton().ToLower(), ExcelLib.ReadData(1, "UserName").ToLower());
}
The problem with using the static PropertiesCollection is that any changes to the static class in one test will be reflected in the other test, making the chances or creating a test dependency very high (as you have discovered).
You have two choices, firstly don't use a static instead create an instance. Alternatively, ensure that in a setup and your teardown methods you set/reset your PropertiesCollection back to it's required state.
Using the OneTimeSetUp attribute is also risky as it only runs once for all the tests in your fixture.
Here's my code :
[TestInitialize]
public void init()
{
_browser = new DefaultSelenium("localhost", 4444, #"*iehta", "http://localhost:4444");
}
[TestMethod]
public void TestLogin()
{
bool hasText;
_browser.Start();
_browser.Open("http://localhost/testSite.asp");
_browser.Type("id=NomUtilisateur", "admin");
_browser.Type("id=UserPassword", "password");
_browser.Click("name=Submit");
_browser.WaitForPageToLoad("30000");
hasText = _browser.IsTextPresent("test");
Assert.IsTrue(hasText, #"The search result does not contain text ""test"".");
}
[TestMethod]
public void TestRequisitionPhotocopie()
{
_browser.Start();
_browser.Open("http://localhost/testSite.asp");
_browser.Type("id=NomUtilisateur", "admin");
_browser.Type("id=UserPassword", "password");
_browser.Click("name=Submit");
_browser.WaitForPageToLoad("30000");
_browser.Click("link=lnkTest");
_browser.WaitForPageToLoad("30000");
}
[TestCleanup]
public void clean()
{
_browser.Stop();
//_browser.Close();
}
If I run the two test method, the secon test always fail with an error message like this :
Remote server don't exist or is unavaible
If I comment one of the test method, it's working, my two test method are working
Where's my mistake.
Thanks
EDIT :
The error is not happening everytime, but the error is in Selenium Remote Control If i debug the error is in selenium-browserbot.js at line 724 : windowToModify.seleniumAlert = windowToModify.alert;
Selenium log console have nothing in it
Have you tried it like this? I always start selenium in the setup of the test. (i'm also using Nunit
[SetUp]
public void init()
{
_browser = new DefaultSelenium("localhost", 4444, #"*iehta", "http://localhost:4444");
_browser.Start();
_browser.Open("http://localhost/testSite.asp");
}
[TestMethod]
public void TestLogin()
{
bool hasText;
_browser.Type("id=NomUtilisateur", "admin");
_browser.Type("id=UserPassword", "password");
_browser.Click("name=Submit");
_browser.WaitForPageToLoad("30000");
hasText = _browser.IsTextPresent("test");
Assert.IsTrue(hasText, #"The search result does not contain text ""test"".");
}
[TestMethod]
public void TestRequisitionPhotocopie()
{
_browser.Type("id=NomUtilisateur", "admin");
_browser.Type("id=UserPassword", "password");
_browser.Click("name=Submit");
_browser.WaitForPageToLoad("30000");
_browser.Click("link=lnkTest");
_browser.WaitForPageToLoad("30000");
}
[TearDown]
public void clean()
{
_browser.Stop();
//_browser.Close();
}