Does any of you know possible solution for taking screenshots on test failures and exceptions?
I've added following code in TearDown() but as a result it also makes screenshots on passed tests, so it is not the best solution:
DateTime time = DateTime.Now;
string dateToday = "_date_" + time.ToString("yyyy-MM-dd") + "_time_" + time.ToString("HH-mm-ss");
Screenshot screenshot = ((ITakesScreenshot)driver).GetScreenshot();
screenshot.SaveAsFile((settings.filePathForScreenShots + "Exception" + dateToday + ".png"), System.Drawing.Imaging.ImageFormat.Png);
I've already found that idea: http://yizeng.me/2014/02/08/take-a-screenshot-on-exception-with-selenium-csharp-eventfiringwebdriver/, to use WebDriverExceptionEventArgs, but for some reasons it makes also some random screenshots without any reasonable explanation.
Other ideas I found are for Java and not for NUnit which I use with Selenium, so they are pretty useless.
If you put the screenshot logic in your TearDown method it will be called after each test finishes, no matter if it succeeded or failed.
I use a base class that has a function which wraps the tests and catches all exceptions. When a test fails the exception is caught and a screenshot is taken.
I use this base class for all my Selenium tests and it looks something like this:
public class PageTestBase
{
protected IWebDriver Driver;
protected void UITest(Action action)
{
try
{
action();
}
catch (Exception ex)
{
var screenshot = Driver.TakeScreenshot();
var filePath = "<some appropriate file path goes here>";
screenshot.SaveAsFile(filePath, ImageFormat.Png);
// This would be a good place to log the exception message and
// save together with the screenshot
throw;
}
}
}
The test classes then look like this:
[TestFixture]
public class FooBarTests : PageTestBase
{
// Make sure to initialize the driver in the constructor or SetUp method,
// depending on your preferences
[Test]
public void Some_test_name_goes_here()
{
UITest(() =>
{
// Do your test steps here, including asserts etc.
// Any exceptions will be caught by the base class
// and screenshots will be taken
});
}
[TearDown]
public void TearDown()
{
// Close and dispose the driver
}
}
In C# I use NUnit 3.4. This offeres the OneTimeTearDown method that is able to access the TestContext including the state of the previous executed test. Do not use TearDown because it is not executed after a test fails ;)
using OpenQA.Selenium;
using System.Drawing.Imaging;
...
[OneTimeTearDown]
public void OneTimeTearDown()
{
if (TestContext.CurrentContext.Result.Outcome != ResultState.Success)
{
var screenshot = ((ITakesScreenshot)driver).GetScreenshot();
screenshot.SaveAsFile(#"C:\TEMP\Screenshot.jpg", ImageFormat.Jpeg);
}
}
For greater justice here is the code for the MSTest
public TestContext TestContext { get; set; }
[TestCleanup]
public void TestCleanup()
{
if (TestContext.CurrentTestOutcome == UnitTestOutcome.Failed)
{
var screenshotPath = $"{DateTime.Now:yyyy-MM-dd_HH-mm-ss.fffff}.png";
MyDriverInstance.TakeScreenshot().SaveAsFile(screenshotPath);
TestContext.AddResultFile(screenshotPath);
}
}
YOu can achieve this easily in TestNG suite FIle
Create a ScreenShot method like Below
public static void CaptureDesktop (String imgpath)
{
try
{
Robot robot = new Robot();
Dimension screensize=Toolkit.getDefaultToolkit().getScreenSize();
Rectangle screenRect = new Rectangle(screensize);
BufferedImage screenshot = robot.createScreenCapture(screenRect);
//RenderedImage screenshot = robot.createScreenCapture(screenRect);
ImageIO.write(screenshot, "png" , new File(imgpath));
}
In above method i used robot class so that you can take screen shot of Dekstop also(window+WebPage) and you can call this method in different Listener class which will implements ITestListener Interface. call your screen Shot method in OntestFailure() of that Listener Class
#Override
public void onTestFailure(ITestResult arg0) {
String methodname = arg0.getMethod().getMethodName();
String imgpath = "./Screenshot/"+methodname+".jpg";
Guru99TakeScreenshot.CaptureDesktop(imgpath);
}
This code is working for me. But this code is written in JAVA. I hope this will work in C# if not i wish this code can help you
Customizing a bit of ExtentReport can give extremely useful report having exception+screenshot captured exactly at time of test failure . Screenshot can be placed alongside exception which user can use to know what was website doing when error occurred.
Report Example
Test
#Test (enabled=true)
public void verifySearch() {
extentlogger = extent.createTest("To verify verifySearch");
//Your other code here.....
soft.assertEquals("xxx", "xxxx");
soft.assertAll();
}
AfterMethod
#AfterMethod
public void getResult(ITestResult result) throws Exception{
if(result.getStatus() == ITestResult.FAILURE)
{
extentlogger.log(Status.FAIL, MarkupHelper.createLabel(result.getThrowable() + " - Test Case Failed", ExtentColor.RED));
try {
// get path of captured screenshot using custom failedTCTakeScreenshot method
String screenshotPath = failedTCTakeScreenshot( result);
extentlogger.fail("Test Case Failed Snapshot is below " + extentlogger.addScreenCaptureFromPath(screenshotPath));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Extension to Andrey Stukalin solution.
If you are using SpecFlow 3.0,
if(TestContext.CurrentTestOutcome == UnitTestOutcome.Failed)
doesn't work anymore.
Have to use
if (scenarioContext.TestError != null)
Related
I have a constructor to wrap around all the test cases to take a screenshot upon failure of the assert. But the screenshot always gives me screenshot-1.png and when the second test is run it overrides the name with that screenshot. won't be able to differentiate screenshot
Code snippet:
public void UITest(Action action)
{
try
{
action();
}
catch (Exception ex)
{
var Screenshot = App.Screenshot($"Assert Error"+ GetType().Name);
throw;
}
}
Test
public void MyMethod(){
UITest(() =>
{
//logic
//Assert
});
}
So whenever this is failed test is taking a screenshot but with same file name. how to give a file name as the current test method name?
You should use
Xamarin.UITest.IApp.ScreenShot(string screenshotname);
as mentioned here in the documentation
To generate different filename each time append randomly generated.
I have a logger that I am adding to my project. Right now for every method I'm going to have to write Logger.DebugLog("Starting Method") at the start of each method and Logger.DebugLog("Completed Method")
this logger - when Debug enabled - allows me to track exactly what methods were called on this run so that if there is an issue I can see how far it got before breaking making it easy to debug. Assume that the method name and line are being captured - my goal here is I do not want to add those two lines on every one of the +100 public or private methods
namespace myProject
{
public class myClass
{
public bool MyPublicMethod(string Message = "someRandomMessage")
{
try
{
myPrivateMethod(1);
writeToLog(Message);
return true;
}
catch(){
return false;
}
}
private bool myPrivateMethod(int passedNumber)
{
try
{
writeToLog(passedNumber);
return true;
}
catch(){
return false;
}
}
}
}
my log file should look like this:
04:00:00 - Starting Method:MyPublicMethod
04:00:00 - Starting Method:myPrivateMethod
04:00:01 - 1
04:00:01 - Completed Method:myPrivateMethod
04:00:02 - someRandomMessage
04:00:02 - Completed Method:MyPublicMethod
What I am forced to do now which looks cluttered is:
namespace myProject
{
public class myClass
{
public bool MyPublicMethod(string Message = "someRandomMessage")
{
try
{
writeToLog("Starting Method");
myPrivateMethod(1);
writeToLog(Message);
writeToLog("Completed Method");
return true;
}
catch(){
return false;
}
}
private bool myPrivateMethod(int passedNumber)
{
try
{
writeToLog("Starting Method");
writeToLog(passedNumber);
writeToLog("Completed Method");
return true;
}
catch(){
return false;
}
}
}
}
is this even possible in .NET or do I have to explicitly list that logging method if I want to use it?
4/6/18 Edit: It is possible - see AOP. Heres a pretty decent article on it http://www.dotnetcurry.com/patterns-practices/1305/aspect-oriented-programming-aop-csharp-using-solid
Here is a good summary of exactly what I was looking for:
Consider the following code:
public class DocumentSource : IDocumentSource
{
//..
public Document[] GetDocuments(string format)
{
try
{
using (var context = CreateEFContext())
{
var documents =
context
.Documents
.Where(c => c.Name.EndsWith("." + format))
.ToArray();
logger.LogSuccess(
"Obtained " + documents.Length + " documents of type " + format +
Environment.NewLine +
"Connection String: " + connectionString);
return documents;
}
}
catch (Exception ex)
{
logger.LogError(
"Error obtaining documents of type " + format +
Environment.NewLine +
"Connection String: " + connectionString, ex);
throw;
}
}
//..
}
Here is how the method would look like without logging:
public Document[] GetDocuments(string format)
{
using (var context = CreateEFContext())
{
return
context
.Documents
.Where(c => c.Name.EndsWith("." + format))
.ToArray();
}
}
Clearly, the logging code has made the original method less readable. It has tangled the real method code with logging code.
This is also a violation of the Single Responsibility Principle.
Also, we expect to find the same logging pattern in many methods all over the code base. Basically, we expect to find the following pattern:
try
{
//Do something here
logger.LogSuccess(…
//..
}
catch (Exception ex)
{
logger.LogError(…
throw;
}
There are some Fody Add-ins that will allow you to add this kind of code into your compiled output at compile-time rather than having to write it yourself.
For example, MethodDecorator allows you to define a specific attribute, and any method you decorate with that attribute will call specific methods prior to entering and leaving the method.
I should note that for a project of any reasonable size, logging the entry and exit for every single method is going to produce more log messages than anyone can reasonably expect to read. I'd suggest you be judicious in which methods add log messages, and what information you include in those log messages.
The vast majority of the time a more useful strategy is to use guard statements to test your assumptions along the way, throwing an exception the moment anything is out of place, and then wrapping exceptions with more useful information (via InnerException) as they go up the call chain, then finally logging the results of those exceptions at the top level of your application. That way, you only produce log messages when something appears to go in a way you don't expect, and the log message in that case has all the information you're likely to need.
I'm not sure this is exactly what you're looking for, but it might point you in the right direction. For demonstration purposes I'm logging to the Console, but you could log to a file instead.
You could create a method that takes in another method to execute, which executes the function and returns the value, and which wraps the beginning and ending of the method call with writes to your log file:
// For methods that return a value
private static TResult LogMethod<TResult>(string displayName, Func<TResult> method)
{
Console.WriteLine($"{DateTime.Now} - Starting method: {displayName}");
TResult result = method();
Console.WriteLine($"{DateTime.Now} - Completed method: {displayName}");
return result;
}
// For void methods
private static void LogMethod(string displayName, Action method)
{
Console.WriteLine($"{DateTime.Now} - Starting method: {displayName}");
method();
Console.WriteLine($"{DateTime.Now} - Completed method: {displayName}");
}
As an example of how to use this, let's say we have the following methods:
private static int GetNumberFromUser(string prompt)
{
int result;
do
{
Console.Write(prompt);
} while (!int.TryParse(Console.ReadLine(), out result));
return result;
}
private static int AddNumbers(int first, int second)
{
return first + second;
}
private static void Tell(string message)
{
Console.WriteLine(message);
}
Now, instead of calling these methods directly, we can call them through our LogMethod function:
private static void Main()
{
var firstNumber = LogMethod("GetNumber",
() => GetNumberFromUser("Enter first number: "));
var secondNumber = LogMethod("GetNumber",
() => GetNumberFromUser("Enter second number: "));
var result = LogMethod("AddNumber",
() => AddNumbers(firstNumber, secondNumber));
LogMethod("Tell", () => Tell($"{firstNumber} + {secondNumber} = {result}"));
GetKeyFromUser("\nDone!! Press any key to exit...");
}
Output
I am writing an automated test suite for my program, and have been looking for ways to simplify and re factor a lot of the code.
Currently, I have a number of try/catch blocks (because I want to be able to move from one test to the next, even in the case of failure) that can log information otherwise. It looks a bit like this:
try
{
PerformTest1();
}
catch(Exception e)
{
file.WriteLine("Unable to perform test number 1.");
file.WriteLine("Error: " + e.Message);
file.WriteLine("StackTrace: " + e.StackTrace);
}
try
{
PerformTest2();
}
catch(Exception e)
{
file.WriteLine("Unable to perform test number 2.");
file.WriteLine("Error: " + e.Message);
file.WriteLine("StackTrace: " + e.StackTrace);
}
And so on. I was wondering if I could make an array of these tests, so I could use a loop. Something like:
foreach(Test t in testsArray)
{
try
{
t.RunTest();
}
catch(Exception e)
{
file.WriteLine(t.failDescription);
file.WriteLine("Error: " + e.Message);
file.WriteLine("StackTrace: " + e.StackTrace);
}
}
How can I execute this, but without making each test it's own class (with a RunTest() method)? I was thinking about making one test class, that has the necessary fields, one of them being a method. Then I could create a test object, and call that objects method. Is such a thing even possible? Or would I have to make multiple test classes?
You would need a Test interface with a runTest() method, but you can implement the logic with anonymous classes or lambda expressions (if you are using Java 8), which would save you the need to create an explicit class for each test.
Example :
Test test1 = new Test () {
public void runTest () {
// logic of first test here
}
};
Actually, you don't need to create your own interface. Use the existing Runnable interface with its run() method.
Runnable test1 = new Runnable () {
public void run () {
// logic of first test here
}
};
or with a lambda expression :
Runnable test1 = () -> {
// logic of first test here
};
Then you can add each of these Runnables to a list (or array) and run them in a loop, as you wanted :
for (Runnable test in testsArray)
{
try {
test.run();
}
catch(Exception e)
{
...
}
}
You could use some Lambda tricks:
var actions = new List<Action>();
actions.Add(PerformTest1); // requires a void PerformTest1() method
actions.Add(PerformTest2);
//... and so on...
and then finally fire all actions:
foreach(var action in actions)
{
action();
}
Have you looked into the available testing frameworks for .Net/ Java? I believe Nunit (or JUnit) provides much of the functionality you are looking for, including writing out the stack trace and exception details in the event of a failure.
The also encapsulate multiple tests within a test fixture, so will allow you to still execute subsequent tests after a failure without bleeding data or state between your individual tests.
public interface Testable {
void doTest();
}
public class Test1 implements Testable {
// implement doTest()
}
private static Testable[] testables = new Testabable[] {
new Test1()
};
// now iterate your testables and call doTest()
Ok, seems to be old school, but it's the first thing, one should learn about interfaces.
In the selenium IDE, there is a verify command. When I exported the command into c# I found that verify is basically an assert in a try catch statement and the error is added into a string.
In my code, I want to use the functionality of the verify command but I do not want to use a try and catch statement for every single assert.
Does anyone have a way to do this?
edit:
public static void AssertVerticalAlignment(CustomWebDriver webDriver, IElement elementA, IElement elementB, double tolerance = 0)
{
try
{
Assert.AreEqual(elementA.Location.X, elementB.Location.X, tolerance);
}
catch (Exception exception)
{
LogHelper.LogException(exception.Message, exception.StackTrace, webDriver.GetScreenshot());
throw;
}
}
What I want to do is add a message in the assert. It should say nameOfElementA is not aligned with nameOfElementB. But I don't want to give elementA and elementB a name property.
This is how I call the method: AssertVerticalAlignment(webdriver, homepage.NameInput, homepage.AgeInput)
Homepage is an object and NameInput is part of Homepage. NameInput is of type IElement, which is basically same as IWebElement, but it cannot interact with html, ie. it doesn't the ability to click, etc.
So I want the message to say NameInput is not aligned with AgeInput
You are essentially asking for a way to do "soft assertions". The way the IDE does it is correct. After all, that's what "soft assertions" are. If something fails a particular assertion, you want it to carry on regardless. That is what the IDE is doing, by catching that exception (notice that in it's code it's only catching the AssertionException).
In order to help avoid messy code, the best you can do is create your own verify methods. Sometimes you don't even need to catch exceptions. For instance take this basic verifyElementIsPresent method:
private class SoftVerifier
{
private StringBuilder verificationErrors;
public SoftVerifier()
{
verificationErrors = new StringBuilder();
}
public void VerifyElementIsPresent(IWebElement element)
{
try
{
Assert.IsTrue(element.Displayed);
}
catch (AssertionException)
{
verificationErrors.Append("Element was not displayed");
}
}
}
Why do you need the exception at all?
private class SoftVerifier
{
private StringBuilder verificationErrors;
public SoftVerifier()
{
verificationErrors = new StringBuilder();
}
public void VerifyElementIsPresent(IWebElement element)
{
if (!element.Displayed)
{
verificationErrors.Append("Element was not displayed");
}
}
}
The sort answer is there are some ways you can make it a little less messy, but overall, no, there isn't much you can do about it.
I would like to have my TearDown method check whether the previous test was a success before it applies some logic. Is there an easy way to do this?
This has been already solved in Ran's answer to similar SO question. Quoting Ran:
Since version 2.5.7, NUnit allows Teardown to detect if last test failed.
A new TestContext class allows tests to access information about themselves including the TestStauts.
For more details, please refer to http://nunit.org/?p=releaseNotes&r=2.5.7
[TearDown]
public void TearDown()
{
if (TestContext.CurrentContext.Result.Status == TestStatus.Failed)
{
PerformCleanUpFromTest();
}
}
If you want to use TearDown to detect status of last test with NUnit 3.5 it should be:
[TearDown]
public void TearDown()
{
if (TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Failed)
{
//your code
}
}
sounds like a dangerous idea unless it's an integration test, with say data to remove say. Why not do it in the test itself?
Obviously a private flag in the class could be set.
This is what Charlie Poole himself has suggested if you must
Only if you do this manually. In fact you even won't know which tests are intend to run. In NUnit IDE one can enable some tests and disable some other. If you want to know if some specific test has run you could include code like this in your test class:
enum TestStateEnum { DISABLED, FAILED, SUCCEDED };
TestStateEnum test1State = TestStateEnum.DISABLED;
[Test]
void Test1()
{
test1State = TestStateEnum.FAILED; // On the beginning of your test
...
test1State = TestStateEnum.SUCCEDED; // On the End of your Test
}
Then you can check the test1State variable. If the test throws an exception it won't set the SUCCEDED. you can also put this in a try catch finally block in your tests with a slightly different logic:
[Test]
void Test1()
{
test1State = TestStateEnum.SUCCEDED; // On the beginning of your test
try
{
... // Your Test
}
catch( Exception )
{
test1State = TestStateEnum.FAILED;
throw; // Rethrows the Exception
}
}
[OneTimeTearDown]
public void AfterEachTest()
{
if (TestContext.CurrentContext.Result.Outcome.Status.Equals(TestStatus.Failed))
{
Console.WriteLine("FAILS");
}
else if (TestContext.CurrentContext.Result.Outcome.Equals(ResultState.Success))
{
Console.WriteLine("SUCESS");
}
}
IMHO tear down logic should be independent of test results.
Ideally you should avoid using setup and teardown completely, a al xunit.net. See here for more info.