I have a constructor to wrap around all the test cases to take a screenshot upon failure of the assert. But the screenshot always gives me screenshot-1.png and when the second test is run it overrides the name with that screenshot. won't be able to differentiate screenshot
Code snippet:
public void UITest(Action action)
{
try
{
action();
}
catch (Exception ex)
{
var Screenshot = App.Screenshot($"Assert Error"+ GetType().Name);
throw;
}
}
Test
public void MyMethod(){
UITest(() =>
{
//logic
//Assert
});
}
So whenever this is failed test is taking a screenshot but with same file name. how to give a file name as the current test method name?
You should use
Xamarin.UITest.IApp.ScreenShot(string screenshotname);
as mentioned here in the documentation
To generate different filename each time append randomly generated.
Related
I have a logger that I am adding to my project. Right now for every method I'm going to have to write Logger.DebugLog("Starting Method") at the start of each method and Logger.DebugLog("Completed Method")
this logger - when Debug enabled - allows me to track exactly what methods were called on this run so that if there is an issue I can see how far it got before breaking making it easy to debug. Assume that the method name and line are being captured - my goal here is I do not want to add those two lines on every one of the +100 public or private methods
namespace myProject
{
public class myClass
{
public bool MyPublicMethod(string Message = "someRandomMessage")
{
try
{
myPrivateMethod(1);
writeToLog(Message);
return true;
}
catch(){
return false;
}
}
private bool myPrivateMethod(int passedNumber)
{
try
{
writeToLog(passedNumber);
return true;
}
catch(){
return false;
}
}
}
}
my log file should look like this:
04:00:00 - Starting Method:MyPublicMethod
04:00:00 - Starting Method:myPrivateMethod
04:00:01 - 1
04:00:01 - Completed Method:myPrivateMethod
04:00:02 - someRandomMessage
04:00:02 - Completed Method:MyPublicMethod
What I am forced to do now which looks cluttered is:
namespace myProject
{
public class myClass
{
public bool MyPublicMethod(string Message = "someRandomMessage")
{
try
{
writeToLog("Starting Method");
myPrivateMethod(1);
writeToLog(Message);
writeToLog("Completed Method");
return true;
}
catch(){
return false;
}
}
private bool myPrivateMethod(int passedNumber)
{
try
{
writeToLog("Starting Method");
writeToLog(passedNumber);
writeToLog("Completed Method");
return true;
}
catch(){
return false;
}
}
}
}
is this even possible in .NET or do I have to explicitly list that logging method if I want to use it?
4/6/18 Edit: It is possible - see AOP. Heres a pretty decent article on it http://www.dotnetcurry.com/patterns-practices/1305/aspect-oriented-programming-aop-csharp-using-solid
Here is a good summary of exactly what I was looking for:
Consider the following code:
public class DocumentSource : IDocumentSource
{
//..
public Document[] GetDocuments(string format)
{
try
{
using (var context = CreateEFContext())
{
var documents =
context
.Documents
.Where(c => c.Name.EndsWith("." + format))
.ToArray();
logger.LogSuccess(
"Obtained " + documents.Length + " documents of type " + format +
Environment.NewLine +
"Connection String: " + connectionString);
return documents;
}
}
catch (Exception ex)
{
logger.LogError(
"Error obtaining documents of type " + format +
Environment.NewLine +
"Connection String: " + connectionString, ex);
throw;
}
}
//..
}
Here is how the method would look like without logging:
public Document[] GetDocuments(string format)
{
using (var context = CreateEFContext())
{
return
context
.Documents
.Where(c => c.Name.EndsWith("." + format))
.ToArray();
}
}
Clearly, the logging code has made the original method less readable. It has tangled the real method code with logging code.
This is also a violation of the Single Responsibility Principle.
Also, we expect to find the same logging pattern in many methods all over the code base. Basically, we expect to find the following pattern:
try
{
//Do something here
logger.LogSuccess(…
//..
}
catch (Exception ex)
{
logger.LogError(…
throw;
}
There are some Fody Add-ins that will allow you to add this kind of code into your compiled output at compile-time rather than having to write it yourself.
For example, MethodDecorator allows you to define a specific attribute, and any method you decorate with that attribute will call specific methods prior to entering and leaving the method.
I should note that for a project of any reasonable size, logging the entry and exit for every single method is going to produce more log messages than anyone can reasonably expect to read. I'd suggest you be judicious in which methods add log messages, and what information you include in those log messages.
The vast majority of the time a more useful strategy is to use guard statements to test your assumptions along the way, throwing an exception the moment anything is out of place, and then wrapping exceptions with more useful information (via InnerException) as they go up the call chain, then finally logging the results of those exceptions at the top level of your application. That way, you only produce log messages when something appears to go in a way you don't expect, and the log message in that case has all the information you're likely to need.
I'm not sure this is exactly what you're looking for, but it might point you in the right direction. For demonstration purposes I'm logging to the Console, but you could log to a file instead.
You could create a method that takes in another method to execute, which executes the function and returns the value, and which wraps the beginning and ending of the method call with writes to your log file:
// For methods that return a value
private static TResult LogMethod<TResult>(string displayName, Func<TResult> method)
{
Console.WriteLine($"{DateTime.Now} - Starting method: {displayName}");
TResult result = method();
Console.WriteLine($"{DateTime.Now} - Completed method: {displayName}");
return result;
}
// For void methods
private static void LogMethod(string displayName, Action method)
{
Console.WriteLine($"{DateTime.Now} - Starting method: {displayName}");
method();
Console.WriteLine($"{DateTime.Now} - Completed method: {displayName}");
}
As an example of how to use this, let's say we have the following methods:
private static int GetNumberFromUser(string prompt)
{
int result;
do
{
Console.Write(prompt);
} while (!int.TryParse(Console.ReadLine(), out result));
return result;
}
private static int AddNumbers(int first, int second)
{
return first + second;
}
private static void Tell(string message)
{
Console.WriteLine(message);
}
Now, instead of calling these methods directly, we can call them through our LogMethod function:
private static void Main()
{
var firstNumber = LogMethod("GetNumber",
() => GetNumberFromUser("Enter first number: "));
var secondNumber = LogMethod("GetNumber",
() => GetNumberFromUser("Enter second number: "));
var result = LogMethod("AddNumber",
() => AddNumbers(firstNumber, secondNumber));
LogMethod("Tell", () => Tell($"{firstNumber} + {secondNumber} = {result}"));
GetKeyFromUser("\nDone!! Press any key to exit...");
}
Output
Does any of you know possible solution for taking screenshots on test failures and exceptions?
I've added following code in TearDown() but as a result it also makes screenshots on passed tests, so it is not the best solution:
DateTime time = DateTime.Now;
string dateToday = "_date_" + time.ToString("yyyy-MM-dd") + "_time_" + time.ToString("HH-mm-ss");
Screenshot screenshot = ((ITakesScreenshot)driver).GetScreenshot();
screenshot.SaveAsFile((settings.filePathForScreenShots + "Exception" + dateToday + ".png"), System.Drawing.Imaging.ImageFormat.Png);
I've already found that idea: http://yizeng.me/2014/02/08/take-a-screenshot-on-exception-with-selenium-csharp-eventfiringwebdriver/, to use WebDriverExceptionEventArgs, but for some reasons it makes also some random screenshots without any reasonable explanation.
Other ideas I found are for Java and not for NUnit which I use with Selenium, so they are pretty useless.
If you put the screenshot logic in your TearDown method it will be called after each test finishes, no matter if it succeeded or failed.
I use a base class that has a function which wraps the tests and catches all exceptions. When a test fails the exception is caught and a screenshot is taken.
I use this base class for all my Selenium tests and it looks something like this:
public class PageTestBase
{
protected IWebDriver Driver;
protected void UITest(Action action)
{
try
{
action();
}
catch (Exception ex)
{
var screenshot = Driver.TakeScreenshot();
var filePath = "<some appropriate file path goes here>";
screenshot.SaveAsFile(filePath, ImageFormat.Png);
// This would be a good place to log the exception message and
// save together with the screenshot
throw;
}
}
}
The test classes then look like this:
[TestFixture]
public class FooBarTests : PageTestBase
{
// Make sure to initialize the driver in the constructor or SetUp method,
// depending on your preferences
[Test]
public void Some_test_name_goes_here()
{
UITest(() =>
{
// Do your test steps here, including asserts etc.
// Any exceptions will be caught by the base class
// and screenshots will be taken
});
}
[TearDown]
public void TearDown()
{
// Close and dispose the driver
}
}
In C# I use NUnit 3.4. This offeres the OneTimeTearDown method that is able to access the TestContext including the state of the previous executed test. Do not use TearDown because it is not executed after a test fails ;)
using OpenQA.Selenium;
using System.Drawing.Imaging;
...
[OneTimeTearDown]
public void OneTimeTearDown()
{
if (TestContext.CurrentContext.Result.Outcome != ResultState.Success)
{
var screenshot = ((ITakesScreenshot)driver).GetScreenshot();
screenshot.SaveAsFile(#"C:\TEMP\Screenshot.jpg", ImageFormat.Jpeg);
}
}
For greater justice here is the code for the MSTest
public TestContext TestContext { get; set; }
[TestCleanup]
public void TestCleanup()
{
if (TestContext.CurrentTestOutcome == UnitTestOutcome.Failed)
{
var screenshotPath = $"{DateTime.Now:yyyy-MM-dd_HH-mm-ss.fffff}.png";
MyDriverInstance.TakeScreenshot().SaveAsFile(screenshotPath);
TestContext.AddResultFile(screenshotPath);
}
}
YOu can achieve this easily in TestNG suite FIle
Create a ScreenShot method like Below
public static void CaptureDesktop (String imgpath)
{
try
{
Robot robot = new Robot();
Dimension screensize=Toolkit.getDefaultToolkit().getScreenSize();
Rectangle screenRect = new Rectangle(screensize);
BufferedImage screenshot = robot.createScreenCapture(screenRect);
//RenderedImage screenshot = robot.createScreenCapture(screenRect);
ImageIO.write(screenshot, "png" , new File(imgpath));
}
In above method i used robot class so that you can take screen shot of Dekstop also(window+WebPage) and you can call this method in different Listener class which will implements ITestListener Interface. call your screen Shot method in OntestFailure() of that Listener Class
#Override
public void onTestFailure(ITestResult arg0) {
String methodname = arg0.getMethod().getMethodName();
String imgpath = "./Screenshot/"+methodname+".jpg";
Guru99TakeScreenshot.CaptureDesktop(imgpath);
}
This code is working for me. But this code is written in JAVA. I hope this will work in C# if not i wish this code can help you
Customizing a bit of ExtentReport can give extremely useful report having exception+screenshot captured exactly at time of test failure . Screenshot can be placed alongside exception which user can use to know what was website doing when error occurred.
Report Example
Test
#Test (enabled=true)
public void verifySearch() {
extentlogger = extent.createTest("To verify verifySearch");
//Your other code here.....
soft.assertEquals("xxx", "xxxx");
soft.assertAll();
}
AfterMethod
#AfterMethod
public void getResult(ITestResult result) throws Exception{
if(result.getStatus() == ITestResult.FAILURE)
{
extentlogger.log(Status.FAIL, MarkupHelper.createLabel(result.getThrowable() + " - Test Case Failed", ExtentColor.RED));
try {
// get path of captured screenshot using custom failedTCTakeScreenshot method
String screenshotPath = failedTCTakeScreenshot( result);
extentlogger.fail("Test Case Failed Snapshot is below " + extentlogger.addScreenCaptureFromPath(screenshotPath));
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Extension to Andrey Stukalin solution.
If you are using SpecFlow 3.0,
if(TestContext.CurrentTestOutcome == UnitTestOutcome.Failed)
doesn't work anymore.
Have to use
if (scenarioContext.TestError != null)
In the selenium IDE, there is a verify command. When I exported the command into c# I found that verify is basically an assert in a try catch statement and the error is added into a string.
In my code, I want to use the functionality of the verify command but I do not want to use a try and catch statement for every single assert.
Does anyone have a way to do this?
edit:
public static void AssertVerticalAlignment(CustomWebDriver webDriver, IElement elementA, IElement elementB, double tolerance = 0)
{
try
{
Assert.AreEqual(elementA.Location.X, elementB.Location.X, tolerance);
}
catch (Exception exception)
{
LogHelper.LogException(exception.Message, exception.StackTrace, webDriver.GetScreenshot());
throw;
}
}
What I want to do is add a message in the assert. It should say nameOfElementA is not aligned with nameOfElementB. But I don't want to give elementA and elementB a name property.
This is how I call the method: AssertVerticalAlignment(webdriver, homepage.NameInput, homepage.AgeInput)
Homepage is an object and NameInput is part of Homepage. NameInput is of type IElement, which is basically same as IWebElement, but it cannot interact with html, ie. it doesn't the ability to click, etc.
So I want the message to say NameInput is not aligned with AgeInput
You are essentially asking for a way to do "soft assertions". The way the IDE does it is correct. After all, that's what "soft assertions" are. If something fails a particular assertion, you want it to carry on regardless. That is what the IDE is doing, by catching that exception (notice that in it's code it's only catching the AssertionException).
In order to help avoid messy code, the best you can do is create your own verify methods. Sometimes you don't even need to catch exceptions. For instance take this basic verifyElementIsPresent method:
private class SoftVerifier
{
private StringBuilder verificationErrors;
public SoftVerifier()
{
verificationErrors = new StringBuilder();
}
public void VerifyElementIsPresent(IWebElement element)
{
try
{
Assert.IsTrue(element.Displayed);
}
catch (AssertionException)
{
verificationErrors.Append("Element was not displayed");
}
}
}
Why do you need the exception at all?
private class SoftVerifier
{
private StringBuilder verificationErrors;
public SoftVerifier()
{
verificationErrors = new StringBuilder();
}
public void VerifyElementIsPresent(IWebElement element)
{
if (!element.Displayed)
{
verificationErrors.Append("Element was not displayed");
}
}
}
The sort answer is there are some ways you can make it a little less messy, but overall, no, there isn't much you can do about it.
I've a method test that does not have a clear assert expression. The returned tested value is a very long string that has to be inspected by a programmer to check if it is correct or not. For this reason, if the code executes without exceptions, I'm calling 'Assert.Inconclusive'.
However, if some kind of Exception is thrown, I want to call 'Assert.Fail' with the exception message. Something like this:
[TestMethod()]
public void Test()
{
try {
string toBeChecked = MethodToBeTested();
//the string is so particular that no clear
//assertion can be made about it.
Console.WriteLine(toBeChecked);
Assert.Inconclusive("Check the console output.");
} catch(Exception e) {
Assert.Fail(e.Message);
}
}
The problem with this code is that if no regular exception is thrown, the Assert.Inconclusive method also throws an exception that is catched, so Assert.Fail is called, and from the IDE test result pane it seems that the test has failed. This is not what I want.
Is there a way to filter the exceptions, such as catching every exception but the Assert-like ones?
(I'm using .NET framework 3.5SP1)
Why not just leave out the Assert.Inconclusive()? In fact, why catch any exceptions at all - if the code throws an exception the unit test framework will mark it as failed. Less is more:
[TestMethod()]
public void Test()
{
string toBeChecked = MethodToBeTested();
Console.WriteLine(toBeChecked);
}
But this is a poor unit test if we can not automatically check the result. All we are checking is that no exception is thrown.
Are there no asserts you can make about the resulting string?
For example, that is is not null or empty? Do we expect it to contain a certain sub-string that we can test?
At least give the test a helpful name, which includes something like: ManualAssertRequired
The Assert.Inconclusive method should throw a AssertInconclusiveException so you can either mark the Test as ExcpectedException(typeof(AssertInconclusiveExcpetion)) or use something like this:
[TestMethod()]
public void Test()
{
try {
string toBeChecked = MethodToBeTested();
//the string is so particular that no clear
//assertion can be made about it.
Console.WriteLine(toBeChecked);
Assert.Inconclusive("Check the console output.");
} catch(AsssertInconclusiveException) {
/* Do nothing */
}
} catch(Exception e) {
Assert.Fail(e.Message);
}
}
Try to catch specific exception type instead of Exception or add another catch for nunit exception that is caused by Assert.Inconclusivemethod which is AssertInconclusiveException...
For example modify it like this:
[TestMethod()]
public void Test()
{
try {
string toBeChecked = MethodToBeTested();
//the string is so particular that no clear
//assertion can be made about it.
Console.WriteLine(toBeChecked);
Assert.Inconclusive("Check the console output.");
} catch(AssertInconclusiveException e) {
// do nothing...
} catch(Exception e) {
Assert.Fail(e.Message);
}
}
is there a MsTest Equivalent of Assert.Warning in MbUnit ?
The closest match is Assert.Inconclusive() - it doesn't make the test fail as such, but it doesn't succeed either. It fall into a third stage called Inconclusive.
A single Inconclusive test will cause an entire test suite to be Inconclusive.
There are overloads that supports custom messages as well:
Assert.Inconclusive("Ploeh");
I have a similar issue as I use NUnit for some projects. Try using
Console.Write("Some Warning");
You may want to use a custom exception.
The trouble with Assert.Inconclusive is that Test Explorer states the test wasn't even run. This may be misleading when running the test in the future, particularly if the test is run by other developers:
The way I've come to prefer is as follows. Firstly, define a custom UnitTestWarningException. I've given mine an additional constructor so I can pass my warning message String.Format-style with arguments:
public class UnitTestWarningException : Exception
{
public UnitTestWarningException(string Message) : base(Message) { }
public UnitTestWarningException(string Format, params object[] Args) : base(string.Format(Format, Args)) { }
}
Then, at the point where you want to end a unit test with a warning, throw a UnitTestWarningException instead:
[TestMethod]
public void TestMethod1()
{
.
.
.
try
{
WorkflowInvoker.Invoke(workflow1, inputDictionary);
}
catch (SqlException ex)
{
if (ex.Errors.Count > 0
&& ex.Errors[0].Procedure == "proc_AVAILABLEPLACEMENTNOTIFICATIONInsert")
{
//Likely to occur if we try to repeat an insert during development/debugging.
//Probably not interested--the mail has already been sent if we got as far as that proc.
throw new UnitTestWarningException("Note: after sending the mail, proc_AVAILABLEPLACEMENTNOTIFICATIONInsert threw an exception. This may be expected depending on test conditions. The exception was: {0}", ex.Message);
}
}
}
The result: Test Explorer then shows that the test has been executed, but failed with a UnitTestWarningException that shows your warning:
Here is my hack on how to have warnings with nunit ( i know this question was about mstest, but this should work too). As always, I am interested in any improvements. This method is working for me.
Background: I have code which checks the tests themselves for correct comments and has logic to detect if someone has copied and pasted another test without changing comments. These are warnings I want to be shown to the developer without normal Assert.Inconclusive blocking the actual test from running. Some are focused on the test and the cleanup refactorings phase is to remove the warnings.
Mission: to have warnings after all other asserts are run. This means even showing the warnings after Assert.Fail that normally occur in tests during development.
Implementation: (best to create a base class for all test files):
public class BaseTestClass
{
public static StringBuilder Warnings;
[SetUp]
public virtual void Test_SetUp()
{
Warnings = new StringBuilder();
}
[TearDown]
public virtual void Test_TearDown()
{
if (Warnings.Length > 0)
{
string warningMessage = Warnings.ToString();
//-- cleared if there is more than one test running in the session
Warnings = new StringBuilder();
if (TestContext.CurrentContext.Result.Status == TestStatus.Failed)
{
Assert.Fail(warningMessage);
}
else
{
Assert.Inconclusive(warningMessage);
}
}
}
Testing Usage
[Test]
public void Sample_Test()
{
if (condition) Warning.AppendLine("Developer warning");
Assert.Fail("This Test Failed!");
}
Actual Result:
"This Test Failed!"
"Developer warning"
Status of test is failed - RED
If the test passed and there was a warning, you will then get the status of Inconclusive - YELLOW.