I am trying to add a testcase name as a folder name. I am adding it in Teardown method to detect first whether to testcase passed or fail then accordingly it adds Folder in Pass or Fail Folder which already exist.
[Test]
public void TestCase12345()
{
string Name = methodBase.Name;
teardown(Name);
}
[TearDown]
public void teardown(string testcase)
{
if (TestContext.CurrentContext.Result.Status == TestStatus.Passed)
{
string sourcepath = #"sourcepath";
string timestamp = DateTime.Now.ToString("yy-MM-dd hh-mm");
string destpath = (#"destinationlocation" + "Test Case - " + testcase + " " + timestamp);
...
}
Error:
Invalid signature for SetUp or TearDown method: teardown
What am I missing over here?
You can't pass parameters to [TearDown] method, NUnit doesn't support it. For example, to pass parameters to [Test] you do something like this
[Test]
[TestCase("abc", 123)]
public void TestCase12345(string str, int number)
{
}
But as I said, NUnit doesn't support it in [TearDown].
As a side note, the check if the test succeeded should be in the test method (I find Assert very useful for that purpose). TearDown should be used only for the "cleaning", i.e. dispose of the WebDriver and any other things you created for the test and doesn't close automatically when the code is finished.
Edit
"Then what is the solution. how can add function name which I am calling it to create folder?"
You can implement an EventListener interface.
EventListeners are able to respond to events that occur in the course of a test run, usually by recording information of some kind.
For example
public class TestEventListaener : EventListener
{
// The test collection started/finished to run.
void RunStarted(string name, int testCount);
void RunFinished(TestResult result );
void RunFinished(Exception exception );
void TestStarted(TestName testName)
{
// create folder with the test name
}
void TestFinished(TestResult result)
{
// if test succeeded insert data to the folder otherwise delete it
}
// more `EventListener` methods
}
In addition to KjetilNodin's answer I would try removing the test case parameter from your tear down function since it is most likely expecting a function without parameters to be used for TearDown.
If you need this functionality I'd remove the TearDown attribute from the function and just call it at the end of your test case as in your example.
Two separate issues here:
1= As others have said, NUnit doesn't allow a teardown method to take an argument.
2 = If you call your method from the test, then it isn't functioning as a TearDown method but as part of the test code. The test has not yet finished, so it has no final result. What's more, if you should have a failed assert, you'll never reach the call.
You should remove the argument and the call. Use TestContext to get the test name as suggested in one of the comments.
Related
In Moq, I want to assert that a method TestMethod is called with a specific parameter. When the test fails, I want to see a useful error message similar to:
TestMethod called with unexpected value X for parameter P where Y expected.
public interface ITestObject
{
void TestMethod(int parameter);
}
Just for sake of illustration, I could achieve this using a handcoded mock as follows:
Assert.AreEqual failed. Expected:<3>. Actual:<2>. TestMethod called with unexpected value for parameter 'actual'.
public class MockTestMethodParameter : ITestObject
{
private readonly int expected;
public MockTestMethodParameter(int expected) { this.expected = expected; }
public void TestMethod(int actual)
{
Assert.AreEqual(actual, expected, $"{nameof(TestMethod)} called with unexpected value for parameter '{nameof(actual)}'.");
}
}
[TestMethod]
public void TestHandcodedMockFailure()
{
var mock = new MockTestMethodParameter(expected: 2);
mock.TestMethod(actual: 3);
}
My problem is, I can't figure out how to do this in Moq. When I set up my Mock and call Verify(), I get the following unexpected and unclear error message, instead of the message I was hoping for:
Expected invocation on the mock at least once, but was never performed: test => test.TestMethod(It.Is(2, GenericEqualityComparer))
Performed invocations:
MockUnitTest1.ITestObject:1 (test):
UnitTest1.ITestObject.TestMethod(3)
[TestMethod]
public void TestMoqFailure()
{
var expected = 2;
var actual = 3;
var mock = new Moq.Mock<ITestObject>(Moq.MockBehavior.Strict);
mock.Setup(test => test.TestMethod(Moq.It.IsAny<int>())).Verifiable();
mock.Object.TestMethod(actual);
mock.Verify(test => test.TestMethod(Moq.It.Is(expected, EqualityComparer<int>.Default)));
}
Granted, the information is there, but I expected something more along the lines of, "TestMethod was invoked, but with incorrect parameters." It confuses me when Moq reports that TestMethod was not invoked because, intuitively, I did invoke the mock. I called TestMethod with It.IsAny(), as declared in the mock setup.
I've tried many different adjustments, but none yielding the desired result:
Custom error message
Setup with Is(3)
Setup with Is(2)
MockBehavior.Loose
Different .NET platforms, Framework 4.6.1, Core 2.1, .NET 6.0
Is this simply the way Moq reports error results for Verify()? I'm new to Moq, and while this behavior is unexpected to me, perhaps it is working as designed.
This appears to be normal behavior for mock frameworks in general. Doing the same thing with Telerik JustMock yields a similar error message:
Occurrence expectation failed. Expected exactly 1 call. Calls so far: 0
[TestMethod]
public void TestJustMockFailure()
{
var expected = 2;
var actual = 3;
var mock = Mock.Create<ITestObject>(Behavior.Strict);
Mock.Arrange(() => mock.TestMethod(Arg.IsAny<int>()));
mock.TestMethod(actual);
Mock.Assert(() => mock.TestMethod(Arg.Is(expected)), Occurs.Once());
}
In summary:
The OP's use case (my use case) is valid, verifying a particular parameter value was passed to a mocked method. However, the error message does not report the parameter mismatch as such. Rather, it reports that the entire mocked method call did not occur. This is by design, and appears to be common among popular mock frameworks.
I'm applying NUnit integration tests on our controller endpoints in a .NET Web API 2 project whose models and controllers are generated via Entity code first from database.
I'm having trouble thinking of what parts of the controller I should test. In the end, we'd just like to be able to automate "can a user with "x" role get this data?"
Looking in the GET portion of this controller, what parts would you test and what's your reasoning?
namespace api.Controllers.myNamespace
{
public class myController : ApiController
{
private string strUserName;
private string strError = "";
private string strApiName = "myTable";
private myDatabase db = new myDatabase();
// ----------------------------------------------------------------------
// GET: api/path
public IQueryable<myTable> GetmyTable()
{
try
{
this.strUserName = this.getUserName();
if
(
// ----- authorize -----
db.view_jnc_role_api_permission.Count
(
view =>
(
view.permission == "get"
&& view.apiName == this.strApiName
&& view.userName == this.strUserName
)
) == 1
// ----- /authorize -----
)
{
// ----- get -----
IQueryable<myTable> data =
from tbl in db.myTable
where tbl.deleted == null
select tbl;
// ----- /get -----
return data;
}
else
{
strError = "Unauthorized.";
throw new HttpResponseException(HttpStatusCode.Forbidden);
}
}
catch (Exception ex)
{
if (strError.Length == 0)
{
if (this.showException())
{
strError = ex.ToString();
}
}
throw new HttpResponseException(ControllerContext.Request.CreateErrorResponse(HttpStatusCode.Forbidden, strError));
}
}
}
For reference, here's what I have so far. Some of these private fields I'm defining shouldn't be here - currently trying to get access to private methods from my test project via AssemblyInfo.cs to fix this:
namespace api.myNamespace
{
[TestFixture]
public class myController : ApiController
{
private string strUserName;
private string strError = "";
private string strApiName = "myTable";
private myDb db = new myDb();
// Using TransactionScope to (hopefully) prevent integration test's changes to database from persisting
protected TransactionScope TransactionScope;
// Instantiate _controller field
private myController _controller;
[SetUp]
public void SetUp() {
TransactionScope = new TransactionScope(TransactionScopeOption.RequiresNew);
// It's possible that one test may leave some state which could impact subsequent tests - so we must reinstantiate _controller at the start of each new test:
_controller = new myController();
}
[TearDown]
public void TearDown()
{
TransactionScope.Dispose();
}
**//------ TESTS -------//
// CanSetAndGetUserName
// AuthorizedUserCanGetData
// UnauthorizedUserCannotGetData
// AuthorizedUserCanPutData
// UnauthorizedUserCannotPutData
// AuthorizedUserCanPostData
// UnauthorizedUserCannotPostData
// AuthorizedUserCanDeleteData
// UnauthorizedUserCannotDeleteData**
[Test]
public void CanGetAndSetUsername()
{
// ARRANGE
var user = _controller.getUserName();
// ACT
// ASSERT
Assert.That(user, Is.EqualTo("my-internal-username"));
}
[Test]
public void UnauthorizedUserCannotGetData()
{
var user = "Mr Unauthorized";
// Unfinished bc integration testing is super abstract, subjective, hard, time consuming and hard. All downvoters are plebs.
Assert.That(user, Is.EqualTo());
}
}
}
}
integration tests means several things:
you setup your test data in the database, via a script for example.
you call the endpoint under test knowing exactly what data you should call it with and what you should get. This is all based on your test data you setup in step 1.
you compare your expected data with the one you got back.
this is an integration test as it touches everything, both api and database.
Now, you said you are having trouble deciding which parts of the controller to test. This suggests you are confusing integration tests with unit tests.
Integration tests we already covered.
Unit tests cover parts of functionality. You do not test controllers, forget about that.
What you really need to consider doing is this:
First, separate your code from the controller. Keep the controller very basic. It receives a call, validates the request model and passes it further to a class library where the functionality happens. This allows you to forget "testing the controller" and focus on your functionality instead. Unit tests will help here and your test cases will become something like this
I have a user, set up in a certain way.
I have some data, set up in a certain way
When I call method X, then I should get this response.
With such a setup in place, you can set your test data any way you like and check every single test case.
The only reason you wonder how you test your controller is because you dumped all your code into it, which of course makes everything hard. Think SOLID, think SOC ( Separation of concerns ).
One piece of advice: never ever return IQueryable from an endpoint, that's not data, that simply a query that hasn't run yet. Return a List, IEnumerable, an singular object, whatever you need, just make sure you execute that first by calling ToList() for example on your IQueryable expression first.
So, the steps are like this:
Setup your IQueryable first
Execute it by calling ToList(), First(), FirstOrDefault() whatever is appropriate and return the result of that.
I'm curious to know if there's any built-in mechanism to retry tests in the Visual Studio 2008 unit testing framework for C#.
Case in point, I have a C# unit test which looks something like:
[TestMethod]
public void MyMethod() {
DoSomething();
Assert.Something();
}
Now, occasionally DoSomething() performs badly; in that case I would like to rerun the DoSomething() method before reaching the assertion. Obviously I can do something like:
...
do {
Initialization();
DoSomething();
} while (PerformedOK() == false);
Assert.Something();
...
Though this is a bit cumbersome because of the added loop and repeating the test initialization which would otherwise be completely handled by other methods / class constructor.
My question is whether there is a more convenient mechanism for retrying a test, something like:
DoSomething();
if (PerformedOK() == false) Retry();
else Assert.Something();
which will automatically retry the test without registering it as a failure, while performing all the regular initialization code as usual.
Seriously...
occasionally DoSomething() performs
badly
A test should be green every time. If the tested code sometimes perform "badly", then you need to fix your code, isolating the different behavior. You should have two test, one where it Asserts correct when DoSomething fails (and is supposed to fail), and one where it Asserts correct when DoSomething is ok (and is supposed to be ok).
Having retry logic in a test is just wrong imo. You should always Assert on the expected outcome, and you should be able to isolate and instrument your code to return what you expect.
[Edit - added some code which could be used for a retry loop]
You could create a loop wrapper which takes whatever method in and calls it X number of times, or until it succeeds. You could also have the Loop function call your init, or pass it as a separate argument. The method could also return a bool if successful. Change the signature to fit your needs.
[TestMethod]
public void something()
{
Loop.LoopMe(TestMethod,3);
Assert.Something();
}
class Loop
{
public static void LoopMe(Action action, int maxRetry)
{
Exception lastException = null;
while (maxRetry > 0)
{
try
{
action();
return;
}
catch (Exception e)
{
lastException = e;
maxRetry--;
}
}
throw lastException;
}
}
Your second example is almost the same lines of code and same complexity. There are tons of ways to skin it, you could not that I am advocating it use recursion.
[TestMethod]
public void MyMethod() {
bool success = DoSomething();
Assert.IsTrue(success);
}
public boolean DoSomething(){
//Do whatever
if(performedOk){
return true;
}else{
//find a way to stop it.
}
}
But the point is it is a unit test. If something is causing it to go wrong, you need to find a way isolate your test, so that it is in a controlled environment.
Unless you have a requirement that says, test should pass eventually. The best retry logic you should use, is after it fails. Click the test and hit run again.
is there a MsTest Equivalent of Assert.Warning in MbUnit ?
The closest match is Assert.Inconclusive() - it doesn't make the test fail as such, but it doesn't succeed either. It fall into a third stage called Inconclusive.
A single Inconclusive test will cause an entire test suite to be Inconclusive.
There are overloads that supports custom messages as well:
Assert.Inconclusive("Ploeh");
I have a similar issue as I use NUnit for some projects. Try using
Console.Write("Some Warning");
You may want to use a custom exception.
The trouble with Assert.Inconclusive is that Test Explorer states the test wasn't even run. This may be misleading when running the test in the future, particularly if the test is run by other developers:
The way I've come to prefer is as follows. Firstly, define a custom UnitTestWarningException. I've given mine an additional constructor so I can pass my warning message String.Format-style with arguments:
public class UnitTestWarningException : Exception
{
public UnitTestWarningException(string Message) : base(Message) { }
public UnitTestWarningException(string Format, params object[] Args) : base(string.Format(Format, Args)) { }
}
Then, at the point where you want to end a unit test with a warning, throw a UnitTestWarningException instead:
[TestMethod]
public void TestMethod1()
{
.
.
.
try
{
WorkflowInvoker.Invoke(workflow1, inputDictionary);
}
catch (SqlException ex)
{
if (ex.Errors.Count > 0
&& ex.Errors[0].Procedure == "proc_AVAILABLEPLACEMENTNOTIFICATIONInsert")
{
//Likely to occur if we try to repeat an insert during development/debugging.
//Probably not interested--the mail has already been sent if we got as far as that proc.
throw new UnitTestWarningException("Note: after sending the mail, proc_AVAILABLEPLACEMENTNOTIFICATIONInsert threw an exception. This may be expected depending on test conditions. The exception was: {0}", ex.Message);
}
}
}
The result: Test Explorer then shows that the test has been executed, but failed with a UnitTestWarningException that shows your warning:
Here is my hack on how to have warnings with nunit ( i know this question was about mstest, but this should work too). As always, I am interested in any improvements. This method is working for me.
Background: I have code which checks the tests themselves for correct comments and has logic to detect if someone has copied and pasted another test without changing comments. These are warnings I want to be shown to the developer without normal Assert.Inconclusive blocking the actual test from running. Some are focused on the test and the cleanup refactorings phase is to remove the warnings.
Mission: to have warnings after all other asserts are run. This means even showing the warnings after Assert.Fail that normally occur in tests during development.
Implementation: (best to create a base class for all test files):
public class BaseTestClass
{
public static StringBuilder Warnings;
[SetUp]
public virtual void Test_SetUp()
{
Warnings = new StringBuilder();
}
[TearDown]
public virtual void Test_TearDown()
{
if (Warnings.Length > 0)
{
string warningMessage = Warnings.ToString();
//-- cleared if there is more than one test running in the session
Warnings = new StringBuilder();
if (TestContext.CurrentContext.Result.Status == TestStatus.Failed)
{
Assert.Fail(warningMessage);
}
else
{
Assert.Inconclusive(warningMessage);
}
}
}
Testing Usage
[Test]
public void Sample_Test()
{
if (condition) Warning.AppendLine("Developer warning");
Assert.Fail("This Test Failed!");
}
Actual Result:
"This Test Failed!"
"Developer warning"
Status of test is failed - RED
If the test passed and there was a warning, you will then get the status of Inconclusive - YELLOW.
I would like to have my TearDown method check whether the previous test was a success before it applies some logic. Is there an easy way to do this?
This has been already solved in Ran's answer to similar SO question. Quoting Ran:
Since version 2.5.7, NUnit allows Teardown to detect if last test failed.
A new TestContext class allows tests to access information about themselves including the TestStauts.
For more details, please refer to http://nunit.org/?p=releaseNotes&r=2.5.7
[TearDown]
public void TearDown()
{
if (TestContext.CurrentContext.Result.Status == TestStatus.Failed)
{
PerformCleanUpFromTest();
}
}
If you want to use TearDown to detect status of last test with NUnit 3.5 it should be:
[TearDown]
public void TearDown()
{
if (TestContext.CurrentContext.Result.Outcome.Status == TestStatus.Failed)
{
//your code
}
}
sounds like a dangerous idea unless it's an integration test, with say data to remove say. Why not do it in the test itself?
Obviously a private flag in the class could be set.
This is what Charlie Poole himself has suggested if you must
Only if you do this manually. In fact you even won't know which tests are intend to run. In NUnit IDE one can enable some tests and disable some other. If you want to know if some specific test has run you could include code like this in your test class:
enum TestStateEnum { DISABLED, FAILED, SUCCEDED };
TestStateEnum test1State = TestStateEnum.DISABLED;
[Test]
void Test1()
{
test1State = TestStateEnum.FAILED; // On the beginning of your test
...
test1State = TestStateEnum.SUCCEDED; // On the End of your Test
}
Then you can check the test1State variable. If the test throws an exception it won't set the SUCCEDED. you can also put this in a try catch finally block in your tests with a slightly different logic:
[Test]
void Test1()
{
test1State = TestStateEnum.SUCCEDED; // On the beginning of your test
try
{
... // Your Test
}
catch( Exception )
{
test1State = TestStateEnum.FAILED;
throw; // Rethrows the Exception
}
}
[OneTimeTearDown]
public void AfterEachTest()
{
if (TestContext.CurrentContext.Result.Outcome.Status.Equals(TestStatus.Failed))
{
Console.WriteLine("FAILS");
}
else if (TestContext.CurrentContext.Result.Outcome.Equals(ResultState.Success))
{
Console.WriteLine("SUCESS");
}
}
IMHO tear down logic should be independent of test results.
Ideally you should avoid using setup and teardown completely, a al xunit.net. See here for more info.