I have a Question for you guys.I have 2 unit tests which are calling webservices .The value that one unit-test returns should be used for another unit test method
Example
namespace TestProject1
{
public class UnitTest1
{
String TID = string.empty;
public void test1
{
//calling webservices and code
Assert.AreNotEqual(HoID, hID);
TID = hID;
}
public void test2
{
//calling webservices and code
string HID = TID // I need the TID value from the Above testcase here
Assert.AreNotEqual(HID, hID);
}
}
}
How can i store a value in one unittest and use that value in another unittest.
In general, you shouldn't write your tests like this. You cannot ensure that your tests will run in any particular order, so there's no nice way to do this.
Instead make the tests independent, but refactor the common part into it's own (non-test) method that you can call as part of your other test.
Don't reuse any values. Order in which tests are run is very often random (most common runners like NUnit's and Resharper's run tests in random order, some might even do so in parallel). Instead, simply call web service again (even if that means having 2 web service calls) in your second test and retrieve the value you need.
Each test (whether it's unit or integration) should have all the data/dependencies available for it to run. You should never rely on other tests to setup environment/data as that's not what they're written for.
Think of your tests in isolation - each single test is a separate being, that sets up, executes, and cleans up all that is necessary to exercise particular scenario.
Here's an example, following the outlines of Oleksi, of how you could organize this
String TID = string.empty;
[TestFixtureSetUp]
public void Given() {
//calling webservices and code
TID = hID;
//calling webservices and code
}
[Test]
public void assertions_call_1() {
...
}
public void assertions_on_call_2() {
if (string.IsNullOrEmpty(TID))
Assert.Inconclusive("Prerequisites for test not met");
...
}
Related
I am having troubles when testing a controller, because there are some lines at my Startup that are null when testing, I want to add a condition for run this lines only if it's not testing.
// Desired method that retrieves if testing
if (!this.isTesting())
{
SwaggerConfig.ConfigureServices(services, this.AuthConfiguration, this.ApiMetadata.Version);
}
The correct answer (although of no help): It should not be able to tell so. The application should to everything it does unaware if it is in productino or test.
However to test the application in a simpler setting, you can use fake modules or mock-up modules that are loaded instead of the heavy-weight production modules.
But in order to use that, you have to refactor your solution and use injection for instance.
Some links I found:
Designing with interfaces
Mock Objects
Some more on Mock objects
It really depends on which framework you use for testing. It can be MSTest, NUnit or whatever.
Rule of thumb, is that your application should not know whether it is tested. It means everything should be configured before actual testing through injection of interfaces. Simple example of how tests should be done:
//this service in need of tests. You must test it's methods.
public class ProductionService: IProductionService
{
private readonly IImSomeDependency _dep;
public ImTested(IImSomeDependency dep){ _dep = dep; }
public void PrintStr(string str)
{
Console.WriteLine(_dep.Format(str));
}
}
//this is stub dependency. It contains anything you need for particular test. Be it some data, some request, or just return NULL.
public class TestDependency : IImSomeDependency
{
public string Format(string str)
{
return "TEST:"+str;
}
}
//this is production, here you send SMS, Nuclear missle and everything else which cost you money and resources.
public class ProductionDependency : IImSomeDependency
{
public string Format(string str)
{
return "PROD:"+str;
}
}
When you run tests you configure system like so:
var service = new ProductionService(new TestDependency());
service.PrintStr("Hello world!");
When you run your production code you configure it like so:
var service = new ProductionService(new ProductionDependency());
service.PrintStr("Hello world!");
This way ProductionService is just doing his work, not knowing about what is inside it's dependencies and don't need "is it testing case №431" flag.
Please, do not use test environment flags inside code if possible.
UPDATE:
See #Mario_The_Spoon explanation for better understanding of dependency management.
I'm struggle with this one quite a long time now. Some background: I created my automated test framework using Selenium. With one project being pure NUnit tests and second one that does all the work for the tests. Now, in test project I keep the directories to all environments I run my tests against. So far many of my tests were read-only and didn't bother much about if tests did not run on environment that they supposed to. This has changed as I started to run some 'write' tests.
I need to prevent this 'Write' tests to run on any other environment then localhost. So far I tried to use method attributes and getting test method names on run time and do work then but this is not quite efficient. Can you guys point me a good solution? Thanks!
I would tag the tests to exclude with a particular category name then define a SetUp function which will stop the tests from running if they are tagged with that name and if you are on a particular environment such as Production. Place the SetUp function in a BaseClass and have all your test fixtures inherit it. The SetUp function will run before every test and prevent it from running if it needs to.
Something like this:
public class BaseSetup
{
protected const string CategoryToExclude = "Write";
[SetUp]
public void Init()
{
string env = ConfigurationManager.GetEnvironment();
if ( env == Constants.Environments.PROD && (TestContext.CurrentContext.Test.Properties["Categories"].Contains(CategoryToExclude)))
{
Assert.Inconclusive(String.Format("Cannot run this test on environment: {0}", env));
}
}
}
[TestFixture]
public class UnitTests : BaseSetup
{
[Test]
[Category(CategoryToExclude)]
public void TestMethod1()
{
Console.WriteLine("TestMethod1");
}
[Test]
public void TestMethod2()
{
Console.WriteLine("TestMethod2");
}
}
Hope this helps!
NUnit have category attribute.
You can group tests by categories and run only wanted categories.
I'm currently running tests through visual studio. Before all the tests are run, I automatically create a set number of users with know credentials, and at the end of the run I delete those users. However, sometimes I need to cancel my tests midway. In these cases the test never gets the chance to clean up, this means that there is left over fake user info from the test run and may causes the next test run to crash (when it attempts to add user info into the DB). Is there anyway to force visual studio/mstest to run a clean up method even if the test is canceled?
I know one option is to have the test check and make sure that the user info doesn't already exist, and if it does remove it before creating the new users. But this still wouldn't solve the issue of the canceled test run leaving unwanted test data.
Update:
Sorry for the miscommunication, however cleaning up the data at the start of the test is not an option. I'm giving a very simplistic view of the issue, but put simply, I have no easy way of making sure that no test data exists at the start of the test. All clean up must occur at the end of the test.
That is impossible. You better find an alternative solution like using separate database for testing and clean all data before each test run, using fixed set of test users or mark test data with some flag. Check Isolating database data in integration tests article by Jimmy Bogard.
There is no built-in way to change MSTest default behavior. In theory you can write MSTest extension that utilizes TestExecution.OnTestStopping event, but that is not an easy process and it requires registry change. Moreover, a lot of people complain that it is not working.
There also MSTest V2, a new version of MSTest with new extensibility points. But it looks like you can't alter cancel behavior with this points, only write attribute decorators. See Extending MSTest V2.
You can't use AppDomain.CurrentDomain.ProcessExit and Process.GetCurrentProcess().Exited events because cancel seems to kill test run process.
NUnit also doesn't support this at the moment. See related NUnit test adapter Run TearDowns on VS Cancel Test Run issue.
Instead of calling the cleanup function at the end of the test, I call mine at the beginning of each test in order to address this exact problem.
Perform the clean up before creating the data as well, this will ensure that you have no leftover data whatever happens. Of course this is only possible if you can identify any leftover data before running the setup.
The idea is that a transaction is initialized before the test begins. In order for the data to be saved in the database, the transaction must be commited, but it is not commited never. It works in case when test a stop, in the case of a successful or unsuccessful completion of the test.
In integration tests we use somthing like this (with NUnit)(It real production code)
public class RollbackAttribute : TestAttribute, ITestAction
{
private TransactionScope _transaction;
public void BeforeTest(ITest test)
{
_transaction = new TransactionScope();
}
public void AfterTest(ITest test)
{
_transaction.Dispose();
}
public ActionTargets Targets => ActionTargets.Test;
}
[TestFixture]
public class SomeTestClass
{
[Rollback] //No need [Test] because Rollback is inherit it.
public void SomeTestMethod()
{
}
}
On MsTest you can make somthing similar, but in this case you should inherit from base class, I hope it works. For example:
public class RollbackTestBase
{
private TransactionScope _transaction;
[TestInitialize]
public virtual void Setup()
{
_transaction = new TransactionScope();
}
[TestCleanup]
public virtual void TearDown()
{
_transaction.Dispose();
}
}
[TestClass]
public class IntegrationTest : RollbackTestBase
{
[TestMethod]
public void TestDataBase()
{
Assert.IsTrue(true);
}
[TestInitialize]
public virtual void Init()
{
}
[TestCleanup]
public virtual void CleanUp()
{
}
}
There are 2 cases which we need to consider while allocating resources in ATPs (Resources might be Creating users, Connection with database). They are
Creation and deletion of resources after each test.
Creation and deletion of resources after set of tests.
Creation and deletion of resources after each test:
If we want to create instance of particular object before execution of a test and want to clean up memory allocated to that object after execution of that test, then we use Test SetUp and Test TearDown attributes of NUnit. In your case object is creation of number of Users.
[SetUp] : Function which is decorated with Test SetUp attribute contains piece of code that executes before the execution of any test.
[TearDown] : Function which is decorated with Test TearDown attributes contains piece of code that executes after execution of any test
Implementation:
[TestClass]
public class UnitTest1
{
[SetUp]
public void SetUP()
{
// Creating Users with proper credentials
}
[TestMethod]
public void TestMethod1()
{
//Write your ATP
}
[TearDown]
public void TearDown()
{
//Clean up
}
}
Creation and deletion of resources after set of tests:
Now If we want to create instance of an object for set of tests and want to clean up memory after execution of all tests then [TestFixtureSetUp] and [TestFixureTearDown] to initialize an object and to clean up memory respectively. Again In your case object can be creation of set of users.
[TestFixtureSetUp] : Function decorated with TestFixtureSetUp will executes once before the execution of group of tests.
[TestFixtureTearDown] : Function decorated with TestFixtureTearDown will executes once after the execution of group of tests.
Implementation
[TestFixture]
public class Tests
{
[TestFixtureSetUp]
public void Setup()
{
//Create users with credentials
}
[Test]
public void _Test1()
{
//Test_1
}
[Test]
public void _Test2()
{
//Test2
}
[TestFixtureTearDown]
public void CleanUp()
{
//Cleanup; Here you need to add code to Delete all users
}
}
Note: I will suggest you, if you are trying to create and delete users for particular ATP then go with SetUp and TearDown. If you are trying same for bunch of ATPs, I would recommend you to go with TestFixtureSetUp and TestFixtureTearDown.
"If your test get pass or fail, SetUp and TearDown functions will execute"
References:
#Shuvra's Answer.
Nunit: SetUp, TearDown, SetUpFixture, TearDownFixture
I think you should open a transaction before you test, create the data and finish test test. But do not commit the transaction. That will ensure that the test can't affect your DB at all.
Update:
The easier approach is use docker container.
You can run a container from your image and remove that container after the test is done. This should definitely reduce the complexity of your test.
Visual studio uses NUNIT therefore, you can use TearDownAttribute. It should run after the test, even if the test is canceled. You can write a function to clean your data.
Please read the reference documentation here: http://nunit.org/docs/2.2/teardown.html
Just to clear more about the NUNIT standrad. Please follow the steps in the Test class:
[TestFixture]
public class _TestClass
{
[TestFixtureSetUp]
public void Setup()
{
//Clearup can be here before start of the tests. But not Recommended
}
[Test]
public void _Test1()
{
}
[Test]
public void _Test2()
{
}
[TestFixtureTearDown]
public void CleanUp()
{
//I will recommend to clean up after all the tests complete
}
}
Reference: http://nunit.org/docs/2.5/fixtureTeardown.html
A better solution to the problem to use what is called "database mocking". In this case you would have your tests run with a different database (or a fake, virtual one).
This article explains how to implement it in C#
https://msdn.microsoft.com/en-us/library/ff650441.aspx
You should begin a transaction and not commit your records to the DB. Thus, all your changes will be automatically rollbacked when the session is over.
Here's the sample static method, say
public static void UpdateSchedule(int selectedScheduleId)
{
using (var dc = new MyDataContext())
{
var selectedSchedule = dc.Schedules.SingleOrDefault(p => p.ScheduleId == selectedScheduleId)
if selectedSchedule != null)
{
selectedSchedule.Name = name;
//and update other properties...
}
dc.SubmitChanges();
}
}
So what would be the correct approach to test on methods like this? Is there a way to avoid calling new MyDataContext() as it might increase the execution time of the unit test.
Also, I am using MsTest test framework in VS2012.
Static functions interfere with testing primarily because they:
Are difficult (sometimes impossible) to substitute from within a consumer
Tend to have "hidden" dependencies
Since you want to test the function itself, number 1 isn't an issue. However, number 2 is problematic because the static function is tightly coupled to the MyDataContext class.
If you want to test the static function without MyDataContext (that is, in isolation) you need to introduce a code seam. This requires some refactoring work, but we can do it fairly painlessly.
Consider if you had the following additional method:
public static void UpdateScheduleWithContext(int selectedScheduleId, IDataContext dc)
{
var selectedSchedule = dc.Schedules.SingleOrDefault(p => p.ScheduleId == selectedScheduleId)
if selectedSchedule != null)
{
selectedSchedule.Name = name;
//and update other properties...
}
dc.SubmitChanges();
}
This new function gives the consumer (i.e., the test) the ability to supply a test double for the data context. This is the seam.
Obviously, though, you don't want all consumers to explicitly provide a data context (which is why you had the original static function to begin with). So you can keep that function around, and just modify it:
public static void UpdateSchedule(int selectedScheduleId)
{
using (var dc = new MyDataContext())
{
UpdateScheduleWithDataContext(selectedScheduleId, dc);
}
}
If you don't have a seam like this, it will be impossible to test the function in isolation. As a result, an integration test would be the best you could hope for.
As Simon Whitehead said, it is impossible to do effective unit testing on static methods and objects. However, using the unit test framework in Visual Studio, you can create a 'unit test' function that is effectively a integration test. Tell Visual Studio to generate a Unit Test for UpdateSchedule, and then modify the function generated to set up the environment/state of the program such that the UpdateSchedule method can execute (create a database connection, instantiate classes, etc.). This includes ensuring that your database has records to update.
Once that is done, you can execute your integration test in the same manner that you would a unit test.
I've been given a task of creating a restful web service with JSON formating using WCF with the below methods using TDD approach which should store the Product as a text file on disk:
CreateProduct(Product product)
GetAProduct(int productId)
URI Templates:
POST to /MyService/Product
GET to /MyService/Product/{productId}
Creating the service and its web methods are the easy part but
How would you approach this task with TDD? You should create a test before creating the SUT codes.
The rules of unit tests say they should also be independent and repeatable.
I have a number of confusions and issues as below:
1) Should I write my unit tests against the actual service implementation by adding a reference to it or against the urls of the service (in which case I'd have to host and run the service)? Or both?
2)
I was thinking one approach could be just creating one test method inside which I create a product, call the CreateProduct() method, then calling the GetAProduct() method and asserting that the product which was sent is the one that I have received. On TearDown() event I just remove the product which was created.
But the issues I have with the above is that
It tests more than one feature so it's not really a unit test.
It doesn't check whether the data was stored on file correctly
Is it TDD?
If I create a separate unit test for each web method then for example for calling GetAProduct() web method, I'd have to put some test data stored physically on the server since it can't rely on the CreateProduct() unit tests. They should be able to run independently.
Please advice.
Thanks,
I'd suggest not worrying about the web service end points and focus on behavior of the system. For the sake of this discussion I'll drop all technical jargon and talk about what I see as the core business problem you're trying to solve: Creating a Product Catalog.
In order to do so, start by thinking through what a product catalog does, not the technical details about how to do it. Use that as your starting points for your tests.
public class ProductCatalogTest
{
[Test]
public void allowsNewProductsToBeAdded() {}
[Test]
public void allowsUpdatesToExistingProducts() {}
[Test]
public void allowsFindingSpecificProductsUsingSku () {}
}
I won't go into detail about how to implement the tests and production code here, but this is a starting point. Once you've got the ProductCatalog production class worked out, you can turn your attention to the technical details like making a web service and marshaling your JSON.
I'm not a .NET guy, so this will be largely pseudocode, but it probably winds up looking something like this.
public class ProductCatalogServiceTest
{
[Test]
public void acceptsSkuAsParameterOnGetRequest()
{
var mockCatalog = new MockProductCatalog(); // Hand rolled mock here.
var catalogService = new ProductCatalogService(mockCatalog);
catalogService.find("some-sku-from-url")
mockCatalog.assertFindWasCalledWith("some-sku-from-url");
}
[Test]
public void returnsJsonFromGetRequest()
{
var mockCatalog = new MockProductCatalog(); // Hand rolled mock here.
mockCatalog.findShouldReturn(new Product("some-sku-from-url"));
var mockResponse = new MockHttpResponse(); // Hand rolled mock here.
var catalogService = new ProductCatalogService(mockCatalog, mockResponse);
catalogService.find("some-sku-from-url")
mockCatalog.assertWriteWasCalledWith("{ 'sku': 'some-sku-from-url' }");
}
}
You've now tested end to end, and test drove the whole thing. I personally would test drive the business logic contained in ProductCatalog and likely skip testing the marshaling as it's likely to all be done by frameworks anyway and it takes little code to tie the controllers into the product catalog. Your mileage may vary.
Finally, while test driving the catalog, I would expect the code to be split into multiple classes and mocking comes into play there so they would be unit tested, not a large integration test. Again, that's a topic for another day.
Hope that helps!
Brandon
Well to answer your question what I would do is to write the test calling the rest service and use something like Rhino Mocks to arrange (i.e setup an expectation for the call), act (actually run the code which calls the unit to be tested and assert that you get back what you expect. You could mock out the expected results of the rest call. An actual test of the rest service from front to back would be an integration test not a unit test.
So to be clearer the unit test you need to write is a test around what actually calls the rest web service in the business logic...
Like this is your proposed implementation (lets pretend this hasn't even been written)
public class SomeClass
{
private IWebServiceProxy proxy;
public SomeClass(IWebServiceProxy proxy)
{
this.proxy = proxy;
}
public void PostTheProduct()
{
proxy.Post("/MyService/Product");
}
public void REstGetCall()
{
proxy.Get("/MyService/Product/{productId}");
}
}
This is one of the tests you might consider writing.
[TestFixture]
public class TestingOurCalls()
{
[Test]
public Void TestTheProductCall()
{
var webServiceProxy = MockRepository.GenerateMock<IWebServiceProxy>();
SomeClass someClass = new SomeClass(webServiceProxy);
webServiceProxy.Expect(p=>p.Post("/MyService/Product"));
someClass.PostTheProduct(Arg<string>.Is.Anything());
webServiceProxy.VerifyAllExpectations();
}
}