So we're using C#/Specflow, and I have a test that reads
Given The publish directory can not be accessed
The app being tested reads/writes files from a directory, firstly checking it exists and if not throws an exception. I am testing that this exception is thrown. What I need to do is make the directory inaccessible for the duration of the test.
The options as I see it are:
Change the directory it's accessing by overriding it's config (Windows registry) for the duration of the test.
Rename the directory it's accessing for the duration of the test.
Change the permissions on the directory for the duration of the test.
None of these seem ideal, I'd like to leave the test server alone if possible. Can anyone tell me of a better solution to this please?
An option that you didn't specify is to mock out the file access behind an interface and then have the mock simulate lack of access.
This has the benefit of not needing to change anything for the test, but means that its not longer an integration tests. If you don't mock and instead change the config/folder access then this test will need to ensure it isn't run when any other tests are running as they might consequently fail due to those changes.
This question has an example of how you might do this simply, and also links this library and this library which might be able to help.
Related
I'm new to .Net and hence don't know if the approach which I'm following below is correct or is there any better way to do it. Can someone please suggest ?
I want to keep different set of configuration for different environments (DEV,QA, UAT, etc) and based on user input load that environment config and start my Nunit tests.
I'm planning to create a different resource file for each of these for ex - QA.resx, DEV.resx, etc and then just load specific resource file based on user input.
for ex.
QA.resx will have
hostname=sample.qa.com
port=1234
DEV.resx will have
hostname=sample.dev.com
port=4321
And then at runtime if I specify something like env=DEV then it should load configuration from DEV.resx and start running the test cases.
Is this a good approach ?
Is this a good approach ?
I don't think so.
First of all, your Unit Test should not depend on environment you're using. True Unit Test should not have any external dependency, like database, file system, external services, etc. Thus UT execution should be the same no matter whether it's launched on a developer workstation or CI server.
If your application requires different configurations for different environments (it's a very common case), the best choice is to use config transformations. Check this article for details.
Right now my Coded UI Tests use their app.config to determine the domain they execute in, which has a 1-1 relationship with environment. To simplify it:
www.test.com
www.UAT.com
www.prod.com
and in App.config I have something like:
<configuration>
<appSettings>
<add key="EnvironmentURLMod" value ="test"/>
and to run the test in a different environment, I manually change the value between runs. For instance the I open the browser like this:
browserWindow.NavigateToUrl(new Uri("http://www."
+ ConfigurationManager.AppSettings.Get("EnvironmentURLMod")
+ ".com"));
Clearly this is inelegant. I suppose I had a vision where we'd drop in a new app.config for each run, but as a spoiler this test will be run in ~10 environments, not 3, and which environments it may run may change.
I know I could decouple these environment URL modifications to yet another XML file, and make the tests access them sequentially in a data-driven scenario. But even this seems like it's not quite what I need, since if one environment fails then the whole test collapses. I've seen Environment Variables as a suggestion, but this would require creating a test agent for each environment, modifying their registries, and running the tests on each of them. If that's what it takes then sure, but it seems like an enormous amount of VM bandwidth to be used for what's a collection of strings.
In an ideal world, I would like to tie these URL mods to something like Test Settings, MTM environments, or builds. I want to execute the suite of tests for each domain and report separately.
In short, what's the best way to parameterize these tests? Is there a way that doesn't involve queuing new builds, or dropping config files? Is Data Driven Testing the answer? Have I structured my solution incorrectly? This seems like it should be such a common scenario, yet my googling doesn't quite get me there.
Any and all help appreciated.
The answer here is data driven testing, and unfortunately there's no total silver bullet even if there's a "Better than most" option.
Using any data source lets you iterate through a test in multiple environments (or any other variable you can think of) and essentially return 3 different test results - one for each permutation or data row. However you'll have to update your assertions to show which environment you're currently executing in, as the test results only show "Data Row 0" or something similar by default. If the test passes, you'll get no clue as to what's actually in the data row for the successful run, unless you embed this information in the action log! I'm lucky that my use case does this automatically since I'm just using a URL mod, but other people may need to do that on their own.
To allow on-the-fly changing of what environments we're testing in, we chose to use a TestCase data source. This has a lot of flexibility - potentially more than using a database or XML for instance - but it comes with its own downsides. Like all data driven scenarios, you have to essentially "Hard Code" the test case ID into the decorator above your test method (Because it's considered a property). I was hoping we could drop an app.config into the build drop location when we wanted to change which test case we used, at least, but it looks like instead we're going to have to do a find + replace across a solution instead.
If anyone knows of a better way to decouple the test ID or any other part of the connection string from the code, I'll give you an answer here. For anyone else, you can find more information on MSDN.
I'm looking for a way to cause unit tests to fail if they access the filesystem or database using NUnit and C#. I would preferably like to be able to set something up in a unit testing base class to prevent filesystem access on all child testing classes.
I currently need to disable the database service, run the unit tests, and check to see which ones fail in order to locate any tests that are accessing resources they aren't supposed to. I'd like to find a way to automate this process so that these tests will fail as soon as they are written.
One way you could test the functionality of the system for particular scenarios (e.g. failing to access the filesystem) is by abstracting the portions of your code where you access the file system and then "plug" these into your classes where they're used via dependency injection. By doing this, you can have tests where your mock file access objects throw specific errors and/or mimic success scenarios. The follow post has some links on dependency injection that you may find useful
Dependency Injection in .NET
You may also want to look into the SOLID principles for writing your code.
I am using MSTest in Visual Studio 2008 with C#. I have a specific environment variable I would I would like to and a path modification I would like to do only during the run of either specific tests or better yet all test within a run configuration.
I tried using the test run configuration Setup script to do this but as I expected since it is a batch file the changes are lost once it exits, so that wont work.
Is there any other way to setup temporary system environment variables that will be valid during all tests being run?
While am not happy with this solution, I was able to get what I needed done with MSTest by using the ClassInitializeAttribute for a test class, and then using Environment.SetEnvironmentVariable to make the changes I need, and then clean this up in the method decorated with theClassCleanupAttribute.
With lack of a better answer this was how I was able to get environment variables set for a group of tests and clean it up when I was done. However I would have prefered this to be handled outside of the CODE and be part of test configuration in some way. Regardless issues has been resolved.
If you trust that your test suite won't be "aborted" mid-test, you can use FixtureSetup and FixtureTeardown methods to set and then remove your changed environment variables.
EDIT FROM COMMENT: I see where you're coming from, but as in my edit, a UT framework is deisgned to be used to create unit tests. The concept of a unit test dictate that it should NOT depend on any outside resources, including environment variables. Tests that do this are integration tests, and require a lot of infrastructure to be in place (and usually take many times longer than a unit test suite of equal LOC).
To create a unit test for code that depends on an environment variable, consider splitting out the lines of code that actually examine the environment variables directly,. and put that into a method in another class, then mock that class using RhinoMocks or whatever to provide a "dummy" value for testing without examining (or changing) actual environment variables.
If this really is an integration test and you really need the environment variable set (say you're changing the path so you can use Process.Start to call your own notepad.exe instead of Windows'), that's what the FixtureSetup and FixtureTeardown methods/attributes are for; to perform complicated setup of a fixed, repeatable environment in which the tests should succeed, and then reset the environment to the way it was, regardless of what happened in the tests. Normally, a test failure throws an exception and ends that test's execution immediately, so code at the end of the test method itself is not guaranteed to run.
I'm trying to unit test file read operations. In this scenario I also need make sure that, if a particular user don't have read access he should get an exception...
But somehow I'm unable to get it working, can anyone suggest something?
PS: I'm using Rhino mock and NUnit
You could use Rhino.Mocks "Do" extension to throw a specific exception:
public delegate void ThrowExceptionDelegate();
mystub.Stub(x => x.ReadFile()).Do(new ThrowExceptionDelegate(delegate()
{ throw new IOException(); }
));
This would allow you to test your exception handling code.
You need to get a test in place which, in place of reading a file is using a mock that throws an exception instead of really reading a file. Then you can verify that the appropriate handling is triggered and things work out as they should.
If you need a better answer, you need to give an example of your classes and maybe the skeleton of the test you've written so far.
I'd go for a proper acceptance test - using mocks too much can be a bit dangerous. In this case it's easy to programmatically set + unset file permissions anyway.
I had a similar problem - wanted to test a permissions problem + came up with the following helper class to wrap the library API for messing around with file permissions
set file permissions for c:\program files\company\app\file for all users