Develop Reference

What's the deal with Unit Tests and sharing static state?

All,
I am using Unit test framework to write my tests in .net. I wrote some unit tests that all access a configuration object (via AppConfig.Current) that is internally a class static.
In the startup I initialize this configuration object and use it immediately in the following line like this:
[AssemblyInitialize]
public static void AssemblyInitialize(TestContext context)
{
var appCfg = AppConfig.Current;
string path = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "conf/app.settings.xml");
appCfg.Load(path);
var _ = AccountingContext.Current; // <= AccountingContext references the configuration object via "AppConfig.Current" to read some configuration values and initialize the accounting context but I receive a different instance (ONLY in release mode) ....
}
After loading configuration, next line uses this configuration object (AccountingContext.Current) to initialize itself from the configuration object.
The issue is, the tests run fine in Debug mode but in Release mode when I reference the configuration object I get a different object (I know this for a fact because the configuration is not loaded).
So what is the deal with using static reference in unit tests in release mode???
UPDATE: SELF ANSWERED
The issued lied in the "AccountingContext" class.
This class is also a static and it initialized itself like this:
public static AccountingContext Current { get; set; } = new AccountingContext();
public AccountingContext()
{
CompanyAccessContext = _CreateCompanyAccessContext();
}
Changing this intialization to using a static ctor resolved this issue and now everything works:
public static AccountingContext Current { get; set; }
static AccountingContext()
{
Current = new AccountingContext();
}
public AccountingContext()
{
CompanyAccessContext = _CreateCompanyAccessContext();
}
Looks like optimizations were made which changed the actual flow of initialization of AccountingContext. That's a bit unwanted, not sure if this is what everyone expects.
I also found this good resource:
Static member variable not being initialized in Release - Compiler/clr bug?

Cause of this issue was not using static ctor which guarantees order of initialization. For more details, see updated comments above.

Nlog log level rules not changed

Having for example this class that inits the NLog config programmatically:
public class NLogConfig
{
public NLogConfiguration(string traceFileName, LogLevel minLogleven, LogLevel maxLogLevel)
{
var config = new NLog.Config.LoggingConfiguration();
// Targets where to log to file
var logfile = new NLog.Targets.FileTarget("logfile") { FileName = $"{traceFileName}.log" };
logfile.Layout = #"${date:format=HH\:mm\:ss}|${level}|${message} ${exception:format=tostring}";
// Rules for mapping loggers to targets
config.AddRule(minLogleven, maxLogLevel, logfile);
// Apply config
LogManager.Configuration = config;
}
}
Now, I call this config in two different classes, e.g.:
public class A
{
public A()
{
var nlogConfig = new NLogConfig("Trace1", LogLevel.Trace, LogLevel.Fatal);
}
}
public class B
{
public B()
{
var nlogConfig = new NLogConfig("Trace2", LogLevel.Info, LogLevel.Fatal);
}
}
Now the problem is, that the two different log level rules will not be taken by the Nlog configuration, so in that case the Trace1.log and the Trace2.log log both with the trace level, while I expected the Trace1.log to log with the level Trace and the Trace2.log to log with the info log level.
How to fix this behavior, so, I can change the log level dynamically?

At the beginning I would try to overview general design of your app and only then I would give you code samples.
Design overview.
We [developers] like to keep things as simple as possible. One of the ways to achieve this is to separation of responsibilities of software modules.
In your case class A (or B) should NOT care what destination file name is or log level. It should just log records and this's all!
But you can ask how to write into different files with different settings? For this purpose you should use "targets" and "rules" concepts. Basically, target is just a log destination. It can be a file, console etc. A rule is filter to select only messages you need . It can be just a log level or/and even special filter by class name or/and way more. Actually, the configuration your are showing in the first example is setup of targets and rules.
This design should help to separate and simplify things. I hope you understand what I'm talking about otherwise imagine simulation when you require to change logging settings in project with 100 classes that uses logger.
Possible solution.
Classes A and B should create log instance and use it.
public class A
{
private static readonly Logger Logger = LogManager.GetCurrentClassLogger();
public A()
{
Logger.Info("Class 'A' created.");
}
}
Nlog configuration usually done at the application startup to execute it only one time during application lifetime.
public static class Program
{
public static void Main()
{
LogConfiguration();
}
private static void LogConfiguration()
{
var config = new LoggingConfiguration()
// Targets where to log to: File and Console
var f1 = new FileTarget("file1") { FileName = "file1.txt" };
var f2 = new FileTarget("file2") { FileName = "file2.txt" };
// Rules for mapping loggers to targets
config.AddRule(LogLevel.Info, LogLevel.Fatal, f1);
config.AddRule(LogLevel.Debug, LogLevel.Fatal, f2);
// Apply config
NLog.LogManager.Configuration = config;
}
}
Possible feature.
From your code seems that you might want to write into "file2.txt" only messages from class B. This is also possible. You need to specify logger name during rule creation as shown here. Small tip: the name you want to use is name of full your class.
From this example you will get what I'm talking about.

Using Methods not Defined in Interface on Fakes within Unit Tests

In my ViewModel, portions of functionality are enabled/disabled depending on the logged-in individual's permissions. The ViewModel relies on a dependency-injected ISecurity object to check if a user has a specific permission. Different portions of functionality require different permissions.
public Interface ISecurity
{
bool UserHasPermision(int userId, string permission);
}
In my production code, the concrete implementation of ISecurity interacts with an external application which does not allow me to change an individual's permissions. I created a FakeSecurity class that would allow me to do this in unit tests.
class FakeSecurity: ISecurity
{
private Dictionary<int, List<string>> permissions = new Dictionary<int, List<string>>();
public bool UserHasPermission(int userId, string permission)
{
return permissions.ContainsKey(userId) &&
permissions[userId].Contains(permission);
}
//Not defined in ISecurity
public void SetPermission(int userId, string permission, bool hasPermission)
{
if (!permissions.ContainsKey(userId))
{
permissions[userId] = new List<string>();
}
List<string> userPermissions = permissions[userId];
if (hasPermission)
{
userPermissions.Add(permission);
}
else
{
userPermissions.Remove(permission);
}
}
}
The problem here is that SetPermission() is not defined in the ISecurity interface, so in order for my Unit Tests to set an individual's permissions I need to cast the ISecurity object registered with my IUnityContainer to a FakeSecurity object. I am told that my unit test should be ignorant of the specific type of implementation that is being used for a particular interface and that calling methods that are not defined in the interface is an anti-pattern.
[TestMethod]
public void UserDoesNotHavePermission()
{
// test setup
IUnityContainer iocContainer = GetIocContainer();
ISecurity sec = iocContainer.Resolve<ISecurity>(); //registered singleton
(sec as FakeSecurity).SetPermission(GetCurrentUser().Id, "Save Colors", false);
var viewModel = iocContainer.Resolve<MaintainColorsViewModel>(); //per-request
// asserts
Assert.IsFalse(viewModel.CanSave);
}
[TestMethod]
public void UserHasPermission()
{
// test setup
IUnityContainer iocContainer = GetIocContainer();
ISecurity sec = iocContainer.Resolve<ISecurity>(); //registered singleton
(sec as FakeSecurity).SetPermission(GetCurrentUser().Id, "Save Colors", true);
var viewModel = iocContainer.Resolve<MaintainColorsViewModel>(); //per-request
// asserts
Assert.IsTrue(viewModel.CanSave);
}
Is this a bad practice or not? I realize that I shouldn't cast my ISecurity instace to a particular type within my application code, but is this really an issue Unit Tests?

I am told that my unit test should be ignorant of the specific type of implementation
This is incorrect. It is completely normal and good practice to let tests use both fake implementations and the class under test directly.
You however, are using the DI container in your unit tests, and that actually is bad practice. Although the use of the DI container is okay when you're writing integration tests (since you want to test components in integration with other components), using the DI library in unit tests leads to hard to read and maintain tests. With unit tests, you test code in isolation. This means that you usually create the class under test by hand, and inject the required fake dependencies to get the test running.
I would therefore expect such unit test to look like this:
public void CanSave_CurrentUserHasNoPermission_ReturnsFalse() {
// Arrange
var noPermission = new FakeSecurity { CurrentUserHasPermission = false };
var viewModel = new MaintainColorsViewModel(noPermission);
// Act
bool actualResult = viewModel.CanSave;
// Assert
Assert.IsFalse(actualResult);
}
public void CanSave_CurrentUserHasPermission_ReturnsTrue() {
// Arrange
var hasPermission = new FakeSecurity { CurrentUserHasPermission = true };
var viewModel = new MaintainColorsViewModel(hasPermission);
// Act
bool actualResult = viewModel.CanSave;
// Assert
Assert.IsTrue(actualResult);
}
public void CanSave_Always_QueriesTheSecurityForTheSaveColorsPermission() {
// Arrange
var security = new FakeSecurity();
var viewModel = new MaintainColorsViewModel(security);
// Act
bool temp = viewModel.CanSave;
// Assert
Assert.IsTrue(security.RequestedPermissions.Contains("Save Colors"));
}
There are a few things to note about this code:
Both the FakeSecurity and the MaintainColorsViewModel are created directly in the tests here; no DI library is used. This makes the tests much more readable and maintainable (and faster).
I considerably simplified the FakeSecurity class (shown below), because you want fake classes to be as simple as possible.
A third test is added to check explicitly whether the MaintainColorsViewModel requests the expected permission.
The AAA pattern (Arrange/Act/Assert) is implemented explicitly.
To allow these tests to be written the way they are, the following change has been made to the ISecurity abstraction:
interface ISecurity
{
bool UserHasPermission(string permission);
}
The userId parameter has been removed from the UserHasPermission method. The reason for this is that the ISecurity implementation will be able to find out who the current user is by itself. Allowing consumers of ISecurity to pass this parameter along only means that the API is getting more complex, there is more code to write, there's a bigger chance of programming errors, and we therefore need more supporting tests. In other words, the sole addition of this userId property forces a lot of extra production and test code to write and maintain.
Here is the simpflified FakeSecurity class:
class FakeSecurity : ISecurity
{
public bool CurrentUserHasPermission;
public List<string> RequestedPermissions = new List<string>();
public bool UserHasPermission(string permission)
{
this.RequestedPermissions.Add(permission);
return this.CurrentUserHasPermission;
}
}
The FakeSecurity class now has very little code and that makes it, just by looking at it, very easy to check for correctness. Remember, test code should be as simple as possible. Side note: replacing this class with a generated mock object, doesn't make our code easier. In most cases it will actually make our unit tests harder to read, understand and maintain.
One reason for developers to start using a DI container inside their unit tests is because the manual creation of the class under test (with all its fake dependencies) causes maintenance issues in their tests. This is true actually; if the MaintainColorsViewModel has multiple dependencies, and we would create that MaintainColorsViewModel in each test, the addition of a single dependency would cause us to change all our MaintainColorsViewModel tests. This often is a reason for developers to either use a DI container -or- revert to mocking frameworks.
This however is not a good reason to start using a DI container or mocking library. A simple refactoring can completely remove the maintenance problem; we just have to create a factory method as follows:
private static MaintainColorsViewModel CreateViewModel(params object[] dependencies) {
return new MaintainColorsViewModel(
dependencies.OfType<ISecurity>().SingleOrDefault() ?? new FakeSecurity(),
dependencies.OfType<ILogger>().SingleOrDefault() ?? new FakeLogger(),
dependencies.OfType<ITimeProvider>().SingleOrDefault() ?? new FakeTimeProvider(),
dependencies.OfType<IUserContext>().SingleOrDefault() ?? new FakeUserContext());
}
Here I assume that the MaintainColorsViewModel contains 4 dependencies (namely ISecurity, ILogger, ITimeProvider and IUserContext). The CreateViewModel factory method allows passing in all dependencies using a params array, and the method tries to get each abstraction from the array and when missing replaces it with the default fake implementation.
With this factory, we can now rewrite our tests to the following:
[TestMethod]
public void CanSave_CurrentUserHasNoPermission_ReturnsFalse()
{
// Arrange
var noPermission = new FakeSecurity { CurrentUserHasPermission = false };
MaintainColorsViewModel viewModel = CreateViewModel(noPermission);
// Act
bool actualResult = viewModel.CanSave;
// Assert
Assert.IsFalse(actualResult);
}
Or we can pass in multiple dependencies if the test requires this:
[TestMethod]
public void CanSave_CurrentUserHasNoPermission_LogsWarning()
{
// Arrange
var logger = new FakeLogger();
var noPermission = new FakeSecurity { CurrentUserHasPermission = false };
MaintainColorsViewModel viewModel = CreateViewModel(logger, noPermission);
// Act
bool temp = viewModel.CanSave;
// Assert
Assert.IsTrue(logger.Entries.Any());
}
Do note that this test is just here for educational purposes. I don't suggest the view model to actually do the logging; that should not be its responsibility.
The moral of the story here is actually that good design can simplify your testing efforts considerably to the point that you can write less code and less tests, while improving the quality of your software.

You shouldn't use a DI container in unit tests, see the answer in this question.
In unit tests, the object graph that you are testing is usually small (usually a single class). So you don't need a DI container.
Without a container, here is how your test would look like:
//Arrange
FakeSecurity fake_security = new FakeSecurity();
fake_security.SetPermission(GetCurrentUser().Id, "Save Colors", false);
MaintainColorsViewModel sut = new MaintainColorsViewModel(fake_security);
//Act
...
Please note that I am assuming that you are using constructor injection to inject ISecurity into MaintainColorsViewModel.
Please note that instead of creating a FakeSecurity class, you can use auto-generated mocks by using mocking frameworks. Here is a link to one of the mocking frameworks called FakeItEasy.

Based on my experience, when you feel something not natural in Unit Test, you may want to re-factor your code.
According to this code, there are a couple choices.
Define the permission dictionary as a property in the interface. So it is easy to set values at unit testing.
Define a permission layer, IPermission, to add/retrieve/remove permission. Then you can mock IPermission in unit testing your Security implementation.
At least I would define the SetPermission method in ISecurity, your current code does not let you define an ISecurity object and let you set permission. Think the following code.
{
ISecurity sec = CreateSecurity()
sec.SetPermission() // ERROR, SetPermission is not a method in ISecurity.
}
private ISecurity CreateSecurity()
{
return new Security()
}
However, I am not sure how to unit test in this case on top of my head.

How to use application variables in UnitTest asp.net

I have a method that uses Application variables to get information from an external file. Since Application variables are not used in unit tests, is there a way I can get the Application variables values from my Global.asax file and be able to use them in the test?
This is my test method:
[TestMethod]
public void TestGetCompanyList()
{
var accController = new AccSerController();
CInt cInt = new CInt();
cIn.Iss = "Other";
cIn.Tick = "BK";
var result
= accController.Clist(cIn) as IEnumerable<CList>;
Assert.IsNotNull(result);
}

Use the repository pattern. Your controller shouldn't have any idea about WebConfiguration.
//This defines the stuff that your controller needs (that your repository should contain)
public interface ISiteConfiguration
{
string Setting1 {get; set;}
}
//Use this in your site. Pull configuration from external file
public class WebConfiguration : ISiteConfiguration
{
public string Setting1 {get; set;}
public WebConfiguration()
{
//Read info from external file here and store in Setting1
Setting1 = File.ReadAllText(HttpContext.Current.Server.MapPath("~/config.txt"));
}
}
//Use this in your unit tests. Manually specify Setting1 as part of "Arrange" step in unit test. You can then use this to test the controller.
public class TestConfiguration : ISiteConfiguration
{
public string Setting1 {get; set;}
}
I'm using Ninject to perform dependency injection, but there's lots of other libraries out there. I'm going to omit some basic Ninject setup from my answer, because there's plenty of resources out there. But the below code shows how you'd specify in your web application to use WebConfiguration to fulfill the needs of an ISiteConfiguration.
private static void RegisterServices(IKernel kernel)
{
kernel.Bind<ISiteConfiguration>().To<WebConfiguration>();
}
Here's where the magic happens. When an instance of your controller is created in your web application, Ninject will look at the constructor and see that it's asking for ISiteConfiguration. And in your Ninject configuration, you told it to use WebConfiguration when it needs ISiteConfiguration. So Ninject will create a new instance of WebConfiguration and provide (inject) it to your controller.
public class AccountServiceController
{
ISiteConfiguration Config {get; set;}
//This is called constructor injection
public AccountServiceController(ISiteConfiguration config)
{
Config = config;
}
public ActionResult Index()
{
//Now you can use Config without needing to know about ISiteConfiguration's implementation details
//Get settings from Config instead of Application
}
}
You can also use Ninject in unit testing, but here's a simpler demo where we're not using it:
[TestMethod]
public void TestGetCompanyList()
{
//Arrange
var config = new TestConfiguration(){ Setting1 = "mysetting" };
var accountController = new AccountServiceController(config);
}
The result of all this is that you can use your controller's action methods easily for unit testing, because you can use whatever implementation of ISiteConfiguration you want.

I've done the following on some of my tests. Not ideal but it gets the job done.
if (System.Web.HttpContext.Current != null)
{
// Fill your application variable
}
else
{
// Get your data from somewhere else
}

There are two ways of unit testing such scenarios as far as I know.
First one is based on splitting controller function into two: one is controller function itself, another one implements the logic (e.g.: this is the one you test). Example:
Before:
public void MyControllerFunction()
{
var x = Context["variable"];
do-something-with-x;
}
After:
public void MyControllerFunction()
{
var x = Context["variable"];
MyControllerLogic(x);
}
internal void MyControllerLogic(object x)
{
do-something-with-x;
}
And then you test MyControllerLogic() function instead of MyControllerFunction() in unit test
Another methodology is create a surrogate context before invoking unit test.
Example:
var controller = new MyController();
controller.Request = new HttpRequestMessage();
controller.Configuration = new HttpConfiguration();
controller.Request.Content = new StringContent("{ x: 21 }",
Encoding.Unicode);
controller.Request.Content.Headers.ContentType.MediaType =
"application/json";
Please note, I did not create HttpContext in 2nd example, I'm not sure if it's a requirement to have. You probably should be able to create it in similar way as well as the other variables you use. It's sort of a hack anyway, so treat it as such

How to mock ConfigurationManager.AppSettings with moq

I am stuck at this point of code that I do not know how to mock:
ConfigurationManager.AppSettings["User"];
I have to mock the ConfigurationManager, but I don't have a clue, I am using Moq.
Someone can give me a tip? Thanks!

I am using AspnetMvc4. A moment ago I wrote
ConfigurationManager.AppSettings["mykey"] = "myvalue";
in my test method and it worked perfectly.
Explanation: the test method runs in a context with app settings taken from, typically a web.config or myapp.config. ConfigurationsManager can reach this application-global object and manipulate it.
Though: If you have a test runner running tests in parallel this is not a good idea.

I believe one standard approach to this is to use a facade pattern to wrap the configuration manager and then you have something loosely coupled that you have control over.
So you would wrap the ConfigurationManager. Something like:
public class Configuration: IConfiguration
{
public string User
{
get
{
return ConfigurationManager.AppSettings["User"];
}
}
}
(You can just extract an interface from your configuration class and then use that interface everywhere in your code)
Then you just mock the IConfiguration. You might be able to implement the facade itself in a few different ways. Above I chose just to wrap the individual properties. You also obtain the side benefit of having strongly typed information to work with rather than weakly typed hash arrays.

Maybe is not what you need to accomplish, but have you considered to use an app.config in your test project?
So the ConfigurationManager will get the values that you put in the app.config and you don't need to mock anything.
This solution works nice for my needs, because I never need to test a "variable" config file.

You can use shims to modify AppSettings to a custom NameValueCollection object. Here is an example of how you can achieve this:
[TestMethod]
public void TestSomething()
{
using(ShimsContext.Create()) {
const string key = "key";
const string value = "value";
ShimConfigurationManager.AppSettingsGet = () =>
{
NameValueCollection nameValueCollection = new NameValueCollection();
nameValueCollection.Add(key, value);
return nameValueCollection;
};
///
// Test code here.
///
// Validation code goes here.
}
}
You can read more about shims and fakes at, Isolating Code Under Test with Microsoft Fakes. Hope this helps.

Have you considered stubbing instead of mocking? The AppSettings property is a NameValueCollection:
[TestClass]
public class UnitTest1
{
[TestMethod]
public void TestMethod1()
{
// Arrange
var settings = new NameValueCollection {{"User", "Otuyh"}};
var classUnderTest = new ClassUnderTest(settings);
// Act
classUnderTest.MethodUnderTest();
// Assert something...
}
}
public class ClassUnderTest
{
private readonly NameValueCollection _settings;
public ClassUnderTest(NameValueCollection settings)
{
_settings = settings;
}
public void MethodUnderTest()
{
// get the User from Settings
string user = _settings["User"];
// log
Trace.TraceInformation("User = \"{0}\"", user);
// do something else...
}
}
The benefits are a simpler implementation and no dependency on System.Configuration until you really need it.

I fear I need to recall what I said. ConfigurationManager.AppSettings sporadically behaves strange, like if it would not always immediately yield the values just written. We had sporadic unit test failures on our build machines due to this. I had to rewrite my code to use a wrapper, returning ConfigurationManager.AppSettings in the usual case and test values in unit tests.
How about just setting what you need? Because, I don't want to mock .NET, do I...?
System.Configuration.ConfigurationManager.AppSettings["myKey"] = "myVal";
You probably should clean out the AppSettings beforehand to make sure the app only sees what you want it to.

That is a static property, and Moq is designed to Moq instance methods or classes that can be mocked via inheritance. In other words, Moq is not going to be any help to you here.
For mocking statics, I use a tool called Moles, which is free. There are other framework isolation tools, like Typemock that can do this too, though I believe those are paid tools.
When it comes to statics and testing, another option is to create the static state yourself, though this can often be problematic (as, I'd imagine it would be in your case).
And, finally, if the isolation frameworks are not an option and you're committed to this approach, the facade mentioned by Joshua is a good approach, or any approach in general where you factor client code of this away from the business logic that you're using to test.

Another way to achieve this goal is to just provide your own IConfiguration, pulling from any file you'd like it to pull from, like this:
var builder = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true).Build();
Now, as long as you have the values you need for testing in this JSON file, it's very easy to override and change values.

I think writing you own app.config provider is a simple task and is more useful then anything else. Especially you should avoid any fakes like shims etc. because as soon as you use them Edit & Continue no longer works.
The providers I use look like this:
By default they get the values from the App.config but for unit tests I can override all values and use them in each test independently.
There's no need for any interfaces or implement it each time over and over again. I have a utilities dll and use this small helper in many projects and unit tests.
public class AppConfigProvider
{
public AppConfigProvider()
{
ConnectionStrings = new ConnectionStringsProvider();
AppSettings = new AppSettingsProvider();
}
public ConnectionStringsProvider ConnectionStrings { get; private set; }
public AppSettingsProvider AppSettings { get; private set; }
}
public class ConnectionStringsProvider
{
private readonly Dictionary<string, string> _customValues = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
public string this[string key]
{
get
{
string customValue;
if (_customValues.TryGetValue(key, out customValue))
{
return customValue;
}
var connectionStringSettings = ConfigurationManager.ConnectionStrings[key];
return connectionStringSettings == null ? null : connectionStringSettings.ConnectionString;
}
}
public Dictionary<string, string> CustomValues { get { return _customValues; } }
}
public class AppSettingsProvider
{
private readonly Dictionary<string, string> _customValues = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
public string this[string key]
{
get
{
string customValue;
return _customValues.TryGetValue(key, out customValue) ? customValue : ConfigurationManager.AppSettings[key];
}
}
public Dictionary<string, string> CustomValues { get { return _customValues; } }
}