I'm using NDepend to analyze a C# project that I'm in the middle of developing. I have most of my business logic and data access layers written, but right now, the only front end application that I have is a "quick and dirty" test application.
So first off, NDepend has all kinds of issues with my test application. Nothing serious, just things like too many methods, too-long methods, etc. Since this is basically a throwaway application, I didn't want to spend a bunch of time refactoring it, so I removed it from the NDepend project.
The problem is that now, since that was my only front end application in the project, NDepend is complaining about things in my business layer like uninstantiated classes, since there is no code that instantiates them except in the test application that I've excluded. I know that these are safe to ignore for now, since they will eventually be instantiated by the REAL front end app, but I really really want to see all of the yellow triangles go away before I do any further development on this app.
Is there a way to make NDepend NOT complain about issues in a particular assembly, but still include it to make queries in OTHER assemblies pass?
If not, any other ideas?
Yes it is possible. First reference again your test application assembly(ies).
Then exclude them by adding a custom query that look like:
// <Name>Discard test assemblies from JustmyCode</Name>
notmycode Application.Assemblies.WithNameIn("TestAsm1","TestAsm2"...)
This query can be saved in the default group Defining JustMyCode (not mandatory but recommended).
Then you need to adapt code rules that are warning about test assemblies dirty stuff to use JustMyCode instead of Application (like from m in JustMyCode.Methods... instead of from m in Application.Methods...).
The notmycode/JustmyCode related documentation can be found here.
Related
I've recently started pushing for TDD where I work. So far things are going well. We're writing tests, we're having them run automatically on commit, and we're always looking to improve our process and tools.
One thing I've identified that could be improved is how we set up our Test Data. In strictly unit tests, we often find ourselves instantiating and populating complex CLR objects. This is a pain, and typically the test is then only run on a handful of cases.
What I'd like to push for is Data Driven tests. I think we should be able to load our test data from files or maybe even generate them on the fly from a schema (though I would only consider doing it on the fly if I could generate every possible configuration of an object, and that number of configurations was small). And there is my problem.
I have yet to find a good strategy for generating test data for C# CLR objects.
I looked into generating XML data from XSDs and then loading that into my tests using the DataSourceAttribute. The seemed like a good approach, but I ran into troubles generating XSD files. xsd.exe falls over because our classes have interface members. I also tried using svcutil.exe on our assembly, but because our code is monolithic the output is huge and tricky (many interdependent .xsd files).
What are other techniques for generating test data? Ideally the generator would follow a schema (maybe an xsd, but preferably the class itself), and could be scripted.
Technical notes (not sure if this is even relevant, but it can't hurt):
We're using Visual Studio's unit testing framework (defined in Microsoft.VisualStudio.TestTools.UnitTesting).
We're using RhinoMocks
Thanks
Extra Info
One reason I'm interested in this is to test an Adapter class we have. It takes a complex and convoluted legacy Entity and converts it to a DTO. The legacy Entity is a total mess of spaghetti and can not be easily split up into logical sub-units defined by interfaces (as suggested). That would be a nice approach, but we don't have that luxury.
I would like to be able to generate a large number of configurations of this legacy Entity and run them through the adapter. The larger the number of configurations, the more likely my test will fail when the next developer (oblivious to 90% of the application) changes the schema of the legacy Entity.
UPDATE
Just to clarify, I am not looking to generate random data for each execution of my tests. I want to be able to generate data to cover multiple configurations of complex objects. I want to generate this data offline and store it as static input for my tests.
I just reread my question and noticed that I had in fact originally ask for random on the fly generation. I'm surprised I ask for that! I've updated the question to fix that. Sorry about the confusion.
What you need is a tool such as NBuilder (http://code.google.com/p/nbuilder).
This allows you to describe objects, then generate them. This is great for unit testing.
Here is a very simple example (but you can make it as complex as you want):
var products = Builder<Product>
.CreateListOfSize(10)
.All().With(x => x.Title = "some title")
.And(x => x.AnyProperty = RandomlyGeneratedValue())
.And(x => x.AnyOtherProperty = OtherRandomlyGeneratedValue())
.Build();
In my experience, what you're looking to accomplish ends up actually being harder to implement and maintain than generating objects in code on a test-by-test basis.
I worked with a client that had a similar issue, and they ended up storing their objects as JSON and deserializing them, with the expectation that it would be easier to maintain and extend. It wasn't. You know what you don't get when editing JSON? Compile-time syntax checking. They just ended up with tests breaking because of JSON that failed to deserialize due to syntax errors.
One thing you can do to reduce your pain is to code to small interfaces. If you have a giant object with a ton of properties, a given method that you'd like to test will probably only need a handful. So instead of your method taking SomeGiantClass, have it take a class that implements ITinySubset. Working with the smaller subset will make it much more obvious what things need to be populated in order for your test to have any validity.
I agree with the other folks who have said that generating random data is a bad idea. I'd say it's a really bad idea. The goal of unit testing is repeatability, which goes zooming out the window the second you generate random data. It's a bad idea even if you're generating the data "offline" and then feeding it in. You have no guarantee that the test object that you generated is actually testing anything worthwhile that's not covered in other tests, or if it's testing valid conditions.
More tests doesn't mean that your code is better. 100% code coverage doesn't mean that your code is bug-free and working properly. You should aim to test the logic that you know matters to your application, not try to cover every single imaginable case.
This is a little different then what you are talking about, but have you looked at Pex? Pex will attempt to generate inputs that cover all of the paths of your code.
http://research.microsoft.com/en-us/projects/Pex/
Generating test data is often an inappropriate and not very useful way of testing - particuarly if you are generating a different set of test data (eg randomly each time) as sometimes a test run will fail and sometimes it wont. It also may be totally irrelevant to what your doing and will make for a confusing group of tests.
Tests are supposed to help document + formalise the specification of a piece of software. If the boundaries of the software are found through bombarding the system with data then these wont be documented properly. They also provide a way of communicating through code that is different from the code itself and as a result are often most useful if they are very specific and easy to read and understand.
That said if you really want to do it though typically you can write your own generator as a test class. I've done this a few times in the past and it works nicely, with the added bonus that you can see exactly what it's doing. You also already know the constraints of the data so there's no problem trying to generalise an approach
From what you say the pain you are having is in setting up objects. This is a common testing issue - I'd suggest focusing on that by making fluent builders for your common object types - this gives you a nice way of filling in less detail every time (you typically would provide only the interesting data (for a given test case) and have valid defaults for everything else). They also reduce the number of dependencies on constructors in test code which means your tests are less likely to get in the way of refactoring later on if you need to change them. You can really get a lot of mileage out of that approach. You can further extend it by having common setup code for builders when you get a lot of them that is a natural point for developers to hang reusable code.
In one system I've worked on we ended up aggregating all these sorts of things together into something which could switch on + off different seams in the application (file access etc), provided builders for objects and setup a comprehensive set of fake view classes (for wpf) to sit on top of our presenters. It effectively provided a test friendly interface for scripting and testing the entire application from very high-level things to very low-level things. Once you get there you're really in the sweet spot as you can write tests that effectively mirror button clicks in the application at a very high level but you have very easy to refactor code as there are few direct dependencies on your real classes in the tests
Actually, there is a Microsoft's way of expressing object instances in markup, and that is XAML.
Don't be scared with the WPF paradigm in the documentation. All you need to do is use correct classes in unit tests to load the objects.
Why I would do this? because Visual Studio project will automatically give you XAML syntax and probably intellisense support when you add this file.
What would be a small problem? markup element classes must have parameterless constructors. But that problem is always present and there are workarounds (e.g. here).
For reference, have a look at:
Create object from text in XAML, and
How to convert XAML File to objects, and
How to Deserialize XML document.
I wish I could show you something done by me on this matter, but I can't.
I have two separate namespaces in my assembly: DataAccess and DomainLogic.
I need a code snippet checking that no class in DomainLogic depends on the namespace DataAccess.
How would you dou it?
PS: I think I saw an example for such a unit test in Mark Seemann's awesome book Dependency Injection in .Net, but I don't have it available here and can't find an example via Google.
Edit
Since all reactions so far point out that I should just split these interdependent classes into two different assemblies, I would like to point out that this is currently not an option (although this is indeed one of my main goals in the end). I'm dealing with legacy code and I just can't refactor it in one big bang right now. The separate namespaces and the test for dependencies between them are an intermediate step. As soon as that test passes, I can go ahead and move a part of the code into a different assembly.
All code within an assembly can legitimately access public and internal code throughout the rest of the assembly. So such unit tests, even if possible, would be a bad idea.
If you split the DataAccess types out into a separate project and made it all internal, then nothing would be able to access it. Clearly not what you'd want. However, by splitting it out, you can ensure that DomainAccess can access DomainLogic, but not vice versa. This presumably is what you want.
In the meantime, rather than try to develop unit tests to check that the rule of "DomainLogic must not access DomainAccess", use code reviews instead. Assuming you are using agile methods (if not, do so!), all activities will be documented as tasks. No task can be considered "done" until someone who understands and embraces your rule has reviewed the code changes for a task. Break the rule and the task fails code review and must be reworked before it's done.
There's a tool that does just this: checks namespace dependencies based on your rules and reports violations at build-time as warnings or errors.
It's called NsDepCop, free, open-source.
The rule config would look something like this:
<NsDepCopConfig IsEnabled="True" CodeIssueKind="Warning">
<Allowed From="*" To="*" />
<Disallowed From="DomainLogic" To="DataAccess" />
</NsDepCopConfig>
My C# .NET solution files are a mess and I am trying to find a way of getting things in order.
I tried to put all close files together in the same folder I am creating for that purpose. For example, I put interfaces, abstract classes, and all their inherited classes at the same folder. By the way - when I do that, I need to write a "using" statement pointing to that folder so I can use those classes in other files (also a mess I guess).
Is there an elegant way of doing things more clean, and not a list of files that I find very confusing?
Is it a good idea to (let's say) open a abstract class file and add nested classes for all the classes derived from it?
Is there a way of telling the solution to automatically set the folder "using" statements above every class I create?
The best way is when your solution file system structure reflects your program architecture and not your code architecture.
For example: if you define an abstract class and after have entities that implement it: put them into the same "basket" (solution folder) if they make a part of the same software architectual unit.
In this case one by looking on your solution tree can see what is your architecture about (more or less) from very top view.
There are different ways to enforce the architecture vision, understanding and felling of the code file system. For example if you use some known frameworks, like NHibernate, or (say) ASP.NET MVC tend to call the things in the name the technolgy calls them, in this way one who is familiar with that technology can easily find itself in your architecture.
For example WPF force you define in code things in some way, but also you need to define byb the way Model, ModelView, View.. which you will do intuitively in seprate files. The technology enforcce you to define your file system in way it was thought.
By the way the topic you're asking for, is broad known dilema/question, not resolved, cuase the code is just characters sequence and nothing else.
Good luck.
It sounds like you're hitting the point where you actually need to break things up a bit, but you're resisting this because more files seems like more complexity. That's true to a point. But there's also a point where files just become big and unmanageable, which is where you might end up if you try to do nested classes.
Keeping code in different namespaces is actually a good thing--that's the "issue" you're running into with the folders and having to add using statements at the top of your files. Namespacing allows you to logically divide your code, and even occasionally reuse a class name, without stepping on other parts of your code base.
What version of Visual Studio are you using? One little known feature of Visual Studio is that it can automatically create the using directive when you type a class name. That would eliminate one pain point.
If I was in your shoes, I'd start looking for logical places to segment my code into different projects. You can definitely go overboard here as well, but it's pretty common to have:
A "core" project that contains your business logic and business objects.
UI projects for the different user interfaces you build, such as a website or Windows Forms app.
A datalayer project that handles all interactions with the database. Your business logic talks to the datalayer instead of directly to the database, which makes it easier to make changes to your database setup down the road.
As your code base grows, a tool like ReSharper starts to become really important. I work on a code base that has ~1 million lines and 10 or so projects in the solution, and I couldn't live without ReSharper's go-to-file navigation feature. It lets you hit a keyboard shortcut and start typing a file name and just jump to it when it finds a match. It's sort of like using Google to find information instead of trying to bookmark every interesting link you come across. Once I made this mental shift, navigating through the code base became so much easier.
Try using multiple projects in the same solution to bring order. Seperate projects for web, entity, data access, setup, testing, etc.
IF the files are in the same namespace you won't need a using statement. If you're breaking your code into multiple projects you'll need to reference the other projects with using statements.
Its up to you. Break things apart logically. Use subfolders where you deem necessary.
Not sure.
Yes, but you'll need to create a template. Search for tuturorials on that.
1) Your solution folders should match your namespace structure. Visual Studio is set up to work this way and will automatically create a matching namespace. Yes, this requires a using for stuff in the folders but that's what it's for.
So yes, group common stuff together under an appropriate namespace.
2) Yes, subclasses should probably live in the same namespace/folder as their abstract base, or a sub folder of it. I'm not sure if you mean all in the same file? If so I would say generally not unless they're very very simple. Different files, same folder.
3) Not that I'm aware of. If you right click the classname when you use it you can get Studio to automatically resolve it and add a using (Ctrl + . also does this)
When working with legacy code, and trying to create tests, I often break out dependencies from classes or methods so I can write unit tests using mocks for these dependencies. Dependencies most often come in the form of calls to static classes and objects created using the new keyword in the constructor or other locations in that class.
In most cases, static calls are handled either by wrapping the static dependency, or if its a singleton pattern (or similar) in the form of StaticClass.Current.MethodCall() passing that dependency by its interface go the constructor instead.
In most cases, uses of the new keyword in the constructor is simply replaced by passing that interface in the constructor instead.
In most cases, uses of the new keyword in other parts of the class, is handled either by the same method as above, or by if needed create a factory, and pass the factory's interface in the constructor.
I always use Resharpers refactoring tools to help me all of these break-outs, however most things are still manual labour (which could be automated), and for some legacy classes and methods that can be a very very tedious process. Is there any other refactoring plugins and/or tools which would help me in this process? Is there a "break out all depencencies from this class in a single click" refactoring tool? =)
It sounds to me like all these steps are common for many developers and a common problem, and before I attempt writing plugin to Resharper or CodeRush, I have to ask, because someone has probably already attempted this..
ADDED:
In reflection to answers below: even if you might not want to break out everything at once (one click total break out might cause more problems than it helps) still being able to simply break out 1 methods dependencies, or 1-2 dependencies easily, would be of big difference.
Also, refactoring code has a measure of "try and see what happens just to learn how everything fits together", and a one click total break out would help that process tons, even if you dont check that code in..
I don't think there is any tool that can automate this for you. Working with legacy code means -as you know- changing code with little steps at a time. The steps are often deliberately small to prevent errors from being made. Usually the first change you should make is one that makes that code testable. After you've written the test you change that part of the code in such way that you fix the bug or implement the RFC.
Because you should take small steps I believe it is hard to use a refactoring tool to magically make all your dependencies disappear. With legacy systems you would hardly ever want to make big changes at once, because the risk of breaking (and not finding out because of the lack of tests) is too big. This however, doesn’t mean refactoring tools aren’t useful in this scenario. On the contrary; they help a lot.
If you haven't already, I'd advise you to read Michael Feathers' book Working Effectively with Legacy Code. It describes in great details a series of patterns that help you refactor legacy code to a more testable system.
Good luck.
When it comes to static call dependencies, you might want to check out Moles. It's able to do code injection at run-time to stub out any static or non-virtual method call with your own test implementation. This is handy for testing legacy code that wasn't designed using testable dependency-injected interfaces.
I thought it will be a common question so I searched for a while but couldn't find it.
I am about to start a new project (C#, .net 3.5) and I was thinking about where I should I write the unit test code. I can create a unit test project and write all code there, or I can write the unit test code with the "class under test" itself.
What do you recommend and why? Things to consider before choosing an approach (caveats?)?
EDIT: About writing unit-test code with "code under test": Removing the test code from production assembly isn't difficult I guess. Thats what conditional compilation is for. Right?
Just throwing this point because answers are rejecting the second option just because production assemblies would be fatty.
Separate project, same solution. Use InternalsVisibleTo if you want to access internals from the test code.
Separating out test from production code:
makes it more obvious what's what
means you don't need dependencies on test frameworks in your production project
keeps your deployed code leaner
avoids including test data files in your deployment assembly
Keeping the code in the same solution:
keeps the test cycle quick
makes it easy to hop between production and test code
I always create a separate project in where I write my TestFixtures.
I do not want to litter my domain model (or whatever) with Test classes.
I do not want to distribute my tests to customers or users of my application, so therefore I put them in a separate project (which also leads to a separate assembly).
If you have the rare case that you want to test internal methods, you can use InternalsVisibleTo.
If you have the very rare case that you want to test private methods, you can use this technique, as explained by Davy Brion.
I prefer the first approach - separating to unit test to its own project.
placing the unit tests within the test subject will make it dirty. furthermore, you don't necessarily want to distribute your project with the unit tests which will make your dll's bigger and possibly expose things that you don't want to expose to the end user.
most of the open source projects that I saw had a different projects for unit tests.
You shoul place the unit tests in a seperate project.
You should also write them in a way, so that the SUT (System under Test) is not modified in a way to make unittests possible. I mean you should have no helper classes in you main project that exist "only" to support you tests.
Mixing test and production code is allways a bad plan, since you dont want to deliver all that extra code out to your clients. Keep the clear separation that another project offers.
I dont think the "keep the tests quick" argument is a really strong one. Make a clear cut... Testing code does not belong into a production enviroment IMHO...
Edit:
Comment on Edit above:
EDIT: About writing unit-test code with "code under test": Removing the test code from production assembly isn't difficult I guess. Thats what conditional compilation is for. Right?
Yes, it is "easy" to remove the code with a conditional compilation flag, but you wont have tested the final assembly you created, you only tested the assembly you created with the code inside it, then you recompile, creating a new,untested assembly and ship that one. Are you sure all your conditional flags are set 100% correct? I guess not, since you cant run the tests ;)