Develop Reference

Should you reuse system functionality in tests, or be explicit?

When writing tests is it acceptable (or should I) to use functionality from elsewhere in the application to assist in a test.
So as an example, the application I am writing tests for uses the CQRS pattern. A lot of the existing tests make use of these commands, queries and handlers when performing the arrange part of a test. They all have their own test cases so I should be OK to accept they function as expected.
I am curious though if this is best practice or if I should be performing setup during the arrange of a test manually (without using other application functionality)? If one of the commands, queries or handlers breaks, then my 'unrelated' test breaks too? Is this good or bad?

When writing tests is it acceptable (or should I) to use functionality from elsewhere in the application to assist in a test.
There are absolutely circumstances where using functionality from elsewhere is going to have good trade offs.
In my experience, it is useful to think about an automated check as consisting of two parts - a measurement that produces a value, and a validation that evaluates whether that value satisfies some specification.
Measurement actual = measurement(args)
assert specification.isSatisfiedBy(actual)
In the specification part, re-using code is commonplace. Consider
String actual = measurement(args)
assert specification.expected.equals(actual)
So here, we have introduced a dependency on String::equals, and that's fine, we have lots and lots of confidence that String::equals is correct, thanks to the robust distributed test program of everybody in the world using it.
Foo actual = measurement(args)
assert specification.expected.equals(actual)
Same idea here, except that instead of some general purpose type we are using our own bespoke equality check. If the bespoke equality check is well tested, then you can be confident that any assertion failures indicate a problem in the measurement. (If not, well then at least the check signals that measurement and specification are in disagreement, and you can investigate why.)
Sometimes, you'll want to have an explicit dependency on other parts of the system, because that's a better description of the actual requirements. For example, compare
int actual = foo("a")
assert 7 == actual
with
assert 7 == bar(0) // This check might be in a different test
assert bar(0) == foo("a")
At a fixed point in time, these spellings are essentially equivalent; but for tests that are expected to evaluate many generations of an evolving system, the verification is somewhat different:
// Future foo should return the same thing as today's foo
assert 7 == foo("a")
// Future foo should return the same thing as future bar
assert bar(0) == foo("a")
Within measurements, the tradeoffs are a bit different, but because you included cqrs I'll offer one specific observation: measurements are about reads.
(Sometimes what we read is "how many times did we crash?" or "what messages did we send?" but, explicit or implicit, we're evaluating the information that comes out of our system).
That means that including a read invocation in your measurement is going to be common, even in designs where you have decoupled reads from writes.
A lot of the existing tests make use of these commands, queries and handlers when performing the arrange part of a test.
Yup and the answer is the same - we're still talking about tradeoffs: does the test detect the problems you want it to? how expensive is it to track down the fault that was detected? How common are false positives (the "fault" is in the test itself, not the test subject)? How much future work are you signing up for just to "maintain" the test (which is related, in part, to how "stable" the dependencies are) during its useful lifetime.

How should I assert in WebAPI integration test?

I'm writing integration tests for Controller classes in my .NET Core WebAPI, and honestly, I'm not sure if Assert.Equal(HttpStatusCode.OK, response.StatusCode); is enough. I'm using my production Startup class with only database different (using InMemory database).
This is one of my tests, named ShouldAddUpdateAndDeleteUser. It's basically:
Send POST request with specific input.
Assert If POST worked. Because POST sends respond with created object, I can Assert on every property and if the Id is greater then 0.
Change the output a little bit and send Update request
Send GET request and assert if update worked.
Send DELETE request
Send GET request and assert if null.
Basically, I test, ADD,UPDATE,DELETE,GET (when item exists), GET (when item doesn't exists).
I have a few questions:
Is it a good practice to have such tests? It does do a lot, but it's not a unit test after all. If it fails, I can be pretty specific and specify which part didn't work. Worst case scenario I can debug it pretty quickly.
Is it integration tests or functional test or neither?
If this is wrong, how can I test DELETE or UPDATE? I'm kinda forced to call GET request after them (They return NoContent)
(Side note: It's not the only test I have for that controller obviously. I also have tests for GET all as well as BedRequest requests)

Is it a good practice to have such tests? It does do a lot, but it's not a unit test after all. If it fails, I can be pretty specific and specify which part didn't work. Worst case scenario I can debug it pretty quickly.
Testing your endpoints full stack does have value but make sure to keep in mind the testing pyramid. Its valuable to have a small number of full stack integration to validate the most important pathways/use cases in your application. Having too many however, can add a lot of extra time to your test runs which could impact release times if those tests are tied to releasing (which they should be).
Is it integration tests or functional test or neither?
Without seeing more of the code I would guess it's probably a bit of both. Functionally, you want to test the output of the endpoints and it seems as though you're testing some level of component integration if you're mocking out data in memory.
If this is wrong, how can I test DELETE or UPDATE? I'm kinda forced to call GET request after them (They return NoContent)
Like I said earlier, a few full stack tests are fine in my opinion as long as they provide value for major workflows in your application. I would suggest one to two integration tests that make database calls that make sure your application can stand up full stack but not bundle that requirement for your functional testing. In my experience you get way more value from modular component testing and a couple end to end functional tests.
I think the test flow you mentioned makes sense. I've done similar patterns throughout my services at work. Sometimes you can't realistically test one component without testing another like your example about getting after a delete.

Strategies for Class/Schema aware test data generation for Data Driven Tests

I've recently started pushing for TDD where I work. So far things are going well. We're writing tests, we're having them run automatically on commit, and we're always looking to improve our process and tools.
One thing I've identified that could be improved is how we set up our Test Data. In strictly unit tests, we often find ourselves instantiating and populating complex CLR objects. This is a pain, and typically the test is then only run on a handful of cases.
What I'd like to push for is Data Driven tests. I think we should be able to load our test data from files or maybe even generate them on the fly from a schema (though I would only consider doing it on the fly if I could generate every possible configuration of an object, and that number of configurations was small). And there is my problem.
I have yet to find a good strategy for generating test data for C# CLR objects.
I looked into generating XML data from XSDs and then loading that into my tests using the DataSourceAttribute. The seemed like a good approach, but I ran into troubles generating XSD files. xsd.exe falls over because our classes have interface members. I also tried using svcutil.exe on our assembly, but because our code is monolithic the output is huge and tricky (many interdependent .xsd files).
What are other techniques for generating test data? Ideally the generator would follow a schema (maybe an xsd, but preferably the class itself), and could be scripted.
Technical notes (not sure if this is even relevant, but it can't hurt):
We're using Visual Studio's unit testing framework (defined in Microsoft.VisualStudio.TestTools.UnitTesting).
We're using RhinoMocks
Thanks
Extra Info
One reason I'm interested in this is to test an Adapter class we have. It takes a complex and convoluted legacy Entity and converts it to a DTO. The legacy Entity is a total mess of spaghetti and can not be easily split up into logical sub-units defined by interfaces (as suggested). That would be a nice approach, but we don't have that luxury.
I would like to be able to generate a large number of configurations of this legacy Entity and run them through the adapter. The larger the number of configurations, the more likely my test will fail when the next developer (oblivious to 90% of the application) changes the schema of the legacy Entity.
UPDATE
Just to clarify, I am not looking to generate random data for each execution of my tests. I want to be able to generate data to cover multiple configurations of complex objects. I want to generate this data offline and store it as static input for my tests.
I just reread my question and noticed that I had in fact originally ask for random on the fly generation. I'm surprised I ask for that! I've updated the question to fix that. Sorry about the confusion.

What you need is a tool such as NBuilder (http://code.google.com/p/nbuilder).
This allows you to describe objects, then generate them. This is great for unit testing.
Here is a very simple example (but you can make it as complex as you want):
var products = Builder<Product>
.CreateListOfSize(10)
.All().With(x => x.Title = "some title")
.And(x => x.AnyProperty = RandomlyGeneratedValue())
.And(x => x.AnyOtherProperty = OtherRandomlyGeneratedValue())
.Build();

In my experience, what you're looking to accomplish ends up actually being harder to implement and maintain than generating objects in code on a test-by-test basis.
I worked with a client that had a similar issue, and they ended up storing their objects as JSON and deserializing them, with the expectation that it would be easier to maintain and extend. It wasn't. You know what you don't get when editing JSON? Compile-time syntax checking. They just ended up with tests breaking because of JSON that failed to deserialize due to syntax errors.
One thing you can do to reduce your pain is to code to small interfaces. If you have a giant object with a ton of properties, a given method that you'd like to test will probably only need a handful. So instead of your method taking SomeGiantClass, have it take a class that implements ITinySubset. Working with the smaller subset will make it much more obvious what things need to be populated in order for your test to have any validity.
I agree with the other folks who have said that generating random data is a bad idea. I'd say it's a really bad idea. The goal of unit testing is repeatability, which goes zooming out the window the second you generate random data. It's a bad idea even if you're generating the data "offline" and then feeding it in. You have no guarantee that the test object that you generated is actually testing anything worthwhile that's not covered in other tests, or if it's testing valid conditions.
More tests doesn't mean that your code is better. 100% code coverage doesn't mean that your code is bug-free and working properly. You should aim to test the logic that you know matters to your application, not try to cover every single imaginable case.

This is a little different then what you are talking about, but have you looked at Pex? Pex will attempt to generate inputs that cover all of the paths of your code.
http://research.microsoft.com/en-us/projects/Pex/

Generating test data is often an inappropriate and not very useful way of testing - particuarly if you are generating a different set of test data (eg randomly each time) as sometimes a test run will fail and sometimes it wont. It also may be totally irrelevant to what your doing and will make for a confusing group of tests.
Tests are supposed to help document + formalise the specification of a piece of software. If the boundaries of the software are found through bombarding the system with data then these wont be documented properly. They also provide a way of communicating through code that is different from the code itself and as a result are often most useful if they are very specific and easy to read and understand.
That said if you really want to do it though typically you can write your own generator as a test class. I've done this a few times in the past and it works nicely, with the added bonus that you can see exactly what it's doing. You also already know the constraints of the data so there's no problem trying to generalise an approach
From what you say the pain you are having is in setting up objects. This is a common testing issue - I'd suggest focusing on that by making fluent builders for your common object types - this gives you a nice way of filling in less detail every time (you typically would provide only the interesting data (for a given test case) and have valid defaults for everything else). They also reduce the number of dependencies on constructors in test code which means your tests are less likely to get in the way of refactoring later on if you need to change them. You can really get a lot of mileage out of that approach. You can further extend it by having common setup code for builders when you get a lot of them that is a natural point for developers to hang reusable code.
In one system I've worked on we ended up aggregating all these sorts of things together into something which could switch on + off different seams in the application (file access etc), provided builders for objects and setup a comprehensive set of fake view classes (for wpf) to sit on top of our presenters. It effectively provided a test friendly interface for scripting and testing the entire application from very high-level things to very low-level things. Once you get there you're really in the sweet spot as you can write tests that effectively mirror button clicks in the application at a very high level but you have very easy to refactor code as there are few direct dependencies on your real classes in the tests

Actually, there is a Microsoft's way of expressing object instances in markup, and that is XAML.
Don't be scared with the WPF paradigm in the documentation. All you need to do is use correct classes in unit tests to load the objects.
Why I would do this? because Visual Studio project will automatically give you XAML syntax and probably intellisense support when you add this file.
What would be a small problem? markup element classes must have parameterless constructors. But that problem is always present and there are workarounds (e.g. here).
For reference, have a look at:
Create object from text in XAML, and
How to convert XAML File to objects, and
How to Deserialize XML document.
I wish I could show you something done by me on this matter, but I can't.

C# application talking with ruby msgpack server which is expecting a ruby-type hash?

I am looking to build a small application to talk with a ruby msgpack server in C#. My only holdup so far is that the API behind the server is expecting to pull out a ruby hash. Can I use a simple dictionary/key-value pair type in C#? If not, what would you suggest?
I will be using the library mentioned on the msgpack website (http://wiki.msgpack.org/display/MSGPACK/QuickStart+for+C+Sharp). However, it only seems to support primitive types? I have tried to go the IronRuby way, however there is a very crippling bug in mono that prevents you from using it. https://bugzilla.xamarin.com/show_bug.cgi?id=2770

It is normal that different part of the system can be built using different technology stacks. Because these parts should be able to talk to each other (this way or another) it is important to specify contracts between subsystems.
It is really important to think first about these contracts as these parts of your system (subsystems) can be (and will be, no doubt) subjects of changes (due to evolving their business logic, bug fixes, etc.).
By having these contracts you allow subsystems to be changed independently without impacting all their "clients" (other subsystems). Otherwise you will end up with "I need to fix this, but may affect tonnes of places I even don't know about" syndrome.
Well, as soon as you hold the contract you can do whatever you want within the given subsystem, which is just a Heaven! :)
This means that instead of "pulling out the ruby hash" you normally want to define a platform-agnostic contract that will be exposed as an aspect in terms of the business logic of your application. This contract then can be consumed by any other subsystem written in any technology.
It also means that instead of just passing some data between subsystems you want to pass some objects. These objects not only contain the data you want to pass, but also describe this data, dive it some meaning. By this "description" I mean the object type, property names, ect. Objects are self-descriptive, you know.
You may declare the contract for your ruby subsystem saying "I accept these queries and I return these results". Both query (method) and result (object) should be formulated in terms of business logic of the specified subsystem. For example, GetProducts contract should probably return a list of Product objects, not some meaningless "ruby hashes". So all the consumers will know what the contract is and what to expect.
You can make it a standard then, saying "between subsystems all the objects passed are serialized to JSON (or XML)", which is more than trivial in Ruby, C# or any other language, as well as truly platform-agnostic.
Therefore, back to your question, you normally just have no such problem in your live as translating ruby types into .NET types using some buggy libraries, or doing similar crazy things :)
Simply defining contracts and standardizing transport (JSON?) helps you in many ways starting from getting rid of this problem and all the way through to having the clean and easily maintainable system.

Testing classes with threads, events, and private methods

general consensus
I've done quite a lot of reading up on the subject of testing complex classes and private methods.
The general consensus seems to be:
"if you need to test private methods then you're class is badly designed"
"if your class is complex, then you need to separate it out"
So, I need your help.
the problem Class
So I have a relatively simple class whose long running job it is to:
poll a datasource
do some very simple mapping of the data
send that data somewhere else
Aditionally:
it needs to be able to be quite fault tolerant by being able to retry various tasks in case of certain errors.
the testing problem
The point of the class is to abstract a lot of the fault tolerance and threading... basically by using a simple Timer Class and some internal lists to keep track of errors etc.
Because of the Timer, certain methods are called on different threads asynchronously... additionally a bunch of the methods rely on global private fields.
How should I test this class... particularly because so many methods are private?
cheers guys

I would extract the code to poll the data into a separate class that can be mocked, and also extract the code that sends that data for the same reason. You might want to extract the data mapping code, depending on how trivial it is.
I would definitely use mock timers in the unit tests, otherwise your tests are hard to set up and slow to run. You can either pass in the timer in the constructor, or expose a property that you can set. I often create a regular timer in the constructor and then overwrite that from my unit test.
You might also be able to extract the retry logic so that it can be tested separately from the other code. Passing in a delegate of the code to try and retry might be a way to decouple the data code from the retry logic. You can also use IEnumerable and the yield statement to generate your data and feed it to the retry code. Essentially, I'm looking for ways to make it so that the retry code doesn't directly call the target code that it's supposed to try and retry. That makes it easier to test and generate all possible errors for, although you can get some of the same benefits by just mocking out that target code.
If you really have multithreaded scenarios that you need to test, there are some tools out there to coordinate threads from within a test. One of them is a port I created of Java MultithreadedTC called TickingTest.

You could try using something like JMock. This will let you replace the real Timer with a mock Timer that you can control. You can then set up test cases that fire method calls in a defined order, and you can also set up error conditions by creating a mock data source.
EDIT: Whoops! Didn't see the C# tag. Maybe there's a C# equivalent to JMock.