Develop Reference

Is it possible to use xUnit with LINQPad?

Is it possible to use xUnit with LINQPad?
It would be great to be able to write some tests for concepts being designed in LINQPad first. It's easier than adding yet another ConsoleApp23423894238 just to be able to quickly write some unit tests.

Yes, it is possible to use xUnit with LinqPad. I have updated this answer to give you the latest information, but the original answer I gave also gives you some insights and links regarding the usage of xUnit, so I kept it for your reference.
UPDATE: Latest version of LinqPad 6 or 7 (since V 6.9.x and higher, see 6.9 Release Notes section) has now built-in XUnit support via the new menu item Query / Add XUnit Test Support. This will add a query with required boilerplate code referenced via #load so you can start testing facts and theories instantly. You can run the example test method by pressing Alt+Shift+T.
For example, add the following new theory to the boilerplate code (right after the main method, put it inside the private::Tests region):
#region private::Tests
[Theory]
[InlineData(1)]
[InlineData(3, 6)]
[InlineData(5, 7, 9)]
void OddNumbers(params int[] values)
{
static bool IsOdd(int value) => value % 2 == 1;
foreach (var value in values)
{
Assert.True(IsOdd(value), $"IsOdd({value}) is false");
}
}
#endregion
Press Alt+Shift+T to run all tests and check the result.
Note: It also adds a file XUnit.linq to your query directory the first time you add XUnit Test support - this is a mini library to support xUnit tests and you should not touch it unless you know exactly what you're doing. For example, you can modify this file to change the way the test summary is being displayed (for example, you can add the parameter Func<UserQuery.TestResultSummary, bool> filter = null to the method RunTests and add .Where(filter) inside the query if filter is not null. This allows you to filter the results of the test summary).
For example: RunTests(filter: f => f.MethodName.StartsWith("Unit")); will only display the tests in the summary which names start with "Unit".
Dynamically skippable unit tests
You can modify the xunit script LinqPad 6 provides to support skippable unit tests (NUGET package XUnit.SkippableFact, detailed description see the link "dynamically skipping test and theory" at the end of this answer).
Make the following changes in xunit:
function RunTests - add:
runner.OnTestSkipped = info => AddTestResult(info);
class TestResultSummary - add:
public bool Skipped() => _testInfo is Xunit.Runners.TestSkippedInfo;
class TestResultSummary / public object Status - add:
_testInfo is Xunit.Runners.TestSkippedInfo ? Util.WithStyle ("Skipped", "color:orange") :
Those changes will enable the xunit script to show when a test was skipped in the results.
To test it, load the NUGET package XUnit.SkippableFact via F4, then add the test framework and finally add the tests:
[SkippableFact]
void Test_Xunit3()
{
Skip.If(true, "skipping");
Assert.True(1 + 1 == 2);
}
[SkippableTheory]
[InlineData(1)]
[InlineData(2)]
void Test_Xunit4(int x)
{
Skip.If(x==2, "skipping");
Assert.True(1 + 1 == 2);
}
Note: Those changes will also work with the plain XUnit. So adding the changes to the code does not harm.
Now if you like, continue reading the original answer (with some useful hints at the end):
Two answers pointed me into the right direction, but the descriptions haven't been complete enough, so I had to experiment a bit. To save you that time, I am describing what I did; I have now working solutions for LinqPad 5 and LinqPad 6 (and 7).
LinqPad 5 - for .NET
Based on what Tom wrote, I was able to get it working with LinqPad 5 (v5.42.01).
There are some more steps required, which aren't obvious and not explained in the accepted answer, so I'll describe what helped on my side.
First of all, you will need to use the paid version of LinqPad, because only then you are able to load NUGET packages.
To load the required packages in LinqPad, press F4. The query properties dialog opens up. In that dialog, click Add NUGET....
The NUGET dialog opens up. In the middle of the NUGET dialog, there is a search textbox. Search for xUnit.Net - once it got displayed, click Add to query, wait until it loaded, then click on Add namespaces. Then do the same for Xunit.Runner.LinqPad.
Then close the NUGET dialog, but keep the query properties open. You need to to there to the Advanced tab: Enable (tick-mark) "Copy all non-framework references to a single local folder." Now you can close the query properties dialog.
Import the following example into the query editor (C# Program mode):
void Main()
{
// Press F4, go to Advanced, then
// tick-mark "Copy all non-framework references to a single local folder"
var me = Assembly.GetExecutingAssembly();
var result = Xunit.Runner.LinqPad.XunitRunner.Run(me);
result.Dump("Xunit runner result");
}
/// <summary>
/// This test class is there for demo purpose, so developers can
/// see how to write test methods.
/// It also verifies that XUnit itself is functioning correctly.
/// </summary>
public class A_XUnitSelfTest
{
[InlineData(2)]
[InlineData(3)]
[Theory]
public void Test_Theory(int a)
{
Assert.True(a == 2 || a == 3); // succeeds
}
[Fact]
public void Test_Fact()
{
Assert.False(1 == 0); // succeeds
}
}
Run it and you will see the output of the test runner.
Running 3 of 3 tests...
Finished: 3 tests in 0,003s (0 failed, 0 skipped)
Modify the test to see how it looks like if a test fails by changing the assert in the Test_Fact() to Assert.False(1 == 1);, then run it again. It should show this time:
Running 3 of 3 tests...
[FAIL] UserQuery+A_XUnitSelfTest.Test_Fact: Assert.False() Failure
Expected: False
Actual: True
at Xunit.Assert.False(Nullable`1 condition, String userMessage)
at Xunit.Assert.False(Boolean condition)
at UserQuery.A_XUnitSelfTest.Test_Fact() in C:\Users...\AppData\Local\Temp\LINQPad5_oyxxqxbu\query_mxsmky.cs:line 62
Finished: 3 tests in 0,005s (1 failed, 0 skipped)
LinqPad 6 - for .NET Core
LinqPad 6 is different, because it is based on .NET Core and hence does require different libraries.
Start with this:
void Main()
{
// Press F4, go to Advanced, then
// tick-mark "Copy all non-framework references to a single local folder"
var me = Assembly.GetExecutingAssembly();
var result = Xunit.Runner.LinqPad.XunitRunner.Run(me);
result.Dump("Xunit runner result");
}
/// <summary>
/// This test class is there for demo purpose, so developers can
/// see how to write test methods.
/// It also verifies that XUnit itself is functioning correctly.
/// </summary>
public class A_XUnitSelfTest
{
[InlineData(2)]
[InlineData(3)]
[Theory]
public void Test_Theory(int a)
{
Assert.True(a == 2 || a == 3); // succeeds
}
[Fact]
public void Test_Fact()
{
Assert.False(1 == 0); // succeeds
}
}
Then add xUnit.net, xUnit.net [Core Unit Testing Framework] and xUnit.net [Runner Utility] and all their namespaces via the F4 dialog (details see LinqPad 5 sesion) and finally copy the XUnit runner source code as mentioned by #ramiroalvarez to the end of the xUnit test example above:
XUnit runner source code
Enable "copy all nuget assemblies to a single local folder" in the F4 dialog, Advanced tab ... that should be enough, but it isn't so read on:
If you try it, it throws the exception InvalidOperationException("Please enable the shadow folder option for none-framework references (F4 -> Advanced).") in the source code.
Now I looked into the XUnit runner, and found that if you comment out 2 lines of code then it is running fine:
Original code (located in method GetTargetAssemblyFilename):
if (shadowFolder != xunitFolder
|| Directory.GetFiles(shadowFolder, "xunit.execution.*.dll").Length == 0)
{
throw new InvalidOperationException("Please enable the shadow folder option ...");
}
// ...
var targetAssembly = Path.Combine(shadowFolder, Path.GetFileName(assemblyFilename));
//Console.WriteLine($"Copy \"{ assemblyFilename }\" -> \"{ targetAssembly }\"");
File.Copy(assemblyFilename, targetAssembly, true);
Comment the lines
// ...
// throw new InvalidOperationException("Please enable the shadow folder option ...");
// ...
// File.Copy(assemblyFilename, targetAssembly, true);
and you're done! (Tested with v6.8.3 version of LinqPad 6, X64 version)
Hints:
Hint 1:
You can append the xUnit runner to your code as described, but there is a more elegant way in LinqPad 6:
Save a copy containing only the xUnit.Runner source code in the same path as your unit test files with the file name xUnit.Runner.LinqPad6.linq.
Then, add the following to the top of your test code:
Now you have cleaned up your code! Every time you create a new test, just remember to add the #load command.
Hint 2:
To be able to skip facts statically, add the following 2 attributes to the code of the test runner:
/// <summary>
/// Skip a Fact (you can override it by setting OverrideSkip=true)
/// </summary>
public class SkipFactAttribute : FactAttribute
{
/// <summary>
/// Override SkipFact: Set OverrideSkip=true to run even skipped tests
/// </summary>
public static bool OverrideSkip = false; // set to true to ignore skip
/// <summary>
/// Constructor. Used to set skip condition.
/// </summary>
public SkipFactAttribute()
{
if (!OverrideSkip) Skip = "Skipped Fact";
}
}
/// <summary>
/// Skip a Theory (you can override it by setting OverrideSkip=true)
/// </summary>
public class SkipTheoryAttribute : TheoryAttribute
{
/// <summary>
/// Override SkipTheory: Set OverrideSkip=true to run even skipped tests
/// </summary>
public static bool OverrideSkip = false; // set to true to ignore skip
/// <summary>
/// Constructor. Used to set skip condition.
/// </summary>
public SkipTheoryAttribute()
{
if (!OverrideSkip) Skip = "Skipped Theory";
}
}
This is useful, if you want to skip tests. To skip a test, simply replace [Fact] by [SkipFact] or [Theory] by [SkipTheory].
If you're busy writing a test and you want to test only that single test, set all attributes to [SkipFact] or likewise [SkipTheory], except the single one you're writing currenlty.
Then, to quickly run all the tests again from time to time - without having to change each and every single attribute - add the following to the initialization part of your test suite (see global setup for startup and teardown):
public class TestsFixture : IDisposable
{
public TestsFixture() // running once before the tests
{
// true: Ignore Skip and run all tests, false: Skip is effective
SkipFactAttribute.OverrideSkip = true;
SkipTheoryAttribute.OverrideSkip = true;
}
public void Dispose() // running once, after the tests
{
// teardown code
}
}
If you're done, restore the [Fact]/ [Theory] attributes and set the .OverrideSkip variables back to false.
NOTE: In case you want to skip facts based on a runtime condition, take a look at dynamically skipping test and theory.
More about unit testing
For some advanced topics about unit testing with xUnit, I recommend to read:
xunit tests in net core
global setup for startup and teardown
shared context
running tests in parallel
running tests in prioritized, specified order
configuring with json
configuring with xml
skipping Test and Theory // description works for TheoryAttribute too
dynamically skipping test and theory
xUnit Theory: Working With InlineData, MemberData, ClassData
If you want to use it with Visual Studio, you can find infos here.

Possible? Sort of.
This is neither a main use case of LINQPad nor of unit testing.
Regarding LINQPad, you could use an existing test runner but probably the only ones that could be integrated with LINQPad are console runners. A test runner is not a trivial program. Try Xunit.Runner.LinqPad.
As for needing to "write some tests for concepts being designed", consider that you might not be getting enough out of your unit testing workflow if you don't find them valuable enough to create a project and keep it in source control. One way to get more value is to use Behavior-Driven Development (BDD) and if you want to write the requirements in a business-readable language, try something like SpecFlow.

Thanks to asherber, xunit now works with linqpad6 (netcore).
https://github.com/asherber/Xunit.Runner.LinqPad/blob/master/Xunit.Runner.LinqPad/XunitRunner.cs
It is important within the query properties, check "copy all nuget assemblies to a single local folder" otherwise, you will have the following error "InvalidOperationException: Please enable the shadow folder option for none-framework references"

I just tried to open a LINQPad query file that tried to load the xunit package and got an error which has never happened to me in using LINQPad for years. So my answer is no.

Dynamically create nunit test cases based off config file

I'm looking for a way to dynamically create nunit test cases based off config file. In my App.config, I have a list of filenames and its expected operation. I would like to be able to add a new entry and have a test case be created dynamically based off that. My code is as follows:
App.config
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<appSettings>
<!--syntax is filename#operation where 0 = Valid File, 1 = Missing Column, 2 = Invalid File-->
<add key="FILENAME_0" value="C:\Users\me\Desktop\file_0.csv#0" />
<add key="FILENAME_1" value="C:\Users\me\Desktop\file_1.csv#1" />
<add key="FILENAME_2" value="C:\Users\me\Desktop\file_2.csv#2" />
</appSettings>
</configuration>
In my test fixture, I initialize a list of test case data, which is created by parsing the App.config file as follows:
[TestFixture]
public class MyTests
{
public enum Operation
{
ValidFile = 0,
MissingColumns = 1,
InvalidFile = 2
};
/// <summary>
/// Gets test case data with the file name and an enum of the expected operation.
/// </summary>
private static IEnumerable<TestCaseData> FilenameCases()
{
int i = 0;
bool moreFilesToTest = true;
var files = new Dictionary<string, Operation>();
while (moreFilesToTest)
{
string filenameAndOp = ConfigurationManager.AppSettings[$"FILENAME_{i}"];
if (filenameAndOp == null)
moreFilesToTest = false;
else
{
string filename = filenameAndOp.Split('#')[0];
int operation = Int32.Parse(filenameAndOp.Split('#')[1]);
Operation op = (Operation)operation;
files.Add(filename, op);
}
i++;
}
foreach (var pair in files)
yield return new TestCaseData(pair.Key, pair.Value);
}
[Test, TestCaseSource("FilenameCases")]
public void ShouldLoadFiles(string FILENAME, Operation expected)
{
// ... run test cases
}
}
This works, however, would need to rebuild the project each time in order for any changes in the App.config to be recognized. The code also only works if the TestCaseData is static. What is the correct way to have new test cases be created dynamically based off the configuration file.

After researching this post, I have found out that the issue is not with the code, but rather the the configuration file. As I was running the test via the command line, the test .dll was not referencing my App.config, but another file.
When I compiled my project, I noticed it produced a config file based off the name of the project. So a test project titled MyTestProject would produce a file called MyTestProject.dll.config. Therefore, when I made changes to that config file, my test cases reflected the changes, without recompiling the project.
Credit to both Charlie and Amittai Shapira, as NUnit doesn't support dynamic test cases and TestCaseSource needs to be static. However, the code I tried was the best work around, as the end result did produced dynamic test cases.

TestCaseSource methods need to be static. There are a few ways around this if you need it but I see no problem with using static in your case. Just because a method is static, that doesn't mean it has to always return the same values.
There is no reason for you to recompile your code when you change the config file. However, you have to reload the test assembly in order for the new values in the config to be read. In some environments, like under VS, it may be easier to just recompile in order to force that to happen.
Essentially, I'd say you have already found the best way to do this, although my personal tendency would be to use a flat file rather than the config file.

When multiple unit tests copy the same file, running all unit tests fail

Description
I am writing unit tests for a method, which copies a file from a source to a destination. Basically it includes this code:
public void MyMethod()
{
// ...
File.Copy(source, destination, true);
// ...
}
In my unit test project, I have a test file: (test.png), which is located in the Resources folder of my unit test project. And I've set the Copy to Output property to Always.
I have 3 unit tests which are testing this method.
When they hit the line of the code which copies the file: source = "Resources\\test.png".
Issue
When I run the unit test individually, they all pass and everything is fine.
However, when I run All Tests in Visual Studio, I get this run time error and unit tests fail:
System.IO.DirectoryNotFoundException
Could not find a part of the path 'Resources\test.png'.
My Thoughts...(UPDATED)
Probably because Visual Studio runs each unit tests simultaneously in a separate thread and they all accessing the same file at the same time?
I think for every unit test, Visual Studio is cleaning bin/Debug and bin/Release folders. Then it copies all the required project files in that folder. This causes sometimes the file actually does not exist?
Question
How can I fix this problem?
Is there any settings of configurations to resolve this?
How can I run all unit tests in Visual Studio (and Team City) when multiple unit tests are accessing the same file?

You could try to rule out the multi-threading issue by following the instructions from MSDN: Executing Unit Tests in parallel on a multi-CPU/core machine, setting parallelTestCount to 1. If the tests now pass, you've narrowed down the problem.
However, if your tests are still failing when you run them in a group - and I think this is the more likely scenario -, then my advice would be to check for any state those tests are sharing. The pattern you describe (i.e. passes in isolation; fails when not in isolation) is a symptom typically exhibited by tests that are (incorrectly) sharing state, and that state is being modified by those tests, causing one or more tests to fail.

Accessing the same file should not be a problem. Make sure you don't have a cleanUp Fixture(TestSuite level) to delete the file. Because from exception it looks like the file is being deleted after running a test.
Also concurrent read operation is fine and perfectly legal. If your unit tests are overwriting the file then it's a problem.

What happened was, since I was using relative path to the testing files, for some reason when running the unit tests in batch, test runner working directory is different than when running individual tests, hence it couldn't find the directory.
So I used this function to build the absolute path to the testing files:
private string GetFilePath([CallerFilePath] string path = "")
{
return path;
}
Then:
string projectDir = Path.GetDirectoryName(GetFilePath());
string testFile = Path.Combine(projectDir, #"Resources\test.png";

I'd speculate that your problem is that one of the tested methods changes directory, given the explicit "Directory Not Found" exception. It's improbable that file locking or any concurrency problems would cause the behaviour described.

If unit testing you shouldn't really be testing whether File.Copy (or any of the File class methods) worked since you didn't write that code. Instead you should test whether your code interacts correctly with the File type (i.e. did it pass the correct source file name, desination file name and overwrite value when you called "Copy"). First create an interface for the File class and a wrapper for it that implements the interface;
public interface IFileWrapper
{
void Copy(string sourceFileName,string destFileName,bool overwrite);
//Other required file system methods and properties here...
}
public class FileWrapper : IFileWrapper
{
public void Copy(string sourceFileName, string destFileName, bool overwrite)
{
File.Copy(sourceFileName, destFileName, overwrite);
}
}
You should then make the class you are testing include an IFileWrapper parameter (dependency injection). In your unit tests you can then use a mocking framework such as Moq or you could write your own mock;
public class MockFileWrapper : IFileWrapper
{
public string SoureFileName { get; set; }
public string DestFileName { get; set; }
public bool Overwrite { get; set; }
public void Copy(string sourceFileName, string destFileName, bool overwrite)
{
SoureFileName = sourceFileName;
DestFileName = destFileName;
Overwrite = overwrite;
}
}
In real implementations pass in FileWrapper as the IFileWrapper parameter but in your unit tests pass in MockFileWrapper. By checking the properties of mockFileWrapper in your unit tests you can now determine whether you class calls Copy and how it is called. Since you are no longer sharing a real file between your unit tests you will avoid the chance of the tests sharing state or potentially locking the file.

As you mentioned in your answer, the test framework does not always run tests with the working directory set to your build output folder.
To instruct the test framework to place build artifacts or other files from your build output into the test directory, you need to use DeploymentItemAttribute. For your case, you would do something like:
const string destination = "Destination.txt";
const string source = "MyData.txt";
[DeploymentItem(source)]
[TestMethod]
public void MyMethod()
{
// …
File.Copy(source, destination, true);
// …
}
[TestCleanup]
public void Cleanup()
{
// Clean up the destination so that subsequent tests using
// the same deploy don’t collide.
File.Delete(destination);
}
Also ensure that your files are marked with a Build Action of Contents and Always Copy. Otherwise, they won’t be in the build output directory and won’t be able to be copied to the test directory.

Visual Studio: Revoke reading permission of test file for Unit Testing

I am working on a Visual Studio (Community 2015) project in C# that reads metadata from video files. Therefore I want to write a Unit Test that tests if the program handles an inaccessible file the right way. So I want to add a test file to my test data for which the Unit Test does not have read permissions.
But what is the best way to do this? I didn't find any option to set permissions in Visual Studio. The only idea I have at the moment is to revoke reading permissions directly in the properties of the file. Through the Windows File Explorer, thus outside of Visual Studio. But then I wouldn't be able to read the file myself. Also, it doesn't seem to be the right way and I think there should be a more clean way to solve this problem directly in Visual Studio.
Any ideas?

If you are writing a unit test then it should just check if your code is written correcly and not about the external issues which are difficult to replicate or may not be available on other machines. So I think you should read about Dependency Injection and how to do mock dependencies in your test. I can give you an easy to understand example but please do write it in your words.
public interface IFileReader
{
string ReadFile( string filePath );
}
public class FileReader : IFileReader
{
public string ReadFile( string filePath )
{
return System.IO.File.ReadAllText( filePath );
}
}
So Suppose you have a class VideoMetaDataReader then in that class you will inject the interface as a dependency and use the ReadFile method to read. And then in your test
[TestFixture]
public class FileReaderTests
{
[Test]
public void ShouldDisplayANiceMessage_WhenFileIsInaccessible()
{
var moq = new Moq.Mock<IFileReader>();
moq
.Setup( x => x.ReadFile( Moq.It.IsAny<string>() ) )
.Throws<Exception>();
var metaDataReader = new MetaDataReader( moq.Object );
metaDataReader.ReadVideoFile( "video.mp4" );
Assert.AreEqual( 1, meaDataReader.Errors.Count );
}
}
I am assuming there will be a errors property which will return encountered errors but it depends how you want to do it (I didn't think too much on this).
Anyways, the point is don't test what is not covered by your code. What your code needs to do is to display a nice message if there is an exception and that you can do if you mock the interface (which means don't call the actual implementation and instead do something which I want in this test) and then check if your code can handle that. btw i used nunit and moq for test and mocking.

Generating methods in design or in build time (C#)

I have an integration testing solution. I have my tests described in XML files. In order to capitalize on Visual Studio 2010 testing infrastructure, I have a C# class where every XML test file has an associated method that loads the XML file and executes its content. It looks like this:
[TestClass]
public class SampleTests
{
[TestMethod]
public void Test1()
{
XamlTestManager.ConductTest();
}
[TestMethod]
public void Test2()
{
XamlTestManager.ConductTest();
}
...
[TestMethod]
public void TestN()
{
XamlTestManager.ConductTest();
}
}
Each method name corresponds to an XML file name. Hence, I have to have the following files in my test directory:
Test1.xml
Test2.xml
...
TestN.xml
XamlTestManager.ConductTest() uses the StackTrace class to get the name of the calling method and this way it can find the correct XML test file to load.
I would like to get rid of the extra administration of adding/removing/renaming test methods whenever I change my tests, adding/removing/renaming an XML test file. How can I automagically generate this class or its methods during the compilation process based on the actual XML files in my test directory?
Option 1:
I have considered PostSharp, but it does not allow me to look up the XML files and generate methods on the fly (or was I superficial?).
Option 2:
The other idea was to build a Visual Studio custom tool that generates my code whenever it is executed. The downside here is the deployment. The custom tool needs to be registered to VS. I want a solution that can be committed into a repository, check it out to another computer and use it right away.
(I believe in simplicity. "Check out and run" just simplifies the life of new developers soooooo much, if they do not need to go through a list of thing to install before they can compile run the application.)
Do you have any recommendation, how to get rid of the unnecessary maintenance issue?
EDIT:
For the request of Justin, I add more details. We use Bizunit (fantastic!!!) as the basis of our framework with a truckload of custom made high level test steps. From these steps we can build our test like from lego blocks in a declarative manner. Our steps include like FileDrop, WebService invokation or even polling, firing up a full blown web server to simulate a partner web application, random data generator, data comparing steps etc. Here is an example test xml (in fact XAML):
<TestCase BizUnitVersion="4.0.154.0" Name="StackOverflowSample" xmlns="clr-namespace:BizUnit.Xaml;assembly=BizUnit" xmlns:nib="clr-namespace:MyCompany.IntegrationTest;assembly=BizUnit.MyCustomSteps" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
<TestCase.SetupSteps>
<nib:ClearStep FailOnError="True" RunConcurrently="False" />
<nib:LaunchSimulatedApp AppKernelCacheKey="provider" FailOnError="True" FireWakeUpCall="False" PortNumber="4000" RepresentedSystem="MyProviderService" RunConcurrently="False" />
<nib:HttpGetStep FailOnError="True" RunConcurrently="False" Url="http://localhost:10000/Home/StartSvgPolling">
<nib:HttpGetStep.Parameters>
<x:String x:Key="PolledAddress">http://localhost:4000/SvgOutputPort.asmx</x:String>
<x:String x:Key="PollingInterval">10</x:String>
<x:String x:Key="FilterFile"></x:String>
</nib:HttpGetStep.Parameters>
</nib:HttpGetStep>
</TestCase.SetupSteps>
<TestCase.ExecutionSteps>
<nib:DocumentMergeStep FailOnError="True" OutputCacheKey="inputDocument" RunConcurrently="False">
<nib:DocumentMergeStep.InputDocuments>
<nib:RandomLoader BoundingBox="Europe" LinkbackUrlPattern="http://MyProviderService/id={0}" MaxAmount="10" MaxID="100" MinAmount="10" MinID="0" NamePattern="EuropeanObject_{0}" NativeFormat="Svg" RepeatableRandomness="False" UriPrefix="European" />
<nib:RandomLoader BoundingBox="PacificIslands" LinkbackUrlPattern="http://MyProviderService/id={0}" MaxAmount="10" MaxID="100" MinAmount="10" MinID="0" NamePattern="PacificObject_{0}" NativeFormat="Svg" RepeatableRandomness="False" UriPrefix="Pacific" />
</nib:DocumentMergeStep.InputDocuments>
</nib:DocumentMergeStep>
<nib:PushToSimulatedApp AppKernelCacheKey="provider" ContentFormat="Svg" FailOnError="True" RunConcurrently="False">
<nib:PushToSimulatedApp.InputDocument>
<nib:CacheLoader SourceCacheKey="inputDocument" />
</nib:PushToSimulatedApp.InputDocument>
</nib:PushToSimulatedApp>
<nib:GeoFilterStep FailOnError="True" OutputCacheKey="filteredDocument" RunConcurrently="False" SelectionBox="Europe">
<nib:GeoFilterStep.InputDocument>
<nib:CacheLoader SourceCacheKey="inputDocument" />
</nib:GeoFilterStep.InputDocument>
</nib:GeoFilterStep>
<nib:DeepCompareStep DepthOfComparision="ID, Geo_2MeterAccuracy, PropertyBag, LinkbackUrl" FailOnError="True" RunConcurrently="False" Timeout="30000" TolerateAdditionalItems="False">
<nib:DeepCompareStep.ReferenceSource>
<nib:CacheLoader SourceCacheKey="filteredDocument" />
</nib:DeepCompareStep.ReferenceSource>
<nib:DeepCompareStep.InvestigatedSource>
<nib:SvgWebServiceLoader GeoFilter="Europe" NvgServiceUrl="http://localhost:10000/SvgOutputPort.asmx"/>
</nib:DeepCompareStep.InvestigatedSource>
</nib:DeepCompareStep>
</TestCase.ExecutionSteps>
<TestCase.CleanupSteps>
<nib:HttpGetStep FailOnError="True" RunConcurrently="False" Url="http://localhost:10000/Home/StopSvgPolling">
<nib:HttpGetStep.Parameters>
<x:String x:Key="PolledAddress">http://localhost:4000/SvgOutputPort.asmx</x:String>
</nib:HttpGetStep.Parameters>
</nib:HttpGetStep>
<nib:KillSimulatedApp AppKernelCacheKey="provider" FailOnError="True" PortNumber="4000" RunConcurrently="False" />
</TestCase.CleanupSteps>
</TestCase>
This is what it does:
Invokes a Clear operation on the test subject
Launches a webserver on port 4000 as a simulated partner app under the name MyProviderService
Invokes the test subject via HTTP Get to poll the simulated partner
Creates a new document containing geo info from two random generated content
Pushes the document to the simulated partner - hence the test subject will pick it up via polling
The test applies a geo filter on the document
The deep compare step loads the filtered document as base of comparision, and loads the content of the test subject via a web service
As clean-up, it stops the polling via an HTTP GET step and kills the simulated partner's web server.
The power of Bizunit is that merges the ease of creating tests in C# with intellisense and ease of maintaining/duplicating it in XAML files. For a quick easy read on how it works: http://kevinsmi.wordpress.com/2011/03/22/bizunit-4-0-overview/

As #GeorgeDuckett said, T4 templates are probably the way to go. In the application I am working on, we use them for a lot, including generating Repositories, Services, ViewModels, Enums and recently unit tests.
They are basically code generating scripts written in either VB or C#, looking at a directory for XML files would be no problem for these kinds of templates.
If you do choose to go the T4 route, the Tangible T4 Editor is definitely a must have, it is a free download.
Here is a quick example of a T4 script which should do or be pretty close to what you want:
<## template language="C#" debug="true" hostspecific="true"#>
<## output extension="g.cs"#>
[TestClass]
public class SampleTests
{
<#
string[] files = Directory.GetFiles(#"C:\TestFiles", "*.xml");
foreach(string filePath in files)
{
string fileName = Path.GetFileNameWithoutExtension(filePath);
#>
[TestMethod]
public void <#=fileName#>()
{
XamlTestManager.ConductTest();
}
<#
}
#>
}
Make sure this is placed in a file with the .tt extension, then on the property windows for this file, ensure the Build Action is None, Custom Tool is TextTemplatingFileGenerator.
Edit: Accessing output directory from T4 template
Add the following two lines to the top of your T4 template, under the <## template ... #> line:
<## assembly name="EnvDTE" #>
<## import namespace="EnvDTE" #>
Then inside your template, you can access and use the visual studio API like so:
IServiceProvider serviceProvider = this.Host as IServiceProvider;
DTE dte = serviceProvider.GetService(typeof(DTE)) as DTE;
object[] activeSolutionProjects = dte.ActiveSolutionProjects as object[];
if(activeSolutionProjects != null)
{
Project project = activeSolutionProjects[0] as Project;
if(project != null)
{
Properties projectProperties = project.Properties;
Properties configurationProperties = project.ConfigurationManager.ActiveConfiguration.Properties;
string projectDirectory = Path.GetDirectoryName(project.FullName);
string outputPath = configurationProperties.Item("OutputPath").Value.ToString();
string outputFile = projectProperties.Item("OutputFileName").Value.ToString();
string outDir = Path.Combine(projectDirectory, outputPath);
string targetPath = Path.Combine(outDir, outputFile);
}
}
outDir and targetPath contain the output directory and the full path to the output file.

Instead of creating a separate test for each set of test data you can create a single test that is repeatedly run for each set of test data:
[TestClass]
public class SampleTests
{
[TestMethod]
public void Test()
{
for (var i = 0; i < 10; ++i)
XamlTestManager.ConductTest(i);
}
}
You can also perform data-driven tests by using the DataSource attribute. This will perform your test for each row in your data set.
[TestClass]
public class SampleTests
{
public TestContext Context { get; set; }
[TestMethod]
[DataSource(...)]
public void Test()
{
var someData = Context.DataRow["SomeColumnName"].ToString();
...
}
}

I actually don't think this is a job for build-time code generation, I think you should use data attributes to drive the tests in this case.
If you used xunit you could do that like this:
public class SampleTests
{
[Theory]
[InlineData(1)]
[InlineData(2)]
[InlineData(...)]
[InlineData(N)]
public void Test(int x)
{
XamlTestManager.ConductTest(x);
}
}
And it will run the test once per InlineData attribute. Also I believe there is another attribute that you can pass a path to a file and it will populate your parameters with values from that file...
I think NUnit has a similar feature but XUnit is much better, I would recommend using XUnit instead.

Just answered alike "code generation from XML with T4" question.
https://stackoverflow.com/a/8554949/753110
Your requirement matches exactly what we did initially (and what lead to discovery of the ADM described on that answer).
We are currently working on test-case based generation, where the test-cases are actually built by the testing-staff, yet the complete integrationtests through code are generated to support them.
Added custom XML based generation demo for that other example, if you want to see:
https://github.com/abstractiondev/DemoSOCase8552428ABS