Develop Reference

Is it possible to use xUnit with LINQPad?

Is it possible to use xUnit with LINQPad?
It would be great to be able to write some tests for concepts being designed in LINQPad first. It's easier than adding yet another ConsoleApp23423894238 just to be able to quickly write some unit tests.

Yes, it is possible to use xUnit with LinqPad. I have updated this answer to give you the latest information, but the original answer I gave also gives you some insights and links regarding the usage of xUnit, so I kept it for your reference.
UPDATE: Latest version of LinqPad 6 or 7 (since V 6.9.x and higher, see 6.9 Release Notes section) has now built-in XUnit support via the new menu item Query / Add XUnit Test Support. This will add a query with required boilerplate code referenced via #load so you can start testing facts and theories instantly. You can run the example test method by pressing Alt+Shift+T.
For example, add the following new theory to the boilerplate code (right after the main method, put it inside the private::Tests region):
#region private::Tests
[Theory]
[InlineData(1)]
[InlineData(3, 6)]
[InlineData(5, 7, 9)]
void OddNumbers(params int[] values)
{
static bool IsOdd(int value) => value % 2 == 1;
foreach (var value in values)
{
Assert.True(IsOdd(value), $"IsOdd({value}) is false");
}
}
#endregion
Press Alt+Shift+T to run all tests and check the result.
Note: It also adds a file XUnit.linq to your query directory the first time you add XUnit Test support - this is a mini library to support xUnit tests and you should not touch it unless you know exactly what you're doing. For example, you can modify this file to change the way the test summary is being displayed (for example, you can add the parameter Func<UserQuery.TestResultSummary, bool> filter = null to the method RunTests and add .Where(filter) inside the query if filter is not null. This allows you to filter the results of the test summary).
For example: RunTests(filter: f => f.MethodName.StartsWith("Unit")); will only display the tests in the summary which names start with "Unit".
Dynamically skippable unit tests
You can modify the xunit script LinqPad 6 provides to support skippable unit tests (NUGET package XUnit.SkippableFact, detailed description see the link "dynamically skipping test and theory" at the end of this answer).
Make the following changes in xunit:
function RunTests - add:
runner.OnTestSkipped = info => AddTestResult(info);
class TestResultSummary - add:
public bool Skipped() => _testInfo is Xunit.Runners.TestSkippedInfo;
class TestResultSummary / public object Status - add:
_testInfo is Xunit.Runners.TestSkippedInfo ? Util.WithStyle ("Skipped", "color:orange") :
Those changes will enable the xunit script to show when a test was skipped in the results.
To test it, load the NUGET package XUnit.SkippableFact via F4, then add the test framework and finally add the tests:
[SkippableFact]
void Test_Xunit3()
{
Skip.If(true, "skipping");
Assert.True(1 + 1 == 2);
}
[SkippableTheory]
[InlineData(1)]
[InlineData(2)]
void Test_Xunit4(int x)
{
Skip.If(x==2, "skipping");
Assert.True(1 + 1 == 2);
}
Note: Those changes will also work with the plain XUnit. So adding the changes to the code does not harm.
Now if you like, continue reading the original answer (with some useful hints at the end):
Two answers pointed me into the right direction, but the descriptions haven't been complete enough, so I had to experiment a bit. To save you that time, I am describing what I did; I have now working solutions for LinqPad 5 and LinqPad 6 (and 7).
LinqPad 5 - for .NET
Based on what Tom wrote, I was able to get it working with LinqPad 5 (v5.42.01).
There are some more steps required, which aren't obvious and not explained in the accepted answer, so I'll describe what helped on my side.
First of all, you will need to use the paid version of LinqPad, because only then you are able to load NUGET packages.
To load the required packages in LinqPad, press F4. The query properties dialog opens up. In that dialog, click Add NUGET....
The NUGET dialog opens up. In the middle of the NUGET dialog, there is a search textbox. Search for xUnit.Net - once it got displayed, click Add to query, wait until it loaded, then click on Add namespaces. Then do the same for Xunit.Runner.LinqPad.
Then close the NUGET dialog, but keep the query properties open. You need to to there to the Advanced tab: Enable (tick-mark) "Copy all non-framework references to a single local folder." Now you can close the query properties dialog.
Import the following example into the query editor (C# Program mode):
void Main()
{
// Press F4, go to Advanced, then
// tick-mark "Copy all non-framework references to a single local folder"
var me = Assembly.GetExecutingAssembly();
var result = Xunit.Runner.LinqPad.XunitRunner.Run(me);
result.Dump("Xunit runner result");
}
/// <summary>
/// This test class is there for demo purpose, so developers can
/// see how to write test methods.
/// It also verifies that XUnit itself is functioning correctly.
/// </summary>
public class A_XUnitSelfTest
{
[InlineData(2)]
[InlineData(3)]
[Theory]
public void Test_Theory(int a)
{
Assert.True(a == 2 || a == 3); // succeeds
}
[Fact]
public void Test_Fact()
{
Assert.False(1 == 0); // succeeds
}
}
Run it and you will see the output of the test runner.
Running 3 of 3 tests...
Finished: 3 tests in 0,003s (0 failed, 0 skipped)
Modify the test to see how it looks like if a test fails by changing the assert in the Test_Fact() to Assert.False(1 == 1);, then run it again. It should show this time:
Running 3 of 3 tests...
[FAIL] UserQuery+A_XUnitSelfTest.Test_Fact: Assert.False() Failure
Expected: False
Actual: True
at Xunit.Assert.False(Nullable`1 condition, String userMessage)
at Xunit.Assert.False(Boolean condition)
at UserQuery.A_XUnitSelfTest.Test_Fact() in C:\Users...\AppData\Local\Temp\LINQPad5_oyxxqxbu\query_mxsmky.cs:line 62
Finished: 3 tests in 0,005s (1 failed, 0 skipped)
LinqPad 6 - for .NET Core
LinqPad 6 is different, because it is based on .NET Core and hence does require different libraries.
Start with this:
void Main()
{
// Press F4, go to Advanced, then
// tick-mark "Copy all non-framework references to a single local folder"
var me = Assembly.GetExecutingAssembly();
var result = Xunit.Runner.LinqPad.XunitRunner.Run(me);
result.Dump("Xunit runner result");
}
/// <summary>
/// This test class is there for demo purpose, so developers can
/// see how to write test methods.
/// It also verifies that XUnit itself is functioning correctly.
/// </summary>
public class A_XUnitSelfTest
{
[InlineData(2)]
[InlineData(3)]
[Theory]
public void Test_Theory(int a)
{
Assert.True(a == 2 || a == 3); // succeeds
}
[Fact]
public void Test_Fact()
{
Assert.False(1 == 0); // succeeds
}
}
Then add xUnit.net, xUnit.net [Core Unit Testing Framework] and xUnit.net [Runner Utility] and all their namespaces via the F4 dialog (details see LinqPad 5 sesion) and finally copy the XUnit runner source code as mentioned by #ramiroalvarez to the end of the xUnit test example above:
XUnit runner source code
Enable "copy all nuget assemblies to a single local folder" in the F4 dialog, Advanced tab ... that should be enough, but it isn't so read on:
If you try it, it throws the exception InvalidOperationException("Please enable the shadow folder option for none-framework references (F4 -> Advanced).") in the source code.
Now I looked into the XUnit runner, and found that if you comment out 2 lines of code then it is running fine:
Original code (located in method GetTargetAssemblyFilename):
if (shadowFolder != xunitFolder
|| Directory.GetFiles(shadowFolder, "xunit.execution.*.dll").Length == 0)
{
throw new InvalidOperationException("Please enable the shadow folder option ...");
}
// ...
var targetAssembly = Path.Combine(shadowFolder, Path.GetFileName(assemblyFilename));
//Console.WriteLine($"Copy \"{ assemblyFilename }\" -> \"{ targetAssembly }\"");
File.Copy(assemblyFilename, targetAssembly, true);
Comment the lines
// ...
// throw new InvalidOperationException("Please enable the shadow folder option ...");
// ...
// File.Copy(assemblyFilename, targetAssembly, true);
and you're done! (Tested with v6.8.3 version of LinqPad 6, X64 version)
Hints:
Hint 1:
You can append the xUnit runner to your code as described, but there is a more elegant way in LinqPad 6:
Save a copy containing only the xUnit.Runner source code in the same path as your unit test files with the file name xUnit.Runner.LinqPad6.linq.
Then, add the following to the top of your test code:
Now you have cleaned up your code! Every time you create a new test, just remember to add the #load command.
Hint 2:
To be able to skip facts statically, add the following 2 attributes to the code of the test runner:
/// <summary>
/// Skip a Fact (you can override it by setting OverrideSkip=true)
/// </summary>
public class SkipFactAttribute : FactAttribute
{
/// <summary>
/// Override SkipFact: Set OverrideSkip=true to run even skipped tests
/// </summary>
public static bool OverrideSkip = false; // set to true to ignore skip
/// <summary>
/// Constructor. Used to set skip condition.
/// </summary>
public SkipFactAttribute()
{
if (!OverrideSkip) Skip = "Skipped Fact";
}
}
/// <summary>
/// Skip a Theory (you can override it by setting OverrideSkip=true)
/// </summary>
public class SkipTheoryAttribute : TheoryAttribute
{
/// <summary>
/// Override SkipTheory: Set OverrideSkip=true to run even skipped tests
/// </summary>
public static bool OverrideSkip = false; // set to true to ignore skip
/// <summary>
/// Constructor. Used to set skip condition.
/// </summary>
public SkipTheoryAttribute()
{
if (!OverrideSkip) Skip = "Skipped Theory";
}
}
This is useful, if you want to skip tests. To skip a test, simply replace [Fact] by [SkipFact] or [Theory] by [SkipTheory].
If you're busy writing a test and you want to test only that single test, set all attributes to [SkipFact] or likewise [SkipTheory], except the single one you're writing currenlty.
Then, to quickly run all the tests again from time to time - without having to change each and every single attribute - add the following to the initialization part of your test suite (see global setup for startup and teardown):
public class TestsFixture : IDisposable
{
public TestsFixture() // running once before the tests
{
// true: Ignore Skip and run all tests, false: Skip is effective
SkipFactAttribute.OverrideSkip = true;
SkipTheoryAttribute.OverrideSkip = true;
}
public void Dispose() // running once, after the tests
{
// teardown code
}
}
If you're done, restore the [Fact]/ [Theory] attributes and set the .OverrideSkip variables back to false.
NOTE: In case you want to skip facts based on a runtime condition, take a look at dynamically skipping test and theory.
More about unit testing
For some advanced topics about unit testing with xUnit, I recommend to read:
xunit tests in net core
global setup for startup and teardown
shared context
running tests in parallel
running tests in prioritized, specified order
configuring with json
configuring with xml
skipping Test and Theory // description works for TheoryAttribute too
dynamically skipping test and theory
xUnit Theory: Working With InlineData, MemberData, ClassData
If you want to use it with Visual Studio, you can find infos here.

Possible? Sort of.
This is neither a main use case of LINQPad nor of unit testing.
Regarding LINQPad, you could use an existing test runner but probably the only ones that could be integrated with LINQPad are console runners. A test runner is not a trivial program. Try Xunit.Runner.LinqPad.
As for needing to "write some tests for concepts being designed", consider that you might not be getting enough out of your unit testing workflow if you don't find them valuable enough to create a project and keep it in source control. One way to get more value is to use Behavior-Driven Development (BDD) and if you want to write the requirements in a business-readable language, try something like SpecFlow.

Thanks to asherber, xunit now works with linqpad6 (netcore).
https://github.com/asherber/Xunit.Runner.LinqPad/blob/master/Xunit.Runner.LinqPad/XunitRunner.cs
It is important within the query properties, check "copy all nuget assemblies to a single local folder" otherwise, you will have the following error "InvalidOperationException: Please enable the shadow folder option for none-framework references"

I just tried to open a LINQPad query file that tried to load the xunit package and got an error which has never happened to me in using LINQPad for years. So my answer is no.

How to run the test cases category wise I have tried with mstest

Hi I have wrote test cases under one class in visual studio 2013.
can I run the test cases through the automation by category wise as each class contain more than 50 test cases and I don't want to run whole at a time wanted to run by category wise, I have tried with the mstest but it gives me no test to execute.
mstest /testcontainer:D:\TFS\OTests.dll /category:Tests
Do anybody can suggest me something else as m not able to do that..

Is there any error message or other logs for further troubleshooting?
Anyway, please make sure you creating and Assigning Test Categories correctly.
Please refer to this article for details:
How to: Group and Run Automated Tests Using Test Categories
UPDATE1:
Please exactly following the rules, you need to add the quotes for the Category name.
Please try mstest /testcontainer:D:\TFS\OTests.dll /category:"Tests"
UPDATE2:
I tested at my side, everything works correctly. Please see the sample: ( You need to run the command with VS2013 Cross Tools command Prompt window)
[TestCategory("nine"), TestMethod()]
public void TestMethod1()
{
Assert.AreEqual(1, 1);
}
[TestCategory("eight"), TestMethod()]
public void TestMethod2()
{
Assert.AreEqual(1, 2);
}

Visual Studio: Revoke reading permission of test file for Unit Testing

I am working on a Visual Studio (Community 2015) project in C# that reads metadata from video files. Therefore I want to write a Unit Test that tests if the program handles an inaccessible file the right way. So I want to add a test file to my test data for which the Unit Test does not have read permissions.
But what is the best way to do this? I didn't find any option to set permissions in Visual Studio. The only idea I have at the moment is to revoke reading permissions directly in the properties of the file. Through the Windows File Explorer, thus outside of Visual Studio. But then I wouldn't be able to read the file myself. Also, it doesn't seem to be the right way and I think there should be a more clean way to solve this problem directly in Visual Studio.
Any ideas?

If you are writing a unit test then it should just check if your code is written correcly and not about the external issues which are difficult to replicate or may not be available on other machines. So I think you should read about Dependency Injection and how to do mock dependencies in your test. I can give you an easy to understand example but please do write it in your words.
public interface IFileReader
{
string ReadFile( string filePath );
}
public class FileReader : IFileReader
{
public string ReadFile( string filePath )
{
return System.IO.File.ReadAllText( filePath );
}
}
So Suppose you have a class VideoMetaDataReader then in that class you will inject the interface as a dependency and use the ReadFile method to read. And then in your test
[TestFixture]
public class FileReaderTests
{
[Test]
public void ShouldDisplayANiceMessage_WhenFileIsInaccessible()
{
var moq = new Moq.Mock<IFileReader>();
moq
.Setup( x => x.ReadFile( Moq.It.IsAny<string>() ) )
.Throws<Exception>();
var metaDataReader = new MetaDataReader( moq.Object );
metaDataReader.ReadVideoFile( "video.mp4" );
Assert.AreEqual( 1, meaDataReader.Errors.Count );
}
}
I am assuming there will be a errors property which will return encountered errors but it depends how you want to do it (I didn't think too much on this).
Anyways, the point is don't test what is not covered by your code. What your code needs to do is to display a nice message if there is an exception and that you can do if you mock the interface (which means don't call the actual implementation and instead do something which I want in this test) and then check if your code can handle that. btw i used nunit and moq for test and mocking.

How to access TestRunParameters within RunSettings file

Reading through https://msdn.microsoft.com/en-us/library/jj635153.aspx I have created a .RunSettings files with a few parameters similar to the example:
<TestRunParameters>
<Parameter name="webAppUrl" value="http://localhost" />
<Parameter name="webAppUserName" value="Admin" />
<Parameter name="webAppPassword" value="Password" />
</TestRunParameters>
I plan on having a .RunSettings file for each of our environments with appropriate URLs and credentials for running a CodedUI test on the specified RunSettings file's environment.
I can see that from command line to reference the Settings file I can run:
vstest.console myTestDll.dll /Settings:Local.RunSettings /Logger:trx
vstest.console myTestDll.dll /Settings:QA.RunSettings /Logger:trx
etc...
But I don't see any way that calls out how to actually utilize the TestRunParameters from within the codedUI test.
What I would like to do is set up test initializers that use the TestRunParameters to determine where to log in, and what credentials to use. Something like this:
[TestInitialize()]
public void MyTestInitialize()
{
// I'm unsure how to grab the RunSettings.TestRunParameters below
string entryUrl = ""; // TestRunParameters.webAppUrl
string userName = ""; // TestRunParameters.webAppUserName
string password = ""; // TestRunParameters.webAppPassword
LoginToPage(entryUrl, userName, password);
}
public void LoginToPage(string entryUrl, string userName, string password)
{
// Implementation
}
Information on how to reference the TestRunParameters is greatly appreciated!
EDIT
/// <summary>
/// Summary description for CodedUITest1
/// </summary>
[CodedUITest]
public class CodedUITest1
{
public static string UserName = string.Empty;
[ClassInitialize]
public static void TestClassInitialize(TestContext context)
{
UserName = context.Properties["webAppUserName"].ToString();
Console.WriteLine(UserName);
}
[TestMethod]
public void CodedUITestMethod1()
{
this.UIMap.RecordedMethod1();
// To generate code for this test, select "Generate Code for Coded UI Test" from the shortcut menu and select one of the menu items.
}
// Rest of the default class - TestContext instantiation, UI map instantiation, etc
}
The exception I'm getting when running:
NullReference Exception
#williamfalconeruk I have updated my test class as above, but I am still getting the same error, any idea what I'm doing wrong?

For those that use Resharper with this issue, I discovered the fix (no need to disable Resharper):
Go to Visual Studio top menu -> Resharper -> Options
Find the Tools section, expand "Unit Testing"
Click on "MsTest". The checkbox should be on enabled, but the Test Settings file path below that may be blank. If it is, click on browse and select the runsettings file you want to use.
Click save, rebuild and try to run the tests, the parameters should now work.
Not sure why but simply selecting the Test Settings file from the Tests menu -> Test Settings does not actually work when using Resharper, so this file needs to be explicitly pointed to directly in the Resharper options.

I also came across this recently, as we wanted to move away from legacy environment variable usage. The solution provided below was suitable for our needs, but there may be a better one...
It turns out you can access these in the TestContext of a ClassInitialize method of your Test fixture. There is a Properties dictionary which has these parameters, and you could access the values here:
[ClassInitialize]
public static void TestClassinitialize(TestContext context)
{
var webAppUrl = context.Properties["webAppUrl"].ToString();
//other settings etc..then use your test settings parameters here...
}
Note: these are static so if you need access to this you may need to set up static properties to access within your Test code.
An alternative suggested is using a Data Driven Test approach.
Here's some basic info on data driven tests here which may also help:
https://msdn.microsoft.com/en-us/library/ms182527.aspx
As I said, the above solution suited our needs at a basic level.
UPDATE: see image below in response to test settings returning null...

This works for me (VS2017-pro):
namespace TestApp.Test
{
[TestClass]
public class UnitTest1
{
// This enables the runner to set the TestContext. It gets updated for each test.
public TestContext TestContext { get; set; }
[TestMethod]
public void TestMethod1()
{
// Arrange
String expectedName = "TestMethod1";
String expectedUrl = "http://localhost";
// Act
String actualName = TestContext.TestName;
// The properties are read from the .runsettings file
String actualUrl = TestContext.Properties["webAppUrl"].ToString();
// Assert
Assert.AreEqual(expectedName, actualName);
Assert.AreEqual(expectedUrl, actualUrl);
}
[TestMethod]
public void TestMethod2()
{
// Arrange
String expectedName = "TestMethod2";
// Act
String actualName = TestContext.TestName;
// Assert
Assert.AreEqual(expectedName, actualName);
}
}
}
Make sure to select the runsettings file you wish to use, here: Test -> Test Settings.

An alternative to disable Resharper is to enable MSTest support and select the test setting file on Resharper Options dialog (->Tools->Unit Testing->MsTest).

I was trying to do this exact thing as well. As many of you may know, running tests through MTM exposes some additional properties to the TestContext, including the name of the Run Settings used. I used this property as a "Foreign Key" of sorts for our test data, allowing us to specify environment URLs etc. without hardcoding them or using the incredibly lackluster "Data Driving" tools that come with out of the box testing.
Of course, there's no way to expose any run-time properties when executing tests as part of a BDT or release workflow besides what #kritner is attempting which microsoft describes HERE. However if you read the comments of that link you'll discover what you may be able to infer here:
You need to use VS 2013 R5 or VS 2015 to use this solution
It will only work for Unit Tests!
Those of us who are trying to execute UI or Load tests as part of a CI or CD workflow are completely screwed. You don't get any additional properties in testContext, even when executing a Plan/Suite with certain test configurations (not settings) created in MTM. #Adam may have been able to get this to work when running vs debugging, but that may have only worked with unit tests. Through CodedUI I've been unable to retrieve the properties without getting a NullReferenceException. Here's an example of the janky code I was using to investigate:
if (testContextInstance.Properties["__Tfs_TestConfigurationName__"] != null) //Exposed when run through MTM
{
TFSTestConfigurationName = testContextInstance.Properties["__Tfs_TestConfigurationName__"].ToString();
}
else TFSTestConfigurationName = "Local"; //Local
var configName = testContextInstance.Properties["configurationName"] ?? "No Config Found";
Trace.WriteLine("Property: " + configName);
And the XML of my .runsettings file:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Parameters used by tests at runtime. These are required as a substitute for TFS/MTM test settings.-->
<!-- File instructions: https://msdn.microsoft.com/en-us/library/jj635153.aspx#example -->
<!-- TFS instructions: https://blogs.msdn.microsoft.com/visualstudioalm/2015/09/04/supplying-run-time-parameters-to-tests/ -->
<TestRunParameters>
<Parameter name="configurationName" value="Local" />
</TestRunParameters>
</RunSettings>
And an excerpt from the .trx produced by the BDT workflow:
Property: No Config Found

For the NullReferenceException issue:
I was also facing the same issue recently and the solution to it is to have the latest update of Visual Studio 2013. Right now the latest update is Update 5. I am not sure which particular update fixes this issue. I applied Update 5 and was able to successfully access the TestRunParameters in the ClassInitialize method.
You can find the updates # https://support.microsoft.com/en-us/kb/2829760
So I had two machines, on one it was all working fine and on the other I was getting the exception. I investigated that the only difference is the Update of VS; applied it and that solved the problem. :)

With NUnit 3, I was able to find the properties in the runsettings file, using
TestContext.Parameters
So in this case it would be:
string entryUrl = TestContext.Parameters["webAppUrl"];
string username = TestContext.Parameters["webAppUserName"];
string password = TestContext.Parameters["webAppPassword"];

I was able to resolve this for Unit tests by disabling Resharper. Wish I could say the same for Coded UI tests.

Why dont you use Application settings?
you can then read them anywhere like
var yesICan= Properties.Settings.Default.IToldYou;
you can create Properties out from them, pretty much you can do a lot.
public string URL_WEBOFFICE
{
get
{
return Properties.Settings.Default.WEBOFFICE_URL.Replace("***", Properties.Settings.Default.SiteName);
}
}

Nunit - TestContext.CurrentContext.Test not working

I am using nunit 2.5.9.10348 and trying to extract the current test name in the TearDown event so I can assign a screengrab filename the test name however it is always null (see the attached image). The private _context variable does have the TestName however this is no use to me!
Has anyone had success using this new TestContext functionality (from 2.5.7).

From your screenshot I see that _context has keys "TestName" and "Properties". But TestAdapter looks for keys "Test.Name" for Name and "Test.Properties" for Properties. So, there is something wrong with TestContext initialization (I think wrong data was put to Remoting.Messaging.CallContext).
After a little investigation (see comments):
NUnit tests should run by NUnit testig environment for Context to be available.

I had the same issue. It occurred when in a TearDown method I executed method, which actually was to make the teardown
[TearDown]
public void CleanUp()
{
TestContext.CurrentContext.Test.FullName; //!=null
someClassInstance.DoTearDown();
}
class SomeClass
{
public void DoTearDown()
{
TestContext.CurrentContext.Test.FullName; //==null
}
}
I have no idea why, but it seemed so. Is it your case?
UPDATE: Now I looked at the screenshot, so it's not your case :)

Same issue with R# test runner. Just downloaded NUnit sources and added a workaround in TestAdapter to make it work with r#
public string Name
{
get
{
return (_context["Test.Name"] ?? _context["TestName"]) as string;
}
}