Does MSTest have an equivalent to NUnit's TestCase? - c#

I find the TestCase feature in NUnit quite useful as a quick way to specify test parameters without needing a separate method for each test. Is there anything similar in MSTest?
[TestFixture]
public class StringFormatUtilsTest
{
[TestCase("tttt", "")]
[TestCase("", "")]
[TestCase("t3a4b5", "345")]
[TestCase("3&5*", "35")]
[TestCase("123", "123")]
public void StripNonNumeric(string before, string expected)
{
string actual = FormatUtils.StripNonNumeric(before);
Assert.AreEqual(expected, actual);
}
}

Microsoft recently announced "MSTest V2" (see blog-article). This allows you to consistently (desktop, UWP, ...) use the DataRow-attribute!
[TestClass]
public class StringFormatUtilsTest
{
[DataTestMethod]
[DataRow("tttt", "")]
[DataRow("", "")]
[DataRow("t3a4b5", "345")]
[DataRow("3&5*", "35")]
[DataRow("123", "123")]
public void StripNonNumeric(string before, string expected)
{
string actual = FormatUtils.StripNonNumeric(before);
Assert.AreEqual(expected, actual);
}
}
Again, Visual Studio Express' Test Explorer unfortunately doesn't recognize these tests. But at least the "full" VS versions now support that feature!
To use it, just install the NuGet packages MSTest.TestFramework and MSTest.TestAdapter (both pre-release as of now).
Older answer:
If don't have to stick with MSTest and you're just using it for being able to run the tests via Test Explorer because you only have a Visual Studio Express edition, then this might be a solution for you:
There's the VsTestAdapter VSIX extension for being able to run NUnit tests via Test Explorer. Unfortunately, VS Express users can't install extensions...
But fortunately the VsTestAdapter comes with a plain NuGet-Package, too!
So, if you're a VS Express user, just install the VsTestAdapter NuGet-Package and enjoy running your NUnit tests/testcases via Test Explorer!
Unfortunately the aforementioned statement isn't true. While it's perfectly possible to install the package via an Express edition, it's useless, since it can't utilize the Test Explorer. There's previously been a side note on an older version of the TestAdapter, which was removed from the 2.0.0's description page:
Note that it doesn't work with VS Express

I know this is a late answer but hopefully it helps others out.
I looked everywhere for an elegant solution and ended up writing one myself. We use it in over 20 projects with thousands of unit tests and hundreds of thousands of iterations. Never once missed a beat.
https://github.com/Thwaitesy/MSTestHacks
1) Install the NuGet package.
2) Inherit your test class from TestBase
public class UnitTest1 : TestBase
{ }
3) Create a Property, Field or Method, that returns IEnumerable
[TestClass]
public class UnitTest1 : TestBase
{
private IEnumerable<int> Stuff
{
get
{
//This could do anything, get a dynamic list from anywhere....
return new List<int> { 1, 2, 3 };
}
}
}
4) Add the MSTest DataSource attribute to your test method, pointing back to the IEnumerable name above. This needs to be fully qualified.
[TestMethod]
[DataSource("Namespace.UnitTest1.Stuff")]
public void TestMethod1()
{
var number = this.TestContext.GetRuntimeDataSourceObject<int>();
Assert.IsNotNull(number);
}
End Result: 3 iterations just like the normal DataSource :)
using Microsoft.VisualStudio.TestTools.UnitTesting;
using MSTestHacks;
namespace Namespace
{
[TestClass]
public class UnitTest1 : TestBase
{
private IEnumerable<int> Stuff
{
get
{
//This could do anything, get a dynamic list from anywhere....
return new List<int> { 1, 2, 3 };
}
}
[TestMethod]
[DataSource("Namespace.UnitTest1.Stuff")]
public void TestMethod1()
{
var number = this.TestContext.GetRuntimeDataSourceObject<int>();
Assert.IsNotNull(number);
}
}
}

I know this is another late answer, but on my team that is locked into using the MS Test framework, we developed a technique that relies only on Anonymous Types to hold an array of test data, and LINQ to loop through and test each row. It requires no additional classes or frameworks, and tends to be fairly easy to read and understand. It's also much easier to implement than the data-driven tests using external files or a connected database.
For example, say you have an extension method like this:
public static class Extensions
{
/// <summary>
/// Get the Qtr with optional offset to add or subtract quarters
/// </summary>
public static int GetQuarterNumber(this DateTime parmDate, int offset = 0)
{
return (int)Math.Ceiling(parmDate.AddMonths(offset * 3).Month / 3m);
}
}
You could use and array of Anonymous Types combined to LINQ to write a tests like this:
[TestMethod]
public void MonthReturnsProperQuarterWithOffset()
{
// Arrange
var values = new[] {
new { inputDate = new DateTime(2013, 1, 1), offset = 1, expectedQuarter = 2},
new { inputDate = new DateTime(2013, 1, 1), offset = -1, expectedQuarter = 4},
new { inputDate = new DateTime(2013, 4, 1), offset = 1, expectedQuarter = 3},
new { inputDate = new DateTime(2013, 4, 1), offset = -1, expectedQuarter = 1},
new { inputDate = new DateTime(2013, 7, 1), offset = 1, expectedQuarter = 4},
new { inputDate = new DateTime(2013, 7, 1), offset = -1, expectedQuarter = 2},
new { inputDate = new DateTime(2013, 10, 1), offset = 1, expectedQuarter = 1},
new { inputDate = new DateTime(2013, 10, 1), offset = -1, expectedQuarter = 3}
// Could add as many rows as you want, or extract to a private method that
// builds the array of data
};
values.ToList().ForEach(val =>
{
// Act
int actualQuarter = val.inputDate.GetQuarterNumber(val.offset);
// Assert
Assert.AreEqual(val.expectedQuarter, actualQuarter,
"Failed for inputDate={0}, offset={1} and expectedQuarter={2}.", val.inputDate, val.offset, val.expectedQuarter);
});
}
}
When using this technique it's helpful to use a formatted message that includes the input data in the Assert to help you identify which row causes the test to fail.
I've blogged about this solution with more background and detail at AgileCoder.net.

Khlr gave a good detailed explanations and apparently this approach started working in VS2015 Express for Desktop.
I tried to leave the comment, but my lack of reputation didn't allow me to do so.
Let me copy the solution here:
[TestClass]
public class StringFormatUtilsTest
{
[TestMethod]
[DataRow("tttt", "")]
[DataRow("", "")]
[DataRow("t3a4b5", "345")]
[DataRow("3&amp;5*", "35")]
[DataRow("123", "123")]
public void StripNonNumeric(string before, string expected)
{
string actual = FormatUtils.StripNonNumeric(before);
Assert.AreEqual(expected, actual);
}
}
To use it, just install the NuGet packages MSTest.TestFramework and MSTest.TestAdapter.
One problem is
Error CS0433 The type 'TestClassAttribute' exists in both
'Microsoft.VisualStudio.QualityTools.UnitTestFramework,
Version=10.0.0.0 and
'Microsoft.VisualStudio.TestPlatform.TestFramework, Version=14.0.0.0
So, please remove Microsoft.VisualStudio.QualityTools.UnitTestFramework from references of the project.
You're very welcome to edit the original reply and delete this one.

Consider using DynamicDataAttribute:
NUnit Test cases
private static readonly IEnumerable<TestCaseData> _testCases = new[]
{
new TestCaseData("input value 1").Returns(new NameValueCollection { { "a", "b" } }),
new TestCaseData("input value 2").Returns(new NameValueCollection { { "a", "b" } }),
/* .. */
};
[TestCaseSource(nameof(_testCases))]
public NameValueCollection test_test(string str)
{
var collection = new NameValueCollection();
collection.TestedMethod(str);
return collection;
}
MSTest Test cases
private static IEnumerable<object[]> _testCases
{
get
{
return new[]
{
new object[] { "input value 1", new NameValueCollection { { "a", "b" } } },
new object[] { "input value 2", new NameValueCollection { { "a", "b" } } },
/* .. */
};
}
}
[TestMethod]
[DynamicData(nameof(_testCases))]
public void test_test(string str, NameValueCollection expectedResult)
{
var collection = new NameValueCollection();
collection.TestedMethod(str);
CollectionAssert.AreEqual(expectedResult, collection);
}

MSTest has the DataSource attribute, which will allow you to feed it a database table, csv, xml, etc. I've used it and it works well. I don't know of a way to put the data right above as attributes as in your question, but it's very easy to set up the external data sources and files can be included in the project. I had it running an hour from when I started, and I'm not an automated test expert.
https://msdn.microsoft.com/en-us/library/ms182527.aspx?f=255&MSPPError=-2147217396 has a full tutorial based on database input.
http://www.rhyous.com/2015/05/11/row-tests-or-paramerterized-tests-mstest-xml/ has a tutorial based on XML file input.

Related

Why my code coverage is zero percent even unit test all passed?

I have created a project with just one method and I have written a unit test on it and the unit test passed locally. But not sure why after running sonar cloud scanner, it shows zero percent coverage.
This is the test class
public class DataStructureTest
{
private readonly DataStructure ds;
public DataStructureTest()
{
ds = new DataStructure();
}
[Theory, MemberData(nameof(LongestString_Return_Longest_String_ShouldPass_Data))]
public void LongestString_Return_Longest_String_ShouldPass(string input, string expect)
{
// Act
var actual = ds.LongestString(input);
// Assert
Assert.Equal(expect, actual);
}
public static TheoryData<string, string> LongestString_Return_Longest_String_ShouldPass_Data()
{
return new TheoryData<string, string>
{
{ "Hello John", "Hello" },
{ "Hi John and Mandy", "Mandy" }
};
}
}
You have to be careful about what these softwares mean when they use some term. For example, SonarQube has following article: https://community.sonarsource.com/t/sonarqube-and-code-coverage/4725
FAQ has this as first question:
Q: After migrating from 5.6 to 6.7 my coverage shows 0%, why is that ?
R: Since SonarQube 6.2 and the implementation of the MMF-345 565, if no coverage information is found the coverage is then set to zero by default.
I think your case may come under this.

How to push LSP Diagnostic using OmniSharp LanguageServer?

I'm using OmniSharp's C# LSP server to implement a simple parsing/language service for a VS Code plugin. I've managed to get the basics up and running, but I've not been able to figure out how to push diagnostic messages to VS Code (like in this typescript sample).
Does anyone have any sample code/hints that would be of use?
Thanks!
Having spoken with #david-driscoll, it turns out I needed to stash a reference to ILanguageServerFacade in my constructor and use the PublishDiagnostics extension method on TextDocument. Ie:
public class TextDocumentSyncHandler : ITextDocumentSyncHandler
{
private readonly ILanguageServerFacade _facade;
public TextDocumentSyncHandler(ILanguageServerFacade facade)
{
_facade = facade;
}
public Task<Unit> Handle(DidChangeTextDocumentParams request, CancellationToken cancellationToken)
{
// Parse your stuff here
// Diagnostics are sent a document at a time, this example is for demonstration purposes only
var diagnostics = ImmutableArray<Diagnostic>.Empty.ToBuilder();
diagnostics.Add(new Diagnostic()
{
Code = "ErrorCode_001",
Severity = DiagnosticSeverity.Error,
Message = "Something bad happened",
Range = new Range(0, 0, 0, 0),
Source = "XXX",
Tags = new Container<DiagnosticTag>(new DiagnosticTag[] { DiagnosticTag.Unnecessary })
});
_facade.TextDocument.PublishDiagnostics(new PublishDiagnosticsParams()
{
Diagnostics = new Container<Diagnostic>(diagnostics.ToArray()),
Uri = request.TextDocument.Uri,
Version = request.TextDocument.Version
});
return Unit.Task;
}
}
For real code, you would want a centralised array of Diagnostic objects, but this shows the basics of how to get it done.
Thank you David!

Ways to test SearchResults<T> Class in Azure.Search.Documents v11.2.0

I'm trying to find out a way to unit test the search results from the SearchResults<T> Class with different parameters.
Compared to DocumentSearchResult<T> Class which was in the previous package Microsoft.Azure.Search.Data v10.1.0, SearchResults<T> Class does not have any constructors, just properties and methods.
DocumentSearchResult<T> Class
https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.search.models.documentsearchresult-1?view=azure-dotnet
SearchResults<T> Class
https://learn.microsoft.com/en-us/dotnet/api/azure.search.documents.models.searchresults-1?view=azure-dotnet
I couldn't seem to find any documentation on that online as it is relatively new, but any ideas or suggestions are most welcome.
Thank you.
All our Azure SDKs for .NET matching package names Azure.* like Azure.Search.Documents are mockable using the same principles as shown here. For pure response objects like SearchResults<T>, we create model factories so that you can create them for testing. In general, there is no other intended purpose for creating them apart from testing.
Using Moq, for example:
// Arrange
var mockResponse = new Mock<Response>();
var mockResults = SearchModelFactory.SearchResults<Model>(new[]
{
SearchModelFactory.SearchResult<Model>(new Model { Name = "a" }, 1.0),
SearchModelFactory.SearchResult<Model>(new Model { Name = "b" }, 0.9),
}, 2, rawResponse: response.Object);
var mockClient = new Mock<SearchClient>();
mockClient.Setup(m => m.Search<Model>(It.IsAny<string>(), It.IsAny<SearchOptions>(), default))
.Returns(Response.FromValue(mockResults, mockResponse.Object));
// Act
var response = mockClient.Object.Search<Model>("anything");
// Assert
Assert.AreEqual(2, response.Value.TotalCount);
See more mocking samples for other ideas.
The SearchModelFactory is the class that will help you to create the models for SearchResults.
This reading can also be helpful for another kind of tests: https://devblogs.microsoft.com/azure-sdk/unit-testing-and-mocking/
Just in case someone else is looking for a version using NSubstitute, here is mine:
public async Task TheTest()
{
// Arrange
var searchResultValues = new List<SearchResult<MyModel>>
{
SearchModelFactory.SearchResult<MyModel>(new MyModel{Id = 1}, score:0.9, highlights:null),
SearchModelFactory.SearchResult<MyModel>(new MyModel{Id = 2}, score:0.8, highlights:null),
};
var searchResults = SearchModelFactory.SearchResults<MyModel>(searchResultValues, totalCount: 2, facets: null, coverage: 1.3, rawResponse: Substitute.For<Response>());
var response = Response.FromValue(searchResults, Substitute.For<Response>());
var searchClient = Substitute.For<SearchClient>();
searchClient.SearchAsync<MyModel>(Arg.Any<string>(), Arg.Any<SearchOptions>()).Returns(Task.FromResult(response));
var sut = new ClassThatUsesTheSearchClient(searchClient);
// Act
var result = await sut.MethodToSearch();
// or directly
var directResult = await searchClient.SearchAsync<MyModel>("text");
// Assert
Assert.AreEqual(2, directResult.Value.TotalCount);
...
}

System.IO.Abstraction.TestingHelpers - testing on the different platforms

I'm writing a unit test to check some methods operating on the files. I've used System.IO.Abstraction on the library side, and System.IO.Abstraction.UnitTesting on the UnitTests side.
I'm using MacOS, but I want to be able to run tests on the Windows too. The problem is around paths because as we know on windows it's like "C:\MyDir\MyFile.pdf", but for Linux/MacOS it's more like "/c/MyDir/MyFile.pdf".
var fileSystem = new MockFileSystem(new Dictionary<string, MockFileData>
{
{ #"/c/scans/myfile.pdf", new MockFileData("Some text") },
{ #"/c/scans/mysecondfile.pdf", new MockFileData("Some text") },
{ #"/c/scans/mydog.jpg", new MockFileData("Some text") }
});
var fileService = new FileService(fileSystem);
var scanDirPath = #"/c/scans/";
I don't know exactly how to deal with this thing. I'm wondering about setting the "initial" path in the constructor of the xunit tests depending on the platform, but I'm not sure if it's a good practice.
I encountered the same scenario where I needed to execute a unit test with the System.IO.Abstraction.TestingHelpers's MockFileSystem on both Windows and Linux. I got it working by adding a check for the platform and then using the expected string format for that platform.
Following the same logic, your tests might look like this:
[Theory]
[InlineData(#"c:\scans\myfile.pdf", #"/c/scans/myfile.pdf")]
[InlineData(#"c:\scans\mysecondfile.pdf", #"/c/scans/mysecondfile.pdf")]
[InlineData(#"c:\scans\mydog.jpg", #"/c/scans/mydog.jpg")]
public void TestName(string windowsFilepath, string macFilepath)
{
// Requires a using statement for System.Runtime.InteropServices;
bool isExecutingOnWindows = RuntimeInformation.IsOSPlatform(OSPlatform.Windows);
bool isExecutingOnMacOS = RuntimeInformation.IsOSPlatform(OSPlatform.OSX);
MockFileSystem fileSystem;
if (isExecutingOnWindows)
{
fileSystem = new MockFileSystem(new Dictionary<string, MockFileData>
{
{ windowsFilepath, new MockFileData("Some text") }
};
}
else if (isExecutingOnMacOS)
{
fileSystem = new MockFileSystem(new Dictionary<string, MockFileData>
{
{ macFilepath, new MockFileData("Some text") }
};
}
else
{
// Throw an exception or handle this however you choose
}
var fileService = new FileService(fileSystem);
// Test logic...
}

Rhino Mocks testing file system io

I have an application that takes a dictionary of files (file type, and list of file names) and copies the files from the original directory into another location.
I've already got the basic code for the copy process, but I need to do some unit tests so it is as robust as possible.
I have wrapper class that I am using so I can test that the System.IO methods are called as I expect, but I am having some difficulty figuring out how to form the tests as there are foreach and switch statements in the code.
Sample code below:
private IFileSystemIO _f;
public CopyFilesToDestination(IFileSystemIO f){
_f = f;
}
public void Cpy_Files(Dictionary<string, List<string>> files)
{
// get a list of the file types in the directory
var listOfFileTypes = new List<string>(files.Keys);
foreach (var fileType in listOfFileTypes){
var fileList = files[fileType].ToList();
foreach (var file in fileList){
switch(fileType){
case ".txt":
_f.Copy(file, #"c:\destination\text");
break;
case ".dat":
_.Copy(file, #"c:\destination\data");
break;
}
}
}
}
To test the above I had thought I would use a mock dictionary object, set up with a list of file types and paths:
public virtual Dictionary<string, List<string>> FakeFiles(){
return fakeDictionary = new Dictionary<string, List<string>>(){
{".txt", new List<string>(){
"c:\test\file1.txt",
"c:\test\file2.txt"
}
},
{".dat", new List<string>(){
"c:\test\file1.dat",
"c:\test\file2.dat"
}
};
}
}
The first test I came up with looks like this:
[Test]
public void Should_Copy_Text_Files(){
var dictionary = new FakeDictionary().FakeFiles();
var mockObject = MockRepository.GenerateMock<IFileSystemIO>();
var systemUnderTest = new CopyFileToDestination(mockObject);
systemUnderTest.Cpy_Files(dictionary);
// I think this means "test the operation, don't check the values in the arguments" but I also think I'm wrong
mockObject.AssertWasCalled(f => f.Copy("something", "something"), o => o.IgnoreArguments());
}
My first problem is: How do I test for a specific file type, such as ".txt"?
Then how do I test the loops? I know with the mocked dictionary that I only have two items, do I leverage this to form the test? How?
I think I may be close to a solution, but I am running out of time/patience hunting it down. Any help is greatly appreciated.
Thanks
Jim
I tried using Roberts solution, but as I stated, I have too many different file types to set up each test case individually. The next thing I tried was setting up a TestCaseSource, but every time I ran the test for that it marked the test as ignored:
[Test, TestCaseSource(typeof(FakeDictionary), "TestFiles")]
public void Cpy_Files_ShouldCopyAllFilesInDictionary(string extension, string fielName) {
// Arrange
var mockObject = MockRepository.GenerateMock<IFileSystemIO>();
var systemUnderTest = new CopyFileToDestination(mockObject);
// Act
systemUnderTest.Cpy_Files(dictionary);
// Assert
mockObject.AssertWasCalled(f => f.Copy(extension, fileName));
}
The data source is below:
public static Dictionary<string, string> TestFiles{
get
{
return new Dictionary<string, string>()
{
{".txt",
"C:\\test\\test1.txt"},
{".txt",
"c:\\test\\test2.txt"}
};
}
}
What I finally worked out uses the times to repeat option in Rhino and is really pretty simple:
[Test]
public void Cpy_Files_ShouldCopyAllFilesInDictionary(){
// Arrange
var mockObject = MockRepository.GenerateMock<IFileSystemIO>();
var systemUnderTest = new CopyFileToDestination(mockObject);
// Act
systemUnderTest.Cpy_Files(dictionary);
// Assert
// I know how many objects are in my fake dictionary so I set the times to repeat as a const
const int timesToRepeat = 2;
// now I just set the values below. I am not testing file names so the test will ignore arguments
mockObject.AssertWasCalled(f => f.Copy("",""), options => options.Repeat.Times(timesToRepeat).IgnoreArguments());
}
I hope this helps someone else with a similar problem.
I would try making use of the TestCase attribute:
[TestCase(".txt", "c:\test\file1.txt")]
[TestCase(".txt", "c:\test\file2.txt")]
[TestCase(".dat", "c:\test\file1.dat")]
[TestCase(".dat", "c:\test\file2.dat")]
public void Should_Copy_Text_Files(string extension, string fileName){
var dictionary = new FakeDictionary().FakeFiles();
var mockObject = MockRepository.GenerateMock<IFileSystemIO>();
var systemUnderTest = new CopyFileToDestination(mockObject);
systemUnderTest.Cpy_Files(dictionary);
mockObject.AssertWasCalled(f => f.Copy(extension, fileName));
}
This will run the test separately for each TestCase attribute, passing the parameters it contains into the test method. That way you can test that each item in your dictionary was "copied" without using multiple asserts in the same test.

Categories

Resources