Does MSTest have standalone GUI similar to nUnit that lets me use it and run test without visual studio? What is the official site for MSTest where I can learn more about how to use it?
MSTest can be used without installing Visual Studio. You will need to install Visual Studio Test Agent, which is a free download from Microsoft.
I think this approach is better from a licensing perspective than manually copying MSTest.exe and its dependencies onto the build server.
See this blog for reference:
http://blogs.msdn.com/b/anutthara/archive/2009/12/16/running-tests-in-mstest-without-installing-the-vs-ide.aspx
It doesn't have a GUI (apart from Visual Studio) but there's a command line tool: MSTest.exe
Here is the official documentation on running MSTest tests.
You can do this with mstest.exe, but the trick is in getting it to work without installing visual studio. This involves the copying of several files and registry entries. I have blogged about it here.
Use Gallio as your test runner... then its not so much of a drama when you enventually drop MsTest and move to a real test framework.
Use VSTest.console.exe part of Microsoft.TestPlatform
Required steps:
Download the test platform from https://www.nuget.org/packages/Microsoft.TestPlatform/
Unzip
In the unzipped folder, copy the \tools\net451\Common7\IDE\Extensions\TestPlatform folder to the machine that has no Visual Studio installed
From cmd.exe run VSTest.console.exe MyTest.dll
More details here:https://learn.microsoft.com/en-us/visualstudio/test/vstest-console-options?view=vs-2017#general-command-line-options
You can also use this tool from codeplex: http://testrunner.codeplex.com...
Uset "dotnet test", it is compatible with all versions, eg:
'dotnet test yourassembly.dll -l console -v detailed'
Related
I'm starting with BDD and spec-flow in C# and Visual Studio. I was following a tutorial at https://www.toolsqa.com/specflow/set-up-specflow/ to set up a project but I cannot be shown the tests in visual studio 2017 to run.
The steps I take are the following:
I create a project as a class library in the .NET Framework
I add spec-flow to the project
I add the NuGet Nunit test adapter and Nunit Framework (version3)
I generate the .feature file
I generate the file steps associated with the feature.
I compile the solution but in the test window I don't see any tests that run
I searched for several tutorials but I can't find what my mistake is or what I'm missing.
Can anyone help me?
Install the "SpecFlow.Tools.MsBuild.Generation" nuget package and rebuild the solution. That should get you going.
We have a "Getting started" guide on the website: https://specflow.org/getting-started/
Also we have some examples here: https://github.com/techtalk/SpecFlow-Examples
You could compare your project with the example if you missed something.
I had the same issue . And when I upgraded my specflow to 3.0.213 , feature.cs and tests were not generated in test explorer. Then I used this MSBUild example with latest specflow to generate feature.cs and tests in test explorer. It uses MSTest as test runner and not Nunit. If test runner doesn't matter to you, use this example. I am running it on visual studio 2019
Had the same issue,
Besides adding the nuget package
SpecFlow.Tools.MsBuild.Generation
as Meza mentioned.
The testes did not appear in the Test Explorer before i added
SpecFlow.NUnit.Runners
besides already having
SpecFlow
SpecFlow.NUnit
NUnit
NUnit3TestAdapter
and the Visual Specflow extension
(VS2019, Specflow 3.1.67)
As an update these were the nuget packages I had in place to finally get an example working, using selenium running under NUnit. (It may be useful to someone)
It is better for you to use visual studio 2015 to do the same things that you want.
I'm using Selenium with C# to create my automated tests. Now i have to use some sort of reporting tools to save the test cases whether they passed,failed...etc.
I've seen a lot of tools like Allure, and Jenkins. But they require an XML file. and i can't find it when i run my tests using Visual 2013. Why is that ?
how to obtain these XML files in details if i'm missing something ?
Is there any way i could achieve this with minimum effort ?
EDIT:
How can i run my tests using Nunit console runner ? where can i find it? i downloaded the .zip for nunit but i couldn't find the runner?
The NUnit Visual Studio Adapter does not currently produce XML results, although there is an issue on GitHub to add the ability. Your best bet is to run your tests using the NUnit Console runner when you want to create reports. It always produces an XML result file.
You can install the NUnit Console by adding the NUnit.Console NuGet package to your test project. The console will be in a directory like packages\NUnit.ConsoleRunner.3.4.0\tools in your solution root.
Another option is to install the MSI for the console runner. It will then be under C:\Program Files (x86)\NUnit.org\nunit-console
For creating reports, ReportUnit creates excellent HTML reports from your test results.
To run the tests, I would recommend setting up a simple command line build that builds your solution, runs the tests and then produces the report. For day to day development, just running your tests in Visual Studio or on the command line will likely be enough. Most developers find the Visual Studio Test Explorer Window to be a poor UI, but usable for seeing passed and failed tests and running/debugging them.
If you want to setup a command line build, one good tool is Cake Build. It will take a bit of time to setup, but it is an excellent way to run your build tasks as your project gets larger.
NUnit testing can be run with the nunit-console.exe application which is installed with nunit under {Project_root}/lib/nunit/nunit-console.exe. It downloads with NuGet when NUnit installs.
It can be passed a list of testing binaries, or testing project files, or an nunit project (listing multiple if needed).
{PathToProject}\lib\nunit\nunit-console.exe {PathToTestDll}\Project1.Tests.dll {PathToTestDll}\Project2.Tests.dll /xml=nunit-result.xml
or create an NUnit Project with the NUnit Project Editor if you want to group all your test projects into a single config file.
{PathToProject}\lib\nunit\nunit-console.exe {PathToNUnitProject}\Project.Tests.nunit /xml=nunit-result.xml
Allure doesn't require xml. At lease now :)
https://github.com/unickq/allure-nunit
You just use [AllureNUnit] attribure
I am trying to use OpenCover for a calculator application, written in C#.
I have opened it in Visual Studio 2013 and installed the OpenCover and Report Generation packages using the package manager console. I cannot understand the other commands to continue. Specifically what does it mean by -target<target application>?
I cannot understand a single step to continue.
When you install OpenCover it would/should have presented you with a readme.txt that provides a link to the wiki http://www.github.com/OpenCover/OpenCover/wiki/usage - there is also a usage.pdf installed with the tool.
-target means the target application, this can be the app you have just written or a coverage tool such as nunit that would run parts of your application code whilst it is running the supplied tests.
If you're happy with any code coverage without it specifically needing to be OpenCover, then see my answer here.
Using the AxoCover plugin makes it very quick and simple to get your NUnit tests displaying their code coverage.
DotTrace can be used to profile tests on TeamCity. But this example works with NUnit, while my tests are written in Visual Studio Tests. When running my tests I use the build-in test engine VSTest, no such option is available for the DotTrace profiler plugin. Instead I need to specify the path for the test engine executable.
I did find an executable VSTest.exe but it's path includes "Microsoft SDKs\Windows Phone\v8.0\ExtensionSDKs", so unsurprisingly it doesn't work. Saying it can't find a start.bat file, that does indeed not exist in that .exe file.
What is the executable path that TeamCity uses for VSTest?
Then I can configure the performance test to use that test engine. I know the tests can be executed using DotTrace, having run it in Visual Studio, just don't want to install Visual Studio and the DotTrace plugin there when there's a TeamCity plugin.
According to this, you can install Visual Studio Test Agent on your TeamCity Agent and then specify vstest.console.exe as a profiling target in the XML config.
Nevertheless, I suggest you to try a new dotTrace plugin version (if it asks you for credentials, specify guest with no password). It's not yet officially announced but fully functional.
The main difference is that the plugin is now integrated into 'unit testing' build steps, e.g. Visual Stiduo Tests, NUnit, MSpec, MSTest. So, it doesn't require a configuration file. Thus, after you install the plugin, all unit testing steps will get the additional option Run build step under dotTrace profiler. All you need to do is add the Visual Studio Tests step, check this option there and specify performance thresholds as usual.
I've got integration tests which I launch from Visual Studio 2008 Development Edition (they are executed as MSTest unit-tests, even though they are actually integration tests). I would like to do some profiling while the tests are being carried out.
In my case, I have an MSTest "ordered" test I'm running. I'd like to collect sampling data while each of the ordered tests is executed.
How do I do this?
p.s. Perhaps I can accomplish this from the command line, if I manually launch mstest from a command prompt, while trying to follow command line directions. Unfortunately, when I try to launch VsPerfCLREnv from the Visual Studio command prompt, it is "file not found." What the heck?
I just got profiling working in VS2008. In case it helps anyone, here is how I got it to work:
Ensure you are in Debug configuration mode
Click Analyse > Launch Performance Wizard
Select Profile a dynamic link library
Enter the following:
Dll path: The dll you want to instrument, located in the bin\Debug of your unit test project
Executable path: C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe
Start Option Arguments: /testmetadata:TestList.vsmdi /runconfig:TestRunConfig1.testrunconfig
Adjust this based on the name of your vsmdi and testrunconfig files.
Working directory: Your solution root path
Choose Instrumentatation. I haven't tried Sampling, so I cannot comment on that.
Run the new performance project
Using this approach I identified the bottleneck in my code, and the unit tests are now 15x faster, and the users are seeing a 10x speed improvement.
Unfortunately Ordered Tests don't have the 'Create Performance Session' option available:
If you had a simple unit test you would be able to create a performance session.
To use command-line profiling you can navigate to \Program Files\Microsoft Visual Studio 9.0\Team Tools\Performance Tools where you will see vsperfclrenv.cmd and other tools mentioned on the profiler blog.