I've set up a build controller and build agent for my TFS 2012.
I have a .NET 4.0 project with coded UI tests that I would like to run there.
I set the build configuration to run with VS Test Runner.
The problem is that the test runner skipping all of my tests!
(I can see Skipped <test_name> for all my tests in the log view when I run the build with Diagnostic logging verbosity)
Any ideas?
update -
I've googled a lot on this and found that adding a .testsettings file in the build configuration should solve this, but when I added one, the build fails with the following error (found nothing about this error :( ):
MSTestAdapter could not run the test as the classic mode is not available. Reason: Exception has been thrown by the target of invocation.
Directory C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\PrivateAssemblies\DataCollectors does not exist
and with this warning:
No tests is available in . Make sure that installed test discovers & executors, platform & framework version settings are appropriate and try again.
To run coded ui test in build server, you needed to run the build agent as an interactive process (Build service property, Run as Interactive Process), you also need setup a test controller and an interactive test agent.
http://msdn.microsoft.com/en-us/library/vstudio/ms181712.aspx#interactive
http://msdn.microsoft.com/en-us/library/dd648127.aspx
in your testsettings you can sepcify the testagent and other settings
http://msdn.microsoft.com/en-us/library/ee256991.aspx
Related
The problem starts with 'dotnet test' command. For some reason, the project with tests can't find a shared library that should be available for it (see a screenshot attached). As you may notice, I'm running the task via Azure DevOps using a custom agent (I think it doesn't really matter). From the logs I realized that the reason it works locally on my machine is that the project I'm talking about is being built in Debug mode in Visual Studio and this is exactly the folder the test project is looking for the library (i.e. \bin\Debug\netstandard2.0\my.dll).
The issue appears when I run 'dotnet test' as a task in the build pipeline in Azure DevOps and the process does not create Debug folder for this exactly project. It's weird, because 'dotnet test' builds solution once again (in the debug mode) and the folder should be generated.
I tried switching the project from 'netstandard2.0' to 'netcoreapp2.2' and adding xunit testing libraries. They're not required in this project but it has started producing Debug folder with compiled libs in it (i.e. \bin\Debug\netcoreapp2.2\my.dll). However, it didn't resolve the issue because the test project still looking for the lib in 'netstandard2.0' folder.
I expect the test projects will be relying on the libs available in Release directory (instead of Debug). Can't get how to achieve this.
You are referencing the DLL in the test project , instead add the reference of the project itself to it.
DotTrace can be used to profile tests on TeamCity. But this example works with NUnit, while my tests are written in Visual Studio Tests. When running my tests I use the build-in test engine VSTest, no such option is available for the DotTrace profiler plugin. Instead I need to specify the path for the test engine executable.
I did find an executable VSTest.exe but it's path includes "Microsoft SDKs\Windows Phone\v8.0\ExtensionSDKs", so unsurprisingly it doesn't work. Saying it can't find a start.bat file, that does indeed not exist in that .exe file.
What is the executable path that TeamCity uses for VSTest?
Then I can configure the performance test to use that test engine. I know the tests can be executed using DotTrace, having run it in Visual Studio, just don't want to install Visual Studio and the DotTrace plugin there when there's a TeamCity plugin.
According to this, you can install Visual Studio Test Agent on your TeamCity Agent and then specify vstest.console.exe as a profiling target in the XML config.
Nevertheless, I suggest you to try a new dotTrace plugin version (if it asks you for credentials, specify guest with no password). It's not yet officially announced but fully functional.
The main difference is that the plugin is now integrated into 'unit testing' build steps, e.g. Visual Stiduo Tests, NUnit, MSpec, MSTest. So, it doesn't require a configuration file. Thus, after you install the plugin, all unit testing steps will get the additional option Run build step under dotTrace profiler. All you need to do is add the Visual Studio Tests step, check this option there and specify performance thresholds as usual.
I was asked to integrate another project (C#) into SonarQube. We use TFS 2013, SonarQube 5.1.2 with Sonar.MSBuild.Runner 1.0.
The Build process template is old (does not support pre and post Powershell scripts) and it is customized. Visual Studio 2013 is unable to edit the old build process template.
I tried to do just the static code analysis and run the MSBuild.SonarQube.Runner with the start parameter and a second time with the end parameter without doing a build in between (The code coverage does not seem important to them). But then I get the error "No ProjectInfo.xml files were found. Check that the analysis targets are referenced by the MSBuild projects being build"?
What options do I have to integrate this project into SonarQube?
To use the MSBuild SonarQube Runner from the command line, you have to:
Prepare the environment: MSBuild.SonarQube.Runner.exe begin /k:SQ_project_key /n:SQ_project_name /v:SQ_project_version (pick arbitrary SonarQube project key, name and version)
Run your build: msbuild
Upload analysis results: MSBuild.SonarQube.Runner.exe end
The MSBuild SonarQube Runner integrates with MSBuild and collects informations as it builds your project: You must run the build during the begin and end invocations, or this mandatory information won't be available and the MSBuild SonarQube Runner will fail with the message you reported. The MSBuild integration is compatible with MSBuild versions 4.0, 12 and 14.
Regarding the integration with TFS XAML builds, if you're using a customized one, you'll have to customize it a bit more to integrate the calls to the beginand end commands of the MSBuild SonarQube Runner.
Code Coverage is supported out of the box on TFS 2013 with the default build definition template, you just need to run tests with coverage enabled. If that doesn't work for you, see this link on how to manually pass the code coverage results path property: http://docs.sonarqube.org/x/CoBh
I'm trying to run unit tests in VisualStudio 2013. My tests are discovered under the Test Explorer but after running the tests, the project builds and stops. All my tests are then categorised under "Not Run".
I'm not getting any error messages or exceptions.
Thanks Jeroen Vannel & gunr2171.
I fixed it by installing xunit.runner to the test project from the NuGet Manager instead of running the installation of the xunit.runner Visual Studio Extension file I downloaded previously.
I try to run dotCover with my NUnit tests, in the TeamCity 8 as a build step. But no metter what I try I always get the same error in the log file:
Step 4/4: Coverage (NUnit) (1s)
[Step 4/4] Starting: C:\TeamCity\buildAgent\plugins\dotnetPlugin\bin\JetBrains.BuildServer.NUnitLauncher.exe #TeamCityImplicit
[Step 4/4] in directory: C:\TeamCity\buildAgent\work\6aee0f0d2626793d
[Step 4/4] ##teamcity[importData type='dotNetCoverage' tool='dotcover' file='C:\TeamCity\buildAgent\temp\buildTmp\coverage_dotcover3226256377023598081.data']
[Step 4/4] Importing data from 'C:\TeamCity\buildAgent\temp\buildTmp\coverage_dotcover3226256377023598081.data' with 'dotNetCoverage' processor
[Step 4/4] Rejected coverage report file: C:\TeamCity\buildAgent\temp\buildTmp\coverage_dotcover3226256377023598081.data size: 0. File is empty or does not exist
[Step 4/4] Process exited with code -2146232576
[Step 4/4] Step Coverage (NUnit) failed
I have tried to use both the included in TeamCity dotCover and the separately installed one, but both are failing with the same error.
My configuration:
If I choose no coverage tool, the tests work fine on its own. But with dotCover selected I always get the same error.
Any help here would be much appreciated.
Check out: http://confluence.jetbrains.com/pages/viewpage.action?pageId=49448495
In the case of internal TeamCity DotCover, you have to add the "ALL APPLICATION PACKAGES" read access rights to the TeamCity installation folder. If using an external DotCover, add the rights there.
This corrected the issue for me, for now.
It's probably because you are running this on build agent without .NET 3.5 (but with 4.0). It's easy to check because you will get exception just after running dotCover.exe on this agent. Install .NET 3.5 (using Server Manager in case of newest Windows).
The next thing is that there is another bug with dotCover in TeamCity 8. You need to use trial dotCover installation until they fix it (and put the path to installed dotCover into the build step configuration).
It solved the problem in my case.
Interestingly, for me at least, it worked for me once I disabled code coverage (i.e. un-selected an item in the .NET Coverage tool drop-down) and ran the build. When I re-enabled code coverage it worked (with no other changes to anything in TeamCity or on the build server). Weird!
I'm currently running TeamCity version 8.0.4 (build 27616).
This is similar to my issue, where the NUnit fails to run when dotCover is enabled.
I resolved this by setting .NET Runtime: Platform: as x86 in the configuration dropdown and I think the error is due to team city fails to figure out which is the correct platform.