Develop Reference

Running (dotnet test command) unit/integration tests dll on remote server through power shell requires csproj file

I have my dlls for the test project deployed to the server. Now I'm trying to run them using dotnet test "Path to tests.dll" but I get an error:
F:\path\tests.dll(1,1): error MSB4025: The project file could not be loaded. Data at the root level is invalid. Line 1, position 1.
Basically it requires csproj file to be there in the same catalog as it was on my local machine. What's the point of having runnable DLLs for testing if I still need csproj to run the tests on the remote server. That doesn't make any sense.
How can I run the tests without having to have csproj file on the server?

I found the answer to my problem. In order to run the tests on a remote server through PowerShell, I had to use "dotnet vstest" instead of "dotnet test". vstest command allows running the tests by simply using dll file without the need for csproj file.

C# Coverlet results always empty

have .net core 3.1 Microsoft.net.sdk projects, with lots of async xUnit tests.
tried - adding coverlet.msbuild 2.9.0 to the project, and then running:
dotnet test Common\Common.csproj /p:CollectCoverage=true /
got 100% displayed, but an empty coverage file created
tried - adding coverlet.collector 1.3.0 to the project and then running:
dotnet test Common\Common.csproj --collect:"XPlat Code Coverage"
got a file created in testresults\{guid}\coverage.cobertura.xml - but it just says lines-covered=0
whereas stdout is saying 88 tests run in 4s. What am I doing wrong?

For me coverlet.msbuild works perfect with command:
dotnet test Common\Common.csproj /p:CollectCoverage=true /p:IncludeTestAssembly=true /p:CoverletOutputFormat=cobertura /p:ExcludeByFile=\"**/Microsoft.NET.Test.Sdk.Program.cs\"
So, I guess you missed CoverletOutputFormat here.

How do I pass runtime parameters to dotnet test on the command line?

I have a set of unit tests in a .NET Core project and using a runsettings file. I am trying to setup Azure DevOps to do automated testing on my deployments. As part of this process, I need to override parameters from the run settings on the command line.
I have a runsettings file with the following section:
<TestRunParameters>
<Parameter name="ApiUrl" value="https://myurl..." />
</TestRunParameters>
I have a static constructor that saves the TestContext like this:
[ClassInitialize]
public static void TestClassInitialize(TestContext context)
{
TestContext = context;
}
I am retrieving settings from the TestContext with the following method:
protected string GetStringSetting(string settingName)
{
return TestContext.Properties[settingName] as string;
}
When I run the test with the runsettings file selected, it gets the TestContext and I see the ApiUrl entry is retrieved successfully.
Now, I want to set this parameter from the command line. I am using a command like:
dotnet test <myproject.csproj> --configuration Release -- ApiUrl=https://newurl
I get an error that says the dictionary does not contain the key "ApiUrl" which indicates that the setting was not processed. From reading documentation, I thought that maybe I need to fully specify the name of the setting with TestRunParameters.ApiUrl. This gives me an XML error.
From everything I have read, I think I am doing this right and can't figure out what is wrong. I am using version 2.1.503 of the tools.
Can someone please give me guidance on how to make this work?

You can use the following argument syntax to do that:
dotnet test <myproject.csproj> --configuration Release -- TestRunParameters.Parameter(name=\"ApiUrl\", value=\"https://newurl\")
Source: https://github.com/Microsoft/vstest-docs/blob/master/docs/RunSettingsArguments.md

You could inject environment variables from your pipeline to overcome the limitations of the dotnet test command. By doing so, you will not need to deal with a temporary *.runsettings file to get your test parameters. You can set environment variables from your CI pipeline and then retrieve them in your tests at runtime.
Your YAML file could set environment variables:
run_tests:
# Set environment variables as you need
variables:
FOO: "bar"
Your test can retrieve environment variables:
[Test]
public void Test()
{
var foo = Environment.GetEnvironmentVariable("FOO"); //set to "bar"
}

This is currently impossible to achieve. Check those closed issues on github:
Runsettings from command line (--) doesnt have capability to pass
TestRunParameters
dotnet test with TestRunParameters defined
should pass parameters to the test runner
Your only option in here is to create another runsettings file with new values and pass it to dotnet test with --settings flag.

You can't substitute the TestRunParameters from the command-line when using the DotNetCoreCLI#2 task or the dotnet test command, but you can create a temporary .runsettings file that contains the valid parameter value as part of your build pipeline in Azure Pipelines:
- powershell: |
[xml]$doc = Get-Content Tests/Settings.runsettings
$doc.RunSettings.TestRunParameters.ChildNodes.Item(0).value = 'https://newurl'
$doc.Save("$(Build.SourcesDirectory)/Tests/UpdatedSettings.runsettings")
displayName: 'Override TestRunParameters'
- task: DotNetCoreCLI#2
displayName: 'Run Tests'
inputs:
command: test
projects: 'Tests/Tests.csproj'
arguments: '-s $(Build.SourcesDirectory)/Tests/UpdatedSettings.runsettings'
- powershell: |
Remove-Item $(Build.SourcesDirectory)/Tests/UpdatedSettings.runsettings
displayName: Remove temporary .runsettings file
You'll find a complete YAML file and a sample test project on GitHub.

Absolute path in dotnet watch run command doesn't work

To run dotnet core application with specified absolute path we need to run following command:
dotnet run -p C:\foo\bar\Project\Project.csproj
But it seems it doesn't work the same with dotnet watch run:
watch : Could not find a MSBuild project file in 'C:\directory\where\we\execute\command'. Specify which project to use with the --project option.
Running the same command with -project instead of -p doesn't help however...
Dotnet watch help specifies -p or -project parameter anyway:
Microsoft DotNet File Watcher 2.1.1-rtm-30846
Usage: dotnet watch [options] [[--] ...]
Options: -?|-h|--help Show help information
-p|--project The project to watch -q|--quiet Suppresses all output except warnings and errors -v|--verbose
Show verbose output --list Lists all discovered
files without starting the watcher --version Show
version information
Environment variables:
DOTNET_USE_POLLING_FILE_WATCHER When set to '1' or 'true',
dotnet-watch will poll the file system for changes. This is required
for some file systems, such as network shares, Docker mounted
volumes, and other virtual file systems.
DOTNET_WATCH dotnet-watch sets this variable to '1' on all child
processes launched.
Remarks: The special option '--' is used to delimit the end of the
options and the beginning of arguments that will be passed to the
child dotnet process. Its use is optional. When the special option
'--' is not used, dotnet-watch will use the first unrecognized
argument as the beginning of all arguments passed into the child
dotnet process.
For example: dotnet watch -- --verbose run
Even though '--verbose' is an option dotnet-watch supports, the use
of '--' indicates that '--verbose' should be treated instead as an
argument for dotnet-run.
Examples: dotnet watch run dotnet watch test
What's wrong then? Why absolute path to project doesn't work with dotnet watch run while works with dotnet run?

You can resolve this by specifying the -p (or the longer --project) option on the watch command rather than on the run command. In your case, that would be:
dotnet watch -p C:\foo\bar\Project\Project.csproj run
There's a note in the docs that covers this:
You can use dotnet watch --project <PROJECT> to specify a project to watch. For example, running dotnet watch --project WebApp run from the root of the sample app will also run and watch the WebApp project.

I'm not 100% sure, but dotnet watch is looking for file changes in the current directory. So if you use absolute path it must know where should it looks for changes. Of course, such implementation is possible but I just think that nobody thinked about it when implementing watch command

In my case, its just a minor error, you have to enter in the project directory before executing dotnet command, like:
cd yourAppName
dotnet watch run
It'll run

.NET Core Unit tests not shown in AppVeyor Tests window (and badge)

Follow up from this question, I'm currently setting up AppVeyor for my project (here) and my .NET Core tests are only shown in the console output but not in the Tests window.
This is the link for the AppVeyor project: ci.appveyor.com/project/Sergio0694/neuralnetwork-net
If some tests fail, the console correctly shows an error and the build is marked as failing, but the Tests window is empty anyways. Same goes for the badge from shields.io which shows 0 total tests, even if I can see many of them being executed from the console output.
Here's the console output:
And here's the Tests window:
Is there something else I have to setup in order for them to be reported correctly outside the console window?

Please add https://www.nuget.org/packages/Appveyor.TestLogger to your test projects.

An arguably cleaner alternative to adding an otherwise unused reference to your test project is to do this in your test script:
cd <test_project_dir>
nuget install Appveyor.TestLogger -Version 2.0.0
cd ..
dotnet test --no-build --no-restore --test-adapter-path:. --logger:Appveyor <test_project_dir>
This has the same effect as adding the reference, in that it makes the testlogger binary available to the test framework, but it doesn't actually change the test project, and therefore doesn't require someone who's not using Appveyor to install the package when they clone and build your repo.
The slight advantage of this solution over outputting and subsequently uploading .trx files (as in the PS script above) is that you should get the test results in real-time, rather than all at the end.
Example appveyor.yml:
version: 0.0.{build}
build_script:
- cmd: dotnet build MySolution.sln
test_script:
- cmd: cd Test
- cmd: nuget install Appveyor.TestLogger -Version 2.0.0
- cmd: cd ..
- cmd: dotnet test --no-build --no-restore --test-adapter-path:. --logger:Appveyor Test

You can add the AppVeyor.TestLogger package to your project, but it can be done without changing your code. You need to output your tests results into an xml file format that AppVeyor understands and then upload it to their HTTP API. The following powershell snippet will iterate through your solution and find each test project, call dotnet test on the csproj and log the output to test-result.trx and then upload the file to AppVeyor.
$config = "release"
# Find each test project and run tests and upload results to AppVeyor
Get-ChildItem .\**\*.csproj -Recurse |
Where-Object { $_.Name -match ".*Test(s)?.csproj$"} |
ForEach-Object {
# Run dotnet test on the project and output the results in mstest format (also works for other frameworks like nunit)
& dotnet test $_.FullName --configuration $config --no-build --no-restore --logger "trx;LogFileName=..\..\test-result.trx"
# if on build server upload results to AppVeyor
if ("${ENV:APPVEYOR_JOB_ID}" -ne "") {
$wc = New-Object 'System.Net.WebClient'
$wc.UploadFile("https://ci.appveyor.com/api/testresults/mstest/$($env:APPVEYOR_JOB_ID)", (Resolve-Path .\test-result.trx))
}
# don't leave the test results lying around
Remove-Item .\test-result.trx -ErrorAction SilentlyContinue
}