Develop Reference

Gitlab does not find csv files required for testing | CI/CD pipeline

I have set up a pipeline for a C# dotnet project. It builds correctly, however the tests do not run - gitlab can't seem to find the csv files that are required for the tests. The CSV files are committed to gitlab and show up in the gitlab build folder. however the error I get is " Error Message:
System.IO.DirectoryNotFoundException : Could not find a part of the path 'C:\Windows\TEMP\Data\xxxxxx.csv'."
I would have thought it would look in the gitlab build folder for the project, but instead it seems to look in C:\windows\temp\Data for the files. Might this be a misconfiguration issue? The pipeline script looks like this:
variables:
SOURCE_CODE_PATH: '"$CI_PROJECT_DIR"'
TEST_DIRECTORY: '"$ProjectName.Test
cache:
paths:
- '$SOURCE_CODE_PATH$TEST_DIRECTORY/Data'
before_script:
- nuget locals all -clear
- echo "starting build for $CI_PROJECT_NAME"
- echo "Restoring NuGet packages..."
- nuget restore "$CI_PROJECT_DIR"
- echo "NuGet Packages restored..."
stages:
- build
- test
build:
tags:
- ProjectName
stage: build
script:
- echo "Release build..."
- 'dotnet build /consoleloggerparameters:ErrorsOnly /maxcpucount /nologo /property:Configuration=Release /verbosity:normal "$CI_PROJECT_DIR"'
artifacts:
untracked: false
test:
tags:
- ProjectName
script:
- echo "starting tests"
- 'dotnet test "$CI_PROJECT_DIR"'
Data is the folder where the files for the tests are stored, so I tried caching them. It doesn't seem to help. The testing framework in use is nunit. Any ideas on what I may be missing or getting wrong here?

using the docfx.console nuget package on linux

At the moment I have a visual studio project and I use the docfx.console nuget package to build the documentation, and everything works fine and as expected... on windows. The point is now I want to make a docker image based on mcr.microsoft.com/dotnet/core/sdk:3.1 which is based on a linux image. And compiling in this docker image running the command:
dotnet publish -c Release -o out
Gives the following error
> [build 9/9] RUN dotnet publish -c Release -o out:
#22 1.080 Microsoft (R) Build Engine version 16.0.450+ga8dc7f1d34 for .NET Core
#22 1.080 Copyright (C) Microsoft Corporation. All rights reserved.
#22 1.080
#22 2.852 Restore completed in 215.94 ms for /app/Documentation/Documentation.csproj.
#22 6.299 Documentation -> /app/Documentation/bin/Release/netcoreapp2.1/Documentation.dll
#22 6.402 /bin/sh: 2: /tmp/tmpbd72ebbe5e6b49c1b3244f1f50c8b57a.exec.cmd: /root/.nuget/packages/docfx.console/2.48.1/build/../tools/docfx.exe: Exec format error
#22 6.407 /root/.nuget/packages/docfx.console/2.48.1/build/docfx.console.targets(57,5): error MSB3073: The command ""/root/.nuget/packages/docfx.console/2.48.1/build/../tools/docfx.exe" "/app/Documentation/docfx.json" -o "" -l "log.txt" --logLevel "Verbose"" exited with code 2. [/app/Documentation/Documentation.csproj]
I already did some prodding around and I believe I have the issue mostly solved. Running file on console.exe shows that this is a PE32 executable (console) Intel 80386 Mono/.Net assembly, for MS Windows. And these kind of files should not be executed on linux using sh but with mono. And indeed running:
mono docfx.exe "/app/Documentation/docfx.json" -o "" -l "log.txt" --logLevel "Verbose"
builds the documentation just fine as expected. At this point of course I have a bunch of workarounds to get the documentation building correctly, just remove docfx.console form the csproj and build it manualy from the command line using a docker command.
But the question is, can I also use the nuget package on linux by changing how the docfx.exe command is run by the nuget package? Or is this only possible by actually fixing this in docfx.console?
p.s. in case it matters, the version of docfx.console that I am using is the most recent one available at the time of writing, namely 2.48.1

But the question is, can I also use the nuget package on linux by changing how the docfx.exe command is run by the nuget package? Or is this only possible by actually fixing this in docfx.console?
Create script docfx that runs docfx.exe using Mono, e.g., like this (assuming docfx.exe is located in /opt/docfx/docfx.exe):
echo '#!/bin/bash\nmono /opt/docfx/docfx.exe $#' > /usr/bin/docfx && chmod +x /usr/bin/docfx
Then, pass MSBuild parameter BuildDocToolPath with path to that script, e.g., like this:
dotnet publish -c Release -o out -p:BuildDocToolPath=/usr/bin/docfx
docfx.console will than use this path to execute DocFX. I think the property BuildDocToolPath isn't documented anywhere, but you can see it in source code.

How to run Selenium tests in GitHub Actions

I'm trying to find the equivalent command to run Selenium tests in GitHub actions. In Azure DevOps, I'd use this YAML to run "Visual Studio Test":
- task: VSTest#2
displayName: 'Run functional smoke tests on website and web service'
inputs:
searchFolder: '$(build.artifactstagingdirectory)'
testAssemblyVer2: |
**\FeatureFlags.FunctionalTests\FeatureFlags.FunctionalTests.dll
uiTests: true
runSettingsFile: '$(build.artifactstagingdirectory)/drop/FunctionalTests/FeatureFlags.FunctionalTests/test.runsettings'
overrideTestrunParameters: |
-ServiceUrl "https://$(WebServiceName)-staging.azurewebsites.net/"
-WebsiteUrl "https://$(WebsiteName)-staging.azurewebsites.net/"
In GitHub actions, what is the equivalent task? It doesn't look like the VsTest.Console.exe exists in the GitHub runner/agent, so the answer may involve installing the Visual Studio Test Platform installer - but obviously I'd like to avoid this as this would severely slow down each build.

Thanks to #Eldar who helped point me in the right direction. The basic answer is (running in a Windows runner):
- name: Functional Tests
run: |
$vsTestConsoleExe = "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise\\Common7\\IDE\\Extensions\\TestPlatform\\vstest.console.exe"
$targetTestDll = "functionaltests\FeatureFlags.FunctionalTests.dll"
$testRunSettings = "/Settings:`"functionaltests\test.runsettings`" "
#Note that the `" is an escape character to quote strings, and the `& is needed to start the command
$command = "`& `"$vsTestConsoleExe`" `"$targetTestDll`" $testRunSettings "
Write-Host "$command"
Invoke-Expression $command
shell: powershell
Using Invoke-Expression seemed to help with some initial errors I was receiving. The complete yaml, with context, is viewable on GitHub:
Note to find the vstest.console.exe file, I used this yaml to search the GitHub Actions runner. There are folders that are locked, so this throws an error if you search on the root C: folder. With the link provided above, I was able to establish that the root folder for Visual Studio and search for vstest.console.exe there:
- name: search for visual studio test runner
run: |
$var1 = Get-Childitem -Path "C:\Program Files (x86)\Microsoft Visual Studio" -Filter "vstest.console.exe" -Recurse | select -ExpandProperty FullName
Write-Host "VS test runner: $var1"
shell: powershell

This is now a lot simplier thanks to the Setup VSTeset.console.exe action that is available on the Marketplace.
It still requires running on a Windows agent but the number of YAML lines is significantly reduced.
The following snippet assumes that an artifact named "functional-tests" that contains the compiled assets (DLLs, chromedriver.exe etc) for your Selenium tests has been uploaded then downloaded into "./functional-tests/".
- name: Add VSTest.console.exe to the PATH
uses: darenm/Setup-VSTest#v1
- name: Run functional tests
run: vstest.console.exe "functional-tests\<functional-tests-project-dll-name-here>.dll"
My experience has been that if you set environment variables at the job level, these will automatically be picked up by vstest.console.exe. This probably holds true for setting them at the step or workflow level but I have not tested that.

.NET Core Unit tests not shown in AppVeyor Tests window (and badge)

Follow up from this question, I'm currently setting up AppVeyor for my project (here) and my .NET Core tests are only shown in the console output but not in the Tests window.
This is the link for the AppVeyor project: ci.appveyor.com/project/Sergio0694/neuralnetwork-net
If some tests fail, the console correctly shows an error and the build is marked as failing, but the Tests window is empty anyways. Same goes for the badge from shields.io which shows 0 total tests, even if I can see many of them being executed from the console output.
Here's the console output:
And here's the Tests window:
Is there something else I have to setup in order for them to be reported correctly outside the console window?

Please add https://www.nuget.org/packages/Appveyor.TestLogger to your test projects.

An arguably cleaner alternative to adding an otherwise unused reference to your test project is to do this in your test script:
cd <test_project_dir>
nuget install Appveyor.TestLogger -Version 2.0.0
cd ..
dotnet test --no-build --no-restore --test-adapter-path:. --logger:Appveyor <test_project_dir>
This has the same effect as adding the reference, in that it makes the testlogger binary available to the test framework, but it doesn't actually change the test project, and therefore doesn't require someone who's not using Appveyor to install the package when they clone and build your repo.
The slight advantage of this solution over outputting and subsequently uploading .trx files (as in the PS script above) is that you should get the test results in real-time, rather than all at the end.
Example appveyor.yml:
version: 0.0.{build}
build_script:
- cmd: dotnet build MySolution.sln
test_script:
- cmd: cd Test
- cmd: nuget install Appveyor.TestLogger -Version 2.0.0
- cmd: cd ..
- cmd: dotnet test --no-build --no-restore --test-adapter-path:. --logger:Appveyor Test

You can add the AppVeyor.TestLogger package to your project, but it can be done without changing your code. You need to output your tests results into an xml file format that AppVeyor understands and then upload it to their HTTP API. The following powershell snippet will iterate through your solution and find each test project, call dotnet test on the csproj and log the output to test-result.trx and then upload the file to AppVeyor.
$config = "release"
# Find each test project and run tests and upload results to AppVeyor
Get-ChildItem .\**\*.csproj -Recurse |
Where-Object { $_.Name -match ".*Test(s)?.csproj$"} |
ForEach-Object {
# Run dotnet test on the project and output the results in mstest format (also works for other frameworks like nunit)
& dotnet test $_.FullName --configuration $config --no-build --no-restore --logger "trx;LogFileName=..\..\test-result.trx"
# if on build server upload results to AppVeyor
if ("${ENV:APPVEYOR_JOB_ID}" -ne "") {
$wc = New-Object 'System.Net.WebClient'
$wc.UploadFile("https://ci.appveyor.com/api/testresults/mstest/$($env:APPVEYOR_JOB_ID)", (Resolve-Path .\test-result.trx))
}
# don't leave the test results lying around
Remove-Item .\test-result.trx -ErrorAction SilentlyContinue
}

Using GitLabCI with C#

I've been working on a C# application and wanted to try the GitLab CI out. All I can see is Ruby and can't find any information on how to build a C# application using it.
When I run the test settings, I make the commit, but I don't have my build.
How should I make a simple build? Which command could I use for that? I don't mind if I get a failed build (but a build).

I just wanted to share my .gitlab-ci.yml complete with unit testing. You will have to adjust your nuget and possibly other paths. This is for a single project in a solution of the same name.
variables:
PROJECT_NAME: "ProjectNameGoesHere"
before_script:
- echo "starting build for %PROJECT_NAME%"
- echo "Restoring NuGet Packages..."
- d:\tools\nuget restore "%PROJECT_NAME%.sln"
stages:
- build
- test
build:
stage: build
script:
- echo "Release build..."
- '"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\msbuild.exe" /consoleloggerparameters:ErrorsOnly /maxcpucount /nologo /property:Configuration=Release /verbosity:quiet "%PROJECT_NAME%.sln"'
artifacts:
untracked: true
test:
stage: test
script:
- echo "starting tests"
- cd %PROJECT_NAME%Tests/bin/Release
- '"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe" /testcontainer:%PROJECT_NAME%Tests.dll'
dependencies:
- build

In order to build a C# application you should have a Windows runner (with shell executor) configured for a project in GitLab CI.
Your .gitlab-ci.yml file should look something like that:
stages:
- build
job:
stage: build
script:
- echo "Restoring NuGet Packages..."
- '"c:\nuget\nuget.exe" restore "MySolution.sln"'
- ''
- echo "Release build..."
- C:\Windows\Microsoft.NET\Framework64\v4.0.30319\msbuild.exe /consoleloggerparameters:ErrorsOnly /maxcpucount /nologo /property:Configuration=Release /verbosity:quiet "MySolution.sln"
tags:
except:
- tags
On a Windows machine you need the following tools:
Runner installed
Git, added to PATH
Latest nuget.exe at C:\nuget (or somewhere else. Just make sure you got the path right in the .gitlab-ci.yml file)

The other answers are good. But I'd like to explain how to install a runner in addition. I use my own local system (Windows), so I chose to run shell. But you could use a Docker image if you'd like.
cd C:\Multi-Runner
gitlab-ci-multi-runner register
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com )
https://gitlab.com
Please enter the gitlab-ci token for this runner
xxx
Please enter the gitlab-ci description for this runner
my-runner
INFO[0034] fcf5c619 Registering runner... succeeded
Please enter the executor: shell, docker, docker-ssh, ssh?
shell
INFO[0037] Runner registered successfully. Feel free to start it, but if it's
running already the config should be automatically reloaded!
Source: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/install/windows.md
Afterwards, you can use a YAML file a like this:
stages:
- build
job:
stage: build
script: '"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\msbuild.exe" "something.sln"'

Installing the build runner on a Windows machine helps a lot, and #prasanth-louis has a great example on how to do that.
As for the .gitlab-ci.yml file, you can simplify it even more by using Cake Build:
stages:
- build
build:
stage: build
script:
- .\build.ps1 -Target Build
tags:
- windows
And your build.cake file can look like this (based of the example repository):
#tool nuget:?package=NUnit.ConsoleRunner&version=3.4.0
var target = Argument("target", "Default");
var configuration = Argument("configuration", "Release");
var solution = "./example-project.sln";
var buildDir = Directory("./example-project/bin");
Task("Default")
.IsDependentOn("Unit-Tests")
.Does(() =>
{
Information("Running Default task!");
});
Task("Clean")
.Does(() =>
{
CleanDirectory(buildDir);
});
Task("PackageRestore")
.IsDependentOn("Clean")
.Does(() =>
{
Information("Restoring NuGet packages for {0}", solution);
NuGetRestore(solution);
});
Task("Build")
.IsDependentOn("PackageRestore")
.Does(() =>
{
Information("Restoring NuGet packages for {0}", solution);
MSBuild(solution, settings => settings.SetConfiguration(configuration));
});
Task("Unit-Tests")
.IsDependentOn("Build")
.Does(() =>
{
NUnit3("./example-project.Tests/**/bin/" + configuration + "/*.Tests.dll");
});
Task("Publish")
.Does(() =>
{
});
RunTarget(target);

Here my working .gitlab-ci.yml file for c# application with NUnit as unit test framework and mono as basic image.
Not very fancy but working:
image: mono:latest
stages:
- build
- test
variables:
solution: "Project.sln"
test: "Project.Test"
before_script:
- nuget restore
build:
stage: build
script:
- msbuild /p:Configuration=Release $solution
test:
stage: test
script:
- msbuild /p:Configuration=Release $solution
- mono ./packages/NUnit.ConsoleRunner.3.10.0/tools/nunit3-console.exe ./$test/bin/Release/$test.dll

The other answers, while informative, miss a few crucial bits of info:
If you are targetting .net core 3.1, .net 5, .net 6 or any newer version then you can build and run your app on linux
If you are targetting .net framework (4.8 or below) then you have to build and run your code on a windows machine.
Now Gitlab runs build jobs using runners (aka build agents), that in turn use executors to specifiy the environment in which the build process happens. Runners can run on the Gitlab infrastructure (Saas runners), or can be installed on premise. There are several types of executors: docker, shell, custom, ...
Most of the gitlab build scripts that can be found around the internet assume a saas runner with a docker executor running on linux, which is based on a specific docker image. This won't work if you want to build your app on windows. For this, either install your own executor, as several other answers instruct to do, or use a windows saas runner which, although in beta, works fine even for production (we have been doing this for months without trouble). While installing your own runner can be useful, e.g. for debugging purposes, I find this defeats the whole purpose of building your software on a hosted cicd platform (github actions, gitlab ci, ...). For debugging, I prefer to base my build script on shell commands which can be run on your local dev box, which minimizes trial and error and makes installing your own runner superfluous.
To get started with a windows runner, see this exemple script. One or two gotchas that I went through:
You cannot choose the VM image of the build machine, it is managed by Gitlab and its content is not documented. This leaves you vulnerable to breaking changes if they decide to change their image.
Strangely, at the time of writing, the .net framework 4.8 developer pack is not installed on the windows executor image, but luckily chocolatey is, so if you target net48 you must install it in your before_script section.
The provisionning of the build machine is somewhat slower than a linux maching running docker, our builds average 15mn which we find acceptable.
That being said, here is a trimmed example based on our build scripts:
variables:
MSBUILD_EXE: 'C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\bin\msbuild.exe'
UPDATE_VERSION: '.\path\to\Update-Version.ps1'
APP_FOLDER: '.\path\to\Artifacts\_PublishedWebsites\MyApp\'
.windows_build_base:
tags:
- shared-windows
- windows
- windows-1809
before_script:
- Set-Variable -Name "time" -Value (date -Format "%H:%m")
- echo ${time}
- echo "started by ${GITLAB_USER_NAME}"
- echo ".NET versions already installed"
- Get-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\*", "HKLM:\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\*"
| Where {$_.DisplayName -like '*.NET*' -and $_.DisplayVersion -like '4.*'}
| Select-Object DisplayName, DisplayVersion
- echo "Installing .NET 4.8 Developer Pack"
- choco install netfx-4.8-devpack -y
stages:
- build
- publish
build:
only:
- prod
- test
extends:
- .windows_build_base
stage: build
script:
- nuget restore
- '& "${UPDATE_VERSION}" -srcDir . | Set-Variable -Name VERSION' # capture output of UPDATE_PRODUCTINFO script in variable VERSION
- echo "VERSION=${VERSION}"
- Add-Content -Path app_version.env -Value "VERSION=${VERSION}" # store value of VERSION in dotenv file to be passed to dependent stages, as per this workaround https://gitlab.com/gitlab-org/gitlab/-/issues/212629#note_430278657
- Get-Content app_version.env
- '& "${MSBUILD_EXE}" /t:solution_path\to\MyApp /p:Configuration=Release /clp:Summary /verbosity:Minimal /p:OutDir=.\Artifacts'
artifacts:
expire_in: 1 day
paths:
- '${APP_FOLDER}'
reports:
dotenv: app_version.env
publish:
only:
- prod
- test
image:
name: octopusdeploy/octo
entrypoint: [""]
stage: publish
dependencies:
- build
script:
- echo "VERSION=$VERSION"
- octo pack --id="MyApp" --format="zip" --version="$VERSION" --basePath="path/to/Artifacts/_PublishedWebsites/MyApp" --outFolder="/output/"
- octo push --overwrite-mode="OverwriteExisting" --server="https://mycompany.octopus.app" --space "MyOctoSpace" --package="/output/MyApp.$VERSION.zip"
This script:
Installs .net 4.8 dev pack
Restores nuget packages
Manually updates the software version and passes it between the build and publish stages
Builds the app
Packs and pushes it to octopus deploy