I'm trying to introduce SonarQube scanning to our code base. I have ASP.NET MVC web application solution. This has roughly 50 projects. The minimum project target framework is .NET 4.5.2 (Some projects are 4.6.2). I can compile this solution absolutely fine in Visual Studio 2015.
I have SonarQube 8.9.3 LTS setup on a development server.
When adding a project on SonarQube the following commands are given to execute (at the solution root) the scanner from your local machine:
Step 1:
SonarScanner.MSBuild.exe begin /k:"MyBiz" /d:sonar.host.url="http://devserver:9600" /d:sonar.login="<token>"
Step 2:
"C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe" MyBiz.sln /t:Rebuild
Step 3:
SonarScanner.MSBuild.exe end /d:sonar.login="<token>"
My problem is that I get an error on step 2 during the build process:
"C:\trunk\Dev4.0\MyBiz\MyBiz.sln" (Rebuild target) (1) ->
"C:\trunk\Dev4.0\Common\src\DataLayer\DataLayer.csproj" (Rebuild target) (4) ->
(CopyFilesToOutputDirectory target) ->
C:\Program Files (x86)\MSBuild\14.0\bin\Microsoft.Common.CurrentVersion.targets(3813,5): error MSB3030: Could not copy the file "obj\Debug\ILRetail.eBusiness.Common.DataLayer.dll" because it was not found. [C:\trunk\Dev4.0\Common\src\DataLayer\DataLayer.csproj]
Obviously this solution compiles fine in Visual Studio so I'm guessing I'm missing some additional parameters on the MSBuild step in this process.
I'm attempted to explicitly use Configuration and Platform properties:
/property:Configuration=Debug /property:Platform="AnyCPU"
But this did not work:
The specified solution configuration "Debug|AnyCPU" is invalid
Is there something obvious I'm missing in the build command that would be the reason this works fine in Visual studio but not from command line?
Note in visual studio I would ordinarily compile using Debug & Mixed Platforms. I've attempted to use these parameters and I still get the same error as supplying no configuration/platform parameter.
EDIT 04/06/18 => Updated question with last status
So I have this working .Net 4.6 Stateful Service that currently run on my Windows Service Fabric cluster deployed on Azure.
Starting from 09/2017, I should be able to move to Linux: https://blogs.msdn.microsoft.com/azureservicefabric/2017/09/25/service-fabric-6-0-release/
So I'm trying to deploy it on Linux so I can save costs.
First things first, I've migrated all my code from .Net 4.6 to .Net Core 2.0. Now I can compile my binaries without issues. I've basically created new .Net Core projects and then moved all my source code from .Net 4.6 projects to the new .Net Core ones.
Then I've updated my Service Fabric application. I removed my previous SF services from my sfproj, then I've added my new .Net Core ones.
Looks like there is a warning (nothing on the output window though), but it's here anyway if I try to create a new empty Statful service using .Net core 2.0 through the template provided by Service Fabric Tools 2.0 (beta):
So I'm going to live with it.
On my dev machine, I've modified the 2 csproj projects that contain my Stateful services so they can run locally as Windows executables. I've used the win7-x64 runtimeIdentifier.
Running my SF cluster locally on my Windows machine is fine.
Then I've slightly changed the previous csproj files for Linux. I used the ubuntu.16.10-x64 runtimeIdentifier.
Also I've changed the ServiceManifest.xml file to target the linux-compatible binary:
<!-- Code package is your service executable. -->
<CodePackage Name="Code" Version="1.9.6">
<EntryPoint>
<ExeHost>
<Program>entryPoint.sh</Program>
</ExeHost>
</EntryPoint>
</CodePackage>
entryPoint.sh is a basic script that eventually executes:
dotnet $DIR/MyService.dll
Then I've successfully deployed to my secured SF Linux cluster from Visual Studio. Unfortunately I have the following errors for both my stateful services:
Error event: SourceId='System.Hosting',
Property='CodePackageActivation:Code:EntryPoint'. There was an error
during CodePackage activation.The service host terminated with exit
code:134
Looks like my binary crashes when starting. So here are my questions:
Is the approach right to deploy a C# .Net Core SF stateful service on Linux from Visual Studio?
EDIT: looking inside the LinuxsyslogVer2v0 table, I get the following error:
starthost.sh[100041]: Unhandled Exception:
System.IO.FileLoadException: Could not load file or assembly
'System.Threading.Thread, Version=4.1.0.0, Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a'. The located assembly's manifest
definition does not match the assembly reference. (Exception from
HRESULT: 0x80131040)
I found the following bug report: https://github.com/dotnet/sdk/issues/1502
Unfortunately, I still get the error without using MSBuild (using dotnet deploy).
EDIT: further clarification:
My boss want me to run on Linux because starting from D1v2 machines, it's half the price compared to Windows machines (no license etc.)
My .NET Core 2.0 services successfully run on Windows. So the .NET Core port should be fine.
So, this was a real pain in the ass to get it working properly. But it works. Well, kind of.
First, Reliable Services are still in preview on Linux: https://github.com/Microsoft/service-fabric/issues/71
Full Linux support should come very soon (actually it should be available already according to the previous link...).
Now for the details about how to procede, here is some information to help others, because there is just nothing about that on Microsoft documentation and I literally lost 3 days trying to make it work.
1. Do use .NET Core 2.0 for your projects.
It is supported on Linux. On preview for now, but it works.
2. Do use the right RID for your projects.
As of today (April 2018), the right RID to use is ubuntu.16.04-x64.
Edit the csproj files of your Reliable Service projects and set the RID like this:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.0</TargetFramework>
<IsServiceFabricServiceProject>True</IsServiceFabricServiceProject>
<RuntimeIdentifier>ubuntu.16.04-x64</RuntimeIdentifier>
<Platforms>AnyCPU;x64</Platforms>
</PropertyGroup>
The fun part is, you should be able to provide multiple RIDs using the RuntimeIdentifiers parameter (with a S at the end) like that:
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.0</TargetFramework>
<IsServiceFabricServiceProject>True</IsServiceFabricServiceProject>
<RuntimeIdentifiers>win7x64;ubuntu.16.04-x64</RuntimeIdentifiers>
<Platforms>AnyCPU;x64</Platforms>
</PropertyGroup>
So you could build Windows binaries and Linux binaries at the same time.
But it simply doesn't work. When building the project from Visual Studio, I end up with the following directory only:
bin/Debug/netcoreapp2.0/
Only DLLs, no valid entry point. No win7-x64 folder, no ubuntu.16.04-x64, no nothing.
This is a bug, supposed to be fixed, but it's not (I use Visual Studio 15.6.2 all up-to-date as of today). See https://github.com/dotnet/core/issues/1039
3. You need a valid Entry Point for your service.
On Windows it's an executable file (*.exe). On Linux it's not. I ended up getting the Linux C# example and copied/pasted the entry point. https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-create-your-first-linux-application-with-csharp
So basically I now have on my ServiceManifest.xml file of each Reliable Service the following EntryPoint :
<?xml version="1.0" encoding="utf-8"?>
<ServiceManifest Name="XXXX"
Version="1.0.0"
xmlns="http://schemas.microsoft.com/2011/01/fabric"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ServiceTypes>
<!-- This is the name of your ServiceType.
This name must match the string used in RegisterServiceType call in Program.cs. -->
<StatefulServiceType ServiceTypeName="YYY" HasPersistedState="true" />
</ServiceTypes>
<!-- Code package is your service executable. -->
<CodePackage Name="Code" Version="1.0.0">
<EntryPoint>
<ExeHost>
<Program>entryPoint.sh</Program>
</ExeHost>
</EntryPoint>
</CodePackage>
entryPoint.sh is as follows:
#!/usr/bin/env bash
check_errs()
{
# Function. Parameter 1 is the return code
if [ "${1}" -ne "0" ]; then
# make our script exit with the right error code.
exit ${1}
fi
}
DIR=`dirname $0`
echo 0x3f > /proc/self/coredump_filter
source $DIR/dotnet-include.sh
dotnet $DIR/NAME_OF_YOUR_SERVICE_DLL.dll $#
check_errs $?
dotnet-include.sh is as follows:
#!/bin/bash
. /etc/os-release
linuxDistrib=$ID
if [ $linuxDistrib = "rhel" ]; then
source scl_source enable rh-dotnet20
exitCode=$?
if [ $exitCode != 0 ]; then
echo "Failed: source scl_source enable rh-dotnet20 : ExitCode: $exitCode"
exit $exitCode
fi
fi
Both are inside the PackageRoot folder. I specified for both their properties so the Build Action is "Content" and the Copy to Output Directory is "Copy always".
4. Do NOT build using MSBuild !!
Yeah it is supposed to build Linux packages too, or at least it seems so, because MSBuild is able to produce the following files when you right click on your project and click "Build":
Don't trust the apparent success of the operation, it will miserably FAIL to properly execute when deployed. Some *.so files missing and other issues. MSBuild is buggy as hell and misbehaves regarding dependencies.
See for instance this bug report: https://github.com/dotnet/sdk/issues/1502
Still not fixed after almost a year...
Or https://github.com/dotnet/core/issues/977 (got this one, too).
5. Do write some PowerShell script to build the stuff by yourself.
I ended up reinventing the wheel using the following script to build my package:
# Creating binaries for service 1
cd DIRECTORY_OF_MY_SERVICE_1
dotnet publish -c Release -r ubuntu.16.04-x64
# Creating binaries for service 2
cd ..\DIRECTORY_OF_MY_SERVICE_2
dotnet publish -c Release -r ubuntu.16.04-x64
# Creating binaries for service 3
cd ..\DIRECTORY_OF_MY_SERVICE_3
dotnet publish -c Release -r ubuntu.16.04-x64
# Copying ApplicationManifest.xml
cd ..
mkdir PKG\ServiceFabricApplication
echo F|xcopy "ServiceFabricApplication\ApplicationPackageRoot\ApplicationManifest.xml" "PKG\ServiceFabricApplication\ApplicationManifest.xml" /sy
# Copying Service1 files
mkdir "PKG\ServiceFabricApplication\Service1Pkg"
mkdir "PKG\ServiceFabricApplication\Service1Pkg\Code"
xcopy "Service1\PackageRoot\*" "PKG\ServiceFabricApplication\Service1Pkg" /sy /D
xcopy "Service1\bin\Release\netcoreapp2.0\ubuntu.16.04-x64\publish\*" "PKG\ServiceFabricApplication\Service1Pkg\Code" /sy
# Copying Service2 files
mkdir "PKG\ServiceFabricApplication\Service2Pkg"
mkdir "PKG\ServiceFabricApplication\Service2Pkg\Code"
xcopy "Service2\PackageRoot\*" "PKG\ServiceFabricApplication\Service2Pkg" /sy /D
xcopy "Service2\bin\Release\netcoreapp2.0\ubuntu.16.04-x64\publish\*" "PKG\ServiceFabricApplication\Service2Pkg\Code" /sy
# Copying Service3 files
mkdir "PKG\ServiceFabricApplication\Service3Pkg"
mkdir "PKG\ServiceFabricApplication\Service3Pkg\Code"
xcopy "Service3\PackageRoot\*" "PKG\ServiceFabricApplication\Service3Pkg" /sy /D
xcopy "Service3\bin\Release\netcoreapp2.0\ubuntu.16.04-x64\publish\*" "PKG\ServiceFabricApplication\Service3Pkg\Code" /sy
# Compresses the package
Write-host "Compressing package..."
Copy-ServiceFabricApplicationPackage -ApplicationPackagePath .\PKG\ServiceFabricApplication -CompressPackage -SkipCopy
sfproj file is a Visual Studio / MSBuild related project, so you need to build everything by yourself.
The script above produces the same content as the pkg folder created by MSBuild when building your sfproj using Visual Studio. It copies everything on a PKG folder at the root of your solution.
The package structure is detailed here: https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/service-fabric/service-fabric-package-apps.md
6. Now it's time to deploy!
At this point I didn't trusted Visual Studio anymore, so I built my own PowerShell script:
. .\ServiceFabricApplication\Scripts\Deploy-FabricApplication.ps1 -ApplicationPackagePath '.\PKG\ServiceFabricApplication' -PublishProfileFile '.\ServiceFabricApplication\PublishProfiles\Cloud.xml' -DeployOnly:$false -ApplicationParameter:#{} -UnregisterUnusedApplicationVersionsAfterUpgrade $false -OverrideUpgradeBehavior 'None' -OverwriteBehavior 'SameAppTypeAndVersion' -SkipPackageValidation:$false -ErrorAction Stop
It reuses the Deploy-FabricApplication.ps1 script provided by the Service Fabric project template inside the sfproj project. This script parses the Cloud.xml PublishProfile and deploys to your service fabric cluster.
So you specifies the rights values on both PublishProfiles/Cloud.xml and ApplicationParameters/Cloud.xml then execute the script.
It only works if you have the certificate used to secure the cluster installed on your machine, of course.
Do note the first dot '.' is important, because if you don't use it, you'll have the following error:
Get-ServiceFabricClusterManifest : Cluster connection instance is null
See https://stackoverflow.com/a/38104087/870604
Oh, and as there are bugs on the Service Fabric SDK too, you might want to shutdown your local SF cluster too...
https://github.com/Azure/service-fabric-issues/issues/821
7. Now it's time for another deception.
It simply doesn't work, the service crashes on startup. After searching hours inside the LinuxsyslogVer2v0 Azure Storage table (the log table for Linux, located in one of the two Azure Storage Accounts created automatically with the SF cluster), I found that Microsoft own Nuget Packages were buggy too.
Specifically, the Nuget package Microsoft.Azure.Devices doesn't work on version 1.6.0. An issue with a reference of a dll not found or whatever. I rollbacked to a previous version, namely 1.5.1, and it was fixed.
At this point I didn't had anymore energy to create another Github issue about that. Sorry MS, I'm not your QA team, I'm getting tired.
8. Build again using the first PowerShell script, deploy using the second PowerShell script, and you're done.
You've finally deployed C# Reliable Services using .NET Core 2.0 from Visual Studio (kind of, as it's buggy and I used PowerShell) on Windows to a Linux SF Cluster.
Now I still have issues with my ASP.NET Core service, but it will be a story for another day.
Conclusion: TL;DR
The whole thing is a mess. Bugs everywhere. In the SDK, in the tools, in some of Microsoft Nuget Packages. Awful experience. But it is supported (in preview for now) and you can make it work. Hope this post will help...
I was having similar issues, but I believe this is the issue:
In this release, .NET Core 2.0 services are only supported on Service Fabric for Windows. Full cross-platform support for .NET Core 2.0 services on Windows and Linux is coming soon.
From the Service Fabric 6.1 Release Notes
So no Linux as long as you are targeting .net core 2.0.
I have success deploy to Linux service fabric with this help
Open all the service .csproj files and update the RuntimeIdentifier as shown below
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.0</TargetFramework>
<IsServiceFabricServiceProject>True</IsServiceFabricServiceProject>
<RuntimeIdentifier>linux-x64</RuntimeIdentifier>
</PropertyGroup>
Update the ServiceManifest.xml to remove .exe extension as shown below
<CodePackage Name="Code" Version="1.0.0">
<EntryPoint>
<ExeHost>
<Program>Web1</Program>
</ExeHost>
</EntryPoint>
</CodePackage>
See https://blogs.msdn.microsoft.com/premier_developer/2018/05/27/running-net-core-2-0-applications-in-a-linux-service-fabric-cluster-on-azure/
*Visual Studio 15.7.3
Folks,
Am trying to run a build using the below command line however there is a prompt everytime which is causing trouble. I have to address this manually and cannot run automated builds.
$currentScriptPath = (pwd).Path
$solution = "$($currentScriptPath)\foobar_Msi.sln"
$project = "$($currentScriptPath)\foobarSetup\foobarSetup.vdproj"
$BuildLog = "$($currentScriptPath)\buildlog.txt"
$devEnvCom = "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\devenv.com"
$devEnvExe = "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\devenv.exe"
$PackagePath = "$($currentScriptPath)\ReleasePackage"
.
.
.
& $devEnvCom $solution /project $project /Build release /Out $BuildLog
An error occured when applying the IIS Express settings to server URL
'http://localhost:2943/' for project 'Foobar.UI'. You may have to
manually edit the applicationHost.config file and make the changes for
your site to run correctly
I cleared IISAppsettings in document, tried non-interactive command line arguments, changed the binding in build servers applicationHost.Config file and nothing works.
On your development machine, copy/move the solution to a local path that exactly matches the one it will be in when the automated build is run. Open the solution in Visual Studio 2012, Save All, and add the hidden [solution path].VS\applicationHost.config to source control.
I got a Jenkins Server set up on Windows 2012 R2. These are my build settings:
Underneath you see the error. It should be said that the pre-build step runs successfully:
[Test CSharp Build Job] $ ...\Jenkins\.jenkins\tools\hudson.plugins.sonar.MsBuildSQRunnerInstallation\MSBuild_2.0\MSBuild.SonarQube.Runner.exe end
SonarQube Scanner for MSBuild 2.0
Default properties file was found at ...\Documents\Jenkins\.jenkins\tools\hudson.plugins.sonar.MsBuildSQRunnerInstallation\MSBuild_2.0\SonarQube.Analysis.xml
Loading analysis properties from ...\Documents\Jenkins\.jenkins\tools\hudson.plugins.sonar.MsBuildSQRunnerInstallation\MSBuild_2.0\SonarQube.Analysis.xml
Post-processing started.
SonarQube Scanner for MSBuild End Step 1.1
No ProjectInfo.xml files were found. Possible causes:
1. The project has not been built - the end step was called right after the begin step, without a build step in between
2. An unsupported version of MSBuild has been used to build the project. Currently MSBuild 12.0 upwards are supported
3. The build step has been launched from a different working folder
Generation of the sonar-properties file failed. Unable to complete SonarQube analysis.
11:18:18.015 Creating a summary markdown file...
Post-processing failed. Exit code: 1
ERROR: Execution of SonarQube Scanner for MSBuild failed (exit code 1)
So in my Jenkins Configuration I have this:
Am I missing something to make this work?
Okay I figured it out
I forgot to add a /t:rebuild in between building with MSBuild and running the End Analysis command.
I'd like to know what the complete set of steps is to build a MonoTouch C# app consisting of the main application assembly containing xib files and a set of library assemblies that also possibly contain xib files.
We're trying to automate these steps via a proper MSBuild script (which MonoTouch has yet to support) for various reasons which I won't go into, to focus on the question here.
So far, here's what I've come up with:
1) Compile each assembly using smcs, e.g.
/Developer/MonoTouch/usr/bin/smcs /noconfig
"/out:/Users/bright/src/MonoTouchAppWithLib/AppLib/bin/Debug/AppLib.dll
"/r:/Developer/MonoTouch/usr/lib/mono/2.1/System.dll"
"/r:/Developer/MonoTouch/usr/lib/mono/2.1/System.Xml.dll"
"/r:/Developer/MonoTouch/usr/lib/mono/2.1/System.Core.dll"
"/r:/Developer/MonoTouch/usr/lib/mono/2.1/monotouch.dll"
/nologo /warn:4 /debug:+ /debug:full /optimize- /codepage:utf8
"/define:DEBUG"
/t:library "/Users/bright/src/MonoTouchAppWithLib/AppLib/Class1.cs"
2) Compile interface definitions: run ibtool on each xib file in each assembly, e.g
/Developer/usr/bin/ibtool
--errors --warnings --notices --output-format human-readable-text
"/Users/bright/src/App/App/ViewController_iPhone.xib"
--compile "/Users/bright/src/App/App/bin/Debug/App.app/ViewController_iPhone.nib"
--sdk "/Developer/Platforms/iPhoneSimulator.platform/Developer/
SDKs/iPhoneSimulator4.3.sdk"
3) Compile to native code:
/Developer/MonoTouch/usr/bin/mtouch
-sdkroot "/Applications/Xcode.app/Contents/Developer"
-v --nomanifest --nosign -sim
"/Users/bright/src/App/App/bin/iPhoneSimulator/Debug/App.app"
-r "/Users/bright/src/App/AppLib/bin/Debug/AppLib.dll"
-r "/Developer/MonoTouch/usr/lib/mono/2.1/System.dll"
-r "/Developer/MonoTouch/usr/lib/mono/2.1/System.Xml.dll"
-r "/Developer/MonoTouch/usr/lib/mono/2.1/System.Core.dll"
-r "/Developer/MonoTouch/usr/lib/mono/2.1/monotouch.dll"
-debug -profiling -nolink -sdk "5.0"
"/Users/bright/src/App/App/bin/iPhoneSimulator/Debug/App.exe"
However, it isn't clear how to do the following (taken from MonoDevelop's build output window), and in what order:
1) Extract embedded content. MonoDevelop just outputs this:
Extracted HelloWorldScreen_iPhone.nib from MtLib.dll
Extracted HelloWorldScreen_iPad.nib from MtLib.dll
2) Update application manifest: There's no command line given in the MonoDevelop build output window.
3) Update debug configuration file: There's no command line given in the MonoDevelop build output window.
4) Update debug settings file: There's no command line given in the MonoDevelop build output window.
And other steps I haven't gotten do yet like app signing and resources.
Hopefully we can get enough information here to make a go of it.
You can run, from a terminal window or from within an MSBuild task, the /Applications/MonoDevelop.app/Contents/MacOS/mdtool tool that is supplied with MonoDevelop. E.g.
/Applications/MonoDevelop.app/Contents/MacOS/mdtool -v build -t:Build "-c:Debug|iPhoneSimulator" /path/to/your/app.csproj
That will build the MonoTouch application, including all your steps above and any future feature that will be added.