Develop Reference

Function to encode image with dlib with c#

I'm trying to make a function with c# with fn project, I did this job with an Dockerized linux containerized API with Visual Studio 2022 and it works great.
The problem is, when I try to make the same with fn on a VM with Centos 9 it doesn't work and I dont know how to view the errors/exceptions. The only message I get is:
{"message": "error receiving function response"}
I modify my Dockerfile like next
FROM fnproject/dotnet:6.0-1.0.9-dev as build-stage
WORKDIR /function
COPY . .
RUN dotnet sln add src/Function/Function.csproj
RUN dotnet build -c Release
RUN dotnet publish src/Function/Function.csproj -c Release -o out
FROM fnproject/dotnet:6.0-1.0.9
WORKDIR /function
COPY --from=build-stage /function/out/ /function/
ENTRYPOINT ["dotnet", "Function.dll"]
CMD ["Function:Encoder:encode"]
And I show my func.yaml
schema_version: 20180708
name: encode-image
version: 0.0.133
runtime: dotnet6.0
build_image: fnproject/dotnet:6.0-1.0.9-dev
run_image: fnproject/dotnet:6.0-1.0.9
cmd: Function:Encoder:encode
entrypoint: dotnet Function.dll
memory: 2048
timeout: 300
triggers:
- name: encode-image
type: http
source: /api/v1/encode-image
And my Function.csproj
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net6.0</TargetFramework>
<!--Nullable>enable</Nullable-->
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Fnproject.Fn.Fdk" Version="1.0.9" />
<PackageReference Include="DlibDotNet" Version="19.21.0.20220724" />
<PackageReference Include="DlibDotNet.Extensions" Version="19.18.0.20200428" />
<PackageReference Include="Magick.NET-Q16-AnyCPU" Version="12.2.2" />
<PackageReference Include="Magick.NET.Core" Version="12.2.2" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.2" />
</ItemGroup>
<ItemGroup>
<None Update="Assets\dlib\dlib_face_recognition_resnet_model_v1.dat">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
<None Update="Assets\dlib\shape_predictor_5_face_landmarks.dat">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
</ItemGroup>
</Project>
I was trying to focus the code which generates the problem and I think is related with next:
detector = Dlib.GetFrontalFaceDetector();
shapePredictor = ShapePredictor.Deserialize("Assets/dlib/shape_predictor_5_face_landmarks.dat");
net = DlibDotNet.Dnn.LossMetric.Deserialize("Assets/dlib/dlib_face_recognition_resnet_model_v1.dat");
And when I call the function with Postman I get:
Dont know how to discover the problem. I think is realted to the load of .dat files or some time the fn framework waits or I dont know.
At last, my method which is called:
public string encode(string strRequest)
{
// Lets generate basic response
Dictionary<string, object> mapResponse = new Dictionary<string, object>();
//List<string> lstErrors = new List<string>();
try
{
JObject jsonRequest = new JObject();
jsonRequest = JObject.Parse(strRequest);
// Lets validate options from request
Dictionary<string, object> mapRequest = validateParameters(jsonRequest);
// Lets get valid parameters
Dictionary<string, object> mapParameters = (Dictionary<string, object>)mapRequest["parameters"];
// If debug mode is true, then add debug info
if (((bool)mapParameters["debug"]) == true) mapResponse["debugging"] = mapParameters;
// Lets form the response
mapResponse["errors"] = mapRequest["errors"];
mapResponse["result"] = string.Empty;
// If there are not errors with parameters and their values then calculate encoding
if (((List<string>) mapRequest["errors"]).Count == 0)
{
// Thinking in future updates, we are going to use a factory
IEncoder encoder = EncoderFactory.createEncoder((string)mapParameters["algorithm"]);
// Lets add the resulting object to the response
mapResponse["result"] = encoder.Encode(mapParameters);
//if(((Encoder_FaceRecognition) encoder).resourceLoaded==true) return "OK";
//return "NOK";
}
}
catch (System.Exception exception)
{
((List<string>)mapResponse["errors"]).Insert(0, "<ERROR> Some exception has occurred." + exception.StackTrace);
((List<string>)mapResponse["errors"]).Insert(0, "<ERROR> Some exception has occurred." + exception.Message);
}
return "<EXITO>";
//return JsonConvert.SerializeObject( mapResponse );
}

ML.NET ImageClassification crash on macOS

I am about to figure out ML.NET and startet with an Microsoft Example which is linked down here:
https://learn.microsoft.com/en-us/dotnet/machine-learning/tutorials/image-classification-api-transfer-learning
I did everything as explained (creating assets-Dir and downloading files etc.) but when I run the Application I always run into a Problem that dotnet crashes away. I can't catch it with "try-catch".
My Code is:
var projectDirectory = Path.GetFullPath(Path.Combine(AppContext.BaseDirectory, "../../../"));
var workspaceRelativePath = Path.Combine(projectDirectory, "workspace");
var assetsRelativePath = Path.Combine(projectDirectory, "assets");
MLContext mlContext = new MLContext();
IEnumerable<ImageData> images = LoadImagesFromDirectory(folder: assetsRelativePath, useFolderNameAsLabel: true);
IDataView imageData = mlContext.Data.LoadFromEnumerable(images);
IDataView shuffledData = mlContext.Data.ShuffleRows(imageData);
var preprocessingPipeline = mlContext.Transforms.Conversion.MapValueToKey(
inputColumnName: "Label",
outputColumnName: "LabelAsKey")
.Append(mlContext.Transforms.LoadRawImageBytes(
outputColumnName: "Image",
imageFolder: assetsRelativePath,
inputColumnName: "ImagePath"));
IDataView preProcessedData = preprocessingPipeline
.Fit(shuffledData)
.Transform(shuffledData);
TrainTestData trainSplit = mlContext.Data.TrainTestSplit(data: preProcessedData, testFraction: 0.3);
TrainTestData validationTestSplit = mlContext.Data.TrainTestSplit(trainSplit.TestSet);
IDataView trainSet = trainSplit.TrainSet;
IDataView validationSet = validationTestSplit.TrainSet;
IDataView testSet = validationTestSplit.TestSet;
var classifierOptions = new ImageClassificationTrainer.Options()
{
FeatureColumnName = "Image",
LabelColumnName = "LabelAsKey",
ValidationSet = validationSet,
Arch = ImageClassificationTrainer.Architecture.ResnetV2101,
MetricsCallback = (metrics) => Console.WriteLine(metrics),
TestOnTrainSet = false,
ReuseTrainSetBottleneckCachedValues = true,
ReuseValidationSetBottleneckCachedValues = true
};
var trainingPipeline = mlContext.MulticlassClassification.Trainers.ImageClassification(classifierOptions)
.Append(mlContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));
As soon as I reach the last line:
mlContext.MulticlassClassification.Trainers.ImageClassification(classifierOptions)
.Append(mlContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));
The App crash and I don't see any error. If I copy the same code onto my Windows-System it's working fine.
I am working with macOS Monterey on my MacBook Pro M1.
ML-Nuget-Packages are Installed in Version 1.6 with Tensorflow-Redist-Package-Version 2.3.1
Do you guys have any idea what's going on there or how can I troubleshoot it?
UPDATE (.csproj-File):
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp3.1</TargetFramework>
</PropertyGroup>
<ItemGroup>
<None Remove="Microsoft.ML.Vision" />
<None Remove="Microsoft.ML.ImageAnalytics" />
<None Remove="Microsoft.ML" />
<None Remove="SciSharp.TensorFlow.Redist" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.ML.Vision" Version="1.6.0" />
<PackageReference Include="Microsoft.ML.ImageAnalytics" Version="1.6.0" />
<PackageReference Include="Microsoft.ML" Version="1.6.0" />
<PackageReference Include="SciSharp.TensorFlow.Redist" Version="2.3.1" />
</ItemGroup>
</Project>
UPDATE
with the newly released Visual Studio 2022 for Mac Version it's now possible to catch the Exception:
Unable to load shared library 'tensorflow' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libtensorflow, 0x0001): tried: 'libtensorflow' (no such file), '/usr/local/lib/libtensorflow' (no such file)
So obviously it cannot find the tensorflowlib. I will try to install the lib by myself now.

How to run tests in parallel using C# Specflow, MsTest Appium with selenium grid

I am unable to run tests parallel in two iOS devices (have different udids) I can see both tests are trying to run on one device only instead of running in parallel mode. One test always fails and other passes for some reason but both passes if I run independently.
I ran out my google search options and posting my question, can someone please help what I am missing in setup.
I have created two iOS simulators and created two node json files and selenium hub and two nodes are running on same machine. Selenium hub is running and registered both nodes successfully.
I have two feature files in my solution and using Test Explorer to run the tests.
Selenium Grid commands used:
Hub
java -jar selenium-server-standalone-3.141.59.jar -role hub
Node 1
appium -p 4726 --nodeconfig iOS14_N2D2_4726.json
Node 2
appium -p 4724 --nodeconfig iOS14_N1D1_4724.json
Packages used:
<ItemGroup>
<PackageReference Include="Appium.WebDriver" Version="4.3.1" />
<PackageReference Include="DotNetSeleniumExtras.WaitHelpers" Version="3.11.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.10.0" />
<PackageReference Include="SpecFlow.Plus.LivingDocPlugin" Version="3.8.35" />
<PackageReference Include="SpecFlow.MsTest" Version="3.8.14" />
<PackageReference Include="MSTest.TestAdapter" Version="2.2.4" />
<PackageReference Include="MSTest.TestFramework" Version="2.2.4" />
<PackageReference Include="FluentAssertions" Version="5.10.3" />
</ItemGroup>
Added this line to one of the Binding class
[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.ClassLevel)]
Folder Structure
calling the driver as below and registering as instance
using BoDi;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using OpenQA.Selenium;
using OpenQA.Selenium.Appium;
using OpenQA.Selenium.Appium.Enums;
using OpenQA.Selenium.Appium.iOS;
using System;
using TechTalk.SpecFlow;
[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.ClassLevel)]
namespace SpecflowParallel.Driver
{
[Binding]
public class DriverSetup
{
private IObjectContainer _objectContainer;
public AppiumDriver<IWebElement> Driver;
public DriverSetup(IObjectContainer objectContainer)
{
_objectContainer = objectContainer;
}
[BeforeScenario]
public void BeforeScenario()
{
Driver = GetDriver();
_objectContainer.RegisterInstanceAs(Driver);
}
public AppiumDriver<IWebElement> GetDriver()
{
var caps = new AppiumOptions();
caps.AddAdditionalCapability(MobileCapabilityType.PlatformName, "IOS");
caps.AddAdditionalCapability(MobileCapabilityType.PlatformVersion, "14.4");
caps.AddAdditionalCapability(MobileCapabilityType.DeviceName, "iPhone 8 Plus");
caps.AddAdditionalCapability(MobileCapabilityType.AutomationName, "XCUITest");
caps.AddAdditionalCapability(MobileCapabilityType.App, "/Users/automation/build/myapp.ios.app.zip");
Driver = new IOSDriver<IWebElement>(new Uri("http://10.1.1.92:4444/wd/hub"), caps);
return Driver;
}
}
}
Node1 - iOS14_N2D2_4726.json
{
"capabilities":
[
{
"browserName": "Safari",
"platformName": "IOS",
"platformVersion": "14.4",
"maxInstances": 2,
"automationName": "XCUITest",
"appPackage": "myapp.droid.integration",
"AppActivity": "myapp.LaunchActivity",
"wdaLocalPort":"8142",
"udid":"188BA7D5-905F-473E-9CF9-3A49555F587C"
}
],
"configuration":
{
"cleanUpCycle":2000,
"timeout":30000,
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"host": "10.1.1.92",
"port": 4726,
"maxSession": 1,
"register": true,
"registerCycle": 5000,
"hubPort": 4444,
"hubHost": "10.1.1.92",
"hubProtocol": "http",
"url":"http://10.1.1.92:4726/wd/hub"
}
}
Node2 - iOS14_N1D1_4724.json
{
"capabilities":
[
{
"browserName": "Safari",
"platformName": "IOS",
"platformVersion": "14.4",
"maxInstances": 2,
"automationName": "XCUITest",
"appPackage": "myapp.droid.integration",
"appActivity": "myapp.LaunchActivity",
"wdaLocalPort":"8141",
"udid":"0227DEF2-DF40-47A1-AE95-EFFBC69FAB37"
}
],
"configuration":
{
"cleanUpCycle":2000,
"timeout":30000,
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"host": "10.1.1.92",
"port": 4724,
"maxSession": 1,
"register": true,
"registerCycle": 5000,
"hubPort": 4444,
"hubHost": "10.1.1.92",
"hubProtocol": "http",
"url":"http://10.1.1.92:4724/wd/hub"
}
}
Happy to provide further info

How do I replace Database.ExecuteSqlQuery when changing TargetFramework away from.NetCore2.1

I have the following which was working fine in .NetCore2.1 with SDKs
Microsoft.AspNetCore.App(2.1.1)
Microsoft.NetCore.App(2.1.0)
My code is
public static int TransitTime(string postcode, ApiDbContext con)
{
var query = "SELECT top 1 Mins from Transit where postcode = #Postcode order by mins desc;";
var p1 = new SqlParameter("#Postcode",postcode);
var result = 0;
using (var dr = con.Database.ExecuteSqlQuery(query,p1))
{
var reader = dr.DbDataReader;
while (reader.Read()) result = (int)reader[0];
}
return Convert.ToInt32(result);
}
Hovering over the word Database I could see it was in
Microsoft.EntityFrameworkCore.Infrastructure.DatabaseFacade
I don't see a specific reference to
Microsoft.EntityFrameworkCore.Infrastructure
It is documented as being part of Entity Framework Core 2.1
inside either SDK so I wonder how it is referenced.
However I needed to add a reference to a Framework 4.7.2 dll
So I switched to the following project file
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Library</OutputType>
<TargetFramework>netstandard2.0</TargetFramework>
<ApplicationIcon />
<StartupObject />
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="2.2.6" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Relational" Version="2.2.6" />
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="2.2.6" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="2.2.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\Common\SBD.Common.csproj" />
</ItemGroup>
</Project>
Now I get an error
CS1061 'DatabaseFacade' does not contain a definition for
'ExecuteSqlQuery'
I tried looking form Microsoft.EntityFrameworkCore.Infrastructure in Nuget Manage Packages for Solution but it does not show.
Looking at this question I decided to try using .FromSQl instead.
I found help in the docs but it does not explain how return non entity types.
How do I do that?

After reading the end of this link I am trying
public static int TransitTime(string postcode, ApiDbContext con)
{
var query = "SELECT top 1 Mins from Transit where postcode = #Postcode order by mins desc;";
var p1 = new SqlParameter("#Postcode", postcode);
var result = 0;
using (var command = con.Database.GetDbConnection().CreateCommand())
{
command.CommandText = query;
command.Parameters.Add(p1);
con.Database.OpenConnection();
using (var reader = command.ExecuteReader())
{
while (reader.Read()) result = (int)reader[0];
}
}
return Convert.ToInt32(result);
}
I see that this is using an extension method in Microsoft.EntityFrameworkCore.Infrastructure

Bundling duplicate content in file on asp.net core

I created an empty project of asp.net core and added BuildBundlerMinifier to test how it works now.
I added two files to be combined, css and js but when i add them to the bundler it duplicates the content of the second file into the first one.
This is only for test, but I do not understand what is wrong.
Project config:
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>netcoreapp2.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.All" Version="2.0.5" />
<PackageReference Include="BuildBundlerMinifier" Version="2.6.362" />
</ItemGroup>
<ItemGroup>
<DotNetCliToolReference Include="Microsoft.VisualStudio.Web.CodeGeneration.Tools" Version="2.0.2" />
</ItemGroup>
</Project>
bundleconfig:
[
{
"outputFileName": "wwwroot/js/test.js",
"inputFiles": [
"wwwroot/js/test.js",
"wwwroot/js/testtest.js"
]
},
{
"outputFileName": "wwwroot/css/customcss.css",
"inputFiles": [
"wwwroot/css/customcss.css",
"wwwroot/css/customcss1.css"
],
"minify": {
"enabled": true,
"renameLocals": true
},
"sourceMap": false
}
]
test.js file has:
function myfunction(hola) {
//Hola
console.log(hola);
}
testtest.js file has:
function testFileTwo(valueP) {
//Hi
console.log(valueP);
}
Why would it combine the files and rewrite them, this behavior only need to be done on the browser, I think.
test.js automatic filling with this:
function myfunction(hola) {
//Hola
console.log(hola);
}
function testFileTwo(valueP) {
//Hi
console.log(valueP);
}
function testFileTwo(valueP) {
//Hi
console.log(valueP);
}
function testFileTwo(valueP) {
//Hi
console.log(valueP);
}
function testFileTwo(valueP) {
//Hi
console.log(valueP);
}
function testFileTwo(valueP) {
//Hi
console.log(valueP);
}
function testFileTwo(valueP) {
//Hi
console.log(valueP);
}
function testFileTwo(valueP) {
//Hi
console.log(valueP);
}

I think this is happening because you have the same name for the bundle result and for the bundle input. Could change it and test?