Develop Reference

Function to encode image with dlib with c#

I'm trying to make a function with c# with fn project, I did this job with an Dockerized linux containerized API with Visual Studio 2022 and it works great.
The problem is, when I try to make the same with fn on a VM with Centos 9 it doesn't work and I dont know how to view the errors/exceptions. The only message I get is:
{"message": "error receiving function response"}
I modify my Dockerfile like next
FROM fnproject/dotnet:6.0-1.0.9-dev as build-stage
WORKDIR /function
COPY . .
RUN dotnet sln add src/Function/Function.csproj
RUN dotnet build -c Release
RUN dotnet publish src/Function/Function.csproj -c Release -o out
FROM fnproject/dotnet:6.0-1.0.9
WORKDIR /function
COPY --from=build-stage /function/out/ /function/
ENTRYPOINT ["dotnet", "Function.dll"]
CMD ["Function:Encoder:encode"]
And I show my func.yaml
schema_version: 20180708
name: encode-image
version: 0.0.133
runtime: dotnet6.0
build_image: fnproject/dotnet:6.0-1.0.9-dev
run_image: fnproject/dotnet:6.0-1.0.9
cmd: Function:Encoder:encode
entrypoint: dotnet Function.dll
memory: 2048
timeout: 300
triggers:
- name: encode-image
type: http
source: /api/v1/encode-image
And my Function.csproj
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net6.0</TargetFramework>
<!--Nullable>enable</Nullable-->
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Fnproject.Fn.Fdk" Version="1.0.9" />
<PackageReference Include="DlibDotNet" Version="19.21.0.20220724" />
<PackageReference Include="DlibDotNet.Extensions" Version="19.18.0.20200428" />
<PackageReference Include="Magick.NET-Q16-AnyCPU" Version="12.2.2" />
<PackageReference Include="Magick.NET.Core" Version="12.2.2" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.2" />
</ItemGroup>
<ItemGroup>
<None Update="Assets\dlib\dlib_face_recognition_resnet_model_v1.dat">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
<None Update="Assets\dlib\shape_predictor_5_face_landmarks.dat">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
</ItemGroup>
</Project>
I was trying to focus the code which generates the problem and I think is related with next:
detector = Dlib.GetFrontalFaceDetector();
shapePredictor = ShapePredictor.Deserialize("Assets/dlib/shape_predictor_5_face_landmarks.dat");
net = DlibDotNet.Dnn.LossMetric.Deserialize("Assets/dlib/dlib_face_recognition_resnet_model_v1.dat");
And when I call the function with Postman I get:
Dont know how to discover the problem. I think is realted to the load of .dat files or some time the fn framework waits or I dont know.
At last, my method which is called:
public string encode(string strRequest)
{
// Lets generate basic response
Dictionary<string, object> mapResponse = new Dictionary<string, object>();
//List<string> lstErrors = new List<string>();
try
{
JObject jsonRequest = new JObject();
jsonRequest = JObject.Parse(strRequest);
// Lets validate options from request
Dictionary<string, object> mapRequest = validateParameters(jsonRequest);
// Lets get valid parameters
Dictionary<string, object> mapParameters = (Dictionary<string, object>)mapRequest["parameters"];
// If debug mode is true, then add debug info
if (((bool)mapParameters["debug"]) == true) mapResponse["debugging"] = mapParameters;
// Lets form the response
mapResponse["errors"] = mapRequest["errors"];
mapResponse["result"] = string.Empty;
// If there are not errors with parameters and their values then calculate encoding
if (((List<string>) mapRequest["errors"]).Count == 0)
{
// Thinking in future updates, we are going to use a factory
IEncoder encoder = EncoderFactory.createEncoder((string)mapParameters["algorithm"]);
// Lets add the resulting object to the response
mapResponse["result"] = encoder.Encode(mapParameters);
//if(((Encoder_FaceRecognition) encoder).resourceLoaded==true) return "OK";
//return "NOK";
}
}
catch (System.Exception exception)
{
((List<string>)mapResponse["errors"]).Insert(0, "<ERROR> Some exception has occurred." + exception.StackTrace);
((List<string>)mapResponse["errors"]).Insert(0, "<ERROR> Some exception has occurred." + exception.Message);
}
return "<EXITO>";
//return JsonConvert.SerializeObject( mapResponse );
}

TimeTrigger Exception "Could not create BlobContainerClient for ScheduleMonitor."

When I run my azure function with TimeTrigger I have this error:
Microsoft.Azure.WebJobs.Extensions.Timers.Storage: Could not create BlobContainerClient for ScheduleMonitor.
I use a host builder:
public static async Task Main()
{
var host = CreateHostBuilder().Build();
using (host)
{
await host.RunAsync();
}
static IHostBuilder CreateHostBuilder() => new HostBuilder()
.UseServiceProviderFactory(new AutofacServiceProviderFactory())
.ConfigureFunctionsWorkerDefaults()
.ConfigureHostConfiguration(configHost =>
{
configHost.SetBasePath(Directory.GetCurrentDirectory());
configHost.AddJsonFile("host.json", optional: true);
configHost.AddEnvironmentVariables();
})
.ConfigureAppConfiguration((hostContext, configApp) =>
{
var env = hostContext.HostingEnvironment;
configApp.AddJsonFile("appsettings.json", optional: true);
configApp.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true);
configApp.AddEnvironmentVariables();
configApp.AddApplicationInsightsSettings(developerMode: !env.IsProduction());
})
.ConfigureServices((hostContext, services) =>
{
[...]
})
.ConfigureContainer<ContainerBuilder>(builder =>
{
builder.RegisterModule<MessagerModule>();
})
.ConfigureLogging((hostContext, configLogging) =>
{
if (hostContext.HostingEnvironment.IsDevelopment())
{
configLogging.AddConsole();
configLogging.AddDebug();
}
})
.UseConsoleLifetime();
and here is the function:
[Function("QueueMessage")]
public async Task QueueMessageAsync(
[TimerTrigger("%MessageQueuerOccurence%", RunOnStartup = true)] TimerInfo timer
)
{
[...]
}
csproj:
<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<AzureFunctionsVersion>v4</AzureFunctionsVersion>
<OutputType>Exe</OutputType>
</PropertyGroup>
<ItemGroup>
<Content Include="**\*.json" Exclude="bin\**\*;obj\**\*" CopyToOutputDirectory="Always" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Autofac.Extensions.DependencyInjection" Version="7.2.0" />
<PackageReference Include="Microsoft.ApplicationInsights.AspNetCore" Version="2.20.0" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Http" Version="3.0.13" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Timer" Version="4.1.0" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Storage" Version="5.0.0" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="1.3.0" OutputItemType="Analyzer" />
<PackageReference Include="Microsoft.Azure.Functions.Worker" Version="1.6.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="6.0.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="6.0.0" />
</ItemGroup>
local.settings.json:
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"MessageQueuerOccurence": "0 */15 * * * *",
"FUNCTIONS_WORKER_RUNTIME": "dotnet-isolated"
}
}
What did I miss?
note: link to Github : https://github.com/Azure/azure-functions-dotnet-worker/issues/779

I was having this issue because I didn't have the storage emulator running...
It worked once I installed and ran azurite, which is the storage emulator that is being maintained (Azure Storage Emulator has been discontinued). You can find more information here on how to install it and use it.
I am using VS Code on a Mac OS, and I found the simplest solution to install azurite extension on VS Code. After the installation, I just had to edit the extension settings, in order to set the location setting (any folder should work). After that I was able to start azurite by running the Azurite: Start from the VS Code command palette.
As per the comments, it might also be necessary to edit the local.settings.json file and change the AzureWebJobsStorage value to UseDevelopmentStorage=true.

How to run tests in parallel using C# Specflow, MsTest Appium with selenium grid

I am unable to run tests parallel in two iOS devices (have different udids) I can see both tests are trying to run on one device only instead of running in parallel mode. One test always fails and other passes for some reason but both passes if I run independently.
I ran out my google search options and posting my question, can someone please help what I am missing in setup.
I have created two iOS simulators and created two node json files and selenium hub and two nodes are running on same machine. Selenium hub is running and registered both nodes successfully.
I have two feature files in my solution and using Test Explorer to run the tests.
Selenium Grid commands used:
Hub
java -jar selenium-server-standalone-3.141.59.jar -role hub
Node 1
appium -p 4726 --nodeconfig iOS14_N2D2_4726.json
Node 2
appium -p 4724 --nodeconfig iOS14_N1D1_4724.json
Packages used:
<ItemGroup>
<PackageReference Include="Appium.WebDriver" Version="4.3.1" />
<PackageReference Include="DotNetSeleniumExtras.WaitHelpers" Version="3.11.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.10.0" />
<PackageReference Include="SpecFlow.Plus.LivingDocPlugin" Version="3.8.35" />
<PackageReference Include="SpecFlow.MsTest" Version="3.8.14" />
<PackageReference Include="MSTest.TestAdapter" Version="2.2.4" />
<PackageReference Include="MSTest.TestFramework" Version="2.2.4" />
<PackageReference Include="FluentAssertions" Version="5.10.3" />
</ItemGroup>
Added this line to one of the Binding class
[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.ClassLevel)]
Folder Structure
calling the driver as below and registering as instance
using BoDi;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using OpenQA.Selenium;
using OpenQA.Selenium.Appium;
using OpenQA.Selenium.Appium.Enums;
using OpenQA.Selenium.Appium.iOS;
using System;
using TechTalk.SpecFlow;
[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.ClassLevel)]
namespace SpecflowParallel.Driver
{
[Binding]
public class DriverSetup
{
private IObjectContainer _objectContainer;
public AppiumDriver<IWebElement> Driver;
public DriverSetup(IObjectContainer objectContainer)
{
_objectContainer = objectContainer;
}
[BeforeScenario]
public void BeforeScenario()
{
Driver = GetDriver();
_objectContainer.RegisterInstanceAs(Driver);
}
public AppiumDriver<IWebElement> GetDriver()
{
var caps = new AppiumOptions();
caps.AddAdditionalCapability(MobileCapabilityType.PlatformName, "IOS");
caps.AddAdditionalCapability(MobileCapabilityType.PlatformVersion, "14.4");
caps.AddAdditionalCapability(MobileCapabilityType.DeviceName, "iPhone 8 Plus");
caps.AddAdditionalCapability(MobileCapabilityType.AutomationName, "XCUITest");
caps.AddAdditionalCapability(MobileCapabilityType.App, "/Users/automation/build/myapp.ios.app.zip");
Driver = new IOSDriver<IWebElement>(new Uri("http://10.1.1.92:4444/wd/hub"), caps);
return Driver;
}
}
}
Node1 - iOS14_N2D2_4726.json
{
"capabilities":
[
{
"browserName": "Safari",
"platformName": "IOS",
"platformVersion": "14.4",
"maxInstances": 2,
"automationName": "XCUITest",
"appPackage": "myapp.droid.integration",
"AppActivity": "myapp.LaunchActivity",
"wdaLocalPort":"8142",
"udid":"188BA7D5-905F-473E-9CF9-3A49555F587C"
}
],
"configuration":
{
"cleanUpCycle":2000,
"timeout":30000,
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"host": "10.1.1.92",
"port": 4726,
"maxSession": 1,
"register": true,
"registerCycle": 5000,
"hubPort": 4444,
"hubHost": "10.1.1.92",
"hubProtocol": "http",
"url":"http://10.1.1.92:4726/wd/hub"
}
}
Node2 - iOS14_N1D1_4724.json
{
"capabilities":
[
{
"browserName": "Safari",
"platformName": "IOS",
"platformVersion": "14.4",
"maxInstances": 2,
"automationName": "XCUITest",
"appPackage": "myapp.droid.integration",
"appActivity": "myapp.LaunchActivity",
"wdaLocalPort":"8141",
"udid":"0227DEF2-DF40-47A1-AE95-EFFBC69FAB37"
}
],
"configuration":
{
"cleanUpCycle":2000,
"timeout":30000,
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"host": "10.1.1.92",
"port": 4724,
"maxSession": 1,
"register": true,
"registerCycle": 5000,
"hubPort": 4444,
"hubHost": "10.1.1.92",
"hubProtocol": "http",
"url":"http://10.1.1.92:4724/wd/hub"
}
}
Happy to provide further info

Log4Net in .Net Core application: LoggingEvent not set properly (LocationInformation porperties are set to "?")

So far we've been using log4net in projects targeting .NET 4.6.1, which works like a charm. Some months ago we started implementing our first .NET core projects. But log4net seems to have it's problems with this framework. We have written our own appender pushing all log events to our logging system. However in core projects essential information is missing in the LoggingEvent objects.
I've set up a minimal test project called Log4NetUnitTest to repeat that behaviour.
The csproj file:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
<IsPackable>false</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="log4net" Version="2.0.8" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.2.0" />
<PackageReference Include="MSTest.TestAdapter" Version="1.4.0" />
<PackageReference Include="MSTest.TestFramework" Version="1.4.0" />
<PackageReference Include="Newtonsoft.Json" Version="12.0.2" />
</ItemGroup>
</Project>
The unit test class configures the logger programmatically (I've also tested configuration by file with the same result):
[TestClass]
public class LoggingTests
{
[TestMethod]
public void Log()
{
var logger = LogManager.GetLogger(typeof(LoggingTests));
var appender = new MyAppender();
var loggerLogger = (log4net.Repository.Hierarchy.Logger)logger.Logger;
loggerLogger.Level = Level.All;
loggerLogger.AddAppender(appender);
loggerLogger.Hierarchy.Configured = true;
try
{
throw new Exception("Some exception");
}
catch (Exception e)
{
logger.Error("Some error text", e);
}
}
}
The custom appender (which simply writes the JSON formatted event to the console):
public sealed class MyAppender : AppenderSkeleton
{
private static readonly JsonSerializerSettings _settings = new JsonSerializerSettings
{
ReferenceLoopHandling = ReferenceLoopHandling.Ignore,
Formatting = Formatting.Indented
};
protected override void Append(LoggingEvent loggingEvent)
{
var loggingObject = new
{
ApplicationName = System.Reflection.Assembly.GetEntryAssembly()?.GetName().Name ?? System.Reflection.Assembly.GetExecutingAssembly().GetName().Name,
Method = loggingEvent.LocationInformation.MethodName,
Line = loggingEvent.LocationInformation.LineNumber,
Class = loggingEvent.LocationInformation.ClassName,
loggingEvent.LoggerName,
Environment.MachineName,
ExceptionMessage = loggingEvent.ExceptionObject?.Message,
Message = loggingEvent.MessageObject,
Level = loggingEvent.Level.Name
};
string serializedLog = JsonConvert.SerializeObject(loggingObject, _settings);
Console.WriteLine(serializedLog);
}
}
When you run the test, you will get the following output:
{
"ApplicationName": "testhost",
"Method": "?",
"Line": "?",
"Class": "?",
"LoggerName": "Log4NetUnitTest.LoggingTests",
"MachineName": "NBG-ITSD-138",
"ExceptionMessage": "Some exception",
"Message": "Some error text",
"Level": "ERROR"
}
So Method, Line and Class cannot be resolved. Also some other properties (which we are not using) like Domain, Identity and UserName are just set to "NOT AVAILABLE". When you change TargetFramework to net461 all properties are set correctly:
{
"ApplicationName": "Log4NetUnitTest",
"Method": "Log",
"Line": "31",
"Class": "Log4NetUnitTest.LoggingTests",
"LoggerName": "Log4NetUnitTest.LoggingTests",
"MachineName": "NBG-ITSD-138",
"ExceptionMessage": "Some exception",
"Message": "Some error text",
"Level": "ERROR"
}
Some years ago someone had a similar problem (not NET core), which could be solved adding loggingEvent.Fix = FixFlags.All; at the beginning of the Append() method. Didn't work for me though:-(
I guess there's some configuration or an additional package I need to add, but I have no clue what's wrong. Any suggestions are greatly appreciated.

Looking at the source for LocationInfo.cs it seems that a lot of functionality is wrapped in a #if !(NETCF || NETSTANDARD1_3) block, which is why none of it is set when using .NET Core. I presume this functionality was not available in .NET Standard 1.3.
There is an issue LOG4NET-606 in the Apache JIRA which mentions that later versions of .NET Standard added the missing functionality, and these values could be restored if log4Net was upgraded to target a higher .NET Standard version. Voting and commenting on this issue may help things along.

Why does C# code compiled on the fly not work when a debugger is attached?

I have the following C# project targetting .NET 4.0 that takes a source code file, compiles it into an assembly on the fly and then executes a static method of a type contained in that assembly.
This works as expected, as long as I don't start the program with a debugger attached. In that case I get an exception on the call to xmlSerializer.Serialize(sw, family);, more precisely a System.NullReferenceException inside a System.TypeInitializationException inside a System.InvalidOperationException.
If I take the same program, include the source code file in the project and compile it directly into the main program assembly, I will not get an exception regardless of whether or not a debugger is attached.
Please note that I my project references the exact same assemblies as those listed when compiling on the fly.
Why does it matter to the code compiled on the fly whether or not a debugger is attached? What am I missing?
Main file Program.cs:
using System;
using System.CodeDom.Compiler;
using System.IO;
using System.Reflection;
using System.Linq;
namespace DebugSerializeCompiler
{
class Program
{
static void Main()
{
if (!Environment.GetCommandLineArgs().Contains("Compile"))
{
DebugSerializeCompiler.SerializerTest.Run();
}
else
{
Assembly assembly;
if (TryCompile("..\\..\\SerializerTest.cs", new[]{ "Microsoft.CSharp.dll",
"System.dll", "System.Core.dll", "System.Data.dll", "System.Xml.dll" },
out assembly))
{
Type type = assembly.GetType("DebugSerializeCompiler.SerializerTest");
MethodInfo methodInfo = type.GetMethod("Run");
methodInfo.Invoke(null, null);
}
}
Console.ReadKey();
}
static bool TryCompile(string fileName, string[] referencedAssemblies,
out Assembly assembly)
{
bool result;
CodeDomProvider compiler = CodeDomProvider.CreateProvider("CSharp");
var compilerparams = new CompilerParameters
{
GenerateExecutable = false,
GenerateInMemory = true
};
foreach (var referencedAssembly in referencedAssemblies)
{
compilerparams.ReferencedAssemblies.Add(referencedAssembly);
}
using (var reader = new StreamReader(fileName))
{
CompilerResults compilerResults =
compiler.CompileAssemblyFromSource(compilerparams, reader.ReadToEnd());
assembly = compilerResults.CompiledAssembly;
result = !compilerResults.Errors.HasErrors;
if (!result)
{
Console.Out.WriteLine("Compiler Errors:");
foreach (CompilerError error in compilerResults.Errors)
{
Console.Out.WriteLine("Position {0}.{1}: {2}",
error.Line, error.Column, error.ErrorText);
}
}
}
return result;
}
}
}
File compiled into separate assembly SerializerTest.cs:
using System;
using System.Collections.Generic;
using System.IO;
using System.Xml.Serialization;
namespace DebugSerializeCompiler
{
public class SerializerTest
{
public static void Run()
{
Console.WriteLine("Executing Run()");
var family = new Family();
var xmlSerializer = new XmlSerializer(typeof(Family));
TextWriter sw = new StringWriter();
try
{
if (sw == null) Console.WriteLine("sw == null");
if (family == null) Console.WriteLine("family == null");
if (xmlSerializer == null) Console.WriteLine("xmlSerializer == null");
xmlSerializer.Serialize(sw, family);
}
catch (Exception e)
{
Console.WriteLine("Exception caught:");
Console.WriteLine(e);
}
Console.WriteLine(sw);
}
}
[Serializable]
public class Family
{
public string LastName { get; set; }
public List<FamilyMember> FamilyMembers { get; set; }
}
[Serializable]
public class FamilyMember
{
public string FirstName { get; set; }
}
}
This is the csproj file used to compile the project using Visual C# 2010 Express on Windows 7:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
<Platform Condition=" '$(Platform)' == '' ">x86</Platform>
<ProductVersion>8.0.30703</ProductVersion>
<SchemaVersion>2.0</SchemaVersion>
<ProjectGuid>{7B8D2187-4C58-4310-AC69-9F87107C25AA}</ProjectGuid>
<OutputType>Exe</OutputType>
<AppDesignerFolder>Properties</AppDesignerFolder>
<RootNamespace>DebugSerializeCompiler</RootNamespace>
<AssemblyName>DebugSerializeCompiler</AssemblyName>
<TargetFrameworkVersion>v4.0</TargetFrameworkVersion>
<TargetFrameworkProfile>Client</TargetFrameworkProfile>
<FileAlignment>512</FileAlignment>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|x86' ">
<PlatformTarget>x86</PlatformTarget>
<DebugSymbols>true</DebugSymbols>
<DebugType>full</DebugType>
<Optimize>false</Optimize>
<OutputPath>bin\Debug\</OutputPath>
<DefineConstants>DEBUG;TRACE</DefineConstants>
<ErrorReport>prompt</ErrorReport>
<WarningLevel>4</WarningLevel>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|x86' ">
<PlatformTarget>x86</PlatformTarget>
<DebugType>pdbonly</DebugType>
<Optimize>true</Optimize>
<OutputPath>bin\Release\</OutputPath>
<DefineConstants>TRACE</DefineConstants>
<ErrorReport>prompt</ErrorReport>
<WarningLevel>4</WarningLevel>
</PropertyGroup>
<ItemGroup>
<Reference Include="System" />
<Reference Include="System.Core" />
<Reference Include="Microsoft.CSharp" />
<Reference Include="System.Data" />
<Reference Include="System.Xml" />
</ItemGroup>
<ItemGroup>
<Compile Include="Program.cs" />
<Compile Include="Properties\AssemblyInfo.cs" />
<Compile Include="SerializerTest.cs">
<SubType>Code</SubType>
</Compile>
</ItemGroup>
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
<!-- To modify your build process, add your task inside one of the targets below and uncomment it.
Other similar extension points exist, see Microsoft.Common.targets.
<Target Name="BeforeBuild">
</Target>
<Target Name="AfterBuild">
</Target>
-->
</Project>

It worked fine for me.
But if I had to guess what's going on for you it would be that because you're compiling the class in with your main project and dynamically compiling it the serializer is getting confused about which assembly to use and is failing. You could try attaching an event to AppDomain.CurrentDomain.AssemblyResolve and see if there are any assemblies failing to resolve there.