Azure Functions - use queue trigger with managed identity - c#

I am trying to use Managed Identity with Azure Functions V3 and a QueueTrigger.
the function code is defined like this:
[Function("ProcessUserData")]
public async Task ProcessUserData([QueueTrigger("%QueueSettings:UserDataQueue%", Connection = "QueueSettings:StorageAccount")] string queueItem, FunctionContext context)
{
var logger = context.GetLogger<QueueListener>();
...
}
According to Microsoft documentation this should be possible by defining some additional configuration properties
<CONNECTION_NAME_PREFIX>__credential
<CONNECTION_NAME_PREFIX>__queueServiceUri
https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference?tabs=blob#local-development-with-identity-based-connections
My local.settings.json looks like this:
// "QueueSettings:StorageAccount": "",
"QueueSettings:StorageAccount__queueServiceUri": "https://mytestfa.queue.core.windows.net/",
"QueueSettings:StorageAccount__credential": "managedidentity",
When trying to run the project locally I get the following error:
[2021-12-06T18:07:53.181Z] The 'ProcessUserData' function is in error: Microsoft.Azure.WebJobs.Host: Error indexing method 'Functions.ProcessUserData'. Microsoft.Azure.WebJobs.Extensions.Storage: Storage account connection string 'AzureWebJobsQueueSettings:StorageAccount' does not exist. Make sure that it is a defined App Setting.
When I use and empty connection string I get another error:
"QueueSettings:StorageAccount": "",
"QueueSettings:StorageAccount__queueServiceUri": "https://mytestfa.queue.core.windows.net/",
"QueueSettings:StorageAccount__credential": "managedidentity",
Error:
[2021-12-06T18:25:20.262Z] The 'ProcessUserData' function is in error: Microsoft.Azure.WebJobs.Host: Error indexing method 'Functions.ProcessUserData'. Microsoft.Azure.WebJobs.Extensions.Storage: Storage account connection string for 'AzureWebJobsQueueSettings:StorageAccount' is invalid.
This works fine when using the full connection string with Account Key, but we have to be using managed identities.
I have upgraded to the latest version of Azure Functions Core Tole (3.0.3904) and am using Visual Studio 2022.
Additional documentation that this should work:
https://devblogs.microsoft.com/azure-sdk/introducing-the-new-azure-function-extension-libraries-beta/
Thanks for any insights.

I was able to resolve this by installing the 5.0.0-beta.4 version of the NuGet package "Microsoft.Azure.Functions.Worker.Extensions.Storage".
Now Managed Identify functionality is working as expected.
Hopefully this will go to GA soon.

Related

DirectoryEntry.Exists runs as CLI but not in WebAPI (dotnet c# activedirectory)

I'm writing an API to check if an OU exists in ActiveDirectory or not.
To perform this check, in C#, I run:
string ouName = "MyOrg";
bool ouExists = DirectoryEntry.Exists ($"LDAP://OU={ouName},DC=test,DC=local");
When I create a new CLI project and run these lines, they work fine (the app is running on the DC itself).
But when called by a Controller in a WebAPI project, they throw a runtime COMException (80004005), with the details being "Unspecified error".
I figure this has to do with how Kestrel runs the code. It should authenticate automatically as the current loggedonuser (i.e. I can't use the username, password optional parameters).
How do I do that? And is this the right way to go about it?
Exception details:
System.Runtime.InteropServices.COMException (0x80004005): Unspecified error
at System.DirectoryServices.DirectoryEntry.Bind(Boolean throwIfFail)
at System.DirectoryServices.DirectoryEntry.Exists(String path)
at OUCheck.Helpers.ActiveDirectoryHelper.OUExists(String ouDN) in /Projects/OUCheck/Helpers/ActiveDirectoryHelper.cs:line 14
System.Runtime.InteropServices.COMException
The format of the paths for the DirectoryEntry are wrong.
I can't find a supporting document, but the following is the difference:
I was making the queries like this: LDAP://{DN}
LDAP://OU=MyOrg,DC=test,DC=local
While it seems the correct way to do it is: LDAP://{domain}/{DN}
LDAP://test.local/OU=MyOrg,DC=test,DC=local
CLI apps work even with the former, perhaps assuming things about the domain.
The following transcript helped me realize, also thanks to Gabriel for some direction!
https://chat.stackoverflow.com/transcript/12432/2012/6/12
Also might be useful: Get all users from Active Directory?

Azure Functions - table storage binding not working

I have what seems to be a common problem with binding table storage in an azure function. I've read many if not all the answers on stack overflow such as
Why is Azure Function v2 unable to bind to CloudTable?
and as much as I can find on github and in ms documents. Despite trying all fixes, none have worked.
It is all the more confusing because I had it working.. when I returned to the project a couple of days later I started getting this error -
Microsoft.Azure.WebJobs.Host: Error indexing method 'GetCompetitions'. >Microsoft.Azure.WebJobs.Host: Cannot bind parameter 'competitionsTable' to type CloudTable. Make >sure the parameter Type is supported by the binding. If you're using binding extensions (e.g. Azure >Storage, ServiceBus, Timers, etc.) make sure you've called the registration method for the >extension(s) in your startup code (e.g. builder.AddAzureStorage(), builder.AddServiceBus(), >builder.AddTimers(), etc.).
To get the function working locally I had to follow several steps once I created the function in Visual Studio, as follows:
install relevant nuget packages and add extensionBundle properties to host.json. Host file modified to:
{
"version": "2.0",
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[1.*, 2.0.0)"
}
}
local.setting.json
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "dotnet"
}
}
Funtion:
[FunctionName("GetCompetitions")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req,
[Table("DailyFF", Connection = "AzureWebJobsStorage")] CloudTable competitionsTable,
ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
TableQuery<ActiveCompetition> projectionQuery = new TableQuery<ActiveCompetition>().Select(
new string[] { "RowKey", "Name" });
var activeCompetitions = await competitionsTable.ExecuteQuerySegmentedAsync(projectionQuery, null);
List<Competition> competitions = new List<Competition>();
foreach(var c in activeCompetitions.Results)
{
competitions.Add(new Competition
{
Id = c.RowKey,
Name = c.Name
});
}
return competitions != null
? (ActionResult)new OkObjectResult(competitions)
: new BadRequestObjectResult("Nothing doing, try something else.");
}
This worked. I successfully queried local table storage several times.
As I say when I returned to work on it it no longer worked with the error mentioned thrown.
A few things to note:
The function does not use any 3rd party libraries so there should be no conflicts.
All libraries use Windows.Azure.Storage 9.3.1 (as generated by visual studio)
I tried targeting.Net standard as suggested in many articles but it made no difference.
It's possible that I installed the asp.net core 3.x framework between writing the function initially and returning but all settings appear to be correct.
I'm stuck, any help/suggestions appreciated, thanks!
[edit]
Something I should add because it's looking like this is the at the root issue.. when I returned to the working function to continue development a new version of the Azure function CLI was installed automatically when I ran the project. I was a little surprised and I had to add a firewall exception as I had done previously.
I don't know which version was installed.
I've tried setting up a new profile and pointing to the latest version of the CLI downloaded from github (that has brought its own issues as I have to manually delete existing profiles from properties\launchSettings.json to get it to work). It doesn't fix the function binding issue however.
Seems to me there's a lot of unreliable "fixes" for this scenario so I'd greatly appreciate any links at all to a working demo of azure functions developed in visual studio 2017.
I'm a bit wary of using functions now I have to say. 2 days work and what was working is turning into something of a mare without even touching the working function.
so I figured it out but still not sure why it worked then stopped working without any changes..
I removed changes to host.json as it says extensionBundles aren't required if you install the pacakges with nuget as described here
https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-register#vs
If you use Install-Package to reference a binding, you don't need to use extension
bundles. This approach is specific for class libraries built in Visual Studio.
I had read that but as the function worked I hadn't paid much attention to it. Removing
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[1.*, 2.0.0)"
}
leaving the host.json file in it's initially generated state fixed the problem.
Just found this:
IQueryable isn't supported in the Functions v2 runtime. An alternative
is to use a CloudTable method parameter to read the table by using the
Azure Storage SDK. Here's an example of a 2.x function that queries an
Azure Functions log table:
If you try to bind to CloudTable and get an error message, make sure
that you have a reference to the correct Storage SDK version.
https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-table

Azure Functions Blob Trigger Dynamic Binding

I need an Azure Functions blob trigger to trigger off a bucket that is given at runtime by an app setting. I read it is possible to do this:
[FunctionName("Process")]
public static async Task Process([BlobTrigger("%BucketName%/{name}", Connection = "AzureWebJobsStorage")] Stream avroBlobStream, string name, TraceWriter log)
{
}
This works locally if I have BucketName ONLY in the Values field in appsettings.json.
{
"IsEncrypted": false,
"Values": {
"BucketName": "capture-bucket",
}
}
If its not in Values field, this is the error:
[6/24/2019 5:52:15 PM] Function 'SomeClass.Process' failed indexing and will be disabled.
[6/24/2019 5:52:15 PM] No job functions found. Try making your job classes and methods public. If you're using binding extensions (e.g. ServiceBus, Timers, etc.) make sure you've called the registration method for the extension(s) in your startup code (e.g. config.UseServiceBus(), config.UseTimers(), etc.).
I put into Azure App function a setting with just BucketName, but it gave me the same error. Could you suggest what the setting should be called or what I am doing wrong when actually in a real Azure environment? Should it be Values:BucketName? But I have never seen an example on Microsoft website with Values: as the prefix.
For your error, I have a test one situation is the Microsoft.Azure.WebJobs.Extensions.Storage package is not installed. After installation, it will work. You could have a try.
As for the dynamic bindings, the there is a description on the official tutorial: Binding expressions - app settings. And when you test locally, app setting values come from the local.settings.json file. I don't know why you are using appsettings.json. The format is just what you paste.
And on Azure, cause the settings in local.settings.json won't be deployed with VS, you have to go to you Azure Function Configuration, and set the the binding name.
I have test, this way it will work, it could process my blob file.
Hope this could help you, if I misunderstand you requirements please let me know.

Why does my .NET Core Web API (deployed on AWS Lambda) return a 502 Bad Gateway error?

I am new to .NET Core, Entity Framework, AWS, and pretty much everything here (please comment and let me know if anything needs to be added to the post).
What I am trying to do is deploy a .NET Core Web API with AWS Lambda. My API seems to be working locally, as I tested it (I have an Angular frontend app, with which I am able to successfully execute all the APIs). However, when I deploy with AWS and run a few of the APIs on Postman, I get a "502 Bad Gateway" error.
My serverless.yml file looks like this:
service: angularawardwebapi
provider:
name: aws
runtime: dotnetcore2.1
region: us-east-1
package:
artifact: bin/release/netcoreapp2.1/deploy-package.zip
functions:
api:
handler: angularawardwebapi::angularawardwebapi.LambdaEntryPoint::FunctionHandlerAsync
events:
- http:
path: /{proxy+}
method: ANY
My Model Class looks something like this:
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;
namespace angularawardwebapi.Models {
public class Student {
[Key]
public int StudentId{get;set;}
[Required]
[Column(TypeName="nvarchar(50)")]
public string StudentFirstName{get;set;}
[Required]
[Column(TypeName="varchar(50)")]
public string StudentLastName{get;set;}
}
}
My DbContext file looks like this:
public class DatabaseContext:DbContext {
public DatabaseContext(DbContextOptions<DatabaseContext> options):base(options) {
}
public DbSet<Student> Students {get;set;}
}
}
I have two methods in my Controller:
[HttpGet("temp")]
public ActionResult<List<String>> GetTemp() {
return Ok("here");
}
[HttpGet]
public ActionResult<IEnumerable<Student>> GetStudent()
{
return Ok(_context.Students);
}
The first of which works without a hitch (as expected). However, when I try to return a list of the Students that are in the DB, it seems that I'm encountering issues.
I tried using the second API locally as well, and received the appropriate response:
[
{
"StudentId": 1,
"StudentFirstName": "John",
"StudentLastName": "Doe"
}
]
I also checked the AWS CloudWatch Logs, which showed something like this:
[Information] Microsoft.AspNetCore.Hosting.Internal.WebHost: Request starting GET https://[myurl]/dev/api/Student/
[Information] Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker: Route matched with {action = "GetStudent", controller = "Student"}. Executing action WebApi.Controllers.StudentController.GetStudent ([myprojectname])
[Information] Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker: Executing action method WebApi.Controllers.StudentController.GetStudent ([myprojectname]) - Validation state: Valid
[Information] Microsoft.EntityFrameworkCore.Infrastructure: Entity Framework Core 2.1.8-servicing-32085 initialized 'DatabaseContext' using provider 'Microsoft.EntityFrameworkCore.SqlServer' with options: None
END RequestId: f57417d3-4075-4680-a78d-e90e2c710760
REPORT RequestId: f57417d3-4075-4680-a78d-e90e2c710760 Duration: 6006.80 ms Billed Duration: 6000 ms Memory Size: 1024 MB Max Memory Used: 141 MB
2019-06-06T14:19:41.116Z f57417d3-4075-4680-a78d-e90e2c710760 Task timed out after 6.01 seconds
This time out issue is not seen when I run the API locally, as I immediately get the correct response (as specified above).
Update
So I now have an RDS Database, which I can successfully connect to and play around with in SSMS. Running APIs locally also still work. However, after I deploy it using serverless deploy -v, I'm still facing the previous issue. As suggested, I tried modifying the permissions for the DB, although I'm not sure I'm doing the right thing there either. What I've done is:
Went into SSMS and modified the database permissions by right-clicking the server, going to Permissions, selecting my login (that I set up while configuring the RDS), and granted the "Alter any database" option. I then went to the database itself and modified some permissions (Delete, Insert and Update).
I also enabled the Windows Management Instrumentation (WMI-In) Windows Firewall rule.
Neither of these seems to have solved my problem.
You're probably using the same connection string you used for testing in your local Db, you should change it to use some database that the api can access to from aws.
If that's the case then your api is trying to access to a local db (which obviously it's not going to find), that would explain the timeout issue.
In order to fix it you should probably deploy your database to AWS as well and change the connection string of the project to the connection string of that database inside of AWS.

Azure Service bus Queue using C# : BadImageFormatException when calling QueueClient.CreateFromConnectionString

I am using VS2017 in Mac( with latest packages added for Azure Service Bus), to pull a message from Service bus Queue in Azure. On execution of below code, getting the error
BadImageFormatException - could not resolve field token 0x0400089c
Its coming from CreateFromConnectionString and the stack points to MessageFactory.create call which happens under the hood, on our call to CreateFromConnectionString.
Got many pointers like x86 issue and all, but none were certain on what to look into. I was using Release x86, then tried Rel AnyCpu as well.
Does anyone faced this issue before or any pointers to resolve this.
string connectionString = "Endpoint=sb://spxxxx.servicebus.windows.net/;SharedAccessKeyName=Root**Key;SharedAccessKey=xxxx.......xxxxxxxx=";
string queueName = "spqueue";
QueueClient client = QueueClient.CreateFromConnectionString(connectionString, queueName);
Also did an trail by creating the MessageFactory in the program itself. Got same error at MessagingFactory.Create
Also connectionString and queue name are fine, as I am able to generate the Authorization token correctly using this code and postman connected to the Q using the same without any issues.
Thanks!
Let me know if any additional details needs to be added.
AFAIK, Visual Studio 2017 for Mac provides the ability for using Xamarin and .NET Core to build mobile,web, and cloud applications on macOS. Per my understanding,
the Microsoft Azure Service Bus 4.1.3 targets on the traditional .NET Framework, you could try to use the next generation Azure Service Bus .NET Standard client library Microsoft.Azure.ServiceBus 0.0.7-preview.

Categories

Resources