I am using Multiple TimerTrigger Functions as below
[FunctionName("Fun1")]
public async Task RunAsync([TimerTrigger("%ScheduleExpression%")] TimerInfo myTimer, ILogger log)
{
log.LogInformation($"Fun1 Timer trigger function executed at: {DateTime.Now}");
}
[FunctionName("Fun2")]
public async Task RunAsync([TimerTrigger("%ScheduleExpression%")] TimerInfo myTimer, ILogger log)
{
log.LogInformation($"Fun2 Timer trigger function executed at: {DateTime.Now}");
}
Likewise, I have a total of 5 functions. I want to run these in a sequence one after another. Because each function will save some data into individual tables and every next function will use the data stored by the previous function.
Is there any way to achieve this?
I would suggest you use Azure Durable Functions with the Function chaining pattern and in this way, you can execute your functions sequentially. You trigger the durable function and then the chain will execute.
See documentation here: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-overview?tabs=csharp#chaining
Related
I am running with a performance issue with the service bus queue-based durable function.
Here is the scenario,
public async Task KickOff(
[ServiceBusTrigger(Constants.QueueName, Connection = "ServiceBusConnectionString")]
ProcessExecution objectFromQueue,
[DurableClient] IDurableOrchestrationClient client,
ILogger logger)
{
//Call Orchestration
await client.StartNewAsync(nameof(SubOrchestration), "My Instance", objectFromQueue);
}
[FunctionName(nameof(SubOrchestration))]
public async Task<Result> SubOrchestration(
[OrchestrationTrigger] IDurableOrchestrationContext context,
ILogger logger)
{
// Call Sub Orchestration
var result = await context.CallSubOrchestratorAsync<Result<MyObjectResult>>(nameof(TaskSubOrchestration), context);
}
[FunctionName(nameof(TaskSubOrchestration))]
public async Task<Result<MyObjectResult>> TaskSubOrchestration(
[OrchestrationTrigger] IDurableOrchestrationContext context,
ILogger logger)
{
//Call context.CallActivityAsync1
//Call context.CallActivityAsync2
//Call context.CallActivityAsync3
}
It shows inconsistent behaviour for all the above Orchestration executions. Sometimes, it takes 15-20 or 50 seconds to complete the entire operation.
I have verified the logs in the Azure App Insights, and I am confused with this behaviour. I see there is no time in the execution of each Activity/SubOrchestration, but it takes time to invoke the SubOrchestration.
In most cases, it takes 15-20 seconds to invoke
SubOrchestration
TaskSubOrchestration
The Orchestration execution time is less, but most of the time is taken to invoke this 2 Orchestration.
Can you please help me if there is any way to optimize the functions in azure or if I am missing anything in the implementation?
If an Azure function executes for say 10 minutes, but has a period of 5 minutes, does the trigger still fire at the 5 minute mark?
The easiest way to answer this question is to test it yourself.
I created a small function that did nothing more than wait for 90 seconds after executing.
The timer is set to run every minute and therefore, the test is to see if the function still executes every minute on the minute or if it delays by the 30 seconds.
The logs show that the answer to your question is ... NO
You can see that it's queued another execution though because as soon as it's finished, it starts again.
You'll also see the start times of each invocation is delayed by the 30 seconds additional it takes for the function to run.
This is the function ...
using System;
public static async Task Run(TimerInfo myTimer, ILogger log)
{
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
await Task.Delay(90000);
}
Monitor Log
Invocations
One of the workaround I tried is:
In host.json, given the function timeout explicitly to test locally:
{
"version": "2.0",
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"excludedTypes": "Request"
}
}
},
"functionTimeout": "00:05:00"
}
In Azure Function, Timer trigger is scheduled at every 10 minutes:
public void Run([TimerTrigger("0 */10 * * * *")]TimerInfo myTimer, ILogger log)
{
log.LogInformation($"C# Timer trigger function1 executed at: {DateTime.Now}");
Even Running in Azure Cloud (Consumption Mode) also, there is no timeout happened as the timer is still running every 10 minutes:
As mentioned in the MS Doc and an Azure blog article, the timeout comes on Azure Function request (it means your business logic should be completed before the function timeout occurs, so at every schedule you can run the same logic by using Timer Trigger).
I have 1 function app project, with a Timer trigger and a http trigger. (using DI or not, does not seem to matter)
The timertrigger:
public static class Timer
{
[FunctionName("Timer")]
public static void Run([TimerTrigger("0 */5 * * * *", RunOnStartup =true)]TimerInfo myTimer, ILogger log)
{
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
Thread.Sleep(10000);//this is just for demonstrating the behavior!!
log.LogInformation($"C# Timer trigger function executed done");
}
}
the http trigger:
public static class Function1
{
[FunctionName("test2")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = null)] HttpRequest req,
ILogger log)
{
return new OkObjectResult("Done");
}
}
The issue is, that when the timer runs, all requests are blocked. Resulting in "Error: connect ECONNREFUSED 127.0.0.1:7071"
I read here and there (mostly in comments) that multiple triggers are (were) not supported, but that doesn't make sense to me as everything seems prepared for it anyway (i mean: i could not find any conclusive documentation about it, VS doesnt throw ANY resistance to add multiple, and even the debug interface and azure interface nicely sums up all the functions it could find...)
Is this behaviour expected, or is there some setting i can change to make it run in parallel or so??
This behavior is expected, since you're completely halting the Azure Functions process by running a Thread.Sleep(). As long as your Function App hasn't scaled to multiple instances, there's only one process running your Functions. This is where working asynchronously comes in.
If you change your timer triggered function to this, it should work fine.
[FunctionName("Timer")]
public static async Task Run([TimerTrigger("0 */5 * * * *", RunOnStartup=true)] TimerInfo myTimer, ILogger log)
{
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
await Task.Delay(10000);
log.LogInformation($"C# Timer trigger function executed done");
}
Next to this, please be advised:
It is not recommended to have your timer triggered Function RunOnStartup set to true
It is not advised to explicitly build in delays in Functions
Multiple triggers for one Function are not supported, having multiple Functions each with their own trigger within one Function App is supported.
A Function App is a collection of one or more Functions
Triggers are what cause a function to run. A trigger defines how a function is invoked and a function must have exactly one trigger. Triggers have associated data, which is often provided as the payload of the function.
Source: Azure Functions triggers and bindings concepts
EDIT:
Tested it, and this also occurs when the Function is async, and also when it's running on Azure. This means the initial run of the Function with runAtStartup=true always blocks any other incoming requests, and with runOnStartup enabled the trigger is invoked whenever your function app is scaled.
This turns out to be because of RunOnStartup. Any scheduled runs after the initial one do not have this issue.
However, if RunOnStartup = false, the trigger still might be called on startup and then it does have the issue.
As stated in the comments, async does not change anything (I actually use async and DI in the real scenario, but simplified the example here.)
I have a time-triggered Azure Function that runs every SECOND. The function reads data from API Servers and stores it into ADLS. How can I optimize the performance of the function so that it can make more that 500 API calls and store per second data for each call in a SECOND.
public static void Run([TimerTrigger("*/1 * * * * *")] TimerInfo myTimer, ILogger log)
{
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
log.LogInformation($"Execution starts at: {DateTime.Now.ToString("hh.mm.ss.ffffff")}");
try
{
var IDs = GetIDs(); //makes 1 API call to fetch list of IDs
foreach(var i in IDs){
ReadAndWriteData(i); //reads data for each ID from API server and stores in ADLS
}
}
catch (Exception e)
{
log.LogError($"An exception has been raised : {e}");
}
log.LogInformation($"C# Timer trigger function execution ended at: {DateTime.Now}");
}
public static async Task<List<string>> GetIDs(){
//List<string> idList = await Task.Run(()=> ReadIDs()); //makes 1 API call to fetch list of IDs
//return idList;
}
public static async Task ReadAndWriteData(String id){
//var result = await Task.Run(()=> ReadData()); //reads data for each ID from API server
...
// uploads data to ADLS
}
What is the best possible way to get data accurately for all IDs per second? I have tried some parallel programming/ TPL methods but still it is giving expected accuracy if I use only one ID, not for all.
First of all, there can be many issues that are causing performance problems to you. You have to debug using Azure Service Profiler or any other tool and check which line of code is taking how much time.
Some of the reasons can be:
There is inefficient algorithm written for fetching IDs/ADLS operations.
You have not added .ConfigureAwait(false) along with await.
Automatic scaling is not enabled for Azure Functions and scaling is hampered due to insufficient manual scaling.
You are using heavy Nuget packages which take a lot of time to create an instance of Azure Functions.
You have not made ReadIDs and ReadData functions as asynchronous.
I have a continuous WebJob with a function using the TimerTrigger to run a process every 30 seconds. A particular call in the function occasionally and seemingly randomly hangs, causing the webjob to wait indefinitely. Current solution is notice the service has stopped, then log into the Azure Dashboard and abort it manually.
Note that I know the correct course of action is to identify the root cause and fix it. Trust me, we're working on that. In the mean time, I want to treat the symptom, and need help doing so.
I'm attempting to have the WebJob detect if status using the Timeout decorator as described in this post on the Azure WebJobs SDK: https://github.com/Azure/azure-webjobs-sdk/issues/590. Implementing the suggestion, I'm able to see that when the problematic call hangs, the Timeout is detected, but the WebJob still doesn't die. What I doing wrong here that won't kill the function to allow subsequent invocations?
Program.cs
static void Main()
{
var config = new JobHostConfiguration();
config.UseTimers();
config.FunctionTimeout = new TimeSpan(0, 15, 0);
var host = new JobHost(config);
Functions.Initialize();
host.RunAndBlock();
}
Functions.cs
[Singleton]
[Timeout("00:05:00")]
public async static Task PeriodicProcess([TimerTrigger("00:00:30", RunOnStartup = true)] TimerInfo timer, CancellationToken cancelToken, TextWriter log)
{
log.WriteLine("-- Processing Begin --");
List<Emails> cases = GetEmailsAndWhatNot();
foreach (Email e in Emails)
{
try
{
ProblematicFunction_SendEmail(e, log);
}
catch(Exception e)
{
// do stuff
}
}
log.WriteLine("-- Processing End -- ");
}
public static void ProblematicFunction_SendEmail(Email e, TextWriter log)
{
// send email
}
WebJob Output During Issues
-- Processing Begin --
Timeout value of 00:05:00 exceeded by function 'Functions.PeriodicProcess' (Id: '0f7438bd-baad-451f-95a6-9461f35bfb2d'). Initiating cancellation.
Despite the webjob initiating cancellation, the function doesn't die. Do I need to monitor the CancellationToken? How far down do I need to propogate asynchronous calling? What am I missing here that will actually abort the process?
As TimerTrigger states about TimerTrigger:
Singleton Locks
TimerTrigger uses the Singleton feature of the WebJobs SDK to ensure that only a single instance of your triggered function is running at any given time.
Scheduling
If your function execution takes longer than the timer interval, another execution won't be triggered until after the current invocation completes. The next execution is scheduled after the current execution completes.
Here is my test for this scenario, you could refer to it:
Use CancellationToken.None and never propogate the cancellation token
Note: The function PeriodicProcess would be time out after 30 s, but the Time-consuming job is still running, and after the long-running job has done, the Processing End log would be printed.
Propogate the cancellation token
Note: If we propogate the cancellation token, the Time-consuming job would be cancelled immediately.
Code snippet
[Timeout("00:00:30")]
[Singleton]
public async static Task PeriodicProcess([TimerTrigger("00:00:10", RunOnStartup = true)] TimerInfo timer, CancellationToken cancelToken, TextWriter log)
{
log.WriteLine($"-- [{DateTime.Now.ToString()}] Processing Begin --");
try
{
await longRunningJob(log, cancelToken);
}
catch (Exception e)
{
// do stuff
}
log.WriteLine($"-- [{DateTime.Now.ToString()}] Processing End -- ");
}
private async static Task longRunningJob(TextWriter log, CancellationToken cancelToken)
{
log.WriteLine($"-- [{DateTime.Now.ToString()}] Begin Time-consuming jobs --");
await Task.Delay(TimeSpan.FromMinutes(1), cancelToken);
log.WriteLine($"-- [{DateTime.Now.ToString()}] Complete Time-consuming jobs --");
}