Develop Reference

Azure Timer Trigger Functions in Sequence

I am using Multiple TimerTrigger Functions as below
[FunctionName("Fun1")]
public async Task RunAsync([TimerTrigger("%ScheduleExpression%")] TimerInfo myTimer, ILogger log)
{
log.LogInformation($"Fun1 Timer trigger function executed at: {DateTime.Now}");
}
[FunctionName("Fun2")]
public async Task RunAsync([TimerTrigger("%ScheduleExpression%")] TimerInfo myTimer, ILogger log)
{
log.LogInformation($"Fun2 Timer trigger function executed at: {DateTime.Now}");
}
Likewise, I have a total of 5 functions. I want to run these in a sequence one after another. Because each function will save some data into individual tables and every next function will use the data stored by the previous function.
Is there any way to achieve this?

I would suggest you use Azure Durable Functions with the Function chaining pattern and in this way, you can execute your functions sequentially. You trigger the durable function and then the chain will execute.
See documentation here: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-overview?tabs=csharp#chaining

Why the "runtimeStatus" in "statusQueryGetUri" not set immediately after timer is finished?

Why the "runtimeStatus" is set to "Completed" only after 52 seconds not 30 as I set in context.CreateTimer() function when checking it with statusQueryGetUri http request?
The documentation that I used
My Code
[FunctionName("H")]
public static async Task<HttpResponseMessage> Start([HttpTrigger(AuthorizationLevel.Anonymous, "get","post",Route = "route/{route}")] HttpRequestMessage req, [DurableClient] IDurableOrchestrationClient client, string route)
{
string id = await client.StartNewAsync("Or1");
return client.CreateCheckStatusResponse(req, id);
}
[FunctionName("Or1")]
public static async Task<string> Or1([OrchestrationTrigger] IDurableOrchestrationContext context, ILogger logger)
{
using (CancellationTokenSource cts = new CancellationTokenSource())
{
DateTime endTime = context.CurrentUtcDateTime.AddSeconds(30);
logger.LogInformation($"*********time now {context.CurrentUtcDateTime}");
logger.LogInformation($"*********end Time {endTime}");
await context.CreateTimer(endTime, cts.Token);
logger.LogInformation($"*********end Time finish {context.CurrentUtcDateTime}");
return "timer finished";
}
}
[FunctionName("Activity1")]
public static async Task A1([ActivityTrigger] IDurableActivityContext context)
{
//Do something
}
The Log
Functions:
H: [GET,POST] http://localhost:7071/api/route/{route}
Activity1: activityTrigger
Or1: orchestrationTrigger
For detailed output, run func with --verbose flag.
[2021-01-13T16:17:06.841Z] Host lock lease acquired by instance ID '000000000000000000000000EB8F9C93'.
[2021-01-13T16:17:24.767Z] Executing 'H' (Reason='This function was programmatically called via the host APIs.', Id=0aeee0e1-6148-4c21-9aa9-d17a43bce8d1)
[2021-01-13T16:17:24.925Z] Executed 'H' (Succeeded, Id=0aeee0e1-6148-4c21-9aa9-d17a43bce8d1, Duration=164ms)
[2021-01-13T16:17:24.995Z] Executing 'Or1' (Reason='(null)', Id=6aa97b04-d526-41b1-9532-afb21c088b18)
[2021-01-13T16:17:25.006Z] *********time now 1/13/2021 4:17:24 PM
[2021-01-13T16:17:25.007Z] *********endTime 1/13/2021 4:17:54 PM
[2021-01-13T16:17:25.017Z] Executed 'Or1' (Succeeded, Id=6aa97b04-d526-41b1-9532-afb21c088b18, Duration=23ms)
[2021-01-13T16:18:16.476Z] Executing 'Or1' (Reason='(null)', Id=9749d719-5789-419a-908f-6523cf497cca)
[2021-01-13T16:18:16.477Z] *********time now 1/13/2021 4:17:24 PM
[2021-01-13T16:18:16.478Z] *********endTime 1/13/2021 4:17:54 PM
[2021-01-13T16:18:16.481Z] *********endTime finish 1/13/2021 4:18:16 PM
[2021-01-13T16:18:16.485Z] Executed 'Or1' (Succeeded, Id=9749d719-5789-419a-908f-6523cf497cca, Duration=9ms)

The azure Orchestrater works on Queue polling which is implemented as a random exponential back-off algorithm to reduce the effect of idle-queue polling on storage transaction costs. When a message is found, the runtime immediately checks for another message; when no message is found, it waits for a period of time before trying again. After subsequent failed attempts to get a queue message, the wait time continues to increase until it reaches the maximum wait time, which defaults to 30 seconds.
If see your logs, you can notice that Orchestrater has triggered Timer at 16:17:24 and when it was finished at 16:17:54 a message was added in the storage queue. As mentioned above due to queue polling it seems that the message was picked at 16:18:16 to resume the orchestration execution.
I believe if you trigger the durable function multiple times then you will notice the total time to finish the orchestration would be different for each instance.
You can read about Azure function orchestration queue polling at here.
You can also check the history table to understand when a message was queued and when picked. Read about it at here.
To show how queuing works you can stop the function as soon timer is triggered. Following is the output in my local environment emulator queue which displays that a message is queued when timer is triggered
Now when Orchestrator function resumes again then it polls the message from queue and pick it to process further.
Note - in my local environment, I tried couple of time with your code as I noticed all instances finishes in ~30 secs.

BackgroundService.StartAsync, should it complete during startup or block?

I don't know if I'm missing some piece of this puzzle but as far as I'm used to working with services in general, any call to Start is supposed to always return so that you know your service has started.
I'm studying BackgroundService:
Background tasks with hosted services in ASP.NET Core
Implement background tasks in microservices with IHostedService and the BackgroundService class
In the examples on both, they're implementing ExecuteAsync (which is called by StartAsync) using a while loop meaning that ExecuteAsync (and by extension StartAsync) is only ever going to return when cancelled (like when the app shuts down).
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
_logger.LogDebug($"GracePeriod task doing background work.");
// This eShopOnContainers method is querying a database table
// and publishing events into the Event Bus (RabbitMQ / ServiceBus)
CheckConfirmedGracePeriodOrders();
await Task.Delay(_settings.CheckUpdateTime, stoppingToken);
}
}
This contradicts what I understand from services because how can you tell that the service has successfully started vs hanging on startup?
I'd really appreciate some clarity on this as I just can't seem to get my head around what's happening and I can't seem to find any further information specific to this.
Edit: Added call to ExecuteAsync from BackgroundService for illustration:
public virtual Task StartAsync(CancellationToken cancellationToken)
{
// Store the task we're executing
_executingTask = ExecuteAsync(_stoppingCts.Token);
// If the task is completed then return it,
// this will bubble cancellation and failure to the caller
if (_executingTask.IsCompleted)
{
return _executingTask;
}
// Otherwise it's running
return Task.CompletedTask;
}

Through the magic of async, even though ExecuteAsync appears to be an infinite loop, it really returns a Task to the caller. The await on the Delay is what allows control flow to leave the ExecuteAsync function, with the next iteration of the loop only running after that Task is resolved.
If you are not used to it, it can be easy to miss.

Must StartNewAsync be awaited on the orchestration client?

I have an Azure orchestration where the orchestration client, which triggers the orchestration, threw a timeout exception.
The orchestration client function only does two things, starting two orchestrations, awaiting each as most example code suggest to.
await orchestrationClient.StartNewAsync("TableOrchestrator", updates);
await orchestrationClient.StartNewAsync("ClientOrchestrator", clientExport);
However, as I understand then the orchestration client is not a special function like the orchestration functions, so it can only run for a max of 10 minutes.
Obviously there is a high chance that the combined run time of my two orchestrations exceeds 10 minutes in total.
Questions:
Is the orchestration client state saved like the actual orchestration functions?
Do I need to await the orchestrations they do not depend on previous orchestration results?
Update Made a complete example of what my code does, and the runtimes as shown below.
It seems that starting an orchestration will await it if there is code written after, but not if the orchestration is the last statement!
Updated Questions:
Will any code after calling StartNewAsync() make the function await till the orchestration really finishes? or will e.g. log statements not trigger this behaviour?
Is it the recommended code practice that StartNewAsync() should only be called after all other code has executed?
.
public static class testOrchestration
{
[FunctionName("Start")]
public static async Task Start([TimerTrigger("0 */30 * * * *", RunOnStartup = true, UseMonitor = false)]TimerInfo myStartTimer, [OrchestrationClient] DurableOrchestrationClient orchestrationClient, ILogger log)
{
var startTime = DateTime.Now;
log.LogInformation(new EventId(0, "Startup"), "Starting Orchestror 1 ***");
await orchestrationClient.StartNewAsync("Orchestrator", "ONE");
log.LogInformation($"Elapsed time, await ONE: {DateTime.Now - startTime}");
await Task.Delay(5000);
log.LogInformation($"Elapsed time, await Delay: {DateTime.Now - startTime}");
log.LogInformation(new EventId(0, "Startup"), "Starting Orchestror 2 ***");
await orchestrationClient.StartNewAsync("Orchestrator", "TWO");
log.LogInformation($"Elapsed time, await TWO: {DateTime.Now - startTime}");
}
[FunctionName("Orchestrator")]
public static async Task<string> TestOrchestrator([OrchestrationTrigger] DurableOrchestrationContextBase context, ILogger log)
{
var input = context.GetInput<string>();
log.LogInformation($"Running {input}");
await Task.Delay(5000);
return $"Done {input}";
}
}
Running this gives me the following output:
Starting Orchestror 1 ***
Elapsed time, await ONE: 00:00:08.5445755
Running ONE
Elapsed time, await Delay: 00:00:13.5541264
Starting Orchestror 2 ***
Elapsed time, await TWO: 00:00:13.6211995
Running TWO

StartNewAsync() just schedules the orchestrators to be started (immediately). To await those calls does not mean that your initial function will really wait for the orchestrators to run - or even to actually start and finish its work.
The StartNewAsync (.NET) or startNew (JavaScript) method on the
orchestration client binding starts a new instance. Internally, this
method enqueues a message into the control queue, which then triggers
the start of a function with the specified name that uses the
orchestration trigger binding.
This async operation completes when the orchestration process is
successfully scheduled
Source
This async operation completes when the orchestration process is successfully scheduled.
So yes: You should await those calls (can also be done in parallel as Miguel suggested). But it will not take longer than a few milliseconds.

If they don't depend on each other, you can run them in parallel using:
var t1 = orchestrationClient.StartNewAsync("TableOrchestrator", updates);
var t2 = orchestrationClient.StartNewAsync("ClientOrchestrator", clientExport);
await Task.WhenAll(t1, t2);

Using Azure WebJob Timeout on Continuous Job with TimerTrigger

I have a continuous WebJob with a function using the TimerTrigger to run a process every 30 seconds. A particular call in the function occasionally and seemingly randomly hangs, causing the webjob to wait indefinitely. Current solution is notice the service has stopped, then log into the Azure Dashboard and abort it manually.
Note that I know the correct course of action is to identify the root cause and fix it. Trust me, we're working on that. In the mean time, I want to treat the symptom, and need help doing so.
I'm attempting to have the WebJob detect if status using the Timeout decorator as described in this post on the Azure WebJobs SDK: https://github.com/Azure/azure-webjobs-sdk/issues/590. Implementing the suggestion, I'm able to see that when the problematic call hangs, the Timeout is detected, but the WebJob still doesn't die. What I doing wrong here that won't kill the function to allow subsequent invocations?
Program.cs
static void Main()
{
var config = new JobHostConfiguration();
config.UseTimers();
config.FunctionTimeout = new TimeSpan(0, 15, 0);
var host = new JobHost(config);
Functions.Initialize();
host.RunAndBlock();
}
Functions.cs
[Singleton]
[Timeout("00:05:00")]
public async static Task PeriodicProcess([TimerTrigger("00:00:30", RunOnStartup = true)] TimerInfo timer, CancellationToken cancelToken, TextWriter log)
{
log.WriteLine("-- Processing Begin --");
List<Emails> cases = GetEmailsAndWhatNot();
foreach (Email e in Emails)
{
try
{
ProblematicFunction_SendEmail(e, log);
}
catch(Exception e)
{
// do stuff
}
}
log.WriteLine("-- Processing End -- ");
}
public static void ProblematicFunction_SendEmail(Email e, TextWriter log)
{
// send email
}
WebJob Output During Issues
-- Processing Begin --
Timeout value of 00:05:00 exceeded by function 'Functions.PeriodicProcess' (Id: '0f7438bd-baad-451f-95a6-9461f35bfb2d'). Initiating cancellation.
Despite the webjob initiating cancellation, the function doesn't die. Do I need to monitor the CancellationToken? How far down do I need to propogate asynchronous calling? What am I missing here that will actually abort the process?

As TimerTrigger states about TimerTrigger:
Singleton Locks
TimerTrigger uses the Singleton feature of the WebJobs SDK to ensure that only a single instance of your triggered function is running at any given time.
Scheduling
If your function execution takes longer than the timer interval, another execution won't be triggered until after the current invocation completes. The next execution is scheduled after the current execution completes.
Here is my test for this scenario, you could refer to it:
Use CancellationToken.None and never propogate the cancellation token
Note: The function PeriodicProcess would be time out after 30 s, but the Time-consuming job is still running, and after the long-running job has done, the Processing End log would be printed.
Propogate the cancellation token
Note: If we propogate the cancellation token, the Time-consuming job would be cancelled immediately.
Code snippet
[Timeout("00:00:30")]
[Singleton]
public async static Task PeriodicProcess([TimerTrigger("00:00:10", RunOnStartup = true)] TimerInfo timer, CancellationToken cancelToken, TextWriter log)
{
log.WriteLine($"-- [{DateTime.Now.ToString()}] Processing Begin --");
try
{
await longRunningJob(log, cancelToken);
}
catch (Exception e)
{
// do stuff
}
log.WriteLine($"-- [{DateTime.Now.ToString()}] Processing End -- ");
}
private async static Task longRunningJob(TextWriter log, CancellationToken cancelToken)
{
log.WriteLine($"-- [{DateTime.Now.ToString()}] Begin Time-consuming jobs --");
await Task.Delay(TimeSpan.FromMinutes(1), cancelToken);
log.WriteLine($"-- [{DateTime.Now.ToString()}] Complete Time-consuming jobs --");
}