We have an Azure function that is supposed to run as soon as a file is inserted into one of our Azure Storage blobs. We are seeing that it actually takes anywhere from 1-10 minutes to run after the file appears in the storage blob. We can't confirm for sure, but it appears that it's polling the storage blob every 10 minutes looking for changes, instead of running instantly upon insert.
Here is the code for the Trigger; the order-requests blob is the one where the file gets inserted:
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using Microsoft.WindowsAzure.Storage.Blob;
namespace Integration
{
public static class IntegrationFunction
{
[FunctionName("AbcIntegration")]
public static async Task Run(
[BlobTrigger("order-requests/{name}", Connection = "BlobStorageConString")] CloudBlockBlob blob,
[Blob("order-responses/{name}")] CloudBlockBlob outputBlob,
ILogger log)
{
var result = await new IntegrationService().IntegrateTask(blob, outputBlob);
log.LogInformation(result);
}
}
}
How can we ensure the function runs the instant the file hits the blob?
After doing some research, there are two options to address this:
Use Event Grid triggering instead of Blob triggering: https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-event-grid?tabs=csharp
Ensure the function app is on an App Service Plan (not a Consumption Plan) and make sure the App is set to Always On.
Implementing #2 is what worked for us. We were on a Consumption Plan, which can take up to 10 minutes to fire-off a trigger for a Function App. We switched the app to an App Service Plan, set it to Always On, and now we are getting immediate execution when a file hits the blob.
It's important to note that additional costs are involved with an Always On App Service Plan. The idea behind a consumption plan is to only pay for when the function is running. This comes at its own cost though of possible cold starts taking up to 10 minutes for executions to occur.
Related
I am currently experiencing difficulties with running a long HTTP request in a Web Job. Same application runs fine when started on local machine. Since I've tried multiple ways of trying to make this work I am now curious if this is possible at all.
My testing code below:
static void Main(string[] args)
{
/// [...]Getting configuration and using it in "SendPayload"
_stopwatch = new Stopwatch();
_stopwatch.Start();
Task.Run(async () => { await SendPayloadAndAwait(configuration); });
while (_isWaiting)
{
if (_stopwatch.ElapsedMilliseconds % 5000 == 0)
{
Console.WriteLine(".");
}
}
}
public static async Task<string> SendPayloadAndAwait (RequestModel targetRequest)
{
/// [...]Preparing client and payload
Console.WriteLine("Start");
Response= curClient.SendAsync(req).Result;
var endContent = Response.Content.ReadAsStringAsync().Result;
Console.WriteLine("End");
_isWaiting = false;
return "Done";
}
The "End" is never printed in Web Job console, it is an infinite (up to the point of timeout) loop of ".". Is such scenario somehow prohibited?
The way I solved at a previous company was by having the long running job in a Web API which was implemented as an Azure Function was
First call (i.e. /api/StartJob)
creates a job entry (in azure table)
starts backgound thread
returns a 202 [Accepted] status with the job entries Id
Background thread performs long running task updating job entry with it's progress
Client Loop (while percentage done < 100)
Requests status (i.e. /api/JobStatus)
If percentage done = 100 exit loop
This was very successful and reliable providing we had a paid for plan
The free plans kill the session after approx 5 minutes (by design)
You will need to enable the "Always on" setting to make sure your long-running job doesn't get terminated.
The web app times out after 20 minutes of inactivity, and only the requests to the actual web app resets this timer. The web jobs are executed asynchronously in a separate process, so they don't keep the web app "awake".
Here's a note from the web jobs documentation:
A web app can time out after 20 minutes of inactivity. and only requests to the actual web app can reset the timer. Viewing the app's configuration in the Azure portal or making requests to the advanced tools site (https://<app_name>.scm.azurewebsites.net) doesn't reset the timer. If you set your web app to run continuous or scheduled (timer-trigger) WebJobs, enable the Always on setting on your web app's Azure Configuration page to ensure that the WebJobs run reliably. This feature is available only in the Basic, Standard, and Premium pricing tiers.
In my dev environment, I have an Azure Functions with 21 functions and the app plan is consumption.
Some functions have a timer trigger and at the end of the process each function sends an email. I have 2 type of timer trigger:
run a function every 20 minutes
run a function once at a particular time in the night
Every 20 minutes the function is doing what I expect. Great.
The problem I'm facing is with the function that they have to start at a particular time. Basically, they don't start until I open the portal and do something on the Azure Function (for example open the monitor for one of them).
In the code point of view, all functions with the timer trigger are defined like that:
[FunctionName("invoiceMonthlyGeneratorTimer")]
public void Run([TimerTrigger("%Timers:GenerateMonthlyInvoices%")] TimerInfo myTimer)
{
// ..
}
[FunctionName("invoiceDunningTimer")]
public async Task Run([TimerTrigger("%Timers:DunningTimer%")] TimerInfo timer)
{
// ...
}
The configuration of the timer is in the settings file like:
"Timers": {
"DunningTimer": "0 0 4 * * *",
"GenerateMonthlyInvoices": "0 0 4 * * *"
}
Generally, speaking, this approach was working in dev environment and it is working perfectly in the production environment.
Because each function sends an email, I expect each morning to find in my inbox some emails but it doesn't happen. Then, I open the Azure portal to see the logs and the monitor.
Open the Azure function in the portal.
Open the monitor for a function
Voila, after a couple of seconds, I start to receive the email for all services! In the production environment I don't have all the function I have in dev because I want to test before deploying. In prod the functions are working fine and start at the right time.
If I look at Application Insights, I can find only the logs for the time I opened the monitor.
There is one interesting thing in the log:
Trigger Details: UnscheduledInvocationReason: IsPastDue, OriginalSchedule: 2020-07-24T05:00:00.0000000+00:00
Update
Apparently, you can't have more than a couple of timer triggers in the same Azure Functions. I opened an issue on Github, so if other devs are facing the same. Something similar with HTTP triggers, look this post.
There are too many functions in your function app, they may interact each other. I met similar problem with this, in that case, the timer trigger functions in one function app and did not work. But when i deploy them to different function apps, they work fine. so you can try to deploy your second function to another function app.
And I suggest you report this problem to Microsoft, they can know more information about this problem, and may have a better solution.
I have an Azure Function that is hooked up to a Service Bus queue. It receives a message and generates records to save in an Azure Table storage.
The Service Bus queue currently has a lot of messages:
Since there are a lot of pending messages, I would expect the scaling of the Azure Functions to happen, however it does not seem to be the case:
There is only one instance of the Function running and I expected more to help empty the queue.
As specified in the Azure Function's documentation on scaling when using a service bus, I made sure the policy used by the Azure Function included the Manage rights to help scaling:
Question
Why is my Azure Function running on a Consumption Plan not scaling to help dequeue all the messages from the Service Bus?
Update
Here is an extract of the Azure Function:
public static class RecordImporter
{
private static readonly IImporter Importer;
static RecordImporter()
{
Importer = new AzureTableImporter();
}
[FunctionName("Importer")]
public static async Task Run([ServiceBusTrigger("records ", Connection = "serviceBusConnectionString")] string serializedRecords, ILogger log)
{
var records = JsonConvert.DeserializeObject<List<Record>>(serializedRecords);
await Importer.AddRecordsAsync(records);
}
}
Just an idea, because some people face the similar problem with service bus trigger scale out:
https://github.com/microsoft/azure-pipelines-tasks/issues/6759
I notice you are using C#, so please do this when publish:
(Clear the Run from package file check box. By default, VS uses Zip deployment, which can be changed to Web deployment through the above steps.)
I need to send a Push Notification once in a day, and I have created the API for the same. but I don't know how to run it from Azure Server.
I have read the below documentation but it is required some files to be uploaded but I want to run an API daily and once in a day.
Link: https://learn.microsoft.com/en-us/azure/app-service-web/web-sites-create-web-jobs
Do I have to create another project and upload in a WebJobs?
Thank You.
You can create a new project and create a webJob, but unless the Web Service you are deploying to has 'Always On' enabled, your app may be asleep when it should be starting.
Because of that I would actually recommend Azure Functions for this feature. under consumption plan you are only charged for the amount of time the logic runs and the execution count.
The function will look something like this in C# Scripting
using System;
public static async Task Run(TimerInfo myTimer, IAsyncCollector<Notification> notification, TraceWriter log)
{
log.Info($"C# Timer trigger function executed at: {DateTime.Now}");
await notification.AddAsync(new Notification(){
// your code here
});
}
EDIT
instructions for Azure Notification Hub
I'm new to Azure WebJobs, I've run a sample where a user uploads an image to blob storage and inserts a record into the Queue, then the job retrieves that from the queue as a signal to do something like resizing the uploaded image. Basically in the code the job uses QueueTrigger attribute on a public static method to do all that.
Now I need a job that just does something like inserting a record into a database table every hour, it does not have any type of trigger, it just runs itself. How do I do this?
I tried to have a static method and in it I do the insert to db, the job did start but I got a message saying:
No functions found. Try making job classes public and methods public
static.
What am I missing?
Edit
After Victor's answer I tried the following,
static void Main()
{
JobHost host = new JobHost();
host.Call(typeof(Program).GetMethod("ManualTrigger"));
}
[NoAutomaticTrigger]
public static void ManualTrigger()
{
// insert records to db
}
but this time I got InvalidOperationException,
'Void ManualTrigger()' can't be invoked from Azure WebJobs SDK. Is it missing Azure WebJobs SDK attributes?
If you don't use any input/output attributes from the WebJobs SDK (QueueTrigger, Blob, Table, etc), you have to decorate the job with the NoAutomaticTrigger Attribute to be recognized by the SDK.
You could use the latest WebJobs SDK, which supports triggering job functions on schedule, based on the same CRON expression format.
You can use it to schedule your job every hour:
[Disable("DisableMyTimerJob")]
public static void TimerJob([TimerTrigger("00:01:00")] TimerInfo timerInfo, TextWriter log)
{
log.WriteLine("Scheduled job fired!");
}
Moreover, the WebJobs SDK also has a DisableAttribute that can be applied to functions, that allows you to enable/disable functions based on application settings. If you change the app setting in the Azure Management Portal, the job will be restarted (https://azure.microsoft.com/en-us/blog/extensible-triggers-and-binders-with-azure-webjobs-sdk-1-1-0-alpha1/).