I'm about to start using hangfire in C# in a asp.net mvc web application, and wonder how to create the right architecture.
As we are going to use HangFire, we are using it as a messagequeue, so we can process(store in the database) the user data directly and then for instance notify other systems and send email later in a separate process.
So our code now looks like this
function Xy(Client newClient)
{
_repository.save(newClient);
_crmConnector.notify(newClient);
mailer.Send(repository.GetMailInfo(), newClient)
}
And now we want to put the last two lines 'on the queue'
So following the example on the hangfire site we could do this
var client = new BackgroundJobClient();
client.Enqueue(() => _crmConnector.notify(newClient));
client.Enqueue(() => mailer.Send(repository.GetMailInfo(), newClient));
but I was wondering whether that is the right solution.
I once read about putting items on a queue and those were called 'commands', and they were classes especially created to wrap a task/command/thing-to-do and put it on a queue.
So for the notify the crm connector this would then be
client.Enqueue(() => new CrmNotifyCommand(newClient).Execute();
The CrmNotifyCommand would then receive the new client and have the knowledge to execute _crmConnector.notify(newClient).
In this case all items that are put on the queue (executed by HangFire) would be wrapped in a 'command'.
Such a command would then be a self containing class which knows how to execute a kind of business functionality. When the command itself uses more than 1 other class it could also be known as a facade I guess.
What do you think about such an architecture?
I once read about putting items on a queue and those were called
'commands', and they were classes especially created to wrap a
task/command/thing-to-do and put it on a queue.
Yes, your intuition is correct.
You should encapsulate all dependencies and explicit functionality in a separate class, and tell Hangfire to simply execute a single method (or command).
Here is my example, that I derived from Blake Connally's Hangfire demo.
namespace HangfireDemo.Core.Demo
{
public interface IDemoService
{
void RunDemoTask(PerformContext context);
}
public class DemoService : IDemoService
{
[DisplayName("Data Gathering Task Confluence Page")]
public void RunDemoTask(PerformContext context)
{
Console.WriteLine("This is a task that ran from the demo service.");
BackgroundJob.ContinueJobWith(context.BackgroundJob.Id, () => NextJob());
}
public void NextJob()
{
Console.WriteLine("This is my next task.");
}
}
}
And then separately, to schedule that command, you'd write something like the following:
BackgroundJob.Enqueue("demo-job", () => this._demoService.RunDemoTask(null));
If you need further clarification, I encourage you to watch Blake Connally's Hangfire demo.
Related
I have an ASP.NET Core web app, with WebAPI controllers. All I am trying to do is, in some of the controllers, be able to kick off a process that would run in the background, but the controller should go ahead and return before that process is done. I don't want the consumers of the service to have to wait for this job to finish.
I have seen all of the posts about IHostedService and BackgroundService, but none of them seem to be what I want. Also, all these examples show you how to set things up, but not how to actually call it, or I am not understanding some of it.
I tried these, but when you register an IHostedService in Startup, it runs immediately at that point in time. This is not what I want. I don't want to run the task at startup, I want to be able to call it from a controller when it needs to. Also, I may have several different ones, so just registering services.AddHostedService() won't work because I might have a MyServiceB and MyServiceC, so how do I get the right one from the controller (I can't just inject IHostedService)?
Ultimately, everything I have seen has been a huge, convoluted mess of code for something that seems like it should be such a simple thing to do. What am I missing?
You have the following options:
IHostedService classes can be long running methods that run in the background for the lifetime of your app. In order to make them to handle some sort of background task, you need to implement some sort of "global" queue system in your app for the controllers to store the data/events. This queue system can be as simple as a Singleton class with a ConcurrentQueue that you pass in to your controller, or something like an IDistributedCache or more complex external pub/sub systems. Then you can just poll the queue in your IHostedService and run certain operations based on it. Here is a microsoft example of IHostedService implementation for handling queues https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-3.1&tabs=visual-studio#queued-background-tasks
Note that the Singleton class approach can cause issues in multi-server environments.
Example implementation of the Singleton approach can be like:
// Needs to be registered as a Singleton in your Startup.cs
public class BackgroundJobs {
public ConcurrentQueue<string> BackgroundTasks {get; set;} = new ConcurrentQueue<string>();
}
public class MyController : ControllerBase{
private readonly BackgroundJobs _backgroundJobs;
public MyController(BackgroundJobs backgroundJobs) {
_backgroundJobs = backgroundJobs;
}
public async Task<ActionResult> FireAndForgetEndPoint(){
_backgroundJobs.BackgroundTasks.Enqueue("SomeJobIdentifier");
}
}
public class MyBackgroundService : IHostedService {
private readonly BackgroundJobs _backgroundJobs;
public MyBackgroundService(BackgroundJobs backgroundJobs)
{
_backgroundJobs = backgroundJobs;
}
public void StartAsync(CancellationToken ct)
{
while(!ct.IsCancellationRequested)
{
if(_backgroundJobs.BackgroundTasks.TryDequeue(out var jobId))
{
// Code to do long running operation
}
Task.Delay(TimeSpan.FromSeconds(1)); // You really don't want an infinite loop here without having any sort of delays.
}
}
}
Create a method that returns a Task, pass in a IServiceProvider to that method and create a new Scope in there to make sure ASP.NET would not kill the task when the controller Action completes. Something like
IServiceProvider _serviceProvider;
public async Task<ActionResult> FireAndForgetEndPoint()
{
// Do stuff
_ = FireAndForgetOperation(_serviceProvider);
Return Ok();
}
public async Task FireAndForgetOperation(IServiceProvider serviceProvider)
{
using (var scope = _serviceProvider.CreateScope()){
await Task.Delay(1000);
//... Long running tasks
}
}
Update: Here is the Microsoft example of doing something similar: https://learn.microsoft.com/en-us/aspnet/core/performance/performance-best-practices?view=aspnetcore-3.1#do-not-capture-services-injected-into-the-controllers-on-background-threads
As I understand from your question you want to create a fire and forget task like logging to database. In this scenario you don't have to wait for log to be inserted database. It also took much of my time to discover an easily implementable solution. Here is what I have found:
In your controller parameters, add IServiceScopeFactory. This will not effect the request body or header. After that create a scope and call your service over it.
[HttpPost]
public IActionResult MoveRecordingToStorage([FromBody] StreamingRequestModel req, [FromServices] IServiceScopeFactory serviceScopeFactory)
{
// Move record to Azure storage in the background
Task.Run(async () =>
{
try
{
using var scope = serviceScopeFactory.CreateScope();
var repository = scope.ServiceProvider.GetRequiredService<ICloudStorage>();
await repository.UploadFileToAzure(req.RecordedPath, key, req.Id, req.RecordCode);
}
catch(Exception e)
{
Console.WriteLine(e);
}
});
return Ok("In progress..");
}
After posting your request, you will immediately receive In Progress.. text but your task will run in the background.
One more thing, If you don't create your task in this way and try to call database operations you will receive an error like this which means your database object is already dead and you are trying to access it;
Cannot access a disposed object. A common cause of this error is disposing a context that was resolved from dependency injection and then later trying to use the same context instance elsewhere in your application. This may occur if you are calling Dispose() on the context, or wrapping the context in a using statement. If you are using dependency injection, you should let the dependency injection container take care of disposing context instances.\r\nObject name: 'DBContext'.
My code is based on Repository pattern. You should not forget to inject service class in your Startup.cs
services.AddScoped<ICloudStorage, AzureCloudStorage>();
Find the detailed documentation here.
What is the simplest way to run a single background task from a controller in .NET Core?
I don't want the consumers of the service to have to wait for this job to finish.
Ultimately, everything I have seen has been a huge, convoluted mess of code for something that seems like it should be such a simple thing to do. What am I missing?
The problem is that ASP.NET is a framework for writing web services, which are applications that respond to requests. But as soon as your code says "I don't want the consumers of the service to have to wait", then you're talking about running code outside of a request (i.e., request-extrinsic code). This is why all solutions are complex: your code has to bypass/extend the framework itself in an attempt to force it to do something it wasn't designed to do.
The only proper solution for request-extrinsic code is to have a durable queue with a separate background process. Anything in-process (e.g., ConcurrentQueue with an IHostedService) will have reliability problems; in particular, those solutions will occasionally lose work.
I have created a Quartz server running in a windows service that has various scheduled jobs.
However, there is one job that I need to be triggered manually from an event in my web application UI.
Quartz.NET job:
public class IntensiveJob : IJob
{
public void Execute(IJobExecutionContext context)
{
// Get job parameters here... BUT HOW?!
// Do some intensive processing here...
}
}
Action that I need to trigger the job in:
public class SomeController : Controller
{
[HttPost]
public ActionResult Run()
{
// Need to be able to trigger the intensive job here...
// Ideally with some arguments too... E.g:
var job = new IntensiveJob();
job.Execute();
return RedirectToAction("Index");
}
}
Any suggestions on best way to implement this or alternative approaches would be great.
The solution that best fits your problem is to create a trigger manually that gets executed now by the scheduler. The problem that arises is if you need to have different data in your trigger each time it is created to be passed to the scheduler. The trigger factory that is created by quartz will always produce an identical object when asked for a new trigger. You would have to instantiate an instance of the Trigger Impl with the necessary details and pass that to the scheduler to run. An example in Java would look something like:
JobDetailImpl jdi = new JobDetailImpl();
jdi.setJobClass(SomeClassWithJob);//The class that contains the task to done.
SimpleTriggerImpl sti = new SimpleTriggerImpl();
sti.setStartTime(new Date(System.currentTimeMillis()));
sti.setRepeatInterval(1);
sti.setRepeatCount(0);
context.getScheduler().scheduleJob(jdi,sti); //pseudocode here
The key thing is to use quartz for the scheduling and not to just insert data into the database. Also, each of the items require a name and group.
I am a complete novice with this but...
I have a small ASP MVC C# application reading an SQL database which I would like to be updated by a background process updating the DB with a JSON request potentially up to every minute or few seconds.
What is the best way to implement the background JSON DB update? In the MVC app on a persistent timer (is that even possible?) or independently (completely outside of the app) in a separate process with an executable running in the background with an internal programmatic timer or else using some kind of scheduler?
EDIT: For the sake of understanding - it is market prices in the JSON string that obviously need to be updated in the DB quite often ie potentially up to every few seconds if desirable or nessesary
I would use a Windows Service combined with the Quartz.net package.
You can run anything you want and on any schedule.
EDIT: From the above discussion I gather that your job would poll for market prices (a web request) that would run every few seconds and on getting the result would update your database.
EDIT2:
This would be your Quartz job:
public class FetchAndSaveFinancialData : IJob
{
public void Execute()
{
//web request to get info
//save to db
}
}
Then your windows service base class:
public class YourFinancialServiceBase : ServiceBase
{
protected override void OnStart(string[] args)
{
ServiceMain();
base.OnStart(args);
}
protected override void OnStop()
{
base.OnStop();
}
protected void ServiceMain()
{
var scheduler = StdSchedulerFactory.GetDefaultScheduler();
var job = JobBuilder.Create<FetchAndSaveFinancialData>().WithIdentity("Job1", "Group1").Build();
ITrigger trigger = TriggerBuilder.Create().WithIdentity("Trigger1","Group1")
.StartNow()
.WithSimpleSchedule(x=>x
.WithIntervalInSeconds(5)
.RepeatForever()
).Build();
scheduler.ScheduleJob(job,trigger);
scheduler.Start();
}
}
EDIT3:
It all depends on what you use to access the data. If you plan on using EntityFramework then I would keep the MVC project and the WindowsService project in the same solution referencing a Library project. The library project would have all your models and allow you to see if changing something effects one or both components.
On the other hand if you rely a lot on stored procedures in your database the above is less relevant. You will have to consolidate any changes in your stored procedures more than in code.
In my experience, I'd rather have to sift through many projects in one solution than trying to find a completely different solution in a different language that is a critical part of your application.
Assume the following scenario:
I have a view model that should automatically close itself after a specific delay.
Something like this:
public AutoCloseViewModel : ViewModelBase
{
public void Close()
{
/* perform actions necessary to close
the view model and its associated view */
}
protected override OnActivate()
{
// schedule job that calls Close on this instance, something like
var trigger = TriggerBuilder.Create()...;
trigger.Job = new DelegateJob(() => Close()); // <----
scheduler.ScheduleJob(trigger);
}
}
The line trigger.Job = new DelegateJob(() => Close()); obviously doesn't work, because of the following reasons:
There exists no Job property on ITrigger.
A job class is supposed to be serializable.
Question:
Is there a way to do what I am trying to do with Quartz.net or am I trying to use it for something it wasn't designed for?
BTW: I am aware of this question but to me it looks like the accepted answer abuses Quartz.net and the other answer wouldn't help me.
For this case, i agree that a timer should be the preferred way to go. I feel a scheduler for this purpose is to complicate things.
Using a scheduler or a timer will both make async stuff.
If you do want to use a scheduler, you can communicate to your viewmodel using a [Messenger] to send a message that the given job is done, and register for that in the viewmodel.1
I created an application that provides several services. Each service provides a specific processing capabilities, except one service (that is the main service) that returns true or false to the clients which request if the specified processing capabilities is available or not.
Now I would modify the application, leaving the main service unchanged and adding the support for the installation of plugin with new processing capabilities: each plugin should add new processing capabilities without the need of implement a new service, but after installing the new plugin, a new service should be avaible. In this way, a plugin should not handle the communication layer. In other words, I would like to separate the layer of the communication and processing, in order to simplify the creation of new plugins.
Is it possible?
I could create two services: the main service and the service for processing.
The first service may be used by clients to know if a certain feature is present on the server (for example, clients may ask the server if it has installed the plugin that provides the functionality for solving differential equations).
The second service could be used to send a generic task and to receive a general result, for example:
Result executeTask(Task task);
where Result and Task are abstract classes...
For example, if I develop a plugin to solve the differential equations, I first create the classes for transferring data:
public class DifferentialEquationTask : Task
// This class contains the data of the differential equation to be solved.
...
public class DifferentialEquationResult : Result
// This class contains the the result.
...
Therefore, the client should instantiate a new object DifferentialEquationTask and pass it to the method of the second service:
DifferentialEquationTask myTask = new DifferentialEquationTask(...);
...
Result result = executeTask(myTask); // called by basic application
// The second service receives myTask as a Task object.
// This Task object also contains the destination plugin, so myTask is send
// to the correct plugin, which converts it to DifferentialEquationTask
...
myResult = result as DifferentialEquationResult;
// received by the client
Moreover, each plugin should have a version for the application server and a version for the client application.
An alternative would be to include the service in the plugin itself: in this way, a new plugin should implement a new functionality and expose it via an additional service.
In summary, I thought the following two alternatives:
a main service to ask the server if it has a plugin or not, and a second service to deliver tasks at the correct plugin;
a main service to ask if the server has a plugin or not, and various additional services (an additional service for each plugin installed).
In order to choose the best approach, I could use the following requirements:
Which of the two alternatives may provide better performance?
What advantages would be obtained using a new service for each plugin than using a single service that delivers tasks at the correct plugin?
Which of the two alternatives simplifies the development of a new plugin?
Being a novice, I was wondering if there was a better approach...
Thanks a lot!
It seems like the main service could maintain a dictionary of plugins, indexed by name. Then for a client to see if the server provides a particular service, all the main service has to do is look up the name in the dictionary. And to process, the service just has to call a method on the object that's in the value portion of the dictionary entry. An example:
You have three abstract classes: Service, ServiceResult, and ServiceTask. The contents of ServiceTask and ServiceResult aren't really important for this discussion. Service must have a parameterless constructor and a method called Process that takes a ServiceTask as its sole parameter. So your differential equation solver would look like:
public class DiffeqSolver : Service
{
public DiffeqSolver()
{
// do any required initialization here
}
public ServiceResult Process(ServiceTask task)
{
DiffeqTask dtask = task as DiffeqTask;
if (dtask == null)
{
// Error. User didn't pass a DiffeqTask.
// Somehow communicate error back to client.
}
// Here, solve the diff eq and return the result.
}
}
The main service is somehow notified of existing plugins. It maintains a dictionary:
Dictionary<string, Service> Services = new Dictionary<string, Service>();
I assume you have some idea how you're going to load the plugins. What you want, in effect, is for the dictionary to contain:
Key = "DiffeqSolver", Value = new DiffeqSolver();
Key = "ServiceType1", Value = new ServiceType1();
etc., etc.
You can then have two methods for the main service: ServiceIsSupported and Process:
bool ServiceIsSupported(string serviceName)
{
return Services.ContainsKey(serviceName);
}
ServiceResult Process(string serviceName, ServiceTask task)
{
Service srv;
if (Services.TryGetValue(serviceName, out srv))
{
return srv.Process(task);
}
else
{
// The service isn't supported.
// Return a failure result
return FailedServiceResult;
}
}
I've simplified that to some extent. In particular, I'm using a Dictionary, which is not thread safe. You'd want to use a ConcurrentDictionary, or use locks to synchronize access to your dictionary.
The more difficult part, I think, will be loading the plugins. But there are many available examples of creating a plugin architecture. I think you can find what you need.