Develop Reference

What is the simplest way to run a single background task from a controller in .NET Core?

I have an ASP.NET Core web app, with WebAPI controllers. All I am trying to do is, in some of the controllers, be able to kick off a process that would run in the background, but the controller should go ahead and return before that process is done. I don't want the consumers of the service to have to wait for this job to finish.
I have seen all of the posts about IHostedService and BackgroundService, but none of them seem to be what I want. Also, all these examples show you how to set things up, but not how to actually call it, or I am not understanding some of it.
I tried these, but when you register an IHostedService in Startup, it runs immediately at that point in time. This is not what I want. I don't want to run the task at startup, I want to be able to call it from a controller when it needs to. Also, I may have several different ones, so just registering services.AddHostedService() won't work because I might have a MyServiceB and MyServiceC, so how do I get the right one from the controller (I can't just inject IHostedService)?
Ultimately, everything I have seen has been a huge, convoluted mess of code for something that seems like it should be such a simple thing to do. What am I missing?

You have the following options:
IHostedService classes can be long running methods that run in the background for the lifetime of your app. In order to make them to handle some sort of background task, you need to implement some sort of "global" queue system in your app for the controllers to store the data/events. This queue system can be as simple as a Singleton class with a ConcurrentQueue that you pass in to your controller, or something like an IDistributedCache or more complex external pub/sub systems. Then you can just poll the queue in your IHostedService and run certain operations based on it. Here is a microsoft example of IHostedService implementation for handling queues https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-3.1&tabs=visual-studio#queued-background-tasks
Note that the Singleton class approach can cause issues in multi-server environments.
Example implementation of the Singleton approach can be like:
// Needs to be registered as a Singleton in your Startup.cs
public class BackgroundJobs {
public ConcurrentQueue<string> BackgroundTasks {get; set;} = new ConcurrentQueue<string>();
}
public class MyController : ControllerBase{
private readonly BackgroundJobs _backgroundJobs;
public MyController(BackgroundJobs backgroundJobs) {
_backgroundJobs = backgroundJobs;
}
public async Task<ActionResult> FireAndForgetEndPoint(){
_backgroundJobs.BackgroundTasks.Enqueue("SomeJobIdentifier");
}
}
public class MyBackgroundService : IHostedService {
private readonly BackgroundJobs _backgroundJobs;
public MyBackgroundService(BackgroundJobs backgroundJobs)
{
_backgroundJobs = backgroundJobs;
}
public void StartAsync(CancellationToken ct)
{
while(!ct.IsCancellationRequested)
{
if(_backgroundJobs.BackgroundTasks.TryDequeue(out var jobId))
{
// Code to do long running operation
}
Task.Delay(TimeSpan.FromSeconds(1)); // You really don't want an infinite loop here without having any sort of delays.
}
}
}
Create a method that returns a Task, pass in a IServiceProvider to that method and create a new Scope in there to make sure ASP.NET would not kill the task when the controller Action completes. Something like
IServiceProvider _serviceProvider;
public async Task<ActionResult> FireAndForgetEndPoint()
{
// Do stuff
_ = FireAndForgetOperation(_serviceProvider);
Return Ok();
}
public async Task FireAndForgetOperation(IServiceProvider serviceProvider)
{
using (var scope = _serviceProvider.CreateScope()){
await Task.Delay(1000);
//... Long running tasks
}
}
Update: Here is the Microsoft example of doing something similar: https://learn.microsoft.com/en-us/aspnet/core/performance/performance-best-practices?view=aspnetcore-3.1#do-not-capture-services-injected-into-the-controllers-on-background-threads

As I understand from your question you want to create a fire and forget task like logging to database. In this scenario you don't have to wait for log to be inserted database. It also took much of my time to discover an easily implementable solution. Here is what I have found:
In your controller parameters, add IServiceScopeFactory. This will not effect the request body or header. After that create a scope and call your service over it.
[HttpPost]
public IActionResult MoveRecordingToStorage([FromBody] StreamingRequestModel req, [FromServices] IServiceScopeFactory serviceScopeFactory)
{
// Move record to Azure storage in the background
Task.Run(async () =>
{
try
{
using var scope = serviceScopeFactory.CreateScope();
var repository = scope.ServiceProvider.GetRequiredService<ICloudStorage>();
await repository.UploadFileToAzure(req.RecordedPath, key, req.Id, req.RecordCode);
}
catch(Exception e)
{
Console.WriteLine(e);
}
});
return Ok("In progress..");
}
After posting your request, you will immediately receive In Progress.. text but your task will run in the background.
One more thing, If you don't create your task in this way and try to call database operations you will receive an error like this which means your database object is already dead and you are trying to access it;
Cannot access a disposed object. A common cause of this error is disposing a context that was resolved from dependency injection and then later trying to use the same context instance elsewhere in your application. This may occur if you are calling Dispose() on the context, or wrapping the context in a using statement. If you are using dependency injection, you should let the dependency injection container take care of disposing context instances.\r\nObject name: 'DBContext'.
My code is based on Repository pattern. You should not forget to inject service class in your Startup.cs
services.AddScoped<ICloudStorage, AzureCloudStorage>();
Find the detailed documentation here.

What is the simplest way to run a single background task from a controller in .NET Core?
I don't want the consumers of the service to have to wait for this job to finish.
Ultimately, everything I have seen has been a huge, convoluted mess of code for something that seems like it should be such a simple thing to do. What am I missing?
The problem is that ASP.NET is a framework for writing web services, which are applications that respond to requests. But as soon as your code says "I don't want the consumers of the service to have to wait", then you're talking about running code outside of a request (i.e., request-extrinsic code). This is why all solutions are complex: your code has to bypass/extend the framework itself in an attempt to force it to do something it wasn't designed to do.
The only proper solution for request-extrinsic code is to have a durable queue with a separate background process. Anything in-process (e.g., ConcurrentQueue with an IHostedService) will have reliability problems; in particular, those solutions will occasionally lose work.

Serilog Not Outputting Anything

Using .Net 4.6 I have a static Serilog helper class - I've stripped down to the essentials as follows:
public static class SerilogHelper
{
private static ILogger log;
private static ILogger CreateLogger()
{
if (log == null)
{
string levelString = SSOSettingsFileManager.SSOSettingsFileReader.ReadString(
"BizTalk.Common", "serilog.minimum-level");
SerilogLevel level = (SerilogLevel)Enum.Parse(typeof(SerilogLevel), levelString);
string conString = SSOSettingsFileManager.SSOSettingsFileReader.ReadString(
"BizTalk.Common", "serilog.connection-string");
var levelSwitch = new LoggingLevelSwitch();
levelSwitch.MinimumLevel = (Serilog.Events.LogEventLevel)level;
log = new LoggerConfiguration()
.MinimumLevel.ControlledBy(levelSwitch)
.WriteTo.MSSqlServer(connectionString: conString, tableName: "Serilog", autoCreateSqlTable: true)
.WriteTo.RollingFile("log-{Date}.txt")
.CreateLogger();
}
return log;
}
public static void WriteString(string content)
{
var logger = CreateLogger();
logger.Information(content);
}
I have the following unit test:
[TestMethod]
public void UN_TestSerilog1()
{
Common.Components.Helpers.SerilogHelper.WriteString("Simple logging");
}
I've stepped through the debugger to be sure that the "level" variable is being set correctly - it's an enum named "Debug" with value of 1.
Although the Sql Server table is created ok, I don't see any rows inserted or any log txt file.
I've also tried calling logger.Error(content) but still no output.
I've used the same helper code previously on a different site / project and it worked ok.
Where did I go wrong this time?

Serilog.Sinks.MSSqlServer is a "periodic batching" sink and by default, it waits 5 seconds before sending the logs to the database. If your test ends before the sink had a chance to write the messages to the database, they are simply lost...
You need to make sure you dispose the logger before your test runner ends, to force the sink to flush the logs to the database at that point. See Lifecycle of Loggers.
((IDisposable) logger).Dispose();
Of course, if you are sharing a static log instance across multiple tests, you can't just dispose the logger inside of a single test as that would mean the next test that runs won't have a logger to write to... In that case, you should look at your testing framework support for executing code once, before the test suite run starts, and once again, for when the a test suite run ends.
I'm guessing you are using MSTest (because of the TestMethod), so you probably want to look into AssemblyInitialize and AssemblyCleanup, which would give you the opportunity to initialize the logger for all tests, and clean up after all tests finished running...
You might be interested in other ideas for troubleshooting Serilog issues: Serilog MSSQL Sink doesn't write logs to database

Masstransit In Memory Testing waiting for consumers to execute

So I've been trying to write a test for mass transit using the in-memory feature. I wondered what peoples approach was to waiting for consumers to execute. In the example below a use a sleep or I've also tried a while loop, but not a fan of either, any better ideas? I basically want to check that the consumer is executed.
[Fact]
public async Task SomeTest()
{
var busControl = Bus.Factory.CreateUsingInMemory(cfg =>
{
cfg.ReceiveEndpoint("commands", ec =>
{
ec.LoadFrom(context);
});
});
var address = new Uri(bus.Address, "commands")
await sendEndpoint.Send(MyExampleCommand());
Thread.Sleep(2000);
//Check nsubstitute mock received
}

Look at the test harness features that are built into MassTransit. They should give you some good ideas of how to test them.
You can look at the harness tests to see how they should be used. Note that they work with any test framework, not just NUnit.
https://github.com/MassTransit/MassTransit/blob/develop/tests/MassTransit.Tests/Testing/ConsumerTest_Specs.cs
The Testing documentation explains how to use the test harnesses.

Clean Up after Canceling tests

I'm currently running tests through visual studio. Before all the tests are run, I automatically create a set number of users with know credentials, and at the end of the run I delete those users. However, sometimes I need to cancel my tests midway. In these cases the test never gets the chance to clean up, this means that there is left over fake user info from the test run and may causes the next test run to crash (when it attempts to add user info into the DB). Is there anyway to force visual studio/mstest to run a clean up method even if the test is canceled?
I know one option is to have the test check and make sure that the user info doesn't already exist, and if it does remove it before creating the new users. But this still wouldn't solve the issue of the canceled test run leaving unwanted test data.
Update:
Sorry for the miscommunication, however cleaning up the data at the start of the test is not an option. I'm giving a very simplistic view of the issue, but put simply, I have no easy way of making sure that no test data exists at the start of the test. All clean up must occur at the end of the test.

That is impossible. You better find an alternative solution like using separate database for testing and clean all data before each test run, using fixed set of test users or mark test data with some flag. Check Isolating database data in integration tests article by Jimmy Bogard.
There is no built-in way to change MSTest default behavior. In theory you can write MSTest extension that utilizes TestExecution.OnTestStopping event, but that is not an easy process and it requires registry change. Moreover, a lot of people complain that it is not working.
There also MSTest V2, a new version of MSTest with new extensibility points. But it looks like you can't alter cancel behavior with this points, only write attribute decorators. See Extending MSTest V2.
You can't use AppDomain.CurrentDomain.ProcessExit and Process.GetCurrentProcess().Exited events because cancel seems to kill test run process.
NUnit also doesn't support this at the moment. See related NUnit test adapter Run TearDowns on VS Cancel Test Run issue.

Instead of calling the cleanup function at the end of the test, I call mine at the beginning of each test in order to address this exact problem.

Perform the clean up before creating the data as well, this will ensure that you have no leftover data whatever happens. Of course this is only possible if you can identify any leftover data before running the setup.

The idea is that a transaction is initialized before the test begins. In order for the data to be saved in the database, the transaction must be commited, but it is not commited never. It works in case when test a stop, in the case of a successful or unsuccessful completion of the test.
In integration tests we use somthing like this (with NUnit)(It real production code)
public class RollbackAttribute : TestAttribute, ITestAction
{
private TransactionScope _transaction;
public void BeforeTest(ITest test)
{
_transaction = new TransactionScope();
}
public void AfterTest(ITest test)
{
_transaction.Dispose();
}
public ActionTargets Targets => ActionTargets.Test;
}
[TestFixture]
public class SomeTestClass
{
[Rollback] //No need [Test] because Rollback is inherit it.
public void SomeTestMethod()
{
}
}
On MsTest you can make somthing similar, but in this case you should inherit from base class, I hope it works. For example:
public class RollbackTestBase
{
private TransactionScope _transaction;
[TestInitialize]
public virtual void Setup()
{
_transaction = new TransactionScope();
}
[TestCleanup]
public virtual void TearDown()
{
_transaction.Dispose();
}
}
[TestClass]
public class IntegrationTest : RollbackTestBase
{
[TestMethod]
public void TestDataBase()
{
Assert.IsTrue(true);
}
[TestInitialize]
public virtual void Init()
{
}
[TestCleanup]
public virtual void CleanUp()
{
}
}

There are 2 cases which we need to consider while allocating resources in ATPs (Resources might be Creating users, Connection with database). They are
Creation and deletion of resources after each test.
Creation and deletion of resources after set of tests.
Creation and deletion of resources after each test:
If we want to create instance of particular object before execution of a test and want to clean up memory allocated to that object after execution of that test, then we use Test SetUp and Test TearDown attributes of NUnit. In your case object is creation of number of Users.
[SetUp] : Function which is decorated with Test SetUp attribute contains piece of code that executes before the execution of any test.
[TearDown] : Function which is decorated with Test TearDown attributes contains piece of code that executes after execution of any test
Implementation:
[TestClass]
public class UnitTest1
{
[SetUp]
public void SetUP()
{
// Creating Users with proper credentials
}
[TestMethod]
public void TestMethod1()
{
//Write your ATP
}
[TearDown]
public void TearDown()
{
//Clean up
}
}
Creation and deletion of resources after set of tests:
Now If we want to create instance of an object for set of tests and want to clean up memory after execution of all tests then [TestFixtureSetUp] and [TestFixureTearDown] to initialize an object and to clean up memory respectively. Again In your case object can be creation of set of users.
[TestFixtureSetUp] : Function decorated with TestFixtureSetUp will executes once before the execution of group of tests.
[TestFixtureTearDown] : Function decorated with TestFixtureTearDown will executes once after the execution of group of tests.
Implementation
[TestFixture]
public class Tests
{
[TestFixtureSetUp]
public void Setup()
{
//Create users with credentials
}
[Test]
public void _Test1()
{
//Test_1
}
[Test]
public void _Test2()
{
//Test2
}
[TestFixtureTearDown]
public void CleanUp()
{
//Cleanup; Here you need to add code to Delete all users
}
}
Note: I will suggest you, if you are trying to create and delete users for particular ATP then go with SetUp and TearDown. If you are trying same for bunch of ATPs, I would recommend you to go with TestFixtureSetUp and TestFixtureTearDown.
"If your test get pass or fail, SetUp and TearDown functions will execute"
References:
#Shuvra's Answer.
Nunit: SetUp, TearDown, SetUpFixture, TearDownFixture

I think you should open a transaction before you test, create the data and finish test test. But do not commit the transaction. That will ensure that the test can't affect your DB at all.
Update:
The easier approach is use docker container.
You can run a container from your image and remove that container after the test is done. This should definitely reduce the complexity of your test.

Visual studio uses NUNIT therefore, you can use TearDownAttribute. It should run after the test, even if the test is canceled. You can write a function to clean your data.
Please read the reference documentation here: http://nunit.org/docs/2.2/teardown.html

Just to clear more about the NUNIT standrad. Please follow the steps in the Test class:
[TestFixture]
public class _TestClass
{
[TestFixtureSetUp]
public void Setup()
{
//Clearup can be here before start of the tests. But not Recommended
}
[Test]
public void _Test1()
{
}
[Test]
public void _Test2()
{
}
[TestFixtureTearDown]
public void CleanUp()
{
//I will recommend to clean up after all the tests complete
}
}
Reference: http://nunit.org/docs/2.5/fixtureTeardown.html

A better solution to the problem to use what is called "database mocking". In this case you would have your tests run with a different database (or a fake, virtual one).
This article explains how to implement it in C#
https://msdn.microsoft.com/en-us/library/ff650441.aspx

You should begin a transaction and not commit your records to the DB. Thus, all your changes will be automatically rollbacked when the session is over.

How to unit test methods that use threads?

How to write unit tests for methods that use threads.
In example below how to test someMethod method ?
public class SomeClass {
private final SomeOtherClass someOtherClassInstance = IoC.getInstance(SomeOtherClass.class);
private final ExecutorService executorService = Executors.newCachedThreadPool();
public void someMethod() {
executorService.execute(new Runnable() {
#Override
public void run() {
someOtherClassInstance.someOtherMethod();
}
});
}
}
Are there any solutions in java and .net for this purpose?

Your question begs 'what is a unit test' (UT). UTs are just one form of automated tests. In this case your question implies that the code calls the OS to start a thread, so by definition, a test that tested it would not be a unit test but probably be an integration test.
Why I bother you with what seems like semantics is that understanding the intent of the test makes it so easy to write the test and structure the code. The amount of working code out there is larger than the amount of testable code out there.
So, how can this code be changed to be unit testable (I will skip the 'why bother stuff') ...
A unit test in C# tests a single type (your class) ... importantly (by definition) nothing else. So your code needs to reflect this. What you want to test is that when I call ABC it does this stuff. Something that stuff includes launching a thread. So you want to test that the method to launch a thread a called. The fundamental here is that your application requires a thread so that is what your asserting.
So how? Create a proxy, and perhaps a factory, for the thread creation. Then you can assert in a unit test that it was called and how it was treated. Sounds hard, but is really easy when you get into the habit.
BTW, Resharper makes creating a proxy for an OS types (e.g. thread). Check out the delegating members stuff in their help. If your not using Resharper you should be.
Yea I know this is not the answer you had hopped for, but believe me it is the answer you need.
I find it useful to think of tests in the categories:
Unit tests
Integration tests (or perhaps subsystem tests)
Functional test (e.g. UI/application/UAT tests)

The jMock Team has described an brilliant approach for Java in the 'jMock Cookbook: Test Multithreaded Code'. The concept of executing "thread" functionality on the same thread as the test can also be applied to C# code (see this blog post). The C# test would look like this:
[Test]
public void TestMultiThreadingCodeSynchronously()
{
// Arrange
SomeOtherClass someOtherClassMock = MockRepository.GenerateMock<SomeOtherClass>();
DeterministicTaskScheduler taskScheduler = new DeterministicTaskScheduler();
SomeClass systemUnderTest = new SomeClass(taskScheduler, someOtherClassMock);
// Act
systemUnderTest.SomeMethod();
// Now execute the new task on
// the current thread.
taskScheduler.RunTasksUntilIdle();
// Assert
someOtherClassMock.AssertWasCalled(x=>x.SomeOtherMethod());
}
Your "system under test" would look like this:
public class SomeClass
{
private readonly TaskScheduler taskScheduler;
private readonly SomeOtherClass instance;
public SomeClass(
TaskScheduler taskScheduler,
SomeOtherClass instance)
{
this.taskScheduler = taskScheduler;
this.instance = instance;
}
public void SomeMethod()
{
Task.Factory.StartNew(
instance.SomeOtherMethod,
new CancellationToken(),
TaskCreationOptions.None,
taskScheduler);
}
}
The C# solution is described in detail in this blog post.

If there is another thread, you have wait until the method has been called. You have to decide how long you should wait before considering the method wasn't called, there is no tool which can do that for you.
Of course you can mock the ExecutorService to run in the current thread and avoid the issue.