How to capture SIGTERM in Linux .NET Hosted service - c#

I have a process which I'm porting from Windows to Linux, and I need it to restart relatively promptly when I call systemctl restart service. Unfortunately, it hangs for the 90 second timeout which is unacceptable.
I've tried several things without effect. Notably, I have added a handler for Domain.CurrentDomain.ProcessExit in my Program.cs file, but that handler never gets hit when I send that command, or any other as far as I can tell. I have reviewed the answers for this question, this question, and this question. I am using IHostBuilder to do this, but I am not in a container. I'm running on Debian 11. I have a background service which implements ExecuteAsync, but it has never served to shut down the service. Even running it from the console and using Ctrl-C doesn't result in the signal handler being called (and in fact doesn't actually terminate the process, even giving it the usual 90 second delay). What am I missing?
This is my Main function and its helpers. When I manually set kill to true via debugger, the program exits (albeit ungracefully). Letting the system run without the exception results in the same hanging behavior. Clearly one of the many threads is hanging onto resources, but they should be guarded with cancellation tokens, which as previously stated are not working.
public static async Task Main(string[] args)
{
IHost builder = CreateHostBuilder(args).Build();
bool kill = false;
void OnCurrentDomainOnProcessExit(object sender, EventArgs args)
{
kill = true;
}
AppDomain.CurrentDomain.ProcessExit +=
OnCurrentDomainOnProcessExit;
await builder.StartAsync();
while (!kill)
{
Thread.Sleep(1000);
}
await builder.StopAsync(TimeSpan.FromSeconds(10));
throw new Exception("Received SIGTERM");
}
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.UseSystemd()
.UseConsoleLifetime()
.ConfigureServices(ConfigureServices);
private static void ConfigureServices(HostBuilderContext _, IServiceCollection services)
{
services.AddLogging();
services.AddSingleton<ITheSingleton, TheSingleton>();
services.AddHostedService<ProcessWorker>();
}
Here's my simple implementation of a BackgroundService:
public class ProcessWorker: BackgroundService
{
ITheProcess _process;
public ProcessWorker(ITheProcess process)
{
_process = process;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
_logger.LogInformation("Worker running at: {time}", DateTimeOffset.Now);
await Task.Delay(1000, stoppingToken);
}
}

Related

Why I can stop an IHostedService when therre are background threads running?

Below is my simple code that runs an IHostedService
internal class Program {
public static Task Main(string[] args) {
var host = new HostBuilder().ConfigureServices((hostcontext, services) => {
services.AddHostedService<MyService>();
}).Build();
host.Run();
return Task.CompletedTask;
}
}
public class MyService : IHostedService {
public Task StartAsync(CancellationToken cancellationToken) {
return Task.Run(() => {
while (true) {
Console.WriteLine("Starting Service");
}
});
}
public Task StopAsync(CancellationToken cancellationToken) {
Console.WriteLine("Stopping service");
return Task.CompletedTask;
}
}
So when I want to stop the service by pressing ctrl + C in the console, I exepct to see the service stops and console prints "Stopping service".
But when I press Ctrl + C, the service continues running in the infinite loop, which I don't understand.
I think Task.Run() queues a work item on the thread pool, then a background thread from thread pool to pick up the job, so in my example, it is a worker thread (worker thread's id is 4, main thread's id is 1) that executes the while loop. So when I press ctrl + S, the service should stop, then why the running background thread stops the service being stopped, isn't that when a application terminals, all background jobs/ threads terminates too? I mean if Task.Run() runs creates a foreground threads then I can understand, because all foreground threads need to finish before the applciation could be stopped.
P.S:
I can pass the CancellationTokento stop the while loop, I understand I can do that and in the while loop, I check if the token is cancalled etc...
but I don't understand why I have to do that, because the running thread is a background thread, not a foreground thread, so why all background threads need to finish first the the StopAsync() can be invoked? i.e how does a running background thread stops the exection flow reach to StopAsync()?
If you want to stop a Task try using a cancellation token in MyServiceClass
Here is my example:
public class MyService : IHostedService
{
private CancellationTokenSource _ts;
public Task StartAsync(CancellationToken cancellationToken)
{
_ts = new CancellationTokenSource();
return Task.Factory.StartNew(() =>
{
while (true)
{
Console.WriteLine("Starting Service");
Thread.Sleep(500);
if (_ts.Token.IsCancellationRequested)
{
Console.WriteLine("Stopping service");
break;
}
}
}, _ts.Token);
}
public Task StopAsync(CancellationToken cancellationToken)
{
_ts.Cancel();
return Task.CompletedTask;
}
}
The reason why Ctrl+C doesn't outright terminate your console application, like you might be used to from "ordinary" console applications, is that there's actually support in .NET to prevent Ctrl+C from terminating the application, and for the application to react to Ctrl+C and "do stuff".
The hosting framework you're using has used this system, an event, to prevent your application from being forcibly terminated as well as take over shutdown procedure.
The event itself is Console.CancelKeyPress and this is a cancellable event meaning that you can return from the event handler and set a flag on the EventArgs object to signal to the code that invoked your event handler that you want to opt out of default handling of Ctrl+C.
The hosting framework has done this.
You can see the exact code here : Microsoft.Extensions.Hosting/Internal/ConsoleLifetime.cs #48-52:
Console.CancelKeyPress += (sender, e) =>
{
e.Cancel = true;
ApplicationLifetime.StopApplication();
};
In addition to cancelling the normal shutdown procedure, the event handler provided by the hosting framework also cancels a CancellationToken that is the token passed to StartAsync. As you say, if you pass this token to your tasks and reacts to it being cancelled, your application shuts down.
The call to ApplicationLifetime.StopApplication eventually ends up in Microsoft.Extensions.Hosting/Internal/ApplicationLifetime.cs #102-112:
private void ExecuteHandlers(CancellationTokenSource cancel)
{
// Noop if this is already cancelled
if (cancel.IsCancellationRequested)
{
return;
}
// Run the cancellation token callbacks
cancel.Cancel(throwOnFirstException: false);
}
So there you have it. The reason your application keeps spinning in an infinite loop is precisely because the hosting framework prevents ordinary shutdown, and instead give your tasks a chance to complete in an orderly fashion. Since your tasks ignore this request, and just keeps running, your process never terminates when you hit Ctrl+C.

How should a Task signal a Windows service shutdown correctly?

I'm working on a bug for a Windows service. The service uses a field to track a reference to a single Task. OnStart, the task executes a single method. The method has a loop inside it and calls a database on a configurable interval to monitor the work another system is doing.
protected override void OnStart(string[] args)
{
_processorTask = Task.Run(() => StartProcessor());
}
We've occasionally had an issue where the Task dies and logs the exception just fine, but the existing plumbing wasn't telling the service to stop, so our service monitors didn't know anything was wrong.
At first, I tried to add just add a call to Stop().
private void StartProcessor()
{
var processor = new PEMonitoringProcessor(_tokenSource.Token);
try
{
// The process loop is in the function. If this method exits, good or bad, the service should stop.
processor.ProcessRun();
}
catch (Exception ex)
{
// An exception caught here is most likely fatal. Log the ex and start the service shutdown.
if (log.IsFatalEnabled) { log.Fatal("A fatal error has occurred.", ex); };
}
finally
{
Stop();
}
}
However, one of my fellow devs noticed in the OnStop method that a token is used to signal the Task to stop and then it waits. If the Task calls Stop and waits for Stop to return and OnStop is waiting for the Task to end, that does not bode well for this code.
protected override void OnStop()
{
_tokenSource.Cancel();
try
{
_processorTask.Wait();
}
// logging & clean-up...
}
I considered a separate Task that doesn't get waited on by OnStop that checks the status of the first and calls Stop if the first Task is Completed, Faulted, etc., but that seemed a touch weird. I also considered raising an Event and trying something like BeginInvoke.
Intentionally stopping the service works fine because the OnStop signals via Token that a shutdown is happening. I'm trying to cover the possibility that the Task method returns or throws unexpectedly and I want the service to stop instead of becoming a zombie.
The most straightforward way I see is something like this:
protected override void OnStart(string[] args) {
_processorTask = Task.Run(() => StartProcessor());
_processorTask.ContinueWith(x => {
// x.Exception contains exception if any, maybe log it here
Stop();
}, TaskContinuationOptions.NotOnCanceled);
}
protected override void OnStop() {
//or !_processorTask.IsCompleted && !_processorTask.IsCanceled && !_processorTask.IsFaulted
if (_processorTask.Status == TaskStatus.Running) {
// only cancel and wait if still running. Won't be the case if service is stopping from ContinueWith above
_tokenSource.Cancel();
_processorTask.Wait();
}
}
Alternative way to do the same:
protected override async void OnStart(string[] args) {
_processorTask = Task.Run(() => StartProcessor());
bool cancelled = false;
try {
await _processorTask;
}
catch (OperationCanceledException) {
// cancelled
cancelled = true;
}
catch (Exception ex) {
// log it?
}
if (!cancelled)
Stop();
}
// OnStop stays the same

How to make 2 tasks run simultaneously in a UWP background app?

I need help running my UWP background app deployed on my Raspberry Pi (hosting Windows 10 IOT Core).
It's almost working but I got a tricky issue... Let me explain:
The app has 2 functionalities:
A Web Server that receives http requests and do some processing according to parameters. (based on this link)
A repeating task that is executed every X minutes (X varies from 10 to 60).
Both functionalities work well if executed seperately but I need them to run simultaneously. I'm rather new to UWP & Raspberry so maybe there's something I'm missing...
Is it a matter of thread conflict?
Is a background task on a raspberry limited to a single thread (so the first/last process executed wins)?
Do all classes need to be "sealed" (microsofts's doc says so but not the example they provide)?
I tought about splitting the project into two independant deployable tasks and I believe it would work but I need the webserver task to control (START/PAUSE/STOP) the repeating task.
I think communication between them could be feasable (using this way) but I'm looking for the simplest/fastest solution here.
Here's the stripped down code (the 3 classes are in separate files):
App class:
public sealed class StartupTask : IBackgroundTask
{
private static BackgroundTaskDeferral _Deferral = null;
public async void Run(IBackgroundTaskInstance taskInstance)
{
_Deferral = taskInstance.GetDeferral();
var webserver = new WebServer();
await ThreadPool.RunAsync(workItem => { webserver.Start(); });
StartRepeatingTask();
}
}
Web server class:
internal class WebServer
{
private const uint BUFFER_SIZE = 8192;
public async void Start()
{
var listener = new StreamSocketListener();
await listener.BindServiceNameAsync("1537");
listener.ConnectionReceived += async (sender, args) =>
{
// Do some stuff
...
Processing.StopProcess();
}
}
}
Processing class
public class Processing
{
private static Task myTask = null;
private static CancellationTokenSource taskCancellation = new CancellationTokenSource();
public static void StartRepeatingTask()
{
taskCancellation = new CancellationTokenSource();
myTask = Task.Run(() => AutoProcess(), taskCancellation.Token);
}
public static void AutoProcess()
{
Process();
myTask = Task.Delay(GetDelayToUse(DELAY_SET_LIST))
.ContinueWith(t => AutoProcess(), taskCancellation.Token);
}
public static void Process()
{
// Do some other stuff
}
public static void StopProcess()
{
taskCancellation.Cancel();
}
}
The two task can run in a background app simultaneously.
Is it a matter of thread conflict?
If there are no commitications or shared data between the two task, there will be no confilct. The important thing is that you need to pay attention to the synchronization of data. If you use a list or queue to share the data between the two threads, you should use the concurrency object. Please refer to ConcurrentQueue example, ConcurrentQueue represents a thread-safe.
Is a background task on a raspberry limited to a single thread (so the
first/last process executed wins)?
Once a background applications are deployed and configured, these applications launch at machine startup and run continuously without any process lifetime management resource use limitations. There are some limitations about background task, but it is not limited to a single thread. More information here. Here you need to differentiate the background application on Windows IoT Core and task.
Do all classes need to be "sealed" (microsofts's doc says so but
not the example they provide)?
Yes, the background task class itself—and all other classes in the background task project—need to be public classes that are sealed (or final). It applies to the background tasks which inherited from interface IBackgroundTask, but not the classes are using for normal thread.

console.writeline within Task does not work

I am learning task aysny based programing and cannot get to make this code work. The console prints the message only once and then disappears.
if I remove the read line and run the program(not debug mode) the console just appears with message saying press a key to continue. When I debug and put the debugger in console.write then it works fine for some time and then the console window disappears and restarts again. If I use for loop <10000 instead of while then also the behaviors is same
Could you please suggest what I am doing wrong.
static void Main(string[] args)
{
multitasker();
}
static async void multitasker()
{
Task task1 = new Task(PrintMessageA);
task1.Start();
await task1;
}
static void PrintMessageA()
{
while(true)
{
Console.WriteLine("Message from A");
Console.ReadLine();
}
}
Your main thread does not block and thus is exiting immediately. You would have to go "await all the way" in a sense and await multitasker as well, but you can't actually do that as seen later.
So first you return a task in multitasker
static async Task multitasker()
{
Task task1 = new Task(PrintMessageA);
task1.Start();
await task1;
}
The problem is you cannot make Main() (the entry point) async, so instead you would need to block that thread by instead calling Wait() on the returned Task
static void Main(string[] args)
{
multitasker().Wait();
}

HttpClient.PostAsync knocks out the app with exit code 0

Everything was working today until it stopped... Below is the minimum source code (I'm using VS 2012 Update 1, .Net 4.5). When I run it, app exits upon calling client.PostAsync() and so it never reaches Console.ReadLine(). Same in debugger, no exception, nothing, exit code 0.
I tried rebooting machine, restarting VS2012 - nothing works.
Again, everything was running today, not sure what changed (no software has been installed etc, all other network apps still work).
Any ideas? I think I'm loosing my mind.
class Program
{
static void Main(string[] args)
{
Run();
}
private async static void Run()
{
using (var client = new System.Net.Http.HttpClient())
{
var headers = new List<KeyValuePair<string, string>>
{
new KeyValuePair<string, string>("submit.x", "48"),
new KeyValuePair<string, string>("submit.y", "15"),
new KeyValuePair<string, string>("submit", "login")
};
var content = new FormUrlEncodedContent(headers);
HttpResponseMessage response = await client.PostAsync("http://www.google.com/", content);
Console.ReadLine();
}
}
}
Your problem is that a program normally exits when its Main() method finishes. And your Main() finishes as soon as you hit the await in Run(), because that's how async methods work.
What you should do is to make Run() into an async Task method and then wait for the Task in your Main() method:
static void Main()
{
RunAsync().Wait();
}
private static async Task RunAsync()
{
…
}
In C# 7.1+ you should use async Main instead:
static async Task Main()
{
await RunAsync();
}
private static async Task RunAsync()
{
…
}
Few more notes:
You should never use async void methods, unless you have to (which is the case of async event handlers).
Mixing await and Wait() in a GUI application or in ASP.NET is dangerous, because it leads to deadlocks. But it's the right solution if you want to use async in a console application.

Categories

Resources