Develop Reference

Multi Thread Worker Service in .Net Core

I'm trying build a worker service on Core 5.0. My tree is basically like that =>
1 -) Program.cs 2-) Worker.cs 3-) MyStartUp.cs 4-) Client.cs
In MyStartUp.cs I am getting a list and calling Client class some servers according to list.
In the Client class, I connect to the devices and write the data I read to the database.
Device count nearly 1200, server way is TCP/IP.
What is your best suggestion for write a worker service like that?
How can I use threads in it best form?
Below is my first try. This form is working but it's so slow for 1000 different client because there is so much reads in client.
public class Worker : BackgroundService
{
private readonly ILogger<Worker> _logger;
public Worker(ILogger<Worker> logger)
{
_logger = logger;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
StartUp startUp = new StartUp();
}
}
public class StartUp
{
public StartUp()
{
//... get client datas and initialize client object
StartClients();
}
public void StartClients()
{
foreach (var item in ClientList)
{
new Thread(item.Run).Start();
}
}
}
public class Client
{
System.Timers.Timer timer ;
public Client()
{
timer = new Timer();
timer.Interval = 100;
timer.Elapsed += Timer_Elapsed;
//... initialize client connection and database
}
public void Run()
{
timer.Start();
}
private void Timer_Elapsed(object sender, ElapsedEventArgs e)
{
//... write values of read client to database
}
}

Say that you have 1k timers that run every 100ms, and say that each timer tick takes 50ms to execute. That means each timer needs 500ms/s, or 50% of one core, and you would need 500 cores to keep up with the work. You probably do not have that many cores, nor IO to process the requests, and that means the work will start piling up and your computer will more or less freeze since it does not have time to update the UI.
50ms might be an overestimation of the time used, but even at 5ms you would probably have issues unless you are running this on a monster server.
The solution would be to decrease the polling frequency to something more reasonable, say every 100s instead of every 100ms. Or to have one or more threads that polls your devices as fast as they can. For example something like:
private BlockingCollection<MyClient> clients = new ();
private List<Task> workers = new ();
public void StartWorker(){
workers.Add(Task.Run(Run));
void Run(){
foreach(var client in clients.GetConsumingEnumerable()){
// Do processing
clients.Add(client); // re-add client to queue
}
}
}
public void CloseAllWorkers(){
clients.CompleteAdding();
Task.WhenAll(workers).Wait();
}
I would note that usages of Thread is mostly superseded by tasks. And that creating a thread just to start a System.Timers.Timer is completely useless since the timer will run the tick event on the threadpool, regardless of the thread that started it. At least unless a synchronization object was specified.

Consumer Producer- Producer thread never executes assigned function

I have .NET Core Web API solution. In each call, I need to perform some database operations. The issue is at a time multiple db connections get opened & close. So to avoid it, I want to implement Queue of objects to be sent to database and then want a separate thread to perform db operation.
I've tried some code as below. But here, Consumer thread never executes assigned function. There is no separate thread for Producer, I am simply feeding queue with object.
What modifications I should do? Need some guidance as I'm new to Threading stuff.
public static class BlockingQueue
{
public static Queue<WebServiceLogModel> queue;
static BlockingQueue()
{
queue = new Queue<WebServiceLogModel>();
}
public static object Dequeue()
{
lock (queue)
{
while (queue.Count == 0)
{
Monitor.Wait(queue);
}
return queue.Dequeue();
}
}
public static void Enqueue(WebServiceLogModel webServiceLog)
{
lock (queue)
{
queue.Enqueue(webServiceLog);
Monitor.Pulse(queue);
}
}
public static void ConsumerThread(IConfiguration configuration)
{
WebServiceLogModel webServiceLog = (WebServiceLogModel)Dequeue();
webServiceLog.SaveWebServiceLog(configuration);
}
public static void ProducerThread(WebServiceLogModel webServiceLog)
{
Enqueue(webServiceLog);
Thread.Sleep(100);
}
}
I've created and started thread in StartUp.cs:
public Startup(IConfiguration configuration)
{
Thread t = new Thread(() => BlockingQueue.ConsumerThread(configuration));
t.Start();
}
In Controller, I've written code to feed the queue:
[HttpGet]
[Route("abc")]
public IActionResult GetData()
{
BlockingQueue.ProducerThread(logModel);
return StatusCode(HttpContext.Response.StatusCode = (int)HttpStatusCode.NotFound, ApplicationConstants.Message.NoBatchHistoryInfo);
}

First of all, try to avoid static classes and methods. Use pattern singleton in that case (and if you really need this).
Second, try to avoid lock, Monitor - those concurrency primitives significantly lower your performance.
In such situation, you can use BlockingCollection<> as 'Adam G' mentioned above, or you can develop your own solution.
public class Service : IDisposable
{
private readonly BlockingCollection<WebServiceLogModel> _packets =
new BlockingCollection<WebServiceLogModel>();
private Task _task;
private volatile bool _active;
private static readonly TimeSpan WaitTimeout = TimeSpan.FromSeconds(1);
public Service()
{
_active = true;
_task = ExecTaskInternal();
}
public void Enqueue(WebServiceLogModel model)
{
_packets.Add(model);
}
public void Dispose()
{
_active = false;
}
private async Task ExecTaskInternal()
{
while (_active)
{
if (_packets.TryTake(out WebServiceLogModel model))
{
// TODO: whatever you need
}
else
{
await Task.Delay(WaitTimeout);
}
}
}
}
public class MyController : Controller
{
[HttpGet]
[Route("abc")]
public IActionResult GetData([FromServices] Service service)
{
// receive model form somewhere
WebServiceLogModel model = FetchModel();
// enqueue model
service.Enqueue(model);
// TODO: return what you need
}
}
And in Startup:
public class Startup
{
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<Service>();
// TODO: other init staffs
}
}
You even can add Start/Stop methods to the service instead of implementing IDisposable and start your service in the startup class in the method Configure(IApplicationBuilder app).

I think your consumer thread is executed just once if there is something in the queue and then immediately returns. If you want to have a thread doing work in background, which is started just once, it should never return and should catch all exceptions. Your thread from BlockingQueue.ConsumerThread is invoked once in Stratup and returns.
Also please be aware that doing such solution is not safe. ASP.NET doesn't guarantee background threads to be running if there are no requests coming in. Your application pool can recycle (and by default it recycles after 20 minutes of inactivity or every 27 hours), so there is a chance that your background code won't be executed for some queue items.
Also, while it doesn't solve all issues, I would suggest using https://www.hangfire.io/ to do background tasks in ASP.NET server. It has persistence layer, can retry jobs and has simple API's. In your request handler you can push new jobs to Hangfire and then have just 1 job processor thread.

frequency of data logging in C#

Dear users & developers,
I have a C# code that writes a variable to a file everytime it changes in value. I want to change it so that it can write every 1 minute (another input to be specified by the user) or so from the time it starts. I am new to data logging, any help/insights regarding the issue would be greatly appreciated.
Thanks
AK

My first tought is to maybe use a System.Threading.Tasks.Task running in the background:
// This is a very crude implementation, but it should serve its purpose.
using System.Threading.Tasks;
public class App
{
public bool isAppActive = true;
public static void Main()
{
Task mytsk = MyLog();
Console.ReadKey();
isAppActive = false;
}
public static async void MyLog()
{
while (App.isAppActive)
{
Thread.Sleep(60000);
// Write your variable to a file (using an async method and await)
}
}
}
... Or maybe using a BackgroundWorker.

Execute function every 5 minutes in background

I have function which reads Data out of an Webservice. With that Data i create Bitmaps. I send the Bitmaps to Panels (Displays) which displays the created Bitmaps. Manually its working like charm. What i need now is, that my Application run this function every 5 min automtically in the Backround.
My Application is running under IIS. How can i do that? Can someone help me with that?

You don't have to be depended on asp.net project, but you can use Cache Callback to do it.
I have found a nice approach, to do it.
actually i don't remember the link so i'll give you a code that i use:
public abstract class Job
{
protected Job()
{
Run();
}
protected abstract void Execute();
protected abstract TimeSpan Interval { get; }
private void Callback(string key, object value, CacheItemRemovedReason reason)
{
if (reason == CacheItemRemovedReason.Expired)
{
Execute();
Run();
}
}
protected void Run()
{
HttpRuntime.Cache.Add(GetType().ToString(), this, null,
Cache.NoAbsoluteExpiration, Interval, CacheItemPriority.Normal, Callback);
}
}
Here is the implementation
public class EmailJob : Job
{
protected override void Execute()
{
// TODO: send email to whole users that are registered
}
protected override TimeSpan Interval
{
get { return new TimeSpan(0, 10, 0); }
}
}

An Asp.Net application is not the correct framework for a task like this.
You should probably create a dedicated service for this type of tasks.
Another option is to create a scheduled task that will run every X minutes
On a side note, if you must do this through your asp.net application, I recommend reading on how to Simulate a Windows Service using ASP.NET to run scheduled jobs

How to Create a C# Listener Service for MSMQ as a Windows Service

I'll start by saying I'm not a .NET developer, but have been thrown into a project where I need to use MSMQ so a classic ASP web application can send messages to a C# Windows Service that handles the processing. I have experience integrating other message queues with other languages, but like I mentioned, I don't have much experience with .NET and Windows development so some guidance would be much appreciated.
Here are my questions...
Could someone provide some basic C# code that listens to an existing MSMQ queue and responds to the new message by doing something simple like writing the current timestamp to a log file or sending an email?
How do I package this code up in Visual Studio .NET to create and install a Windows Service? (What type of project should it be, etc. I'm using Visual C# 2010 Express.)
Finally, I'm not sure which version and/or implementation of MSMQ I need to be using for my requirements with classic ASP. I think the COM version is what I need, but I've also read about a new WCF version, as well as differences between 3.0 and 4.0. Could someone please give me direction on which version I should be using?
Many thanks!

You can wait for a message on a given queue using the following code (You want to use the private queue named SomeQueue on your computer, named ComputerName => QueueName = #"ComputerName\private$\SomeQueue")
public void AsyncWatchQueue(object encapsulatedQueueName)
{
Message newMessage;
MessageQueue queue;
string queueName = encapsulatedQueueName as string;
if (queueName == null)
return;
try
{
if (!MessageQueue.Exists(queueName))
MessageQueue.Create(queueName);
else
{
queue = new MessageQueue(queueName);
if (queue.CanRead)
newMessage = queue.Receive();
}
HandleNewMessage(newMessage); // Do something with the message
}
// This exception is raised when the Abort method
// (in the thread's instance) is called
catch (ThreadAbortException e)
{
//Do thread shutdown
}
finally
{
queue.Dispose();
}
}
Note: the Receove method will block untill a message is received at which point it'll remove the message from the queue and return it.
edit: added code for the implementation of the multithreaded portion (and renamed the above method signature)
Thread Creation Code:
public Thread AddWatchingThread(string QueueName)
{
Thread Watcher =
new Thread(new ParameterizedThreadStart(AsyncWatchQueue));
Watcher.Start(QueueName);
// The thread instance is used to manipulate (or shutdown the thread)
return Watcher;
}
I'll just note, that this is untested cod, it's just an quick example

As far as I know, Visual Studio Express does not have a project template for a service. That does not mean you cannot write a windows service with VSE, just that you will not have a template to get you started.
To create a service you can just create a normal Console application. Create the service class which will be responsible for the actual service implementation. It will look something like this
using System.ServiceProcess;
namespace WindowsService1
{
public partial class Service1 : ServiceBase
{
public Service1()
{
InitializeComponent();
}
protected override void OnStart(string[] args)
{
}
protected override void OnStop()
{
}
}
}
Then the Service startup code can go into the Main function of your service.
using System.ServiceProcess;
namespace WindowsService1
{
static class Program
{
static void Main()
{
ServiceBase[] ServicesToRun;
ServicesToRun = new ServiceBase[]
{
new Service1()
};
ServiceBase.Run(ServicesToRun);
}
}
}
That should give you the basic framework for your service. The other thing you will need is to add an installer for the service so that it can be installed as a service. The following should get you started, note I have note tested this.
using System.ComponentModel;
using System.Configuration.Install;
using System.ServiceProcess;
namespace WindowsService1
{
[RunInstaller(true)]
public class ProjectInstaller : Installer
{
private ServiceProcessInstaller serviceProcessInstaller1;
private ServiceInstaller serviceInstaller1;
public ProjectInstaller()
{
this.serviceProcessInstaller1 = new ServiceProcessInstaller();
this.serviceInstaller1 = new ServiceInstaller();
this.serviceProcessInstaller1.Password = null;
this.serviceProcessInstaller1.Username = null;
this.serviceInstaller1.ServiceName = "Service1";
this.Installers.AddRange(new System.Configuration.Install.Installer[] {
this.serviceProcessInstaller1,
this.serviceInstaller1});
}
}
}
Given the above, you should have enough to search around or ask for more details around the service creation. As for the MSMQ listener, you can use the following MSDN article as a starting point
http://msdn.microsoft.com/en-us/library/ms978425.aspx