I'm newbie to use threading logic.
I declared ThreadPool.SetMaxThreads(10, 10) and create multiple thread same number of 10.
First request working well, I requested 2 more each different browser. following request fallen hang until finish first request's thread work.
Does ThreadPool.SetMaxThreads affected entire IIS application pool?
public ActionResult Index()
{
ThreadPool.SetMaxThreads(10, 10);
for (int i = 0; i < 10; i++)
{
Task.Factory.StartNew(() =>
{
try
{
Thread.Sleep(30000);
}
finally
{
}
});
}
return View();
}
Here's new code using semaphoreslim.
My Actual goal is only specified count of threads run at once.
for example I will download 9 files and assign each thread 1 download job, max 3 threads work.
public ActionResult Index()
{
int maxConcurrency = 3;
using (SemaphoreSlim concurrencySemaphore = new SemaphoreSlim(maxConcurrency))
{
List<Task> tasks = new List<Task>();
for (int i = 0; i < 9; i++)
{
concurrencySemaphore.Wait();
var t = Task.Factory.StartNew(() =>
{
try
{
// Here to actual my job logic.
Thread.Sleep(10000);
}
finally
{
concurrencySemaphore.Release();
}
});
tasks.Add(t);
}
Task.WaitAll(tasks.ToArray());
}
return View();
}
Related
Let's say I have 10 tasks which should run simultaneously visualizing their progress using GUI. For simplicity let them be just counters 1 to 100. Main app should be console application. GUI controls should be generated on the fly.
Here's the code I've written:
static void Main(string[] args)
{
Enumerable.Range(0, 10).ToList().ForEach(x =>
frm.Controls.Add(new Label()
{
Left = 10,
Top = 22 * x,
}));
Thread mThread = new Thread(delegate ()
{
frm.ShowDialog();
});
mThread.SetApartmentState(ApartmentState.STA);
mThread.Start();
Jobs();
}
static void Jobs()
{
Task[] tasks = new Task[10];
for (int i = 0; i < tasks.Length; i++)
{
tasks[i] = PayLoad(i);
}
Task.WaitAll(tasks);
}
static async Task PayLoad(int taskNumber)
{
Console.WriteLine($"Task {taskNumber} is starting...");
//await Task.Delay(1000);
for (int i = 1; i < 100; i++)
{
Console.WriteLine($"Task {taskNumber} - iteration {i} ");
Action act = delegate () { frm.Controls[taskNumber].Text = i.ToString(); };
frm.BeginInvoke(act);
await Task.Delay(1000);
}
}
The problem is the progress stops when all counters reach 1. If I comment the 2nd await the code completes but I can see it works synchronously. All the labels are filled one by one.
Am I missing something?
I have this function which checks for proxy servers and currently it checks only a number of threads and waits for all to finish until the next set is starting. Is it possible to start a new thread as soon as one is finished from the maximum allowed?
for (int i = 0; i < listProxies.Count(); i+=nThreadsNum)
{
for (nCurrentThread = 0; nCurrentThread < nThreadsNum; nCurrentThread++)
{
if (nCurrentThread < nThreadsNum)
{
string strProxyIP = listProxies[i + nCurrentThread].sIPAddress;
int nPort = listProxies[i + nCurrentThread].nPort;
tasks.Add(Task.Factory.StartNew<ProxyAddress>(() => CheckProxyServer(strProxyIP, nPort, nCurrentThread)));
}
}
Task.WaitAll(tasks.ToArray());
foreach (var tsk in tasks)
{
ProxyAddress result = tsk.Result;
UpdateProxyDBRecord(result.sIPAddress, result.bOnlineStatus);
}
tasks.Clear();
}
This seems much more simple:
int numberProcessed = 0;
Parallel.ForEach(listProxies,
new ParallelOptions { MaxDegreeOfParallelism = nThreadsNum },
(p)=> {
var result = CheckProxyServer(p.sIPAddress, s.nPort, Thread.CurrentThread.ManagedThreadId);
UpdateProxyDBRecord(result.sIPAddress, result.bOnlineStatus);
Interlocked.Increment(numberProcessed);
});
With slots:
var obj = new Object();
var slots = new List<int>();
Parallel.ForEach(listProxies,
new ParallelOptions { MaxDegreeOfParallelism = nThreadsNum },
(p)=> {
int threadId = Thread.CurrentThread.ManagedThreadId;
int slot = slots.IndexOf(threadId);
if (slot == -1)
{
lock(obj)
{
slots.Add(threadId);
}
slot = slots.IndexOf(threadId);
}
var result = CheckProxyServer(p.sIPAddress, s.nPort, slot);
UpdateProxyDBRecord(result.sIPAddress, result.bOnlineStatus);
});
I took a few shortcuts there to guarantee thread safety. You don't have to do the normal check-lock-check dance because there will never be two threads attempting to add the same threadid to the list, so the second check will always fail and isn't needed. Secondly, for the same reason, I don't believe you need to ever lock around the outer IndexOf either. That makes this a very highly efficient concurrent routine that rarely locks (it should only lock nThreadsNum times) no matter how many items are in the enumerable.
Another solution is to use a SemaphoreSlim or the Producer-Consumer Pattern using a BlockinCollection<T>. Both solution support cancellation.
SemaphoreSlim
private async Task CheckProxyServerAsync(IEnumerable<object> proxies)
{
var tasks = new List<Task>();
int currentThreadNumber = 0;
int maxNumberOfThreads = 8;
using (semaphore = new SemaphoreSlim(maxNumberOfThreads, maxNumberOfThreads))
{
foreach (var proxy in proxies)
{
// Asynchronously wait until thread is available if thread limit reached
await semaphore.WaitAsync();
string proxyIP = proxy.IPAddress;
int port = proxy.Port;
tasks.Add(Task.Run(() => CheckProxyServer(proxyIP, port, Interlocked.Increment(ref currentThreadNumber)))
.ContinueWith(
(task) =>
{
ProxyAddress result = task.Result;
// Method call must be thread-safe!
UpdateProxyDbRecord(result.IPAddress, result.OnlineStatus);
Interlocked.Decrement(ref currentThreadNumber);
// Allow to start next thread if thread limit was reached
semaphore.Release();
},
TaskContinuationOptions.OnlyOnRanToCompletion));
}
// Asynchronously wait until all tasks are completed
// to prevent premature disposal of semaphore
await Task.WhenAll(tasks);
}
}
Producer-Consumer Pattern
// Uses a fixed number of same threads
private async Task CheckProxyServerAsync(IEnumerable<ProxyInfo> proxies)
{
var pipe = new BlockingCollection<ProxyInfo>();
int maxNumberOfThreads = 8;
var tasks = new List<Task>();
// Create all threads (count == maxNumberOfThreads)
for (int currentThreadNumber = 0; currentThreadNumber < maxNumberOfThreads; currentThreadNumber++)
{
tasks.Add(
Task.Run(() => ConsumeProxyInfo(pipe, currentThreadNumber)));
}
proxies.ToList().ForEach(pipe.Add);
pipe.CompleteAdding();
await Task.WhenAll(tasks);
}
private void ConsumeProxyInfo(BlockingCollection<ProxyInfo> proxiesPipe, int currentThreadNumber)
{
while (!proxiesPipe.IsCompleted)
{
if (proxiesPipe.TryTake(out ProxyInfo proxy))
{
int port = proxy.Port;
string proxyIP = proxy.IPAddress;
ProxyAddress result = CheckProxyServer(proxyIP, port, currentThreadNumber);
// Method call must be thread-safe!
UpdateProxyDbRecord(result.IPAddress, result.OnlineStatus);
}
}
}
If I'm understanding your question properly, this is actually fairly simple to do with await Task.WhenAny. Basically, you keep a collection of all of the running tasks. Once you reach a certain number of tasks running, you wait for one or more of your tasks to finish, and then you remove the tasks that were completed from your collection and continue to add more tasks.
Here's an example of what I mean below:
var tasks = new List<Task>();
for (int i = 0; i < 20; i++)
{
// I want my list of tasks to contain at most 5 tasks at once
if (tasks.Count == 5)
{
// Wait for at least one of the tasks to complete
await Task.WhenAny(tasks.ToArray());
// Remove all of the completed tasks from the list
tasks = tasks.Where(t => !t.IsCompleted).ToList();
}
// Add some task to the list
tasks.Add(Task.Factory.StartNew(async delegate ()
{
await Task.Delay(1000);
}));
}
I suggest changing your approach slightly. Instead of starting and stopping threads, put your proxy server data in a concurrent queue, one item for each proxy server. Then create a fixed number of threads (or async tasks) to work on the queue. This is more likely to provide smooth performance (you aren't starting and stopping threads over and over, which has overhead) and is a lot easier to code, in my opinion.
A simple example:
class ProxyChecker
{
private ConcurrentQueue<ProxyInfo> _masterQueue = new ConcurrentQueue<ProxyInfo>();
public ProxyChecker(IEnumerable<ProxyInfo> listProxies)
{
foreach (var proxy in listProxies)
{
_masterQueue.Enqueue(proxy);
}
}
public async Task RunChecks(int maximumConcurrency)
{
var count = Math.Max(maximumConcurrency, _masterQueue.Count);
var tasks = Enumerable.Range(0, count).Select( i => WorkerTask() ).ToList();
await Task.WhenAll(tasks);
}
private async Task WorkerTask()
{
ProxyInfo proxyInfo;
while ( _masterList.TryDequeue(out proxyInfo))
{
DoTheTest(proxyInfo.IP, proxyInfo.Port)
}
}
}
I'm working on a WPF-MVVM project and I need to implement asynchronous infinite loops in some background threads. What I have done in the ViewModel is
public TestVM()
{
LineIO_Task();
//some other work
}
and LineIO_Task is defined as
public void LineIO_Task()
{
for (int i = 0; i < 7; i++)
{
Task GetP = new Task(() => { EnPost(Lines[i]); }, TaskCreationOptions.LongRunning);
GetP.Start();
}
}
Lines is an ObservableCollection that is initialized in TestVm.
And EnPost is defined as
public async void EnPost(Line l)
{
int last = 0;
while (true)
{
// do the work in the loop
int pno = l.Com.ReadPostNo();//read a serial port
if (pno != 0 && pno != last)
{
log.WriteLog(pno + " to " + l.ToString());
Dispatcher.Invoke(() =>
{
// update the UI
l.Posts.First(x => x.IsValid).Num = pno;
l.Posts.First(x => x.IsValid).IsValid = false;
LocalDb.InsertPost(l.Num, AssignedPost.ToList().Find(x => x.Num == pno));
});
pno = last;
}
await Task.Delay(500);
}
}
I've tried Task.Run(() => Method()), Task.Factory.StartNew(() => Method()),，async Task EnPost() and using a System.Timers.Timer. But no matter which way I use, the EnPost method just doesn't run. I put break-points in the method. It doesn't hit there. Am I using Task wrong?
I'm guessing this is a "captured variable" issue; try:
for (int i = 0; i < 7; i++)
{
int index = i;
Task GetP = new Task(() => { EnPost(Lines[index]); }, TaskCreationOptions.LongRunning);
GetP.Start();
}
(fixed by question edit) Note however that using the thread-pool for a very long lived task is not a good idea. You might want to use a full thread instead. Also; your TaskDelay may want to be inside the while loop, in which case you can ignore the previous comment, as it is no longer actually a very long lived single piece.
Thanks for #Marc and #Brian's answer. They mention the"captured variable" issue, so I tried
foreach (Line l in Lines)
{ ... }
It works finally.
I have a Parallel.For loop which I use to peform a lot of HTTP request at a certain point when a scheduled task occurs like this:
Parallel.For(0, doc.GetElementsByTagName("ItemID").Count, i => {
var xmlResponse = PerformHttpRequestMethod();
});
Is there any way for me to set the loop to pause after the counter value hits 2,4,6,8,10 and so on...
so every 2 method calls it performs, sleep for 2 minutes lets say..
Is there any way I could achieve this ?
I recommend you to use Task.Delay.
Now your method is asyncronous using async/await.
public async Task DoSomething()
{
int i = 0;
while (i < doc.GetElementsByTagName("ItemID").Count)
{
Task.Run(() => PerformHttpRequestMethod());
if(i%2==0){
await Task.Delay(TimeSpan.FromMinutes(2));
//or simply:
await Task.Delay(120000);//120000 represents 2 minutes.
}
i++;
}
}
OR simply if you want to use for loop.
public async Task DoSomething()
{
for (int i = 0; i < doc.GetElementsByTagName("ItemID").Count; i++)
{
Task.Run(() => PerformHttpRequestMethod());
if(i%2==0){
await Task.Delay(TimeSpan.FromMinutes(2));
}
}
}
How would this 2nd example look if I'd want to do iterations from 0 to 4 then sleep 5 to 9 and so on... ?
public async Task DoSomething()
{
for (int i = 0; i < doc.GetElementsByTagName("ItemID").Count; i=i+5)
{
if( i%10 == 0 ){
for( int j=i;j<=i+4;j++){
Task.Run(() => PerformHttpRequestMethod());
}
}
else{
for(int j=i;j<=i+4;j++){
await Task.Delay(TimeSpan.FromMinutes(2));
}
}
}
}
Let's test the correctitude of algorithm.
i=0 -> (0%10==0 ->true) ,then will execute Task.Run(() => PerformHttpRequestMethod()) for i=(0,4)
i=5 -> (5%10==0 ->false), then will execute await Task.Delay(TimeSpan.FromMinutes(2)); for i=(5,9).
And so on...
I don't really see the point of using a Parallel.For if you want to sleep for x number of minutes or seconds every other iteration...how about using a plain old for loop?:
for(int i = 0; i < doc.GetElementsByTagName("ItemID").Count; ++i)
{
var xmlResponse = PerformHttpRequestMethod();
if (i % 2 == 0)
{
System.Threading.Thread.Sleep(TimeSpan.FromMinutes(2));
}
}
Or maybe you want to keep track of the how many iterations that are currently in flight?:
int inFlight = 0;
Parallel.For(0, doc.GetElementsByTagName("ItemID").Count, i => {
System.Threading.Interlocked.Increment(ref inFlight);
if (inFlight % 2 == 0)
System.Threading.Thread.Sleep(TimeSpan.FromMinutes(2));
var xmlResponse = PerformHttpRequestMethod();
System.Threading.Interlocked.Decrement(ref inFlight);
});
You can do that by combining a Parallel.For and a normal for loop:
for(var i = 0;i<doc.GetElementsByTagName("ItemID").Count;i = i+2)
{
Parallel.For(0, 2, i => {
var xmlResponse = PerformHttpRequestMethod();
});
Thread.Sleep(2000);
}
This is the first time I'm attempting multiple threads in a project so bear with me. The idea is this. I have a bunch of documents I need converted to pdf. I am using itextsharp to do the conversion for me. When run iteratively, the program runs fine but slow.
I have a list of items that need to be converted. I take that list and split it into 2 lists.
for (int i = 0; i < essaylist.Count / 2; i++)
{
frontessay.Add(essaylist[i]);
try
{
backessay.Add(essaylist[essaylist.Count - i]);
}
catch(Exception e)
{
}
}
if (essaylist.Count > 1)
{
var essay1 = new Essay();
Thread t1 = new Thread(() => essay1.StartThread(frontessay));
Thread t2 = new Thread(() => essay1.StartThread(backessay));
t1.Start();
t2.Start();
t1.Join();
t2.Join();
}
else
{
var essay1 = new Essay();
essay1.GenerateEssays(essaylist[1]);
}
I then create 2 threads that run this code
public void StartThread(List<Essay> essaylist)
{
var essay = new Essay();
List<System.Threading.Tasks.Task> tasklist = new List<System.Threading.Tasks.Task>();
int threadcount = 7;
Boolean threadcomplete = false;
int counter = 0;
for (int i = 0; i < essaylist.Count; i++)
{
essay = essaylist[i];
var task1 = System.Threading.Tasks.Task.Factory.StartNew(() => essay.GenerateEssays(essay));
tasklist.Add(task1);
counter++;
if (tasklist.Count % threadcount == 0)
{
tasklist.ForEach(t => t.Wait());
//counter = 0;
tasklist = new List<System.Threading.Tasks.Task>();
threadcomplete = true;
}
Thread.Sleep(100);
}
tasklist.ForEach(t => t.Wait());
Thread.Sleep(100);
}
For the majority of the files, the code runs as it should. However, for example I have 155 items that need to be convereted. When the program finishes and I look at the results I have 149 items instead of 155. It seems like the results are something like the total = list - threadcount. In this case its 7. Any ideas on how to correct this? Am I even doing threads/tasks correctly?
Also the essay.GenerateEssays code is the actual itextsharp that converts the info from the db to the actual pdf.
How about using TPL. It seems that all your code can be replaced with this
Parallel.ForEach(essaylist, essay =>
{
YourAction(essay);
});