I have some code that needs to retry whenever it cannot make a database connection, for example 20 times for each login. The website has dozens of users, what would be the best way to handle this? I was thinking of using a counter but as multiple users will be logged on at the same time perhaps some threading would be required, which I am unfamiliar with.
EDIT :
I get the error 'method name expected' on the line where I am instantiating the Thread object. Here is my code :
private static List<Field.Info> FromDatabase(this Int32 _campId)
{
List<Field.Info> lstFields = new List<Field.Info>();
Field.List.Response response = new Field.List.Ticket
{
campId = _campId
}.Commit();
if (response.status == Field.List.Status.success)
{
lstFields = response.fields;
lock (campIdLock)
{
loadedCampIds.Add(_campId);
}
}
if (response.status == Field.List.Status.retry)
{
Thread th1 = new Thread(new ThreadStart(FromDatabase(_campId)));
FromDatabase(_campId);
}
return lstFields;
}
My approach would be an event, which increments the counter and calls an asynchronous methode if counter <= 20.
Then i would block the current thread and wait for the connection result.
To thread-safe increment your counter use
Interlocked.Increment(ref int counter);
Related
I am trying to create some email sending process. Without the multi-threading approach, it is working as expected. But it took more time to send all emails.
After I decided to use multi-threading and I also do some exception handling for the multi-threading process as well.
Here are my methods,
public enum EmailRequestProcesStatus
{
Pending = 1,
Processing = 2,
Complete = 3
}
public async Task<int> InvokeSendingEmailAsync(string serverFolderPath)
{
// update pending requests
await UpdateEmailRequestToProcessAsync(EmailRequestProcesStatus.Processing);
var invokedDateTime = DateTime.Now;
// first get the CustomerEmail list
var emailList= await GetPendingEmailList();
IList<CustomerRule> customerRules = new List<CustomerRule>();
int ruleLength = 0;
List<Thread> threadList = new List<Thread>();
// email count list
List<int> sendEmailCountList = new List<int>();
foreach (var customer in emailList)
{
ruleLength++;
customerRules.Add(customer);
if (ruleLength % 20 == 0)
{
var custList = customerRules;
Thread t = new Thread(() => ProcessEmailSend(custList, serverFolderPath, invokedDateTime, sendEmailCountList));
threadList.Add(t);
t.Start();
customerRules = new List<CustomerRule>();
}
}
if (customerRules.Count > 0)
{
Thread t = new Thread(() => ProcessEmailSend(customerRules, serverFolderPath, invokedDateTime, sendEmailCountList));
threadList.Add(t);
t.Start();
}
// join the all threads
foreach (var thread in threadList)
{
thread.Join();
}
return sendEmailCountList.Count;
}
private void ProcessEmailSend(IList<CustomerRule> customerList, string serverFolderPath, DateTime invokedDateTime, List<int> sendEmailCountList)
{
foreach(var cust in customerList)
{
var customerDetailList = get customerEmailDetails(cust);
// access some file from file server by giving details
var result = getPdf(customerDetailList.files);
// do the rest of work using
}
}
private filesDto getPdf(filesDto files)
{
try
{
//here I call the file server
}
catch(Exception e)
{
UpdateEmailServiceWhenException();
throw;
}
}
After the throw, it will be captured by the global exception handler.
If one worker thread got an exception for file not found from the file server then it inserted to the table as an exception and throw again.
From that, all the processes will be terminated and the IIS app pool also shutdown. There can be ambiguous multiple records are in the table as well.
How do I sort it?
Most likely, multiple unhandled exceptions being thrown in that worker thread results in the Rapid-Fail Protection feature.
https://learn.microsoft.com/en-us/iis/configuration/system.applicationHost/applicationPools/add/failure
Simply speaking, enough unhandled exceptions in a configurable timespan (the default is 5 in 5 minutes) will shut down the App Pool, causing the 503 Service Unavailable response.
Please check the advanced setting in the Application pool.
To fix this, we need to handle the exception or prevent the exception from being thrown.
Please refer to the below discussion.
Why does my IIS7 application pool shutdown after an exception in a DLL called from an ASP.NET page?
I have a fairly simple WPF application that uses Entity Framework. The main page of the application has a list of records that I am getting from a database on startup.
Each record has a picture, so the operation can be a little slow when the wireless signal is poor. I'd like this (and many of my SQL operations) to perform in the background if possible. I have async/await setup and at first it seemed to be working exactly as I wanted but now I'm seeing that my application is becoming unresponsive when accessing the DB.
Eventually I'm thinking I'm going to load up the text in one query and the images in another background operation and load them as they come in. This way I get the important stuff right away and the pictures can come in in the background, but the way things are going it's still looking like it will lock up if I do this.
On top of that, I'm trying to implement something to handle connectivity issues (in case the wifi cuts out momentarily) so that the application notifies the user of a connection issue, automatically retries a few times, etc. I put a try catch for SQL exception which seems to be working for me, but the whole application locks up for about a minute while it is trying to connect to the DB.
I tried testing my async/await using await Task.Delay() and everything is very responsive as expected while awaiting the delay, but everything locks up when awaiting the .ToListAsync(). Is this normal and expected? My understanding of async/await is pretty limited.
My code is kind of messy (I'm new) but it does what I need it to do for the most part. I understand there's probably plenty of improvements I can make and better ways to do things, but one step at a time here. My main goal right now is to keep the application from crashing during database accessing exceptions and to keep the user notified of what the application is doing (searching, trying to access db, unable to reach DB and retrying, etc) as opposed to being frozen, which is what they're going to think when they see it being unresponsive for over a minute.
Some of my code:
In my main view model
DataHelper data = new DataHelper();
private async void GetQualityRegisterQueueAsync()
{
try
{
var task = data.GetQualityRegisterAsync();
IsSearching = true;
await task;
IsSearching = false;
QualityRegisterItems = new ObservableCollection<QualityRegisterQueue>(task.Result);
OrderQualityRegisterItems();
}
catch (M1Exception ex)
{
Debug.WriteLine(ex.Message);
Debug.WriteLine("QualityRegisterLogViewModel.GetQualityRegisterQueue() Operation Failed");
}
}
My Data Helper Class
public class DataHelper
{
private bool debugging = false;
private const int MAX_RETRY = 2;
private const double LONG_WAIT_SECONDS = 5;
private const double SHORT_WAIT_SECONDS = 0.5;
private static readonly TimeSpan longWait = TimeSpan.FromSeconds(LONG_WAIT_SECONDS);
private static readonly TimeSpan shortWait = TimeSpan.FromSeconds(SHORT_WAIT_SECONDS);
private enum RetryableSqlErrors
{
ServerNotFound = -1,
Timeout = -2,
NoLock = 1204,
Deadlock = 1205,
}
public async Task<List<QualityRegisterQueue>> GetQualityRegisterAsync()
{
if(debugging) await Task.Delay(5000);
var retryCount = 0;
using (M1Context m1 = new M1Context())
{
for (; ; )
{
try
{
return await (from a in m1.QualityRegisters
where (a.qanClosed == 0)
//orderby a.qanAssignedDate descending, a.qanOpenedDate
orderby a.qanAssignedDate.HasValue descending, a.qanAssignedDate, a.qanOpenedDate
select new QualityRegisterQueue
{
QualityRegisterID = a.qanQualityRegisterID,
JobID = a.qanJobID.Trim(),
JobAssemblyID = a.qanJobAssemblyID,
JobOperationID = a.qanJobOperationID,
PartID = a.qanPartID.Trim(),
PartRevisionID = a.qanPartRevisionID.Trim(),
PartShortDescription = a.qanPartShortDescription.Trim(),
OpenedByEmployeeID = a.qanOpenedByEmployeeID.Trim(),
OpenedByEmployeeName = a.OpenedEmployee.lmeEmployeeName.Trim(),
OpenedDate = a.qanOpenedDate,
PartImage = a.JobAssembly.ujmaPartImage,
AssignedDate = a.qanAssignedDate,
AssignedToEmployeeID = a.qanAssignedToEmployeeID.Trim(),
AssignedToEmployeeName = a.AssignedEmployee.lmeEmployeeName.Trim()
}).ToListAsync();
}
catch (SqlException ex)
{
Debug.WriteLine("SQL Exception number = " + ex.Number);
if (!Enum.IsDefined(typeof(RetryableSqlErrors), ex.Number))
throw new M1Exception(ex.Message, ex);
retryCount++;
if (retryCount > MAX_RETRY) throw new M1Exception(ex.Message, ex); ;
Debug.WriteLine("Retrying. Count = " + retryCount);
Thread.Sleep(ex.Number == (int)RetryableSqlErrors.Timeout ?
longWait : shortWait);
}
}
}
}
}
Edit: Mostly looking for general guidance here, though a specific example of what to do would be great. For these types of operations where I am downloading data, is it just a given that if I need the application to be responsive I need to be making multiple threads? Is that a common solution to this type of problem? Is this not something I should be expecting async/await to solve?
If you call this method from your UI thread, you will overload the capture of UI thread context and back on itself. Also, your service will not be necessarily "Performant" because it must wait until the UI thread is free before it can continue.
The solution is simple: just call the method passing the ConfigureAwait "false" parameter when you made the call.
.ToListAsync().ConfigureAwaiter(false);
I hope it helps
maybe it is a bad question, but I have a Client App which sends some Commands to the server. I made a little function for sending commands, which simply waits until a message is received over AutoResetEvent.WaitOne() and then returns the message as string. I used this function very often in my client app, because it was a fast way to implement sth.
Now I want to show a LoadingCircle in this app while it is waiting from the response from the Server. I already wrote sth for that in the message receiving function, which simply adds the control to the Form. This is working find, but UserControl(Loading Circle), which redraws after some milliseconds, doesnt redraw anymore, because AutoResetEvent.WaitOne() is blocking the GUI Thread. I know that I could move the message receiving part into another Thread, but I dont want to change the concept of this function, because I used the message receiving function over 150x.
So my question is: Is there a lazy way to do that, so that It does GUI Events while its waiting, or maybe I can move the drawing part of the LoadingCircle Control into another Thread?
Any help will be greatly appreciated
You have essentially taken an async operation and turned it into a synchronous operation via AutoResetEvent.WaitOne. And now you want to turn it back into an async operation.
My suggestion would be to separate the function from the UI. Then tool a synchronous and an async option. That way you get to keep a fully functioning app and can incrementally release during your code re-write of the 150 instances that need changed.
The new async and await keywords in 4.5 should serve you well here.
I found a small fix for that here, which works well(I cant take credit for it)
private static TimeSpan InfiniteTimeout = TimeSpan.FromMilliseconds(-1);
private const Int32 MAX_WAIT = 100;
public static bool Wait(WaitHandle handle, TimeSpan timeout)
{
Int32 expireTicks;
bool signaled;
Int32 waitTime;
bool exitLoop;
// guard the inputs
if (handle == null) {
throw new ArgumentNullException("handle");
}
else if ((handle.SafeWaitHandle.IsClosed)) {
throw new ArgumentException("closed wait handle", "handle");
}
else if ((handle.SafeWaitHandle.IsInvalid)) {
throw new ArgumentException("invalid wait handle", "handle");
}
else if ((timeout < InfiniteTimeout)) {
throw new ArgumentException("invalid timeout <-1", "timeout");
}
// wait for the signal
expireTicks = (int)Environment.TickCount + timeout.TotalMilliseconds;
do {
if (timeout.Equals(InfiniteTimeout)) {
waitTime = MAX_WAIT;
}
else {
waitTime = (expireTicks - Environment.TickCount);
if (waitTime <= 0) {
exitLoop = true;
waitTime = 0;
}
else if (waitTime > MAX_WAIT) {
waitTime = MAX_WAIT;
}
}
if ((handle.SafeWaitHandle.IsClosed)) {
exitLoop = true;
}
else if (handle.WaitOne(waitTime, false)) {
exitLoop = true;
signaled = true;
}
else {
if (Application.MessageLoop) {
Application.DoEvents();
}
else {
Thread.Sleep(1);
}
}
}
while (!exitLoop);
return signaled;
}
I'm building a multithreaded app in .net.
I have a thread that listens to a connection (abstract, serial, tcp...).
When it receives a new message, it adds it to via AddMessage. Which then call startSpool. startSpool checks to see if the spool is already running and if it is, returns, otherwise, starts it in a new thread. The reason for this is, the messages HAVE to be processed serially, FIFO.
So, my questions are...
Am I going about this the right way?
Are there better, faster, cheaper patterns out there?
My apologies if there is a typo in my code, I was having problems copying and pasting.
ConcurrentQueue<IMyMessage > messages = new ConcurrentQueue<IMyMessage>();
const int maxSpoolInstances = 1;
object lcurrentSpoolInstances;
int currentSpoolInstances = 0;
Thread spoolThread;
public void AddMessage(IMyMessage message)
{
this.messages.Add(message);
this.startSpool();
}
private void startSpool()
{
bool run = false;
lock (lcurrentSpoolInstances)
{
if (currentSpoolInstances <= maxSpoolInstances)
{
this.currentSpoolInstances++;
run = true;
}
else
{
return;
}
}
if (run)
{
this.spoolThread = new Thread(new ThreadStart(spool));
this.spoolThread.Start();
}
}
private void spool()
{
Message.ITimingMessage message;
while (this.messages.Count > 0)
{
// TODO: Is this below line necessary or does the TryDequeue cover this?
message = null;
this.messages.TryDequeue(out message);
if (message != null)
{
// My long running thing that does something with this message.
}
}
lock (lcurrentSpoolInstances)
{
this.currentSpoolInstances--;
}
}
This would be easier using BlockingCollection<T> instead of ConcurrentQueue<T>.
Something like this should work:
class MessageProcessor : IDisposable
{
BlockingCollection<IMyMessage> messages = new BlockingCollection<IMyMessage>();
public MessageProcessor()
{
// Move this to constructor to prevent race condition in existing code (you could start multiple threads...
Task.Factory.StartNew(this.spool, TaskCreationOptions.LongRunning);
}
public void AddMessage(IMyMessage message)
{
this.messages.Add(message);
}
private void Spool()
{
foreach(IMyMessage message in this.messages.GetConsumingEnumerable())
{
// long running thing that does something with this message.
}
}
public void FinishProcessing()
{
// This will tell the spooling you're done adding, so it shuts down
this.messages.CompleteAdding();
}
void IDisposable.Dispose()
{
this.FinishProcessing();
}
}
Edit: If you wanted to support multiple consumers, you could handle that via a separate constructor. I'd refactor this to:
public MessageProcessor(int numberOfConsumers = 1)
{
for (int i=0;i<numberOfConsumers;++i)
StartConsumer();
}
private void StartConsumer()
{
// Move this to constructor to prevent race condition in existing code (you could start multiple threads...
Task.Factory.StartNew(this.spool, TaskCreationOptions.LongRunning);
}
This would allow you to start any number of consumers. Note that this breaks the rule of having it be strictly FIFO - the processing will potentially process "numberOfConsumer" elements in blocks with this change.
Multiple producers are already supported. The above is thread safe, so any number of threads can call Add(message) in parallel, with no changes.
I think that Reed's answer is the best way to go, but for the sake of academics, here is an example using the concurrent queue -- you had some races in the code that you posted (depending upon how you handle incrementing currnetSpoolInstances)
The changes I made (below) were:
Switched to a Task instead of a Thread (uses thread pool instead of incurring the cost of creating a new thread)
added the code to increment/decrement your spool instance count
changed the "if currentSpoolInstances <= max ... to just < to avoid having one too many workers (probably just a typo)
changed the way that empty queues were handled to avoid a race: I think you had a race, where your while loop could have tested false, (you thread begins to exit), but at that moment, a new item is added (so your spool thread is exiting, but your spool count > 0, so your queue stalls).
private ConcurrentQueue<IMyMessage> messages = new ConcurrentQueue<IMyMessage>();
const int maxSpoolInstances = 1;
object lcurrentSpoolInstances = new object();
int currentSpoolInstances = 0;
public void AddMessage(IMyMessage message)
{
this.messages.Enqueue(message);
this.startSpool();
}
private void startSpool()
{
lock (lcurrentSpoolInstances)
{
if (currentSpoolInstances < maxSpoolInstances)
{
this.currentSpoolInstances++;
Task.Factory.StartNew(spool, TaskCreationOptions.LongRunning);
}
}
}
private void spool()
{
IMyMessage message;
while (true)
{
// you do not need to null message because it is an "out" parameter, had it been a "ref" parameter, you would want to null it.
if(this.messages.TryDequeue(out message))
{
// My long running thing that does something with this message.
}
else
{
lock (lcurrentSpoolInstances)
{
if (this.messages.IsEmpty)
{
this.currentSpoolInstances--;
return;
}
}
}
}
}
Check 'Pipelines pattern': http://msdn.microsoft.com/en-us/library/ff963548.aspx
Use BlockingCollection for the 'buffers'.
Each Processor (e.g. ReadStrings, CorrectCase, ..), should run in a Task.
HTH..
I have a multithreaded app that uses sqlite. When two threads try to update the db at once i get the exception
Additional information: The database file is locked
I thought it would retry in a few milliseconds. My querys arent complex. The most complex one (which happens frequently) is update, select, run trivial code update/delete, commit. Why does it throw the exception? How can i make it retry a few times before throwing an exception?
SQLite isn't thread safe for access, which is why you get this error message.
You should synchronize the access to the database (create an object, and "lock" it) whenever you go to update. This will cause the second thread to block and wait until the first thread's update finishes automatically.
try to make your transaction / commit blocks as short as possible. The only time you can deadlock/block is with a transaction -- thus if you don't do them you won't have the problem.
That said, there are times when you need to do transactions (mostly on data updates), but don't do them while you are "run trivial code" if you can avoid it.
A better approach may be to use an update queue, if you can do the database updates out of line with the rest of your code. For example, you could do something like:
m_updateQueue.Add(()=>InsertOrder(o));
Then you could have a dedicated update thread that processed the queue.
That code would look similar to this (I haven't compiled or tested it):
class UpdateQueue : IDisposable
{
private object m_lockObj;
private Queue<Action> m_queue;
private volatile bool m_shutdown;
private Thread m_thread;
public UpdateQueue()
{
m_lockObj = new Object();
m_queue = new Queue<Action>();
m_thread = new Thread(ThreadLoop);
m_thread.Start();
}
public void Add(Action a)
{
lock(m_lockObj)
{
m_queue.Enqueue(a);
Monitor.Pulse(m_lockObj);
}
}
public void Dispose()
{
if (m_thread != null)
{
m_shutdown = true;
Monitor.PulseAll(m_lockObj);
m_thread.Join();
m_thread = null;
}
}
private void ThreadLoop()
{
while (! m_shutdown)
{
Action a;
lock (m_lockObj)
{
if (m_queue.Count == 0)
{
Monitor.Wait(m_lockObj);
}
if (m_shutdown)
{
return;
}
a = m_queuue.Dequeue();
}
a();
}
}
}
Or, you could use something other than Sql Lite.