Develop Reference

Block Controller Method while already running

I have a controller which returns a large json object. If this object does not exist, it will generate and return it afterwards. The generation takes about 5 seconds, and if the client sent the request multiple times, the object gets generated with x-times the children. So my question is: Is there a way to block the second request, until the first one finished, independent who sent the request?
Normally I would do it with a Singleton, but because I am having scoped services, singleton does not work here

Warning: this is very oppinionated and maybe not suitable for Stack Overflow, but here it is anyway
Although I'll provide no code... when things take a while to generate, you don't usually spend that time directly in controller code, but do something like "start a background task to generate the result, and provide a "task id", which can be queried on another different call).
So, my preferred course of action for this would be having two different controller actions:
Generate, which creates the background job, assigns it some id, and returns the id
GetResult, to which you pass the task id, and returns either different error codes for "job id doesn't exist", "job id isn't finished", or a 200 with the result.
This way, your clients will need to call both, however, in Generate, you can check if the job is already being created and return an existing job id.
This of course moves the need to "retry and check" to your client: in exchange, you don't leave the connection to the server opened during those 5 seconds (which could potentially be multiplied by a number of clients) and return fast.
Otherwise, if you don't care about having your clients wait for a response during those 5 seconds, you could do a simple:
if(resultDoesntExist) {
resultDoesntExist = false; // You can use locks for the boolean setters or Interlocked instead of just setting a member
resultIsBeingGenerated = true;
generateResult(); // <-- this is what takes 5 seconds
resultIsBeingGenerated = false;
}
while(resultIsBeingGenerated) { await Task.Delay(10); } // <-- other clients will wait here
var result = getResult(); // <-- this should be fast once the result is already created
return result;
note: those booleans and the actual loop could be on the controller, or on the service, or wherever you see fit: just be wary of making them thread-safe in however method you see appropriate
So you basically make other clients wait till the first one generates the result, with "almost" no CPU load on the server... however with a connection open and a thread from the threadpool used, so I just DO NOT recommend this :-)
PS: #Leaky solution above is also good, but it also shifts the responsability to retry to the client, and if you are going to do that, I'd probably go directly with a "background job id", instead of having the first (the one that generates the result) one take 5 seconds. IMO, if it can be avoided, no API action should ever take 5 seconds to return :-)

Do you have an example for Interlocked.CompareExchange?
Sure. I'm definitely not the most knowledgeable person when it comes to multi-threading stuff, but this is quite simple (as you might know, Interlocked has no support for bool, so it's customary to represent it with an integral type):
public class QueryStatus
{
private static int _flag;
// Returns false if the query has already started.
public bool TrySetStarted()
=> Interlocked.CompareExchange(ref _flag, 1, 0) == 0;
public void SetFinished()
=> Interlocked.Exchange(ref _flag, 0);
}
I think it's the safest if you use it like this, with a 'Try' method, which tries to set the value and tells you if it was already set, in an atomic way.
Besides simply adding this (I mean just the field and the methods) to your existing component, you can also use it as a separate component, injected from the IOC container as scoped. Or even injected as a singleton, and then you don't have to use a static field.
Storing state like this should be good for as long as the application is running, but if the hosted application is recycled due to inactivity, it's obviously lost. Though, that won't happen while a request is still processing, and definitely won't happen in 5 seconds.
(And if you wanted to synchronize between app service instances, you could 'quickly' save a flag to the database, in a transaction with proper isolation level set. Or use e.g. Azure Redis Cache.)
Example solution
As Kit noted, rightly so, I didn't provide a full solution above.
So, a crude implementation could go like this:
public class SomeQueryService : ISomeQueryService
{
private static int _hasStartedFlag;
private static bool TrySetStarted()
=> Interlocked.CompareExchange(ref _hasStartedFlag, 1, 0) == 0;
private static void SetFinished()
=> Interlocked.Exchange(ref _hasStartedFlag, 0);
public async Task<(bool couldExecute, object result)> TryExecute()
{
if (!TrySetStarted())
return (couldExecute: false, result: null);
// Safely execute long query.
SetFinished();
return (couldExecute: true, result: result);
}
}
// In the controller, obviously
[HttpGet()]
public async Task<IActionResult> DoLongQuery([FromServices] ISomeQueryService someQueryService)
{
var (couldExecute, result) = await someQueryService.TryExecute();
if (!couldExecute)
{
return new ObjectResult(new ProblemDetails
{
Status = StatusCodes.Status503ServiceUnavailable,
Title = "Another request has already started. Try again later.",
Type = "https://tools.ietf.org/html/rfc7231#section-6.6.4"
})
{ StatusCode = StatusCodes.Status503ServiceUnavailable };
}
return Ok(result);
}
Of course possibly you'd want to extract the 'blocking' logic from the controller action into somewhere else, for example an action filter. In that case the flag should also go into a separate component that could be shared between the query service and the filter.
General use action filter
I felt bad about my inelegant solution above, and I realized that this problem can be generalized into basically a connection number limiter on an endpoint.
I wrote this small action filter that can be applied to any endpoint (multiple endpoints), and it accepts the number of allowed connections:
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
public class ConcurrencyLimiterAttribute : ActionFilterAttribute
{
private readonly int _allowedConnections;
private static readonly ConcurrentDictionary<string, int> _connections = new ConcurrentDictionary<string, int>();
public ConcurrencyLimiterAttribute(int allowedConnections = 1)
=> _allowedConnections = allowedConnections;
public override async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
var key = context.HttpContext.Request.Path;
if (_connections.AddOrUpdate(key, 1, (k, v) => ++v) > _allowedConnections)
{
Close(withError: true);
return;
}
try
{
await next();
}
finally
{
Close();
}
void Close(bool withError = false)
{
if (withError)
{
context.Result = new ObjectResult(new ProblemDetails
{
Status = StatusCodes.Status503ServiceUnavailable,
Title = $"Maximum {_allowedConnections} simultaneous connections are allowed. Try again later.",
Type = "https://tools.ietf.org/html/rfc7231#section-6.6.4"
})
{ StatusCode = StatusCodes.Status503ServiceUnavailable };
}
_connections.AddOrUpdate(key, 0, (k, v) => --v);
}
}
}

Best practice for threading in a Web API

I have a simple Web API with a single GET controller only. The GET controller reads XML data and returns it in JSON format.
The GET API is supposed to be excessed every minute but in case if there is no GET request for more than 1 minute then I need to start backing up the data (simply in XML format..nothing fancy).
My GET Api
//GET api/
public IHttpActionResult Get()
{
try
{
//Read XML
XDocument xDoc = XDocument.Load(#"D:\myfile.xml");
//Convert XML data into JSON string
string jsonStr = JsonConvert.SerializeXNode(xDoc);
JObject json = JObject.Parse(jsonStr);
return Ok(json);
}
catch (Exception ex)
{
return NotFound();
}
}
Sample XML:
<CurrentStatus>
<Time Stamp= "20181011133631244">
<price>12</price>
<amount>100</amount>
</Time>
</CurrentStatus>
Sample Backup in XML:
<CurrentStatus>
<Time Stamp= "20181011133631244">
<price>12</price>
<amount>100</amount>
</Time>
<Time Stamp= "20181011133633354">
<price>11</price>
<amount>120</amount>
</Time>
<Time Stamp= "20181011133633204">
<price>15</price>
<amount>90</amount>
</Time>
</CurrentStatus>
My Planned Logic: I am planning to declare a static variable _lastAccessedTimestamp, which will be monitored by a separate thread. If currentTime - _lastAccessedTimestamp > 1 min then start backing-up the data.
Question: Good-practice to implement threading in Web API for this kind of scenario (or any other better option)?
PS: Web Service will run on a localhost IIS server.

After each request I would set/reset a timer for 1 min. Having a variable that you poll from a different thread is not efficient and it raises other problems.
Even more, you can use a proper job handling lib (like Hangfire - https://www.hangfire.io/) so you can just schedule a job (after 1 min) and reschedule it after each request, in case a new request comes.
Be aware that IIS can stop your process at any time, you might want to look into that.
http://docs.hangfire.io/en/latest/deployment-to-production/making-aspnet-app-always-running.html
For the implementation, you can use a middleware/filter that does this at the end of each request automatically. No need to add this in the controller. It's also easier to add multiple controllers without duplicating the code.

You can use something like Hangfire to fire off a task, and then that task can be handled by something. That could be a console app, another web application, an Azure Function, whatever. The point is that it's external to your main web application. Then, you can immediately return, without waiting for that thing, whatever it is, to finish. You can, also, utilize something like SignalR and web workers to push status updates to the client.

I would do this using System.Timers.Timer. Further reading here -> Time Class
Please know that my answer uses dependency injection but could easily be refactored to instantiating normal classes.
I would Implement a timer service which is configured using an interval in your case 1 minute. Im going to share my timer implementation below , it is registered in the dependency injection container as a singleton but should give you something to work with.
public class TimerService : ITimerService
{
private readonly System.Timers.Timer _timer;
private DateTime _startTime = DateTime.Now;
private double _timerSettings;
public TimerService()
{
_timer = new System.Timers.Timer
{
AutoReset = true,
Interval = 60000,
};
_timer.Elapsed += (sender, args) =>
{
//Backup Data method here
_startTime = DateTime.Now;
};
}
public double GetTimerInterval()
{
return _timer.Interval;
}
public void StopTimer()
{
if (_timer == null)
{
throw new ApplicationException("Timer not primed.");
}
_timer.Stop();
}
public void StartTimer()
{
if (_timer == null)
{
throw new ApplicationException("Timer not primed.");
}
_startTime = DateTime.Now;
_timer.Start();
}
}
On application start you must kick off the timer , after every minute it will start doing the backup of data. Due to the AutoReset = true.
Now simply reset the timer in your controller
[Route("api/[controller]/[action]")]
public class XmlController : Controller
{
private readonly ITimerService _timerService;
public XmlController(ITimerService timerService)
{
//Injected in
_timerService = timerService;
}
[HttpGet]
public IActionResult ProccessXML(object someXMLObject)
{
_timerService.StopTimer();
SomeMethodWithXml(someXMLObject)
//Reset Timer
_timerService.StartTimer();
return Ok();
}
}

Shared object among different requests

I'm working with .NET 3.5 with a simple handler for http requests. Right now, on each http request my handler opens a tcp connection with 3 remote servers in order to receive some information from them. Then closes the sockets and writes the server status back to Context.Response.
However, I would prefer to have a separate object that every 5 minutes connects to the remote servers via tcp, gets the information and keeps it. So the HttpRequest, on each request would be much faster just asking this object for the information.
So my questions here are, how to keep a shared global object in memory all the time that can also "wake" an do those tcp connections even when no http requests are coming and have the object accesible to the http request handler.

A service may be overkill for this.
You can create a global object in your application start and have it create a background thread that does the query every 5 minutes. Take the response (or what you process from the response) and put it into a separate class, creating a new instance of that class with each response, and use System.Threading.Interlocked.Exchange to replace a static instance each time the response is retrieved. When you want to look the the response, simply copy a reference the static instance to a stack reference and you will have the most recent data.
Keep in mind, however, that ASP.NET will kill your application whenever there are no requests for a certain amount of time (idle time), so your application will stop and restart, causing your global object to get destroyed and recreated.
You may read elsewhere that you can't or shouldn't do background stuff in ASP.NET, but that's not true--you just have to understand the implications. I have similar code to the following example working on an ASP.NET site that handles over 1000 req/sec peak (across multiple servers).
For example, in global.asax.cs:
public class BackgroundResult
{
public string Response; // for simplicity, just use a public field for this example--for a real implementation, public fields are probably bad
}
class BackgroundQuery
{
private BackgroundResult _result; // interlocked
private readonly Thread _thread;
public BackgroundQuery()
{
_thread = new Thread(new ThreadStart(BackgroundThread));
_thread.IsBackground = true; // allow the application to shut down without errors even while this thread is still running
_thread.Name = "Background Query Thread";
_thread.Start();
// maybe you want to get the first result here immediately?? Otherwise, the first result may not be available for a bit
}
/// <summary>
/// Gets the latest result. Note that the result could change at any time, so do expect to reference this directly and get the same object back every time--for example, if you write code like: if (LatestResult.IsFoo) { LatestResult.Bar }, the object returned to check IsFoo could be different from the one used to get the Bar property.
/// </summary>
public BackgroundResult LatestResult { get { return _result; } }
private void BackgroundThread()
{
try
{
while (true)
{
try
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("http://example.com/samplepath?query=query");
request.Method = "GET";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (StreamReader reader = new StreamReader(response.GetResponseStream(), System.Text.Encoding.UTF8))
{
// get what I need here (just the entire contents as a string for this example)
string result = reader.ReadToEnd();
// put it into the results
BackgroundResult backgroundResult = new BackgroundResult { Response = result };
System.Threading.Interlocked.Exchange(ref _result, backgroundResult);
}
}
}
catch (Exception ex)
{
// the request failed--cath here and notify us somehow, but keep looping
System.Diagnostics.Trace.WriteLine("Exception doing background web request:" + ex.ToString());
}
// wait for five minutes before we query again. Note that this is five minutes between the END of one request and the start of another--if you want 5 minutes between the START of each request, this will need to change a little.
System.Threading.Thread.Sleep(5 * 60 * 1000);
}
}
catch (Exception ex)
{
// we need to get notified of this error here somehow by logging it or something...
System.Diagnostics.Trace.WriteLine("Error in BackgroundQuery.BackgroundThread:" + ex.ToString());
}
}
}
private static BackgroundQuery _BackgroundQuerier; // set only during application startup
protected void Application_Start(object sender, EventArgs e)
{
// other initialization here...
_BackgroundQuerier = new BackgroundQuery();
// get the value here (it may or may not be set quite yet at this point)
BackgroundResult result = _BackgroundQuerier.LatestResult;
// other initialization here...
}

My timer loop intermittently slows down, not sure why

I have a API request queue that I have looped using System.Timers.Timer. I set it up like this:
private static void SetupTimerLoop()
{
queueLoopTimer.Elapsed += new ElapsedEventHandler(OnTimedEvent);
queueLoopTimer.Interval = 100;
queueLoopTimer.Start();
}
Periodically OnTimedEvent will only get called once every second, almost exactly for large spans of time. It will then speed back up to once every 100ms. I cannot accurately reproduce these results, sometimes it happens, sometimes it does not. I have watched my CPU usage and it doesn't spike during these times of slowdown, if anything it goes down.
If I make breakpoints it shows that the timers interval is still only 100ms.
What could be going on here? Is there anything I can do to further troubleshoot this?
Potentially related, when this happens all my HTTP requests that are initiated from the loop (these are put onto Tasks so they return asynchronously) stop returning.
Request related:
private T Get<T>(string endpoint, IRequest request) where T : class
{
var client = InitHttpClient();
string debug = BaseUrl + endpoint + ToQueryString(request);
HttpResponseMessage response = client.GetAsync(BaseUrl + endpoint + ToQueryString(request)).Result;
string body = response.Content.ReadAsStringAsync().Result;
if (response.IsSuccessStatusCode)
{
T result = JsonConvert.DeserializeObject<T>(body, _serializerSettings);
return result;
}
var error = JsonConvert.DeserializeObject<HelpScoutError>(body);
throw new HelpScoutApiException(error, body);
}
I never get to: string body = response.Content.ReadAsStringAsync().Result; as long as the loop seems to be running slow.
Not sure if that could have something to do with it, hopefully you guys have some insight.
Edit: I don't flood the API with requests. I throttle the number of requests that go out per rolling 60-second period. During that time I just use an if statement to pass over the API call for that loop.

First of all, I'm not too clear when you say you have a loop. Your "OnTimedEvent" is an event handler that gets triggered at a preset interval (100ms).
As for a hint, you might want to try adding logic at the beginning of your "OnTimedEvent" handler to prevent re-entrance, just in-case your "OnTimedEvent" handler is taking longer than 100ms.
static int working = 0;
private static void OnTimedEvent(Object source, System.Timers.ElapsedEventArgs e)
{
if (working > 0)
return;
working++;
//do your normal work here
working--;
}

Using WCF service via async interface from worker thread, how do I ensure that events are sent from the client "in order"

I am writing a Silverlight class library to abstract the interface to a WCF service. The WCF service provides a centralized logging service. The Silverlight class library provides a simplified log4net-like interface (logger.Info, logger.Warn, etc) for logging. From the class library I plan to provide options such that logged messages can be accumulated on the client and sent in "bursts" to the WCF logging service, rather than sending each message as it occurs. Generally, this is working well. The class library does accumulate messages and it does send collections of messages to the WCF logging service, where they are logged by an underlying logging framework.
My current problem is that the messages (from a single client with a single thread - all logging code is in button click events) are becoming interleaved in the logging service. I realize that the at least part of this is probably due to the instancing (PerCall) or Synchronization of the WCF logging service. However, it also seems that my messages are occurring in such rapid succession that that the "bursts" of messages leaving on the async calls are actually "leaving" the client in a different order than they were generated.
I have tried to set up a producer consumer queue as described here with a slight (or should that be "slight" with air quotes) change that the Work method blocks (WaitOne) until the async call returns (i.e. until the async callback executes). The idea is that when one burst of messages is sent to the WCF logging service, the queue should wait until that burst has been processed before sending the next burst.
Maybe what I am trying to do is not feasible, or maybe I am trying to solve the wrong problem, (or maybe I just don't know what I am doing!).
Anyway, here is my producer/consumer queue code:
internal class ProducerConsumerQueue : IDisposable
{
EventWaitHandle wh = new AutoResetEvent(false);
Thread worker;
readonly object locker = new object();
Queue<ObservableCollection<LoggingService.LogEvent>> logEventQueue = new Queue<ObservableCollection<LoggingService.LogEvent>>();
LoggingService.ILoggingService loggingService;
internal ProducerConsumerQueue(LoggingService.ILoggingService loggingService)
{
this.loggingService = loggingService;
worker = new Thread(Work);
worker.Start();
}
internal void EnqueueLogEvents(ObservableCollection<LoggingService.LogEvent> logEvents)
{
//Queue the next burst of messages
lock(locker)
{
logEventQueue.Enqueue(logEvents);
//Is this Set conflicting with the WaitOne on the async call in Work?
wh.Set();
}
}
private void Work()
{
while(true)
{
ObservableCollection<LoggingService.LogEvent> events = null;
lock(locker)
{
if (logEventQueue.Count > 0)
{
events = logEventQueue.Dequeue();
if (events == null || events.Count == 0) return;
}
}
if (events != null && events.Count > 0)
{
System.Diagnostics.Debug.WriteLine("1. Work - Sending {0} events", events.Count);
//
// This seems to be the key...
// Send one burst of messages via an async call and wait until the async call completes.
//
loggingService.BeginLogEvents(events, ar =>
{
try
{
loggingService.EndLogEvents(ar);
System.Diagnostics.Debug.WriteLine("3. Work - Back");
wh.Set();
}
catch (Exception ex)
{
}
}, null);
System.Diagnostics.Debug.WriteLine("2. Work - Waiting");
wh.WaitOne();
System.Diagnostics.Debug.WriteLine("4. Work - Finished");
}
else
{
wh.WaitOne();
}
}
}
#region IDisposable Members
public void Dispose()
{
EnqueueLogEvents(null);
worker.Join();
wh.Close();
}
#endregion
}
In my test it is essentially called like this:
//Inside of LogManager, get the LoggingService and set up the queue.
ILoggingService loggingService = GetTheLoggingService();
ProducerConsumerQueue loggingQueue = new ProducerConsumerQueue(loggingService);
//Inside of client code, get a logger and log with it
ILog logger = LogManager.GetLogger("test");
for (int i = 0; i < 100; i++)
{
logger.InfoFormat("logging message [{0}]", i);
}
Internally, logger/LogManager accumulates some number of logging messages (say 25) before adding that group of messages to the queue. Something like this:
internal void AddNewMessage(string message)
{
lock(logMessages)
{
logMessages.Add(message);
if (logMessages.Count >= 25)
{
ObservableCollection<LogMessage> messages = new ObservableCollection<LogMessage>(logMessages);
logMessages.Clear();
loggingQueue.EnqueueLogEvents(messages);
}
}
}
So, in this case I would expect to have 4 bursts of 25 messages each. Based on the Debug statements in my ProducerConsumerQueue code (maybe not the best way to debug this?), I would expect to see something like this:
Work - Sending 25 events
Work - Waiting
Work - Back
Work - Finished
Repeated 4 times.
Instead I am seeing something like this:
*1. Work - Sending 25 events
*2. Work - Waiting
*4. Work - Finished
*1. Work - Sending 25 events
*2. Work - Waiting
*3. Work - Back
*4. Work - Finished
*1. Work - Sending 25 events
*2. Work - Waiting
*3. Work - Back
*4. Work - Finished
*1. Work - Sending 25 events
*2. Work - Waiting
*3. Work - Back
*3. Work - Back
*4. Work - Finished
(Added leading * so that the lines would not be autonumbered by SO)
I guess I would have expected that, the queue would have allowed multiple bursts of messages to be added, but that it would completely process one burst (waiting on the acync call to complete) before processing the next burst. It doesn't seem to be doing this. It does not seem to be reliably waiting on the completion of the async call. I do have a call to Set in the EnqueueLogEvents, maybe that is cancelling the WaitOne from the Work method?
So, I have a few questions:
1. Does my explanation of what I am trying to accomplish make sense (is my explanation clear, not is it a good idea or not)?
Is what I am trying to (transmit - from the client - the messages from a single thread, in the order that they occurred, completely processing one set of messages at a time) a good idea?
Am I close?
Can it be done?
Should it be done?
Thanks for any help!
[EDIT]
After more investigation and thanks to Brian's suggestion, we were able to get this working. I have copied the modified code. The key is that we are now using the "wh" wait handle strictly for ProducerConsumerQueue functions. Rather than using wh to wait for the async call to complete, we are now waiting on res.AsyncWaitHandle, which is returned by the BeginLogEvents call.
internal class LoggingQueue : IDisposable
{
EventWaitHandle wh = new AutoResetEvent(false);
Thread worker;
readonly object locker = new object();
bool working = false;
Queue<ObservableCollection<LoggingService.LogEvent>> logEventQueue = new Queue<ObservableCollection<LoggingService.LogEvent>>();
LoggingService.ILoggingService loggingService;
internal LoggingQueue(LoggingService.ILoggingService loggingService)
{
this.loggingService = loggingService;
worker = new Thread(Work);
worker.Start();
}
internal void EnqueueLogEvents(ObservableCollection<LoggingService.LogEvent> logEvents)
{
lock (locker)
{
logEventQueue.Enqueue(logEvents);
//System.Diagnostics.Debug.WriteLine("EnqueueLogEvents calling Set");
wh.Set();
}
}
private void Work()
{
while (true)
{
ObservableCollection<LoggingService.LogEvent> events = null;
lock (locker)
{
if (logEventQueue.Count > 0)
{
events = logEventQueue.Dequeue();
if (events == null || events.Count == 0) return;
}
}
if (events != null && events.Count > 0)
{
//System.Diagnostics.Debug.WriteLine("1. Work - Sending {0} events", events.Count);
IAsyncResult res = loggingService.BeginLogEvents(events, ar =>
{
try
{
loggingService.EndLogEvents(ar);
//System.Diagnostics.Debug.WriteLine("3. Work - Back");
}
catch (Exception ex)
{
}
}, null);
//System.Diagnostics.Debug.WriteLine("2. Work - Waiting");
// Block until async call returns. We are doing this so that we can be sure that all logging messages
// are sent FROM the client in the order they were generated. ALSO, we don't want interleave blocks of logging
// messages from the same client by sending a new block of messages before the previous block has been
// completely processed.
res.AsyncWaitHandle.WaitOne();
//System.Diagnostics.Debug.WriteLine("4. Work - Finished");
}
else
{
wh.WaitOne();
}
}
}
#region IDisposable Members
public void Dispose()
{
EnqueueLogEvents(null);
worker.Join();
wh.Close();
}
#endregion
}
As I mentioned in my initial question and in my comments to Jon and Brian, I still don't know if doing all of this work is a good idea, but at least the code does what I wanted it to do. That means that I at least have the choice of doing it this way or some other way (such as restoring order after the fact) rather than not having the choice.

Can I suggest that there's a simple alternative to all this coordination? Have a sequence using a cheap monotonically increasing ID (e.g. with Interlocked.Increment()) so that no matter what order things happen at the client or server, you can regenerate the original ordering later on.
That should let you be efficient and flexible, sending whatever you want asynchronously without waiting for acknowledgement, but without losing the ordering.
Obviously that means the ID (or possibly a guaranteed-unique timestamp field) would need to be part of your WCF service, but if you control both ends that should be reasonably simple.

The reason you are getting that kind of sequencing is because you are trying to use the same wait handle that the producer-consumer queue is using for a different purpose. That is going to cause all kinds of chaos. At some point things will go from bad to worse and the queue will get live-locked eventually. You really should create a separate WaitHandle to wait for completion of the logging service. Or if the BeginLoggingEvents fits the standard pattern it will return a IAsyncResult that contains a WaitHandle that you can use instead of creating your own.
As a side note, I really do not like the producer-consumer pattern presented on the Albarahi website. The problem is that it is not safe for multiple consumers (obviously that is of no concern to you). And I say that with all due respect because I think his website is one of the best resources for multithreaded programming. If BlockingCollection is available to you then use that instead.