I have my c# form running two threads one thread listens for data to come in and the other is processing the data so i can use it. for some reason once the process thread starts the listen thread isn't executed any more.
Thread th1 = new Thread(new ThreadStart(zeroMQConn.Listen));
th1.Start();
Thread th2 = new Thread(() => ProcessData(zeroMQConn));
th2.Start();
when i debug this it starts the th1 goes into it and then th2 starts and it never goes back to th1 and my data comes back null.
public void Listen()
{
while (true)
{
try
{
byte[] zmqBuffer = new byte[102400];
int messageLength;
lockForZMQ.EnterWriteLock();
messageLength = socket.Receive(zmqBuffer);
lockForZMQ.ExitWriteLock();
byte[] message = new byte[messageLength];
Buffer.BlockCopy(zmqBuffer, 0, message, 0, messageLength);
PriceBookData priceBook = PriceBookData.CreateBuilder().MergeFrom(message).Build();
double Type = priceBook.GetPb(0).QuoteType;
if (Type == 0.0)
{
lockForList.EnterWriteLock();
CachedBidBooks = priceBook;
lockForList.ExitWriteLock();
}
else
{
lockForList.EnterWriteLock();
CachedAskBooks = priceBook;
lockForList.ExitWriteLock();
}
}
catch (ZmqException ex)
{
MessageBox.Show(ex.Message);
}
}
}
public void ProcessData(object connection)
{
while (true)
{
priceBookData = ((ZeroMQClass)connection).GetPriceBook();
}
}
public List<PriceBookData> GetPriceBook()
{
List<PriceBookData> AskAndBid = new List<PriceBookData>();
lockForList.EnterWriteLock();
if (CachedAskBooks != null && CachedBidBooks != null)
{
AskAndBid.Add(CachedBidBooks);
AskAndBid.Add(CachedAskBooks);
CachedBidBooks = null;
CachedAskBooks = null;
lockForList.ExitWriteLock();
return AskAndBid;
}
lockForList.ExitWriteLock();
return null;
}
What you have here is a producer-consumer model, but you aren't properly synchronizing them. The problem is that rather than some sort of buffer or collection of data that's ready to be processed, you have a single variable, and you completely synchronize access to that variable. This means that the producer can't ever be working while the consumer is working.
The BlockingCollection<T> is a fantastic class whenever dealing with producer/consumer queues.
var queue = new BlockingCollection<PriceBookData>();
Task.Factory.StartNew(() =>
{
while (true)
{
byte[] zmqBuffer = new byte[102400];
int messageLength;
socket.Receive(zmqBuffer);
byte[] message = new byte[messageLength];
Buffer.BlockCopy(zmqBuffer, 0, message, 0, messageLength);
PriceBookData priceBook = PriceBookData.CreateBuilder().MergeFrom(message).Build();
double Type = priceBook.GetPb(0).QuoteType;
queue.Add(priceBook);
}
}, TaskCreationOptions.LongRunning);
Task.Factory.StartNew(() =>
{
foreach (var item in queue.GetConsumingEnumerable())
{
//do stuff with item
}
}, TaskCreationOptions.LongRunning);
Related
Please correct me if I have some errors in this logic (not some elegancy things like getting rid of constructor initialization and using Init method instead for Poll). I have not had experience with timer callbacks so far. The code is pretty self-explanatory, I hope. What confuses me a bit is some mix of async things (like connection client creation) and further code - though, I just reused IClient class, it's not mine):
public async Task<WaitForLoanActivationDto> WaitForLoanActivation(string userName, string accountGuid, int timeout)
{
const int dueTime = 0;
const int pollPeriod = 500;
Poll<WaitForLoanActivationDto> state = new Poll<WaitForLoanActivationDto>
{
Client = await _rpcConnectionPool.GetClientAsync(userName),
AutoResetEvent = new AutoResetEvent(false),
StartTime = DateTime.Now,
Timeout = timeout,
Parameters = new Variant[] { accountGuid },
Result = new WaitForLoanActivationDto { Active = false }
};
Timer timer = new Timer(new TimerCallback(WaitForLoanActivationCallback), state, dueTime, pollPeriod);
state.AutoResetEvent.WaitOne();
timer.Dispose(state.AutoResetEvent);
if (state.ThreadException != null)
{
throw state.ThreadException;
}
return state.Result;
}
private void WaitForLoanActivationCallback(object state)
{
Poll<WaitForLoanActivationDto> pollState = (Poll<WaitForLoanActivationDto>)state;
if (pollState.StartTime.AddMilliseconds(pollState.Timeout) >= DateTime.Now)
{
try
{
using (RPCReply reply = ResultHelper.Check(pollState.Client.ExecuteRemoteCommand(WaitForLoanActivationRpcName, pollState.Parameters)))
{
pollState.Result.Active = reply[2].IDList["Active"].AsBoolean().Value;
VariantList statusList = reply[2].IDList["statuses"].List;
if (statusList.Count > 0)
{
var statuses = CustomerInformationConverter.GetStatusesList(statusList);
pollState.Result.Statuses = statuses.ToArray();
}
if (pollState.Result.Active)
{
pollState.AutoResetEvent.Set();
}
}
}
catch (Exception ex)
{
pollState.Result = null;
pollState.ThreadException = ex;
pollState.AutoResetEvent.Set();
}
}
else
{
pollState.AutoResetEvent.Set();
}
}
Thank you guys.
#ckuri, based on your idea I came up with the code below. I have not used Task.Delay though, because as I understood it creates delay even if the task is successfully complete - after it. The objective of my case is to run RPC method each pollPeriod milliseconds during timeout milliseconds. It the method returns Active == false - keep polling, otherwise - return the result. RPC execution time might take more than pollPeriod milliseconds, so if it's already running - no sense to spawn another task.
public async Task<WaitForLoanActivationDto> WaitForLoanActivation(string userName, string accountGuid, int timeout)
{
var cancellationTokenSource = new CancellationTokenSource();
try
{
const int pollPeriod = 500;
IClient client = await _rpcConnectionPool.GetClientAsync(userName);
DateTime startTime = DateTime.Now;
WaitForLoanActivationDto waitForLoanActivationDto = null;
while (startTime.AddMilliseconds(timeout) >= DateTime.Now)
{
waitForLoanActivationDto = await Task.Run(() => WaitForLoanActivationCallback(client, accountGuid), cancellationTokenSource.Token);
if (waitForLoanActivationDto.Active)
{
break;
}
else
{
await Task.Delay(pollPeriod, cancellationTokenSource.Token);
}
}
return waitForLoanActivationDto;
}
catch (AggregateException ex)
{
cancellationTokenSource.Cancel();
throw ex.InnerException;
}
}
private WaitForLoanActivationDto WaitForLoanActivationCallback(IClient client, string accountGuid)
{
using (RPCReply reply = ResultHelper.Check(client.ExecuteRemoteCommand(WaitForLoanActivationRpcName, accountGuid)))
{
var waitForLoanActivationDto = new WaitForLoanActivationDto
{
Active = reply[2].IDList["Active"].AsBoolean().Value
};
VariantList statusList = reply[2].IDList["statuses"].List;
if (statusList.Count > 0)
{
var statuses = CustomerInformationConverter.GetStatusesList(statusList);
waitForLoanActivationDto.Statuses = statuses.ToArray();
}
return waitForLoanActivationDto;
}
}
I just finished my C# proxy containing 2 sockets (one client-socket receiving data from a certain client, one server-socket receiving data from a certain server).
- Method I)
The fully working receive/send procedure from a previous early-alpha version uses the following memory eating, quick and dirty method I won't implement again:
while (true)
{
if (ClientSocket.Available > 0)
{
// Received data from the game client
byte[] buf = new byte[ClientSocket.Available];
ClientSocket.Receive(buf);
Packet p = new Packet(buf);
Logger.Log(p.ID, LogType.PACKET);
// Forward re-encrypted data back to the official server
ServerSocket.Send(p.Encrypt());
}
if (ServerSocket.Available > 0)
{
// Received Data from the official server
byte[] buf = new byte[ServerSocket.Available];
ServerSocket.Receive(buf);
Packet p = new Packet(buf);
Logger.Log(p.ID, LogType.PACKET);
// Forward re-encrypted data back to the game client
ClientSocket.Send(p.Encrypt());
}
}
- Method II)
The not working receive/send procedure from the actual version uses the following memory friendly method split into 2 threads:
class ClientReceiveThread
{
public Thread T { get; set; }
public ClientReceiveThread(Socket ClientSocket, Socket ServerSocket)
{
T = new Thread(() =>
{
try
{
while (ClientSocket.Available > 0)
{
// Received data from the game client
byte[] buf = new byte[ClientSocket.Available];
ClientSocket.Receive(buf);
Packet p = new Packet(buf);
Logger.Log(p.ID, LogType.PACKET);
// Forward re-encrypted data back to the official server
ServerSocket.Send(p.Encrypt());
}
}
catch (Exception e)
{
ExceptionHandler.Handle(e);
}
});
T.Start();
}
}
class ServerReceiveThread
{
public Thread T { get; set; }
public ServerReceiveThread(Socket ClientSocket, Socket ServerSocket)
{
T = new Thread(() =>
{
try
{
while (ServerSocket.Available > 0)
{
// Received Data from the official server
byte[] buf = new byte[ServerSocket.Available];
ServerSocket.Receive(buf);
Packet p = new Packet(buf);
Logger.Log(p.ID, LogType.PACKET);
// Forward re-encrypted data back to the game client
ClientSocket.Send(p.Encrypt());
}
}
catch (Exception e)
{
ExceptionHandler.Handle(e);
}
});
T.Start();
}
}
Method I) does insofar work that both Client- and Serversockets receive data, while method II) only receives data from the ClientReceiveThread.
Why does ServerReceiveThread in method II) not receive data? It's basically the same code as in the while(true) loop, only ported to a seperate thread.
Any suggestions or answers are highly appreciated.
Thanks in advance!
Fixed it by avoiding the "Available" property:
class ClientReceiveThread
{
public Thread T { get; set; }
public ClientReceiveThread(Socket ClientSocket, Socket ServerSocket)
{
T = new Thread(() =>
{
try
{
byte[] buf = new byte[1024];
while (ClientSocket.Receive(buf) > 0)
{
// Received data from the game client
Packet p = new Packet(buf);
Logger.Log(p.ID, LogType.PACKET);
// Forward re-encrypted data back to the official server
ServerSocket.Send(p.Encrypt());
}
}
catch (Exception e)
{
ExceptionHandler.Handle(e);
}
});
T.Start();
}
}
class ServerReceiveThread
{
public Thread T { get; set; }
public ServerReceiveThread(Socket ClientSocket, Socket ServerSocket)
{
T = new Thread(() =>
{
try
{
byte[] buf = new byte[1024];
while (ServerSocket.Receive(buf) > 0)
{
// Received Data from the official server
Packet p = new Packet(buf);
Logger.Log(p.ID, LogType.PACKET);
// Forward re-encrypted data back to the game client
ClientSocket.Send(p.Encrypt());
}
}
catch (Exception e)
{
ExceptionHandler.Handle(e);
}
});
T.Start();
}
}
I have a telephony application, in which I want to invoke simultaneous calls,. Each call will occupy a channel or port. So I added all channels to a BlockingCollection. The application is a windows service.
Let's see the code.
public static BlockingCollection<Tuple<ChannelResource, string>> bc = new BlockingCollection<Tuple<ChannelResource, string>>();
public static List<string> list = new List<string>();// then add 100 test items to it.
The main application has the code:
while (true)
{
ThreadEvent.WaitOne(waitingTime, false);
lock (SyncVar)
{
Console.WriteLine("Block begin");
for (int i = 0; i < ports; i++)
{
var firstItem = list.FirstOrDefault();
if (bc.Count >= ports)
bc.CompleteAdding();
else
{
ChannelResource cr = OvrTelephonyServer.GetChannel();
bc.TryAdd(Tuple.Create(cr, firstItem));
list.Remove(firstItem);
}
}
pc.SimultaneousCall();
Console.WriteLine("Blocking end");
if (ThreadState != State.Running) break;
}
Now for the simultaneous call code:
public void SimultaneousCall()
{
Console.WriteLine("There are {0} channels to be processed.", bc.Count);
var workItemBlock = new ActionBlock<Tuple<ChannelResource, string>>(
workItem =>
{
ProcessEachChannel(workItem);
});
foreach (var workItem in bc.GetConsumingEnumerable())
{
bool result = workItemBlock.SendAsync(workItem).Result;
}
workItemBlock.Complete();
}
private void ProcessEachChannel(Tuple<ChannelResource, string> workItem)
{
ChannelResource cr = workItem.Item1;
string sipuri = workItem.Item2;
VoiceResource vr = workItem.Item1.VoiceResource;
workItem.Item1.Disconnected += new Disconnected(workItemItem1_Disconnected);
bool success = false;
try
{
Console.WriteLine("Working on {0}", sipuri);
DialResult dr = new DialResult();
// blah blah for calling....
}
catch (Exception ex)
{
Console.WriteLine("Exception: {0}", ex.Message);
}
finally
{
if (cr != null && cr.VoiceResource != null)
{
cr.Disconnect();
cr.Dispose();
cr = null;
Console.WriteLine("Release channel for item {0}.", sipuri);
}
}
}
The question was when I tested the application with 4 ports, I thought the code should reach at
Console.WriteLine("Blocking end");
However it was not. Please see the snapshot.
The application is just hanging on after releasing the last channel. I guess that I may use the blockingcollection incorrectly. Thanks for help.
UPDATE:
Even I changed the code by using POST action as below, the situation is still unchanged.
private bool ProcessEachChannel(Tuple<ChannelResource, string> workItem)
{
// blah blah to return true or false respectively.
public void SimultaneousCall()
{
Console.WriteLine("There are {0} channels to be processed.", bc.Count);
var workItemBlock = new ActionBlock<Tuple<ChannelResource, string>>(
workItem =>
{
bool success = ProcessEachChannel(workItem);
});
foreach (var workItem in bc.GetConsumingEnumerable())
{
workItemBlock.Post(workItem);
}
workItemBlock.Complete();
}
I believe the problem is that you never call bc.CompleteAdding(): the if means it would be called in ports + 1-th iteration of the loop, but the loop iterates only ports-times. Because of this, GetConsumingEnumerable() returns a sequence that never ends, which means the foreach inside SimultaneousCall() blocks forever.
I think the right solution is to call bc.CompleteAdding() after the for loop, not in an impossible condition inside it.
I require a Dataflow block which delays the forward of the message to the next block based on the timestamp in the message (LogEntry).
This is what i came up with but it feels not right. Any suggestions for improvements?
private IPropagatorBlock<LogEntry, LogEntry> DelayedForwardBlock()
{
var buffer = new ConcurrentQueue<LogEntry>();
var source = new BufferBlock<LogEntry>();
var target = new ActionBlock<LogEntry>(item =>
{
buffer.Enqueue(item);
});
Task.Run(() =>
{
LogEntry entry;
while (true)
{
entry = null;
if (buffer.TryPeek(out entry))
{
if (entry.UtcTimestamp < (DateTime.UtcNow - TimeSpan.FromMinutes(5)))
{
buffer.TryDequeue(out entry);
source.Post(entry);
}
}
}
});
target.Completion.ContinueWith(delegate
{
LogEntry entry;
while (buffer.TryDequeue(out entry))
{
source.Post(entry);
}
source.Complete();
});
return DataflowBlock.Encapsulate(target, source);
}
You could simply use a single TransformBlock that asynchronously waits out the delay using Task.Delay:
IPropagatorBlock<TItem, TItem> DelayedForwardBlock<TItem>(TimeSpan delay)
{
return new TransformBlock<TItem, TItem>(async item =>
{
await Task.Delay(delay);
return item;
});
}
Usage:
var block = DelayedForwardBlock<LogEntry>(TimeSpan.FromMinutes(5));
I am using following code to initiate Jobs.
List<Thread> threads = new List<Thread>();
List<Job> foundedJobs = new List<Job>();
public void StartAllJobs()
{
try
{
if (CanExecuteJobs == false) return;
var jobs = GetAllTypesImplementingInterface(typeof(Job)); // get all job implementations of this assembly.
var enumerable = jobs as Type[] ?? jobs.ToArray();
if (jobs != null && enumerable.Any()) // execute each job
{
Job instanceJob = null;
var index = 0;
foreach (var job in enumerable)
{
if (IsRealClass(job)) // only instantiate the job its implementation is "real"
{
try
{
instanceJob = (Job)Activator.CreateInstance(job); // instantiate job by reflection
foundedJobs.Add(instanceJob);
var thread = new Thread(new ThreadStart(instanceJob.ExecuteJob))
{
IsBackground = true,
Name = "SID" + index
}; // create thread for this job execution method
thread.Start();// start thread executing the job
threads.Add(thread);
index++;
}
catch (Exception ex)
{
App.Logger.Error(ex);
}
}
}
}
}
catch (Exception ex)
{
App.Logger.Error(ex);
}
}
How I can access Job class from thread later in some other class?
What I mean is that we create Thread like this
var thread = new Thread(new ThreadStart(instanceJob.ExecuteJob))
{
IsBackground = true,
Name = "SID" + index
};
So can I use thread somehow to access instanceJob?
Thanks!
Threads aren't really associated with any particular class, and its ThreadStart is not exposed, so given a thread, there isn't really a class context to be extracted.
Instead, what it looks like is that you need to create some dictionary with Thread as key, and your job as value. Then, given a Thread instance you can query the dictionary for the associated job instance.
Dictionary<Thread, Job> threads = new Dictionary<Thread, Job>();
List<Job> foundedJobs = new List<Job>();
public void StartAllJobs()
{
try
{
if (CanExecuteJobs == false) return;
var jobs = GetAllTypesImplementingInterface(typeof(Job)); // get all job implementations of this assembly.
var enumerable = jobs as Type[] ?? jobs.ToArray();
if (jobs != null && enumerable.Any()) // execute each job
{
Job instanceJob = null;
var index = 0;
foreach (var job in enumerable)
{
if (IsRealClass(job)) // only instantiate the job its implementation is "real"
{
try
{
instanceJob = (Job)Activator.CreateInstance(job); // instantiate job by reflection
foundedJobs.Add(instanceJob);
var thread = new Thread(new ThreadStart(instanceJob.ExecuteJob))
{
IsBackground = true,
Name = "SID" + index
}; // create thread for this job execution method
thread.Start();// start thread executing the job
threads.Add(thread, job);
index++;
}
catch (Exception ex)
{
App.Logger.Error(ex);
}
}
}
}
}
catch (Exception ex)
{
App.Logger.Error(ex);
}
}
in other places where you need to get job for a thread do:
job = threads[myThreadInstance]; // Where myThreadInstance is an instance of Thread class...
I would recommend using .NET 4's Task<T> to hold this, instead of using Thread. You can then directly return the Job within the Task<Job>:
public List<Task<Job>> StartAllJobs()
{
if (CanExecuteJobs == false) return;
var jobs = GetAllTypesImplementingInterface(typeof(Job)); // get all job implementations of this assembly.
return jobs.Where(IsRealClass)
.Select(job => (Job)Activator.CreateInstance(job))
.Select(job => Task.Factory.StartNew(() => { job.ExecuteJob(); return job; }))
.ToList();
}
This will give you a list of Tasks that will be completed once the job's ExecuteJob method is done, and the Task.Result will be the job that executed.