I have such code:
public void IssueOrders(List<OrderAction> actions)
{
foreach (var action in actions)
{
if (action is AddOrder)
{
uint userId = apiTransactions.PlaceOrder((action as AddOrder).order);
Console.WriteLine("order is placing userId = " + userId);
}
// TODO: implement other actions
}
// how to wait until OnApiTransactionsDataMessageReceived for all userId is received?
// TODO: need to update actions with received data here
}
private void OnApiTransactionsDataMessageReceived(object sender, DataMessageReceivedEventArgs e)
{
var dataMsg = e.message;
var userId = dataMsg.UserId;
apiTransactions.PlaceOrder is asynchronous so I receive userId as result but I will receive data in callback OnApiTransactionsDataMessageReceived.
So for example If I place 3 orders, i will receive 3 userId, for example 1, 3, and 4. Now I need to wait until data for all these userId is received.
userId is always increasing if this is important. This is almost integer numbers sequence, but some numbers may be ommited due parallel execution.
UPD Note:
IssueOrders can be executed parallel from different threads
callack may be called BEFORE PlaceOrder returns
UPD2
Likely I need to refactor PlaceOrder code below so I can guarantee that userId is known before "callback" is received:
public uint PlaceOrder(Order order)
{
Publisher pub = GetPublisher();
SchemeDesc schemeDesc = pub.Scheme;
MessageDesc messageDesc = schemeDesc.Messages[0]; //AddMM
FieldDesc fieldDesc = messageDesc.Fields[3];
Message sendMessage = pub.NewMessage(MessageKeyType.KeyName, "FutAddOrder");
DataMessage smsg = (DataMessage)sendMessage;
uint userId = counter.Next();
FillDataMessageWithPlaceOrder(smsg, order, userId);
System.Console.WriteLine("posting message dump: {0}", sendMessage);
pub.Post(sendMessage, PublishFlag.NeedReply);
sendMessage.Dispose();
return userId;
}
So I need to split PlaceOrder to two methods: userId CreateOrder and void PostOrder. This will guarantee that when callback is received I know userId.
I'd check out the ForkJoin method in the Reactive Framework. It will block until multiple async calls have completed.
Edit: It seems that ForkJoin() was only ever included in an experimental release of Rx. Here's a discussion of what you want based on Merge().
One of the most silly and working approaches would be:
public void IssueOrders(List<OrderAction> actions)
{
var userIds = new List<uint>();
lock(theHashMap)
theHashMap[userIds] = "blargh";
foreach (var action in actions)
{
if (action is AddOrder)
{
lock(userIds)
{
uint userId = apiTransactions.PlaceOrder((action as AddOrder).order);
Console.WriteLine("order is placing userId = " + userId);
userIds.Add(userId);
}
}
// TODO: implement other actions
}
// waiting:
do
{
lock(userIds)
if(userIds.Count == 0)
break;
Thread.Sleep(???); // adjust the time depending on how long you wait for a callback on average
}while(true);
lock(theHashMap)
theHashMap.Remove(userIds);
// now you have the guarantee that all were received
}
private Dictionary<List<uint>, string> theHashMap = new Dictionary<List<uint>,string>();
private void OnApiTransactionsDataMessageReceived(object sender, DataMessageReceivedEventArgs e)
{
var dataMsg = e.message;
var userId = dataMsg.UserId;
// do some other things
lock(theHashMap)
foreach(var list in theHashMap.Keys)
lock(list)
if(list.Remove(userId))
break;
}
but, this is quite crude approach.. Its hard to suggest anything more unless you explain what do yo umean by wait - as Jon asked in the comments. For example, you may might want to leave the IssueOrders, wait anywhere, and just be sure that the some extra job is done when all have arrived? Or maybe you cannot leave the IssueOrders unless all are received? etc..
Edit: please note that near ADD, the lock must be before PlaceOrder, or else, when the callback arrive hyper-fast, the callback may attempt to remove the ID before it is added. Also, note that this implementation is very naiive: the callback must search and lock through all the lists at each time. With a few additional dictionary/maps/indexes, it may be optimized much, but I did not do that here for readability.
In case you are able to change the API, consider to use Task Parallel Library, your code will get much easier with that.
Otherwise AutoResetEvent might help you:
private Dictionary<int, AutoResetEvent> m_Events = new ...;
public void IssueOrders(List<OrderAction> actions)
{
foreach (var action in actions)
{
if (action is AddOrder)
{
uint userId = apiTransactions.PlaceOrder((action as AddOrder).order);
// Attention: Race condition if PlaceOrder finishes
// before the MRE is created and added to the dictionary!
m_Events[userId] = new ManualResetEvent(false);
Console.WriteLine("order is placing userId = " + userId);
}
// TODO: implement other actions
}
WaitHandle.WaitAll(m_Events.Values);
// TODO: Dispose the created MREs
}
private void OnApiTransactionsDataMessageReceived(object sender, DataMessageReceivedEventArgs e)
{
var dataMsg = e.message;
var userId = dataMsg.UserId;
m_Events[userId].Set();
}
Related
I have a thread that handles the message receiving every 10 seconds and have another one write these messages to the database every minute.
Each message has a different sender which is named serialNumber in my case.
Therefore, I created a ConcurrentDictionary like below.
public ConcurrentDictionary<string, ConcurrentQueue<PacketModel>> _dicAllPackets;
The key of the dictionary is serialNumber and the value is the collection of 1-minute messages. The reason I want to collect a minute of data is instead of going database every 10 seconds is go once in every minute so I can reduce the process by 1/6 times.
public class ShotManager
{
private const int SLEEP_THREAD_FOR_FILE_LIST_DB_SHOOTER = 25000;
private bool ACTIVE_FILE_DB_SHOOT_THREAD = false;
private List<Devices> _devices = new List<Devices>();
public ConcurrentDictionary<string, ConcurrentQueue<PacketModel>> _dicAllPackets;
public ShotManager()
{
ACTIVE_FILE_DB_SHOOT_THREAD = Utility.GetAppSettings("AppConfig", "0", "ACTIVE_LIST_DB_SHOOT") == "1";
init();
}
private void init()
{
using (iotemplaridbContext dbContext = new iotemplaridbContext())
_devices = (from d in dbContext.Devices select d).ToList();
if (_dicAllPackets is null)
_dicAllPackets = new ConcurrentDictionary<string, ConcurrentQueue<PacketModel>>();
foreach (var device in _devices)
{
if(!_dicAllPackets.ContainsKey(device.SerialNumber))
_dicAllPackets.TryAdd(device.SerialNumber, new ConcurrentQueue<PacketModel> { });
}
}
public void Spinner()
{
while (ACTIVE_FILE_DB_SHOOT_THREAD)
{
try
{
Parallel.ForEach(_dicAllPackets, devicePacket =>
{
Thread.Sleep(100);
readAndShot(devicePacket);
});
Thread.Sleep(SLEEP_THREAD_FOR_FILE_LIST_DB_SHOOTER);
//init();
}
catch (Exception ex)
{
//init();
tLogger.EXC("Spinner exception for write...", ex);
}
}
}
public void EnqueueObjectToQueue(string serialNumber, PacketModel model)
{
if (_dicAllPackets != null)
{
if (!_dicAllPackets.ContainsKey(serialNumber))
_dicAllPackets.TryAdd(serialNumber, new ConcurrentQueue<PacketModel> { });
else
_dicAllPackets[serialNumber].Enqueue(model);
}
}
private void readAndShot(KeyValuePair<string, ConcurrentQueue<PacketModel>> keyValuePair)
{
StringBuilder sb = new StringBuilder();
if (keyValuePair.Value.Count() <= 0)
{
return;
}
sb.AppendLine($"INSERT INTO ......) VALUES(");
//the reason why I don't use while(TryDequeue(out ..)){..} is there's constantly enqueue to this dictionary, so the thread will be occupied with a single device for so long
for (int i = 0; i < 10; i++)
{
keyValuePair.Value.TryDequeue(out PacketModel packet);
if (packet != null)
{
/*
*** do something and fill the sb...
*/
}
else
{
Console.WriteLine("No packet found! For Device: " + keyValuePair.Key);
break;
}
}
insertIntoDB(sb.ToString()[..(sb.Length - 5)] + ";");
}
}
EnqueueObjectToQueue caller is from a different class like below.
private void packetToDictionary(string serialNumber, string jsonPacket, string messageTimeStamp)
{
PacketModel model = new PacketModel {
MachineData = jsonPacket,
DataInsertedAt = messageTimeStamp
};
_shotManager.EnqueueObjectToQueue(serialNumber, model);
}
How I call the above function is from the handler function itself.
private void messageReceiveHandler(object sender, MessageReceviedEventArgs e){
//do something...parse from e and call the func
string jsonPacket = ""; //something parsed from e
string serialNumber = ""; //something parsed from e
string message_timestamp = DateTime.Now().ToString("yyyy-MM-dd HH:mm:ss");
ThreadPool.QueueUserWorkItem(state => packetToDictionary(serialNumber, str, message_timestamp));
}
The problem is sometimes some packets are enqueued under the wrong serialNumber or repeat itself(duplicate entry).
Is it clever to use ConcurrentQueue in a ConcurrentDictionary like this?
No, it's not a good idea to use a ConcurrentDictionary with nested ConcurrentQueues as values. It's impossible to update atomically this structure. Take this for example:
if (!_dicAllPackets.ContainsKey(serialNumber))
_dicAllPackets.TryAdd(serialNumber, new ConcurrentQueue<PacketModel> { });
else
_dicAllPackets[serialNumber].Enqueue(model);
This little piece of code is riddled with race conditions. A thread that is running this code can be intercepted by another thread at any point between the ContainsKey, TryAdd, the [] indexer and the Enqueue invocations, altering the state of the structure, and invalidating the conditions on which the correctness of the current thread's work is based.
A ConcurrentDictionary is a good idea when you have a simple Dictionary that contains immutable values, you want to use it concurrently, and using a lock around each access could potentially create significant contention. You can read more about this here: When should I use ConcurrentDictionary and Dictionary?
My suggestion is to switch to a simple Dictionary<string, Queue<PacketModel>>, and synchronize it with a lock. If you are careful and you avoid doing anything irrelevant while holding the lock, the lock will be released so quickly that rarely other threads will be blocked by it. Use the lock just to protect the reading and updating of a specific entry of the structure, and nothing else.
Alternative designs
A ConcurrentDictionary<string, Queue<PacketModel>> structure might be a good option, under the condition that you never removed queues from the dictionary. Otherwise there is still space for race conditions to occur. You should use exclusively the GetOrAdd method to get or add atomically a queue in the dictionary, and also use always the queue itself as a locker before doing anything with it (either reading or writing):
Queue<PacketModel> queue = _dicAllPackets
.GetOrAdd(serialNumber, _ => new Queue<PacketModel>());
lock (queue)
{
queue.Enqueue(model);
}
Using a ConcurrentDictionary<string, ImmutableQueue<PacketModel>> is also possible because in this case the value of the ConcurrentDictionary is immutable, and you won't need to lock anything. You'll need to use always the AddOrUpdate method, in order to update the dictionary with a single call, as an atomic operation.
_dicAllPackets.AddOrUpdate
(
serialNumber,
key => ImmutableQueue.Create<PacketModel>(model),
(key, queue) => queue.Enqueue(model)
);
The queue.Enqueue(model) call inside the updateValueFactory delegate does not mutate the queue. Instead it creates a new ImmutableQueue<PacketModel> and discards the previous one. The immutable collections are not very efficient in general. But if your goal is to minimize the contention between threads, at the cost of increasing the work that each thread has to do, then you might find them useful.
I have been working on a webscraping project.
I am having two issues, one being presenting the number of urls processed as percentage but a far larger issue is that I can not figure out how I know when all the threads i am creating are totaly finished.
NOTE: I am aware of that the a parallel foreach once done moves on BUT this is within a recursive method.
My code below:
public async Task Scrape(string url)
{
var page = string.Empty;
try
{
page = await _service.Get(url);
if (page != string.Empty)
{
if (regex.IsMatch(page))
{
Parallel.For(0, regex.Matches(page).Count,
index =>
{
try
{
if (regex.Matches(page)[index].Groups[1].Value.StartsWith("/"))
{
var match = regex.Matches(page)[index].Groups[1].Value.ToLower();
if (!links.Contains(BaseUrl + match) && !Visitedlinks.Contains(BaseUrl + match))
{
Uri ValidUri = WebPageValidator.GetUrl(match);
if (ValidUri != null && HostUrls.Contains(ValidUri.Host))
links.Enqueue(match.Replace(".html", ""));
else
links.Enqueue(BaseUrl + match.Replace(".html", ""));
}
}
}
catch (Exception e)
{
log.Error("Error occured: " + e.Message);
Console.WriteLine("Error occured, check log for further details."); ;
}
});
WebPageInternalHandler.SavePage(page, url);
var context = CustomSynchronizationContext.GetSynchronizationContext();
Parallel.ForEach(links, new ParallelOptions { MaxDegreeOfParallelism = 25 },
webpage =>
{
try
{
if (WebPageValidator.ValidUrl(webpage))
{
string linkToProcess = webpage;
if (links.TryDequeue(out linkToProcess) && !Visitedlinks.Contains(linkToProcess))
{
ShowPercentProgress();
Thread.Sleep(15);
Visitedlinks.Enqueue(linkToProcess);
Task d = Scrape(linkToProcess);
Console.Clear();
}
}
}
catch (Exception e)
{
log.Error("Error occured: " + e.Message);
Console.WriteLine("Error occured, check log for further details.");
}
});
Console.WriteLine("parallel finished");
}
}
catch (Exception e)
{
log.Error("Error occured: " + e.Message);
Console.WriteLine("Error occured, check log for further details.");
}
}
NOTE that Scrape gets called multiple times(recursive)
call the method like this:
public Task ExecuteScrape()
{
var context = CustomSynchronizationContext.GetSynchronizationContext();
Scrape(BaseUrl).ContinueWith(x => {
Visitedlinks.Enqueue(BaseUrl);
}, context).Wait();
return null;
}
which in turn gets called like so:
static void Main(string[] args)
{
RunScrapper();
Console.ReadLine();
}
public static void RunScrapper()
{
try
{
_scrapper.ExecuteScrape();
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
}
my result:
How do I solve this?
(Is it ethical for me to answer a question about web page scraping?)
Don't call Scrape recursively. Place the list of urls you want to scrape in a ConcurrentQueue and begin processing that queue. As the process of scraping a page returns more urls, just add them into the same queue.
I wouldn't use just a string, either. I recommend creating a class like
public class UrlToScrape //because naming things is hard
{
public string Url { get; set; }
public int Depth { get; set; }
}
Regardless of how you execute this it's recursive, so you have to somehow keep track of how many levels deep you are. A website could deliberately generate URLs that send you into infinite recursion. (If they did this then they don't want you scraping their site. Does anybody want people scraping their site?)
When your queue is empty that doesn't mean you're done. The queue could be empty, but the process of scraping the last url dequeued could still add more items back into that queue, so you need a way to account for that.
You could use a thread safe counter (int using Interlocked.Increment/Decrement) that you increment when you start processing a url and decrement when you finish. You're done when the queue is empty and the count of in-process urls is zero.
This is a very rough model to illustrate the concept, not what I'd call a refined solution. For example, you still need to account for exception handling, and I have no idea where the results go, etc.
public class UrlScraper
{
private readonly ConcurrentQueue<UrlToScrape> _queue = new ConcurrentQueue<UrlToScrape>();
private int _inProcessUrlCounter;
private readonly List<string> _processedUrls = new List<string>();
public UrlScraper(IEnumerable<string> urls)
{
foreach (var url in urls)
{
_queue.Enqueue(new UrlToScrape {Url = url, Depth = 1});
}
}
public void ScrapeUrls()
{
while (_queue.TryDequeue(out var dequeuedUrl) || _inProcessUrlCounter > 0)
{
if (dequeuedUrl != null)
{
// Make sure you don't go more levels deep than you want to.
if (dequeuedUrl.Depth > 5) continue;
if (_processedUrls.Contains(dequeuedUrl.Url)) continue;
_processedUrls.Add(dequeuedUrl.Url);
Interlocked.Increment(ref _inProcessUrlCounter);
var url = dequeuedUrl;
Task.Run(() => ProcessUrl(url));
}
}
}
private void ProcessUrl(UrlToScrape url)
{
try
{
// As the process discovers more urls to scrape,
// pretend that this is one of those new urls.
var someNewUrl = "http://discovered";
_queue.Enqueue(new UrlToScrape { Url = someNewUrl, Depth = url.Depth + 1 });
}
catch (Exception ex)
{
// whatever you want to do with this
}
finally
{
Interlocked.Decrement(ref _inProcessUrlCounter);
}
}
}
If I was doing this for real the ProcessUrl method would be its own class, and it would take HTML, not a URL. In this form it's difficult to unit test. If it were in a separate class then you could pass in HTML, verify that it outputs results somewhere, and that it calls a method to enqueue new URLs it finds.
It's also not a bad idea to maintain the queue as a database table instead. Otherwise if you're processing a bunch of urls and you have to stop, you'd have start all over again.
Can't you add all tasks Task d to some type of concurrent collection you thread through all recursive calls (via method argument) and then simply call Task.WhenAll(tasks).Wait()?
You'd need an intermediate method (makes it cleaner) that launches the base Scrape call and passes in the empty task collection. When the base call returns you have in hand all tasks and you simply wait them out.
public async Task Scrape (
string url) {
var tasks = new ConcurrentQueue<Task>();
//call your implementation but
//change it so that you add
//all launched tasks d to tasks
Scrape(url, tasks);
//1st option: Wait().
//This will block caller
//until all tasks finish
Task.WhenAll(tasks).Wait();
//or 2nd option: await
//this won't block and will return to caller.
//Once all tasks are finished method
//will resume in WriteLine
await Task.WhenAll(tasks);
Console.WriteLine("Finished!"); }
Simple rule: if you want to know when something finishes, the first step is to keep track of it. In your current implementation you are essentially firing and forgetting all launched tasks...
I need to import customer related data from legacy DB and perform several transformations during the process. This means a single entry needs to perform additional "events" (synchronize products, create invoices, etc.).
My initial solution was a simple parallel approach.
It works okay, but sometimes it has issues. If the currently processed customers need to wait for the same type of events, their processing queues might got stuck and eventually time out, causing every underlying events to fail too (they depend on the one which failed). It doesn't happen all the time, yet it's annoying.
So I got another idea, work in batches. I mean not only limiting the number of customers being processed at the same time, but also the number of the events which are broadcasted to the queues. While searching around for ideas, I found this answer, which points to the TPL DataFlow.
I made a skeleton to get familiar with it. I set up a simple pipeline, but I'm a bit confused about the usage of Complete() and awaiting Completion().
The steps are the following
Make a list of numbers (the ids of the customers to be imported) - this is outside the import logic, it just there to be able to trigger the rest of the logic
Create a BatchBlock (to be able to limit the number of customers to be processed at the same time)
Create a single MyClass1 item based on the id (TransformBlock<int, MyClass1>)
Perform some logic and generate a collection of MyClass2 (TransformManyBlock<MyClass1, MyClass2>) - as example, sleep for 1 second
Perform some logic on every item of the collection (ActionBlock<MyClass2>) - as example, sleep for 1 second
Here's the full code:
public static class Program
{
private static void Main(string[] args)
{
var batchBlock = new BatchBlock<int>(2);
for (var i = 1; i < 10; i++)
{
batchBlock.Post(i);
}
batchBlock.Complete();
while (batchBlock.TryReceive(null, out var ids))
{
var transformBlock = new TransformBlock<int, MyClass1>(delegate (int id)
{
Console.WriteLine($"TransformBlock(id: {id})");
return new MyClass1(id, "Star Wars");
});
var transformManyBlock = new TransformManyBlock<MyClass1, MyClass2>(delegate (MyClass1 myClass1)
{
Console.WriteLine($"TransformManyBlock(myClass1: {myClass1.Id}|{myClass1.Value})");
Thread.Sleep(1000);
return GetMyClass22Values(myClass1);
});
var actionBlock = new ActionBlock<MyClass2>(delegate (MyClass2 myClass2)
{
Console.WriteLine($"ActionBlock(myClass2: {myClass2.Id}|{myClass2.Value})");
Thread.Sleep(1000);
});
transformBlock.LinkTo(transformManyBlock);
transformManyBlock.LinkTo(actionBlock);
foreach (var id in ids)
{
transformBlock.Post(id);
}
// this is the point when I'm not 100% sure
//transformBlock.Complete();
//transformManyBlock.Complete();
//transformManyBlock.Completion.Wait();
actionBlock.Complete();
actionBlock.Completion.Wait();
}
Console.WriteLine();
Console.WriteLine("Press any key to continue...");
Console.ReadKey();
}
private static IEnumerable<MyClass2> GetMyClass22Values(MyClass1 myClass1)
{
return new List<MyClass2>
{
new MyClass2(1, myClass1.Id+ " did this"),
new MyClass2(2, myClass1.Id+ " did that"),
new MyClass2(3, myClass1.Id+ " did this again")
};
}
}
public class MyClass1
{
public MyClass1(int id, string value)
{
Id = id;
Value = value;
}
public int Id { get; set; }
public string Value { get; set; }
}
public class MyClass2
{
public MyClass1(int id, string value)
{
Id = id;
Value = value;
}
public int Id { get; set; }
public string Value { get; set; }
}
So the point I struggle with is the end, where I'd need to call Complete() or wait for Completion. I can't seem to find the right combination. I'd like to see an output as follows:
TransformBlock(id: 1)
TransformBlock(id: 2)
TransformManyBlock(myClass1: 1|Star Wars)
TransformManyBlock(myClass1: 2|Star Wars)
ActionBlock(myClass2: 1|1 did this)
ActionBlock(myClass2: 2|1 did that)
ActionBlock(myClass2: 3|1 did this again)
ActionBlock(myClass2: 1|2 did this)
ActionBlock(myClass2: 2|2 did that)
ActionBlock(myClass2: 3|2 did this again)
TransformBlock(id: 3)
TransformBlock(id: 4)
TransformManyBlock(myClass1: 3|Star Wars)
TransformManyBlock(myClass1: 4|Star Wars)
ActionBlock(myClass2: 1|3 did this)
ActionBlock(myClass2: 2|3 did that)
ActionBlock(myClass2: 3|3 did this again)
ActionBlock(myClass2: 1|4 did this)
ActionBlock(myClass2: 2|4 did that)
ActionBlock(myClass2: 3|4 did this again)
[the rest of the items]
Press any key to exit...
Anyone can point me to the right direction?
You're almost there, you need to call Complete on the first block in the pipeline then await Completion on the last block. Then in your links you need to propagate completion like this:
private async static void Main(string[] args) {
var transformBlock = new TransformBlock<int, MyClass1>(delegate (int id)
{
Console.WriteLine($"TransformBlock(id: {id})");
return new MyClass1(id, "Star Wars");
});
var transformManyBlock = new TransformManyBlock<MyClass1, MyClass2>(delegate (MyClass1 myClass1)
{
Console.WriteLine($"TransformManyBlock(myClass1: {myClass1.Id}|{myClass1.Value})");
Thread.Sleep(1000);
return GetMyClass22Values(myClass1);
});
var actionBlock = new ActionBlock<MyClass2>(delegate (MyClass2 myClass2)
{
Console.WriteLine($"ActionBlock(myClass2: {myClass2.Id}|{myClass2.Value})");
Thread.Sleep(1000);
});
//propagate completion
transformBlock.LinkTo(transformManyBlock, new DataflowLinkOptions() { PropagateCompletion = true });
transformManyBlock.LinkTo(actionBlock, new DataflowLinkOptions() { PropagateCompletion = true});
foreach(var id in ids) {
transformBlock.Post(id);
}
//Complete the first block
transformBlock.Complete();
//wait for completion to flow to the last block
await actionBlock.Completion;
}
You can also incorporate the batch block into your pipeline and remove the need for the TryRecieve call but that seems like another part of your flow.
Edit
Example of propagating completion to multiple blocks:
public async static void Main(string[] args) {
var sourceBlock = new BufferBlock<int>();
var processBlock1 = new ActionBlock<int>(i => Console.WriteLine($"Block1 {i}"));
var processBlock2 = new ActionBlock<int>(i => Console.WriteLine($"Block2 {i}"));
sourceBlock.LinkTo(processBlock1);
sourceBlock.LinkTo(processBlock2);
var sourceBlockCompletion = sourceBlock.Completion.ContinueWith(tsk => {
if(!tsk.IsFaulted) {
processBlock1.Complete();
processBlock2.Complete();
} else {
((IDataflowBlock)processBlock1).Fault(tsk.Exception);
((IDataflowBlock)processBlock2).Fault(tsk.Exception);
}
});
//Send some data...
sourceBlock.Complete();
await Task.WhenAll(sourceBlockCompletion, processBlock1.Completion, processBlock2.Completion);
}
Trying to Request data from Web-service via Async call.
Started a similar Wait until event has finished post, in this case i was able to use a None-Async method. This is not the case now. There is no None-Async method available with current web-service. So how would i go a head and determine if the event from the Event "RespService_Send_Completed" has finished before initiate / move on with the next row in the loop. If event is successful the return from the webservice is a UUID in "e.Result".
foreach (string id in uuid)
{
WebRef.ResponderService RespService = new WebRef.ResponderService();
_uuid = id;
RespService.SendDataAsync(id);
RespService.SendCompleted += RespService_Send_Completed;
}
The code works fine when calling method
public void InvokeSend(string[] uuid)
with one value in string[] uuid. But when array containes more than one the code will fail.
public class SendReciveSoapData
{
private string _uuid { get; set; }
public void InvokeSend(string[] uuid)
{
foreach (string id in uuid)
{
WebRef.ResponderService RespService = new WebRef.ResponderService();
_uuid =id;
RespService.SendDataAsync(id);
RespService.SendCompleted += RespService_Send_Completed;
}
}
void RespService_Send_Completed(object sender, WebRef.CompletedEventArgs e)
{
//Saving Response Data to database
string SuccessID = e.Result;
string TransactionID = _uuid;
DataBase db = new DataBase();
db.UpdateResponseID(SuccessID, TransactionID);
}
}
The _uuid private field is likely cause of your issue.
Specifically, the foreach loop is not going to wait for the completed event to occur before continuing to the next id. The _uuid field will be overwritten with each iteration, and since the loop is going to be fast -- unless there are a lot (thousands) of calls to create, the loop will finish before any of the completed events are raised. There is simply no way to know what value _uuid will be set to when the completed event is rasied.
Most likely the loop will have finished and _uuid will be the last id when all of the completed events are raised.
I would need to know more about the API you are calling to know for certain, but xxxxAsync() methods usually return a Task<>. Regardless of if it is a Task<> or something else, save all of the return values in an array and remove from the array when completed.
Again, can't be more specific without more information about the API.
I solved the problem by, calling the "InvokeSend" -method from the event. Then working through the array that was sent to the Class, removing "Current" uuid-value after each iteration. Until array is empty.
public class SendReciveSoapData
{
private string[] UUID_array { get; set; }
private string CurrentUUID { get; set; }
public void InvokeSend(string[] uuid_array)
{
int len = uuid_array.Length;
if (len > 0)
{
CurrentUUID = uuid_array[0].ToString();
string strToRemove = CurrentUUID;
UUID_array = uuid_array.Where(val => val != strToRemove).ToArray();
invokeSend(CurrentUUID);
}
}
private void invokeSend(string uuid)
{
CurrentUUID=uuid;
WebRef.ResponderService RespService = new WebRef.ResponderService();
RespService.SendDataAsync(uuid);
RespService.SendCompleted += RespService_Send_Completed;
}
void RespService_Send_Completed(object sender, WebRef.CompletedEventArgs e)
{
//Saving Response Data to database
string SuccessID = e.Result;
string TransactionID = CurrentUUID;
DataBase db = new DataBase();
db.UpdateResponseID(SuccessID, TransactionID);
InvokeSend(UUID_array);
}
}
I have a tricky situation and was wondering if anyone could shed any light on the matter:
I have a blocking collection action worker that is called like this
ultraHash hash = new ultraHash(lblFolder.Text, (Action) (() => {
textBox1.Text = self.getMD5();
});
runQueue.addAction(hash);
now the "self" thing is there is pseudo code, and this is why, this is the worker (where runQueue is the BlockingCollection container class)
class ultraHash
{
string _filePath;
Action _onComplete;
string _md5;
public ultraHash(string filePath)
{
_filePath = filePath;
}
public ultraHash(string filePath, Action onComplete) : this(filePath) //constructor chaining
{
_onComplete = onComplete;
}
public override void action()
{
using (var md5 = MD5.Create())
{
try
{
using (var stream = File.OpenRead(_filePath))
{
_md5 = MakeHashString(md5.ComputeHash(stream));
if (_onComplete != null) _onComplete();
}
}
catch (IOException)
{
/***
* file is Busy
*/
}
}
}
public string getMD5()
{
return _md5;
}
private string MakeHashString(byte[] hashBytes)
{
StringBuilder hash = new StringBuilder(32);
foreach (byte b in hashBytes)
{
hash.Append(b.ToString("X2").ToLower());
}
return hashBytes.ToString();
}
}
now what I want to happen is for the anonymous method Action (() => {} in the first snippet to execute after the queue has completed and be able to contain the MD5 sum. I know there is thread safety issues here, so I understand I have to invoke this back to the parent thread where textBox1 resides, but at the moment I am just lost as to how to do this (is it even possible?)
EDIT
I don't normally "edit" answers but in case anyone else hits this issue this is the code I eventually used, thanks to the answer from usr. Notice the BeginInvoke
ultraHash hash = null;
hash = new ultraHash(lblFolder.Text, (Action) (() => {
this.BeginInvoke((Action) (() => {
txtMain.Text = hash.getMD5();
}));
}));
runQueue.addAction(hash);
also, if you are wondering why I did this, it is merely so that I can "watch" a folder for any file changes and then store meta data changes, since entire folders can be dragged in/out there needs to be a queueing mechanism in a FIFO queue, also the parent folders are hashed too, so any file alterations need to bubble up, lastly it may be that a file is locked when it is trying to be hashed in which case a FIFO queue can wait
Probably, you should pass the hash as an argument to your onComplete function. getMD5 does not need to exist.
If you insists on doing it just this way you need a little initialization dance:
ultraHash hash = null;
hash = new ultraHash(lblFolder.Text, (Action) (() => {
textBox1.Text = hash.getMD5();
});
The whole design of the ultraHash seems strange. Is it really necessary for that class to exist? A single static method that computes the hash should be enough.