Develop Reference

Task does not always return upon completion

So I am trying to build a program to control a machine. Communications with said machine is via a serial port for which I have written a driver. Continuous polling to the machine is necessary for status feedback etc. In my program I have a dedicated ExecutionEngine() class to handle serial send and receive. I also need to have two separate control sequences running, which I have put into methods RunSequenceA() and RunSequenceB() respectively. During normal operation, all three methods need to run until both control sequences finish, at which point the StopSequence() method is called. My issue is that sometimes, for whatever reason, the StopSequence() method is never called, leaving my ExecutionEngine() method in an infinite loop!
Code for ExecutionEngine():
private static void ExecutionEngine()
{
// Clear both lists in case they have old data
_commandList.Clear();
_pollingList.Clear();
// Poll while user has not yet clicked "STOP"
while (!_cTokenSource.Token.IsCancellationRequested)
{
// If there are commands to be sent, send them first
if (_commandList.Count > 0)
{
Command[] tempCommandArray;
lock (_commandList)
tempCommandArray = _commandList.ToArray();
foreach (var c in tempCommandArray)
{
if (_cTokenSource.Token.IsCancellationRequested)
break;
var response = SerialDriver.ComCycle(c.CommandBytes, _serialPort);
var success = CheckErrorReturn(response, false);
if (success)
{
AddPolling(c);
RemoveCommand(c);
}
}
}
// Do polling operation on applicable controllers
if (_pollingList.Count > 0)
{
Command[] tempPollingArray;
lock (_pollingList)
tempPollingArray = _pollingList.ToArray();
foreach (var c in tempPollingArray)
{
if (_cTokenSource.Token.IsCancellationRequested)
break;
var response = SerialDriver.ComCycle(c.PollBytes, _serialPort);
var success = ProcessPollReturn(response);
if (success)
{
c.FlagDone();
RemovePolling(c);
}
}
}
if (_commandList.Count + _pollingList.Count == 0)
{
// Will get stuck here if neither list gets new items added
Console.WriteLine("Bad place");
Thread.Sleep(500);
}
}
// Cancellation has been requested
lock (_commandList)
_commandList.Clear();
lock (_pollingList)
_pollingList.Clear();
ResetTriggers();
var endCommand = new Command("GL_SYSCMD", 0);
SerialDriver.ComCycle(endCommand.CommandBytes, _serialPort);
_serialPort.Close();
_vm.SequenceRunning = false;
return;
}
Code for running sequences:
private static async Task RunSequencesAsync()
{
var taskArray = new Task[2];
var a = new Action(RunSequenceA);
var b = new Action(RunSequenceB);
taskArray[0] = Task.Run(a);
taskArray[1] = Task.Run(b);
await Task.WhenAll(taskArray).ConfigureAwait(continueOnCapturedContext: false);
// Sometimes this never fires, WHY?
UpdateStatus("All done!");
StopSequence();
}
// Run A sequence
internal static void RunSequenceA()
{
if (_sequenceA1 != null && _sequenceA1.Count > 0)
{
foreach (var s in _sequenceA1)
{
if (_cTokenSource.Token.IsCancellationRequested)
return;
s.Execute();
if (s.Reference != null && TriggerStepCompleted != null)
TriggerStepCompleted(s, EventArgs.Empty);
}
// This part always fires
Console.WriteLine("Sequence A finished");
return;
}
else
return;
}
And finally, the methods to start and stop everything:
private static async Task StartSequenceAsync()
{
_serialPort.PortName = _vm.SelectedComPort;
_serialPort.Open();
_serialPort.DiscardInBuffer();
_serialPort.DiscardOutBuffer();
// Start
_cTokenSource = new CancellationTokenSource();
_vm.SequenceRunning = true;
var taskArray = new Task[2];
taskArray[0] = Task.Run(() => ExecutionEngine());
Thread.Sleep(50);
taskArray[1] = Task.Run(() => RunSequencesAsync());
await Task.WhenAll(taskArray).ConfigureAwait(continueOnCapturedContext: false);
}
private static void StopSequence()
{
_cTokenSource.Cancel();
}
To reiterate, the problem doesn't happen every time. In fact, most times the program runs fine. It seems that problems only arise if I manually call the StopSequence() method half way through execution. Then it's 50/50 as to whether the problem shows up. I'm pretty sure my issue is threading related, but not sure exactly what is going wrong. Any help pointing me in the right direction will be greatly appreciated!

Problem with passing constantly reading serial buffer data to another class

Long story short. ;
I have a class named Scope. And this class contains all logic for scope operations etc. It also starts backround thread that constantly read serial port data (in my case events was unreliable):
Thread BackgroundReader = new Thread(ReadBuffer);
BackgroundReader.IsBackground = true;
BackgroundReader.Start();
private void ReadBuffer()
{
SerialPort.DiscardInBuffer();
while (!_stopCapture)
{
int bufferSize = SerialPort.BytesToRead;
byte[] buffer = new byte[bufferSize];
if(bufferSize > 5)
{
SerialPort.Read(buffer, 0, bufferSize);
Port_DataReceivedEvent(buffer, null);
}
Thread.Sleep(_readDelay);
}
CurrentBuffer = null;
}
In Scope class there is a public field named Buffer
public byte[] Buffer
{
get
{
return CurrentBuffer;
}
}
And here is event fired while there is new data readed
private void Port_DataReceivedEvent(object sender, EventArgs e)
{
//populate buffer
Info(sender, null);
CurrentBuffer = ((byte[])sender);
foreach(byte data in CurrentBuffer)
{
DataBuffer.Enqueue(data);
}
if (DataBuffer.Count() > _recordLength)
{
GenerateFrame(DataBuffer.ToArray());
DataBuffer.Clear(); ;
}
}
To make code more manageable, I splitted it in several classes. One of this classes is for searching specific data pattern in current stream and create specific object from this data. This code works in way that send to serial port specific command and expect return frame. If reponse is not received or not ok, send is performed again and again until correct response arrives or there will be timeout. Response is expected to be in current buffer. Those strange string manipulation is for debug purposes.
public class GetAcknowledgedFrame
{
byte[] WritedData;
string lastEx;
string stringData;
public DataFrame WriteAcknowledged(Type SendType, Type ReturnType, JyeScope scope)
{
var stopwatch = new Stopwatch();
stopwatch.Restart();
while (stopwatch.ElapsedMilliseconds < scope.TimeoutTime)
{
try
{
if (SendType == typeof(GetParameters))
{
WriteFrame(new ScopeControlFrames.GetParameters(), scope.SerialPort);
}
else if(SendType == typeof(GetConfig))
{
WriteFrame(new ScopeControlFrames.GetConfig(), scope.SerialPort);
}
else if (SendType == typeof(EnterUSBScopeMode))
{
WriteFrame(new ScopeControlFrames.EnterUSBScopeMode(), scope.SerialPort);
}
return ReturnFrame(ReturnType, scope.Buffer, scope.TimeoutTime);
}
catch (InvalidDataFrameException ex)
{
lastEx = ex.Message;
System.Threading.Thread.Sleep(10);
}
}
stringData = "";
foreach (var data in scope.Buffer)
{
stringData += data + ",";
}
stringData.Remove(stringData.Length - 1);
throw new TimeoutException($"Timeout while waiting for frame acknowledge: " + SendType.ToString() + ", " + ReturnType.ToString() + Environment.NewLine+ "Add. err: "+lastEx);
}
private DataFrame ReturnFrame(Type FrameType, byte[] buffer, int timeoutTime)
{
if (FrameType == typeof(DataFrames.DSO068.CurrConfigDataFrame))
{
DataFrames.DSO068.CurrConfigDataFrame CurrConfig = new DataFrames.DSO068.CurrConfigDataFrame(buffer);
return CurrConfig;
}
else if (FrameType == typeof(DataFrames.DSO112.CurrConfigDataFrame))
{
DataFrames.DSO112.CurrConfigDataFrame CurrParam = new DataFrames.DSO112.CurrConfigDataFrame(buffer);
return CurrParam;
}
else if (FrameType == typeof(CurrParamDataFrame))
{
CurrParamDataFrame CurrParam = new CurrParamDataFrame(buffer);
return CurrParam;
}
else if (FrameType == typeof(DataBlockDataFrame))
{
DataBlockDataFrame CurrData = new DataBlockDataFrame(buffer);
return CurrData;
}
else if (FrameType == typeof(DataSampleDataFrame))
{
DataSampleDataFrame CurrData = new DataSampleDataFrame(buffer);
return CurrData;
}
else if (FrameType == typeof(ScopeControlFrames.ScopeReady))
{
ScopeControlFrames.ScopeReady ready = new ScopeControlFrames.ScopeReady(buffer);
return ready;
}
else
{
throw new InvalidOperationException("Wrong object type");
}
}
private bool WriteFrame(DataFrame frame, IStreamResource port)
{
WritedData = frame.Data;
port.Write(frame.Data, 0, frame.Data.Count());
return true;
}
}
From main class (and main thread) I call method in this class, for example:
var Ready = (ScopeControlFrames.ScopeReady)new GetAcknowledgedFrame().WriteAcknowledged
(typeof(ScopeControlFrames.EnterUSBScopeMode), typeof(ScopeControlFrames.ScopeReady), this);
The problem is when I pass "this" object (that has thread working in background) to my helper class. It seems like helper class not see changing data in this object. The problem started when I separate code of my helper class from main class.
My questions:
- I know that object are passed by reference, that means I think that when object is dynamically changing its state (in this case data buffer should changing while new data is received) all classes that has reference to this object are also seeing this changes. Maybe I'm missing something?
- I tried passing array (by ref), arrays are also reference types. But this not help me at all. Maybe I'm missing something?
I tried changing this class to static, it not helped.
Many thanks for help.

The code below;
Info(sender, null);
CurrentBuffer = ((byte[])sender);
is creating a new reference variable called CurrentBuffer. Any other code holding a reference 'pointer' to the CurrentBuffer value prior to this line of code will not get the new value of CurrentBuffer when its reset.

Multithreaded c# console app to scrape data from sites

I have written an app that goes through our own properties and scraps the data. To make sure I don't run through the same URLs, I am using a MySQL database to store the URL, flag it once its processed. All this was being done in a single thread and it's fine if I had only few thousand entries. But I have few hundred thousand entries that I need to parse so I need to make changes in the code (I am newbie in multithreading in general). I found an example and was trying to copy the style but doesn't seem to work. Anyone know what the issue is with the following code?
EDIT: Sorry didn't mean to make people guess the issue but was stupid of me to include the exception. Here is the exception
"System.InValidCastException: 'Specified cast is not valid.'"
When I start the process it collects the URLs from the database and then never hits DoWork method
//This will get the entries from the database
List<Mappings> items = bot.GetUrlsToProcess(100);
if (items != null)
{
var tokenSource = new CancellationTokenSource();
var token = tokenSource.Token;
Worker.Done = new Worker.DoneDelegate(WorkerDone);
foreach (var item in items)
{
urls.Add(item.Url);
WaitingTasks.Enqueue(new Task(id => new Worker().DoWork((int)id, item.Url, token), item.Url, token));
}
LaunchTasks();
}
static async void LaunchTasks()
{
// keep checking until we're done
while ((WaitingTasks.Count > 0) || (RunningTasks.Count > 0))
{
// launch tasks when there's room
while ((WaitingTasks.Count > 0) && (RunningTasks.Count < MaxRunningTasks))
{
Task task = WaitingTasks.Dequeue();
lock (RunningTasks) RunningTasks.Add((int)task.AsyncState, task);
task.Start();
}
UpdateConsole();
await Task.Delay(300); // wait before checking again
}
UpdateConsole(); // all done
}
static void UpdateConsole()
{
Console.Write(string.Format("\rwaiting: {0,3:##0} running: {1,3:##0} ", WaitingTasks.Count, RunningTasks.Count));
}
static void WorkerDone(int id)
{
lock (RunningTasks) RunningTasks.Remove(id);
}
public class Worker
{
public delegate void DoneDelegate(int taskId);
public static DoneDelegate Done { private get; set; }
public async void DoWork(object id, string url, CancellationToken token)
{
if (token.IsCancellationRequested) return;
Content obj;
try
{
int tries = 0;
bool IsUrlProcessed = true;
DateTime dtStart = DateTime.Now;
string articleDate = string.Empty;
try
{
ScrapeWeb bot = new ScrapeWeb();
SearchApi searchApi = new SearchApi();
SearchHits searchHits = searchApi.Url(url, 5, 0);
if (searchHits.Hits.Count() == 0)
{
obj = await bot.ReturnArticleObject(url);
if (obj.Code != HttpStatusCode.OK)
{
Console.WriteLine(string.Format("\r Status is {0}", obj.Code));
tries = itemfound.UrlMaxTries + 1;
IsUrlProcessed = false;
itemfound.HttpCode = obj.Code;
}
else
{
string title = obj.Title;
string content = obj.Contents;
string description = obj.Description;
Articles article = new Articles();
article.Site = url.GetSite();
article.Content = content;
article.Title = title;
article.Url = url.ToLower();
article.Description = description;
string strThumbNail = HtmlHelper.GetImageUrl(url, obj.RawResponse);
article.Author = HtmlHelper.GetAuthor(url, obj.RawResponse);
if (!string.IsNullOrEmpty(strThumbNail))
{
//This condition needs to be added to remove ?n=<number> from EP thumbnails
if (strThumbNail.Contains("?"))
{
article.ImageUrl = strThumbNail.Substring(0, strThumbNail.IndexOf("?")).Replace("http:", "https:");
}
else
article.ImageUrl = strThumbNail.Replace("http:", "https:");
}
else
{
article.ImageUrl = string.IsNullOrEmpty(strThumbNail) ? article.Url.GetDefaultImageUrls() : strThumbNail.Replace("http:", "https:");
}
articleDate = HtmlHelper.GetPublishDate(url, obj.RawResponse);
if (string.IsNullOrEmpty(articleDate))
article.Pubdate = DateTime.Now;
else
article.Pubdate = DateTime.Parse(articleDate);
var client = new Index(searchApi);
var result = client.Upsert(article);
itemfound.HttpCode = obj.Code;
if (result)
{
itemfound.DateCreated = DateTime.Parse(articleDate);
itemfound.DateModified = DateTime.Parse(articleDate);
UpdateItem(itemfound);
}
else
{
tries = itemfound.UrlMaxTries + 1;
IsUrlProcessed = false;
itemfound.DateCreated = DateTime.Parse(articleDate);
itemfound.DateModified = DateTime.Parse(articleDate) == null ? DateTime.Now : DateTime.Parse(articleDate);
UpdateItem(itemfound, tries, IsUrlProcessed);
}
}
}
else
{
tries = itemfound.UrlMaxTries + 1;
IsUrlProcessed = true;
itemfound.HttpCode = HttpStatusCode.OK;
itemfound.DateCreated = DateTime.Parse(articleDate);
itemfound.DateModified = DateTime.Parse(articleDate) == null ? DateTime.Now : DateTime.Parse(articleDate);
}
}
catch (Exception e)
{
tries = itemfound.UrlMaxTries + 1;
IsUrlProcessed = false;
itemfound.DateCreated = DateTime.Parse(articleDate);
itemfound.DateModified = DateTime.Parse(articleDate) == null ? DateTime.Now : DateTime.Parse(articleDate);
}
finally
{
DateTime dtEnd = DateTime.Now;
Console.WriteLine(string.Format("\r Total time taken to process items is {0}", (dtEnd - dtStart).TotalSeconds));
}
}
catch (Exception e)
{
Console.WriteLine(e);
}
Done((int)id);
}
}
All this code is based from Best multi-thread approach for multiple web requests this link. Can someone tell me how to get this approach running?

I think the problem is in the way you're creating your tasks:
new Task(id => new Worker().DoWork((int)id, item.Url, token), item.Url, token)
This Task constructor overload expected Action<object> delegate. That means id will be typed as object and you need to cast it back to something useful first.
Parameters
action
Type: System.Action<Object>
The delegate that represents the code to execute in the task.
state
Type: System.Object
An object representing data to be used by the action.
cancellationToken
Type: System.Threading.CancellationToken
-The CancellationToken that that the new task will observe.
You decided to cast it to int by calling (int)id, but you're passing item.Url as the object itself. I can't tell you 100% what the type of Url is but I don't expect Url-named property to be of type int.

Based on what #MarcinJuraszek said I just went back to my code and added an int as I couldn't find another way to resolve it. Here is the change I made
int i=0
foreach (var item in items)
{
urls.Add(item.Url);
WaitingTasks.Enqueue(new Task(id => new Worker().DoWork((string)id, item.Url, token), item.Url, token));
i++;
}

Task<List<T>> returns List.Count=0 but has 116 Elements

I really don't know what the problem is. I've an win8.1 App which is calling a WCF Method for getting some Data. When i debug the WCF it returns a Task<List<T>> with List.Count=116 Elements in the List but when the List (Task) "arrives" in my App it always says List.Count=0.
Tested with WCF-Testclient too - result:
And the result i get in my Win8 App:
Call:
private List<Person> getPersonList()
{
IService client = CustomServiceBinding.GetClientForIService();
try
{
if (client != null && ((IClientChannel)client).State == CommunicationState.Opened))
{
var personList = client.GetPersonStateList(terminalName);
personList.Wait(Timeout.Infinite);
if (personList.IsCompleted)
{
if (personList.Result == null || personList.Result.Count <= 0)
{
// i'm ALWAYS landing here because the List has no Elements
// but when the method in the WCF returns the List had 116 Elements
// where do they get lost?
}
else
{
foreach (var item in personList.Result)
{
loginTextbox.Text += "Yeah";
}
}
}
}
}
catch (CommunicationObjectFaultedException cofe)
{
ExceptionHandling(cofe);
if (client != null)
{
((IClientChannel)client).Abort();
}
}
catch (Exception ex)
{
ExceptionHandling(ex);
if (client != null)
{
((IClientChannel)client).Abort();
}
}
finally
{
if (client != null)
{
((IClientChannel)client).Close();
((IClientChannel)client).Dispose();
}
}
}
Method in WCF:
public async Task<List<Person>> GetPersonStateList(string terminal)
{
Task<List<Person>> t = Task.Run(() =>
{
List<Person> personList = new List<Person>();
try
{
//calling another WCF - returns 116 Elements
//validatePerson always returns true (hardcoded for testing)
currentPersonList = ServiceBinding.getCurrentPersonList().GetPersonEnumeration().Where(x => validatePerson(x));
}
catch (Exception ex)
{
currentPersonList = null;
ExceptionContainer myException = new ExceptionContainer("Message: " + ex.Message);
throw new FaultException<ExceptionContainer>(myException);
}
if (currentPersonList != null && currentPersonList.Count() > 0)
{
foreach (var item in currentPersonList)
{
TimeSpan ts;
var dailyTimes = item.GetDailyTimeEnumeration();
foreach (var day in dailyTimes)
{
ts = new TimeSpan(0, 0, (int)day.RawValue);
personList.Add(new Person() { Name = item.firstname + " " + item.lastname, dailyTime = ts });
}
}
}
else
{
// i never get in here
personList = null;
}
// at this point the personList has 116 valid Elements
return personList;
});
// this "t" has a List with 116 Elements at the return point
return await t;
}
What am i missing here? Did i forget something? Do i have to lock something (if yes, WHAT should i lock and where?)?
** EDITED **
Added more code examples in WCF Method and two Screenshots from the results i get in WCF and in Win8 App.
** EDITED **
Now i've changed a few things in coding in the given methods above.
Second one is not returning a Task<List<Person>> but just a List<Person>.
The first Method is shortened and looks like this:
private async void updateUI(object sender, object e)
{
try
{
List<Person> list = new List<Person>();
TM2K_Service client = IAGServiceBinding.GetClientForTM2K();
try
{
if (client != null)
{
string terminalName = getTerminalName().ToLower();
// asynchronous
var personList = await client.GetPersonStateList(terminalName);
if (personList == null || personList.Count <= 0)
{
// BUT I'M STILL STUCK HERE WITH A LIST WITH ZERO ELEMENTS (Count=0)
}
else
{
foreach (var item in personList.Result)
{
loginTextbox.Text += "Yeah";
}
}
}
}
catch (Exception x)
{
ExceptionHandling(x);
}
}
catch (Exception ex)
{
ExceptionHandling(ex);
}
}
I get the same Results when i'm calling the IIS published WCF as when i call the WCF-Testclient var personList Count=0.
** EDIT **
I found out it's a problem with deserializing of collections or generic lists. When you add a servicereference via visual studio (clicking) it sets default collection types for deserialization. I'm working with ChannelFactory so i have to set it as KnownType(typeof)? But i'm fairly new to this niveau of coding in WCF or setting programmatically servicereferences on clientside - how and where do i set these attributes? And i tried the known type IList<> too or an Array or an ObservableCollection. No chance to deserialize received data.
How and which attributes do i have to set for deserialization?

Not reach the code as expected

I have a telephony application, in which I want to invoke simultaneous calls,. Each call will occupy a channel or port. So I added all channels to a BlockingCollection. The application is a windows service.
Let's see the code.
public static BlockingCollection<Tuple<ChannelResource, string>> bc = new BlockingCollection<Tuple<ChannelResource, string>>();
public static List<string> list = new List<string>();// then add 100 test items to it.
The main application has the code:
while (true)
{
ThreadEvent.WaitOne(waitingTime, false);
lock (SyncVar)
{
Console.WriteLine("Block begin");
for (int i = 0; i < ports; i++)
{
var firstItem = list.FirstOrDefault();
if (bc.Count >= ports)
bc.CompleteAdding();
else
{
ChannelResource cr = OvrTelephonyServer.GetChannel();
bc.TryAdd(Tuple.Create(cr, firstItem));
list.Remove(firstItem);
}
}
pc.SimultaneousCall();
Console.WriteLine("Blocking end");
if (ThreadState != State.Running) break;
}
Now for the simultaneous call code:
public void SimultaneousCall()
{
Console.WriteLine("There are {0} channels to be processed.", bc.Count);
var workItemBlock = new ActionBlock<Tuple<ChannelResource, string>>(
workItem =>
{
ProcessEachChannel(workItem);
});
foreach (var workItem in bc.GetConsumingEnumerable())
{
bool result = workItemBlock.SendAsync(workItem).Result;
}
workItemBlock.Complete();
}
private void ProcessEachChannel(Tuple<ChannelResource, string> workItem)
{
ChannelResource cr = workItem.Item1;
string sipuri = workItem.Item2;
VoiceResource vr = workItem.Item1.VoiceResource;
workItem.Item1.Disconnected += new Disconnected(workItemItem1_Disconnected);
bool success = false;
try
{
Console.WriteLine("Working on {0}", sipuri);
DialResult dr = new DialResult();
// blah blah for calling....
}
catch (Exception ex)
{
Console.WriteLine("Exception: {0}", ex.Message);
}
finally
{
if (cr != null && cr.VoiceResource != null)
{
cr.Disconnect();
cr.Dispose();
cr = null;
Console.WriteLine("Release channel for item {0}.", sipuri);
}
}
}
The question was when I tested the application with 4 ports, I thought the code should reach at
Console.WriteLine("Blocking end");
However it was not. Please see the snapshot.
The application is just hanging on after releasing the last channel. I guess that I may use the blockingcollection incorrectly. Thanks for help.
UPDATE:
Even I changed the code by using POST action as below, the situation is still unchanged.
private bool ProcessEachChannel(Tuple<ChannelResource, string> workItem)
{
// blah blah to return true or false respectively.
public void SimultaneousCall()
{
Console.WriteLine("There are {0} channels to be processed.", bc.Count);
var workItemBlock = new ActionBlock<Tuple<ChannelResource, string>>(
workItem =>
{
bool success = ProcessEachChannel(workItem);
});
foreach (var workItem in bc.GetConsumingEnumerable())
{
workItemBlock.Post(workItem);
}
workItemBlock.Complete();
}

I believe the problem is that you never call bc.CompleteAdding(): the if means it would be called in ports + 1-th iteration of the loop, but the loop iterates only ports-times. Because of this, GetConsumingEnumerable() returns a sequence that never ends, which means the foreach inside SimultaneousCall() blocks forever.
I think the right solution is to call bc.CompleteAdding() after the for loop, not in an impossible condition inside it.