Develop Reference

Why the Task gets Cancelled

I'd like to spawn some threads and in each thread sequentially make calls to an API and aggregate the results (some sort of stress testing). Here is my attempt:
private async Task DoWork()
{
var allResponses = new List<int>();
for (int i = 0; i < 10; i++)
{
await Task.Run(() =>
{
var responses = Enumerable.Range(0, 50).Select(i => CallApiAndGetStatusCode());
allResponses.AddRange(responses);
});
}
// do more work
}
private int CallApiAndGetStatusCode()
{
try
{
var request = new HttpRequestMessage(httpMethod.Get, "some url");
var responseResult = httpClient.SendAsync(request).Result;
return (int)responseResult.StatusCode;
}
catch (Exception e)
{
logger.LogError(e, "Calling API failed");
}
}
However, this code always ends up the catch with the inner exception being {"A task was canceled."}. What am I doing wrong here?

There is no benefit to using either Enumerable.Range or .AddRange in your example, since you do not need the seeded number. Your code must be converted to async/await to avoid deadlocks and in doing so, you can simply loop inside of each task and avoid any odd interactions between Enumerable.Select and await:
private async Task DoWork()
{
var allTasks = new List<Task>(10);
var allResponses = new List<int>();
for (int i = 0; i < 10; i++)
{
allTasks.Add(Task.Run(async () =>
{
var tempResults = new List<int>();
for (int i = 0; i < 50; i++)
{
var result = await CallApiAndGetStatusCode();
if (result > 0) tempResults.Add(result);
}
if (tempResults.Count > 0)
{
lock (allResponses)
{
allResponses.AddRange(tempResults);
}
}
}));
}
await Task.WhenAll(allTasks);
// do more work
}
private async Task<int> CallApiAndGetStatusCode()
{
try
{
var request = new HttpRequestMessage(HttpMethod.Get, "some url");
var responseResult = await httpClient.SendAsync(request);
return (int)responseResult.StatusCode;
}
catch (Exception e)
{
logger.LogError(e, "Calling API failed");
}
return -1;
}
Note that this code is overly protective, locking the overall batch before adding the temp results.

I changed your code to this and work
async Task DoWork()
{
var allResponses = new List<int>();
for (int i = 0; i < 10; i++)
{
await Task.Run(() =>
{
var responses = Enumerable.Range(0, 3).Select(i => CallApiAndGetStatusCodeAsync());
allResponses.AddRange(responses.Select(x => x.Result));
});
}
// do more work
}
async Task<int> CallApiAndGetStatusCodeAsync()
{
try
{
var request = new HttpRequestMessage(HttpMethod.Get, "http://www.google.com");
var responseResult = await httpClient.SendAsync(request);
return (int)responseResult.StatusCode;
}
catch (Exception e)
{
logger.LogError(e, "Calling API failed");
return -1;
}
}

Trying to run 100 parallel tasks on an iteration doesn't work

Well, I'm trying to run a task 100 times on each run (with paralellism) but I can't manage this to work.
I'm trying to bruteforce an API, for this the API allows me to concatenate as many IDS as possible (without exceeding the timeout)
// consts:
// idsNum = 1000
// maxTasks = 100
// We prepare the ids that we will process (in that case 100,000)
var ids = PrepareIds(i * idsNum * maxTasks, idsNum, maxTasks);
// This was my old approach (didn't work)
//var result = await Task.WhenAll(ids.AsParallel().Select(async x => (await client.GetItems(x)).ToArray()));
// This is my new approach (also this didn't worked...)
var items = new List<item[]>();
ids.AsParallel().Select(x => client.GetItems(x).GetAwaiter().GetResult()).ForAll(item =>
{
//Console.WriteLine("processed!");
items.Add(item.ToArray());
});
var result = items.ToArray();
As you can see I put Console.WriteLine("processed!"); statment, in order to check if anything worked... But I can't manage this to work.
Those are my other methods:
private static IEnumerable<ulong[]> PrepareIds(int startingId, int idsNum = 1000, int maxTasks = 100)
{
for (int i = 0; i < maxTasks; i++)
yield return Range((ulong)startingId, (ulong)(startingId + idsNum)).ToArray();
}
And...
public static async Task<IEnumerable<item>> GetItems(this HttpClient client, ulong[] ids, Action notFoundCallback = null)
{
var keys = PrepareDataInKeys(type, ids); // This prepares the data to be sent to the API server
var content = new FormUrlEncodedContent(keys);
content.Headers.ContentType =
new MediaTypeHeaderValue("application/x-www-form-urlencoded") { CharSet = "UTF-8" };
client.DefaultRequestHeaders.ExpectContinue = false;
// We create a post request
var response = await client.PostAsync(new Uri(FILE_URL), content);
string contents = null;
JObject jObject;
try
{
contents = await response.Content.ReadAsStringAsync();
jObject = JsonConvert.DeserializeObject<JObject>(contents);
}
catch (Exception ex)
{
Console.WriteLine(ex);
Console.WriteLine(contents);
return null;
}
// Then we read the items from the parsed JObject
JArray items;
try
{
items = jObject
.GetValue("...")
.ToObject<JObject>()
.GetValue("...").ToObject<JArray>();
}
catch (Exception ex)
{
Console.WriteLine(ex);
return null;
}
int notFoundItems = 0;
int nonxxxItems = 0;
int xxxItems = 0;
var all = items.BuildEnumerable(notFoundCallback, item =>
{
if (item.Result != 1)
++notFoundItems;
else if (item.id != 5)
++nonxxxItems;
else
++xxxItems;
});
CrawledItems += ids.Length;
NotFoundItems += notFoundItems;
NonXXXItems += nonxxxItems;
XXXItems += xxxItems;
return all;
}
private static IEnumerable<item> BuildEnumerable(this JArray items, Action notFoundCallback, Action<item> callback = null)
{
foreach (var item in items)
{
item _item;
try
{
_item = new item(item.ToObject<JObject>());
callback?.Invoke(_item);
}
catch (Exception ex)
{
if (notFoundCallback == null)
Console.WriteLine(ex, Color.Red);
else
notFoundCallback();
continue;
}
yield return _item;
}
}
So as you can see I create 100 parallel post requests using an HttpClient. But I can't manage it to work.
So the thing that I want to achieve is to retrieve as many items as possible because I need to crawl +2,000,000,000 items.
But any breakpoint is triggered, neither any caption is updated on Console (I'm using Konsole project in order to print values at a fixed position on console), so any advice can be given there?

Message body empty on quickblox ios in xamarin native c#

i've translated the quickblox's sample app for Xamarin forms in Xamarin iOS and Android native.
Everything works "except" that the retrieving of the message fails.
I send the message from one client, and the event is catched from the other chat occupant:
XMPP: DispatchEvents ====> <message id="580735ed335fb760ae0017ec" xmlns="jabber:client" from="18029700-46533#chat.quickblox.com/1220770403-quickblox-68179" type="chat" to="18976912-46533#chat.quickblox.com"><extraParams xmlns="jabber:client"><save_to_history>1</save_to_history><dialog_id>5800aea1a28f9a1c1f000010</dialog_id><message_id>580735ed335fb760ae0017ec</message_id><date_sent>1476867565</date_sent></extraParams><body>test+message</body><thread>5800aea1a28f9a1c1f000010</thread></message>
XMPP: OnMessageReceived ====> From: 18029700 To: 18976912 Body: DateSent 1476867565 FullXmlMessage: <message id="580735ed335fb760ae0017ec" xmlns="jabber:client" from="18029700-46533#chat.quickblox.com/1220770403-quickblox-68179" type="chat" to="18976912-46533#chat.quickblox.com"><extraParams xmlns="jabber:client"><save_to_history>1</save_to_history><dialog_id>5800aea1a28f9a1c1f000010</dialog_id><message_id>580735ed335fb760ae0017ec</message_id><date_sent>1476867565</date_sent></extraParams><body>test+message</body><thread>5800aea1a28f9a1c1f000010</thread></message>
as you can see the Body part into the event OnMessageReceived is empty!
But the html part in the end of this snippet contains the message "test+message"
this is the PrivateChat Class
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using QbChat.Pcl.Repository;
using Quickblox.Sdk.GeneralDataModel.Models;
using Quickblox.Sdk.Modules.ChatXmppModule;
using Quickblox.Sdk.Modules.UsersModule.Models;
using UIKit;
using Xmpp.Im;
namespace KeepInTouch.iOS
{
public partial class PrivateChat : BaseChat
{
PrivateChatManager privateChatManager;
public PrivateChat(string dialogId, string nibname) : base(dialogId, nibname) { }
public async override void ViewWillAppear(bool animated)
{
base.ViewWillAppear(animated);
AppDelegate.R.NavController.SetNavigationBarHidden(true, true);
await MessageProvider.Reconnect();
}
public override void ViewDidLoad()
{
base_tb_chat = tb_chat;
base_txt_chat = txt_chat;
base.ViewDidLoad();
view_top.BackgroundColor = UIColor.FromPatternImage(UIImage.FromFile("navbar_top_kit.png"));
tb_chat.TableFooterView = new UIView();
IsBusyIndicatorVisible = true;
var dialog = Database.Instance().GetDialog(dialogId);
var opponentId = dialog.OccupantIds.Split(',').Select(int.Parse).First(id => id != QbChat.UserId);
ChatXmppClient xmpc = QbChat.QbProvider.GetXmppClient();
privateChatManager = xmpc.GetPrivateChatManager(opponentId, dialogId);
privateChatManager.MessageReceived += OnMessageReceived;
DialogName = dialog.Name;
//xmpc.MessageReceived += OnMessageReceived;
btn_chat.TouchUpInside += async delegate {
await SendMessageCommandExecute();
txt_chat.Text = "";
};
txt_chat.ShouldReturn += (textField) => {
textField.ResignFirstResponder();
return true;
};
txt_chat.EditingChanged += async delegate {
MessageText = txt_chat.Text;
await MessageProvider.Reconnect();
};
btn_back.TouchUpInside += delegate {
DismissViewController(false, null);
};
IsBusyIndicatorVisible = true;
Task.Factory.StartNew(async () => {
var users = await QbChat.QbProvider.GetUsersByIdsAsync(dialog.OccupantIds);
var opponentUser = users.FirstOrDefault(u => u.Id != QbChat.UserId);
if (opponentUser != null && opponentUser.BlobId.HasValue) {
await QbChat.QbProvider.GetImageAsync(opponentUser.BlobId.Value).ContinueWith((task, result) => {
//var bytes =
task.ConfigureAwait(true).GetAwaiter().GetResult();
}, TaskScheduler.FromCurrentSynchronizationContext());
}
opponentUsers = new List<User> { opponentUser };
await LoadMessages();
InvokeOnMainThread(() =>
IsBusyIndicatorVisible = false
);
});
}
async void OnMessageReceived(object sender, MessageEventArgs messageEventArgs)
{
if (messageEventArgs.MessageType == MessageType.Chat ||
messageEventArgs.MessageType == MessageType.Groupchat) {
string decodedMessage = System.Net.WebUtility.UrlDecode(messageEventArgs.Message.MessageText);
var messageTable = new MessageTable();
messageTable.SenderId = messageEventArgs.Message.SenderId;
messageTable.DialogId = messageEventArgs.Message.ChatDialogId;
messageTable.DateSent = messageEventArgs.Message.DateSent;
if (messageEventArgs.Message.NotificationType != 0) {
if (messageEventArgs.Message.NotificationType == NotificationTypes.GroupUpdate) {
if (messageEventArgs.Message.AddedOccupantsIds.Any()) {
var userIds = new List<int>(messageEventArgs.Message.AddedOccupantsIds);
userIds.Add(messageEventArgs.Message.SenderId);
var users = await QbChat.QbProvider.GetUsersByIdsAsync(string.Join(",", userIds));
var addedUsers = users.Where(u => u.Id != messageEventArgs.Message.SenderId);
var senderUser = users.First(u => u.Id == messageEventArgs.Message.SenderId);
messageTable.Text = senderUser.FullName + " added users: " + string.Join(",", addedUsers.Select(u => u.FullName));
} else if (messageEventArgs.Message.DeletedOccupantsIds.Any()) {
var userIds = new List<int>(messageEventArgs.Message.DeletedOccupantsIds);
var users = await QbChat.QbProvider.GetUsersByIdsAsync(string.Join(",", userIds));
messageTable.Text = string.Join(",", users.Select(u => u.FullName)) + " left this room";
}
//var dialogInfo = await QbChat.QbProvider.GetDialogAsync(messageEventArgs.Message.ChatDialogId);
//if (dialogInfo == null)
//{
// return;
//}
//var dialog = new DialogTable(dialogInfo);
//Database.Instance().SaveDialog(dialog);
}
} else {
messageTable.Text = decodedMessage;
}
await SetRecepientName(messageTable);
Messages.Add(messageTable);
InvokeOnMainThread(async () => {
tb_chat.ReloadData();
await ScrollList();
});
}
}
public async Task LoadMessages()
{
List<Message> messages;
try {
messages = await QbChat.QbProvider.GetMessagesAsync(dialogId);
} catch (Exception ex) {
Console.WriteLine(ex);
return;
}
if (messages != null) {
messages = messages.OrderBy(message => message.DateSent).ToList();
foreach (var message in messages) {
var chatMessage = new MessageTable();
chatMessage.DateSent = message.DateSent;
chatMessage.SenderId = message.SenderId;
chatMessage.MessageId = message.Id;
if (message.RecipientId.HasValue)
chatMessage.RecepientId = message.RecipientId.Value;
chatMessage.DialogId = message.ChatDialogId;
chatMessage.IsRead = message.Read == 1;
await SetRecepientName(chatMessage);
chatMessage.Text = System.Net.WebUtility.UrlDecode(message.MessageText);
InvokeOnMainThread(() =>
Messages.Add(chatMessage)
);
}
InvokeOnMainThread(async () => {
tb_chat.ReloadData();
await ScrollList();
});
}
}
async Task SendMessageCommandExecute()
{
var message = MessageText != null ? MessageText.Trim() : string.Empty;
if (!string.IsNullOrEmpty(message)) {
var m = new MessageTable();
m.SenderId = QbChat.UserId;
m.Text = message;
m.DialogId = dialogId;
m.RecepientFullName = "Me";
try {
await MessageProvider.Reconnect();
var encodedMessage = System.Net.WebUtility.UrlEncode(message);
privateChatManager.SendMessage(encodedMessage);
} catch (Exception ex) {
Console.WriteLine(ex);
return;
}
long unixTimestamp = DateTime.UtcNow.Ticks - new DateTime(1970, 1, 1).Ticks;
unixTimestamp /= TimeSpan.TicksPerSecond;
m.DateSent = unixTimestamp;
m.ID = Database.Instance().SaveMessage(m);
var dialog = Database.Instance().GetDialog(dialogId);
dialog.LastMessage = m.Text;
dialog.LastMessageSent = DateTime.UtcNow;
Database.Instance().SaveDialog(dialog, true);
Messages.Add(m);
MessageText = "";
InvokeOnMainThread(async () => {
tb_chat.ReloadData();
await ScrollList();
});
}
}
public override void DidReceiveMemoryWarning()
{
base.DidReceiveMemoryWarning();
}
}
}
long story short the sdk receive the message but apparently fail to parse the body message. fire the handler but if empty don't call the privateChatManager.MessageReceived.
could anyone help please?
Thank you

Multithreading issue ,Maybe a DeadLock using Foreach

Parallel.ForEach keeps on running and my program does not end. I am unable to trace where it goes after the first iteration. My guess is that gets a deadlock and keeps on doing context switching.
private void ReadInputFile()
{
var collection = new ConcurrentBag<PropertyRecord>();
var lines = System.IO.File.ReadLines(InputFileName);
int i = 0;
int RecordsCount = lines.Count();
Parallel.ForEach(lines, line =>
{
if (string.IsNullOrWhiteSpace(line))
{
return;
}
var tokens = line.Split(',');
var postalCode = tokens[0];
var country = tokens.Length > 1 ? tokens[1] : "england";
SetLabelNotifyTwoText(
string.Format(
"Reading PostCode {0} out of {1}"
i,
lines.Length));
var tempRecord = GetAllAddesses(postalCode, country);
if (tempRecord != null)
{
foreach (PropertyRecord r in tempRecord)
{
collection.Add(r);
}
}
});
}
private List<PropertyRecord> GetAllAddesses(
string postalCode,
string country = "england")
{
SetLabelNotifyText("");
progressBar1.Value = 0;
progressBar1.Update();
var records = new List<PropertyRecord>();
using (WebClient w = new WebClient())
{
var url = CreateUrl(postalCode, country);
var document = w.DownloadString(url);
var pagesCount = GetPagesCount(document);
if (pagesCount == null)
{
return null;
}
for (int i = 0; i < pagesCount; i++)
{
SetLabelNotifyText(
string.Format(
"Reading Page {0} out of {1}",
i,
pagesCount - 1));
url = CreateUrl(postalcode,country, i);
document = w.DownloadString(url);
var collection = Regex.Matches(
document,
"<div class=\"soldDetails\">(.|\\n|\\r)*?class=" +
"\"soldAddress\".*?>(?<address>.*?)(</a>|</div>)" +
"(.|\\n|\\r)*?class=\\\"noBed\\\">(?<noBed>.*?)" +
"</td>|</tbody>");
foreach (var match in collection)
{
var r = new PropertyRecord();
var bedroomCount = match.Groups["noBed"].Value;
if(!string.IsNullOrEmpty(bedroomCount))
{
r.BedroomCount = bedroomCount;
}
else
{
r.BedroomCount = "-1";
}
r.address = match.Groups["address"].Value;
var line = string.Format(
"\"{0}\",{1}",
r.address
r.BedroomCount);
OutputLines.Add(line);
Records.Add(r);
}
}
}
return Records;
}
It runs fine without Parallel.ForEach, but using Parallel.ForEach is in requirements.
I have debugged it and after returning from GetAllAdresses-method first time, Step Next button halts and it just keep on debugging in the background. It doesn't come back on any bookmark I have placed.

As you said in comments, your SetLabelNotifyText and SetLabelNotifyTwoText methods calls Control.Invoke.
For Control.Invoke to work, Main thread has to be free, but in your case you seem to block the main thread by invoking Parallel.ForEach in it.
Here is a minimal reproduction:
private void button1_Click(object sender, EventArgs e)
{
Parallel.ForEach(Enumerable.Range(1, 100), (i) =>
{
Thread.Sleep(10);//Simulate some work
this.Invoke(new Action(() => SetText(i)));
});
}
private void SetText(int i)
{
textBox1.Text = i.ToString();
}
Main thread waits for Parallel.ForEach and worker threads waits for Main thread, and thus results in deadlock.
How to fix: Don't use Invoke simply use BeginInvoke or don't block the MainThread.
If this isn't the case post sscce, that will be helpful for us

Change your code like this, to use async and await. This is the modern alternative to using BeginInvoke and other asynchronous code models.
private async Task ReadInputFile()
{
var collection = new ConcurrentBag<PropertyRecord>();
var lines = System.IO.File.ReadLines(InputFileName);
int i = 0;
int RecordsCount = lines.Count();
Parallel.ForEach(lines, line =>
{
if (string.IsNullOrWhiteSpace(line))
{
return;
}
var tokens = line.Split(',');
var postalCode = tokens[0];
var country = tokens.Length > 1 ? tokens[1] : "england";
SetLabelNotifyTwoText(
string.Format(
"Reading PostCode {0} out of {1}"
i,
lines.Length));
var tempRecord = await GetAllAddesses(postalCode, country);
if (tempRecord != null)
{
foreach (PropertyRecord r in tempRecord)
{
collection.Add(r);
}
}
});
}
private async Task<List<PropertyRecord>> GetAllAddesses(
string postalCode,
string country = "england")
{
SetLabelNotifyText("");
progressBar1.Value = 0;
progressBar1.Update();
var records = new List<PropertyRecord>();
using (WebClient w = new WebClient())
{
var url = CreateUrl(postalCode, country);
var document = await w.DownloadStringTaskAsync(url);
var pagesCount = GetPagesCount(document);
if (pagesCount == null)
{
return null;
}
for (int i = 0; i < pagesCount; i++)
{
SetLabelNotifyText(
string.Format(
"Reading Page {0} out of {1}",
i,
pagesCount - 1));
url = CreateUrl(postalcode,country, i);
document = await w.DownloadStringTaskAsync(url);
var collection = Regex.Matches(
document,
"<div class=\"soldDetails\">(.|\\n|\\r)*?class=" +
"\"soldAddress\".*?>(?<address>.*?)(</a>|</div>)" +
"(.|\\n|\\r)*?class=\\\"noBed\\\">(?<noBed>.*?)" +
"</td>|</tbody>");
foreach (var match in collection)
{
var r = new PropertyRecord();
var bedroomCount = match.Groups["noBed"].Value;
if(!string.IsNullOrEmpty(bedroomCount))
{
r.BedroomCount = bedroomCount;
}
else
{
r.BedroomCount = "-1";
}
r.address = match.Groups["address"].Value;
var line = string.Format(
"\"{0}\",{1}",
r.address
r.BedroomCount);
OutputLines.Add(line);
Records.Add(r);
}
}
}
return Records;
}
Then call it like this
ReadInputFile.Wait();
or, even better, is the caller is async,
await ReadInputFile();

How to create async http requests in a loop?

Yesterday I've found out how to create several async http requests without async/await. But today I need to do it in a loop: if some of responses don't satisfy some condition - I need to change a request for them and send these requests again. It may be repeated several times.
I've tried this code:
do
{
var loadingCoordinatesTasks = new List<Task<Terminal>>();
var totalCountOfTerminals = terminalPresetNode.ChildNodes.Count;
var uiTaskScheduler = TaskScheduler.FromCurrentSynchronizationContext();
foreach (var terminal in terminals.Except(_terminalsWithCoordinates))
{
var address = terminal.GetNextAddress();
var webRequest = (HttpWebRequest)WebRequest.Create(GeoCoder.GeoCodeUrl + address);
var webRequestTask = Task.Factory.FromAsync<WebResponse>(webRequest.BeginGetResponse,
webRequest.EndGetResponse,
terminal);
var parsingTask = webRequestTask.ContinueWith(antecedent =>
{
// Parse the response
});
loadingCoordinatesTasks.Add(parsingTask);
}
Task.Factory.ContinueWhenAll(loadingCoordinatesTasks.ToArray(), antecedents =>
{
foreach (var antecedent in antecedents)
{
var terminalWithCoordinates = antecedent.Result;
if (antecedent.Status == TaskStatus.RanToCompletion &&
!terminalWithCoordinates.Coordinates.AreUnknown)
{
_terminalsWithCoordinates.Add(terminalWithCoordinates);
_countOfProcessedTerminals++;
}
}
});
} while (_countOfProcessedTerminals < totalCountOfTerminals);
but is it possible to check the condition in while just after every single set of requests executed?

You can perform the check after increasing the count:
_countOfProcessedTerminals++;
if (_countOfProcessedTerminals >= totalCountOfTerminals)
{
break;
}
Is _countOfProcessedTerminals thread-safe though?

I manage to do it using recursion:
public void RunAgainFailedTasks(IEnumerable<Task<Terminal>> tasks)
{
Task.Factory.ContinueWhenAll(tasks.ToArray(), antecedents =>
{
var failedTasks = new List<Task<Terminal>>();
foreach (var antecedent in antecedents)
{
var terminal = antecedent.Result;
// Previous request was failed
if (terminal.Coordinates.AreUnknown)
{
string address;
try
{
address = terminal.GetNextAddress();
}
catch (FormatException) // No versions more
{
continue;
}
var getCoordinatesTask = CreateGetCoordinatesTask(terminal, address);
failedTasks.Add(getCoordinatesTask);
}
else
{
_terminalsWithCoordinates.Add(terminal);
}
}
if (failedTasks.Any())
{
RunAgainFailedTasks(failedTasks);
}
else
{
// Display a map
}
}, CancellationToken.None,
TaskContinuationOptions.None,
TaskScheduler.FromCurrentSynchronizationContext());
}
private Task<Terminal> CreateGetCoordinatesTask(Terminal terminal, string address)
{
var webRequest = (HttpWebRequest)WebRequest.Create(GeoCoder.GeoCodeUrl + address);
webRequest.KeepAlive = false;
webRequest.ProtocolVersion = HttpVersion.Version10;
var webRequestTask = Task.Factory.FromAsync<WebResponse>(webRequest.BeginGetResponse,
webRequest.EndGetResponse,
terminal);
var parsingTask = webRequestTask.ContinueWith(webReqTask =>
{
// Parse the response
});
return parsingTask;
}