Develop Reference

Finding out is user fully completed download or not in C#

I have created a class that downloads files from a web client and the completed method returns a message in the console of the downloaded yes or no. This works, but is it also possible to read out how big the file is before downloading, and if the download is interrupted and not completely downloaded to delete this file?
How can I add this detection? I was thinking to read out the size of each file via readallbytes to get the size but so far without success
This is my code so far
public class FileDownload
{
private volatile bool _completed;
public bool DownloadCompleted
{
get
{
return _completed;
}
}
public async Task DownloadFile(string address, string location)
{
using (WebClient client = new WebClient())
{
Uri Uri = new Uri(address);
_completed = false;
client.Headers.Add("Authorization", await Header.getAuthorizationHeader());
client.DownloadFileCompleted += new System.ComponentModel.AsyncCompletedEventHandler(Completed);
client.DownloadProgressChanged += new DownloadProgressChangedEventHandler(DownloadProgress);
client.DownloadFileAsync(Uri, location);
}
}
private void Completed(object sender, System.ComponentModel.AsyncCompletedEventArgs e)
{
if (e.Cancelled == true)
{
Console.WriteLine("Download has been canceled.");
_completed = false;
}
else if (e.Error != null)
{
Console.WriteLine("Download error!");
_completed = false;
}
else
{
Console.WriteLine("Download completed!");
_completed = true;
}
}
private void DownloadProgress(object sender, DownloadProgressChangedEventArgs e)
{
// Displays the operation identifier, and the transfer progress.
Console.WriteLine("{0} downloaded {1} of {2} bytes. {3} % complete...",
(string)e.UserState,
e.BytesReceived,
e.TotalBytesToReceive,
e.ProgressPercentage);
}
}
// override DownloadFileAsync timeout to 10 second
public class WebClientWithTimeout : WebClient
{
//10 secs default
public int Timeout
{
get;
set;
} = 10000;
//the above will not work for async requests :(
//let's create a workaround by hiding the method
//and creating our own version of DownloadStringTaskAsync
public new async Task<string> DownloadStringTaskAsync(Uri address)
{
var t = base.DownloadStringTaskAsync(address);
if (await Task.WhenAny(t, Task.Delay(Timeout)).ConfigureAwait(false) != t) //time out!
{
CancelAsync();
}
return await t.ConfigureAwait(false);
}
//for sync requests
protected override WebRequest GetWebRequest(Uri uri)
{
var w = base.GetWebRequest(uri);
w.Timeout = Timeout; //10 seconds timeout
return w;
}

Instead of having complicated logic, you could download it as another name and only rename it to the correct name after the download completed. If all your temporary files followed a pattern, you could delete those files when you start the program, since they obviously were left over from a prior run that failed.
You may also want to delete the temporary file if you find out the download failed (you already know, sice you are printing it to console).

Threading concept to improve the webbrowsers URL navigation process in C#

I have developed a simple Id checking windows forms with C# application to check a set of given Ids valid or not by passing to a webpage using webbrowser control and getting the reply and everything is working fine,its taking 40 - 60 seconds for 20 Ids.one by one.Now i want to speed up the same process using advance threading concept in C# .
Code is working fine i want to improve the performance using threading. any simple suggestion would be great help today
private void button2_Click(object sender, EventArgs e)
{
string url = "https://idscheckingsite.com";
WebBrowser wb = new WebBrowser();
wb.ScriptErrorsSuppressed = true;
wb.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(Final_DocumentCompleted);
wb.Navigate(url);
}
private void Final_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
WebBrowser wbs = sender as WebBrowser;
wbs.Document.GetElementById("pannumber").InnerText = ListsofIds[ids];
wbs.Document.GetElementById("frmType1").SetAttribute("value", "24Q");
HtmlElement btnlink = wbs.Document.GetElementById("clickGo1");
btnlink.InvokeMember("Click");
//string response = wbs.DocumentText;
wbs.DocumentCompleted -= new WebBrowserDocumentCompletedEventHandler(Final_DocumentCompleted);
wbs.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(Final_result);
}
private void Final_result(object sender, WebBrowserDocumentCompletedEventArgs e)
{
WebBrowser wbResult = sender as WebBrowser;
string status = wbResult.Document.GetElementById("status").InnerText;
string name = wbResult.Document.GetElementById("name").InnerText;
wbResult.DocumentCompleted -= new WebBrowserDocumentCompletedEventHandler(Final_result);
wbResult.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(Final_DocumentCompleted);
DataRow dr = dt.NewRow();
dr[0] = PANNumber[ids];
dr[1] = status;
dr[2] = name;
dt.Rows.Add(dr);
++ids;
if (ids < 20)
wbResult.Navigate(vurl);
else
{
dataGridView1.DataSource = dt;
}
}
Working fine but need to improve the performance to the max using advance C# threading concepts if any .

Here is my suggestion. When the button2 is clicked, a number of worker tasks are started. A reasonable number is 4, but you can try different numbers until you get the best performance. Each worker task will use its own WebBrowser control, and will invoke a subset of the ids. For example the worker task #0 will invoke the ids 4, 8, 12, 16, and 20, the worker task #1 will invoke 1, 5, 9, 13, and 17 etc. Then all worker tasks will be waited to complete, and then the DataGridView can be updated. There is no multithreading involved. Everything happens in the UI thread. No locking or other thread synchronization is required.
private async void button2_Click(object sender, EventArgs e)
{
string url = "https://idscheckingsite.com";
const int WORKER_TASKS_COUNT = 4;
var workerTasks = new Task[WORKER_TASKS_COUNT];
for (int i = 0; i < WORKER_TASKS_COUNT; i++)
{
workerTasks[i] = DoWorkAsync(i);
}
await Task.WhenAll(workerTasks);
dataGridView1.DataSource = dt;
async Task DoWorkAsync(int workerIndex)
{
using (var wb = new WebBrowser())
{
wb.ScriptErrorsSuppressed = true;
for (int i = 0; i < ListsofIds.Length; i++)
{
if (i % WORKER_TASKS_COUNT != workerIndex) continue;
wb.Navigate(url);
await wb; // await for the next DocumentCompleted
wb.Document.GetElementById("pannumber").InnerText = ListsofIds[i];
wb.Document.GetElementById("frmType1").SetAttribute("value", "24Q");
HtmlElement btnlink = wb.Document.GetElementById("clickGo1");
btnlink.InvokeMember("Click");
await wb; // await for the next DocumentCompleted
string status = wb.Document.GetElementById("status").InnerText;
string name = wb.Document.GetElementById("name").InnerText;
DataRow dr = dt.NewRow();
dr[0] = PANNumber[i];
dr[1] = status;
dr[2] = name;
dt.Rows.Add(dr);
}
}
}
}
The code above uses an interesting technique to simplify the navigation of the WebBrowser control. Instead of subscribing and unsubscribing manually to the DocumentCompleted event, it is doing it automatically by awaiting the WebBrowser control. Normally this is not possible, but we can make it possible by creating an extension method that returns a TaskAwaiter:
public static class WebBrowserExtensions
{
public static TaskAwaiter<Uri> GetAwaiter(this WebBrowser wb)
{
var tcs = new TaskCompletionSource<Uri>();
WebBrowserDocumentCompletedEventHandler handler = null;
handler = (_, e) =>
{
wb.DocumentCompleted -= handler;
tcs.TrySetResult(e.Url);
};
wb.DocumentCompleted += handler;
return tcs.Task.GetAwaiter();
}
}
Update: After using my code myself I found await wb to be a bit confusing, because the WebBrowser control has many events that could be awaited. So I made it more explicit and extensible be creating an async version of the event (instead of an awaiter):
public static class WebBrowserExtensions
{
public static Task<Uri> DocumentCompletedAsync(this WebBrowser wb)
{
var tcs = new TaskCompletionSource<Uri>();
WebBrowserDocumentCompletedEventHandler handler = null;
handler = (_, e) =>
{
wb.DocumentCompleted -= handler;
tcs.TrySetResult(e.Url);
};
wb.DocumentCompleted += handler;
return tcs.Task;
}
}
It can be used like this:
await wb.DocumentCompletedAsync();
Then it becomes trivial to create more extension methods like NavigatedAsync or DocumentTitleChangedAsync for example.
Update: Waiting endlessly is not very nice, so a timeout (expressed in milliseconds) could be added as an argument in the awaited extension method. Since the whole code is intended to run exclusively in the UI thread I used a System.Windows.Forms.Timer, although a CancellationToken would be propably more convenient in general. The code is a bit involved to avoid memory leaks, that could be an issue for an application intended to run for many hours, and do thousands web requests.
public static class WebBrowserExtensions
{
public static Task<Uri> DocumentCompletedAsync(this WebBrowser wb, int timeout)
{
var tcs = new TaskCompletionSource<Uri>();
WebBrowserDocumentCompletedEventHandler handler = null;
var timeoutRegistration = WithTimeout(tcs, timeout,
() => wb.DocumentCompleted -= handler);
handler = (_, e) =>
{
wb.DocumentCompleted -= handler;
timeoutRegistration.Unregister();
tcs.TrySetResult(e.Url);
};
wb.DocumentCompleted += handler;
return tcs.Task;
}
public static Task<Uri> DocumentCompletedAsync(this WebBrowser wb)
{
return wb.DocumentCompletedAsync(30000); // Default timeout 30 sec
}
private static TimeoutRegistration WithTimeout<T>(
TaskCompletionSource<T> tcs, int timeout, Action eventRemove)
{
if (timeout == Timeout.Infinite) return default;
var timer = new System.Windows.Forms.Timer();
timer.Tick += (s, e) =>
{
timer.Enabled = false;
timer = null;
eventRemove();
eventRemove = null;
tcs.SetException(new TimeoutException());
tcs = null;
};
timer.Interval = timeout;
timer.Enabled = true;
return new TimeoutRegistration(() =>
{
if (timer == null) return;
timer.Enabled = false;
// Make everything null to avoid memory leaks
timer = null;
eventRemove = null;
tcs = null;
});
}
private struct TimeoutRegistration
{
private Action _unregister;
public TimeoutRegistration(Action unregister)
{
_unregister = unregister;
}
public void Unregister()
{
if (_unregister == null) return;
_unregister();
_unregister = null;
}
}
}
Update: As a side note, I see that you are suppressing script errors by using wb.ScriptErrorsSuppressed = true. Are you aware that you can configure the Internet Explorer version emulated by the WebBrowser control? To make the control emulate the latest (and final) version of Internet Explorer, the version 11, add this code at the start of your program:
Registry.SetValue(#"HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_BROWSER_EMULATION",
AppDomain.CurrentDomain.FriendlyName, 11000); // IE11

C# async web browser runs very slow

this program reads a list of web site then saves them.
i found it runs good for the first 2 url requests. then goes very slow (about 5 min per request)
the time spend on row 1 and row 2 are only 2 second.
Then all other will be about 5 min each.
When i debug , i see it actually tooks long in wb.Navigate(url.ToString());
public static async Task<bool> test()
{
long totalCnt = rows.Count();
long procCnt = 0;
foreach (string url in rows)
{
procCnt++;
string webStr = load_WebStr(url).Result;
Console.WriteLine(DateTime.Now+ "["+procCnt + "/" + totalCnt+"] "+url);
}
return true;
}
public static async Task<string> load_WebStr(string url)
{
var tcs = new TaskCompletionSource<string>();
var thread = new Thread(() =>
{
EventHandler idleHandler = null;
idleHandler = async (s, e) =>
{
// handle Application.Idle just once
Application.Idle -= idleHandler;
// return to the message loop
await Task.Yield();
// and continue asynchronously
// propogate the result or exception
try
{
var result = await webBrowser_Async(url);
tcs.SetResult(result);
}
catch (Exception ex)
{
tcs.SetException(ex);
}
// signal to exit the message loop
// Application.Run will exit at this point
Application.ExitThread();
};
// handle Application.Idle just once
// to make sure we're inside the message loop
// and SynchronizationContext has been correctly installed
Application.Idle += idleHandler;
Application.Run();
});
// set STA model for the new thread
thread.SetApartmentState(ApartmentState.STA);
// start the thread and await for the task
thread.Start();
try
{
return await tcs.Task;
}
finally
{
thread.Join();
}
}
public static async Task<string> webBrowser_Async(string url)
{
string result = "";
using (var wb = new WebBrowser())
{
wb.ScriptErrorsSuppressed = true;
TaskCompletionSource<bool> tcs = null;
WebBrowserDocumentCompletedEventHandler documentCompletedHandler = (s, e) =>
tcs.TrySetResult(true);
tcs = new TaskCompletionSource<bool>();
wb.DocumentCompleted += documentCompletedHandler;
try
{
wb.Navigate(url.ToString());
// await for DocumentCompleted
await tcs.Task;
}
catch
{
Console.WriteLine("BUG!");
}
finally
{
wb.DocumentCompleted -= documentCompletedHandler;
}
// the DOM is ready
result = wb.DocumentText;
}
return result;
}

I recognize a slightly modified version of the code I used to answer quite a few WebBrowser-related questions. Was it this one? It's always a good idea to include a link to the original source.
Anyhow, the major problem in how you're using it here is perhaps the fact that you create and destroy an instance of WebBrowser control for every URL from your list.
Instead, you should be re-using a single instance of WebBrowser (or a pool of WebBrowser objects). You can find both versions here.

Asynchronous approach for long running sync methods

Hello guys I'm pretty new to the whole async stuff and it would be nice if you could give me some advice. I'm not really sure if my approach is OK.
Lets say I have a bus device that is reading data and it fires an event if a telegram is completed. Now I want to check each telegram for its length. If length == expectation -> OK if not try until is OK or timeout. But it want to check for length 1, 2 and 5 at the same time.
UPDATE:
OK I changed my example to an async approach but I still can't figure out how this should help me with my problem? OK on the plus side I don't have threads anymore that are blocked most of the time, but this wasn't my problem :(
So I try to explain in a different way. I want a async method that listens on the bus and returns the telegram that match the defined length
async Task<byte[]> GetTelegramAsync(int length, Timespan timeout)
I want to do something like this
Task<byte[]> t1 = GetTelegramAsync(1);
Task<byte[]> t2 = GetTelegramAsync(6);
Task<byte[]> t4 = GetTelegramAsync(4);
Task t4 = DoOtherStuffAsync();
DoStuff();
Task.WaitAll(AsyncRsp(t1), AsyncRsp(t2), AsyncRsp(t3), t4);
/* Output
Get telegram with length of 1
Get telegram with length of 6
Get telegram with length of 4
Start doing other async stuff
Sync stuff done...
Telegram found 0x00 0x01 0x02 0x03 0x04 0x05
Async stuff done...
Telegram found 0xFF
Telegram with length 4 not found
*/
Here is my first BusDevice class. A thread starts that listens on the bus, if a telegram is received an event fires.
class BusDeviceThread
{
private readonly Random _r = new Random();
private Thread _t;
public event EventHandler<TelegramReceivedArgs> TelegramReceived;
public void Connect()
{
_t = new Thread(FetchBusData)
{
Name = "FetchBusData",
Priority = ThreadPriority.Normal
};
_t.Start();
}
public void Close()
{
_t.Abort();
_t.Join();
}
private void FetchBusData()
{
while (true)
{
Thread.Sleep(_r.Next(100, 1000));
var buffer = new byte[_r.Next(1, 10)];
_r.NextBytes(buffer);
OnTelegramReceived(new TelegramReceivedArgs(buffer));
}
}
private void OnTelegramReceived(TelegramReceivedArgs e)
{
var handler = TelegramReceived;
if (handler != null) handler(this, e);
}
}
And here is the changed BusDevice class utilizing async await.
class BusDeviceAsync
{
private readonly Random _r = new Random();
public event EventHandler<TelegramReceivedArgs> TelegramReceived;
public async Task Connect(CancellationToken token)
{
while (!token.IsCancellationRequested)
{
var telegram = await FetchBusData();
OnTelegramReceived(new TelegramReceivedArgs(telegram.ToArray()));
}
}
private async Task<IEnumerable<byte>> FetchBusData()
{
await Task.Delay(_r.Next(100, 1000));
var buffer = new byte[_r.Next(1, 10)];
_r.NextBytes(buffer);
return buffer;
}
private void OnTelegramReceived(TelegramReceivedArgs e)
{
var handler = TelegramReceived;
if (handler != null) handler(this, e);
}
}
Like I said it doesn't help me with my problem, the
async Task<byte[]> GetTelegramAsync(int length, Timespan timeout)
implementation stays the same or do I miss a point here?
byte[] GetTelegram(int length, TimeSpan timeout)
{
byte[] telegram = null;
using (var resetEvent = new AutoResetEvent(false))
{
EventHandler<TelegramReceivedArgs> handler = (sender, e) =>
{
var t = e.Telegram;
if (Check(t, length))
{
telegram = t;
resetEvent.Set();
}
};
_d.TelegramReceived += handler;
resetEvent.WaitOne(timeout.Milliseconds);
_d.TelegramReceived -= handler;
}
return telegram ?? new byte[0];
}
async Task<byte[]> GetTelegramAsync(int length, TimeSpan timeout)
{
return await Task.Run(() => GetTelegram(length, timeout));
}

I updated my example, but I can't figure out the difference regarding
my problem. Well I certainly have fixed the blocked thread "problem".
This is not exactly what I meant, you're still using the pull model for your data (now with help of Task.Delay), not the push model (where the notification is coming asynchronously from the bus driver, as shown here).
Anyway, I think the following implementation might be what you're looking for. Note it doesn't explicitly use threads at all, beside for async I/O bus read simulation. Substitute the real device APM API for readBus:
using System;
using System.Threading;
using System.Threading.Tasks;
namespace ConsoleApplication
{
class Program
{
public class TelegramEventArg: EventArgs
{
public byte[] Data { get; set; }
}
public EventHandler<TelegramEventArg> TelegramEvent = delegate { };
async Task<byte[]> ReadTelegramAsync(int size, CancellationToken token)
{
var tcs = new TaskCompletionSource<byte[]>();
EventHandler<TelegramEventArg> handler = null;
bool subscribed = false;
handler = (s, e) =>
{
if (e.Data.Length == size)
{
this.TelegramEvent -= handler;
subscribed = false;
tcs.TrySetResult(e.Data);
}
};
this.TelegramEvent += handler;
try
{
subscribed = true;
using (token.Register(() => tcs.TrySetCanceled()))
{
await tcs.Task.ConfigureAwait(false);
return tcs.Task.Result;
}
}
finally
{
if (subscribed)
this.TelegramEvent -= handler;
}
}
async Task ReadBusAsync(CancellationToken token)
{
while (true)
{
// get data from the bus
var data = await Task.Factory.FromAsync(
(asyncCallback, asyncState) =>
readBus.BeginInvoke(asyncCallback, asyncState),
(asyncResult) =>
readBus.EndInvoke(asyncResult),
state: null).ConfigureAwait(false);
token.ThrowIfCancellationRequested();
this.TelegramEvent(this, new TelegramEventArg { Data = data });
}
}
// simulate the async bus driver with BeginXXX/EndXXX APM API
static readonly Func<byte[]> readBus = () =>
{
var random = new Random(Environment.TickCount);
Thread.Sleep(random.Next(1, 500));
var data = new byte[random.Next(1, 5)];
Console.WriteLine("A bus message of {0} bytes", data.Length);
return data;
};
static void Main(string[] args)
{
try
{
var program = new Program();
var cts = new CancellationTokenSource(Timeout.Infinite); // cancel in 10s
var task1 = program.ReadTelegramAsync(1, cts.Token);
var task2 = program.ReadTelegramAsync(2, cts.Token);
var task3 = program.ReadTelegramAsync(3, cts.Token);
var busTask = program.ReadBusAsync(cts.Token);
Task.WaitAll(task1, task2, task3);
Console.WriteLine("All telegrams received");
cts.Cancel(); // stop ReadBusAsync
}
catch (Exception ex)
{
while (ex is AggregateException)
ex = ex.InnerException;
Console.WriteLine(ex.Message);
}
Console.ReadLine();
}
}
}
Also, this scenario appears to be an ideal candidate for implementation using Reactive Extensions (Rx). As time allows, I'll show how to do that.

DownloadFileCompleted event doesn't fire inside while loop c# .net

I'm a bit confused why the event doesn't fire when the file has downloaded.
The file it self downloads perfectly fine.
I'm assuming there is some error in the way I am using this, in that the event doesn't fire inside a loop.
Thanks for any help anyone can give me
class DownloadQueue
{
public List<string[]> DownloadItems { get; set; }
public int CurrentDownloads;
public int DownloadInProgress;
string url = #"http://www.google.co.uk/intl/en_uk/images/logo.gif";
bool downloadComplete;
public DownloadQueue()
{
CurrentDownloads = 0;
DownloadItems = new List<string[]>();
Console.Write("new download queue made");
}
public void startDownloading(int maxSimulatiousDownloads)
{
downloadComplete = true;
DownloadInProgress = 0;
WebClient client = new WebClient();
client.DownloadFileCompleted +=
new AsyncCompletedEventHandler(this.downloadCompleteMethod);
while(DownloadInProgress != DownloadItems.Count )
{
if (downloadComplete == true)
{
downloadComplete = false;
client.DownloadFileAsync(new Uri(DownloadItems.ElementAt(DownloadInProgress).ElementAt(0).ToString()), DownloadItems.ElementAt(DownloadInProgress).ElementAt(1).ToString());
}
}
Console.Write("all downloads completed");
}
private void downloadCompleteMethod(object sender, AsyncCompletedEventArgs e)
{
downloadComplete = true;
DownloadInProgress++;
Console.Write("file Downloaded");
}
}

Where are you waiting for the DownloadFileAsync() call to complete? How about something like
manualResetEvent AllDone = new mre(false)
just before console.WriteLine
AllDone.WaitOne()
And in download complete
if (interlocked.decrement(ref downloadComplete) == 0) { AllDone.Set(); }

Most likely there is no time for the message to come. I suspect if you put an Application.DoEvents at the end of your loop, it would start firing the events. It is less than ideal to use it, but with the design you have, I can't think of a better way.