I'm having trouble getting DownloadProgressChangedEventHandler to fire. I understand that, worst case, the event handler should fire every 64Kb. The URL I'm trying to download data from creates 680Kb of XML data on the fly, but the handler does not fire at all.
Here's test code that demonstrates the problem. Unfortunately I cannot share the specific URL as it contains proprietary data.
static void Main(string[] args)
{
Console.WriteLine("Downloading data");
string url = "https://MyUrlThatRespondsWith680KbOfData";
string outputPath = #"C:\Temp\DeleteMe.xml";
using (WebClient webClient = new WebClient())
{
webClient.DownloadProgressChanged += new DownloadProgressChangedEventHandler(webClient_DownloadProgressChanged);
webClient.DownloadFile(url, outputPath);
}
Console.Write("Done");
Console.ReadKey(true);
}
static void webClient_DownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e)
{
Console.WriteLine("Download progress: " + e.BytesReceived);
}
Your code looks ok, but documentation says "This event is raised each time an asynchronous download makes progress" while you are using synchronous version of the download. Switch to use DownloadFileAsync.
My code was structured so that the WebClient was already used on a non-UI thread, so I extended WebClient to allow for a synchronous call to get events. I also extended it to allow for a custom connection timeout (I was calling a web service that could take quite a while to respond). The new method DownloadFileWithEvents internally calls DownloadFileAsync and properly blocks until the appropriate complete event is received.
Here's the code in case it's useful to anyone:
public class MyWebClient : WebClient
{
//time in milliseconds
private int timeout;
public int Timeout
{
get
{
return timeout;
}
set
{
timeout = value;
}
}
public MyWebClient(int timeout = 60000)
{
this.timeout = timeout;
this.DownloadFileCompleted += new System.ComponentModel.AsyncCompletedEventHandler(MyWebClient_DownloadFileCompleted);
}
EventWaitHandle asyncWait = new ManualResetEvent(false);
public void DownloadFileWithEvents(string url, string outputPath)
{
asyncWait.Reset();
Uri uri = new Uri(url);
this.DownloadFileAsync(uri, outputPath);
asyncWait.WaitOne();
}
void MyWebClient_DownloadFileCompleted(object sender, System.ComponentModel.AsyncCompletedEventArgs e)
{
asyncWait.Set();
}
protected override WebRequest GetWebRequest(Uri address)
{
var result = base.GetWebRequest(address);
result.Timeout = this.timeout;
return result;
}
}
Related
I have created a class that downloads files from a web client and the completed method returns a message in the console of the downloaded yes or no. This works, but is it also possible to read out how big the file is before downloading, and if the download is interrupted and not completely downloaded to delete this file?
How can I add this detection? I was thinking to read out the size of each file via readallbytes to get the size but so far without success
This is my code so far
public class FileDownload
{
private volatile bool _completed;
public bool DownloadCompleted
{
get
{
return _completed;
}
}
public async Task DownloadFile(string address, string location)
{
using (WebClient client = new WebClient())
{
Uri Uri = new Uri(address);
_completed = false;
client.Headers.Add("Authorization", await Header.getAuthorizationHeader());
client.DownloadFileCompleted += new System.ComponentModel.AsyncCompletedEventHandler(Completed);
client.DownloadProgressChanged += new DownloadProgressChangedEventHandler(DownloadProgress);
client.DownloadFileAsync(Uri, location);
}
}
private void Completed(object sender, System.ComponentModel.AsyncCompletedEventArgs e)
{
if (e.Cancelled == true)
{
Console.WriteLine("Download has been canceled.");
_completed = false;
}
else if (e.Error != null)
{
Console.WriteLine("Download error!");
_completed = false;
}
else
{
Console.WriteLine("Download completed!");
_completed = true;
}
}
private void DownloadProgress(object sender, DownloadProgressChangedEventArgs e)
{
// Displays the operation identifier, and the transfer progress.
Console.WriteLine("{0} downloaded {1} of {2} bytes. {3} % complete...",
(string)e.UserState,
e.BytesReceived,
e.TotalBytesToReceive,
e.ProgressPercentage);
}
}
// override DownloadFileAsync timeout to 10 second
public class WebClientWithTimeout : WebClient
{
//10 secs default
public int Timeout
{
get;
set;
} = 10000;
//the above will not work for async requests :(
//let's create a workaround by hiding the method
//and creating our own version of DownloadStringTaskAsync
public new async Task<string> DownloadStringTaskAsync(Uri address)
{
var t = base.DownloadStringTaskAsync(address);
if (await Task.WhenAny(t, Task.Delay(Timeout)).ConfigureAwait(false) != t) //time out!
{
CancelAsync();
}
return await t.ConfigureAwait(false);
}
//for sync requests
protected override WebRequest GetWebRequest(Uri uri)
{
var w = base.GetWebRequest(uri);
w.Timeout = Timeout; //10 seconds timeout
return w;
}
Instead of having complicated logic, you could download it as another name and only rename it to the correct name after the download completed. If all your temporary files followed a pattern, you could delete those files when you start the program, since they obviously were left over from a prior run that failed.
You may also want to delete the temporary file if you find out the download failed (you already know, sice you are printing it to console).
I am trying to just request the webpage by using webclient DownloadString method with proxy.
The following is my code:
using (WebClient wc = new WebClient())
{
try
{
wc.Encoding = Encoding.UTF8;
wc.Proxy = new WebProxy(myProxy);
string result = wc.DownloadString(url);
}
catch (Exception ex)
{
//log exception
}
}
I have few proxies and almost all of them work well with above code.
However, sometimes the response return time is very very long (over an hour) and I believe it's due to my proxy slow issue.
But I don't understand that why it didn't throw exception since the webclient class should have an default timeout (I search default timeout that it should be 100 seconds).
Therefore I want to ask how to avoid this case?
Thanks everyone.
It will be complicated as you need two threads - one for download and one for cancellation on timeout.
wc.Encoding = Encoding.UTF8;
wc.Proxy = new WebProxy(myProxy);
string result = null;
var waitCancel = new CancellationTokenSource();
wc.DownloadStringCompleted += (sender, e) =>
{
if (e.Cancelled) return;
waitCancel.Cancel();
result = e.Result;
};
wc.DownloadStringAsync(url);
waitCancel.Token.WaitHandle.WaitOne(30 * 1000);
if (waitCancel.IsCancellationRequested == false)
{
wc.CancelAsync();
}
Console.WriteLine("Result: " + result);
First you need to make use of the "new" HttpClient for .Net, below I will illustrate in two steps
HttpClient has handlers you can place in a chain and then control how the whole client behaves, so in this case we will add a progress handler that will dictate behavior in regard to the speed of the transfer
if the transfer is too slow then cancel it and start a new one with different proxy in ur case ...
Create the progress handler (to tell us how the progress is going .. If too slow then throw an exception)
Type :
ProgressMessageHandler
"https://msdn.microsoft.com/en-us/library/system.net.http.handlers.progressmessagehandler(v=vs.118).aspx"
Note : Package (System.Net.Http.Formatting.Extension) required
We will wrap it with an own object to erase testing and configuring to certain expected transfer rate
Code :
public class MyOwnProgressHandlerContainer
{
private readonly long _expectedBytesTransferred;
private long _lastRecoredBytesTransferred = 0;
public MyOwnProgressHandlerContainer(long expectedBytesTransferred)
{
_expectedBytesTransferred = expectedBytesTransferred;
}
public void ReceivedProgressHandler(object sender, HttpProgressEventArgs e)
{
// you can uses e.ProgressPercentage but this is calculated based on content length
// http header which is very much ignored nowadays
if (_lastRecoredBytesTransferred != 0 && e.BytesTransferred < (_lastRecoredBytesTransferred + _expectedBytesTransferred))
throw new Exception("Too Slow, Abort !");
_lastRecoredBytesTransferred = e.BytesTransferred;
}
public void SendProgressHandler(object sender, HttpProgressEventArgs e)
{
// write stuff to handle here when sending data (mainly for post or uploads)
Console.WriteLine("Sent data ...");
}
}
Create the HttpClient and inject the handlers that we wrapped :)
Type :
HttpClient
https://msdn.microsoft.com/en-us/library/system.net.http.httpclient(v=vs.118).aspx
Code :
// set expected rate
int expectedTransferRate = 25000; // AKA 2.5Mbps
// create own handler instance
MyOwnProgressHandlerContainer myHandler = new MyOwnProgressHandlerContainer(expectedTransferRate);
// create "ProgressMessageHandler"
ProgressMessageHandler progresshandler = new ProgressMessageHandler();
// Hookup event (send)
progresshandler.HttpSendProgress += myHandler.SendProgressHandler; // these are delegates so they can be a part of a stat-full class
// Hookup event (Receive)
progresshandler.HttpReceiveProgress += myHandler.ReceivedProgressHandler; // these are delegates so they can be a part of a stat-full class
// Create proxy handler
HttpClientHandler httpClientProxyHandler =
new HttpClientHandler
{
Proxy = new WebProxy("http://localhost:8888", false),
UseProxy = true
};
// Create client from factory with progress and "proxy" in your case
using (HttpClient httpClient = HttpClientFactory.Create(progresshandler, httpClientProxyHandler))
{
try
{
string downloadResult =
httpClient
// get result (progress handlers are notified based on sent / received data)
.GetAsync("https://httpbin.org/image/svg")
// could be a stream to read file content
.Result.Content.ReadAsStringAsync().Result;
Console.WriteLine(downloadResult);
}
catch (Exception)
{
// inspected if the exception is the same as the on u throw in MyOwnProgressHandlerContainer
throw;
}
}
WebClient doesnt have Timeout. You must extend it like this:
using System;
using System.Net;
namespace ConsoleApplication1
{
internal class Program
{
private static void Main(string[] args)
{
using (var webClient = new WebClientTimeOut())
{
webClient.Encoding = Encoding.UTF8;
webClient.Proxy = new WebProxy(myProxy);
try
{
var response = webClient.DownloadString("http://www.go1ogle.com");
Console.WriteLine(response);
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
}
}
public class WebClientTimeOut : WebClient
{
protected override WebRequest GetWebRequest(Uri uri)
{
var w = base.GetWebRequest(uri);
w.Timeout = 1000; // Timeout 1 second
return w;
}
}
}
I'm trying to get the current user's network download speed. After hitting a dead end with NetworkInterfaces and all I tried a solution I found online. I edited it a bit and it works great but it's not asynchronous.
public static void GetDownloadSpeed(this Label lbl)
{
double[] speeds = new double[5];
for (int i = 0; i < 5; i++)
{
int fileSize = 407; //Size of File in KB.
WebClient client = new WebClient();
DateTime startTime = DateTime.Now;
if (!Directory.Exists($"{CurrentDir}/tmp/speedtest"))
Directory.CreateDirectory($"{CurrentDir}/tmp/speedtest");
client.DownloadFile(new Uri("https://ajax.googleapis.com/ajax/libs/threejs/r69/three.min.js"), "/tmp/speedtest/three.min.js");
DateTime endTime = DateTime.Now;
speeds[i] = Math.Round((fileSize / (endTime - startTime).TotalSeconds));
}
lbl.Text = string.Format("{0}KB/s", speeds.Average());
}
That function is called within a timer at an interval of 2 minutes.
MyLbl.GetDownloadSpeed()
I've tried using WebClient.DownloadFileAsync but that just shows the unlimited symbol.My next try would be to use HttpClient but before I go on does anyone have a recommended way of getting the current users download speed asynchronously (without lagging the main GUI thread)?
As it was suggested you could make an async version of GetDownloadSpeed():
async void GetDownloadSpeedAsync(this Label lbl, Uri address, int numberOfTests)
{
string directoryName = #"C:\Work\Test\speedTest";
string fileName = "tmp.dat";
if (!Directory.Exists(directoryName))
Directory.CreateDirectory(directoryName);
Stopwatch timer = new Stopwatch();
timer.Start();
for (int i = 0; i < numberOfTests; ++i)
{
using (WebClient client = new WebClient())
{
await client.DownloadFileTaskAsync(address, Path.Combine(directoryName, fileName), CancellationToken.None);
}
}
lbl.Text == Convert.ToString(timer.Elapsed.TotalSeconds / numberOfTests);
}
WebClient class being relatively old does not have awaitable DownloadFileAsync().
EDITED
As it was correctly pointed out WebClient in fact has a task-based async method DownloadFileTaskAsync(), which i advise to use. The code below can still help addressing the case when async method returning Task is not provided.
We can fix it with the help of TaskCompletionSource<T>:
public static class WebClientExtensions
{
public static Task DownloadFileAwaitableAsync(this WebClient instance, Uri address,
string fileName, CancellationToken cancellationToken)
{
TaskCompletionSource<object> tcs = new TaskCompletionSource<object>();
// Subscribe for completion event
instance.DownloadFileCompleted += instance_DownloadFileCompleted;
// Setup cancellation
var cancellationRegistration = cancellationToken.CanBeCanceled ? (IDisposable)cancellationToken.Register(() => { instance.CancelAsync(); }) : null;
// Initiate asyncronous download
instance.DownloadFileAsync(address, fileName, Tuple.Create(tcs, cancellationRegistration));
return tcs.Task;
}
static void instance_DownloadFileCompleted(object sender, System.ComponentModel.AsyncCompletedEventArgs e)
{
((WebClient)sender).DownloadDataCompleted -= instance_DownloadFileCompleted;
var data = (Tuple<TaskCompletionSource<object>, IDisposable>)e.UserState;
if (data.Item2 != null) data.Item2.Dispose();
var tcs = data.Item1;
if (e.Cancelled)
{
tcs.TrySetCanceled();
}
else if (e.Error != null)
{
tcs.TrySetException(e.Error);
}
else
{
tcs.TrySetResult(null);
}
}
}
Try `await Task.Run(()=> { //your code });
Edit: #JustDevInc I still think you should use DownloadAsync. Task.Run(delegate) creates a new thread and you might want to avoid that. If you want, post some of your old code so we can try to fix it.
Edit: The first solution turned out to be the only one of the two working. DownloadFileAsync doesn't return task, so can't it awaited.
I can get an html code from web site this way:
public void Test()
{
WebClient client = new WebClient();
client.DownloadStringCompleted +=
new DownloadStringCompletedEventHandler(client_DownloadStringCompleted);
client.DownloadStringAsync(new Uri("http://testUrl.xml"));
}
void client_DownloadStringCompleted(object sender,
DownloadStringCompletedEventArgs e)
{
string html = e.Result;
//Now do something with the string...
}
But I need to get updated html each 30 seconds, so I wrote:
public void TestMain()
{
DispatcherTimer Timer = new DispatcherTimer()
{
Interval = TimeSpan.FromSeconds(30)
};
Timer.Tick += (s, t) =>
{
Test();
};
Timer.Start();
}
I changed the xml but I get the same html, what is wrong?
There's a cache included in the WebClient. If you request the same URI twice, the second time it will fetch the whole content directly from the cache.
There's no way to disable cache on WebClient, so you have two workarounds:
Use HttpWebRequest instead of WebClient
Add a random parameter to the URI:
client.DownloadStringAsync(new Uri("http://testUrl.xml?nocache=" + Guid.NewGuid()));
I'm a bit confused why the event doesn't fire when the file has downloaded.
The file it self downloads perfectly fine.
I'm assuming there is some error in the way I am using this, in that the event doesn't fire inside a loop.
Thanks for any help anyone can give me
class DownloadQueue
{
public List<string[]> DownloadItems { get; set; }
public int CurrentDownloads;
public int DownloadInProgress;
string url = #"http://www.google.co.uk/intl/en_uk/images/logo.gif";
bool downloadComplete;
public DownloadQueue()
{
CurrentDownloads = 0;
DownloadItems = new List<string[]>();
Console.Write("new download queue made");
}
public void startDownloading(int maxSimulatiousDownloads)
{
downloadComplete = true;
DownloadInProgress = 0;
WebClient client = new WebClient();
client.DownloadFileCompleted +=
new AsyncCompletedEventHandler(this.downloadCompleteMethod);
while(DownloadInProgress != DownloadItems.Count )
{
if (downloadComplete == true)
{
downloadComplete = false;
client.DownloadFileAsync(new Uri(DownloadItems.ElementAt(DownloadInProgress).ElementAt(0).ToString()), DownloadItems.ElementAt(DownloadInProgress).ElementAt(1).ToString());
}
}
Console.Write("all downloads completed");
}
private void downloadCompleteMethod(object sender, AsyncCompletedEventArgs e)
{
downloadComplete = true;
DownloadInProgress++;
Console.Write("file Downloaded");
}
}
Where are you waiting for the DownloadFileAsync() call to complete? How about something like
manualResetEvent AllDone = new mre(false)
just before console.WriteLine
AllDone.WaitOne()
And in download complete
if (interlocked.decrement(ref downloadComplete) == 0) { AllDone.Set(); }
Most likely there is no time for the message to come. I suspect if you put an Application.DoEvents at the end of your loop, it would start firing the events. It is less than ideal to use it, but with the design you have, I can't think of a better way.