Develop Reference

How to fix download issues for some users using WebClient DownloadFileAsync?

I created a Windows Form Application in C# that users of my game modification can use to download updates automatically.
The 'Launcher', as I call it, uses WebClient to download the updates. But the first release of the mod is very big (2,7 GB zipped). The launcher works perfect for me and most users, but for some users the extraction of the zip file logs an error where the file is corrupted and not readable.
I searched already on stack, and it is possible that the file might be corrupted or truncated due to bad internet connection. But how do I build in a method that fix that problem?
//Start downloading file
using (WebClient webClient = new WebClient())
{
webClient.DownloadFileCompleted += new
AsyncCompletedEventHandler(Client_DownloadFileCompleted);
webClient.DownloadProgressChanged += new
DownloadProgressChangedEventHandler(Client_DownloadProgressChanged);
webClient.DownloadFileAsync(new Uri("http://www.dagovaxgames.com/api/downloads/+ patch.path), downloadPath);
}
private void Client_DownloadFileCompleted(object sender, AsyncCompletedEventArgs e)
{
//install the update
InstallUpdate();
}
private void InstallUpdate()
{
var file = currentPatchPath;
//get the size of the zip file
fileInfo = new FileInfo(file);
_fileSize = fileInfo.Length;
installBackgroundWorker = new BackgroundWorker();
installBackgroundWorker.DoWork += ExtractFile_DoWork;
installBackgroundWorker.ProgressChanged += ExtractFile_ProgressChanged;
installBackgroundWorker.RunWorkerCompleted += ExtractFile_RunWorkerCompleted;
installBackgroundWorker.WorkerReportsProgress = true;
installBackgroundWorker.RunWorkerAsync();
}
EDIT, just showing install code so that you know I am using a backgroundworker to extract the zip.

Common approach here is to download large file in small chunks and put them together on client after completion. Using this approach you can: 1. run few downloads in parallel and 2. in case of problem with network you don't have to download entire file again, just download incomplete chunks.

I faced a similar issue many years ago and created a subclass of WebClient that uses the DownloadProgressChanged event and a timer to abort downloads that get hung up and cancels the download smoother than the underlying internet transport layer does. The code also supports a callback to notify the calling code of progress. I found that sufficient to smoothly handle occasional hiccups downloading 1GB-ish files.
The idea of breaking your download into multiple pieces also has merit. You could leverage a library such as 7-Zip to both create the chunks and piece them back together (many compression libraries have that feature; I'm personally most familiar with 7-Zip).
Here's the code I wrote. Feel free to use and/or modify in any way that's helpful to you.
public class JWebClient : WebClient, IDisposable
{
public int Timeout { get; set; }
public int TimeUntilFirstByte { get; set; }
public int TimeBetweenProgressChanges { get; set; }
public long PreviousBytesReceived { get; private set; }
public long BytesNotNotified { get; private set; }
public string Error { get; private set; }
public bool HasError { get { return Error != null; } }
private bool firstByteReceived = false;
private bool success = true;
private bool cancelDueToError = false;
private EventWaitHandle asyncWait = new ManualResetEvent(false);
private Timer abortTimer = null;
private bool isDisposed = false;
const long ONE_MB = 1024 * 1024;
public delegate void PerMbHandler(long totalMb);
public delegate void TaggedPerMbHandler(string tag, long totalMb);
public event PerMbHandler NotifyMegabyteIncrement;
public event TaggedPerMbHandler NotifyTaggedMegabyteIncrement;
public JWebClient(int timeout = 60000, int timeUntilFirstByte = 30000, int timeBetweenProgressChanges = 15000)
{
this.Timeout = timeout;
this.TimeUntilFirstByte = timeUntilFirstByte;
this.TimeBetweenProgressChanges = timeBetweenProgressChanges;
this.DownloadFileCompleted += new System.ComponentModel.AsyncCompletedEventHandler(MyWebClient_DownloadFileCompleted);
this.DownloadProgressChanged += new DownloadProgressChangedEventHandler(MyWebClient_DownloadProgressChanged);
abortTimer = new Timer(AbortDownload, null, TimeUntilFirstByte, System.Threading.Timeout.Infinite);
}
protected void OnNotifyMegabyteIncrement(long totalMb)
{
NotifyMegabyteIncrement?.Invoke(totalMb);
}
protected void OnNotifyTaggedMegabyteIncrement(string tag, long totalMb)
{
NotifyTaggedMegabyteIncrement?.Invoke(tag, totalMb);
}
void AbortDownload(object state)
{
cancelDueToError = true;
this.CancelAsync();
success = false;
Error = firstByteReceived ? "Download aborted due to >" + TimeBetweenProgressChanges + "ms between progress change updates." : "No data was received in " + TimeUntilFirstByte + "ms";
asyncWait.Set();
}
private object disposeLock = new object();
void MyWebClient_DownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e)
{
if (cancelDueToError || isDisposed) return;
long additionalBytesReceived = e.BytesReceived - PreviousBytesReceived;
PreviousBytesReceived = e.BytesReceived;
BytesNotNotified += additionalBytesReceived;
if (BytesNotNotified > ONE_MB)
{
OnNotifyMegabyteIncrement(e.BytesReceived);
OnNotifyTaggedMegabyteIncrement(Tag, e.BytesReceived);
BytesNotNotified = 0;
}
firstByteReceived = true;
try
{
lock (disposeLock)
{
if (!isDisposed) abortTimer.Change(TimeBetweenProgressChanges, System.Threading.Timeout.Infinite);
}
}
catch (ObjectDisposedException) { } // Some strange timing issue causes this to throw now and then
}
public string Tag { get; private set; }
public bool DownloadFileWithEvents(string url, string outputPath, string tag = null)
{
Tag = tag;
asyncWait.Reset();
Uri uri = new Uri(url);
this.DownloadFileAsync(uri, outputPath);
asyncWait.WaitOne();
return success;
}
void MyWebClient_DownloadFileCompleted(object sender, System.ComponentModel.AsyncCompletedEventArgs e)
{
if (e.Error != null) success = false;
if (cancelDueToError || isDisposed) return;
asyncWait.Set();
}
protected override WebRequest GetWebRequest(Uri address)
{
var result = base.GetWebRequest(address);
result.Timeout = this.Timeout;
return result;
}
void IDisposable.Dispose()
{
lock (disposeLock)
{
isDisposed = true;
if (asyncWait != null) asyncWait.Dispose();
if (abortTimer != null) abortTimer.Dispose();
base.Dispose();
}
}
}

I fixed it using a combination of a BackgroundWorker and downloading in small chunks (following up mtkachenko's solution). It also checks if the length of the downloaded file is the same as the one on the server. Using that, it can continue the download where the connection was interupted.
private void DownloadPatch(Patch patch){
//using background worker now!
downloadBackgroundWorker = new BackgroundWorker();
downloadBackgroundWorker.DoWork += (sender, e) => DownloadFile_DoWork(sender, e, patch);
downloadBackgroundWorker.ProgressChanged += DownloadFile_ProgressChanged;
downloadBackgroundWorker.RunWorkerCompleted += DownloadFile_RunWorkerCompleted;
downloadBackgroundWorker.WorkerReportsProgress = true;
downloadBackgroundWorker.RunWorkerAsync();
}
private void DownloadFile_DoWork(object sender, DoWorkEventArgs e, Patch patch)
{
string startupPath = Application.StartupPath;
string downloadPath = Path.Combine(Application.StartupPath, patch.path);
string path = ("http://www.dagovaxgames.com/api/downloads/" + patch.path);
long iFileSize = 0;
int iBufferSize = 1024;
iBufferSize *= 1000;
long iExistLen = 0;
System.IO.FileStream saveFileStream;
// Check if file exists. If true, then check amount of bytes
if (System.IO.File.Exists(downloadPath))
{
System.IO.FileInfo fINfo =
new System.IO.FileInfo(downloadPath);
iExistLen = fINfo.Length;
}
if (iExistLen > 0)
saveFileStream = new System.IO.FileStream(downloadPath,
System.IO.FileMode.Append, System.IO.FileAccess.Write,
System.IO.FileShare.ReadWrite);
else
saveFileStream = new System.IO.FileStream(downloadPath,
System.IO.FileMode.Create, System.IO.FileAccess.Write,
System.IO.FileShare.ReadWrite);
System.Net.HttpWebRequest hwRq;
System.Net.HttpWebResponse hwRes;
hwRq = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(path);
hwRq.AddRange((int)iExistLen);
System.IO.Stream smRespStream;
hwRes = (System.Net.HttpWebResponse)hwRq.GetResponse();
smRespStream = hwRes.GetResponseStream();
iFileSize = hwRes.ContentLength;
//using webclient to receive file size
WebClient webClient = new WebClient();
webClient.OpenRead(path);
long totalSizeBytes = Convert.ToInt64(webClient.ResponseHeaders["Content-Length"]);
int iByteSize;
byte[] downBuffer = new byte[iBufferSize];
while ((iByteSize = smRespStream.Read(downBuffer, 0, downBuffer.Length)) > 0)
{
if (stopDownloadWorker == true)
{
autoDownloadReset.WaitOne();
}
saveFileStream.Write(downBuffer, 0, iByteSize);
long downloadedBytes = new System.IO.FileInfo(downloadPath).Length;
// Report progress, hint: sender is your worker
int percentage = Convert.ToInt32(100.0 / totalSizeBytes * downloadedBytes);
(sender as BackgroundWorker).ReportProgress(percentage, null);
}
}
As you can see, I report the progress, so that I have a working progress bar as well.
private void DownloadFile_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
statusTextLabel.Text = "Downloading updates for version " + currentDownloadingPatch + " (" + e.ProgressPercentage + "%)";
progressBar.Value = e.ProgressPercentage;
}

DownloadFileAsync is blocking my App

I am trying to download a large file(around 1GB) my server.When I start downloading I am unable to use the app till the download completes. It is blocking the UI and its becoming unresponsive.
In below code I am calling DownloadFile method when the user download button on the UI.And then download starts, but the UI is freezed.
I read that DownloadFileAsync won't block the UI. But here its blocking. How to use it in correct way. There are several answers but none is working when I am testing.
Code:
Button call:
private void Button_Click(object sender, RoutedEventArgs e)
{
Debug.WriteLine("1");
DownloadGamefile DGF = new DownloadGamefile();
Debug.WriteLine("2" + Environment.CurrentDirectory);
DGF.DownloadFile("URL(https link to zip file)", Environment.CurrentDirectory + #"\ABC.zip");
Debug.WriteLine("3");
}
Download code:
class DownloadGamefile
{
private volatile bool _completed;
public void DownloadFile(string address, string location)
{
WebClient client = new WebClient();
Uri Uri = new Uri(address);
_completed = false;
client.DownloadFileCompleted += new AsyncCompletedEventHandler(Completed);
client.DownloadProgressChanged += new DownloadProgressChangedEventHandler(DownloadProgress);
client.DownloadFileAsync(Uri, location);
}
private void DownloadProgress(object sender, DownloadProgressChangedEventArgs e)
{
// Displays the operation identifier, and the transfer progress.
Console.WriteLine("{0} downloaded {1} of {2} bytes. {3} % complete...",
(string)e.UserState,
e.BytesReceived,
e.TotalBytesToReceive,
e.ProgressPercentage);
}
private void Completed(object sender, AsyncCompletedEventArgs e)
{
if (e.Cancelled == true)
{
Console.WriteLine("Download has been canceled.");
}
else
{
Console.WriteLine("Download completed!");
}
_completed = true;
}
}

Please refer to this link. The actual problem is getting lots of feedback about the number of bytes downloaded(about progress of download process). Take a timer to get progress for every 2 seconds or any time, this solved the problem.

WebClient.DownloadProgressChanged never gets called

I added an event handler to the WebClient's DownloadProgressChanged event, but it never seems to fire. The file successfully downloads, but without having its progress updated.
public class DownloadFile
{
private File file = null;
public DownloadFile(File file)
{
this.file = file;
}
public void startDownloadThread()
{
Console.WriteLine("Starting Download : "+file.URL);
var t = new Thread(() => DownloadThread(file));
t.Start();
}
public Action<string> action_error_downloadFailed = Console.WriteLine;
private void DownloadThread(File file) //Unnecessary argument but whatever ;D
{
try
{
string url = file.URL;
string savepath = file.DestinationDir + "\\" + file.Filename;
WebClient_B client = new WebClient_B();
client.Proxy = null; //default to no proxy
client.DownloadProgressChanged += new DownloadProgressChangedEventHandler(client_DownloadProgressChanged);
client.DownloadFile(url, savepath);
Console.WriteLine("Download finished :" + file.Filename);
}
catch (Exception ex)
{
if (action_error_downloadFailed != null)
action_error_downloadFailed("Download failed :"+ex.Message);
}
}
private void client_DownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e)
{
try
{
if (file.TotalSize == 0)
file.TotalSize = (int)e.TotalBytesToReceive;
file.CurrentSize = (int)e.BytesReceived;
Form_DownloadManager.rebuildQueue();
Console.WriteLine("{0} downloaded {1} of {2} bytes. {3} % complete...",
(string)e.UserState,
e.BytesReceived,
e.TotalBytesToReceive,
e.ProgressPercentage);
}
catch (Exception ex) { Console.WriteLine("client_DownloadProgressChanged error : "+ex.Message); }
}
}
Output:
Starting Download : http://x.x.x/y/z.zip
'projectname.vshost.exe' (Managed (v4.0.30319)): Loaded 'C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
The thread '<No Name>' (0x3b8c) has exited with code 0 (0x0).
Download finished :z.zip
I am using WebClient_B because I had to add useragent+cookiecontainer functionality to the WebClient class since my server kept rejecting the download request. The event never fired with the 'standard' WebClient class either, though. So that shouldn't be the problem. But anyway; link to class

client.DownloadFile(url, savepath);
You have to use the async version to download a file, currently you use the blocking, synchronous version.
From the msdn docs for WebClient.DownloadProgressChanged:
Occurs when an asynchronous download operation successfully transfers
some or all of the data.
In your case that would be:
client.DownloadProgressChanged += new DownloadProgressChangedEventHandler(client_DownloadProgressChanged);
client.DownloadFileAsync (url, savepath);
Since your method does not directly return any result, this refactoring shouldn't be a problem, note though that the download most likely has not been completed by the time the method returns to the caller.

If you want to use WebClient synchronously, while getting progress updates, you can rely on this method detailed here:
http://alexfeinberg.wordpress.com/2014/09/14/how-to-use-net-webclient-synchronously-and-still-receive-progress-updates/
public void DownloadFile(Uri uri, string destination)
{
using (var wc = new WebClient())
{
wc.DownloadProgressChanged += HandleDownloadProgress;
wc.DownloadFileCompleted += HandleDownloadComplete;
var syncObject = new object();
lock (syncObject)
{
wc.DownloadFileAsync(uri, destination, syncObject);
// This would block the thread until download completes
Monitor.Wait(syncObject);
}
}
// Do more stuff after download was complete
}
private void HandleDownloadComplete(object sender, AsyncCompletedEventArgs args)
{
lock (args.UserState)
{
// releases blocked thread
Monitor.Pulse(args.UserState);
}
}
private void HandleDownloadProgress(object sender, DownloadProgressChangedEventArgs args)
{
// Process progress updates here
}

WebClient DownloadProgressChangedEventHandler Not Firing

I'm having trouble getting DownloadProgressChangedEventHandler to fire. I understand that, worst case, the event handler should fire every 64Kb. The URL I'm trying to download data from creates 680Kb of XML data on the fly, but the handler does not fire at all.
Here's test code that demonstrates the problem. Unfortunately I cannot share the specific URL as it contains proprietary data.
static void Main(string[] args)
{
Console.WriteLine("Downloading data");
string url = "https://MyUrlThatRespondsWith680KbOfData";
string outputPath = #"C:\Temp\DeleteMe.xml";
using (WebClient webClient = new WebClient())
{
webClient.DownloadProgressChanged += new DownloadProgressChangedEventHandler(webClient_DownloadProgressChanged);
webClient.DownloadFile(url, outputPath);
}
Console.Write("Done");
Console.ReadKey(true);
}
static void webClient_DownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e)
{
Console.WriteLine("Download progress: " + e.BytesReceived);
}

Your code looks ok, but documentation says "This event is raised each time an asynchronous download makes progress" while you are using synchronous version of the download. Switch to use DownloadFileAsync.

My code was structured so that the WebClient was already used on a non-UI thread, so I extended WebClient to allow for a synchronous call to get events. I also extended it to allow for a custom connection timeout (I was calling a web service that could take quite a while to respond). The new method DownloadFileWithEvents internally calls DownloadFileAsync and properly blocks until the appropriate complete event is received.
Here's the code in case it's useful to anyone:
public class MyWebClient : WebClient
{
//time in milliseconds
private int timeout;
public int Timeout
{
get
{
return timeout;
}
set
{
timeout = value;
}
}
public MyWebClient(int timeout = 60000)
{
this.timeout = timeout;
this.DownloadFileCompleted += new System.ComponentModel.AsyncCompletedEventHandler(MyWebClient_DownloadFileCompleted);
}
EventWaitHandle asyncWait = new ManualResetEvent(false);
public void DownloadFileWithEvents(string url, string outputPath)
{
asyncWait.Reset();
Uri uri = new Uri(url);
this.DownloadFileAsync(uri, outputPath);
asyncWait.WaitOne();
}
void MyWebClient_DownloadFileCompleted(object sender, System.ComponentModel.AsyncCompletedEventArgs e)
{
asyncWait.Set();
}
protected override WebRequest GetWebRequest(Uri address)
{
var result = base.GetWebRequest(address);
result.Timeout = this.timeout;
return result;
}
}

DownloadFileCompleted event doesn't fire inside while loop c# .net

I'm a bit confused why the event doesn't fire when the file has downloaded.
The file it self downloads perfectly fine.
I'm assuming there is some error in the way I am using this, in that the event doesn't fire inside a loop.
Thanks for any help anyone can give me
class DownloadQueue
{
public List<string[]> DownloadItems { get; set; }
public int CurrentDownloads;
public int DownloadInProgress;
string url = #"http://www.google.co.uk/intl/en_uk/images/logo.gif";
bool downloadComplete;
public DownloadQueue()
{
CurrentDownloads = 0;
DownloadItems = new List<string[]>();
Console.Write("new download queue made");
}
public void startDownloading(int maxSimulatiousDownloads)
{
downloadComplete = true;
DownloadInProgress = 0;
WebClient client = new WebClient();
client.DownloadFileCompleted +=
new AsyncCompletedEventHandler(this.downloadCompleteMethod);
while(DownloadInProgress != DownloadItems.Count )
{
if (downloadComplete == true)
{
downloadComplete = false;
client.DownloadFileAsync(new Uri(DownloadItems.ElementAt(DownloadInProgress).ElementAt(0).ToString()), DownloadItems.ElementAt(DownloadInProgress).ElementAt(1).ToString());
}
}
Console.Write("all downloads completed");
}
private void downloadCompleteMethod(object sender, AsyncCompletedEventArgs e)
{
downloadComplete = true;
DownloadInProgress++;
Console.Write("file Downloaded");
}
}

Where are you waiting for the DownloadFileAsync() call to complete? How about something like
manualResetEvent AllDone = new mre(false)
just before console.WriteLine
AllDone.WaitOne()
And in download complete
if (interlocked.decrement(ref downloadComplete) == 0) { AllDone.Set(); }

Most likely there is no time for the message to come. I suspect if you put an Application.DoEvents at the end of your loop, it would start firing the events. It is less than ideal to use it, but with the design you have, I can't think of a better way.