I have the follow code section for a web client:
var client = new WebClient();
client.DownloadProgressChanged += (sender, args) =>
client_DownloadProgressChanged(sender, args, this.Context.ConnectionId);
client.DownloadDataCompleted += (sender, evt) =>
{
byte[] result = evt.Result;
aLongRunningTask(result);**
}
My problem is that the "aLongRunningTask" function needs to be initiated upon the completion of the downloaddata method, which is working fine, but since the file I receive is very large, I end up having a string in memory taking up a lot of memory space, that there is no need for since I only need byte[].
evt.Result is sadly a readonly property so I can not empty it, and I can not null the client, since I am running the aLongRunningTask.
Is there any way to either overwrite the evt.Result so it can be cleaned, or another way to get that memory usage emptied.
How about using using
like this
using(var client = new WebClient()){
//your logic
}
Related
Okay so I'm working on my file transfer service, and I can transfer the files fine with WCF streaming. I get good speeds, and I'll eventually be able to have good resume support because I chunk my files into small bits before streaming.
However, I'm running into issues with both the server side transfer and the client side receiving when it comes to measuring a detailed transfer speed as the messages are streamed and written.
Here's the code where the file is chunked, which is called by the service every time it needs to send another chunk to the client.
public byte[] NextChunk()
{
if (MoreChunks) // If there are more chunks, procede with the next chunking operation, otherwise throw an exception.
{
byte[] buffer;
using (BinaryReader reader = new BinaryReader(File.OpenRead(FilePath)))
{
reader.BaseStream.Position = currentPosition;
buffer = reader.ReadBytes((int)MaximumChunkSize);
}
currentPosition += buffer.LongLength; // Sets the stream position to be used for the next call.
return buffer;
}
else
throw new InvalidOperationException("The last chunk of the file has already been returned.");
In the above, I basically write to the buffer based on the chunk size I am using(in this case it's 2mb which I found to have the best transfer speeds compared to larger or smaller chunk sizes). I then do a little work to remember where I left off, and return the buffer.
The following code is the server side work.
public FileMessage ReceiveFile()
{
if (!transferSpeedTimer.Enabled)
transferSpeedTimer.Start();
byte[] buffer = chunkedFile.NextChunk();
FileMessage message = new FileMessage();
message.FileMetaData = new FileMetaData(chunkedFile.MoreChunks, buffer.LongLength);
message.ChunkData = new MemoryStream(buffer);
if (!chunkedFile.MoreChunks)
{
OnTransferComplete(this, EventArgs.Empty);
Timer timer = new Timer(20000f);
timer.Elapsed += (sender, e) =>
{
StopSession();
timer.Stop();
};
timer.Start();
}
//This needs to be more granular. This method is called too infrequently for a fast and accurate enough progress of the file transfer to be determined.
TotalBytesTransferred += buffer.LongLength;
return message;
}
In this method, which is called by the client in a WCF call, I get information for the next chunk, create my message, do a little bit with timers to stop my session once the transfer is complete and update the transfer speeds. Shortly before I return the message I increment my TotalBytesTransferred with the length of the buffer, which is used to help me calculate transfer speed.
The problem with this, is it takes a while to stream the file to the client, and so the speeds I'm getting are false. What I'm trying to aim for here is a more granular modification to the TotalBytesTransferred variable so I have a better representation of how much data is being sent to the client at any given time.
Now, for the client side code, which uses an entirely different way of calculating transfer speed.
if (Role == FileTransferItem.FileTransferRole.Receiver)
{
hostChannel = channelFactory.CreateChannel();
((IContextChannel)hostChannel).OperationTimeout = new TimeSpan(3, 0, 0);
bool moreChunks = true;
long bytesPreviousPosition = 0;
using (BinaryWriter writer = new BinaryWriter(File.OpenWrite(fileWritePath)))
{
writer.BaseStream.SetLength(0);
transferSpeedTimer.Elapsed += ((sender, e) =>
{
transferSpeed = writer.BaseStream.Position - bytesPreviousPosition;
bytesPreviousPosition = writer.BaseStream.Position;
});
transferSpeedTimer.Start();
while (moreChunks)
{
FileMessage message = hostChannel.ReceiveFile();
moreChunks = message.FileMetaData.MoreChunks;
writer.BaseStream.Position = filePosition;
// This is golden, but I need to extrapolate it out and do the stream copy myself so I can do calculations on a per byte basis.
message.ChunkData.CopyTo(writer.BaseStream);
filePosition += message.FileMetaData.ChunkLength;
// TODO This needs to be more granular
TotalBytesTransferred += message.FileMetaData.ChunkLength;
}
OnTransferComplete(this, EventArgs.Empty);
}
}
else
{
transferSpeedTimer.Elapsed += ((sender, e) =>
{
totalElapsedSeconds += (int)transferSpeedTimer.Interval;
transferSpeed = TotalBytesTransferred / totalElapsedSeconds;
});
transferSpeedTimer.Start();
host.Open();
}
Here, my TotalBytesTransferred is also based on the length of the chunk coming in. I know I can get a more granular calculation if I do the stream writing myself instead of using the CopyTo for the stream, but I'm not exactly sure how to best go about this.
Can anybody help me out here? Outside of this class I have another class polling the property of TransferSpeed as it's updated internally.
I apologize if I posted too much code, but I wasn't sure what to post and what not.
EDIT: I realize at least with the Server side implementation, the way I can get a more granular reading on how many bytes have been transferred, is by reading the position of the return message value of the stream. However, I don't know a way to do this to ensure absolute integrity on my count. I thought about maybe using a timer and polling the position as the stream was being transferred, but then the next call might be made and I would quickly become out of sync.
How can I poll data from the returning stream and know immediately when the stream finishes so I can quickly add up the remainder of what was left of the stream into my byte count?
Okay I have found what seems to be ideal for me. I don't know if it's perfect, but it's pretty darn good for my needs.
On the Server side, we have this code that does the work of transferring the file. The chunkedFile class obviously does the chunking, but this is the code that sends the information to the Client.
public FileMessage ReceiveFile()
{
byte[] buffer = chunkedFile.NextChunk();
FileMessage message = new FileMessage();
message.FileMetaData = new FileMetaData(chunkedFile.MoreChunks, buffer.LongLength, chunkedFile.CurrentPosition);
message.ChunkData = new MemoryStream(buffer);
TotalBytesTransferred = chunkedFile.CurrentPosition;
UpdateTotalBytesTransferred(message);
if (!chunkedFile.MoreChunks)
{
OnTransferComplete(this, EventArgs.Empty);
Timer timer = new Timer(20000f);
timer.Elapsed += (sender, e) =>
{
StopSession();
timer.Stop();
};
timer.Start();
}
return message;
}
The client basically calls this code, and the server proceeds to get a new chunk, put it in a stream, update the TotalBytesTransferred based on the position of the chunkedFile(which keeps track of the underlying file system file that is used to draw the data from). I'll show the method UpdateTotalBytesTransferred(message) in a moment, as that is where all the code for the server and client reside to achieve the more granular polling of the TotalBytesTransferred.
Next up is the client side work.
hostChannel = channelFactory.CreateChannel();
((IContextChannel)hostChannel).OperationTimeout = new TimeSpan(3, 0, 0);
bool moreChunks = true;
using (BinaryWriter writer = new BinaryWriter(File.OpenWrite(fileWritePath)))
{
writer.BaseStream.SetLength(0);
while (moreChunks)
{
FileMessage message = hostChannel.ReceiveFile();
moreChunks = message.FileMetaData.MoreChunks;
UpdateTotalBytesTransferred(message);
writer.BaseStream.Position = filePosition;
message.ChunkData.CopyTo(writer.BaseStream);
TotalBytesTransferred = message.FileMetaData.FilePosition;
filePosition += message.FileMetaData.ChunkLength;
}
OnTransferComplete(this, EventArgs.Empty);
}
This code is very simple. It calls the host to get the file stream, and also utilizes the UpdateTotalBytesTransferred(message) method. It does a little bit of work to remember the position of the underlying file that is being written, and copies the stream to that file while also updating the TotalBytesTransferred after finishing.
The way I achieved the granularity I was looking for was with the UpdateTotalBytesTransferred method as follows. It works exactly the same for both the Server and the Client.
private void UpdateTotalBytesTransferred(FileMessage message)
{
long previousStreamPosition = 0;
long totalBytesTransferredShouldBe = TotalBytesTransferred + message.FileMetaData.ChunkLength;
Timer timer = new Timer(500f);
timer.Elapsed += (sender, e) =>
{
if (TotalBytesTransferred + (message.ChunkData.Position - previousStreamPosition) < totalBytesTransferredShouldBe)
{
TotalBytesTransferred += message.ChunkData.Position - previousStreamPosition;
previousStreamPosition = message.ChunkData.Position;
}
else
{
timer.Stop();
timer.Dispose();
}
};
timer.Start();
}
What this does is take in the FileMessage which is basically just a stream and some information about the file itself. It has a variable previousStreamPosition to remember the last position it was when it was polling the underlying stream. It also does a simple calculation with totalBytesTransferredShouldBe based on how many bytes are already transferred plus the total length of the stream.
Finally, a timer is created and executed, which upon every tick checks to see if it needs to be incrementing the TotalBytesTransferred. If it's not supposed to update it anymore(reached the end of the stream basically), it stops and disposes of the timer.
This all allows me to get very small reads of how many bytes have been transferred, which lets me better calculate the total progress in a more fluid way, as more accurately measure the file transfer speeds achieved.
How can I get this error from with in the DownloadStringCompleted Event? Doesn't that mean, it's finished? Is there another event I can fire this from?
I get this error extremely rarely, but once in a while it will happen on my WP7 phone. I have a web client that I fire over and over, and I fire it again from the completed event. Is this happening because there is still some stale connection open? Is there a way to prevent this 100%?
I have checked to see if there is a chance for the thread to walk over itself, but it is only fired from within the completed event.
How can I be sure, when the complete event is fired, the client is no longer isBusy? One suggestion was to add a while with a thread sleep while the client is busy.
Some pseudo code.
var client = new WebClient("URL 1");
client.CompletedEvent += CompletedEvent;
client.downloadasync();
void CompletedEvent(){
Dosomestuff;
client.downloadasync(); //This is where we break.
}
The WebClient only supports a single operations, it cannot download multiple files. You haven't shown your code, but my guess is that you are somehow firing a new request before the old is completed. My bet is that WebClient.IsBusy is true when you attempt to perform another fetch.
See the following thread:
wb.DownloadFileAsync throw "WebClient does not support concurrent I/O operations." exception
The only answer is to create a new webclient within the scope of the Completed Event. You can't set it to new since webclient is readonly. Creating a new client is the only solution. This allows the old client to complete in the background. This does have slight memory implications since you are creating a new instance instead of reusing an old. But the garbage collector should keep it clean if your scope is setup right.
Instead of using WebClient use HttpClient to do parallel HTTP calls. Below code shows how to download files.
HttpClient httpClient = new HttpClient();
var documentList=_documentManager.GetAllDocuments();
documentList.AsParallel().ForAll(doc =>
{
var responseResult= httpClient.GetAsync(doc.FileURLPath);
using (var memStream = responseResult.Result.Content.ReadAsStreamAsync().Result)
{
using (var fileStream =File.Create($"{filePath}\\{doc.FileName}"))
{
memStream.CopyTo(fileStream);
}
}
});
The solution, I found is to use multiple WebClient objects, so to modify your pseudocode example; try
var client = new WebClient("URL 1");
client.CompletedEvent += CompletedEvent;
client.downloadasync();
void CompletedEvent(){
Dosomestuff;
var client2 = new WebClient();
client2.downloadasync();
}
Create a new Web Client for each new request. Don't reuse an existing Web Client instance.
This allows the first request to complete before starting the new one. This is a standard way of creating new requests.
private async Void SyncParcelStatus(List<string> Urls)
{
try
{
foreach (var URL in WebhookUrls)
{
Task.Factory.StartNew(() => AsyncDownLoad(URL));
}
}
catch (Exception ex)
{
//log Exception
}
}
private async void AsyncDownLoad(string URL)
{
using (WebClient myWebClient = new WebClient())
{
try
{
Uri StringToUri = new Uri(URL);
myWebClient.DownloadStringAsync(StringToUri);
}
catch (Exception ex)
{
//log Exception
}
}
}
I'm looking into how i can create a webclient method that i can reuse in my code. Now the below code would call client_DownloadStringCompleted and i would have to deal with the returned data there, but i'd like to do it in the request method so i can return it.
private string request(string json, string url) {
WebClient client = new WebClient();
client.DownloadStringCompleted += new DownloadStringCompletedEventHandler(client_DownloadStringCompleted);
client.DownloadStringAsync(new Uri(url);
}
The WebClient call is asynchronous, and it's that for a reason. The reason is usually to not block the UI thread during the wait time when the call has been initiated and the result has come back through the wire.
I think you are making a design mistake by enforcing an old school blocking call.
See my response in this question for solutions. WebClient - wait until file has downloaded
As the web request is performed asynchronously there isn't an easy way to have that method block and only return when (if?) the response is received. (There is a way to do this but it is recommended against for perf, usability and maintainability reasons.)
Instead you should write code that is designed to be run asynchronously.
The general pattern for your situation is to specify a callback method which takes an action as a final parameter.
The simplest way to implement the action is to take a single parameter which is the webresponse object. To aid code reuse and separation of concerns you'll probably want to move to having the action take a tuple of an Exception or the raw (or possibly formatted) response content.
You'll want to write your method something like:
private string request(string json,
string url,
Action<Exception, string> callback)
{
WebClient client = new WebClient();
client.DownloadStringCompleted += (s, e) =>
{
// add better error handling than this!!!
try
{
callback(e.Error, e.Result);
}
catch (Exception exc)
{
callback(exc, null);
}
};
client.DownloadStringAsync(new Uri(url);
}
Alternativey, you could use the async ctp http://msdn.microsoft.com/en-us/vstudio/gg316360
This sychronous method can be used, but it will block your thread until the download is completed and is therefore not the preferred method, but I think it will answer your question:
private string Request(string url)
{
WebClient client = new WebClient();
return client.DownloadString(new Uri(url));
}
I need to make a simple http client in C# that must be asynchronous and must support a persistent connection to the server. So i'm trying to use the WebClient class, but i'm having some problems, my code is this:
void sendMessage()
{
ServicePointManager.ServerCertificateValidationCallback += new System.Net.Security.RemoteCertificateValidationCallback(bypassAllCertificateStuff);
string loginRequest = #"{'IDENTIFIER':'patient1','PASSWORD':'asdasd','DEVICE_ID':'knt-01'}";
client = new WebClient();
// add event handlers for completed and progress changed
client.UploadProgressChanged += new UploadProgressChangedEventHandler(client_UploadProgressChanged);
client.UploadStringCompleted += new UploadStringCompletedEventHandler(client_UploadStringCompleted);
client.DownloadProgressChanged += new DownloadProgressChangedEventHandler(client_DownloadProgressChanged);
// carry out the operation as normal
client.UploadStringAsync(new Uri("Https://192.168.1.100/PaLogin"), "POST", loginRequest);
}
void client_DownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e)
{
Console.WriteLine("downloadProgressChanged");
}
void client_UploadProgressChanged(object sender, UploadProgressChangedEventArgs e)
{
// Console.WriteLine(e.ProgressPercentage);
if (e.ProgressPercentage != 50)
{
Console.WriteLine("uploadProgressChanged");
}
}
void client_UploadStringCompleted(object sender, UploadStringCompletedEventArgs e)
{
if (e.Result != null)
{
Console.WriteLine("Done");
}
}
The problem is that i should receive a response from the server, but neither the client_UploadStringCompleted nor client_DownloadProgressChanged callbacks are ever called.
The only thing I see on the console is: client_DownloadProgressChanged
So basically what i'm trying to do is:
1- I send some data to the server without closing the connection
2- I receive the server response but the connection must still be open when i have received it.
What am I missing?
Thank you. :-)
You are missing the whole HTTP protocol here.
HTTP is a stateless request-response protocol. HTTP 1.1 provides optional guidelines for keeping connections open purely for the sake of performance - although as for the request response paradigm, there is no change. [Yet I have seen many cases where client or server have decided not to respect it and closed the connection.] It also provides chunked encoding to facilitate streaming, but that is all it is as far as HTTP is concerned.
So basically in HTTP, client will wait for a reply (and keep connection open) until it receives a response or timeout. There is no way to change/better this behaviour.
NOW, back to you problem.
I think something is going wrong with connecting to the server so you need to use Fiddler to see what is happening. My hunch is it does not connect to server (firewall, server down, etc) since the certificate check is not even called.
Http server push mechanism can do this.
See this:
http://en.wikipedia.org/wiki/Comet_(programming))
c# client:
using (var client = new WebClient())
using (var reader = new StreamReader(client.OpenRead(uri), Encoding.UTF8, true))
{
string line;
while ((line = reader.ReadLine()) != null)
{
Console.WriteLine(line);
}
}
(che รจ quello che vi dicevo questo pomeriggio)
Trying to implement downloadStringAsync() to prevent UI freezing for 10 seconds when downloading one byte of data. However, even though the download completes, it is freezing the UI just as if I used downloadString().
Here is my code:
public void loadHTML()
{
WebClient client = new WebClient();
// Specify that the DownloadStringCallback2 method gets called
// when the download completes.
client.DownloadStringCompleted += new DownloadStringCompletedEventHandler(loadHTMLCallback);
client.DownloadStringAsync(new Uri("http://www.example.com"));
return;
}
public void loadHTMLCallback(Object sender, DownloadStringCompletedEventArgs e)
{
// If the request was not canceled and did not throw
// an exception, display the resource.
if (!e.Cancelled && e.Error == null)
{
string result = (string)e.Result;
// Do cool stuff with result
}
}
Encountered the same problem, and found a solution.
Quite complex discussion here:
http://social.msdn.microsoft.com/Forums/en-US/a00dba00-5432-450b-9904-9d343c11888d/webclient-downloadstringasync-freeze-my-ui?forum=ncl
In short, the problem is web client is searching for proxy servers and hanging the app.
The following solution helps:
WebClient webClient = new WebClient();
webClient.Proxy = null;
... Do whatever else ...