I'm trying to understand the differences among WebClient.OpenRead, WebClient.OpenReadAsync and WebClient.OpenReadTaskAsync.
It looks like these have differences regarding blocking a thread, but I don't understand it well.
Could you please explain the differences? It would be great if you could give me some example (examples don't have to be sample code, but would be great if you could provide)
As you said, the difference is in thread blocking behavior. First one (OpenRead()) is thread blocking operation, other two - not. For example, let assume, that your network latency to reach google.com is 300ms. When you do var stream = webClient.OpenRead(#"https://www.google.com"); your application is "paused" for this 300ms, so code next to this line is not executed until your webClient return a stream to stream variable. This is calling Thread blocking.
When you do this in UI-thread (in example: in a button click handler) - your application become freezing and not responding to user actions. This is bad user experience, so never-ever call a thread blocking stuff in your UI. Here is example for console application:
var address = #"https://www.google.com/";
Console.WriteLine($"Opening stream from {address}");
using (var stream = webClient.OpenRead(address)) // this will block for 300ms
{
Console.WriteLine("Stream is open!");
// some other code
}
Second method (OpenReadAsync()) is asynchronous and return nothing immediately after a call, so your thread is not blocked. After awhile (300ms) OpenReadCompleted event will be raised by your webClient and all attached listeners will handle opened stream one-by-one. Here is an example:
public partial class MainForm : Form
{
private WebClient _client = new WebClient();
public MainForm()
{
InitializeComponents();
_client.OpenReadCompleted += OpenReadCompletedHandler;
}
private void ButtonClickHandler(object sender, EventArgs e)
{
_client.OpenReadAsync(#"https://www.google.com/");
}
private void OpenReadCompletedHandler(object sender, OpenReadCompletedEventArgs e)
{
// this event will be raiesed 300ms after 'Button' click
var stream = e.Result; // <- here is your stream
// some other code
}
}
The last one (OpenReadTaskAsync()) is all about TPL (Task Parallel Library) and async/await keywords. It runs all stuff in a Task which is returned by this method. Here is an example:
public partial class MainForm : Form
{
private WebClient _client = new WebClient();
public MainForm()
{
InitializeComponents();
}
private async void ButtonClickHandler(object sender, EventArgs e)
{
// after 'await' keyword, execution will be returned from this method immediately
// meanwhile, actual acquiring of 'stream' is running in a background thread
using (var stream = await _client.OpenReadTaskAsync(#"https://www.google.com/"))
{
// after 300ms, this code will be continued in UI thread
// result will be automaticly unpacked to 'stream' variable
// some other code
}
}
}
Hope this helps.
I'd suggest WebClient is more or less obsolete now, HttpClient is more appropriate for anything targeting .NET framework 4.5+ or .NET core. Just watch out that the latter does not automatically throw exceptions on HTTP error codes (400+).
Related
I have a webpage with a list of recipes. In my code, I loop through the page and download each recipe. Here is a pseudo code of what I am doing :
//This is the Recipe class
//The constructor accepts a link to the recipe
//This method scrapes the recipe
public Task ScrapeAsync(){
return Task.Run(async () => {
string WebPage;
WebRequest request = WebRequest.Create(link);
request.Method = "GET";
using (WebResponse response = await request.GetResponseAsync())
using (StreamReader reader = new StreamReader(response.GetResponseStream())) {
WebPage = await reader.ReadToEndAsync();
}
//Non - async code here which is used to scrape the webpage
}
}
I used Task.Run because there are both async and blocking code in the Method.
//This is the RecipePage class
//This method scrapes the page
public Task GetRecipeListAsync(){
return Task.Run(async () => {
//I get the page using WebRequest and WebResponse in the same way as above
//Non - async/blocking code here which scrapes the page and finds links to recipes. I do await Recipe.ScrapeAsync() somewhere here.
//And I add the recipe objects to a public list in this class
}
}
In the form, it loops through a list of pages and do await RecipePage.GetRecipeList() and other things.
Here's where the for loop is:
private async void go_Click(object sender, EventArgs e){
for (int i = (int)startingPageNUD.Value; i <= (int)finishingPageNUD.Value; ++i) {
RecipePage page = new RecipePage(page + i);
await page.GetRecipeListAsync();
//Some other code here
}
}
My problem is that whenever an exception happens in the ScrapeAsync method, Visual Studio 2013 points to Application.Run(new Form1())
static void Main()
{
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
Application.Run(new Form1());
}
and tells me that Reflection.TargetInvocationException has occured. It does not show the actual exception in my code. For example, if I get a NullReferenceException in the code, it does not show that.
Because of this, I am having to write both async and non-async code and use non-async code to debug. Is there any way to solve this?
I have another question too. Am I using async/await and Task in the correct way?
Am I using async/await and Task in the correct way?
Pretty close. I don't see a need for Task.Run in this scenario (it should only be used to move CPU-intensive operations off the UI thread, and HTML scraping is usually fast enough to stay on the UI without any adverse effects).
The other recommendation I'd make is to have async Task methods return values as much as possible, instead of modifying member variables as side effects. In your case, consider having ScrapeAsync return its own list instead of updating a shared list, and having the calling code do the list merging.
Regarding your exception, TargetInvocationException will have an InnerException with the details. Exceptions in async void event handlers are managed in practically the same say as exceptions in regular void event handlers: if one escapes the event handler, then it goes to the application main loop. If you don't want this, you'll have to catch it (for both synchronous and asynchronous handlers).
Okay. Thanks to your edits I now can answer.
Your problem is this function:
private async void go_Click(object sender, EventArgs e){
for (int i = (int)startingPageNUD.Value; i <= (int)finishingPageNUD.Value; ++i) {
RecipePage page = new RecipePage(page + i);
await page.GetRecipeListAsync();
//Some other code here
}
}
The issue is that the function returns to what ever calls it once it reaches the await. Now if page.GetRecipeListAsync(); throws a exception, this exception is thrown inside the continuation handler. This handler is executed in the task queue of your UI. A exception thrown there crashes the task loop of your UI and this has all sorts of funny effects including strange exceptions.
In a async void function you should always handle any incoming exceptions by wrapping all code inside into a try…catch. If you crash the application if a exception occurs there or not is yours to decide.
The general way the things work is that any exception thrown inside a Task and by extend in a async function are thrown again by the await. But the async void functions are not awaited anywhere, so that does not work.
Regarding the usage of the async. I don't think you need to wrap every function into a Task but you can do that. Usually you don't need to force it into the background like this.
Encapsulate the getRecipeList code inside a try/catch.
In the catch code, get exception message and stack trace and assign them to variable(s) of Recipe class that you will test/display in your main thread when action is completed.
private string TheException = "" ;
public Task GetRecipeListAsync()
{
TheException = "" ;
Task result = Task.Run(async () =>
{
try
{
//I get the page using WebRequest and WebResponse in the same way as above
...
}
catch (Exception Ex)
}
if (TheException!="") MessageBox.show(TheException) ;
// or Throw an exception with
return result ;
}
you may also test "TheExceptio" in the GoClick procedure.
I'm trying to write a program that catches the HTTP get requests.
I have found Fiddler-core a genius library that should do exactly what I want.
The thing is, I'm trying to execute a big piece code-work inside the void FiddlerApplication_BeforeRequest(Session oSession) and it seems to block all the request and damage my surfing speed a great deal.
I have tried to use threads/tasks with no avail.
What am I doing wrong?
This is my code:
public event RequestCapture RequestCaptured;
private CancellationTokenSource cancelTokenSource = new CancellationTokenSource();
//...stat public function:
public void RunWatch() {
Fiddler.FiddlerApplication.BeforeRequest += FiddlerApplication_BeforeRequest;
Fiddler.FiddlerApplication.Startup(0, FiddlerCoreStartupFlags.Default);
}
void FiddlerApplication_BeforeRequest(Session oSession)
{
if (RequestCaptured != null)
{
CancellationToken ct = cancelTokenSource.Token;
Task.Factory.StartNew(() =>RequestCaptured(oSession.fullUrl), ct);
//Handle the event in a new thread, so the Listener will continue to listen
}
}
//close public function:
public void Close() {
try
{
FiddlerApplication.Shutdown();
cancelTokenSource.Cancel();
}
catch { }
}
now i have i different class that do that:
public Form1()
{
Listiner = new HttpWatcher.Listner();
Listiner.RequestCaptured += RequestCaptured;
Listiner.RunWatch();
}
void RequestCaptured(string url)
{
System.Threading.Thread.Sleep(10000);
}
edit
The question is: Is there a better way using fiddler-core? or am i to build a simple proxy for that? using something else? Thanks!
edit2
I have edited the code, so it would fill the missing parts.
Just to be clear here, FiddlerCore processes each Session on a threadpool thread. If you need blocking behavior, there's no need to spin up an additional thread or anything like that. If you don't need to process things in a blocking manner, then feel free to queue the data on a background queue and use tasks or another asynchronous mechanism to perform processing.
You should explain exactly what you mean when you say damage my surfing speed a great deal, and whether or not you see different behavior when using Fiddler rather than your application.
I want to create an HTTPRequest on a periodic task in a windows phone 7 background agent.
To keep it simple I just want to call a method on a shared class between the backgroundAgent and the application.
The shared method is a simple HTTPRequest.
On the SharedClass.cs makeTheRequest()
public static void makeTheRequest(){
var request = (HttpWebRequest)WebRequest.Create(new Uri("http://foo.bar"));
request.BeginGetResponse(r =>
{
NotifyComplete();
}, request);
}
I cannot call the notifyComplete() here because is not in the scope.
On the BackgroundAgent.cs onInvoke()
protected override void OnInvoke(ScheduledTask task)
{
if (task is PeriodicTask)
{
SharedClass.makeTheRequest();
NotifyComplete();
}
}
When I call it here, probably makeTheRequest() never gets done because the process is killed before it gets completed
I have read something about Taks Parallel library, but I don't know if thas is the right way to do it nor how to do it.
thanks
I'd change your makeTheRequest() method so that you can pass it an Action to fire upon request completion.
In the call from the agent you can include the call to NotifyComplete() but from the app you don't do this.
Note also that you should include timeout handling in the agent as repeated failing of the request from within the agent, due to timing out, can lead to the agent being disabled.
Update
An Example:
protected override void OnInvoke(ScheduledTask task)
{
if (task is PeriodicTask)
{
SharedClass.makeTheRequest(this.NotifyComplete);
}
}
public class SharedClass
{
public static void makeTheRequest(Action callback)
{
var request = (HttpWebRequest)WebRequest.Create(new Uri("http://foo.bar"));
request.BeginGetResponse(r => callback.Invoke(), request);
}
}
Once the main bgAgent thread exits, the HttpWebRequest will be killed.
We should be using synchronous HttpWebRequest here, but we can't because MS took them away from gelded framework.
We have to mimic thread-blocking behaviour using thread synchronization objects, like
ManualResetEvent.
protected override void OnInvoke(ScheduledTask task)
{
var evnt = new ManualResetEvent(false);//initialize to unsignalled state (false)
var request = (HttpWebRequest)WebRequest.Create(new Uri("http://foo.bar"));
request.BeginGetResponse(r =>
{
//do work here
evnt.Set();//signal main thread to continue
}, request);
evnt.WaitOne();//block execution of main thread
NotifyComplete();
return;
}
This way neither main thread will exit not NotifyComplete will be called before you finish your work.
You should make that WaitOne with a timeout (around 25 seconds) to ensure your task won't get killed and (worse) unsheduled because of 30 secs limit. This will make things much more complicated, as you'll have to protect your both threads (main and http) from messing each other up.
The evnt.Close() issue is also not shown here. Main thread may close the handle before http finishes and tries to Set(). Or you can rely on garbage collection Do I need to call Close() on a ManualResetEvent?
(btw, ManualResetEvent has nothing to do with concept of C# event. It's and event in Win32 sense, from same gang as Mutex and Semaphore).
You MUST use Delegate
In SharedClass.cs
public delegate void MyDelegate();
public MyDelegate MyUpdate;
In BackgroundAgent.cs
Maybe
void UpdateLiveTile() { ..... NotifyComplete(); }
In BackgroundAgent.cs onInvoke()
var cs = new SharedClass();
cs.MyUpdate= new SharedClass.MyDelegate(UpdateLiveTile);
cs.makeTheRequest();
In public static void makeTheRequest()
public static void makeTheRequest()
{
var request ....
request.BeginGetResponse(r =>
{
.........
MyUpdate();
}, request
Now the WebClient issue is fixed and can return on a background thread i'd like to start using it in this way.
After much searching I've come up with this code that appears to work fine, is this all there is to it?
BackgroundWorker bw = new BackgroundWorker();
bw.DoWork += (s,e) =>
{
WebClient wc = new WebClient();
wc.DownloadStringCompleted += DownloadStringCompleted;
wc.DownloadStringAsync(url);
};
bw.RunWorkerAsync();
In DownloadStringCompleted I send the result back to the UI thread.
Have I missed anything important or is it really this simple?
I don't get why you would want to run the WebClient on a background thread in the first place, since the WebClient is already creating a thread for the downloading part.
The difference is that the WebClient is running it's DownloadStringCompleted event on the UI thread. Which it still would do in your code.
I would suggest you use WebRequest class instead. The use of the WebRequest class can be greatly simplified with a simple extension method, that makes it behave like the WebClient.
public static class WebRequestEx
{
public static void DownloadStringAsync(this WebRequest request, Action<string> callback)
{
if (request == null)
throw new ArgumentNullException("request");
if (callback == null)
throw new ArgumentNullException("callback");
request.BeginGetResponse((IAsyncResult result) =>
{
try
{
var response = request.EndGetResponse(result);
using (var reader = new StreamReader(response.GetResponseStream()))
{
callback(reader.ReadToEnd());
}
}
catch (WebException e)
{
// Don't perform a callback, as this error is mostly due to
// there being no internet connection available.
System.Diagnostics.Debug.WriteLine(e.Message);
}
}, request);
}
}
The issue I referred to was that in 7.0 WebClient always returned on the UI thread regardless of where it was created, potentially making the UI unresponsive.
In the WP SDK 7.1 WebClient will return on the thread it was created from, so if it is created from a background thread DownloadStringCompleted will now return on a background thread.
If you test my example without marshalling the response you will see an Invalid Cross Thread Exception.
It seems to me unless you have a reason not to, why not use WebClient now?
Seems that easy.
Check only if everything which can be disposed get's disposed.
I using a API http://themoviedbapi.codeplex.com/ but if I call for example 7 instances of this method it takes 2-3 sek to run and in the meantime my app is locked. So is any part this method implemented wrongly so it aint run async?
#region GetMovieInfoAsyncMethods
private delegate void GetMovieInfoDelegate(int id, object userState, AsyncOperation asyncOp);
public void GetMovieInfoAsync(int id)
{
GetMovieInfoAsync(id, null);
}
public void GetMovieInfoAsync(int id, object userState)
{
AsyncOperation asyncOp = AsyncOperationManager.CreateOperation(null);
GetMovieInfoDelegate worker = new GetMovieInfoDelegate(GetMovieInfoWorker);
worker.BeginInvoke(id, userState, asyncOp, null, null);
}
private void GetMovieInfoWorker(int id, object userState, AsyncOperation asyncOp)
{
Exception exception = null;
TmdbMovie movie = null;
try
{
movie = GetMovieInfo(id);
}
catch (Exception ex)
{
exception = ex;
}
ImdbMovieInfoCompletedEventArgs args = new ImdbMovieInfoCompletedEventArgs(movie, exception, false, userState);
asyncOp.PostOperationCompleted(
delegate(object e) { OnGetMovieInfoCompleted((ImdbMovieInfoCompletedEventArgs)e); },
args);
}
protected virtual void OnGetMovieInfoCompleted(ImdbMovieInfoCompletedEventArgs e)
{
if (GetMovieInfoCompleted != null)
GetMovieInfoCompleted(this, e);
}
#endregion
I think you should put some Debug.Write()-style tracing in there so you can see where things are starting and stopping.
My initial guess from your sourcecode posting is that when GetMovieInfoCompleted is fired, something that subscribes to it takes a long time to run (e.g. updating many UI components).
What you're doing in your code sample looks okay at first glance because you're calling Control.BeginInvoke() which wouldn't block. The whole point of BeginInvoke as opposed to Invoke is it's non-blocking.
UPDATE:
You had an interesting comment about BeginInvoke still blocking the UI thread. I wasn't aware of that because I don't use it. Instead, consider using one of the following:
The BackgroundWorker component.
ThreadPool.QueueUserWorkItem() (although that'll require more cross-thread marshalling on your part that BackgroundWorker would do for you).
If you're on .NET 4.0, look at the Parallel Task Library.