I have an application that downloading urls using threadPool in different threads, but recently I've read an article (http://www.codeproject.com/KB/IP/Crawler.aspx) that it says HttpWebRequest.GetResponse() is working only in one thread and the other threads is waiting for that thread. first I want to know is it true ? how can i monitor which one of my threads is actually downloading with its status ?
I doubt that HttpWebRequest.GetResponse would block other threads - but you can verify that easily using tools such as Fiddler. You can launch fiddler and run your program. The request would appear in Fiddler as soon as your program makes it and you can quickly determine if they are simultaneous or one by one.
Yes, GetResponse is a blocking call (check MSDN) which can only return when the server replies or a request timeout occurs. After that, just check the status code and use GetResponseStream to start downloading the returning content. Like this:
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == 200)
{
Stream content = response.GetResponseStream();
// Read the content and report the downloading progress...
...
}
Related
I noticed, that HttpResponseMessage.Content has ReadAsStringAsync() method. What is the sense of making it async when this operation requires cpu and task reation will add more CPU work?
It's not purely a CPU-bound operation. In a request with a large response, you can start reading the response body before the entire response body has been received over the network.
The first part of the response will likely already be in memory, but you could very well hit a point while reading the response where you need to wait for the rest of the data to be received over the network. That allows you to wait asynchronously and not block the thread.
I know I can perform a normal HTTP get and check the response but in my case i have some limitations as the video feed server is only allowing 1 connection at a time, which mean any HTTP request while watching the video stream on any player will interrupt the main stream and make it stop.
I have tried different headers with the HTTP GET request (AddRange , ReadWriteTimeout) but all were causing the feed to stop playing on any player
I have tried the HEAD request but for some reason its stuck at GetResponse() as it try to download the content which goes into infinity
var request = WebRequest.Create("http://xyss.ts") as HttpWebRequest;
request.Method = "HEAD";
var response = request.GetResponse(); //doesn't stop trying to download content
request.Abort();
response.Close();
Any idea how we can test if the feed still exist without interrupting the main server feed so it doesn't affect any client streaming
I have set up the following web request:
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(gotoWebinerUrl);
request.Accept = "text/json";
request.Timeout = 5000;
// Allows us to track with Fiddler, for dev use only
request.Proxy = new WebProxy("127.0.0.1", 8888);
try
{
WebResponse response = request.GetResponse();
Stream responseStream = response.GetResponseStream();
...
}
catch (Exception exception)
{
...
}
In Fiddler, I have installed this plugin which delays requests http://fiddlerdelayext.codeplex.com/. Using the plugin, I have added a rule which delays the webinar request URL by 60,000ms (1 min).
I would expect my application to delay for 5 seconds, fail and be caught by my exception. However, it delays for the full 60 seconds.
I'm not sure if this is the plugins problem or my application, but I suspect the latter. In the 60 seconds delay, I can navigate to other web pages and see the requests in Fiddler, so I don't believe the problem exists with the plugin.
I found a couple of similar questions (How to terminate HttpWebRequest Connection in C#?It doesn't work even set timeout or readwritetimeout), but I'm not very familiar with threads and if they apply here or not.
Additional Info
A bit more research into this stepping through the code and I've noticed some interesting behaviour. The GetResponse is fired successfully and I see the request made in Fiddler. I then have my 60 seconds wait time. However, when I can continue stepping through, I notice that it has been caught as a TimeoutException, despite the fact it's actually waited the full 60s and has received a successful response. This does suggest that maybe something in the plugin is fooling the application into holding on somehow.
I want to create console application in c# which is going to connect to local push server and will receive messages from it.
I have created the application which is able to read response from push server for 'single' connection made by using following code:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://192.168.15.174/MvisumPushServer/PushServer.ashx?op=2&DeviceUniqueID=50211&ConnectionTimeout=30&ConnectionMechanism=Wifi&OSVersion=10&DeviceMake=Android&DeviceModel=sdk");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream stream = null;
stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream);
s = reader.ReadLine();
Console.WriteLine(s);
Now I want to do same thing for 5 or more connections from single program using threads.
These many connections should receive all messages individually and should display it to console. For example if 5 notifications are generated from push server on console total 25 alerts should be written i.e. each connection receiving each message from push server.I have tried it using Lock(this) but total 5 alerts are getting displayed on console.
If it matters your approach is about a device which is turn on periodically. This means that if you want to test the server, you have to perform a poll by using a timer.An array of timers calling a routine with the code you already have at specific or random intervals will do the job, as long as you use a thread safe Queue to enqueue the responses and separately dequeue them and write them on console. In addition this routine may accept different DeviceUniqueIDs (as parameter) if you connect a DeviceUniqueID to each timer (with another array of the same length).
I suspect the problem is actually the lock. I suspect some thread is acquiring the lock and then blocking on ReadLine and thus preventing the other threads from making progress. Assuming WriteLine is thread safe, you should be able to execute your code above in five threads with no problems.
It doesn't sound like your threading approach is responsible for the problem. It sounds like your server is sending one event per connection (5 connections) rather than five events per connection.
I have an application that needs to download several files in a row in succession (sometimes a few thousand). However, what ends up happening when several files need to be downloaded is I get an exception with an inner exception of type SocketException and the error code 10048 (WSAEADDRINUSE). I did some digging and basically it's because the server has run out of sockets (and they are all waiting for 240s or so before they become available again) - not coincidentally it starts happening around the 1024 file range. I would expect that the HttpWebRequest/ServicePointManager would be reusing my connection, but apparently it is not (and the files are https, so that may be part of it). I never saw this problem in the C++ code that this was ported from (but that doesn't mean it didn't ever happen - I'd be surprised if it was, though).
I am properly closing the WebRequest object and the HttpWebRequest object has KeepAlive set to true by default. Next my intent is to fiddle around with ServicePointManager.SetTcpKeepAlive(). However, I can't see how more people haven't run into this problem.
Has anyone else run into the problem, and if so, what did you do to get around it? Currently I have a retry scheme that detects this error and waits it out, but that doesn't seem like the right thing to do.
Here's some basic code to verify what I'm doing (just in case I'm missing closing something):
WebRequest webRequest = WebRequest.Create(uri);
webRequest.Method = "GET";
webRequest.Credentials = new NetworkCredential(username, password);
WebResponse webResponse = webRequest.GetResponse();
try
{
using(Stream stream = webResponse.GetResponseStream())
{
// read the stream
}
}
finally
{
webResponse.Close()
}
What kind of application is this? You mentioned that the server is running out of ports, but then you mentioned HttpWebRequest. Are you running this code in a webservice or ASP.NET page, which is trying to then download multiple files for the same incoming request from the client?
What kind of authentication is the page using? If it is using NTLM authentication, then the connections cannot be shared if the credentials being used are different for each request.
What I would suggest is to group your request per credential. So, for eg, all requests using username "John" would be grouped. You can specify the "ConnectionGroupName" property on the service point, so the system will try to reuse connections for the same credential and server.
If that also doesnt work, you will need to do one or more of the following:
1) Throttle your requests.
2) Increase the wildcard port range.
3) Use the BindIPConnectionCallback on ServicePoint to make it bind to a non-wildcard port (i.e a port in the range 1024-16384)
More digging seems to point to it possibly being due to authentication and the UnsafeAuthenticatedConnectionSharing property might alleviate this. However, I'm not sure that's the best thing, either.