I have the following code:
System.Net.WebRequest req = System.Net.WebRequest.Create(url);
req.Credentials = new NetworkCredential("admin", "password");
System.Net.WebResponse resp = req.GetResponse();
System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream());
var result = sr.ReadToEnd().Trim();
When I run the code the result is just an empty string. However when I step through the code the result is a string with data in it, as I was expecting, when I put a breakpoint on this line:
System.Net.WebResponse resp = req.GetResponse();
So I think the problem lies with this or the subsequent line. Not sure how to proceed, help would be appreciated.
I came across a similar issue whilst using CopyToAsync() on a WebResponse, it turned out that the Stream's pointer was ending up at the end of the Stream (it's pointer position was equal to it's length).
If this is the case, you can reset the pointer before reading the contents of the string with the following...
var responseStream = resp.GetResponseStream();
responseStream.Seek(0, SeekOrigin.Begin);
var sr = new StreamReader(responseStream);
var result = sr.ReadToEnd().Trim();
Although, since you're reading the stream directly, and not copying it into a new MemoryStream, this may not apply to your case.
May be "req.GetResponse();" taking more time..... When your putting the break point its getting time to complete the task.
You need to check
resp.StatusDescription
before
System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream());
Related
I'm trying to get an image from an url using a byte stream. But i get this error message:
This stream does not support seek operations.
This is my code:
byte[] b;
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
WebResponse myResp = myReq.GetResponse();
Stream stream = myResp.GetResponseStream();
int i;
using (BinaryReader br = new BinaryReader(stream))
{
i = (int)(stream.Length);
b = br.ReadBytes(i); // (500000);
}
myResp.Close();
return b;
What am i doing wrong guys?
You probably want something like this. Either checking the length fails, or the BinaryReader is doing seeks behind the scenes.
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
WebResponse myResp = myReq.GetResponse();
byte[] b = null;
using( Stream stream = myResp.GetResponseStream() )
using( MemoryStream ms = new MemoryStream() )
{
int count = 0;
do
{
byte[] buf = new byte[1024];
count = stream.Read(buf, 0, 1024);
ms.Write(buf, 0, count);
} while(stream.CanRead && count > 0);
b = ms.ToArray();
}
edit:
I checked using reflector, and it is the call to stream.Length that fails. GetResponseStream returns a ConnectStream, and the Length property on that class throws the exception that you saw. As other posters mentioned, you cannot reliably get the length of a HTTP response, so that makes sense.
Use a StreamReader instead:
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
WebResponse myResp = myReq.GetResponse();
StreamReader reader = new StreamReader(myResp.GetResponseStream());
return reader.ReadToEnd();
(Note - the above returns a String instead of a byte array)
You can't reliably ask an HTTP connection for its length. It's possible to get the server to send you the length in advance, but (a) that header is often missing and (b) it's not guaranteed to be correct.
Instead you should:
Create a fixed-length byte[] that you pass to the Stream.Read method
Create a List<byte>
After each read, call List.AddRange to append the contents of your fixed-length buffer onto your byte list
Note that the last call to Read will return fewer than the full number of bytes you asked for. Make sure you only append that number of bytes onto your List<byte> and not the whole byte[], or you'll get garbage at the end of your list.
If the server doesn't send a length specification in the HTTP header, the stream size is unknown, so you get the error when trying to use the Length property.
Read the stream in smaller chunks, until you reach the end of the stream.
With images, you don't need to read the number of bytes at all. Just do this:
Image img = null;
string path = "http://www.example.com/image.jpg";
WebRequest request = WebRequest.Create(path);
req.Credentials = CredentialCache.DefaultCredentials; // in case your URL has Windows auth
WebResponse resp = req.GetResponse();
using( Stream stream = resp.GetResponseStream() )
{
img = Image.FromStream(stream);
// then use the image
}
Perhaps you should use the System.Net.WebClient API. If already using client.OpenRead(url) use client.DownloadData(url)
var client = new System.Net.WebClient();
byte[] buffer = client.DownloadData(url);
using (var stream = new MemoryStream(buffer))
{
... your code using the stream ...
}
Obviously this downloads everything before the Stream is created, so it may defeat the purpose of using a Stream. webClient.DownloadData("https://your.url") gets a byte array which you can then turn into a MemoryStream.
The length of a stream can not be read from the stream since the receiver does not know how many bytes the sender will send. Try to put a protocol on top of http and send i.e. the length as first item in the stream.
I am creating HTTP WebRequests in c# to navigate a website. I create the request, get a response and read it into a StreamReader. I then use the same request variable to create a new request and get a new response...
Below is my code:
HttpWebRequest request;
WebResponse response;
Stream responseStream;
StreamReader reader;
string responseFromServer;
request = (HttpWebRequest)WebRequest.Create("https://www.firstRequest.com");
//set cookies and headers here...
response = request.GetResponse();
responseStream = response.GetResponseStream();
reader = new StreamReader(responseStream);
responseFromServer = reader.ReadToEnd();
//second request using the same variables
request = (HttpWebRequest)WebRequest.Create("https://www.secondRequest.com");
response = request.GetResponse();
responseStream = response.GetResponseStream();
reader = new StreamReader(responseStream);
responseFromServer = reader.ReadToEnd();
I know I have to close the response, stream and streamreader to free up resources.
Do I have to do it in between each request or just at the end of everything?
Yes, you need to clean up after yourself every time. Just make use of the IDisposable features that you're ignoring:
So, instead, you could
using(response = request.GetResponse())
{
using(responseStream = response.GetResponseStream())
{
using(reader = new StreamReader(responseStream))
{
responseFromServer = reader.ReadToEnd();
}
}
}
or more concisely:
using(response = request.GetResponse())
using(responseStream = response.GetResponseStream())
using(reader = new StreamReader(responseStream))
{
responseFromServer = reader.ReadToEnd();
}
...now you don't need to worry about calling Close. The IDisposable implementation takes care of it for you (when the using block ends).
Alltough #spender has a point when he's saying you should be using using directives, I don't think that his answer answers your question or is particularly helpful.
When thinking about problems like this, think about the fundamentals of C# and objects that implement IDisposable. Classes that do so make use of unmanaged resources that need to be freed when the class is no longer used. This happens either at the end of the using-block or when Dispose is called.
After a call to Dispose however, the object should not be used anymore. This is a fundamental difference to Close. Using Close the response frees resources and the same object can be reused by another request.
MSDN for Close
Now let's consider what would happen if you simply overrode the response variable. classes are a reference type in C#, the variable would now reference a new object and the old response object would vanish in nirvana, waiting to be eventually collected by the GC without Dispose being invoked. Certainly not what you want.
So if you want to reuse variables, make sure you use Close
I am trying to retrieve HTML code from a webpage using HttpWebRequest and HttpWebResponse.
response = (HttpWebResponse)request.GetResponse();
...
Stream stream = response.GetResponseStream();
The response object has a ContentLength value of 106142. When I look at the stream object, it has a length of 65536. When reading the stream with a StreamReader using ReadToEnd(), only the first 65536 characters are returned.
How can I get the whole code?
Edit:
Using the following code segment:
catch (WebException ex)
{
errorMessage = errorMessage + ex.Message;
if (ex.Response != null) {
if (ex.Response.ContentLength > 0)
{
using (Stream stream = ex.Response.GetResponseStream())
{
using (StreamReader reader = new StreamReader(stream))
{
string pageOutput = reader.ReadToEnd().Trim();
ex.Response.ContentLength = 106142
ex.Response.GetResponseStream().Length = 65536
stream.Length = 65536
pageOutput.Length = 65534 (because of the trim)
And yes, the code is actually truncated.
You can find an answer in this topic in System.Net.HttpWebResponse.GetResponseStream() returns truncated body in WebException
You have to manage the HttpWebRequest object and change DefaultMaximumErrorResponseLength property.
For example :
HttpWebRequest.DefaultMaximumErrorResponseLength = 1048576;
ReadToEnd does specifically just that, it reads to the end of the stream. I would check to make sure that you were actually being sent the entire expected response.
There seems to be a problem when calling the GetResponseStream() method on the HttpWebResponse returned by the exception. Everything works as expected when there is no exception.
I wanted to get the HTML code from the error returned by the server.
I guess I'll have to hope the error doesn't exceed 65536 characters...
I get a response from a web-server using StreamReader... now I want to parse this response (it's an XML document file) to get its values, but every time I try to do it I get a error: Root element is missing.
If I read the same XML file directly, the file is well formatted and I can read it.
This is the stream:
WebResponse response = webRequest.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
string responseString = responseReader.ReadToEnd();
And this is how I try to read the XML file:
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.Load(responseReader);
XmlNodeList address = xmlDoc.GetElementsByTagName("original");
You have called ReadToEnd(), hence consumed all the data (into a string). This means the reader has nothing more to give. Just: don't do that. Or, do that and use LoadXml(reaponseString).
The Load method is capable of fetching XML documents from remote resources. So you could simplify your code like this:
var xmlDoc = new XmlDocument();
xmlDoc.Load("http://example.com/foo.xml");
var address = xmlDoc.GetElementsByTagName("original");
No need of any WebRequests, WebResponses, StreamReaders, ... (which by the way you didn't properly dispose). If this doesn't work it's probably because the remote XML document is not a real XML document and it is broken.
If you do it with the exact code you pasted in your question, then the problem is that you first read the whole stream into string, and then try to read the stream again when calling
xmlDoc.Load(responseReader)
If you have already read the whole stream to the string, use that string to create the xml document
xmlDoc.Load(responseString)
Check what's the content of responseString: probably it contains some additional headers that makes the xmlparser unhappy.
The error you are getting means, that XML you receive lacks first element that wraps the whole content. Try wrapping the answer you receive with some element, for example:
WebResponse response = webRequest.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
string responseString = responseReader.ReadToEnd();
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.LoadXML( "<root>" + responseString + "</root>" );
XmlNodeList address = xmlDoc.GetElementsByTagName("original")
Hope this helped
I'm trying to get an image from an url using a byte stream. But i get this error message:
This stream does not support seek operations.
This is my code:
byte[] b;
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
WebResponse myResp = myReq.GetResponse();
Stream stream = myResp.GetResponseStream();
int i;
using (BinaryReader br = new BinaryReader(stream))
{
i = (int)(stream.Length);
b = br.ReadBytes(i); // (500000);
}
myResp.Close();
return b;
What am i doing wrong guys?
You probably want something like this. Either checking the length fails, or the BinaryReader is doing seeks behind the scenes.
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
WebResponse myResp = myReq.GetResponse();
byte[] b = null;
using( Stream stream = myResp.GetResponseStream() )
using( MemoryStream ms = new MemoryStream() )
{
int count = 0;
do
{
byte[] buf = new byte[1024];
count = stream.Read(buf, 0, 1024);
ms.Write(buf, 0, count);
} while(stream.CanRead && count > 0);
b = ms.ToArray();
}
edit:
I checked using reflector, and it is the call to stream.Length that fails. GetResponseStream returns a ConnectStream, and the Length property on that class throws the exception that you saw. As other posters mentioned, you cannot reliably get the length of a HTTP response, so that makes sense.
Use a StreamReader instead:
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
WebResponse myResp = myReq.GetResponse();
StreamReader reader = new StreamReader(myResp.GetResponseStream());
return reader.ReadToEnd();
(Note - the above returns a String instead of a byte array)
You can't reliably ask an HTTP connection for its length. It's possible to get the server to send you the length in advance, but (a) that header is often missing and (b) it's not guaranteed to be correct.
Instead you should:
Create a fixed-length byte[] that you pass to the Stream.Read method
Create a List<byte>
After each read, call List.AddRange to append the contents of your fixed-length buffer onto your byte list
Note that the last call to Read will return fewer than the full number of bytes you asked for. Make sure you only append that number of bytes onto your List<byte> and not the whole byte[], or you'll get garbage at the end of your list.
If the server doesn't send a length specification in the HTTP header, the stream size is unknown, so you get the error when trying to use the Length property.
Read the stream in smaller chunks, until you reach the end of the stream.
With images, you don't need to read the number of bytes at all. Just do this:
Image img = null;
string path = "http://www.example.com/image.jpg";
WebRequest request = WebRequest.Create(path);
req.Credentials = CredentialCache.DefaultCredentials; // in case your URL has Windows auth
WebResponse resp = req.GetResponse();
using( Stream stream = resp.GetResponseStream() )
{
img = Image.FromStream(stream);
// then use the image
}
Perhaps you should use the System.Net.WebClient API. If already using client.OpenRead(url) use client.DownloadData(url)
var client = new System.Net.WebClient();
byte[] buffer = client.DownloadData(url);
using (var stream = new MemoryStream(buffer))
{
... your code using the stream ...
}
Obviously this downloads everything before the Stream is created, so it may defeat the purpose of using a Stream. webClient.DownloadData("https://your.url") gets a byte array which you can then turn into a MemoryStream.
The length of a stream can not be read from the stream since the receiver does not know how many bytes the sender will send. Try to put a protocol on top of http and send i.e. the length as first item in the stream.