I'm trying to read the reponse from a webserver using httpwebrequests in C#.
I use the following code:
UriBuilder urib = new UriBuilder();
urib.Host = "wikipedia.com";
HttpWebRequest req = WebRequest.CreateHttp(urib.Uri);
req.KeepAlive = false;
req.Host = "wikipedia.com/";
req.Method = "GET";
HttpWebResponse response = (HttpWebResponse) req.GetResponse();
byte[] buffer = new byte[response.ContentLength];
System.IO.Stream stream = response.GetResponseStream();
stream.Read(buffer, 0, buffer.Length);
Console.WriteLine(System.Text.Encoding.ASCII.GetString(buffer, 0, buffer.Length));
The code does indeed retrieve the correct amount of data (I compared the contentlength used to create the buffer, with the length of the console output, they're the same.
My problem is that the last 80% or so of the response is blank chars. They're all 0x00.
I tested this with several pages, including wikipedia.com and it just cuts off mid-file for some reason.
Have I misunderstood/misused the way to use webrequests or can anyone spot an error here?
Try to use this method:
public static String GetResponseString(Uri url, CookieContainer cc)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.Method = WebRequestMethods.Http.Get;
request.CookieContainer = cc;
request.AutomaticDecompression = DecompressionMethods.GZip;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
String responseString = reader.ReadToEnd();
response.Close();
return responseString;
}
There are a couple of issues with your code:
Your trying to read the entire response in one go using Stream.Read - that's not what it was designed for. This should be used for more optimal reading e.g. 4KB chunks.
Your reading a HTML response as ASCII encoding - are you sure the page doesn't contain any Unicode characters? I would stick to UTF-8 encoding to be on the safe side (or alternatively read the Content-Type header in the response).
When reading characters from a byte stream (which is what your response is essentially) the recommended approach is to use StreamReader. More specifically, if you want to read the entire stream in one go then use StreamReader.ReadToEnd.
Your code could be shortened to:
HttpWebRequest req = WebRequest.CreateHttp(new Uri("http://wikipedia.org"));
req.Method = WebRequestMethods.Http.Get;
using (var response = (HttpWebResponse)req.GetResponse())
using (var reader = new StreamReader(response.GetResponseStream()))
{
Console.WriteLine(reader.ReadToEnd());
}
Related
I am trying to use an API and I don't have any problems with GET and POST but PUT isn't working. I tried with a lot of different examples and finally by chance I discovered that waiting more that 5 seconds (with 5000ms it is not working and with 5100ms it does) it starts working properly. But why is that happening? And how can I avoid this? 5 seconds for each registry update is to much waiting and I really don't understand why POST works well without waiting and PUT needs 5 seconds to work.
Here I put the method that I am using with the Thread.Sleep(5100). As I said without this line when I make WebResponse response = request.GetResponse(); gives me an error.
public void call(string url, object jsonObj)
{
try
{
// Create a request using a URL that can receive a post.
HttpWebRequest request = (HttpWebRequest) HttpWebRequest.Create(urlSplio);
// Create POST data and convert it to a byte array.
request.Method = "PUT";
// Set the ContentType property of the WebRequest.
request.ContentType = "application/json";
request.Credentials = new NetworkCredential(WebConfigurationManager.AppSettings["User"], "WebConfigurationManager.AppSettings["Key"]");
string json = JsonConvert.SerializeObject(jsonObj);
byte[] byteArray = Encoding.UTF8.GetBytes(json);
// Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length;
// Get the request stream.
Stream dataStream = request.GetRequestStream();
// Write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);
// Close the Stream object.
dataStream.Close();
Thread.Sleep(5100);
// Get the response.
WebResponse response = request.GetResponse();
// Display the status.
Console.WriteLine(((HttpWebResponse)response).StatusDescription);
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd();
// Display the content.
Console.WriteLine(responseFromServer);
// Clean up the streams.
dataStream.Close();
response.Close();
}
catch (Exception ex)
{
}
}
I think you might want to rewrite the response stream code
Take a look at this walkthrough on MS MS walkthrough
private byte[] GetURLContents(string url)
{
// The downloaded resource ends up in the variable named content.
var content = new MemoryStream();
// Initialize an HttpWebRequest for the current URL.
var webReq = (HttpWebRequest)WebRequest.Create(url);
// Send the request to the Internet resource and wait for
// the response.
// Note: you can't use HttpWebRequest.GetResponse in a Windows Store app.
using (WebResponse response = webReq.GetResponse())
{
// Get the data stream that is associated with the specified URL.
using (Stream responseStream = response.GetResponseStream())
{
// Read the bytes in responseStream and copy them to content.
responseStream.CopyTo(content);
}
}
// Return the result as a byte array.
return content.ToArray();
}
I am trying to get the access token from MNS for Push notifications and the WebRequest.GetRequestStreamAsync() method timesout each time. Any suggestions?
Reference: http://msdn.microsoft.com/en-us/library/windows/apps/hh913756.aspx
Below is the code I use
HttpWebRequest webRequest = (HttpWebRequest)HttpWebRequest.Create("https://login.live.com/accesstoken.srf");
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
string postString = String.Format("grant_type=client_credentials&client_id={0}&client_secret={1}&scope=notify.windows.com",
SID,
SECRET_KEY);
byte[] data = Encoding.UTF8.GetBytes(postString);
Stream newStream = await webRequest.GetRequestStreamAsync();
newStream.Write(data, 0, data.Length);
Try rewriting it like this and see if it makes a difference. I've sometimes had problems with HttpWebRequest where WebRequest worked fine. Also make sure you close your streams.
WebRequest webRequest = WebRequest.Create("https://login.live.com/accesstoken.srf");
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
string postString = String.Format("grant_type=client_credentials&client_id={0}&client_secret={1}&scope=notify.windows.com",
SID,
SECRET_KEY);
byte[] data = Encoding.UTF8.GetBytes(postString);
Stream newStream = await webRequest.GetRequestStreamAsync();
newStream.Write(data, 0, data.Length);
newStream.Close();
WebResponse response = webRequest.GetResponse();
StreamReader requestReader = new StreamReader( response.GetResponseStream() );
string webResponse = requestReader.ReadToEnd();
response.Close();
Call .ConfigureAwait(false) on your Async method.
This blog post should explain the why and how.
We had the same problem and it turned out to be a problem in different place than one would originally guess.
You need to .Dispose() or at least .Close() the response that you get from .GetResponseAsync, otherwise the next call to .GetRequestStreamAsync hangs.
It seems that the code behind this holds some limited (rather low) amount of sockets or locks, that disallow further requests to even begin until previous request has completed.
Despite trying lots of things (see below), I can't get rid of the "Bytes to be written to the stream exceed the Content-Length bytes size specified." error that's thrown in
writer.Close();
This is the code that tries to post data from an ASP.NET to a php site. The script works fine as long as there are no special characters in the code - note the German Umlaut in 'Wörld'.
Uri uri = new Uri("http://mydomain/test.php");
string data = #"data=Hello Wörld";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
request.Method = WebRequestMethods.Http.Post;
request.ContentLength = data.Length;
request.ContentType = "application/x-www-form-urlencoded";
StreamWriter writer = new StreamWriter(request.GetRequestStream());
writer.Write(data);
writer.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string tmp = reader.ReadToEnd();
response.Close();
Response.Write(tmp);
I have tried different variations using UTF-8 encodings, like:
request.ContentLength = System.Text.Encoding.UTF8.GetByteCount(data);
and/or
StreamWriter writer = new StreamWriter(request.GetRequestStream(), Encoding.UTF8);
I have also tried to convert the data to UTF-8 before sending it (somewhat ugly):
data = System.Text.Encoding.UTF8.GetString(System.Text.Encoding.Convert(System.Text.Encoding.UTF8, System.Text.Encoding.UTF8, System.Text.Encoding.UTF8.GetBytes(data)));
Yet the error remains. My feeling is that I just don't get the UTF-8 handling right. Any help is greatly appreciated, also any hint where I can find a perfectly working script that posts to php from ASP.NET (server side).
use
byte[] bdata = Encoding.UTF8.GetBytes(data);
and
request.ContentLength = bdata.Length;
and
Stream writer = request.GetRequestStream();
writer.Write(bdata, 0, bdata.Length);
writer.Close();
I'm pretty RESTless right now because I keep getting incomplete responses from Amazon. I'm using the Product Advertising API, making one ItemLookup request to the server.
The C# code is pretty basic:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
string resultString;
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
resultString = sr.ReadToEnd();
}
The number of chars I recieve is 17408- pretty constant but I've seen something around 16k as well.
This is how it always ends:
...ount><CurrencyCode>EUR</CurrencyCode><FormattedPrice>EUR 11,33</FormattedPri
Is there something I don't know about HttpWebRequest or Amazon's API?
the url (don't care about the key) edit: bad idea, limit exceeded...
This worked for me:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
Stream s = response.GetResponseStream();
using (StreamReader sr = new StreamReader(s))
{
s.Flush();
resultString = sr.ReadToEnd();
...
}
I've got a problem with creating an HTTP post request in .NET. When I do this request in ruby it does work.
When doing the request in .NET I get following error:
<h1>FOXISAPI call failed</h1><p><b>Progid is:</b> carejobs.carejobs
<p><b>Method is:</b> importvacature/
<p><b>Parameters are:</b>
<p><b> parameters are:</b> vacature.deelnemernr=478
</b><p><b>GetIDsOfNames failed with err code 80020006: Unknown name.
</b>
Does anyone knows how to fix this?
Ruby:
require 'net/http'
url = URI.parse('http://www.carejobs.be/scripts/foxisapi.dll/carejobs.carejobs.importvacature')
post_args = {
'vacature.deelnemernr' => '478',
}
resp, data = Net::HTTP.post_form(url, post_args)
print resp
print data
C#:
Uri address = new Uri(url);
// Create the web request
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
// Set type to POST
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
// Create the data we want to send
StringBuilder data = new StringBuilder();
data.Append("vacature.deelnemernr=" + HttpUtility.UrlEncode("478"));
// Create a byte array of the data we want to send
byte[] byteData = UTF8Encoding.UTF8.GetBytes(data.ToString());
// Set the content length in the request headers
request.ContentLength = byteData.Length;
// Write data
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(byteData, 0, byteData.Length);
}
// Get response
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
StreamReader reader = new StreamReader(response.GetResponseStream());
// Console application output
result = reader.ReadToEnd();
}
return result;
Don't you need the ? after the URL in order to do a post with parameters? I think that Ruby hides this behind the scenes.
I found the problem! The url variable in the C# code was "http://www.carejobs.be/scripts/foxisapi.dll/carejobs.carejobs.importvacature/"
It had to be "http://www.carejobs.be/scripts/foxisapi.dll/carejobs.carejobs.importvacature" without the backslash.