I am attempting to send XML to a URL and read the response, but the response is coming back empty every time. I think this is because its being processed Asynchronously and so the receiving code hasn't had a chance to complete by the time I read the response. In Javascrpt I would use
xmlhttp.Open("POST", url, false);
to send a request Synchronously. How can I achieve it in C#?
My code is currently
HttpWebRequest objRequest = (HttpWebRequest)WebRequest.Create(url);
objRequest.Credentials = CredentialCache.DefaultCredentials;
objRequest.Method = "POST";
objRequest.ContentType = "text/xml";
Stream dataStream = objRequest.GetRequestStream();
byte[] bytes = new byte[UpliftJobXMLString.Length * sizeof(char)];
System.Buffer.BlockCopy(UpliftJobXMLString.ToCharArray(), 0, bytes, 0, bytes.Length);
dataStream.Write(bytes, 0, bytes.Length);
dataStream.Close();
HttpWebResponse response = (HttpWebResponse)objRequest.GetResponse();
System.IO.StreamReader sr = new System.IO.StreamReader(response.GetResponseStream());
string respString = System.Web.HttpUtility.HtmlDecode(sr.ReadToEnd()); //always empty
Thanks
I'm fairly certain that this is not an async issue. Have you checked what sr.ReadToEnd() returns before the HtmlDecode?
Furthermore, you should check that the server is returning what you're expecting for it to return. Check the response StatusCode and StatusDescription. If your server is throwing an internal server exception (500) or something similar, the response string you read would come up empty as the content of the response would not be sent by the server in the first place.
I don't think your problem is related to sync/async operations. Your code to convert the string to byte array
byte[] bytes = new byte[UpliftJobXMLString.Length * sizeof(char)];
System.Buffer.BlockCopy(UpliftJobXMLString.ToCharArray(), 0, bytes, 0, bytes.Length);
is similar to Unicode encoding(2 bytes per char).
See the differences among encodings
string UpliftJobXMLString = "abcÜ";
byte[] bytesASCII = Encoding.ASCII.GetBytes(UpliftJobXMLString);
byte[] bytesUTF8 = Encoding.UTF8.GetBytes(UpliftJobXMLString);
byte[] bytesUnicode = Encoding.Unicode.GetBytes(UpliftJobXMLString);
Therefore, either set the content-encoding to unicode or use another encoding. For ex;
objRequest.ContentType = "text/xml; charset=utf-8";
Related
I have this code, calling this sample API call of the game Guild Wars 2
HttpWebRequest request = HttpWebRequest.Create("https://api.guildwars2.com/v1/events.json?world_id=1001") as HttpWebRequest;
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
byte[] bytes = new byte[response.ContentLength];
Stream stream = response.GetResponseStream();
stream.Read(bytes, 0, bytes.Length);
string result = Encoding.Default.GetString(bytes);
It is a sample call to the official Guild Wars 2 API and is supposed to return a list of events and their status in JSON format.
If you call the address in a browser and paste the data into a text editor, it gives a correct JSON string, of a little more than 300 kb.
However, when called by this code and, looking at the resulting byte array, the first 3800-3900 bytes are correctly filled (the number varies slightly from call to call), but the rest is all zero.
The response.ContentLength indicates the correct length of the stream, a little more than 300k, and testing with stream.ReadByte() tells me the stream does deliver those 300k, it's just zeroes after that 3850 byte mark.
Is there an error in my code, or what could cause this problem? Could I call the API in another way to get the correct, full response?
The Read operation on a stream returns the number of bytes actually read. You should keep calling stream.Read() until it returns 0. The data simply has not arrived yet.
The reason Read won't block is that some applications may already begin processing the partial data.
You could use the following code:
HttpWebRequest request = HttpWebRequest.Create("https://api.guildwars2.com/v1/events.json?world_id=1001") as HttpWebRequest;
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
byte[] bytes;
using (Stream stream = response.GetResponseStream())
using (MemoryStream buffer = new MemoryStream((int)response.ContentLength))
{
byte[] chunk = new byte[4096];
int bytesRead;
while ((bytesRead = stream.Read(chunk, 0, chunk.Length)) > 0)
{
buffer.Write(chunk, 0, bytesRead);
}
bytes = buffer.ToArray();
}
I'm trying to read the reponse from a webserver using httpwebrequests in C#.
I use the following code:
UriBuilder urib = new UriBuilder();
urib.Host = "wikipedia.com";
HttpWebRequest req = WebRequest.CreateHttp(urib.Uri);
req.KeepAlive = false;
req.Host = "wikipedia.com/";
req.Method = "GET";
HttpWebResponse response = (HttpWebResponse) req.GetResponse();
byte[] buffer = new byte[response.ContentLength];
System.IO.Stream stream = response.GetResponseStream();
stream.Read(buffer, 0, buffer.Length);
Console.WriteLine(System.Text.Encoding.ASCII.GetString(buffer, 0, buffer.Length));
The code does indeed retrieve the correct amount of data (I compared the contentlength used to create the buffer, with the length of the console output, they're the same.
My problem is that the last 80% or so of the response is blank chars. They're all 0x00.
I tested this with several pages, including wikipedia.com and it just cuts off mid-file for some reason.
Have I misunderstood/misused the way to use webrequests or can anyone spot an error here?
Try to use this method:
public static String GetResponseString(Uri url, CookieContainer cc)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.Method = WebRequestMethods.Http.Get;
request.CookieContainer = cc;
request.AutomaticDecompression = DecompressionMethods.GZip;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
String responseString = reader.ReadToEnd();
response.Close();
return responseString;
}
There are a couple of issues with your code:
Your trying to read the entire response in one go using Stream.Read - that's not what it was designed for. This should be used for more optimal reading e.g. 4KB chunks.
Your reading a HTML response as ASCII encoding - are you sure the page doesn't contain any Unicode characters? I would stick to UTF-8 encoding to be on the safe side (or alternatively read the Content-Type header in the response).
When reading characters from a byte stream (which is what your response is essentially) the recommended approach is to use StreamReader. More specifically, if you want to read the entire stream in one go then use StreamReader.ReadToEnd.
Your code could be shortened to:
HttpWebRequest req = WebRequest.CreateHttp(new Uri("http://wikipedia.org"));
req.Method = WebRequestMethods.Http.Get;
using (var response = (HttpWebResponse)req.GetResponse())
using (var reader = new StreamReader(response.GetResponseStream()))
{
Console.WriteLine(reader.ReadToEnd());
}
I am trying to send contents of 1GB text file over the network. I modified the suggested code for basic authentication and kept it as follows :
WRequest = (HttpWebRequest)HttpWebRequest.Create(URL);
WRequest.Credentials = Credentials;
WRequest.PreAuthenticate = true;
WRequest.ContentType = "text/plain";
WRequest.Method = "POST";
WRequest.AllowWriteStreamBuffering = false;
WRequest.Timeout = 10000;
FileStream ReadIn = new FileStream(filename, FileMode.Open, FileAccess.Read);
ReadIn.Seek(0, SeekOrigin.Begin);
WRequest.ContentLength = ReadIn.Length;
Byte[] FileData = new Byte[ReadIn.Length];
int DataRead = 0;
Stream tempStream = WRequest.GetRequestStream();
do
{
DataRead = ReadIn.Read(FileData, 0, 2048);
if (DataRead > 0)
{
tempStream.Write(FileData, 0, DataRead);
Array.Clear(FileData, 0, 2048);
}
} while (DataRead > 0);
// The response
WResponse = (HttpWebResponse)WRequest.GetResponse();
However, now it gives me System.Net.ProtocolViolationException error : "You must write ContentLength bytes to the request stream before calling [Begin]GetResponse". I checked HttpWebRequest.BeginGetRequestResponse ... and found from debugging that the contentlength for WRequest is not -1. What else could be going wrong ? How should I get the response ?
Update :
The code which worked for small files is as followed :
WebRequest request = WebRequest.Create(url);
request.Method = "POST";
request.Credentials = Credentials;
using (StreamReader reader = new StreamReader(filename))
{
postData = reader.ReadToEnd();
}
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
request.ContentType = "text/plain";
request.ContentLength = byteArray.Length;
Stream dataStream = request.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
// The response
WebResponse response = request.GetResponse();
Console.WriteLine(((HttpWebResponse)response).StatusDescription);
dataStream = response.GetResponseStream();
using (StreamReader reader = new StreamReader(dataStream))
{
responseFromServer = reader.ReadToEnd();
}
dataStream.Close();
response.Close();
The article you referenced says
If the Microsoft Internet Information Services (IIS) Web server is configured to use Basic authentication, and you must set the HttpWebRequest.AllowWriteStreamBuffering property to false, you must send a HEAD request to pre-authenticate the connection before you send the POST or PUT request.
EDIT - now with more clarification!
To restate the article, if you want to send a large file to a destination which requires basic authentication, you'll need to issue two separate requests. The key here is that you are setting PreAuthenticate = true. Read the statement literally -- by setting the property to true, you are saying that you will authenticate any requests that you make before you actually attempt them! The framework doesn't know how you want to accomplish this pre-authentication, so you need to perform that action yourself, by sending a HEAD request to the destination. Think of the HEAD HTTP method as being a prologue to the actual request - it describes (or requests information about) a particular resource.
So the process goes like this:
Make a HEAD request to http://someurl/aresource containing the credentials you want to use when making future requests from this client to that server for the listed resource
The server will respond (ideally) with "OK - you may proceed. You're authenticated"
The server immediately regrets its' decision to allow the operation as it finds itself saving a very large file :-)
I don't see you making that HEAD request anywhere in the code you posted - if it's not already there, add this at the beginning of your code (snipped from the sample article ref in OP):
//preAuth the request
// You can add logic so that you only pre-authenticate the very first request.
// You should not have to pre-authenticate each request.
WRequest = (HttpWebRequest)HttpWebRequest.Create(URL);
// Set the username and the password.
WRequest.Credentials = new NetworkCredential(user, password);
WRequest.PreAuthenticate = true;
WRequest.UserAgent = "Upload Test";
WRequest.Method = "HEAD";
WRequest.Timeout = 10000;
WResponse = (HttpWebResponse)WRequest.GetResponse();
WResponse.Close();
// Make the real request.
I am working with a REST API and I am trying to do a PUT method to it. I found this code I was going to give a try:
static void Main()
{
string xml = "<xml>...</xml>";
byte[] arr = System.Text.Encoding.UTF8.GetBytes(xml);
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("http://localhost/");
request.Method = "PUT";
request.ContentType = "text/xml";
request.ContentLength = arr.Length;
Stream dataStream = request.GetRequestStream();
dataStream.Write(arr, 0, arr.Length);
dataStream.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string returnString = response.StatusCode.ToString();
Console.WriteLine(returnString);
}
One thing I want to do if possible, and can't seem to find anything about it. I would like to pass the data of text fields so, txtEmail.Text, txtFirstName.Text, etc. Is this possible? If, so how would I go about doing this? Does this code look like it would work? Unfortunately the API I'm working with has very very little documentation. Thanks!
The code lines
Stream dataStream = request.GetRequestStream();
dataStream.Write(arr, 0, arr.Length);
dataStream.Close();
do write something to the remote website. The request stream is the way to provide data to the server, so you would create a string / object that you write to the stream to transfer to the server. In your example <xml>...</xml> is sent to the server.
Despite trying lots of things (see below), I can't get rid of the "Bytes to be written to the stream exceed the Content-Length bytes size specified." error that's thrown in
writer.Close();
This is the code that tries to post data from an ASP.NET to a php site. The script works fine as long as there are no special characters in the code - note the German Umlaut in 'Wörld'.
Uri uri = new Uri("http://mydomain/test.php");
string data = #"data=Hello Wörld";
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
request.Method = WebRequestMethods.Http.Post;
request.ContentLength = data.Length;
request.ContentType = "application/x-www-form-urlencoded";
StreamWriter writer = new StreamWriter(request.GetRequestStream());
writer.Write(data);
writer.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string tmp = reader.ReadToEnd();
response.Close();
Response.Write(tmp);
I have tried different variations using UTF-8 encodings, like:
request.ContentLength = System.Text.Encoding.UTF8.GetByteCount(data);
and/or
StreamWriter writer = new StreamWriter(request.GetRequestStream(), Encoding.UTF8);
I have also tried to convert the data to UTF-8 before sending it (somewhat ugly):
data = System.Text.Encoding.UTF8.GetString(System.Text.Encoding.Convert(System.Text.Encoding.UTF8, System.Text.Encoding.UTF8, System.Text.Encoding.UTF8.GetBytes(data)));
Yet the error remains. My feeling is that I just don't get the UTF-8 handling right. Any help is greatly appreciated, also any hint where I can find a perfectly working script that posts to php from ASP.NET (server side).
use
byte[] bdata = Encoding.UTF8.GetBytes(data);
and
request.ContentLength = bdata.Length;
and
Stream writer = request.GetRequestStream();
writer.Write(bdata, 0, bdata.Length);
writer.Close();