I'm trying to send a POST request to a website in C#, and then parse the html in the response to get certain item names from it. However, I am getting a 408 error about 50% of the time I run the program. Here is my code (most is taken from here: https://msdn.microsoft.com/en-us/library/debx8sh9.aspx):
using System;
using System.IO;
using System.Net;
using System.Text;
using System.Collections.Generic;
using HtmlAgilityPack;
using System.Net;
using System.Diagnostics;
using Newtonsoft.Json.Linq;
using System.Linq;
using Fizzler.Systems.HtmlAgilityPack;
class FinderClass
{
//some irrelevant code here
public int getItemIndex(string itemName)
{
itemName = itemName.Replace(" ", "+"); //formatting for request
itemName = itemName.Replace("|", "%7C");
//taken from https://msdn.microsoft.com/en-us/library/debx8sh9(v=vs.110).aspx :
WebRequest request = WebRequest.Create("http://csgolounge.com/ajax/tradeCsRightTmp.php"); //address to send request
request.Method = "POST";
string postData = "type=Type+-+All&quality=0&exterior=0&fraze=" + itemName + "&search=1&page=1"; //request parameters
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = byteArray.Length;
Stream dataStream = request.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
WebResponse response = request.GetResponse();
Debug.WriteLine(((HttpWebResponse)response).StatusDescription);
dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = reader.ReadToEnd();
Debug.WriteLine(responseFromServer); //print response to debug console (temporary)
reader.Close();
dataStream.Close();
response.Close();
return -1; //placeholder for when item index is parsed from html and returned
}
}
The exception that is thrown:
An unhandled exception of type 'System.Net.WebException' occurred in System.dll
Additional information: The remote server returned an error: (408) Request Timeout.
Normally I would think there is a straightforward solution to this, but since it only happens half the time I run the program, it might be something more complicated.
Any time that I hear "so and such network problem happens after a while", I tend to look at the code and see if using blocks are being used correctly.
using blocks should be used whenever you
Create some resource, and
Use that resource, and
Finish with that resource in the same method
(there are some other times, but this is the most frequent case)
In your code, you are creating a WebResponse, a Stream, and a StreamReader, you're using them, then you're finishing with them, and you're not cleaning up.
Manually using a Dispose or Close call isn't always enough. These will get skipped over if an exception is thrown in the code and unhandled. A using block will ensure that the cleanup happens, even in the presence of unhandled exceptions.
This error is because of a timeout. On the face of it, it looks as though the server is timing out. So, I'd investigate the server side to see how long the call is taking. However, there's a chance that it may just be a client side problem because I sometimes get this error when I pause the debugger on the client side. I.e. the Request is sent to the server, the server sends the Response back, but the client doesn't accept it. I know this sounds funny, and I haven't been able to figure out why this happens myself. Anyway, you can try setting the timeout to something much larger in the WebRequest on the client side:
request.Timeout = 60000;
Related
I have class in c# which makes POST call to a REST API. BUt it gives me a 'connection time out' error with error code as 10060. In the POST call I am trying to make some transaction adjustments in the client's system. When I use Fiddler or Postman to make the api call, the request seems to go through but not from the c# class. Can you see where I am going wrong?
Below is my sample code.
HttpWebRequest request = (HttpWebRequest)WebRequest.CreateHttp(clientURL);
byte[] data = Encoding.UTF8.GetBytes(urlParameters);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = data.Length;
using (Stream stream = request.GetRequestStream()) **//exception occurs at this point**
{
stream.Write(data, 0, data.Length);
}
Thanks.
You have to close your webrequest by using "using" or adding a call to the HttpWebRequest.Abort() method
The default timeout for HttpWebRequest is 100 seconds. You can change that value by setting that property (in ms):
request.Timeout = 120000;
I'm trying to understand something about exception handling with a HttpWebRequest.
I have a client library and it's making a request to a WebAPI controller;
HttpWebRequest r = (HttpWebRequest)WebRequest.Create(url);
r.Method = "POST";
r.ContentType = "application/json";
foreach (var header in request.Headers)
{
r.Headers.Add(header.Key, header.Value.ToString());
}
r.ContentLength = request.RequestBody.Length;
using (StreamWriter writer = new StreamWriter(r.GetRequestStream()))
writer.Write(request.RequestBody);
I know the request will throw an exception, and contain the message entity already exists - 1234.
When I get the response;
using (HttpWebResponse response = (HttpWebResponse)r.GetResponse())
{
if (response.StatusCode == HttpStatusCode.OK)
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
return reader.ReadToEnd();
return "Invalid";
}
I get a WebException thrown. So, the caller of the request has a try..catch in it. And I get the WebException. What I get is a protocol error, not the 500 internal server error that was thrown (using correct status codes to represent the message comes later). Now if I read the Response of the WebException, it does contain my message and the stacktrace.
Questions
Why do I not get a status code of 500 in my response, why does it throw a protocol error?
Is there a more correct way of handling the request?
I have searched around and found some people getting this issue when not using the correct headers etc. But as far as I can tell, I have added all the headers that I can and still get the same behavior.
An 500 internal server error usually means that the API received the request but threw an unhandled exception while processing it, thus the "Internal Server Error".
You may log to a database or file all your API's unhandled exceptions to help your debugging process. Good luck.
I'm trying to get data from the Microsoft Live API. However, when I try to get the access_token, I instead get a 415(Unsupported Media Type) error message. I have looked pretty much everywhere, but I can't find any answer (that worked for me).
Here is my (partial) code that tries to get the token (dataToWrite is cut-up for readability, it's one line in the actual code):
WebRequest request;
request = WebRequest.Create("https://login.live.com/oauth20_token.srf");
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
var dataToWrite = "code=[code]&
client_id=[client_id]&
client_secret=[client_secret]&
redirect_uri=[redirect_uri]&
grant_type=authorization_code";
var buffer = Encoding.ASCII.GetBytes(dataToWrite);
request.ContentLength = buffer.Length;
var dataStream = request.GetRequestStream();
dataStream.Write(buffer, 0, buffer.Length);
dataStream.Close();
var response = request.GetResponse();
var responseStream = response.GetResponseStream();
Where the '[]' are:
[code] is a string, given by Microsoft after user logs in (this part of the code works);
[client_id] is a string, given by Microsoft, representing my client id;
[client_secret] is a string, given by Microsoft, representing my client secret;
[redirect_uri] is the URL of the site's return location (same as the URL used in the code for the user consent(see [code]))
According to the manual of Microsoft Live API(http://msdn.microsoft.com/en-us/library/live/hh243647.aspx) this should work. However, the documentation isn't very detailed.
Does anyone know why I keep getting the error?
Thanks!
Never mind, I'm an idiot...
It does work after all. I did another request after this one. And that one failed because I did not include the parameters there.
Good day.
I really need help on this issue. I have tried every possible option here.
I use a REST API in an Outlook add-in using C#. The code links outlook items to CRM records, one way. The add-in works 100% fine but after a couple of calls outs i keep on getting the error "The operation has timed out".
When I use the Google Chrome App "Advanced REST Client" I can post the same request 50 times after each other with no time out error.
From within the add-in I use POST, GET and PATCH HttpWebRequest and I get the error for all of them. The error happens at the code line System.IO.Stream os = req.GetRequestStream();
Below is the method:
public static string HttpPatch(string URI, string Parameters)
{
var req = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(URI);
if (GlobalSettings.useproxy.Equals("true"))
{
req.Proxy = WebRequest.DefaultWebProxy;
req.Credentials = new NetworkCredential(GlobalSettings.proxyusername, GlobalSettings.proxypassword, GlobalSettings.proxydomain);
req.Proxy.Credentials = new NetworkCredential(GlobalSettings.proxyusername, GlobalSettings.proxypassword, GlobalSettings.proxydomain);
}
req.Headers.Add("Authorization: OAuth " + GlobalSettings.token.access_token);
req.ContentType = "application/json";
req.Method = "PATCH";
byte[] data = System.Text.Encoding.Unicode.GetBytes(Parameters);
req.ContentLength = data.Length;
using (System.IO.Stream os = req.GetRequestStream())
{
os.Write(data, 0, data.Length);
os.Close();
}
WebResponse resp;
try
{
resp = req.GetResponse();
}
catch (WebException ex)
{
if (ex.Message.Contains("401"))
{
}
}
}
I suspect the problem is that you're not disposing of the WebResponse. That means the connection pool thinks that the connection is still in use, and will wait for the response to be disposed before reusing it for another request. The connection is needed in order to get a request stream, and it won't become available unless the finalizer happens to kick in at a useful time, hence the timeout.
Simply change your code using the response to use a using statement - or in your case, potentially something a little more complicated using a finally block as you're assigning it within a try block. (We can't really see how you're using the response, which makes it hard to give sample code around that. But fundamentally, you need to dispose it.)
My app currently uses OAuth to communicate with the Twitter API. Back in December, Twitter upped the rate limit for OAuth to 350 requests per hour. However, I am not seeing this. I am still getting 150 from the account/rate_limit_status method.
I was told that I needed to use the X-RateLimit-Limit HTTP header to get the new rate limit. However, in my code, I do not see that header.
Here is my code...
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(newURL);
request.Method = "GET";
request.ServicePoint.Expect100Continue = false;
request.ContentType = "application/x-www-form-urlencoded";
using (WebResponse response = request.GetResponse())
{
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
responseString = reader.ReadToEnd();
}
}
If I inspect the response, I can see that it has a property for Headers, and that there are 16 headers. However, I do not have X-RateLimit-Limit in the list.
(source: yfrog.com)
Any idea what I am doing wrong?
You should simple be able to use:
using (WebResponse response = request.GetResponse())
{
string limit = response.Headers["X-RateLimit-Limit"];
...
}
If that doesn't work as expected, you can do a watch on response.Headers and see what's in there.
Look at the raw response text (e.g., with Fiddler). If the header isn't there, no amount of C# code is going to make it appear. :) From what you've shown, it seems the header isn't in the response.
Update: When I go to: http://twitter.com/account/rate_limit_status.xml there is no X-RateLimit-Limit header. But when I go to http://twitter.com/statuses/public_timeline.xml, it's there. So I think you just need to use a different method.
It still says 150, though!