I am wondering how can I check if the RestSharp request I made failed because the server is down vs something else.
When I shutdown my server I get a status code of "NotFound" but that could be a particular record was not found(which I do on my site if say they try to find a record that might be recently deleted).
How can I figure out the server is actually down?
Edit
here is my code
private readonly RestClient client = new RestClient(GlobalVariables.ApiUrl);
var request = new RestRequest("MyController", Method.POST);
request.AddParameter("UserId", "1");
request.AddParameter("Name", name.Trim());
var asyncHandle = client.ExecuteAsync(request, response =>
{
var status = response.StatusCode;
});
When the server is down, it should not return a "404 NotFound" error.
The most appropriate in this case is HTTP Error 503 - Service unavailable
The Web server (running the Web site) is currently unable to handle
the HTTP request due to a temporary overloading or maintenance of the
server. The implication is that this is a temporary condition which
will be alleviated after some delay. Some servers in this state may
also simply refuse the socket connection, in which case a different
error may be generated because the socket creation timed out.
That way checking that RestResponse.StatusCode is 503 it will tell you that the server is down.
I am having the same issue.
I decided to check for the ContentLength too. At least my webservice always returns ContentLength>0 (even for NotFound occurrences). This seems to work out.
if ( response.StatusCode == System.Net.HttpStatusCode.NotFound &&
response.ContentLength == -1 ){
==> client couldn't connect to webservice
}
In my scenario, I needed to check if anything happened to the connection, e.g. connection time out, can't resolve the host name, etc. So I also added following code:
try
{
var client = new RestClient(_configuration.ServerUrl);
var request = new RestRequest
{
Resource = _configuration.SomeUrl,
Method = Method.POST
};
var response = client.Execute(request);
if (response.ErrorException != null)
{
throw response.ErrorException;
}
}
catch (WebException ex)
{
// TODO: check ex.Status, if it matches one of needed conditions.
}
For more information about WebExceptionStatus Enumeration, please, see following link https://msdn.microsoft.com/en-us/library/system.net.webexceptionstatus(v=vs.110).aspx
Related
I am working on Windows Service in visual studio 2017. In the rest api's call, getting exceptions while debugging code. Sometimes first 2 3 calls working after that getting exceptions.
System.Net.WebException: 'The remote server returned an error: (503)
Server Unavailable.'
The remote server returned an error: (429)
Unable to connect to the remote server
When calling same api's from Postman, getting response successfully.
This is my code
private void timer1_Tick(object sender, ElapsedEventArgs e)
{
WriteToFile("timer1_Tick method called..");
try
{
string jsonString = "";
string jsonstring2 = "";
string prodfetchurl = HOST;
var req = WebRequest.Create(prodfetchurl) as HttpWebRequest;
req.Method = "GET";
InitializeRequest(req);
req.Accept = MIME_TYPE;
//System.Threading.Thread.Sleep(5000);
var response = (HttpWebResponse)req.GetResponse();
WriteToFile("First service called...");
if (response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
jsonString = responseReader.ReadToEnd();
}
var deserialsseobj = JsonConvert.DeserializeObject<ProductList>(jsonString).Products.Where(i => i.Failed > 0).ToList();
foreach (var a in deserialsseobj)
{
var pid = a.ID;
string url = FailedDevicesUrl + pid.Value + "/failed";
var req2 = WebRequest.Create(url) as HttpWebRequest;
req2.Method = "GET";
InitializeRequest(req2);
req2.Timeout = 300000;
req2.Accept = MIME_TYPE;
var response1 = (HttpWebResponse)req2.GetResponse();
Stream responsestream2 = response1.GetResponseStream();
WriteToFile("Second service called...");
if (response1.StatusCode == HttpStatusCode.OK)
{
StreamReader responsereader1 = new StreamReader(responsestream2);
jsonstring2 = responsereader1.ReadToEnd();
}
var output = JsonConvert.DeserializeObject<List<FailedDeviceList>>(jsonstring2); // Will get List of the Failed devices
List<int> deviceids = new List<int>();
Reprocessdata reproc = new Reprocessdata();
Reprocessdata.DeviceId rprod = new Reprocessdata.DeviceId();
reproc.ForceFlag = true;
reproc.ProductID = pid.Value;
foreach (var dd in output)
{
rprod.ID = dd.DeviceId;
reproc.DeviceIds.Add(rprod);
}
// Reprocess the Product in Devices
var req3 = WebRequest.Create(ReprocessUrl) as HttpWebRequest;
req3.Method = "POST";
InitializeRequest(req3);
req3.Accept = MIME_TYPE;
req3.Timeout = 300000;
req3.ContentType = "application/json";
using (StreamWriter writer = new StreamWriter(req3.GetRequestStream()))
{
string json = new JavaScriptSerializer().Serialize(reproc);
writer.Write(json);
writer.Close();
}
System.Threading.Thread.Sleep(5000);
var response5 = (HttpWebResponse)req3.GetResponse();
WriteToFile("Third service called...");
if (response5.StatusCode == HttpStatusCode.OK)
{
string result;
using (StreamReader rdr = new StreamReader(response5.GetResponseStream()))
{
result = rdr.ReadToEnd();
}
}
}
response.Close();
}
catch (Exception ex)
{
WriteToFile("Simple Service Error on: {0} " + ex.Message + ex.StackTrace);
}
}
Methods used in above code
protected override void OnStart(string[] args)
{
base.OnStart(args);
timer1 = new System.Timers.Timer();
timer1.Interval = 60000; //every 1 min
timer1.Elapsed += new System.Timers.ElapsedEventHandler(timer1_Tick);
timer1.Enabled = true;
WriteToFile("Service has started..");
}
public void InitializeRequest(HttpWebRequest request)
{
request.Headers.Add("aw-tenant-code", API_TENANT_CODE);
request.Credentials = new NetworkCredential(USER_NAME, PASSWORD);
request.KeepAlive = false;
request.AddRange(1024);
}
When I contacted service provide they said everything fine from there side. Is this my code is buggy or windows service not reliable? How can I fix this issue?
Note: All APIS are working fine from Angular application using Visual Studio Code. It means my code is not working.
Edit1: Three below services I am using from this document of VMware.
private const string HOST = "https:host/api/mdm/products/search?";
private const string FailedDevicesUrl = "https:host/api/mdm/products/";
private const string ReprocessUrl = "https:host/api/mdm/products/reprocessProduct";
Response http code 429 indicates that you sending too many requests on target web service.
This means service you trying to send requests has a policies that blocks some requests by request-per-time limit.
Also I admit that external service can be manually configured to throw 403 code in specific cases that you can't know about. If that, this information can be explained in external service documentation... or not :)
What you can do with this?
Fit in limitations
You can make detailed research what limits target webservice has and set up your code to fit in this limitations. For example if service has limitation for receiving only one request per 10 minutes - you must set up your timer to send one request each 10 or more minutes. If documentation not provide such information - you can test it manually by finding some patterns with external service responses.
Use proxy
Every limitation policy based on information about requests senders. Usually this information consists of IP address of sender only. This means if you send 2 requests from two different IP addresses - limitation policy will perceive that like 2 different computers sending these requests. So you can find/buy/rent some proxy IP addresses and send requests through there on target web server.
How to connect through proxy in C# using WebRequest you can see in this answer.
Negotiate with external service provider
If you have possibility to communicate with external service developers or help center, you can ask their to reduce limitations for your IP address (if it static) or provide some mechanisms to avoid limitation policy for you. If for some reason they cannot provide this opportunity, at least you can ask detailed information about limitations.
Repetition mechanism
Some times 503 error code that is outer exception you received may be caused by service unavailable. It means that server can be under maintenance or temporary overloaded. So you can write repetition mechanism to make continious sending requests to server until it'll be accessible.
Polly library may help you with repetition mechanism creation
The inner error of that 503 is:
The remote server returned an error: (429)
HTTP 429 indicates too many requests. Maybe your upstream server can’t process all requests sent.
This can happen when you reached rate limiting / throttling value if you’re calling a third party API.
UPDATE
As per page 28 in the API docs, you could configure throttling when creating a new API. Check if the throttling is too small or maybe turn off the throttling and see if that could fix the error?
I am sending cURL request using HttpClient through the method described here under.
The parameter used for this method are:
SelectedProxy = a custom class that stores my proxy's parameters
Parameters.WcTimeout = the timeout
url, header, content = the cURL request (based on this tool to convert to C# https://curl.olsh.me/).
const SslProtocols _Tls12 = (SslProtocols)0x00000C00;
const SecurityProtocolType Tls12 = (SecurityProtocolType)_Tls12;
ServicePointManager.SecurityProtocol = Tls12;
string source = "";
using (var handler = new HttpClientHandler())
{
handler.UseCookies = usecookies;
WebProxy wp = new WebProxy(SelectedProxy.Address);
handler.Proxy = wp;
using (var httpClient = new HttpClient(handler))
{
httpClient.Timeout = Parameters.WcTimeout;
using (var request = new HttpRequestMessage(new HttpMethod(HttpMethod), url))
{
if (headers != null)
{
foreach (var h in headers)
{
request.Headers.TryAddWithoutValidation(h.Item1, h.Item2);
}
}
if (content != "")
{
request.Content = new StringContent(content, Encoding.UTF8, "application/x-www-form-urlencoded");
}
HttpResponseMessage response = new HttpResponseMessage();
try
{
response = await httpClient.SendAsync(request);
}
catch (Exception e)
{
//Here the exception happens
}
source = await response.Content.ReadAsStringAsync();
}
}
}
return source;
If I am running this without proxy, it works like a charm.
When I send a request using a proxy which I tested first from Chrome, I have the following error on my try {} catch {}. Here is the error tree
{"An error occurred while sending the request."}
InnerException {"Unable to connect to the remote server"}
InnerException {"A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond [ProxyAdress]"}
SocketErrorCode: TimedOut
By using a Stopwatch I see that the TimedOut occurred after around 30 sec.
I tried a few different handler based on the following links What's the difference between HttpClient.Timeout and using the WebRequestHandler timeout properties?, HttpClient Timeout confusion or with the WinHttpHandler.
It's worth noting that WinHttpHandler allow for a different error code, i.e. Error 12002 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'The operation timed out'. The underlying reason is the same though it helped to target where it bugs (i.e. WinInet) which confirms also what #DavidWright was saying regarding that timeouts from HttpClient manages a different part of the request sending.
Hence my issue is coming from the time it takes to establish a connection to the server, which triggers the 30sec timeout from WinInet.
My question is then How to change those timeout?
On a side note, it's worth noting that Chrome, which uses WinInet, does not seem to suffer from this timeout, nor Cefsharp on which a big part of my app is based, and through which the same proxies can properly send requests.
So thanks to #DavidWright I understand a few things:
Before that the HttpRequestMessage is sent and the timeout from HttpClient starts, a TCP connection to the server is initiated
The TCP connection has its own timeout, defined at OS level, and we do not identified a way to change it at run time from C# (question pending if anyone want to contribute)
Insisting on trying to connect works as each try benefits from previous tries, though proper exception management & manual timeout counter needs to be implemented (I actually considered a number of tries in my code, assuming each try is around 30sec)
All this together ended up in the following code:
const SslProtocols _Tls12 = (SslProtocols)0x00000C00;
const SecurityProtocolType Tls12 = (SecurityProtocolType)_Tls12;
ServicePointManager.SecurityProtocol = Tls12;
var sp = ServicePointManager.FindServicePoint(endpoint);
sp.ConnectionLeaseTimeout = (int)Parameters.ConnectionLeaseTimeout.TotalMilliseconds;
string source = "";
using (var handler = new HttpClientHandler())
{
handler.UseCookies = usecookies;
WebProxy wp = new WebProxy(SelectedProxy.Address);
handler.Proxy = wp;
using (var client = new HttpClient(handler))
{
client.Timeout = Parameters.WcTimeout;
int n = 0;
back:
using (var request = new HttpRequestMessage(new HttpMethod(HttpMethod), endpoint))
{
if (headers != null)
{
foreach (var h in headers)
{
request.Headers.TryAddWithoutValidation(h.Item1, h.Item2);
}
}
if (content != "")
{
request.Content = new StringContent(content, Encoding.UTF8, "application/x-www-form-urlencoded");
}
HttpResponseMessage response = new HttpResponseMessage();
try
{
response = await client.SendAsync(request);
}
catch (Exception e)
{
if(e.InnerException != null)
{
if(e.InnerException.InnerException != null)
{
if (e.InnerException.InnerException.Message.Contains("A connection attempt failed because the connected party did not properly respond after"))
{
if (n <= Parameters.TCPMaxTries)
{
n++;
goto back;
}
}
}
}
// Manage here other exceptions
}
source = await response.Content.ReadAsStringAsync();
}
}
}
return source;
On a side note, my current implementation of HttpClient may be problematic in the future. Though being disposable, HttpClient should be defined at App level through a static, and not within a using statement. To read more about this go here or there.
My issue is that I want to renew the proxy at each request and that it is not set on a per request basis. While it explains the reasdon of the new ConnectionLeaseTimeout parameter (to minimize the time the lease remains open) it is a different topic
I have had the same problem with HttpClient. Two things need to happen for SendAsync to return: first, setting up the TCP channel over which the communication occurs (the SYN, SYN/ACK, ACK handshake, if you're familiar with that) and second getting back the data that constitutes the HTTP response over that TCP channel. HttpClient's timeout only applies to the second part. The timeout for the first part is governed by the OS's network subsystem, and it's quite difficult to change that timeout in .NET code.
(Here's how you can reproduce this effect. Set up a working client/server connection between two machines, so you know that name resolution, port access, listening, and client and server logic all works. Then unplug the network cable on the server and re-run the client request. It will time out with the OS's default network timeout, regardless of what timeout you set on your HttpClient.)
The only way I know around this is to start your own delay timer on a different thread and cancel the SendAsync task if the timer finishes first. You can do this using Task.Delay and Task.WaitAny or by creating a CancellationTokenSource with your desired timeone (which essentially just does the first way under the hood). In either case you will need to be careful about cancelling and reading exceptions from the task that loses the race.
I am currently running a method that takes a string (a domain name) and checks to see if the site is available or not and passes the information into a Domain object I have created. Currently, I am running into an issue where one of the sites is down and is in turn crashing the application. Below is the method:
private Domain GetStatus(string x)
{
string status = "";
WebRequest req = HttpWebRequest.Create("http://www." + x);
WebResponse res = req.GetResponse();
HttpWebResponse response = (HttpWebResponse)res;
if ((int)response.StatusCode > 226 || response.StatusCode == HttpStatusCode.NotFound)
{
status = "ERROR: " + response.StatusCode.ToString();
}
else
{
status = "LIVE";
}
Domain temp = new Domain(x, status);
return temp;
}
Initial thoughts were that the response.StatusCode == HttpStatusCode.NotFound would handle such an error but it is currently crashing on the line WebResponse res = req.GetResponse(); with the following response:
System.Net.WebException: 'The remote name could not be resolved: 'www.DOMAIN.com''
The issue is due to the fact that your own code is raising an exception.
This can be due to the lack of an internet connection, or a dns resolve issue (which could be caused by the remote party).
So, if the remote server throws an error, you'll get HTTP 500 Internal Server Error, if you can't reach it; your code throws an exception and you'll need to handle that.
To fix this, you can use a try/catch block, something like this:
private Domain GetStatus(string x)
{
string status = "";
try
{
WebRequest req = HttpWebRequest.Create("http://www." + x);
WebResponse res = req.GetResponse();
HttpWebResponse response = (HttpWebResponse)res;
if ((int)response.StatusCode > 226 ||
response.StatusCode == HttpStatusCode.NotFound)
{
status = "ERROR: " + response.StatusCode.ToString();
}
else
{
status = "LIVE";
}
}
catch (Exception e)
{
status = "ERROR: Something bad happend: " + e.ToString();
}
Domain temp = new Domain(x, status);
return temp;
}
By the way, the message,
The remote name could not be resolved
indicates that the host cannot be resolved.
Most likely cause is that your internet is down or, the domain is misspelled or the route to the domain is faulty (e.g. on intranet environments).
HttpWebRequest is all about HTTP protocol, which is kind of a agreed upon language.
But if the person on the other end doesn't exists, so how should you expect him to return you an "Hello" for example ?
So StatusCode is really just about if the actual remote site did response, what did the response state was according to the request resource, is it Successful(200) ? Not Found(404) ? Unauthorized(401) and so on.
Exception means, i couldn't reach the site because of many reasons.
StatusCode means the resource you requested has return this response type.
But a more actual check if the site is alive or not, is querying a static page and not getting exception, a more healthy check, will querying a static page, you will always count as being Healthy; meaning will return a 200 OK response.
So it all depends on what LIVE means for you (or the client using it).
Is it the remote host is actually receiving requests, meaning no Exceptions.
Or it actually means, he's able to get requests and returning me a valid StatusCode response that i expect him to return (Healthy).
So I manage to create a post requests but I have no idea on how to know if it worked?
How can I get the html code of the website after the requests?
here is my code:
var client = new WebClient();
var method = "POST"; // If your endpoint expects a GET then do it.
var parameters = new NameValueCollection();
parameters.Add("utf8", "✓");
parameters.Add("style", data);
parameters.Add("size", size);
parameters.Add("commit", "add to basket");
var response_data = client.UploadValues(url_add_to_cart, method, parameters);
Moreover, after this post requests I need to do another post requests with the same session I used before. Is it possible by doing everything in the same webclient??
Thank you very much for your answers!
If you need more info I'll add it
Simply add a try catch block in this line :
try
{
var response_data = client.UploadValues(url_add_to_cart, method, parameters);
}
catch(Exception e)
{
Console.WriteLine(e);
}
If you get an error on your HTTP request, program should execute the catch block.
Try debbuging your code and see what is inside the response_data var.
HTTP error are metioned here btw :
https://www.npmjs.com/package/http-errors
WebClient will throw WebException if the server returns a non-success (200) code.
Examine WebException.Status for more details.
https://msdn.microsoft.com/en-us/library/system.net.webexception.status.aspx
Here is my code
internal static void ValidateUrl(string url)
{
Uri validUri;
if(Uri.TryCreate(url,UriKind.Absolute,out validUri))
{
using (HttpClient client = new HttpClient())
{
try
{
HttpResponseMessage response = client.Get(url);
response.EnsureStatusIsSuccessful();
}
catch (Exception ex)
{
//exception handler goes here
}
}
}
}
This code when i run it produces this result.
ProxyAuthenticationRequired (407) is not one of the following:
OK (200), Created (201), Accepted (202), NonAuthoritativeInformation
(203), NoContent (204), ResetContent (205), PartialContent (206).
All i want to do is make this code validate whether a given website is up and running.
Any ideas?
This basically means exactly what it says: That you are trying to access the service via a proxy that you are not authenticated to use.
I guess that means your server was reached from the Web Service, but that it was not permitted to access the URL it tried to reach, since it tried to access it through a proxy it was not authenticated for.
It's what EnsureStatusIsSuccessful() does, it throws an exception if status code (returned from web server) is not one of that ones.
What you can do, to simply check without throwing an exception is to use IsSuccessStatusCode property. Like this:
HttpResponseMessage response = client.Get(url);
bool isValidAndAccessible = response.IsSuccessStatusCode;
Please note that it simply checks if StatusCode is within the success range.
In your case status code (407) means that you're accessing that web site through a proxy that requires authentication then request failed. You can do following:
Provide settings for Proxy (in case defaults one doesn't work) with WebProxy class.
Do not download page but just try to ping web server. You won't know if it's a web page or not but you'll be sure it's accessible and it's a valid URL. If applicable or not depends on context but it may be useful if HTTP requests fails.
Example from MSDN using WebProxy with WebRequest (base class for HttpWebRequest):
var request = WebRequest.Create("http://www.contoso.com");
request.Proxy = new WebProxy("http://proxyserver:80/",true);
var response = (HttpWebResponse)request.GetResponse();
int statusCode = (int)response.StatusCode;
bool isValidAndAccessible = statusCode >= 200 && statusCode <= 299;
You are invoking EnsureStatusIsSuccessful() which rightfully complains that the request was not successful because there's a proxy server between you and the host which requires authentication.
If you are on framework 4.5, I've included a slightly enhanced version below.
internal static async Task<bool> ValidateUrl(string url)
{
Uri validUri;
if(Uri.TryCreate(url,UriKind.Absolute,out validUri))
{
var client = new HttpClient();
var response = await client.GetAsync(validUri, HttpCompletionOption.ResponseHeadersRead);
return response.IsSuccessStatusCode;
}
return false;
}