I m trying to invoke a URL in C#, I am just interested in invoking, and dont care about response. When i have the following, does it mean that I m invoking the URL?
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
You need to actually perform the request:
var request = (HttpWebRequest)WebRequest.Create(url);
request.GetResponse();
The call to GetResponse makes the outbound call to the server. You can discard the response if you don't care about it.
First) Create WebRequest to execute URL.
Second) Use WebResponse to get response.
Finally) Use StreamReader to decode response and convert it to normal string.
string url = "Your request url";
WebRequest request = HttpWebRequest.Create(url);
WebResponse response = request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string responseText = reader.ReadToEnd();
You can use this:
string address = "http://www.yoursite.com/page.aspx";
using (WebClient client = new WebClient())
{
client.DownloadString(address);
}
No when you say request.GetResponse(); then you invoke it.
Probably not. See: http://www.codeproject.com/KB/webservices/HttpWebRequest_Response.aspx
You're allowed to set the Method, ContentType, etc., all which would have to be done before the request is actually sent. It looks like GetResponse() actually sends the request. You can simply ignore the return value.
Related
Trying to call WebServices from C# and getting below error:
System.Net.WebException: 'The remote server returned an error: (502) Bad Gateway
Code:
WebRequest request = WebRequest.Create("https://xxxxx/cgi/webservice.pl?function=get_latest_ts_values&site_list=130105B&datasource=AT&varfrom=10.00&varto=10.00&lookback=60&format=csv");
request.Method = "GET";
WebResponse response = request.GetResponse();
using (Stream dataStream = response.GetResponseStream() )
{
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = reader.ReadToEnd();
Console.WriteLine(responseFromServer);
Console.ReadLine();
}
But works fine when i use Postman or just copy url in browser and also works fine with below python code:
import requests
dataload = {}
dataurl = "https://xxxxx/cgi/webservice.pl?function=get_latest_ts_values&site_list=130105B&datasource=AT&varfrom=10.00&varto=10.00&lookback=60"
headers = {}
response = requests.request("GET", dataurl, headers=headers, data=dataload)
for dataresp in response:
print(dataresp)
What am I doing wrong with C# code?
The uri for the WebRequest has the query parameter &format=csv. Maybe this is why you are getting a 502. The Python request is missing that query parameter. Did you try the WebRequest by removing that part?
Could be incorrect content type or user agent having the wrong information. Postman could be setting these values without your knowledge. Might try in the exception seeing if there is a a response stream and read it through a streamreader to see if there is any more information you're not seeing to point you in the correct direction.
Ended up using RestSharp and it works fine. (https://www.nuget.org/packages/RestSharp)
string Uri = "https://xxxx/cgi/webservice.pl?xxxx";
var client = new RestSharp.RestClient(Uri);
client.Timeout = -1;
var request = new RestRequest(Method.GET);
IRestResponse response = client.Execute(request);
Console.WriteLine(response.Content);
I'm trying to get the order book from GDAX (link to documentation of the call) but when doing it from the c# executable I always get Error 400 - Bad request.
When taking the actual URL and pasting it into my browser, it works fine.
String URL = "https://api.gdax.com/products/BTC-USD/book?level=2";
WebRequest request = WebRequest.Create(URL);
WebResponse response = request.GetResponse();
The actual issue with your API call is , the API is expecting a user-agent string while making the request: Below is the code in working condition:
try
{
String URL = "http://api.gdax.com/products/BTC-USD/book?level=2";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(URL);
request.UserAgent = ".NET Framework Test Client";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
var encoding = ASCIIEncoding.ASCII;
using (var reader = new System.IO.StreamReader(response.GetResponseStream(), encoding))
{
string responseText = reader.ReadToEnd();
}
}
catch(WebException ex)
{
HttpWebResponse xyz = ex.Response as HttpWebResponse;
var encoding = ASCIIEncoding.ASCII;
using (var reader = new System.IO.StreamReader(xyz.GetResponseStream(), encoding))
{
string responseText = reader.ReadToEnd();
}
}
Basically ProtocolError indicates that you have received the response but there is an error related to protocol, which you can observe, when you read the response content from exception. I have added catch to handle the exception and read ex.Response (which is HttpWebResponse) and could see that the API is asking for user-agent to be suppllied while making the call. I got to see the error as "{"message":"User-Agent header is required."}"
You can ignore the code inside the exception block, I used it only to see what is the actual response message, which contains actual error details
Note: I have boxed WebRequest to HttpWebRequest to have additional http protocol related properties and most importantly "UserAgent" property which is not available with the WebRequest class.
You need to Accept the certificarte, Google for access to a https webrequest.
Like this
Using the WebClient class I can get the title of a website easily enough:
WebClient x = new WebClient();
string source = x.DownloadString(s);
string title = Regex.Match(source,
#"\<title\b[^>]*\>\s*(?<Title>[\s\S]*?)\</title\>",
RegexOptions.IgnoreCase).Groups["Title"].Value;
I want to store the URL and the page title. However when following a link such as:
http://tinyurl.com/dbysxp
I'm clearly going to want to get the Url I'm redirected to.
QUESTIONS
Is there a way to do this using the WebClient class?
How would I do it using HttpResponse and HttpRequest?
If I understand the question, it's much easier than people are saying - if you want to let WebClient do all the nuts and bolts of the request (including the redirection), but then get the actual response URI at the end, you can subclass WebClient like this:
class MyWebClient : WebClient
{
Uri _responseUri;
public Uri ResponseUri
{
get { return _responseUri; }
}
protected override WebResponse GetWebResponse(WebRequest request)
{
WebResponse response = base.GetWebResponse(request);
_responseUri = response.ResponseUri;
return response;
}
}
Just use MyWebClient everywhere you would have used WebClient. After you've made whatever WebClient call you needed to do, then you can just use ResponseUri to get the actual redirected URI. You'd need to add a similar override for GetWebResponse(WebRequest request, IAsyncResult result) too, if you were using the async stuff.
I know this is already an answered question, but this works pretty to me:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://tinyurl.com/dbysxp");
request.AllowAutoRedirect = false;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string redirUrl = response.Headers["Location"];
response.Close();
//Show the redirected url
MessageBox.Show("You're being redirected to: "+redirUrl);
Cheers.! ;)
With an HttpWebRequest, you would set the AllowAutoRedirect property to false. When this happens, any response with a status code between 300-399 will not be automatically redirected.
You can then get the new url from the response headers and then create a new HttpWebRequest instance to the new url.
With the WebClient class, I doubt you can change it out-of-the-box so that it does not allow redirects. What you could do is derive a class from the WebClient class and then override the GetWebRequest and the GetWebResponse methods to alter the WebRequest/WebResponse instances that the base implementation returns; if it is an HttpWebRequest, then set the AllowAutoRedirect property to false. On the response, if the status code is in the range of 300-399, then issue a new request.
However, I don't know that you can issue a new request from within the GetWebRequest/GetWebResponse methods, so it might be better to just have a loop that executes with HttpWebRequest/HttpWebResponse until all the redirects are followed.
I got the Uri for the redirected page and the page contents.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(strUrl);
request.AllowAutoRedirect = true;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream dataStream = response.GetResponseStream();
strLastRedirect = response.ResponseUri.ToString();
StreamReader reader = new StreamReader(dataStream);
string strResponse = reader.ReadToEnd();
response.Close();
In case you are only interested in the redirect URI you can use this code:
public static string GetRedirectUrl(string url)
{
HttpWebRequest request = (HttpWebRequest) HttpWebRequest.Create(url);
request.AllowAutoRedirect = false;
using (HttpWebResponse response = HttpWebResponse)request.GetResponse())
{
return response.Headers["Location"];
}
}
The method will return
null - in case of no redirect
a relative url - in case of a redirect
Please note: The using statement (or a final response.close()) is essential. See MSDN Library for details. Otherwise you may run out of connections or get a timeout when executing this code multiple times.
HttpWebRequest.AllowAutoRedirect can be set to false. Then you'd have to manually http status codes in the 300 range.
// Create a new HttpWebRequest Object to the mentioned URL.
HttpWebRequest myHttpWebRequest=(HttpWebRequest)WebRequest.Create("http://www.contoso.com");
myHttpWebRequest.MaximumAutomaticRedirections=1;
myHttpWebRequest.AllowAutoRedirect=true;
HttpWebResponse myHttpWebResponse=(HttpWebResponse)myHttpWebRequest.GetResponse();
The WebClient class has an option to follow redirects. Set that option and you should be fine.
Ok this is really hackish, but the key is to use the HttpWebRequest and then set the AllowAutoRedirect property to true.
Here's a VERY hacked together example
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://tinyurl.com/dbysxp");
req.Method = "GET";
req.AllowAutoRedirect = true;
WebResponse response = req.GetResponse();
response.GetResponseStream();
Stream responseStream = response.GetResponseStream();
// Content-Length header is not trustable, but makes a good hint.
// Responses longer than int size will throw an exception here!
int length = (int)response.ContentLength;
const int bufSizeMax = 65536; // max read buffer size conserves memory
const int bufSizeMin = 8192; // min size prevents numerous small reads
// Use Content-Length if between bufSizeMax and bufSizeMin
int bufSize = bufSizeMin;
if (length > bufSize)
bufSize = length > bufSizeMax ? bufSizeMax : length;
StringBuilder sb;
// Allocate buffer and StringBuilder for reading response
byte[] buf = new byte[bufSize];
sb = new StringBuilder(bufSize);
// Read response stream until end
while ((length = responseStream.Read(buf, 0, buf.Length)) != 0)
sb.Append(Encoding.UTF8.GetString(buf, 0, length));
string source = sb.ToString();string title = Regex.Match(source,
#"\<title\b[^>]*\>\s*(?<Title>[\s\S]*?)\</title\>",RegexOptions.IgnoreCase).Groups["Title"].Value;
enter code here
I have to use HttpWebRequest in C#, to retrieve an access token.
My base url redirects me to another url that contains the access token.
Is there a way to use this to get the access token?
This is my code after applying suggestions from the answer:
// Create a request for the URL.
string url = "https://stackexchange.com/oauth/dialog?client_id=2532&scope=no_expiry&redirect_uri=https://stackexchange.com/oauth/login_success";
WebRequest request = WebRequest.Create(
url);
request.Method = "GET";
// If required by the server, set the credentials.
request.Credentials = CredentialCache.DefaultCredentials;
// Get the response.
WebResponse response = request.GetResponse();
// Display the status.
Console.WriteLine(((HttpWebResponse)response).StatusDescription);
// Get the stream containing content returned by the server.
Stream dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd();
// Display the content.
Console.WriteLine(responseFromServer);
// Clean up the streams and the response.
reader.Close();
response.Close();
Debug breaks at line: WebResponse response = request.GetResponse();
VS tells me that the server has the a generic error: 500.
If I try with https://www.google.com, no errors.
You can get the resulting url from the ResponseUri property.
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
NaveValueCollection urlParameters = HttpUtility.ParseQueryString(response.ResponseUri.Query);
// extract the access token from the url.
string accessToken = urlParameters["access_token"];
}
How might I change the verb of a WebClient request? It seems to only allow/default to POST, even in the case of DownloadString.
try
{
WebClient client = new WebClient();
client.QueryString.Add("apiKey", TRANSCODE_KEY);
client.QueryString.Add("taskId", taskId);
string response = client.DownloadString(TRANSCODE_URI + "task");
result = JsonConvert.DeserializeObject<Dictionary<string, dynamic>>(response);
}
catch (Exception ex )
{
result = null;
error = ex.Message + " " + ex.InnerException;
}
And Fiddler says:
POST http://someservice?apikey=20130701-234126753-X7384&taskId=20130701-234126753-258877330210884 HTTP/1.1
Content-Length: 0
If you use HttpWebRequest instead you would get more control of the call. You can change the REST verb by the Method property (default is GET)
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(HostURI);
request.Method = "GET";
String test = String.Empty;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
Stream dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
test = reader.ReadToEnd();
reader.Close();
dataStream.Close();
}
DeserializeObject(test ...)
Not sure if you can use WebClient for that. But why not use HttpClient.GetAsync Method (String) http://msdn.microsoft.com/en-us/library/hh158944.aspx
As one can see in the source code of .NET, the HTTP Method of the DownloadString depends on the state of the private WebClient instance field m_Method, which is cleared to null upon each new request method call (link) and defaults to the Web request Creator (depends on the URI, for example ftp protocol gets another creator), but this is not thread safe.
Perhaps you are sharing this WebClient instance among several calls simultaneously?
So it gets confused. Either this or the URI confuses the WebRequest creator.