Develop Reference

How can I pull data from website using C#

Web-page data into the application

You can replicate the request the website makes to get a list of relevant numbers. The following code might be a good start.
var httpRequest = (HttpWebRequest) WebRequest.Create("<url>");
httpRequest.Method = "POST";
httpRequest.Accept = "application/json";
string postData = "{<json payload>}";
using (var streamWriter = new StreamWriter(httpRequest.GetRequestStream())) {
streamWriter.Write(postData);
}
var httpResponse = (HttpWebResponse) httpRequest.GetResponse();
string result;
using (var streamReader = new StreamReader(httpResponse.GetResponseStream())) {
result = streamReader.ReadToEnd();
}
Console.WriteLine(result);
Now, for the <url> and <json payload> values:
Open the web inspector in your browser.
Go to the Network tab.
Set it so Fetch/XHR/AJAX requests are shown.
Refresh the page.
Look for a request that you want to replicate.
Copy the request URL.
Copy the Payload (JSON data, to use it in your code you'll have to add a \ before every ")
Side note: The owner of the website you are making automated requests to might not be very happy about your tool, and you/it might be blocked if it makes too many requests in a short time.

Insert photo ERROR: The remote server returned an error: (403) Forbidden

I need your help!.
Im trying to insert a new photo into a Picasa Album using Oauth 2.0 and a simple HttpRequest process. The result is that I cant insert a new photo into my Picasa web album after following the instructions listed on: https://developers.google.com/picasa-web/docs/2.0/developers_guide_protocol#Auth
I also have to say that I tried using the .Net library that they provide with the same results.
The implementation that I'm using now is the following:
public static string PostImage(
string streamImageConvertedToString)
{
string url = string.Format("https://picasaweb.google.com/data/feed/api/user/{0}/albumid/{1}", "username#gmail.com", "idAlbum");
HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
request.ContentType = "image/jpeg";
request.ContentLength = Encoding.UTF8.GetByteCount(data);
request.Method = "POST";
request.Headers.Add("GData-Version", "2");
request.Headers.Add("Slug", "cute_baby_kitten.jpg");
request.Headers.Add("Authorization", "Bearer " + GetToken());
if (data != null)
{
using (StreamWriter writer = new StreamWriter(request.GetRequestStream()))
{
writer.Write(data);
}
}
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
string result = string.Empty;
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
result = reader.ReadToEnd();
}
return result;
}
private static string GetToken() {
const string ServiceAccountEmail = "someid#developer.gserviceaccount.com";
var servicio = new PicasaService(null);
var certificate = new X509Certificate2(HttpContext.Current.Server.MapPath("/key2.p12"), "notasecret", X509KeyStorageFlags.Exportable);
var serviceAccountCredentialInitializer =
new ServiceAccountCredential.Initializer(ServiceAccountEmail)
{
Scopes = new[] { "https://picasaweb.google.com/data/" }
}.FromCertificate(certificate);
var credential = new ServiceAccountCredential(serviceAccountCredentialInitializer);
if (!credential.RequestAccessTokenAsync(System.Threading.CancellationToken.None).Result)
throw new InvalidOperationException("Access token request failed.");
return credential.Token.AccessToken;
}
Any help is welcome!!

(403) Forbidden
Means that you are trying to use a method insert which requires authorization to do.
you are connecting to service account someid#developer.gserviceaccount.com which should give you access to someid#developer.gserviceaccount.com pictures then.
you appear to be trying to access username#gmail.com unless you have given someid#developer.gserviceaccount.com access to insert pictures on behalf of username#gmail.com (Which I am not even sure is possible) you are not going to have permission to do this.
Remember a service account is a sudo user it has its own drive account, calendar account ... it does not have access to a random users data unless that user has given them access like they would any other user.
Note: Google .net client library does not support gdata APIs. Picasa is a gdata library I like how are trying to merge the two I am have to test this.

You're best (imho) approach would be to forget libraries and forget service accounts. Get a refresh token for the google user account you're trying to insert to, and use the raw HTTP REST API to invoke Picasa.

Request parameter in Windows Mobile for REST API's

I have one Windows Handheld device application on .Net framework 3.5 which has the requirement of accessing a REST API. The REST API gives me JSON output which I am going to process later. I have the following code for that:-
HttpWebRequest webRequest;
string result = String.Empty;
try
{
webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.Method = "POST";
webRequest.KeepAlive = false;
webRequest.ContentType = "application/x-www-form-urlencoded";
using (WebResponse response = webRequest.GetResponse())
{
using (StreamReader streamReader = new StreamReader(response.GetResponseStream()))
{
result = streamReader.ReadToEnd();
}
}
}
catch (Exception ex)
{
result = ex.Message;
}
The url variable is holding the url for the API with some query parameters in it.
For Example "http://www.something.com/Login?id=test&pwd=test".
Now my problem is I dont want to use the query string parameters rather I want to use Request parameters because the above approach does not work every time perfectly. Some times I get an "Unauthorized" error. And also I have one tokenId which I need to send everytime I am calling the API and the token Id is in base64 format.
Can anyone please help me how can I use the Request Parameter feature to send the parameter values?

use the Headers property of your request object.
webRequest.Headers.Add("id", "test");
webRequest.Headers.Add("pwd", "test");
webRequest.Headers.Add("token", myToken);

Can't access Web of Trust (WoT) API w/ JSON.Net

I'm new to JSON & am using VS 2013/C#. Here's the code for the request & response. Pretty straightforward, no?
Request request = new Request();
//request.hosts = ListOfURLs();
request.hosts = "www.cnn.com/www.cisco.com/www.microsoft.com/";
request.callback = "process";
request.key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
string output = JsonConvert.SerializeObject(request);
//string test = "hosts=www.cnn.com/www.cisco.com/www.microsoft.com/&callback=process&key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
try
{
var httpWebRequest = (HttpWebRequest) WebRequest.Create("http://api.mywot.com/0.4/public_link_json2?);
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "POST";
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
string json = output;
streamWriter.Write(json);
}
var httpResponse = (HttpWebResponse) httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var responseText = streamReader.ReadToEnd();
}
}
catch (WebException e)
{
MessageBox.Show(e.ToString());
}
//response = true.
//no response = false
return true;
}
When I run this, I get a 405 error indicating method not allowed.
It seems to me that there are at least two possible problems here: (1) The WoT API (www.mywot.com/wiki/API) requires a GET request w/ a body, & httpWebRequest doesn't allow a GET in the httpWebRequest.Method; or (2) the serialized string isn't serialized properly.
NOTE: In the following I've had to remove the leading "http://" since I don't have enough rep to post more than 2 links.
It should look like:
api.mywot.com/0.4/public_link_json2?hosts=www.cnn.com/www.cisco.com/www.microsoft.com/&callback=process&key=xxxxxxxxxxxxxx
but instead looks like:
api.mywot.com/0.4/public_link_json2?{"hosts":"www.cnn.com/www.cisco.com/www.microsoft.com/","callback":"process","key":"xxxxxxxxxxxxxxxxxxx"}.
If I browse to:api.mywot.com/0.4/public_link_json2?hosts=www.cnn.com/www.cisco.com/www.microsoft.com/&callback=process&key=xxxxxxxxxxxxxx; I get the expected response.
If I browse to: api.mywot.com/0.4/public_link_json2?{"hosts":"www.cnn.com/www.cisco.com/www.microsoft.com/","callback":"process","key":"xxxxxxxxxxxxxxxxxxx"}; I get a 403 denied error.
If I hardcode the request & send as a GET like below:
var httpWebRequest = (HttpWebRequest) WebRequest.Create("api.mywot.com/0.4/public_link_json2? + "test"); it also works as expected.
I'd appreciate any help w/ this & hope I've made the problem clear. Thx.

Looks to me like the problem is that you are sending JSON in the URL. According to the API doc that you referenced, the API is expecting regular URL encoded parameters (not JSON), and it will return JSON to you in the body of the response:
Requests
The API consists of a number of interfaces, all of which are called using normal HTTP GET requests to api.mywot.com and return a response in XML or JSON format if successful. HTTP status codes are used for returning error information and parameters are passed using standard URL conventions. The request format is as follows:
http://api.mywot.com/version/interface?param1=value1&param2=value2
You should not be serializing your request; you should be deserializing the response. All of your tests above bear this out.

Problems consuming WebService in .Net (ReCaptcha)

I am having difficulty in consuming the reCaptcha Web Service using C#/.Net 3.5. Although I think the problem is with consuming web services in general.
String validate = String.Format("http://api-verify.recaptcha.net/verify?privatekey={0}&remoteip={1}&challenge={2}&response={3}", PrivateKey, UserIP, Challenge, Response);
WebClient serviceRequest = new WebClient();
serviceRequest.Headers.Add("ContentType","application/x-www-form-urlencoded")
String response = serviceRequest.DownloadString(new Uri(validate ));
It keeps telling me that the error is: nverify-params-incorrect. Which means:
The parameters to /verify were incorrect, make sure you are passing all the required parameters.
But it's correct. I am using the private key, the IP address (locally) is 127.0.0.1, and the challenge and response seem fine. However the error keeps occurring.
I am pretty sure this is a issue with how I am requesting the service as this is the first time I have actually used webservices and .Net.
I also tried this as it ensures the data is posted:
String queryString = String.Format("privatekey={0}&remoteip={1}&challenge={2}&response={3}",PrivateKey, UserIP, Challenge, Response);
String Validate = "http://api-verify.recaptcha.net/verify" + queryString;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new Uri(Validate));
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = Validate.Length;
**HttpWebResponse captchaResponse = (HttpWebResponse)request.GetResponse();**
String response;
using (StreamReader reader = new StreamReader(captchaResponse.GetResponseStream()))
response = reader.ReadToEnd();
Seems to stall at the point where I get response.
Any advice?
Thanks in advance

Haven't worked with the recaptcha service previously, but I have two troubleshooting recommendations:
Use Fiddler or Firebug and watch what you're sending outbound. Verifying your parameters would help you with basic troubleshooting, i.e. invalid characters, etc.
The Recaptcha Wiki has an entry about dealing with development on Vista. It doesn't have to be limited to Vista, though; if you're system can handle IPv6, then your browser could be communicating in that format as a default. It appears as if Recaptcha deals with IPv4. Having Fiddler/Firebug working would tell you about those other parameters that could be causing you grief.
This may not help solve your problem but it might provide you with better troubleshooting info.

So got this working, for some reason I needed to write the request to a stream like so:
//Write data to request stream
using (Stream requestSteam = request.GetRequestStream())
requestSteam.Write(byteData, 0, byteData.Length);
Could anyone explain why this works. I didn't think I would need to do this, don't completely understand what's happening behind the scenes..

Damien's answer is correct of course, but just to be clear about the order of things (I was a little confused) and to have a complete code sample...
var uri = new Uri("http://api-verify.recaptcha.net/verify");
var queryString = string.Format(
"privatekey={0}&remoteip={1}&challenge={2}&response={3}",
privateKey,
userIP,
challenge,
response);
var request = (HttpWebRequest)HttpWebRequest.Create(uri);
request.Method = WebRequestMethods.Http.Post;
request.ContentLength = queryString.Length;
request.ContentType = "application/x-www-form-urlencoded";
using (var writer = new StreamWriter(request.GetRequestStream()))
{
writer.Write(queryString);
}
string result;
using (var webResponse = (HttpWebResponse)request.GetResponse())
{
var reader = new StreamReader(webResponse.GetResponseStream());
result = reader.ReadToEnd();
}
There's a slight difference in that I'm writing the post variables to the request, but the core of it is the same.