Develop Reference

How can I pull data from website using C#

Web-page data into the application

You can replicate the request the website makes to get a list of relevant numbers. The following code might be a good start.
var httpRequest = (HttpWebRequest) WebRequest.Create("<url>");
httpRequest.Method = "POST";
httpRequest.Accept = "application/json";
string postData = "{<json payload>}";
using (var streamWriter = new StreamWriter(httpRequest.GetRequestStream())) {
streamWriter.Write(postData);
}
var httpResponse = (HttpWebResponse) httpRequest.GetResponse();
string result;
using (var streamReader = new StreamReader(httpResponse.GetResponseStream())) {
result = streamReader.ReadToEnd();
}
Console.WriteLine(result);
Now, for the <url> and <json payload> values:
Open the web inspector in your browser.
Go to the Network tab.
Set it so Fetch/XHR/AJAX requests are shown.
Refresh the page.
Look for a request that you want to replicate.
Copy the request URL.
Copy the Payload (JSON data, to use it in your code you'll have to add a \ before every ")
Side note: The owner of the website you are making automated requests to might not be very happy about your tool, and you/it might be blocked if it makes too many requests in a short time.

C# Asana POST new task returns error 400 (Bad request)

I have a problem with creating new task in Asana from my app.
Post method:
protected static T Post<T>(string route, object action = null, object parameters = null) where T : BaseResult, new()
{
var result = new T();
try
{
var actionUrl = GetActionUrl(route, action);
var request = (HttpWebRequest)WebRequest.Create(actionUrl);
request.Credentials = CredentialCache.DefaultCredentials;
request.Accept = "application/json";
request.Method = WebRequestMethods.Http.Post;
request.Proxy.Credentials = CredentialCache.DefaultCredentials;
request.Headers.Add("Authorization: Bearer " + ApiKey);
if (parameters != null)
{
var contentJSON = JsonConvert.SerializeObject(parameters);
request.ContentType = "application/json";
using (var s = request.GetRequestStream())
using (var sw = new StreamWriter(s, Encoding.UTF8))
sw.Write(contentJSON);
}
var response = (HttpWebResponse) request.GetResponse();
using (var reader = new StreamReader(response.GetResponseStream()))
{
var data = reader.ReadToEnd();
result = JsonConvert.DeserializeObject<T>(data);
}
}
catch (Exception ex)
{
result.IsOk = false;
result.Message = ex.GetMessage();
}
return result;
}
Action URL: https://app.asana.com/api/1.0/workspaces/MyWorkspace/tasks
JSON:{"data":{"name":"TestTask1","notes":"Test note","workspace":"*MyWorkspace*","assignee":"*MyAssignee"}}
But Asana returns me "The remote server returned an error: (400) Bad Request."
If I change request.ContentType to "application/x-www-form-urlencoded", I get no errors, but Asana returns me new task with empty fields.
What my next steps to fix issue should be?
Thank you

If you're using an ApiKey (and not a Personal Access Token), I believe that your Authorization Header should be
"Authorization: Basic " + EncodedAuthInfo
where
EncodedAuthInfo = Convert.ToBase64String(Encoding.Default.GetBytes(ApiKey + ":"))
See How do I connect to the Asana Rest API using c#? or the Using Basic Authentication section in https://asana.com/developers/documentation/getting-started/auth for details on using basic authentication.
I'm also a little confused by what you mean when you say that
JSON = {"data": {"name": "TestTask1"} ...
Is this the HTTP response that you expect?
Anyways, hopefully what I've outlined helps.

Hmm. I think I've got what's blocking you sorted out.
Imagine the scenario where you post to
https://app.asana.com/api/1.0/workspaces/123456/tasks
and you pass in the request body the parameter
"workspace":"789012"
What should the Asana platform do with this data? You've inadvertently specified the workspace twice with conflicting numbers. For this reason, you cannot specify the workspace id in the data when hitting an endpoint which also contains the workspace id in the URL.
The documentation is confusing on this point, because we don't clarify which parameters are found in the URL and which parameters are found in the JSON in the request body. I'm actually fixing this very soon! If this is indeed what's causing the issue, I'm sorry that we were not clear on this.
Personally, I think it might be a better user experience to allow the workspace to be duplicated in the parameter data if and only if it's identical to the one in the URL, but right now, we simply check to see that there is only one value for the workspace id. If there are more than one, even if they are the same one, we return the 400 error code.
You might consider parsing the response body, even on errors. In it, we try to provide fairly decent information about what was wrong about the request. For example, when testing my hunch about your request, what I got back was:
"errors":[{"message":"Duplicate field: workspace", ...}]
If we've done a good job about sending back informative messages, I hope you'll find this even more useful than an Asana sandbox! If this is not the issue, feel free to comment; I'll be happy to dive into this further.

Get web page contents from Firefox in a C# program

I need to write a simple C# app that should receive entire contents of a web page currently opened in Firefox. Is there any way to do it directly from C#? If not, is it possible to develop some kind of plug-in that would transfer page contents? As I am a total newbie in Firefox plug-ins programming, I'd really appreciate any info on getting me started quickly. Maybe there are some sources I can use as a reference? Doc links? Recommendations?
UPD: I actually need to communicate with a Firefox instance, not get contents of a web page from a given URL

It would help if you elaborate What you are trying to achieve. May be plugins already out there such as firebug can help.
Anways, if you really want to develop both plugin and C# application:
Check out this tutorial on firefox extension:
http://robertnyman.com/2009/01/24/how-to-develop-a-firefox-extension/
Otherwise, You can use WebRequest or HttpWebRequest class in .NET request to get the HTML source of any URL.

I think you'd almost certainly need to write a Firefox plugin for that. However there are certainly ways to request a webpage, and receive its HTML response within C#. It depends on what your requirements are?
If you're requirements are simply receive the source from any website, leave a comment and I'll point you towards the code.

Uri uri = new Uri(url);
System.Net.HttpWebRequest req = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(uri.AbsoluteUri);
req.AllowAutoRedirect = true;
req.MaximumAutomaticRedirections = 3;
//req.UserAgent = _UserAgent; //"Mozilla/6.0 (MSIE 6.0; Windows NT 5.1; Searcharoo.NET)";
req.KeepAlive = true;
req.Timeout = _RequestTimeout * 1000; //prefRequestTimeout
// SIMONJONES http://codeproject.com/aspnet/spideroo.asp?msg=1421158#xx1421158xx
req.CookieContainer = new System.Net.CookieContainer();
req.CookieContainer.Add(_CookieContainer.GetCookies(uri));
System.Net.HttpWebResponse webresponse = null;
try
{
webresponse = (System.Net.HttpWebResponse)req.GetResponse();
}
catch (Exception ex)
{
webresponse = null;
Console.Write("request for url failed: {0} {1}", url, ex.Message);
}
if (webresponse != null)
{
webresponse.Cookies = req.CookieContainer.GetCookies(req.RequestUri);
// handle cookies (need to do this incase we have any session cookies)
foreach (System.Net.Cookie retCookie in webresponse.Cookies)
{
bool cookieFound = false;
foreach (System.Net.Cookie oldCookie in _CookieContainer.GetCookies(uri))
{
if (retCookie.Name.Equals(oldCookie.Name))
{
oldCookie.Value = retCookie.Value;
cookieFound = true;
}
}
if (!cookieFound)
{
_CookieContainer.Add(retCookie);
}
}
string enc = "utf-8"; // default
if (webresponse.ContentEncoding != String.Empty)
{
// Use the HttpHeader Content-Type in preference to the one set in META
doc.Encoding = webresponse.ContentEncoding;
}
else if (doc.Encoding == String.Empty)
{
doc.Encoding = enc; // default
}
//http://www.c-sharpcorner.com/Code/2003/Dec/ReadingWebPageSources.asp
System.IO.StreamReader stream = new System.IO.StreamReader
(webresponse.GetResponseStream(), System.Text.Encoding.GetEncoding(doc.Encoding));
webresponse.Close();

This does what you want.
using System.Net;
var cli = new WebClient();
string data = cli.DownloadString("http://www.heise.de");
Console.WriteLine(data);

Native messaging enables an extension to exchange messages with a native application installed on the user's computer.

Sending and receiving SOAP messages

I am writing a web service client in C# and do not want to create and serialize/deserialize objects, but rather send and receive raw XML.
Is this possible in C#?

Here is part of an implementation I just got running based on John M Gant's example. It is important to set the content type request header. Plus my request needed credentials.
protected virtual WebRequest CreateRequest(ISoapMessage soapMessage)
{
var wr = WebRequest.Create(soapMessage.Uri);
wr.ContentType = "text/xml;charset=utf-8";
wr.ContentLength = soapMessage.ContentXml.Length;
wr.Headers.Add("SOAPAction", soapMessage.SoapAction);
wr.Credentials = soapMessage.Credentials;
wr.Method = "POST";
wr.GetRequestStream().Write(Encoding.UTF8.GetBytes(soapMessage.ContentXml), 0, soapMessage.ContentXml.Length);
return wr;
}
public interface ISoapMessage
{
string Uri { get; }
string ContentXml { get; }
string SoapAction { get; }
ICredentials Credentials { get; }
}

You can use the System.Net classes, such as HttpWebRequest and HttpWebResponse to read and write directly to an HTTP connection.
Here's a basic (off-the-cuff, not compiled, non-error-checking, grossly oversimplified) example. May not be 100% correct, but at least will give you an idea of how it works:
HttpWebRequest req = (HttpWebRequest) HttpWebRequest.Create(url);
req.ContentLength = content.Length;
req.Method = "POST";
req.GetRequestStream().Write(Encoding.ASCII.GetBytes(content), 0, content.Length);
HttpWebResponse resp = (HttpWebResponse) req.getResponse();
//Read resp.GetResponseStream() and do something with it...
This approach works well. But chances are whatever you need to do can be accomplished by inheriting the existing proxy classes and overriding the members you need to have behave differently. This type of thing is best reserved for when you have no other choice, which is not very often in my experience.

Yes - you can simply declare the inputs and outputs as XmlNode's
[WebMethod]
public XmlNode MyMethod(XmlNode input);

You can have your web service method return a string containing the xml, but do heed the comment above about making things more error-prone.

Problems consuming WebService in .Net (ReCaptcha)

I am having difficulty in consuming the reCaptcha Web Service using C#/.Net 3.5. Although I think the problem is with consuming web services in general.
String validate = String.Format("http://api-verify.recaptcha.net/verify?privatekey={0}&remoteip={1}&challenge={2}&response={3}", PrivateKey, UserIP, Challenge, Response);
WebClient serviceRequest = new WebClient();
serviceRequest.Headers.Add("ContentType","application/x-www-form-urlencoded")
String response = serviceRequest.DownloadString(new Uri(validate ));
It keeps telling me that the error is: nverify-params-incorrect. Which means:
The parameters to /verify were incorrect, make sure you are passing all the required parameters.
But it's correct. I am using the private key, the IP address (locally) is 127.0.0.1, and the challenge and response seem fine. However the error keeps occurring.
I am pretty sure this is a issue with how I am requesting the service as this is the first time I have actually used webservices and .Net.
I also tried this as it ensures the data is posted:
String queryString = String.Format("privatekey={0}&remoteip={1}&challenge={2}&response={3}",PrivateKey, UserIP, Challenge, Response);
String Validate = "http://api-verify.recaptcha.net/verify" + queryString;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new Uri(Validate));
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = Validate.Length;
**HttpWebResponse captchaResponse = (HttpWebResponse)request.GetResponse();**
String response;
using (StreamReader reader = new StreamReader(captchaResponse.GetResponseStream()))
response = reader.ReadToEnd();
Seems to stall at the point where I get response.
Any advice?
Thanks in advance

Haven't worked with the recaptcha service previously, but I have two troubleshooting recommendations:
Use Fiddler or Firebug and watch what you're sending outbound. Verifying your parameters would help you with basic troubleshooting, i.e. invalid characters, etc.
The Recaptcha Wiki has an entry about dealing with development on Vista. It doesn't have to be limited to Vista, though; if you're system can handle IPv6, then your browser could be communicating in that format as a default. It appears as if Recaptcha deals with IPv4. Having Fiddler/Firebug working would tell you about those other parameters that could be causing you grief.
This may not help solve your problem but it might provide you with better troubleshooting info.

So got this working, for some reason I needed to write the request to a stream like so:
//Write data to request stream
using (Stream requestSteam = request.GetRequestStream())
requestSteam.Write(byteData, 0, byteData.Length);
Could anyone explain why this works. I didn't think I would need to do this, don't completely understand what's happening behind the scenes..

Damien's answer is correct of course, but just to be clear about the order of things (I was a little confused) and to have a complete code sample...
var uri = new Uri("http://api-verify.recaptcha.net/verify");
var queryString = string.Format(
"privatekey={0}&remoteip={1}&challenge={2}&response={3}",
privateKey,
userIP,
challenge,
response);
var request = (HttpWebRequest)HttpWebRequest.Create(uri);
request.Method = WebRequestMethods.Http.Post;
request.ContentLength = queryString.Length;
request.ContentType = "application/x-www-form-urlencoded";
using (var writer = new StreamWriter(request.GetRequestStream()))
{
writer.Write(queryString);
}
string result;
using (var webResponse = (HttpWebResponse)request.GetResponse())
{
var reader = new StreamReader(webResponse.GetResponseStream());
result = reader.ReadToEnd();
}
There's a slight difference in that I'm writing the post variables to the request, but the core of it is the same.