Although i can grasp the concepts of the .Net framework and windows apps, i want to create an app that will involve me simulating website clicks and getting data/response times from that page. I have not had any experience with web yet as im only a junior, could someone explain to me (in english!!) the basic concepts or with examples, the different ways and classes that could help me communicate with a website?
what do you want to do?
send a request and grab the response in a String so you can process?
HttpWebRequest and HttpWebResponse will work
if you need to connect through TCP/IP, FTP or other than HTTP then you need to use a more generic method
WebRequest and WebResponse
All the 4 methods above are in System.Net Namespace
If you want to build a Service in the web side that you can consume, then today and in .NET please choose and work with WCF (RESTfull style).
hope it helps you finding your way :)
as an example using the HttpWebRequest and HttpWebResponse, maybe some code will help you understand better.
case: send a response to a URL and get the response, it's like clicking in the URL and grab all the HTML code that will be there after the click:
private void btnSendRequest_Click(object sender, EventArgs e)
{
textBox1.Text = "";
try
{
String queryString = "user=myUser&pwd=myPassword&tel=+123456798&msg=My message";
byte[] requestByte = Encoding.Default.GetBytes(queryString);
// build our request
WebRequest webRequest = WebRequest.Create("http://www.sendFreeSMS.com/");
webRequest.Method = "POST";
webRequest.ContentType = "application/xml";
webRequest.ContentLength = requestByte.Length;
// create our stram to send
Stream webDataStream = webRequest.GetRequestStream();
webDataStream.Write(requestByte, 0, requestByte.Length);
// get the response from our stream
WebResponse webResponse = webRequest.GetResponse();
webDataStream = webResponse.GetResponseStream();
// convert the result into a String
StreamReader webResponseSReader = new StreamReader(webDataStream);
String responseFromServer = webResponseSReader.ReadToEnd().Replace("\n", "").Replace("\t", "");
// close everything
webResponseSReader.Close();
webResponse.Close();
webDataStream.Close();
// You now have the HTML in the responseFromServer variable, use it :)
textBox1.Text = responseFromServer;
}
catch (Exception ex)
{
textBox1.Text = ex.Message;
}
}
The code does not work cause the URL is fictitious, but you get the idea. :)
You could use the System.Net.WebClient class of the .NET Framework. See the MSDN documentation here.
Simple example:
using System;
using System.Net;
using System.IO;
public class Test
{
public static void Main (string[] args)
{
if (args == null || args.Length == 0)
{
throw new ApplicationException ("Specify the URI of the resource to retrieve.");
}
WebClient client = new WebClient ();
// Add a user agent header in case the
// requested URI contains a query.
client.Headers.Add ("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)");
Stream data = client.OpenRead (args[0]);
StreamReader reader = new StreamReader (data);
string s = reader.ReadToEnd ();
Console.WriteLine (s);
data.Close ();
reader.Close ();
}
}
There are other useful methods of the WebClient, which allow developers to download and save resources from a specified URI.
The DownloadFile() method for example will download and save a resource to a local file. The UploadFile() method uploads and saves a resource to a specified URI.
UPDATE:
WebClient is simpler to use than WebRequest. Normally you could stick to using just WebClient unless you need to manipulate requests/responses in an advanced way. See this article where both are used: http://odetocode.com/Articles/162.aspx
Related
I am working on a c# console application where I am making a Http Post request to a web api by using xml file and I'm kind of new to XML and web services but I figured out the following code for the request but failed to pass xml data to the method
static void Main(string[] args)
{
string desturl=#"https://xyz.abcgroup.com/abcapi/";
Program p = new Program();
System.Console.WriteLine(p.WebRequestPostData(desturl, #"C:\Applications\TestService\FixmlSub.xml"));
}
public string WebRequestPostData(string url, string postData)
{
System.Net.WebRequest req = System.Net.WebRequest.Create(url);
req.ContentType = "text/xml";
req.Method = "POST";
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(postData);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
using (System.Net.WebResponse resp = req.GetResponse())
{
if (resp == null) return null;
using (System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream()))
{
return sr.ReadToEnd().Trim();
}
}
}
For obvious reasons the above code throws 404 error as I think I am not passing the xml data properly
May I know how I can fix this?
You're not posting xml, your posting the string C:\Applications\TestService\FixmlSub.xml
Change your method call from:
System.Console.WriteLine(p.WebRequestPostData(desturl, #"C:\Applications\TestService\FixmlSub.xml"));
to
var xml = XElement.Load(#"C:\Applications\TestService\FixmlSub.xml");
System.Console.WriteLine(p.WebRequestPostData(desturl, xml.ToString(SaveOptions.DisableFormatting));
If you are trying to learn post / receive, go for it. But there are open source libraries that are already well tested to use if you want to use them.
The non-free version of Servicestack.
And their older free-version. I think the older free version is great. I've never tried the newer one. You deal with objects, like say an Employee and pass that to the library and it does the translation to xml or whatever the web-service wants.
You can post whole strings if you want. They have great extension methods to help you with that too.
I am preparing a windows application in that I want to use oAuth2 to access info from google plus.
But I am getting bad request for my following code.. I am able to create app in google console and fetch the "code" for application access.
WebRequest request = WebRequest.Create(
GoogleAuthenticationServer.Description.TokenEndpoint
);
// You must use POST for the code exchange.
request.Method = "POST";
// Create POST data.
string postData = FormPostData(code);
byte[] byteArray = Encoding.UTF8.`enter code here`GetBytes(postData);
// Set up the POST request for the code exchange.
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = byteArray.Length;
Stream dataStream = request.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
// Perform the POST and retrieve the server response with
// the access token and/or the refresh token.
WebResponse response = request.GetResponse();
dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = reader.ReadToEnd();
reader.Close();
dataStream.Close();
response.Close();
// Convert the response JSON to an object and return it.
return JsonConvert.DeserializeObject<OAuthResponseObject>(
responseFromServer);
Now, I am trying to use that code to get the access token. .which is giving me BAD request.
I also followed few post on stackoverflow. but none of them working for me.
Thanks
EDIT:
Thanks for the reply. I was able to do it my own somehow :)
i think you can begin this way :
Google plus OAuth
For web applications, I would strongly recommend you use a one-time-code flow as demonstrated in the Google+ Quickstart sample. Please try working through the Quickstart instructions to make sure you're not missing any steps. When you do Google+ Sign-In this way, you will be able to get over-the-air installs of Android apps (if you have one for your site) and will be applying best practices for authorization.
All of the code for doing this is available in the sample which also demonstrates integration with the Google client libraries - this opens up access to all of the Google APIs and product integrations.
From Windows apps or installed apps, you would need to do more of the heavy lifting yourself. The following blog article covers how you could do the authorization in legacy scenarios:
http://gusclass.com/blog/2012/08/31/using-the-google-net-client-library-with-google/
There is an example as well:
http://gusclass.com/projects/PlusDotNet.zip
A couple notes:
When you create your client ID, make sure it's for an installed application.
The authorization code is taken from the window title after the user authorizes; this is a little dodgy and you should be doing this in a window you host in your app.
Performing authorization this way will not allow you to have over-the-air installs to Android. For doing this, you could possibly host a webview inside of the application and use that for a one-time-code flow but I have never seen this working from Windows.
I was able to do it my own using following code..
private void button2_Click(object sender, EventArgs e)
{
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("https://accounts.google.com/o/oauth2/token");
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.Method = "POST";
authLink.AppendFormat("code={0}", code);
authLink.AppendFormat("&client_id={0}", "996688211762.apps.googleusercontent.com");
authLink.AppendFormat("&client_secret={0}", "nprfJuBUOyU2hsb3tqt1XDnB");
authLink.AppendFormat("&redirect_uri={0}", "urn:ietf:wg:oauth:2.0:oob");
authLink.Append("&grant_type=authorization_code");
UTF8Encoding utfenc = new UTF8Encoding();
byte[] bytes = utfenc.GetBytes(authLink.ToString());
Stream os = null;
try // send the post
{
webRequest.ContentLength = bytes.Length; // Count bytes to send
os = webRequest.GetRequestStream();
os.Write(bytes, 0, bytes.Length); // Send it
}
catch (Exception ex) { MessageBox.Show(ex.Message); }
try // get the response
{
HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse();
if (webResponse == null) { MessageBox.Show("null"); }
StreamReader sr = new StreamReader(webResponse.GetResponseStream());
textBox1.Text = sr.ReadToEnd().Trim();
//MessageBox.Show(sr.ReadToEnd().Trim());
}
catch (Exception ex) { MessageBox.Show(ex.Message); }
}
private void button1_Click(object sender, EventArgs e)
{
StringBuilder authLink = new StringBuilder();
authLink.Append("https://accounts.google.com/o/oauth2/auth");
authLink.AppendFormat("?scope={0}", "https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.profile");
authLink.AppendFormat("&client_id={0}", "xxxxxx.apps.googleusercontent.com");
authLink.AppendFormat("&redirect_uri={0}", "urn:ietf:wg:oauth:2.0:oob");
authLink.Append("&response_type=code");
string u = #"https://accounts.google.com/o/oauth2/auth?scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.profile&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&client_id=xxxxxxxx.apps.googleusercontent.com";
webBrowser1.Navigate(u);
}
I am assuming to have two button on window.. button1 is used to get CODE from google and button2 uses that code and get the access_token which is what I was looking for.
I'm trying to get an HttpRequest to post this URL
https://www.iformbuilder.com/exzact/_emptyTable.php?PAGE_ID=1234&TABLE_NAME=table_name_here&USERNAME=yo#yo.com&PASSWORD=What!What!
I've tried using
WebClient rar = new WebClient();
rar.OpenReadAsync(new Uri(#"https://www.iformbuilder.com/exzact/_emptyTable.php?PAGE_ID=1234&TABLE_NAME=table_name_here&USERNAME=yo#yo.com&PASSWORD=What!What!"));
rar.DownloadStringAsync(new Uri(#"https://www.iformbuilder.com/exzact/_emptyTable.php?PAGE_ID=1234&TABLE_NAME=table_name_here&USERNAME=yo#yo.com&PASSWORD=What!What!"));
This is suppose to delete my information on their site, but it's not taking. i'm following this documentation.
http://getsatisfaction.com/exzact/topics/how_can_we_delete_old_records_not_manually
and they state that all I have to do is paste the proper URL into a web browser and hit enter and it will work. How would I do this equivalent in c#? Any help would be awesome! Thanks!
Use WebClient.DownloadString instead of DownloadStringAsync. Async indicates asynchronous methods that do not block the current thread.
Try setting your User-Agent header in your WebClient before you submit it to see if that fixes things.
rar.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)")
A lot of web servers are set up to simply ignore requests if the User-Agent header is missing.
Subsequently, you're using HTTPS here so you'll want to set up your ServicePointManager.ServerCertificateValidationCallback as well.
Try to use System.Web classes, for example like this:
HttpWebRequest req = null;
HttpWebResponse resp = null;
try
{
req = (HttpWebRequest)HttpWebRequest.Create(url); // enter your url
req.Method = "post";
resp = (HttpWebResponse)req.GetResponse();
}
catch (Exception)
{
throw;
}
It's example for post method, you can use any other HTTP method like this. Check the documentation.
This isn't a direct answer to your question but check out Hammock for REST
string uriString = #"https://www.iformbuilder.com/exzact/_emptyTable.php?PAGE_ID=1234&TABLE_NAME=table_name_here&USERNAME=yo#yo.com&PASSWORD=What!What!";
using (WebClient webClient = new WebClient { Encoding = Encoding.UTF8 })
{
try
{
string content = webClient.DownloadString(uriString);
//do stuff with the answer you got back from the site
}
catch (Exception exception)
{
//handle exceptions
}
}
I need to write a simple C# app that should receive entire contents of a web page currently opened in Firefox. Is there any way to do it directly from C#? If not, is it possible to develop some kind of plug-in that would transfer page contents? As I am a total newbie in Firefox plug-ins programming, I'd really appreciate any info on getting me started quickly. Maybe there are some sources I can use as a reference? Doc links? Recommendations?
UPD: I actually need to communicate with a Firefox instance, not get contents of a web page from a given URL
It would help if you elaborate What you are trying to achieve. May be plugins already out there such as firebug can help.
Anways, if you really want to develop both plugin and C# application:
Check out this tutorial on firefox extension:
http://robertnyman.com/2009/01/24/how-to-develop-a-firefox-extension/
Otherwise, You can use WebRequest or HttpWebRequest class in .NET request to get the HTML source of any URL.
I think you'd almost certainly need to write a Firefox plugin for that. However there are certainly ways to request a webpage, and receive its HTML response within C#. It depends on what your requirements are?
If you're requirements are simply receive the source from any website, leave a comment and I'll point you towards the code.
Uri uri = new Uri(url);
System.Net.HttpWebRequest req = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(uri.AbsoluteUri);
req.AllowAutoRedirect = true;
req.MaximumAutomaticRedirections = 3;
//req.UserAgent = _UserAgent; //"Mozilla/6.0 (MSIE 6.0; Windows NT 5.1; Searcharoo.NET)";
req.KeepAlive = true;
req.Timeout = _RequestTimeout * 1000; //prefRequestTimeout
// SIMONJONES http://codeproject.com/aspnet/spideroo.asp?msg=1421158#xx1421158xx
req.CookieContainer = new System.Net.CookieContainer();
req.CookieContainer.Add(_CookieContainer.GetCookies(uri));
System.Net.HttpWebResponse webresponse = null;
try
{
webresponse = (System.Net.HttpWebResponse)req.GetResponse();
}
catch (Exception ex)
{
webresponse = null;
Console.Write("request for url failed: {0} {1}", url, ex.Message);
}
if (webresponse != null)
{
webresponse.Cookies = req.CookieContainer.GetCookies(req.RequestUri);
// handle cookies (need to do this incase we have any session cookies)
foreach (System.Net.Cookie retCookie in webresponse.Cookies)
{
bool cookieFound = false;
foreach (System.Net.Cookie oldCookie in _CookieContainer.GetCookies(uri))
{
if (retCookie.Name.Equals(oldCookie.Name))
{
oldCookie.Value = retCookie.Value;
cookieFound = true;
}
}
if (!cookieFound)
{
_CookieContainer.Add(retCookie);
}
}
string enc = "utf-8"; // default
if (webresponse.ContentEncoding != String.Empty)
{
// Use the HttpHeader Content-Type in preference to the one set in META
doc.Encoding = webresponse.ContentEncoding;
}
else if (doc.Encoding == String.Empty)
{
doc.Encoding = enc; // default
}
//http://www.c-sharpcorner.com/Code/2003/Dec/ReadingWebPageSources.asp
System.IO.StreamReader stream = new System.IO.StreamReader
(webresponse.GetResponseStream(), System.Text.Encoding.GetEncoding(doc.Encoding));
webresponse.Close();
This does what you want.
using System.Net;
var cli = new WebClient();
string data = cli.DownloadString("http://www.heise.de");
Console.WriteLine(data);
Native messaging enables an extension to exchange messages with a native application installed on the user's computer.
Is there any way that I could get the source of a website (as a string preferably), let's say www.google.com, from some c# code inside code behind of asp.net website?
edit: of course i mean html code - in every browser you can view it using "view source" in context menu.
Assuming you want to retrieve the html:
class Program
{
static void Main(string[] args)
{
using (WebClient client = new WebClient())
using (Stream stream = client.OpenRead("http://www.google.com"))
using (StreamReader reader = new StreamReader(stream))
{
Console.WriteLine(reader.ReadToEnd());
}
}
}
For C#, I prefer to use HttpWebRequest over WebClient because you can have more option in the future like having GET/POST parameter, using Cookies, etc.
You can have a shortest explication at MSDN.
Here is the example from MSDN:
// Create a new HttpWebRequest object.
HttpWebRequest request=(HttpWebRequest) WebRequest.Create("http://www.contoso.com/example.aspx");
// Set the ContentType property.
request.ContentType="application/x-www-form-urlencoded";
// Set the Method property to 'POST' to post data to the URI.
request.Method = "POST";
// Start the asynchronous operation.
request.BeginGetRequestStream(new AsyncCallback(ReadCallback), request);
// Keep the main thread from continuing while the asynchronous
// operation completes. A real world application
// could do something useful such as updating its user interface.
allDone.WaitOne();
// Get the response.
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream streamResponse = response.GetResponseStream();
StreamReader streamRead = new StreamReader(streamResponse);
string responseString = streamRead.ReadToEnd();
Console.WriteLine(responseString);
// Close the stream object.
streamResponse.Close();
streamRead.Close();
// Release the HttpWebResponse.
response.Close();
it's not the most obvious (and the best) way but i found out that in windows forms you can use WebBrowser control (if you actually need it), fill it's Url property with the url you need and when it's loaded, read the DocumentText property - it contains the html code of the viewed site.