I am trying to use public proxy server (http://www.unblockwebnow.info/) to send HTTP request to destination site, say http://stackoverflow.com :)
My HTTP client has following architecture:
string url = "http://stackoverflow.com";
HttpWebRequest HttpWRequest = (HttpWebRequest)WebRequest.Create(url);
HttpWRequest.Method = "GET";
WebProxy myProxy = new WebProxy();
myProxy.Address = new Uri("http://www.unblockwebnow.info/");
HttpWRequest.Proxy = myProxy;
HttpWebResponse HttpWResponse = (HttpWebResponse)HttpWRequest.GetResponse();
StreamReader sr = new StreamReader(HttpWResponse.GetResponseStream(), encoding);
var rawHTML = sr.ReadToEnd();
sr.Close();
After executing the code for rawHTML I get "pageok -managed by puppet - hostingcms02 pageok"
If I comment out HttpWRequest.Proxy = myProxy; line, I get the site content.
This seems to work, but not with your proxy (don't know port number for unblockwebnow.info).
Added port number after ":" in URI
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.IO;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
string url = "http://stackoverflow.com";
HttpWebRequest HttpWRequest = (HttpWebRequest)WebRequest.Create(url);
HttpWRequest.Method = "GET";
WebProxy myProxy = new WebProxy();
//United States proxy, from http://www.hidemyass.com/proxy-list/
myProxy.Address = new Uri("http://72.64.146.136:8080");
HttpWRequest.Proxy = myProxy;
HttpWebResponse HttpWResponse = (HttpWebResponse)HttpWRequest.GetResponse();
StreamReader sr = new StreamReader(HttpWResponse.GetResponseStream(), true);
var rawHTML = sr.ReadToEnd();
sr.Close();
Console.Out.WriteLine(rawHTML);
Console.ReadKey();
}
}
}
Related
I'm trying to implement an IPN-Listener for Confirming Paypal-Payments of my Customers. I tried to implement an IPN-Listener and when I tested it with the Paypal simulator it says verified handshake. Nevertheless when I try to buy something with a sandbox account, and after the payment its linked to my confirmation site, it always says that the transaction is invalid. Do you guys have an idea?
Here is my listener code
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
public partial class Confirmation : System.Web.UI.Page
{
protected void Page_Load(object sender, System.EventArgs e)
{
// Post back to either sandbox or live
string strSandbox = "https://www.sandbox.paypal.com/cgi-bin/webscr";
string strLive = "https://www.paypal.com/cgi-bin/webscr";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(strSandbox);
// Set values for the request back
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
byte[] Param = Request.BinaryRead(HttpContext.Current.Request.ContentLength);
string strRequest = Encoding.ASCII.GetString(Param);
strRequest = strRequest + "&cmd=_notify-validate";
req.ContentLength = strRequest.Length;
// for proxy
// Dim proxy As New WebProxy(New System.Uri("http://url:port#"))
// req.Proxy = proxy
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls;
ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
ServicePointManager.SecurityProtocol = (SecurityProtocolType)3072;
// Send the request to PayPal and get the response
StreamWriter streamOut = new StreamWriter(req.GetRequestStream(), Encoding.ASCII);
streamOut.Write(strRequest);
streamOut.Close();
StreamReader streamIn = new StreamReader(req.GetResponse().GetResponseStream());
string strResponse = streamIn.ReadToEnd();
streamIn.Close();
if (strResponse == "VERIFIED")
{
test.InnerText = "Verified";
}
else if (strResponse == "INVALID")
{
test.InnerText = "invalid";
}
else
{
test.InnerText = "error";
}
Span1.InnerText = strRequest;
test.InnerText = strResponse;
}
}
EDIT: In the IPN history I get the 200 HTTP Response and its says its verified. But why my code says its INVALID?
I'm doing some research, and I want to logging into the ecac site using my company's certificate ... using Chrome I can access the system without any problem because chrome asks for the certificate to authenticate the login. I tried to implement a small test crawler, where I put my certificate in the request, but, the government website always returns error ... So I tried to use selenium with the chrome driver to login, but with that it is necessary that I select the certificate manually, and my idea is to do this automatically.
My test source is:
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using System.Threading.Tasks;
namespace TestCrawler
{
class Program
{
static void Main(string[] args)
{
string host = #"https://cav.receita.fazenda.gov.br/autenticacao/login";
string certName = #"certificado.pfx";
string password = #"senha";
try
{
X509Certificate2 certificate = new X509Certificate2(certName, password);
ServicePointManager.CheckCertificateRevocationList = false;
ServicePointManager.ServerCertificateValidationCallback = (a, b, c, d) => true;
ServicePointManager.Expect100Continue = true;
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(host);
req.PreAuthenticate = true;
req.AllowAutoRedirect = true;
req.ClientCertificates.Add(certificate);
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
string postData = "login-form-type=cert";
byte[] postBytes = Encoding.UTF8.GetBytes(postData);
req.ContentLength = postBytes.Length;
Stream postStream = req.GetRequestStream();
postStream.Write(postBytes, 0, postBytes.Length);
postStream.Flush();
postStream.Close();
WebResponse resp = req.GetResponse();
Stream stream = resp.GetResponseStream();
using (StreamReader reader = new StreamReader(stream))
{
string line = reader.ReadLine();
while (line != null)
{
Console.WriteLine(line);
line = reader.ReadLine();
}
}
stream.Close();
}
catch (Exception e)
{
Console.WriteLine(e);
}
}
}
}
When analyzing the source of the ecac site, I have identified that in the login button by the certificate, a JS code is executed
onclick="javascript:document.loginCert.submit(); return false;
My another test:
using System;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography.X509Certificates;
namespace TestCrawler
{
class Program
{
private static string host = #"https://cav.receita.fazenda.gov.br/";
private static string certName = #"certificado.pfx";
private static string password = #"senha";
static async void Login()
{
X509Certificate2 certificate = new X509Certificate2(certName, password);
var handler = new WebRequestHandler();
handler.ClientCertificates.Add(certificate);
using (var client = new HttpClient(handler))
{
client.BaseAddress = new Uri(host);
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("text/html"));
// New code:
using (HttpResponseMessage response = await client.GetAsync("autenticacao/Login/Certificado"))
{
using (HttpContent content = response.Content)
{
string result = await content.ReadAsStringAsync();
}
}
}
}
static void Main(string[] args)
{
Login();
Console.ReadLine();
}
}
}
I use the code below to call an OData service (which is the working service from Odata.org) from C# and I don't get any result.The error is in the response.GetResponseStream().
Here is the error :
Length = 'stream.Length' threw an exception of type 'System.NotSupportedException'
I want to call to the service and parse the data from it, what is the simpliest way to do that?
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Web;
using System.Net;
using System.IO;
using System.Xml;
namespace ConsoleApplication1
{
public class Class1
{
static void Main(string[] args)
{
Class1.CreateObject();
}
private const string URL = "http://services.odata.org/OData/OData.svc/Products?$format=atom";
private static void CreateObject()
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(URL);
request.Method = "GET";
request.ContentType = "application/xml";
request.Accept = "application/xml";
using (WebResponse response = request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
XmlTextReader reader = new XmlTextReader(stream);
}
}
}
}
}
I ran your code on my machine, and it executed fine, I was able to traverse all XML elements retrieved by the XmlTextReader.
var request = (HttpWebRequest)WebRequest.Create(URL);
request.Method = "GET";
request.ContentType = "application/xml";
request.Accept = "application/xml";
using (var response = request.GetResponse())
{
using (var stream = response.GetResponseStream())
{
var reader = new XmlTextReader(stream);
while (reader.Read())
{
Console.WriteLine(reader.Value);
}
}
}
But as #qujck suggested, take a look at HttpClient. It's much easier to use.
If you're running .NET 4.5 then take a look at HttpClient (MSDN)
HttpClient client = new HttpClient();
client.DefaultRequestHeaders.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/json"));
HttpResponseMessage response = await client.GetAsync(endpoint);
Stream stream = await response
.Content.ReadAsStreamAsync().ConfigureAwait(false);
response.EnsureSuccessStatusCode();
See here and here for complete examples
I want to execute a QueryString using WebClient but using the POST Method
That is what I got so far
CODE:
using (var client = new WebClient())
{
client.QueryString.Add("somedata", "value");
client.DownloadString("uri");
}
It is working but unfortunately it is using GET not POST and the reason I want it to use POST is that I am doing a web scraping and this is how the request is made as I see in WireShark. [IT USES POST AS A METHOD BUT NO POST DATA ONLY THE QUERY STRING.]
In answer to your specific question:
client.Headers.Add("Content-Type", "application/x-www-form-urlencoded");
byte[] response = client.UploadData("your url", "POST", new byte[] { });
//get the response as a string and do something with it...
string s = System.Text.Encoding.Default.GetString(response);
But using WebClient can be a PITA since it doesn't accept cookies nor allow you to set a timeout.
this will help you, use WebRequest instead of WebClient.
using System;
using System.Net;
using System.Threading;
using System.IO;
using System.Text;
class ThreadTest
{
static void Main()
{
WebRequest req = WebRequest.Create("http://www.yourDomain.com/search");
req.Proxy = null;
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
string reqString = "searchtextbox=webclient&searchmode=simple";
byte[] reqData = Encoding.UTF8.GetBytes(reqString);
req.ContentLength = reqData.Length;
using (Stream reqStream = req.GetRequestStream())
reqStream.Write(reqData, 0, reqData.Length);
using (WebResponse res = req.GetResponse())
using (Stream resSteam = res.GetResponseStream())
using (StreamReader sr = new StreamReader(resSteam))
File.WriteAllText("SearchResults.html", sr.ReadToEnd());
System.Diagnostics.Process.Start("SearchResults.html");
}
}
As the title states, I just need help with the first part of OAuth1.0 authentication: obtaining the request token. I am doing this in a console application using C#. I have been working on this for 3 days now and I have tried numerous samples from the internet, but so far nothing works. Here is my current attempt:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Net.Http;
using System.Net.Http.Headers;
namespace MCAPIClient
{
class Program
{
static void Main(string[] args)
{
RunAsync().Wait();
}
static async Task RunAsync()
{
using (var client = new HttpClient())
{
var values = new Dictionary<string, string>
{
{ "oauth_consumer_key", "<my consumer key>" },
{ "oauth_consumer_secret", "<my secret key>" }
};
var content = new FormUrlEncodedContent(values);
var response = await client.PostAsync("https://app.masteryconnect.com/oauth/request_token", content);
var responseString = await response.Content.ReadAsStringAsync();
Console.WriteLine(response.Headers);
Console.WriteLine("Press any key to continue...");
Console.ReadKey(true);
}
}
}
}
This works beautifully when consuming an API without authentication (like http://rest-service.guides.spring.io/greeting), but receives 401 Forbidden when running it like this. What am I missing? BTW, I'm brand new to API's.
You must have a key and secret, or username and password. Both of them are very necessary to fetch access_token.
i suggest use google postman tool it makes life lot easier in case of api's.
Below code will solve your problem.
WebRequest req = WebRequest.Create(url);
req.Method = "POST";
req.Headers["Authorization"] = "Basic " + Convert.ToBase64String(Encoding.Default.GetBytes("key:Secret"));
req.Credentials = new NetworkCredential("username", "password");
var postData = "grant_type=client_credentials";
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = byteArray.Length;
Stream dataStream = req.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
WebResponse response = req.GetResponse();
// Console.WriteLine(((HttpWebResponse)response).StatusDescription);
dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = reader.ReadToEnd();
Newtonsoft.Json.Linq.JObject o = Newtonsoft.Json.Linq.JObject.Parse(responseFromServer);
String status = (string)o.SelectToken(".access_token");