The error says:
"The remote server returned an error:
(http://www.tgv.com.my/movies/man-city-v-arsenal-HO00005174)
Forbidden"
below is my code:
string url = https://translate.google.com/translate_a/single?client=t&sl=en&tl=vi&hl=vi&dt=bd&dt=ex&dt=ld&dt=md&dt=qca&dt=rw&dt=rm&dt=ss&dt=t&dt=at&ie=UTF-8&oe=UTF-8&otf=1&srcrom=1&ssel=0&tsel=0&kc=5&tk=520987|10880&q=" + keyword;
var request = (HttpWebRequest)WebRequest.Create(url);
WebProxy proxy = (WebProxy)WebProxy.GetDefaultProxy();
if (proxy.Address != null)
{
proxy.Credentials = proxy.Credentials = new NetworkCredential("username", "pw");
WebRequest.DefaultWebProxy = new System.Net.WebProxy(proxy.Address, proxy.BypassProxyOnLocal, proxy.BypassList, proxy.Credentials);
}
request.Proxy = proxy;
var postData = "";
var data = Encoding.ASCII.GetBytes(postData);
request.Method = "POST";
request.ContentType = "text/html; charset=UTF-8";
request.ContentLength = data.Length;
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36";
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
using (var stream = request.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}
var response = (HttpWebResponse)request.GetResponse();
var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
return responseString;
Thanks!
Using your Url, it's not a POST request, it's a GET request and could be done like this:
string url = "https://translate.google.com/translate_a/single?client=t&sl=de&tl=en&hl=de&dt=at&dt=bd&dt=ex&dt=ld&dt=md&dt=qca&dt=rw&dt=rm&dt=ss&dt=t&ie=UTF-8&oe=UTF-8&otf=2&ssel=0&tsel=0&kc=4&tk=767774.885916&q=hallo%20du";
var request = (HttpWebRequest)WebRequest.Create(url);
var response = (HttpWebResponse)request.GetResponse();
var responseString = new StreamReader(response.GetResponseStream()).ReadToEnd();
But if your q should have another value, other values must be changed too, otherwise you will have the Error 403 whitch tells you, that you do not have permission to do your request.
Using Google Translate, consider having a look at the Google Translate API.
There you can do your request like that:
https://www.googleapis.com/language/translate/v2?key=YOUR_API_KEY&q=hello%20world&source=en&target=de
But this is a payd service...
Related
I want to login to Rockstar Social Club page https://pl.socialclub.rockstargames.com
I have this script
public static void Login()
{
string firstUrl = "https://pl.socialclub.rockstargames.com/profile/signin";
string formParams = string.Format("login-field={0}&password-field={1}", "mynickname", "mypassword");
string cookieHeader;
WebRequest req = WebRequest.Create(firstUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
string pageSource;
string getUrl = "https://pl.socialclub.rockstargames.com/games/gtav/pc/career/overview/gtaonline";
WebRequest getRequest = WebRequest.Create(getUrl);
getRequest.Headers.Add("Cookie", cookieHeader);
WebResponse getResponse = getRequest.GetResponse(); //Here returns me this error: System.Net.WebException: 'An error occurred while sending the request"
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
}
Error occures in WebResponse getResponse = getRequest.GetResponse();
System.Net.WebException: 'An error occurred while sending the request'
I don't know how to repair this, and succesfully login to this website.
I have accomplished what you are attempting to do, but on a different website.
Basically - a few years ago, I wanted to create a website that would track my Guild/Company details on Final Fantasy XIV.
They didn't have an API, so I made one.
In order to get the information I required, I needed to use a mix of HtmlAgilityPack along with the C# WebBrowser control.
In order to pass the verification token stage above, you need to run the page source in a Web Browser control.
This will allow dynamic fields and data to be generated.
You then need to take that data, and submit it with your post data.
This is to fool it into thinking the request is coming from the page.
Be warned, when doing your posts - you may need to allow for redirects and you may need to mirror the referrer and host fields to match the website you are emulating.
The specific process I followed was:
Navigate to login page in WebBrowser control
Get page source
Load into HtmlAgilityPack HtmlDocument class
Use XPath to scrape the login form.
Take _verification tokens, csrf tokens etc make note of them.
Post a web-request with the necessary data to the form target destination url.
Read the response
Be aware - sometimes the response will actually be html code that tells it to do a Javascript redirect - in my case with Final Fantasy XIV - it was loading up another form and performing an autopost on page load.
You will also want to use
LoggedInCookies = new CookieContainer();
In your first HttpWebRequest
followed by:
request.CookieContainer = LoggedInCookies;
for each subsequent request.
The cookie container will trap and persist the authentication related cookies, while the WebBrowser control and HtmlAgilityPack will allow you to scrape the fields from the web forms that you need to break through.
Adding some code from wayback when I solved this for Final Fantasy XIV's lodestone website.
This code is very old and may not work anymore, but the process it follows could be adapted for sites that do not use Javascript as part of the login process.
Pay attention to the areas where it allows the request to be redirected, this is because the Server endpoint you are calling may do Action redirects etc
If your request does not allow those redirects, then it will not be emulating the login process.
class LoggedInClient
{
public static CookieContainer LoginCookie(string user, string pass)
{
string sStored = "";
string url = "http://eu.finalfantasyxiv.com/lodestone/account/login/";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
CookieContainer cookies = new CookieContainer();
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
HttpWebResponse response1 = (HttpWebResponse)request.GetResponse();
Console.WriteLine(cookies.Count.ToString());
string sPage = "";
using (var vPage = new StreamReader(response1.GetResponseStream()))
{
sPage = vPage.ReadToEnd();
}
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(sPage);
sStored = doc.DocumentNode.SelectSingleNode("//input[#type='hidden' and #name='_STORED_']").Attributes["value"].Value;
string param = "sqexid="+user+"8&password="+pass+"&_STORED_=" + sStored;
string postURL = doc.DocumentNode.SelectSingleNode("//form[#name='mainForm']").Attributes["action"].Value;
//Console.WriteLine(sStored);
postURL = "https://secure.square-enix.com/oauth/oa/" + postURL;
request.Method = "POST";
byte[] paramAsBytes = Encoding.Default.GetBytes(param);
request = (HttpWebRequest)WebRequest.Create(postURL);
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
request.AllowAutoRedirect = false;
try
{
using (Stream stream = request.GetRequestStream())
{
stream.Write(paramAsBytes, 0, paramAsBytes.Length);
}
}
catch (Exception ee)
{
Console.WriteLine(ee.ToString());
}
string sGETPage = "";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (var vPage = new StreamReader(response.GetResponseStream()))
{
sPage = vPage.ReadToEnd();
sGETPage = response.Headers["Location"];
}
}
// Console.WriteLine(sPage);
request = (HttpWebRequest)WebRequest.Create(sGETPage);
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
HttpWebResponse response2 = (HttpWebResponse)request.GetResponse();
Console.WriteLine(cookies.Count.ToString());
sPage = "";
using (var vPage = new StreamReader(response2.GetResponseStream()))
{
sPage = vPage.ReadToEnd();
}
// Console.WriteLine(sPage);
doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(sPage);
string _c = doc.DocumentNode.SelectSingleNode("//input[#type='hidden' and #name='_c']").Attributes["value"].Value;
string cis_sessid = doc.DocumentNode.SelectSingleNode("//input[#type='hidden' and #name='cis_sessid']").Attributes["value"].Value;
string action = doc.DocumentNode.SelectSingleNode("//form[#name='mainForm']").Attributes["action"].Value;
string sParams = "_c=" + _c + "&cis_sessid=" + cis_sessid;
byte[] bData = Encoding.Default.GetBytes(sParams);
// Console.WriteLine(sStored);
request = (HttpWebRequest)WebRequest.Create(action);
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
request.AllowAutoRedirect = true;
try
{
using (Stream stream = request.GetRequestStream())
{
stream.Write(bData, 0, bData.Length);
}
}
catch (Exception ee)
{
Console.WriteLine(ee.ToString());
}
string nextPage = "";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (var vPage = new StreamReader(response.GetResponseStream()))
{
nextPage = vPage.ReadToEnd();
}
}
// Console.WriteLine(nextPage);
doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(nextPage);
string csrf_token = doc.DocumentNode.SelectSingleNode("//input[#type='hidden' and #name='csrf_token']").Attributes["value"].Value;
string cicuid = "51624738";
string timestamp = Convert.ToInt32(DateTime.UtcNow.Subtract(new DateTime(1970, 1, 1)).TotalSeconds).ToString() + "100";
action = "http://eu.finalfantasyxiv.com/lodestone/api/account/select_character/";
sParams = "csrf_token=" + csrf_token + "&cicuid=" + cicuid + "×tamp=" + timestamp;
bData = Encoding.Default.GetBytes(sParams);
request = (HttpWebRequest)WebRequest.Create(action);
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
request.AllowAutoRedirect = true;
try
{
using (Stream stream = request.GetRequestStream())
{
stream.Write(bData, 0, bData.Length);
}
}
catch (Exception ee)
{
Console.WriteLine(ee.ToString());
}
nextPage = "";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (var vPage = new StreamReader(response.GetResponseStream()))
{
nextPage = vPage.ReadToEnd();
}
}
return cookies;
}
}
I am trying to get an access OAuth2 token for Yelp Fusion Api using C# as mentioned in the documentation: https://www.yelp.com/developers/documentation/v3/get_started
However, I am getting the error :
client_id or client_secret not found. Make sure to provide client_id and client_secret in the body with the application application/x-www-form-urlencoded
The following is the code snippet:
<code>
string baseURL = "https://api.yelp.com/oauth2/token";
Dictionary<string, string> query = new Dictionary<string, string>();
query["grant_type"] = "client_credentials";
query["client_id"] = CONSUMER_KEY;
query["client_secret"] = CONSUMER_SECRET;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(baseURL);
request.Accept = "application/json";
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
StreamWriter requestWriter = new StreamWriter(request.GetRequestStream());
requestWriter.Write(query.ToString());
requestWriter.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
var stream = new StreamReader(response.GetResponseStream(), Encoding.UTF8);
Console.WriteLine(stream.ReadToEnd());
</code>
As per the Yelp Fusion documentation here you need to make a POST call and parameters should be sent in application/x-www-form-urlencoded format. So the method used above is incorrect.
This should help:
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(baseURL);
webRequest.Method = "POST";
webRequest.AllowAutoRedirect = true;
webRequest.Timeout = 20 * 1000;
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
webRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36";
//write the data to post request
String postData = "client_id=" + CLIENT_ID + "&client_secret=" + CLIENT_SECRET + "&grant_type=client_credentials";
byte[] buffer = Encoding.Default.GetBytes(postData);
if (buffer != null)
{
webRequest.ContentLength = buffer.Length;
webRequest.GetRequestStream().Write(buffer, 0, buffer.Length);
}
HttpWebResponse response = (HttpWebResponse)webRequest.GetResponse();
Stream dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
string strResponse = reader.ReadToEnd();
Please note that the example above will return data in String format. To read the actual values, you will have to serialize the data.
Edit: Since March 1, 2018, the authentication process for yelp fusion API has changed. This is not applicable any more.
I'm having a problem when sending HttpWebRequest to a particular url.
The response is generated but it is not that I want.
It is giving a response like "Session Timeout! Please Login to continue".
Whereas,
When I'm sending the same request through Python 3.4 it is giving me the required response.
The response is like "--some html is here--" which I want.
I don't know why this is happening because I wan't it to do throught C#.
The CodeBehind (giving the wrong response) is:
string postData = "somequerystring";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("someurl");
request.KeepAlive = true;
request.ProtocolVersion = HttpVersion.Version10;
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36";
request.Method = "POST";
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = byteArray.Length;
request.Headers.Add(HttpRequestHeader.CacheControl, "no-cache");
using (var dataStream = request.GetRequestStream())
{
dataStream.Write(byteArray, 0, byteArray.Length);
}
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string responseFromServer = new StreamReader(response.GetResponseStream()).ReadToEnd();
response.Close();
Context.Response.Write(responseFromServer);
The Python Script (giving the correct response) is:
import requests
url = "someurl"
payload = "somequerystring"
headers = {
'content-type': "application/x-www-form-urlencoded"
}
response = requests.request("POST", url, data=payload, headers=headers)
print(response.text)
Help me out?
When making a specific GET request, it seems there is a chunk of html missing from the center of the html response. I can't seem to figure out how its being removed. If you run the sample code and record it with Wireshark. You'll see that Wireshark has more html data than the result in the sample has. (Search for 'french' in both; you'll notice that it is missing from the sample yet it shows up in wireshark).
static string ReadHTML(string urlAddress)
{
urlAddress = "http://www.ebay.com/sch/Dress-Shirts-/57991/i.html?_fln=1&_dmd=1&_ipg=200&_from=R40&_dcat=57991&LH_ItemCondition=3000&_nkw=&LH_Complete=1&LH_Sold=1";
HttpWebRequest request = WebRequest.Create(urlAddress) as HttpWebRequest;
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/538.1 (KHTML, like Gecko) PhantomJS/2.1.1 Safari/538.1";
request.Host = "www.ebay.com";
request.Headers.Add("Accept-Encoding", "gzip, deflate, sdch");
request.Headers.Add("Accept-Language", "en-US,*");
//request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(new System.Net.Cookie("JSESSIONID", "989EDC0DF3234BF32A2544E4FA96C314") { Domain = request.Host });//JSESSIONID=989EDC0DF3234BF32A2544E4FA96C314
request.CookieContainer.Add(new System.Net.Cookie("npii", "btguid/4948662a1540a1c448711d0effdd49bb58fe33af^cguid/4948bab11540a56603d04eb0fd0d7c0c58fe33af^") { Domain = request.Host });//npii=btguid/4948662a1540a1c448711d0effdd49bb58fe33af^cguid/4948bab11540a56603d04eb0fd0d7c0c58fe33af^
request.CookieContainer.Add(new System.Net.Cookie("ebay", "%5Esbf%3D%2320000000000000000000210%5E") { Domain = request.Host });//ebay=%5Esbf%3D%2320000000000000000000210%5E
request.CookieContainer.Add(new System.Net.Cookie("ns1", "=BAQAAAVQr0QFlAAaAANgASVkAJ2pjNjl8NjAxXjE0NjE1MTgzNjIxNjleXjFeM3wyfDV8NHw3XjFeMl40XjNeMTJeMTJeMl4xXjFeMF4xXjBeMV42NDQyNDU5MDc17HliWNBjA7nLq7R3ubPYrU6hFOk*") { Domain = request.Host });
request.CookieContainer.Add(new System.Net.Cookie("dp1", "bu1p/QEBfX0BAX19AQA**5900276a^bl/US5ae15aea^") { Domain = request.Host });
request.CookieContainer.Add(new System.Net.Cookie("nonsession", "CgADLAAFXHvryMwDKACBghPVqNDk0ODY2MmExNTQwYTFjNDQ4NzExZDBlZmZkZDQ5YmI2iA27") { Domain = request.Host });
request.CookieContainer.Add(new System.Net.Cookie("s", "CgAD4ACBXIEVqNTBkYjAwODExNTQwYTU2MTc4NDcxNWIyZmZjMzkyNWIA7gCVVyBFajMGaHR0cDovL3d3dy5lYmF5LmNvbS9zY2gvRHJlc3MtU2hpcnRzLS81Nzk5MS9pLmh0bWw/X2Zsbj0xJl9kbWQ9MSZfaXBnPTIwMCZfZnJvbT1SNDAmX2RjYXQ9NTc5OTEmTEhfSXRlbUNvbmRpdGlvbj0zMDAwJl9ua3c9JkxIX0NvbXBsZXRlPTEmTEhfU29sZD0xGTda/A**") { Domain = request.Host });
if (request == null)
return "";
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
if (response == null)
return "";
if (response.StatusCode != HttpStatusCode.OK)
return "";
Stream s2 = response.GetResponseStream();
if (response.ContentEncoding.ToLower().Contains("gzip"))
s2 = new GZipStream(s2, CompressionMode.Decompress);
else if (response.ContentEncoding.ToLower().Contains("deflate"))
s2 = new DeflateStream(s2, CompressionMode.Decompress);
StreamReader sr = null;
if (response.CharacterSet == null)
{
sr = new StreamReader(s2);
}
else
{
sr = new StreamReader(s2, Encoding.GetEncoding(response.CharacterSet));
}
string data = sr.ReadToEnd();
response.Close();
return data;
}
You can also simulate the GET request through PostMan (plugin for chrome) and see the exact same html response that Wireshark is getting from the request which is different from the result in the sample code.
Looks like the 'Text Visualizer' in Visual Studio was showing an incomplete response. The data was always there. Just the debugger was wrong.
Now I know not to trust it.
Recently i made a simple software using c# http request but now it seems there is a problem logging in facebook getting and sending request
here is few lines of code that used work
string email = "email";string pw = "pw";string PostData = String.Format("email={0}&pass={1}", email, pw);
CookieCollection cookies = new CookieCollection();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.facebook.com");
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(cookies);
//Get the response from the server and save the cookies from the first request..
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
cookies = response.Cookies;
string getUrl = "https://www.facebook.com/login.php? login_attempt=1";
string postData = String.Format("email={0}&pass={1}", "anything#gmail.com", "yourpassword");
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
getRequest.CookieContainer = new CookieContainer();
getRequest.CookieContainer.Add(cookies); //recover cookies First request
getRequest.Method = WebRequestMethods.Http.Post;
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
getRequest.AllowWriteStreamBuffering = true;
getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
byte[] byteArray = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = byteArray.Length;
Stream newStream = getRequest.GetRequestStream(); //open connection
newStream.Write(byteArray, 0, byteArray.Length); // Send the data.
newStream.Close();
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
string sourceCode = sr.ReadToEnd();
webBrowser1.ScriptErrorsSuppressed = true;
webBrowser1.DocumentText = sourceCode;
}
First ,saving cookies after sending request then using it to login on facebook.(so that it won't mention "Please enable cookies...")
and Now it doesn't work
Anyone know the issue or should i switch to httpclient (though i haven't used httpclient much)
thanks in advance