Develop Reference

Web Response 'an error occured while sending this request' while trying to login into Rockstar social club page

I want to login to Rockstar Social Club page https://pl.socialclub.rockstargames.com
I have this script
public static void Login()
{
string firstUrl = "https://pl.socialclub.rockstargames.com/profile/signin";
string formParams = string.Format("login-field={0}&password-field={1}", "mynickname", "mypassword");
string cookieHeader;
WebRequest req = WebRequest.Create(firstUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
string pageSource;
string getUrl = "https://pl.socialclub.rockstargames.com/games/gtav/pc/career/overview/gtaonline";
WebRequest getRequest = WebRequest.Create(getUrl);
getRequest.Headers.Add("Cookie", cookieHeader);
WebResponse getResponse = getRequest.GetResponse(); //Here returns me this error: System.Net.WebException: 'An error occurred while sending the request"
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
}
Error occures in WebResponse getResponse = getRequest.GetResponse();
System.Net.WebException: 'An error occurred while sending the request'
I don't know how to repair this, and succesfully login to this website.

I have accomplished what you are attempting to do, but on a different website.
Basically - a few years ago, I wanted to create a website that would track my Guild/Company details on Final Fantasy XIV.
They didn't have an API, so I made one.
In order to get the information I required, I needed to use a mix of HtmlAgilityPack along with the C# WebBrowser control.
In order to pass the verification token stage above, you need to run the page source in a Web Browser control.
This will allow dynamic fields and data to be generated.
You then need to take that data, and submit it with your post data.
This is to fool it into thinking the request is coming from the page.
Be warned, when doing your posts - you may need to allow for redirects and you may need to mirror the referrer and host fields to match the website you are emulating.
The specific process I followed was:
Navigate to login page in WebBrowser control
Get page source
Load into HtmlAgilityPack HtmlDocument class
Use XPath to scrape the login form.
Take _verification tokens, csrf tokens etc make note of them.
Post a web-request with the necessary data to the form target destination url.
Read the response
Be aware - sometimes the response will actually be html code that tells it to do a Javascript redirect - in my case with Final Fantasy XIV - it was loading up another form and performing an autopost on page load.
You will also want to use
LoggedInCookies = new CookieContainer();
In your first HttpWebRequest
followed by:
request.CookieContainer = LoggedInCookies;
for each subsequent request.
The cookie container will trap and persist the authentication related cookies, while the WebBrowser control and HtmlAgilityPack will allow you to scrape the fields from the web forms that you need to break through.
Adding some code from wayback when I solved this for Final Fantasy XIV's lodestone website.
This code is very old and may not work anymore, but the process it follows could be adapted for sites that do not use Javascript as part of the login process.
Pay attention to the areas where it allows the request to be redirected, this is because the Server endpoint you are calling may do Action redirects etc
If your request does not allow those redirects, then it will not be emulating the login process.
class LoggedInClient
{
public static CookieContainer LoginCookie(string user, string pass)
{
string sStored = "";
string url = "http://eu.finalfantasyxiv.com/lodestone/account/login/";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
CookieContainer cookies = new CookieContainer();
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
HttpWebResponse response1 = (HttpWebResponse)request.GetResponse();
Console.WriteLine(cookies.Count.ToString());
string sPage = "";
using (var vPage = new StreamReader(response1.GetResponseStream()))
{
sPage = vPage.ReadToEnd();
}
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(sPage);
sStored = doc.DocumentNode.SelectSingleNode("//input[#type='hidden' and #name='_STORED_']").Attributes["value"].Value;
string param = "sqexid="+user+"8&password="+pass+"&_STORED_=" + sStored;
string postURL = doc.DocumentNode.SelectSingleNode("//form[#name='mainForm']").Attributes["action"].Value;
//Console.WriteLine(sStored);
postURL = "https://secure.square-enix.com/oauth/oa/" + postURL;
request.Method = "POST";
byte[] paramAsBytes = Encoding.Default.GetBytes(param);
request = (HttpWebRequest)WebRequest.Create(postURL);
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
request.AllowAutoRedirect = false;
try
{
using (Stream stream = request.GetRequestStream())
{
stream.Write(paramAsBytes, 0, paramAsBytes.Length);
}
}
catch (Exception ee)
{
Console.WriteLine(ee.ToString());
}
string sGETPage = "";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (var vPage = new StreamReader(response.GetResponseStream()))
{
sPage = vPage.ReadToEnd();
sGETPage = response.Headers["Location"];
}
}
// Console.WriteLine(sPage);
request = (HttpWebRequest)WebRequest.Create(sGETPage);
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
HttpWebResponse response2 = (HttpWebResponse)request.GetResponse();
Console.WriteLine(cookies.Count.ToString());
sPage = "";
using (var vPage = new StreamReader(response2.GetResponseStream()))
{
sPage = vPage.ReadToEnd();
}
// Console.WriteLine(sPage);
doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(sPage);
string _c = doc.DocumentNode.SelectSingleNode("//input[#type='hidden' and #name='_c']").Attributes["value"].Value;
string cis_sessid = doc.DocumentNode.SelectSingleNode("//input[#type='hidden' and #name='cis_sessid']").Attributes["value"].Value;
string action = doc.DocumentNode.SelectSingleNode("//form[#name='mainForm']").Attributes["action"].Value;
string sParams = "_c=" + _c + "&cis_sessid=" + cis_sessid;
byte[] bData = Encoding.Default.GetBytes(sParams);
// Console.WriteLine(sStored);
request = (HttpWebRequest)WebRequest.Create(action);
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
request.AllowAutoRedirect = true;
try
{
using (Stream stream = request.GetRequestStream())
{
stream.Write(bData, 0, bData.Length);
}
}
catch (Exception ee)
{
Console.WriteLine(ee.ToString());
}
string nextPage = "";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (var vPage = new StreamReader(response.GetResponseStream()))
{
nextPage = vPage.ReadToEnd();
}
}
// Console.WriteLine(nextPage);
doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(nextPage);
string csrf_token = doc.DocumentNode.SelectSingleNode("//input[#type='hidden' and #name='csrf_token']").Attributes["value"].Value;
string cicuid = "51624738";
string timestamp = Convert.ToInt32(DateTime.UtcNow.Subtract(new DateTime(1970, 1, 1)).TotalSeconds).ToString() + "100";
action = "http://eu.finalfantasyxiv.com/lodestone/api/account/select_character/";
sParams = "csrf_token=" + csrf_token + "&cicuid=" + cicuid + "&timestamp=" + timestamp;
bData = Encoding.Default.GetBytes(sParams);
request = (HttpWebRequest)WebRequest.Create(action);
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36";
request.CookieContainer = cookies;
request.AllowAutoRedirect = true;
try
{
using (Stream stream = request.GetRequestStream())
{
stream.Write(bData, 0, bData.Length);
}
}
catch (Exception ee)
{
Console.WriteLine(ee.ToString());
}
nextPage = "";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (var vPage = new StreamReader(response.GetResponseStream()))
{
nextPage = vPage.ReadToEnd();
}
}
return cookies;
}
}

Cookie count difference between web request and c# request

I have a https website, when I access it via the browser and check the cookies (via firebug), it gives me 3 cookies. but when I make the request via c# code, the cookie count is 2. one of them is missing. Can someone help what is happening ?
public void Login(string url)
{
//Create request to URL.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create( url);
//Set request headers.
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0";
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
request.Headers.Set(HttpRequestHeader.AcceptLanguage, "en-US,en;q=0.5");
request.Headers.Set(HttpRequestHeader.AcceptEncoding, "gzip, deflate, br");
request.ContentType = "application/x-www-form-urlencoded";
request.KeepAlive = true;
request.Headers.Add("Upgrade-Insecure-Requests", #"1");
//Set request method
request.Method = "POST";
request.ServicePoint.Expect100Continue = false;
// Set request body.
string body = #"";
byte[] postBytes = System.Text.Encoding.UTF8.GetBytes(body);
request.ContentLength = postBytes.Length;
Stream stream = request.GetRequestStream();
stream.Write(postBytes, 0, postBytes.Length);
stream.Close();
CookieContainer cookieJar = new CookieContainer();
request.CookieContainer = cookieJar;
//Get response to request.
var response = (HttpWebResponse)request.GetResponse();
var xx = ReadResponse(response);
int cookieCount = cookieJar.Count;
var items = cookieJar.List();
fixCookies(response);
items = cookieJar.List();
}
private static void fixCookies(HttpWebResponse response)
{
for (int i = 0; i < response.Headers.Count; i++)
{
string name = response.Headers.GetKey(i);
if (name != "Set-Cookie")
continue;
string value = response.Headers.Get(i);
foreach (var singleCookie in value.Split(','))
{
//I get only two values here.
}
}
}

401 error when try to connect to Lighthouse API

I'm trying to connect to the Lighthouse api using C# code. This is the php example https://support.lighthouserocks.com/hc/en-gb/articles/201319732-API-The-Basics which it describe how to do this. But I'm fails with it. I was try both NetworkCredentials & send in the Header but still have 401 Unauthorized access to it, here is the code:
public string RequestResponse()
{
HttpWebRequest webRequest = WebRequest.Create(HomeUrl) as HttpWebRequest;
webRequest.Method = "GET";
webRequest.ContentType = "application/json";
webRequest.ServicePoint.Expect100Continue = false;
webRequest.Timeout = 20000;
string auth = CreateAuthorization("domain.lhlive.com", "user", "token");
webRequest.Headers["auth"] = "Basic " + auth;
//webRequest.Credentials = new NetworkCredential("user", "token");
//webRequest.PreAuthenticate = true;
//webRequest.Headers.Add("auth", "user, token");
webRequest.Accept = "application/vnd.lighthouse.v1.hal+json";
Stream responseStream = null;
StreamReader responseReader = null;
string responseData = "";
try
{
WebResponse webResponse = webRequest.GetResponse();
responseStream = webResponse.GetResponseStream();
responseReader = new StreamReader(responseStream);
responseData = responseReader.ReadToEnd();
}
finally
{
if (responseStream != null)
{
responseStream.Close();
responseReader.Close();
}
}
return responseData;
}
public void Test()
{
using (var client = new WebClient())
{
client.Headers["User-Agent"] = "Mozilla/4.0 (Compatible; Windows NT 5.1; MSIE 6.0) (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)";
client.Headers["ContentType"] = "application/json";
client.Headers["Accept"] = "application/vnd.lighthouse.v1.hal+json";
//client.Headers["Lighthouse Username"] = "user";
//client.Headers["API Key"] = "token";
client.Headers["WWW-Authenticate"] = "user, token";
byte[] arr = client.DownloadData("https://domain.lhlive.com/contacts");
Console.WriteLine("--- WebClient result ---");
Console.WriteLine(arr.Length);
}
}
anybody know what I should to do?

Hard to say because I can't access Lighthouse but try the following (notice how auth header is set).
var webRequest = WebRequest.Create("http://some.endpoint.com/") as HttpWebRequest;
webRequest.Method = "GET";
webRequest.Accept = "application/vnd.lighthouse.v1.hal+json";
webRequest.ContentType = "application/json";
webRequest.Headers["Authorization"] = string.Format("Basic {0}",
Convert.ToBase64String(Encoding.Default.GetBytes(
string.Format("{0}:{1}", "your username", "your API key"))));
var response = webRequest.GetResponse();
var stream = response.GetResponseStream();
var data = (new StreamReader(stream)).ReadToEnd();
In this case your Authorization header looks like "Basic eW91ciB1c2VybmFtZTp5b3VyIEFQSSBrZXk=".

missing post data by using HttpWebRequest

I got a problem on posting data by using HttpWebRequest.
There is a string(ie. key1=value1&key2=value2&key3=value3) and I have post it to a site (ie. www.*.com/edit), but ,I don't know why that sometimes it's nothing wrong , but sometimes ,the first key=value1 will be missing, only key2=value&key3=value3 that can find in HttpAnalyzer.
public static string SubmitData(string Url, string FormData, CookieContainer _Cc, string ContentType)
{
Stream RequestStream = null, ResponseStream = null; StreamReader Sr = null;
HttpWebRequest HRequest = (HttpWebRequest)WebRequest.Create(Url);
try
{
HRequest.CookieContainer = _Cc;
HRequest.Method = "POST";
HRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0)";
HRequest.ContentType = ContentType;
HRequest.ContentLength = FormData.Length;
//byte[] BFromData = new ASCIIEncoding().GetBytes(FormData);
byte[] BFromData = Encoding.ASCII.GetBytes(FormData);
BFromData = Encoding.Convert(Encoding.ASCII, Encoding.UTF8, BFromData);//ascii → utf8
RequestStream = HRequest.GetRequestStream();
RequestStream.Write(BFromData, 0, BFromData.Length);
//RequestStream.Write(utf8Bytes,0,utf8Bytes.Length );
HttpWebResponse HResponse = (HttpWebResponse)HRequest.GetResponse();
ResponseStream = HResponse.GetResponseStream();
Sr = new StreamReader(ResponseStream, Encoding.UTF8);
return Sr.ReadToEnd();
}
catch
{
return "";
}
finally
{
if (null != RequestStream) RequestStream.Close();
if (null != ResponseStream) ResponseStream.Close();
if (null != Sr) Sr.Close();
}
}

Use Fiddler to see how the request looks like when you click on the form then try using this approach and modify what you need for your request.
public static void PostDataAndDoSomething()
{
string URI = "http://www.something.com";
//make your request payload
string requestBody = String.Format("{{'param1': {0}, 'param2': {1}}}",value1, value2); //json format
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(URI); //make request
// set request headers as you need
request.ContentType = "application/json; charset=UTF-8";
request.Accept = "application/json, text/javascript;
request.Method = "POST";
request.UserAgent = "";
request.Headers.Add("X-Requested-With", "XMLHttpRequest");
using (StreamWriter writer = new StreamWriter(request.GetRequestStream()))
{
writer.Write(requestBody); //write your request payload
}
WebResponse response = request.GetResponse();
string jsonData = String.Empty;
using (var reader = new StreamReader(response.GetResponseStream()))
{
jsonData = reader.ReadToEnd();
}
response.Close();
//do something with your data, deserialize, Regex etc....
}

Cannot login website with httprequest POST in C#?

I'm trying to write a bit of code to login to a website. But it's not working. Please can you give me some advice. This is my a bit of code:
static void Main(string[] args)
{
CookieContainer container = new CookieContainer();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://pagehype.com/login.php");
request.Method = "POST";
request.Timeout = 10000;
request.ReadWriteTimeout = 30000;
request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) (Prevx 3.0.5)";
request.CookieContainer = container;
ASCIIEncoding encoding = new ASCIIEncoding();
string postData = "username=user&password=password&processlogin=1&return=";
byte[] data = encoding.GetBytes(postData);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = data.Length;
Stream newStream = request.GetRequestStream();
newStream.Write(data, 0, data.Length);
newStream.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string htmldoc = reader.ReadToEnd();
response.Close();
Console.Write(htmldoc);
}
Many thanks,

Use http://www.fiddler2.com/fiddler2/ to view the http request sent went you login in using a browser and ensure that the request you build in code is the same.

PHP logins use PHPSESSID cookie. You'll need to capture this and pass it back in the CookieContainer. This is how the server will recognise you as an authenticated user.
The cookie is set in the Set-Cookie header in the initial response. You'll need to parse it to recreate the cookie in your container (don't forget the path (and domain?)
var setCookie = response.GetResponseHeader("Set-Cookie");
response.Close();
container = new CookieContainer();
foreach (var cookie in setCookie.Split(','))
{
var split = cookie.Split(';');
string name = split[0].Split('=')[0];
string value = split[0].Split('=')[1];
var c = new Cookie(name, value);
if (cookie.Contains(" Domain="))
c.Domain = split.Where(x => x.StartsWith(" Domain")).First().Split('=')[1];
else
{
c.Domain = ".pagehype.com";
}
if (cookie.Contains(" Path="))
c.Path = split.Where(x => x.StartsWith(" Path")).First().Split('=')[1];
container.Add(c);
}
Then add this container to your request.