Develop Reference

send request and receive response with selenium and cookies [duplicate]

I log into the site https://dmarket.com. I want to save cookies and use later. In order not to visit the site next time.
private void login_Click(object sender, EventArgs e)
{
string login = textBox1.Text;
string password = textBox2.Text;
string steamguard = textBox3.Text;
IWebDriver driver = new ChromeDriver();
driver.Navigate().GoToUrl(#"https://steamcommunity.com/openid/login?openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.mode=checkid_setup&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&openid.realm=https%3A%2F%2Fapi.dmarket.live&openid.return_to=https%3A%2F%2Fapi.dmarket.live%2Fauth%2Fv1%2Fcallback%2Fsteam%2F901e7d34-06c1-44b0-82b4-2f982c058361");
driver.FindElement(By.XPath("//*[#id=\"steamAccountName\"]")).SendKeys(login);
driver.FindElement(By.XPath("//*[#id=\"steamPassword\"]")).SendKeys(password);
driver.FindElement(By.XPath("//*[#id=\"imageLogin\"]")).Click();
driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(150);
driver.FindElement(By.XPath("//*[#id=\"twofactorcode_entry\"]")).SendKeys(steamguard);
driver.FindElement(By.XPath("//*[#id=\"login_twofactorauth_buttonset_entercode\"]/div[1]")).Click();
var cookies = driver.Manage().Cookies.AllCookies;
driver.Manage().Cookies.AddCookie(cookies);
}
But an error occurs: Error CS1503 Argument 1: Unable to convert from "System.Collections.ObjectModel.ReadOnlyCollection <OpenQA.Selenium.Cookie>" to "OpenQA.Selenium.Cookie". Maybe I'm doing something wrong. And maybe it should have been done differently.
Thank you!

From my experience dealing high level with cookies will fail you , to master and get root of the problem my way :
Get Cookie manager extension for Firefox or whatever browser you are using.
see how many cookies key/value you are getting after logging.
install fiddler sniffer and see how many of them sent in request after login when browsing the website.
extract that cookies and inject them in HttpClient or similar class and track the requests with fiddler if succeeded or not .
once socket request succeeded , i add the same headers and cookies to selenium request, and continue doing my selenium stuff.
maybe longer approach but always worked with me , let me show you an example with instagram login :
var ig_did = driver.Manage().Cookies.GetCookieNamed("ig_did");
var sessionid = driver.Manage().Cookies.GetCookieNamed("sessionid");
var mid = driver.Manage().Cookies.GetCookieNamed("mid");
var ig_nrcb = driver.Manage().Cookies.GetCookieNamed("ig_nrcb");
var rur = driver.Manage().Cookies.GetCookieNamed("rur");
var csrftoken = driver.Manage().Cookies.GetCookieNamed("csrftoken");
var ds_user_id = driver.Manage().Cookies.GetCookieNamed("ds_user_id");
string ig_did_value = ig_did.ToString().Substring(0, ig_did.ToString().IndexOf(";")).Replace("ig_did=", "");
string sessionid_value = sessionid.ToString().Substring(0, sessionid.ToString().IndexOf(";")).Replace("sessionid=", "");
string mid_value = mid.ToString().Substring(0, mid.ToString().IndexOf(";")).Replace("mid=", "");
string ig_nrcb_value = ig_nrcb.ToString().Substring(0, ig_nrcb.ToString().IndexOf(";")).Replace("ig_nrcb=", "");
string rur_value = rur.ToString().Substring(0, rur.ToString().IndexOf(";")).Replace("rur=", "");
string ds_user_id_value = ds_user_id.ToString().Substring(0, ds_user_id.ToString().IndexOf(";")).Replace("ds_user_id=", "");
string csrftoken_value = csrftoken.ToString().Substring(0, csrftoken.ToString().IndexOf(";")).Replace("csrftoken=", "");
Then inject them to HttpClient and sniff them with fiddler :
var baseAddress = new Uri("https://www.instagram.com");
var cookieContainer = new CookieContainer();
using (var handler = new HttpClientHandler()
{
CookieContainer = cookieContainer,
Proxy = new WebProxy("127.0.0.1:8888", false),
UseProxy = true,
AllowAutoRedirect = true
})
using (httpclient = new HttpClient(handler) { BaseAddress = baseAddress })
{
httpclient.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.82 Safari/537.36");
httpclient.DefaultRequestHeaders.Add("X-CSRFToken", csrftoken_value);
httpclient.DefaultRequestHeaders.Add("Referer", "My_Instagram_URL");
httpclient.DefaultRequestHeaders.Add("X-IG-App-ID", Ig_app_Id_value);
httpclient.DefaultRequestHeaders.Add("Origin", "https://www.instagram.com");
httpclient.DefaultRequestHeaders.Add("Connection", "keep-alive");
httpclient.DefaultRequestHeaders.Add("X-Requested-With", "XMLHttpRequest");
httpclient.DefaultRequestHeaders.Add("Sec-Fetch-Site", "same-origin");
httpclient.DefaultRequestHeaders.Add("Sec-Fetch-Mode", "cors");
httpclient.DefaultRequestHeaders.Add("Sec-Fetch-Dest", "empty");
cookieContainer.Add(baseAddress, new System.Net.Cookie("ig_did", ig_did_value));
cookieContainer.Add(baseAddress, new System.Net.Cookie("mid", mid_value));
cookieContainer.Add(baseAddress, new System.Net.Cookie("ig_nrcb", ig_nrcb_value));
cookieContainer.Add(baseAddress, new System.Net.Cookie("csrftoken", csrftoken_value));
cookieContainer.Add(baseAddress, new System.Net.Cookie("sessionid", sessionid_value));
cookieContainer.Add(baseAddress, new System.Net.Cookie("rur", rur_value));
string url = "My_Instagram_URL";
var response = await httpclient.GetAsync(url);
}
As said it's looks long approach but this is will always work.
Good Luck.

POST Json web API C#

I am trying post api through HTTP client in C#, For authorization ,we send Bearer token.
But it is not getting inserted in the client side. It throws an error.
My code here:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3 | SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;
string result = "";
using (var client = new HttpClient(new HttpClientHandler() { UseDefaultCredentials = true }))
{
client.BaseAddress = new Uri(baseURL);
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Add("Cookie", "ebpPermHash=-396055074);
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.Add("Authorization", "Bearer " + authtoken);
client.DefaultRequestHeaders.Add("Accept-Encoding", "gzip, deflate");
client.DefaultRequestHeaders.Add("Accept-Language", "en-GB,en-US;q=0.9,en;q=0.8");
client.DefaultRequestHeaders.Add("Cache-Control", "no-cache")
client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36");
client.DefaultRequestHeaders.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8");
HttpResponseMessage response = client.PostAsJsonAsync("/v2/bb/sa", payModel).Result;
var responseString = response.Content.ReadAsStringAsync();
result = responseString.Result;
dynamic data = JObject.Parse(result);
string throwresult = Convert.ToString(data);
result = data.authToken;
logErr.ControlLog(throwresult);
}
return result;
But I am getting error like:
exceptionType": "com.ebpsource.exception.AuthenticationFailedException",
"name": "AUTHENTICATION_REQUIRED",
"message": "Authentication is required to call 'Resource.execute'",
"address": "/service/Resource.execute",
"httpRequestBody": {
"header": {
"config": {
"action": "insert",
"resourceName": "sa"
}
Thanks in advance.

I suggest you check your backend with Postman first, to verify if it is a backend issue or a client issue

you have a bug, a wrong syntax to add a token. And plus to many headers. Try this
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
var contentType = new MediaTypeWithQualityHeaderValue("application/json");
client.DefaultRequestHeaders.Accept.Add(contentType);
client.BaseAddress = new Uri(baseAddress);
var jsonData = JsonConvert.SerializeObject(payModel);
var contentData = new StringContent(jsonData, Encoding.UTF8, "application/json");
var response = client.PostAsync("/v2/bb/sa", contentData).Result;
if (response.IsSuccessStatusCode)
{
var stringData = response.Content.ReadAsStringAsync().Result;
var result = JsonConvert.DeserializeObject<object>(stringData);
}
}
UPDATE
if you need to add cookies you can do it this way
var cookieContainer = new CookieContainer();
using (var handler = new HttpClientHandler() { CookieContainer = cookieContainer })
using (var client = new HttpClient(handler))
{
....
cookieContainer.Add(baseAddress, new Cookie("CookieName", "cookie_value"));
.....
}

Trying to get authorization token from url works in browser, but not in WebRequest

I'm working on setting up an authorized restful request and I'm having a hell of a time getting a valid response back. If I paste the request URL into a browser(Firefox Quantum and Chrome) I can get a response of Status:Authenticated;token:[token string] but when I try WebRequest.Create([url]) I keep getting a response of "400: bad request". I'm copying the URL straight from debug code so I know it's valid. I'm pretty sure I'm doing something simple wrong. Would someone point me in the right direction?
string loginReq = _authPath + "?user=" + _username + "&pw=" + _password;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(loginReq);
request.Accept = "text/html, application/xhtml + xml, */*";
request.UserAgent = "Mozilla/5.0(Windows NT 6.1; WOW64; Trident/7.0; rv: 11.0) like Gecko";
request.KeepAlive = true;
WebResponse response = request.GetResponse();
Console.WriteLine(response.ResponseUri);
Console.Read();
Ok, after doing some more poking it looks like the site I'm calling is refusing the request because it thinks I'm using IE9. Here's the latest version of the code
private static string GetAuthorization() {
string token = string.Empty;
string loginReq = _authPath + "?user=" + _ftpUsername + "&pw=" + _ftpPassword;
string task = SendWebRequest(loginReq);
//HttpWebRequest request = (HttpWebRequest)WebRequest.Create(loginReq);
//request.Accept = "text/html, application/xhtml + xml, */*";
//request.UserAgent = "Mozilla/5.0(Windows NT 6.1; WOW64; Trident/7.0; rv: 11.0) like Gecko";
//request.KeepAlive = true;
//request.Headers.Add("Accept-Encoding", "gzip, deflate");
//request.Headers.Add("Cache-Control", "no-cache");
//request.Headers.Add("Accept-Language", "en-US");
//request.Headers.Add("DNT", "1");
//request.Method = "GET";
//request.CookieContainer = new CookieContainer();
//request.Headers.Add("Request", "GET /xc2/QAPI_Upload?user=user#ottrtest1.com&pw=ETqDJeQ1! HTTP/1.1");
//HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Console.WriteLine(task);
Console.Read();
return token;
}
public static string SendWebRequest(string requestUrl) {
using (HttpClient client = new HttpClient())
using (HttpResponseMessage response = client.GetAsync(requestUrl).GetAwaiter().GetResult())
return response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
}
I keep trying different things but I'm getting the same results(400 Bad Request) Here's the latest version of what I'm working with
string loginReq = $"{_authPath}?user={_userName}&pw={_passWord}";
string result;
using (WebClient wc = new WebClient()) {
var json = wc.DownloadString(loginReq);
result = json.ToString();
Console.WriteLine(json);
}
If I change the url to "https://www.google.com" my code works. if I paste loginReq into SoapUI it works, I can't get the url and my code to work together...
Fiddler found the problem. Once I reviewed the request in fiddler I saw that I needed to set the security protocol type to tls1.0, tls1.1, or tls1.2. Once I did that I finally got the call to work. Here's the working code in case anyone needs it for reference:
ServicePointManager.SecurityProtocol = (SecurityProtocolType)192 | (SecurityProtocolType)768 | (SecurityProtocolType)3072;
string loginReq = $"{_authPath}?user={_userName}&pw={_passWord}";
string result;
using (WebClient wc = new WebClient()) {
var json = wc.DownloadString(loginReq);
result = json.ToString();
Console.WriteLine(json);
}
return result;

Fiddler found the problem. Once I reviewed the request in fiddler I saw that I needed to set the security protocol type to tls1.0, tls1.1, or tls1.2. Once I did that I finally got the call to work. Here's the working code in case anyone needs it for reference:
ServicePointManager.SecurityProtocol = (SecurityProtocolType)192 | (SecurityProtocolType)768 | (SecurityProtocolType)3072;
string loginReq = $"{_authPath}?user={_userName}&pw={_passWord}";
string result;
using (WebClient wc = new WebClient()) {
var json = wc.DownloadString(loginReq);
result = json.ToString();
Console.WriteLine(json);
}
return result;

Calling SharePoint with HttpClient PostAsync() results into forbidden response

I am trying to send HttpClient PostAsync() request to company's internal sharepoint site but its returning response with forbidden error. I have all necessary access permission for site to load and have also passed required headers to the HttpClient object.
Here is code snippet.
HttpClient client = new System.Net.Http.HttpClient (new HttpClientHandler { UseDefaultCredentials = true });
client.BaseAddress = new Uri (string.Format (API_URL, p_siteNumber));
client.DefaultRequestHeaders.Accept.Add (new MediaTypeWithQualityHeaderValue (#"application/atom+xml"));
client.DefaultRequestHeaders.TryAddWithoutValidation ("Accept-Encoding", "gzip, deflate");
client.DefaultRequestHeaders.TryAddWithoutValidation ("Accept-Language", "en-US, en;q=0.8, hi;q=0.6");
client.DefaultRequestHeaders.TryAddWithoutValidation ("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0");
client.DefaultRequestHeaders.TryAddWithoutValidation ("Accept-Charset", "ISO-8859-1");
HttpResponseMessage httpResponse = await client.PostAsync (urlHttpPost, new StringContent (string.Empty));
string response = await httpResponse.Content.ReadAsStringAsync ();
Can anyone help me with this?
Thanks in advance.

I ran into the same problem I wanted to send the file and some string contents with it.
so below code helped me!!
using (var client = new HttpClient())
{
//client.DefaultRequestHeaders.Add("User-Agent", "CBS Brightcove API Service");
string authorization = GenerateBase64();
client.DefaultRequestHeaders.Add("Authorization", authorization);
using (var content = new MultipartFormDataContent())
{
string fileName = Path.GetFileName(textBox1.Text);
//Content-Disposition: form-data; name="json"
var stringContent = new StringContent(InstancePropertyObject);
stringContent.Headers.Remove("Content-Type");
stringContent.Headers.Add("Content-Type", "application/json");
stringContent.Headers.Add("Content-Disposition", "form-data; name=\"instance\"");
content.Add(stringContent, "instance");
var fileContent = new ByteArrayContent(filecontent);
fileContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
{
FileName = fileName
};
content.Add(fileContent);
var result = client.PostAsync(targetURL, content).Result;
}
}

Read cookie in WPF application

in a wpf application I have to call an external rest service which returns a cookie with session id in it. In all subsequent calls I have to send session id in cookie otherwise it does not return any data.
So how can I retrieve the cookie in wpf code behind class?
CookieContainer cookies = new CookieContainer();
HttpClientHandler handler = new HttpClientHandler();
handler.CookieContainer = cookies;
HttpClient client = new HttpClient(handler);
var domain = EndPoint;
HttpResponseMessage response2 = client.PostAsync(domain, new StringContent(parameters)).Result;
Uri uri = new Uri(domain);
IEnumerable<Cookie> responseCookies = cookies.GetCookies(uri).Cast<Cookie>();
var cookieWithId = responseCookies.Single(o => o.Name == "JESSSIONID");
where EndPoint has http://mydomain.com:38080/workshop/ and parameters has rest/login?username=usr&password=pwd

Here's an example on how to read cookies from response.
CookieContainer cookies = new CookieContainer();
HttpClientHandler handler = new HttpClientHandler();
handler.CookieContainer = cookies;
HttpClient client = new HttpClient(handler);
var domain = "http://yourServiceURL.com";
HttpResponseMessage response = client.GetAsync(domain).Result;
Uri uri = new Uri(domain);
IEnumerable<Cookie> responseCookies = cookies.GetCookies(uri).Cast<Cookie>();
var cookieWithId = responseCookies.Single(o => o.Name == "SessionId");