The title explains it mostly. I have declared my HttpClient, HttpClientHandler, and CookieContainer as class variables.
private HttpClient client;
private HttpClientHandler handler;
private CookieContainer cookies;
Then in the form creation I initialize the variables like so
public FrmMain()
{
InitializeComponent();
handler = new HttpClientHandler();
cookies = new CookieContainer();
handler.AllowAutoRedirect = true;
handler.UseCookies = true;
handler.CookieContainer = cookies;
client = new HttpClient(handler);
client.DefaultRequestHeaders.Connection.Clear();
client.DefaultRequestHeaders.ConnectionClose = false;
client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36");
}
Later on in the program, when I call the requests, I am able to log in to the url (in this case a device on my local net) just fine. As part of the troubleshooting for this, I started printing the cookie data to the console each time a request is made. When I initially log on, it gives me single cookie, a sessionID. Any subsequent requests that I make using the same client gives me a new sessionID. This causes my requests to get a return code of badRequest, most likely because it is trying to route me back to the login page. I know that I am successfully logging in with the first request because printing the response content gives me the HTML of the index page that I am redirected to upon a successful login. I've tested all the data I'm sending via Postman, where I'm able to do a log in request, then do whatever other requests I need without issue. The only difference between Postman and my program is that in my program I am getting a new sessionID for every request instead of it persisting. Anyone know why my cookies are not persisting despite the client handler, client, and cookie container all being declared in the class scope?
It turns out my issue was not a cookie issue. The cookie was supposed to change to a new sessionID after logging in. The sessionID never changes after that. HttpClient was saving the cookie persistently. The issue was with a hidden CSRFToken that I was submitting with the formdata. That changes with each request and while I was doing the steps to get the new one before each POST, I was not actually assigning it.
I'd like to thank Jonathan. If I hadn't jumped back in to try some tweaks to make sure the program was only loading the initializations once, I probably wouldn't be looking in the area where I was neglecting to assign the new CSRFToken.
Related
so basically i started learning http request with restsharp and to be honest there is a total confusion.
So i managed to log into my reddit account, catch upvote post request and i tried to post the method and it actually worked on the active session.(yayy after a week...)
Here is my code:
var client = new RestClient("https://www.reddit.com/")
CookieContainer cookieContainer = new CookieContainer();
var options = new RestClientOptions()
{
MaxTimeout = -1
};
var request = new RestRequest("https://oauth.reddit.com/api/vote?redditWebClient=desktop2x&app=desktop2x-client-production&raw_json=1&gilding_detail=1", Method.Post);
request.AddHeader("accept", "*/*");
request.AddHeader("accept-language", "en-US,en;q=0.9");
request.AddHeader("authorization", "Bearer 1909307768910-Y9YshL45Wi1uTiaXGIDzIYS1CvawRw");
request.AddHeader("content-type", "application/x-www-form-urlencoded");
request.AddHeader("origin", "https://www.reddit.com");
request.AddHeader("referer", "https://www.reddit.com/");
request.AddHeader("sec-ch-ua", "\".Not/A)Brand\";v=\"99\", \"Google Chrome\";v=\"103\", \"Chromium\";v=\"103\"");
request.AddHeader("sec-ch-ua-mobile", "?0");
request.AddHeader("sec-ch-ua-platform", "\"Windows\"");
request.AddHeader("sec-fetch-dest", "empty");
request.AddHeader("sec-fetch-mode", "cors");
request.AddHeader("sec-fetch-site", "same-site");
request.AddHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36");
request.AddHeader("x-reddit-loid", "0000000000od4gjr66.2.1654778423000.Z0FBQUFBQmlzX05MUTZyNUxqOW4wcnNLNzhiNjFWeG5kR21saVptZXl6RllTcmVuMTJLV2hneEVBU3Z6SDB0VUJOQzJORzE3d0R2aWRHT1YxWFN2akdmbjg4T2Z2RVBUaWR4VjNHRk1paEFaUU9HblQydDBIbXFXZjY3WDAyaS10dGRRaDU0aWFoS2I");
request.AddHeader("x-reddit-session", "infdgnamhcklpjejeb.0.1656153357171.Z0FBQUFBQml0dVVORWozUVZwQ2RqTGdMc3MwbGJQczJvWW1UVUdmdjVHVjJqMElFV2ZwUlhOaGxmSlh0Wm02aDZsWmx2a0NtVXJKNHEyRG1Hc25tQ0FKMVdlbGlPcDN3aGVXdS1xaXJJRU9KMGVKRFQ1ZXJHTEFzSXpqSm1RUmFIa0dCNFBjU2tnYlo");
request.AddHeader("Cookie", "csv=2; edgebucket=EzrHckI9fWuza20bDr; loid=0000000000p5fofqq6.2.1656153609022.Z0FBQUFBQml0dVlKM2FDVUo5NUN2eEZBRDk4NDV5WWtDNk1mU01vRmktRnB1eWYwMzY4X3ZCdkx1eXZUNERzTVN1MkhDZXdnZFpHblMwU0k4LTc4M1VnNGRkQ3hOQUp3dTVxaFZYbndFQ19WN3NUaUFsX21MaE1SNTRaQWRsQU1sZFBUTDJQWlBoRnM; session_tracker=zoCNAuHADuPm2Hy4AF.0.1656153616293.Z0FBQUFBQml0dVlRRTZCc2NBNHRmV2xxUWhwVFllT0JaUGhDN3JDNTYxRk1DLURsTmJ4U1ZVOUNLRjN4RjZHNExpZUZjcVd6MzktMGVZMS12X193SDJ2OUd0R19kVjJ3dEtnWGFPOEVHSWd5dHBIWFVZcWNReEZPb0Q5UG9mRVlyWUFXOC1lMEYxZ20");
request.AddParameter("id", "t3_qbl6b2");
request.AddParameter("id", "t3_vk7gmb");
request.AddParameter("dir", "1");
request.AddParameter("api_type", "json");
RestResponse response = client.Execute(request);
Console.WriteLine(response.Content);
Ok the thing here is im not really interested into official reddit api or any official api as im only trying to learn and i have no precise goal. I have some questions that i need to be precised. I found some explanations on youtube, here or google but its not really clear and i seen that restsharp have updated recently.
1- So far what i understand, its that i can do this post request because the active session is using the same Bearer and cookies but i guess intercepting the post request each time to generate these data aint really practical so how you actually get the session id/loid/cookies/bearer and all dynamic data that you need to make succesful request before actually doing the request(logic lol)?
2- How to handle cookies?
3- Do you need to make a get request before a post?
4- How to handle objects and data from the response?
I want to learn so if you guys have some knowledge to share, it will be appreciated!
Ok so it was totally not about oauth at all.
-So basically, you need to make a GET request that will deliver an html code.
-In this html code you will find an csrf token. You will use regex to put it in a string.
-You will create a POST login request that return username, password and csrf token.
You're now logged!!!
About handling cookies, restsharp do it automatically as long as you keep the same RestClient object. I put them in a string array and use them in the request using String.Format
So i am just getting to learn HttpWebRequests and it's functions.
I've gotten to the point where I want to learn how to capture cookies in a CookieContainer and parse through them.
The issue is that some websites return a 503 error and I am not sure.
One of the websites will be used in this example.
From what Iäve read online a 503 error is this.
The HyperText Transfer Protocol (HTTP) 503 Service Unavailable server
error response code indicates that the server is not ready to handle
the request.
Common causes are a server that is down for maintenance or that is
overloaded. This response should be used for temporary conditions and
the Retry-After HTTP header should, if possible, contain the estimated
time for the recovery of the service.
Which doesnt seem to fit in at all since the website is up and running.
Why is my request returning a 503 status code and what should I do to resolve this issue in a propper manner?
static void Main(string[] args)
{
//1. Create a HTTP REQUEST
//Build the request
Uri site = new Uri("https://ucp.nordvpn.com/login/");
//Inizializing a new instance of the HttpWebRequest and casting it as a WebRequest
//And calling the Create function and using our site as a paramter which the Create function takes.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(site);
//Inizialize a new instance of the CookieContainer
CookieContainer cookies = new CookieContainer();
//The request has a CookieContainer, which is null by default, so we are just assinging the newly inizialized instance
//of our CookieContainer to our requests CookieContainer
request.CookieContainer = cookies;
//Print out the number of cookies before the response (of course it will be blank)
Console.WriteLine(cookies.GetCookieHeader(site));
//Get the response and print out the cookies again
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
Console.WriteLine(cookies.GetCookieHeader(site));
}
Console.ReadKey();
}
The URL that you are trying to get to appears to be protected by CloudFlare. You can't use the basic HttpWebRequest for that type of request without some additional work. While I haven't tried this, it may be an option for you to get around that protection:
CloudFlareUtilities
The url you are trying to access is using cloud hosting which use many security measurement including which browser are accessing the site
for that to work you need to change the userAgent property of HttpWebRequest
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0";
I have rather strange problem. With all that questions over internet how to add and get cookies, I want the opposite ;)
When I try to send request via HttpHandler it is adding it's own Cookie header. I have to get rid of it. Without digging into details - when it is added, server I am trying to request is giving me wrong answer. It works without this cookie (tried in fiddler).
But back to the problem, code:
string domain = "someMysteriousDomain";
var handler = new HttpClientHandler();
handler.UseDefaultCredentials = false;
handler.AllowAutoRedirect = true;
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.UseCookies = false;
var httpClient = new HttpClient(handler);
var request = new HttpRequestMessage(HttpMethod.Get, domain);
request.Headers.UserAgent.Add(new ProductInfoHeaderValue("Mozilla", "5.0"));
request.Headers.AcceptEncoding.Add(new StringWithQualityHeaderValue("gzip"));
var response = await httpClient.SendAsync(request);
Raw request seen in fiddler:
GET https://domain HTTP/1.1
Accept-Encoding: gzip
User-Agent: Mozilla/5.0
Host: domain
Connection: Keep-Alive
Cookie: cadata477E7C1824F44800AF0077724F65345="51595d316-0286-44bb-bc6f-ffb1fd311a92SqJA36rA69YW7aBg+iHXYi9LAcBLN6DBWE8a3MLejd2VCluO/UQ5eF6F6T4NWh4NhdRcv4rea15Hs0e2q6GatMac59UVbljhREYdH6PRbzZC/2qn8QHtpc6go5B56R"; mobile=0
I don't want to add that cookie! How to delete/clear/whatever it?
I am using Visual Studio Community 2015, with Windows Universal Project.
What is interesting, after rebooting my pc after few hours, I was able to make 2 or 3 requests without this cookie (using THE SAME code) and then mysterios cookie returned.
What it is about? How to get rid of it?
Thank you for reporting this issue - this is a known issue with the System.Net.Http.HttpClientHandler API implementation on Windows 10 and we are working on fixing it in an upcoming release.
In the meanwhile, a possible workaround is to use the Windows.Web.Http.HttpClient API with the underlying HttpBaseProtocolFilter class. This class has a property called CookieManager that stores all the cookies for each URI. You can write a method to delete the cookies from the CookieManager for the destination URI before sending a request. This will ensure that no cookies get sent. You can see this sample for how to delete cookies from the CookieManager: https://github.com/Microsoft/Windows-universal-samples/tree/master/httpclient
Thanks
Sidharth [MSFT]
i want to make a web request while rearrange cookies in c#.
I'm handling cookies with
public CookieContainer _cookies = new CookieContainer();
when i look at the _cookies element i see that non-public members -> domain=.local
I want to change .local variable.
becuase i think website gets authenticated with this value.
i tried to add this cookie but website gives me 401 error
request.CookieContainer.Add(new Uri(url), new Cookie("Domain", "bb.local"));
how can i manage cookies
I need help with cookies. I'm planing use cookies to download web content. To get the content I need to log into a website because only authorized users can download web content or files. I'm using
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
and then I'm scraping source code, and I need to get link to file, but I can't get because I'm not authorized, so I need to set cookies. I have not worked with cookies before.
How do I to do this?
If you are talking about creating a asp.net web app that can create cookies, then doing this should work:
Not sure if this would work in asp.net mvc though, this works in asp.net web forms
Response.Cookies["nameOfCookie"].Value = "someValue";
Response.Cookies["nameOfCookies].Expires = DateTime.Now.AddDays(1);
then on a post back on say same page or different page, can get cookie by
string value = string.Empty;
if (Request.Cookies["nameOfCookie"] != null)
value = Request.Cookies["nameOfCookie"].Value;
I've created a quick little application that helps with generating web requests for me
public class HttpRequestHandler {
private CookieContainer cookies;
public HttpRequestHandler() {
cookies = new CookieContainer();
}
public HttpWebRequest GenerateWebRequest(string url) {
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new System.Uri(url));
request.CookieContainer = cookies;
request.AllowAutoRedirect = true;
request.KeepAlive = true;
request.Referer = HttpUtility.UrlEncode(referer);
request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)";
request.Headers.Add("Pragma", "no-cache");
request.Timeout = 40000;
return request;
}
}
Your problem is probably related to lack of a CookieContainer. If you create a cookie container you can save/use cookies in your web requests.
You should set the CookieContainer property of the HTTPWebRequest class to an instance of a CookieContainer class. From MSDN, it's stated that:
CookieContainer is null by default. You must assign a CookieContainer object to the property to have cookies returned in the Cookies property of the HttpWebResponse returned by the GetResponse method.
In other words, after you have set the CookieContainer property of the HTTPWebRequest object in your code, you can get the corresponding Cookies in the HTTPWebResponse object in your code. The sample code in the MSDN link shared above should get you started.
Could you be more speciffic about the project? is it a Desktop Application, ASP.NET, ASP.NET MVC?
in ASP.NET MVC I used SetPersistenCookie method every time when user is logging in, and for methods I use Authorize Attribute, every time when a user wants to acces somenthig and he is not logged in it redirects him to LogIn page...