I'm trying to get ASP.NET_SessionId from a HttpWebResponse but it seems that no such data comes on the response.
Basically I'm trying to simulate some steps over some pages, where authentication is required. The problem is not in the authentication, my problem is to get ASP.NET_SessionId after the authentication so I can use it in my future requests/steps.
From Chrome on developer tools > network, I can see the ASP.NET_SessionId on the headers, but it does not come in my HttpWebResponse. Any ideia why this is happening?
There is my code:
var httpWebRequest = (HttpWebRequest) WebRequest.Create(url);
httpWebRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.93 Safari/537.36"; httpWebRequest.Method = "POST";
httpWebRequest.ContentType = "application/x-www-form-urlencoded";
httpWebRequest.ContentLength = 0;
var httpWebResponse = (HttpWebResponse) httpWebRequest.GetResponse();
After my request I should see a ASP.NET_SessionId header Set-Cookie, but no luck. Any ideia?
I've seen some people say that
httpWebResponse.Headers["ASP.NET_SessionId"]
or
httpWebResponse.Headers["SESSION_ID"]
should work but no, no session id header is set nor any Cookie.
After many research, the answer was here
Basically we have to keep the same CookieContainer object reference across all requests. I was extracting some Set-Cookie from the responses and adding them into my requests, but now I don't need to do anything, CookieContainer manages all of it transparently.
Set-Cookie from responses are set on CookieContainer of your request. It's the way they found to resolve possible security issues, so don't lose more time and just keep a reference to your CookieContainer because you will not be able to access session id (and you don't need it).
There's the example of my code now.
var cookieContainer = new CookieContainer();
var httpWebRequest1 = (HttpWebRequest) WebRequest.Create(url);
httpWebRequest1.CookieContainer = cookieContainer;
// do the request and some logic
var httpWebRequest2 = (HttpWebRequest) WebRequest.Create(anotherUrl);
httpWebRequest2.CookieContainer = cookieContainer; // same cookieContainer reference
Everything is working great now, hope it helps someone.
Related
I am posting the following JSON to my Flask server:
{"comment": "astute observation", "ua": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36", "cookie": "ua=f8256d37159e3faf28ae61a6406601c3; platform=pc; bs=bxa7ziiq07dufk31prvoo0mbfm44sayh; ss=994139249854417186; il=v1qyca8PU7X6jSwgiqceXsySwWB60HnCjSJ1HjNmsSxRUxNjQzNjA3OTEycUVTTFoxWUpSZDFTZ3kwclIzenBHa19FbnlmMTlIN0hZeHNDQ1FOLQ..; expiredEnterModalShown=1", "parent": "860245961"}
When sent from Postman, the request works flawlessly. However, when sent from my C# app, like so, the server returns a 400 Bad Request error code.
var data = $"{{\"comment\": \"{text}\", \"ua\": \"{userAgent}\", \"cookie\": \"{cookie}\", \"parent\": \"{parent}\"}}";
var url = "http://127.0.0.1:5000/";
var request = HttpWebRequest.CreateHttp(url);
request.Method = "POST";
request.ContentType = "application/json";
await using var sw = new StreamWriter(request.GetRequestStream());
await sw.WriteAsync(data);
var response = await request.GetResponseAsync();
To make sure that the JSON is properly formatted, I set a breakpoint and inspected the "data" variable. I copied/pasted that value into Postman, set the Content-Type to application/json, and the request succeeds from there, but fails from my C# application.
Server code:
from flask import Flask, request
app = Flask(__name__)
#app.route('/', methods=['GET', 'POST'])
def process():
print(request.json)
return 'Hello'
app.run(debug=True)
Edit: After posting this, I realized I wasn't sending a user agent from my C# app. I tried adding one and the server still responds with a 400 Bad Request error.
Edit 2: I tried simplifying the data variable for testing purposes. var data = "{\"msg\": \"david\"}"; and setting request.ContentLength = data.Length, still having the same problem. Really strange.
Very odd, but this solved the problem. Using the StreamWriter class was causing the issue. Doing this instead results in a 200 OK response finally.
using var stream = request.GetRequestStream();
await stream.WriteAsync(Encoding.UTF8.GetBytes(data));
So i am just getting to learn HttpWebRequests and it's functions.
I've gotten to the point where I want to learn how to capture cookies in a CookieContainer and parse through them.
The issue is that some websites return a 503 error and I am not sure.
One of the websites will be used in this example.
From what Iäve read online a 503 error is this.
The HyperText Transfer Protocol (HTTP) 503 Service Unavailable server
error response code indicates that the server is not ready to handle
the request.
Common causes are a server that is down for maintenance or that is
overloaded. This response should be used for temporary conditions and
the Retry-After HTTP header should, if possible, contain the estimated
time for the recovery of the service.
Which doesnt seem to fit in at all since the website is up and running.
Why is my request returning a 503 status code and what should I do to resolve this issue in a propper manner?
static void Main(string[] args)
{
//1. Create a HTTP REQUEST
//Build the request
Uri site = new Uri("https://ucp.nordvpn.com/login/");
//Inizializing a new instance of the HttpWebRequest and casting it as a WebRequest
//And calling the Create function and using our site as a paramter which the Create function takes.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(site);
//Inizialize a new instance of the CookieContainer
CookieContainer cookies = new CookieContainer();
//The request has a CookieContainer, which is null by default, so we are just assinging the newly inizialized instance
//of our CookieContainer to our requests CookieContainer
request.CookieContainer = cookies;
//Print out the number of cookies before the response (of course it will be blank)
Console.WriteLine(cookies.GetCookieHeader(site));
//Get the response and print out the cookies again
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
Console.WriteLine(cookies.GetCookieHeader(site));
}
Console.ReadKey();
}
The URL that you are trying to get to appears to be protected by CloudFlare. You can't use the basic HttpWebRequest for that type of request without some additional work. While I haven't tried this, it may be an option for you to get around that protection:
CloudFlareUtilities
The url you are trying to access is using cloud hosting which use many security measurement including which browser are accessing the site
for that to work you need to change the userAgent property of HttpWebRequest
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0";
I'm having trouble getting a WebRequest to pass authentication when trying to download a file from a URL. I have verified that, hitting the url directly and being prompted with a username / password form, my credentials go through just fine and the file is downloaded to my computer.
However, when using a WebRequest (with a proxy) i'm getting a (401) Authentication failed.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Credentials = new NetworkCredential(username, password);
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
WebProxy proxy = new WebProxy("myProxyAddress:myProxyPort");
proxy.UseDefaultCredentials = false;
proxy.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;
request.Proxy = proxy;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
It's definitely passing proxy auth as i was a getting a 407 before. I've tried a myriad of settings such as setting PreAuthenticate to true, UseDefaultCredentials to false etc but to no avail.
Is there something i'm missing here? Could it be down to the type of auth the 3rd party url is requiring?
Trying To Loading Html Content Of "http://links.casemakerlegal.com/states/CA/books/Case_Law/results?search[Cite]=214 Cal.App.3d 533" but HttpWebResponse object Giving This Error "(500) Internal Server Error"
And Code Is------
request = WebRequest.Create(urlCheck); request.Timeout = 100000; response = request.GetResponse(); strmRead = new StreamReader(response.GetResponseStream(),System.Text.Encoding.UTF8); result = strmRead.ReadToEnd();
You need to use a tool like Wireshark or Ethereal, or the developer tools in your browser to investigate this further. It is likely the browser is sending some values in the HTTP Header that your code is not, and the server is returning a 500 due to these missing values. Try replicating all of the headers that the browser is using in your code to see if this resolves the problem.
It is usually browser agent. try adding a valid browser agent to your request headers along with Accept and Accept-Encoding headers,
*Edit: For example:
request.UserAgent = "Mozilla/5.0 (Windows NT 5.2; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1";
request.Headers.Add("Accept-Encoding: gzip, deflate");
request.Headers.Add("Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
request.Headers.Add("Accept-Language: en;q=0.8");
I need help with cookies. I'm planing use cookies to download web content. To get the content I need to log into a website because only authorized users can download web content or files. I'm using
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
and then I'm scraping source code, and I need to get link to file, but I can't get because I'm not authorized, so I need to set cookies. I have not worked with cookies before.
How do I to do this?
If you are talking about creating a asp.net web app that can create cookies, then doing this should work:
Not sure if this would work in asp.net mvc though, this works in asp.net web forms
Response.Cookies["nameOfCookie"].Value = "someValue";
Response.Cookies["nameOfCookies].Expires = DateTime.Now.AddDays(1);
then on a post back on say same page or different page, can get cookie by
string value = string.Empty;
if (Request.Cookies["nameOfCookie"] != null)
value = Request.Cookies["nameOfCookie"].Value;
I've created a quick little application that helps with generating web requests for me
public class HttpRequestHandler {
private CookieContainer cookies;
public HttpRequestHandler() {
cookies = new CookieContainer();
}
public HttpWebRequest GenerateWebRequest(string url) {
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(new System.Uri(url));
request.CookieContainer = cookies;
request.AllowAutoRedirect = true;
request.KeepAlive = true;
request.Referer = HttpUtility.UrlEncode(referer);
request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)";
request.Headers.Add("Pragma", "no-cache");
request.Timeout = 40000;
return request;
}
}
Your problem is probably related to lack of a CookieContainer. If you create a cookie container you can save/use cookies in your web requests.
You should set the CookieContainer property of the HTTPWebRequest class to an instance of a CookieContainer class. From MSDN, it's stated that:
CookieContainer is null by default. You must assign a CookieContainer object to the property to have cookies returned in the Cookies property of the HttpWebResponse returned by the GetResponse method.
In other words, after you have set the CookieContainer property of the HTTPWebRequest object in your code, you can get the corresponding Cookies in the HTTPWebResponse object in your code. The sample code in the MSDN link shared above should get you started.
Could you be more speciffic about the project? is it a Desktop Application, ASP.NET, ASP.NET MVC?
in ASP.NET MVC I used SetPersistenCookie method every time when user is logging in, and for methods I use Authorize Attribute, every time when a user wants to acces somenthig and he is not logged in it redirects him to LogIn page...