Fiddler Website web request Mimicking for screen scraping - c#

I'm trying to log into a website to download data through my account. This is the raw Fiddler Request for the POST login form.
POST login/login.jsp HTTP/1.1
Host: server.com
Connection: keep-alive
Content-Length: 73
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Origin: https://server.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Referer: https://server.com/login/login.jsp
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Cookie: __utma=109610308.114257620.1370889472.1373479499.1371761934.3; __utmc=109613338; __utmz=109610308.1373249472.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided); _bcvm_vid_424161365915852877=4393500994580715020; _bcvm_vrid_424161365915852877=4393492275825713189; WT_FPC=id=199.234.233.42-2645888112.30303753:lv=1371356395815:ss=1371758333825; JSESSIONID=RGJGy4yQ2WCXRPbnhxCTKGb2rZh39b67d8g8PktTQLqfsBQTlTlYLTD!1154156211; BIGipServeresuite_prod_pool=295635768.2713643.0000
It then responds with:
HTTP/1.1 302 Moved Temporarily
Date: Fri, 21 Jun 2013 12:39:46 GMT
Location: https://server.com/login/redirect.jsp?APPLICATION=0
Content-Type: text/html
Set-Cookie: SECURITY_SESSION_ID=383826514*198399234219875960; domain=.server.com; path=/
Connection: Close
Set-Cookie: BIGipServeresuite_prod_pool=294168768.27163.0000; expires=Fri, 21-Jun-2013 13:09:47 GMT; path=/
Content-Length: 3669
That SECURITY_SESSION_ID is what's needed to do anything on the site.
To mimic it I coded wrote this:
//GET the Login page - I preform a quick get to pick up the first two important cookies
HttpWebRequest GETLoginRequest = (HttpWebRequest)HttpWebRequest.Create("https://server.com/login/login.jsp");
GETLoginRequest.Method = "GET";
GETLoginRequest.Accept = "application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*";
GETLoginRequest.AllowAutoRedirect = false;
GETLoginRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)";
GETLoginRequest.CookieContainer = cookieJar;
HttpWebResponse GETLoginResponse = (HttpWebResponse)GETLoginRequest.GetResponse(); //Gets the JSession and BIGipServer cookies
Console.Write(" \n 3rd count after GETLoginResponse : " + cookieJar.Count + "\n");
//POST Login
HttpWebRequest POSTLoginRequest = (HttpWebRequest)HttpWebRequest.Create("https://server.com/login/login.jsp");
POSTLoginRequest.Method = "POST";
WebHeaderCollection myWebHeaderCollection = POSTLoginRequest.Headers;
POSTLoginRequest.AllowAutoRedirect = true;
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
///Cache
POSTLoginRequest.Headers.Add(HttpRequestHeader.CacheControl, "max-age=0");
//Client
POSTLoginRequest.Accept = "application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*";
POSTLoginRequest.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate,sdch");
myWebHeaderCollection.Add("Accept-Language:en-US");
POSTLoginRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)";
//Cookies/Login
POSTLoginRequest.CookieContainer = cookieJar; //The cookie jar contains mainly added
//Entity
POSTLoginRequest.ContentLength = bytes.Length;
POSTLoginRequest.ContentType = "Content-Type: application/x-www-form-urlencoded";
//Miscellanous
POSTLoginRequest.Headers.Add("Origin: https://server.com");
POSTLoginRequest.Referer = "https://server.com/login/login.jsp";
//Transport
//Fix I found to allow Connection: Keep-Alive
var sp = POSTLoginRequest.ServicePoint;
var prop = sp.GetType().GetProperty("HttpBehaviour", BindingFlags.Instance | BindingFlags.NonPublic);
prop.SetValue(sp, (byte)0, null);
ServicePointManager.Expect100Continue = false;
POSTLoginRequest.Host = "server.com";
using (Stream os = POSTLoginRequest.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
HttpWebResponse POSTLoginResponse = (HttpWebResponse)POSTLoginRequest.GetResponse();
Console.Write(" \n 4th count after POSTLoginResponse : " + cookieJar.Count + "\n");
at the end of the day my request in fiddler looks like this:
POST /login/login.jsp HTTP/1.1
Cache-Control: max-age=0
Accept: application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)
Content-Type: Content-Type: application/x-www-form-urlencoded
Origin: https://server.com
Referer: https://server.com/login/login.jsp
Host: esuite.pjm.com
Cookie: __utma=1096103034.235016339.1371048460.1371048460.1371048460.1; __utmz=109610428.1371048460.1.1.utmcsr=bing|utmccn=(organic)|utmcmd=organic|utmctr=pjm; _bcvm_vrid_424161365915852877=4393493862784729423; WT_FPC=id=199.234.233.42-3603288592.30304123:lv=1371044861062:ss=1371044859892; JSESSIONID=RGbYQd7JnPdNkTvtGCzQ9NLyFgfBnnyLFzbvKPg2Y0gLnhL2hp8F!-1770592471; BIGipServeresuite_prod_pool=327723200.27163.0000
Content-Length: 73
Connection: Keep-Alive
which is pretty much the same and yet I get this as a response:
HTTP/1.1 200 OK
Date: Fri, 21 Jun 2013 13:58:17 GMT
Content-Length: 3356
Content-Type: text/html
Set-Cookie: BIGipServeresuite_prod_pool=327723200.27163.0000; expires=Fri, 21-Jun-2013 14:28:17 GMT; path=/
I think one thing has to do with that in the browser the reply is a 302 Moved Temporarily but I really don't know. The two important cookies seem to be JSession and BIGIp because
those are set by the site. The other cookies I manually added in and they seem to be google analytic cookies and I don't think they matter all to much. Anyway the headers are nearly
identicaly but it's still not responding with the SECURITY_SESSION_ID that I'm looking for. Does anyone have any idea on what I'm doing wrong?

Your 'accept' headers are different, and your Content-Type part of the submit is wrong:
POSTLoginRequest.ContentType = "Content-Type: application/x-www-form-urlencoded";
Should be
POSTLoginRequest.ContentType = "application/x-www-form-urlencoded";

Make sure the post parameters are the same as in fiddler example:
So do this first before converting them to bytes.
HttpUtility.Urlencode(forumparameters)
See if that works.
As you are missing counts in ContentLength too, so must be cause of this.

Related

Unable to get request data in asp.net web forms

in asp.net pageload method I am trying to get request data using
Request.Form["keyname"]
Request.Query
but all are empty but when I run fiddler in inspector>>raw I am getting this
POST http://localhost:50844/Success.aspx HTTP/1.1
Host: localhost:50844
Connection: keep-alive
Content-Length: 15583
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
Origin: null
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cookie: ASP.NET_SessionId=ax0e4j1ucaghsarowzmvgoep
valuestr=abc&randomnum=99389
I need to get values of valuestr and randomnum,how can I do this
You can use the below code as per the values in your URL.
if your URl is = http://localhost:3039/test.aspx?valuestr=abc&randomnum=99389
then you can use the below code
string valuestr = Request.QueryString["valuestr"];
string randomnum= Request.QueryString["valuestr"];
Response.Write(valuestr );
Response.Write(randomnum);
do let me know in case you require any help

ASP.Net / Webforms: Missing the range header

I have a webforms application that has a download handler in an ashx file that's redirected to from an urlMapping in the web.config. This works alright, however I'd like to also support continuing downloads- which uses the Range request header. However, though the Range header is sent, it is not visible on the server.
My test request looks like this:
GET http://localhost:81/Downloads/MyFile.zip HTTP/1.1
Host: localhost:81
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36
Accept-Language: en-US,en;q=0.8,nl;q=0.6
Range: bytes=3-
However, when on the server I try read the Range header, I get nothing. The following code:
var request = HttpContext.Current.Request;
Debug.WriteLine("/" + request.HttpMethod + " " + request.RawUrl);
foreach (var key in headers.AllKeys)
{
Debug.WriteLine(key + ": " + context.Request.Headers[key]);
}
The output that gives me is:
/GET /Downloads/MyFile.zip
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.8,nl;q=0.6
Host: localhost:81
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36
Somehow, the Range header is gone and a "Content-Length: 0" header has been added.
My question is: Why is the range header being removed, how can I get it back?

HttpWebRequest Cookie Issue - (500) Internal Server Error

I've recently run into an issue which I believe is related with the CookieContainer.
It is a basic HttpWebRequest to a url that was previously working fine (2 days ago) but is now having issues.
Below is the code I am using.
string url = "http://www.footywire.com/afl/footy/ft_match_statistics?mid=5827";
var cookieJar = new CookieContainer();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.CookieContainer = cookieJar;
request.AllowAutoRedirect = false;
request.Method = "GET";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0";
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
WebHeaderCollection myWebHeaderCollection = request.Headers;
myWebHeaderCollection.Add("Accept-Encoding", "gzip, deflate");
myWebHeaderCollection.Add("Accept-Language", "en-US,en;q=0.5");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
int cookieCount = cookieJar.Count;
response.Close();
I am getting a (500) Internal Server Error during the GetResponse() call.
The URL works fine in the browser, but fails if I use an incognito/private window which makes me think it has to be cookie related.
I've monitored the request/responses using fiddler and can see the request is not handling cookies at all.
Here is a working request through the browser with cookies enabled.
GET http://www.footywire.com/afl/footy/ft_match_statistics?mid=5827 HTTP/1.1
Host: www.footywire.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: __utma=123692316.616048912.1368166198.1400452319.1400457786.25; __gads=ID=c95b71c78d8a7516:T=1368166203:S=ALNI_MbHrLnKztzLrfUEkZNoQfcimetGNw; _em_vt=3938ffa289939f5bd80fd60798185181c3303dddd6-9262208553794a3e; __utmz=123692316.1395356585.4.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmc=123692316; JSESSIONID=62B598BD99DC2D228C4FC1E1549ED6FB; __utmb=123692316.1.10.1400457786; _em_v=5cc514d9ac92155cd445ac583ffd53794a3e7088d0-4657961653794a3e
Connection: keep-alive
This is the request the code is sending. The cookie is clearly missing.
GET http://www.footywire.com/afl/footy/ft_match_statistics?mid=5827 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.5
Host: www.footywire.com
Connection: Keep-Alive
I'm at a bit of a loss as to why its not working now, but it was 2 days ago. I'm also not sure of what to do next.
Any help would be appreciated. Thanks.

C# - Connection: keep-alive Header is Not Being Sent During HttpWebRequest

I'm trying to send to send the following header with my HttpWebRequest:
Connection: keep-alive
However, the header is never sent. Fiddler2 is showing that whenever I request the page in Google Chrome, the header is sent. However, my application refuses to send this header for some reason.
I have set the KeepAlive property to true (it's true by default anyway), yet the header still does not get sent.
I am trying to send this header with multiple HttpWebRequests, but they all basically look like this:
HttpWebRequest logIn6 = (HttpWebRequest)WebRequest.Create(new Uri(responseFromLogIn5));
logIn6.CookieContainer = cookies;
logIn6.KeepAlive = true;
logIn6.Referer = "https://login.yahoo.com/config/login?.src=spt&.intl=us&.lang=en-US&.done=http://football.fantasysports.yahoo.com/";
logIn6.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.220 Safari/535.1";
logIn6.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
logIn6.Headers.Add("Accept-Encoding:gzip,deflate,sdch");
logIn6.Headers.Add("Accept-Language:en-US,en;q=0.8");
logIn6.Headers.Add("Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3");
logIn6.AllowAutoRedirect = false;
HttpWebResponse logIn6Response = (HttpWebResponse)logIn6.GetResponse();
string responseFromLogIn6 = logIn6Response.GetResponseHeader("Location");
cookies.Add(logIn6Response.Cookies);
logIn6Response.Close();
Does anyone know what I have to do to make sure this header is sent?
Fiddler2 Raw From Chrome:
GET xxx HTTP/1.1
Host: accounts.google.com
Connection: keep-alive
Referer: https://login.yahoo.com/config/login?.src=spt&.intl=us&.lang=en-US&.done=http://football.fantasysports.yahoo.com/
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.220 Safari/535.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: xxx
HTTP/1.1 302 Moved Temporarily
Set-Cookie: xxx
Set-Cookie: xxx
Location: xxx
Content-Type: text/html; charset=UTF-8
P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
Date: Sat, 17 Sep 2011 22:27:09 GMT
Expires: Sat, 17 Sep 2011 22:27:09 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Content-Length: 2176
Server: GSE
Fiddler2 Raw From My Application:
GET xxx HTTP/1.1
Referer: https://login.yahoo.com/config/login?.src=spt&.intl=us&.lang=en-US&.done=http://football.fantasysports.yahoo.com/
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.220 Safari/535.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Host: accounts.google.com
HTTP/1.1 302 Moved Temporarily
Location: xxx
Content-Type: text/html; charset=UTF-8
Date: Sun, 18 Sep 2011 00:05:40 GMT
Expires: Sun, 18 Sep 2011 00:05:40 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Content-Length: 573
Server: GSE
I'm trying to get the second Fiddler2 raw information to look like the first Fiddler2 raw information.
I've had the same issue: The Connection: Keep-Alive header is not sent except the first request, and the server I accessed won't give me the correct response if it is missing. So, here are my workarounds to this issue:
First is set the ProtocolVersion property of HttpWebRequest instance to HttpVersion.Version10. Except the http command will become GET xxx HTTP/1.0, it works and uses only the public API.
The second way uses the reflection to modify the internal property ServicePoint.HttpBehaviour of HttpWebRequest instance, like this:
var req = (HttpWebRequest)WebRequest.Create(someUrl);
var sp = req.ServicePoint;
var prop = sp.GetType().GetProperty("HttpBehaviour",
BindingFlags.Instance | BindingFlags.NonPublic);
prop.SetValue(sp, (byte)0, null);
req.GetResponse().Close();
Hope this helps.
I struggled with this problem for half a day! And dear old Fiddler (my guardian angel) was inadvertantly part of the problem:
Whenever I tested my HTTP POSTs using Fiddler monitoring ON - the problem DIDN'T appear
Whenever I tested my HTTP POSTs with Fiddler monitoring OFF - the problem DID appear
My POSTS were sent with protocol 1.1 and the Keep-Alive was ignored/redundant/why after the initial connection. i.e. I could see it in the header of the first POST (via Fiddler!), but not in subsequent POSTs despite using the same code. Hey ho ...
But the remote server would only respond if Keep-Alive was sent. Now I can't prove this, but I suspect that Fiddler monitoring the connection caused the remote server to think or believe that the connection was still active (despite no Keep-Alives sent after my first POST) and responded correctly. As I said, the second I turned Fiddler off, the absence of Keep-Alives caused the remote server to timeout on me..
I implemented the 1.0 solution described above and my POSTS now work, with or without Fiddler on or off. Hope this helps somebody else stuck somewhere ...
You doing it right. The code should result in following header added:
Connection: Keep-Alive
Post the code that you use for sending request and Raw output from Fiddler if you don't see this header. You may also ignore this because HTTP 1.1 connection is keep-alive by default.
Update: it looks like .NET only sets Keep-Alive explicitly for the first (!) request. Further requests to the same host/url will not have this header presumably because underlying tcp connection is already being reused.
After downloading HttpWebRequest source code, noticed that every property checks some known headers for HeaderCollection. To get rid of that doing some reflection stuff on that collection make it work
var webRequest = (HttpWebRequest) WebRequest.Create(url);
webRequest.Headers.GetType().InvokeMember("ChangeInternal",
BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.InvokeMethod,
Type.DefaultBinder, webRequest.Headers, new object[] {name, value}
);
I know the answer for this as I had the same problem and managed to solve it by inheriting the webclient and overriding it's Get Web Request method.
See the code below:
public class CookieAwareWebClient : WebClient
{
public CookieContainer CookieContainer { get; set; }
public CookieAwareWebClient()
: this(new CookieContainer())
{ }
public CookieAwareWebClient(CookieContainer c)
{
this.CookieContainer = c;
}
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
var castRequest = request as HttpWebRequest;
if (castRequest != null)
{
castRequest.KeepAlive = true; //<-- this what you want! The rest you don't need.
castRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
castRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.104 Safari/537.36";
castRequest.Referer = "https://www.jobserve.com/gb/en/Candidate/Login.aspx?url=48BB4C724EA6A1F2CADF4243A0D73C13225717A29AE8DAD6913D";
castRequest.Headers.Add("Accept-Encoding", "gzip,deflate,sdch");
castRequest.Headers.Add("Accept-Language", "en-GB,en-US;q=0.8,en;q=0.6");
castRequest.CookieContainer = this.CookieContainer;
}
return request;
}
}
As you can see I am not only enabling keep-alive but I am utilizing cookies and other headers also!
I hope that helps!
Kiran

Get content of response with StatusCode 401

I'm trying to establish a connection to a server that sends 401 Authentication Error for all my requests along with the normal html response. e.g.
However, I also want to read the HTML response that is sent alongwith so that I can parse that. An example header exchange captured using LiveHTTPHeaders:
Clearly, content-length is non-zero. Firefox shows it to be javascript.
https://172.31.1.251:1003/fgtauth?73e285357b2dc5cc
Request:
GET /fgtauth?73e285357b2dc5cc HTTP/1.1
Host: 172.31.1.251:1003
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Response:
HTTP/1.x 401 Unauthorized
WWW-Authenticate: Fortigate
Content-Length: 1091
Connection: Keep-Alive
Cache-Control: no-cache
Content-Type: text/html
At this point, a form opens in firefox that asks me to enter my username and password.
https://172.31.1.251:1003/
Request:
POST / HTTP/1.1
Host: 172.31.1.251:1003
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1b4) Gecko/20090423 Firefox/3.5b4 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: https://172.31.1.251:1003/
Content-Type: application/x-www-form-urlencoded
Content-Length: 93
magic=73e285357b2dc5cc&username=uuxx&password=xxuu&4Tredir=http%3A%2F%2Fwww.google.com%2F
Response:
HTTP/1.x 401 Unauthorized
WWW-Authenticate: Fortigate
Content-Length: 924
Connection: Keep-Alive
Cache-Control: no-cache
Content-Type: text/html
At this point I am redirected to another URL. However, the problem that I have is how to get the content of lenght 924 that is sent along with the 401 Unauthorized, because that content will help me in doing what I want to do further. But the very line:
WebResponse loginResponse = loginRequest.GetResponse();
throws an exception.
I will be grateful for any suggestions to help me get to the actual content.
Thanks.
You need to read the WebException's Response property, like this:
WebResponse loginResponse;
try {
loginResponse = loginRequest.GetResponse();
} catch(WebException ex) {
if (ex.Status == WebExceptionStatus.ProtocolError) {
loginResponse = ex.Response;
} else
throw;
}
//Do something with the response, and remember to dispose it.

Categories

Resources