Develop Reference

How can I use cookies to re-order the site, since in order to get a json from the website I need to be logged in?

I'm already getting login cookies from the website. How can I use these cookies to make another requests to the site, since in order to get a json from the website I need to be logged in?
I'm doing this:
string formParams = string.Format("user[email]={0}&user[password]={1}", "****#********.com", "********");
string formUrl = "https://*************.com/users/sign_in";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(formUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
Now i need to make a request to this link: https://********.com/parishioners.json?entity_id=13723
Someone can help me?

GET Request after POST(login)

I have a web site which i check my product list daily. I want to make a desktop program for it.
I need to login to the web site first then i go to site.com/v1/ProductList which is an xml document. I have managed to login with this code:
CookieCollection cookies = new CookieCollection();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(FirstURL);
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(cookies);
//Get the response from the server and save the cookies from the first request..
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
cookies = response.Cookies;
string postData = "Username=x&Password=x&List=1&Submit=Submit";
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
getRequest.CookieContainer = new CookieContainer();
getRequest.CookieContainer.Add(cookies); //recover cookies First request
getRequest.Method = WebRequestMethods.Http.Post;
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
getRequest.AllowWriteStreamBuffering = true;
getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
byte[] byteArray = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = byteArray.Length;
Stream newStream = getRequest.GetRequestStream(); //open connection
newStream.Write(byteArray, 0, byteArray.Length); // Send the data.
newStream.Close();
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
string sourceCode = sr.ReadToEnd();
}
Here i am successfully logged in.
But after this, if i create a new get request for my list(site.com/v1/ProductList) and get request it redirects me to the login page.
Edit: I just realized that i cant get any cookies after i login. It says "'enumeration yielded no results'".
I have no idea how to fix it right now.
Thanks

Change this line:
getRequest.AllowAutoRedirect = true;
to this:
getRequest.AllowAutoRedirect = false;
Make sure you actually set it to false and don't just delete the line because, by default, it gets set to true.

Cannot login website with httprequest POST in C#?

I'm trying to write a bit of code to login to a website. But it's not working. Please can you give me some advice. This is my a bit of code:
static void Main(string[] args)
{
CookieContainer container = new CookieContainer();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://pagehype.com/login.php");
request.Method = "POST";
request.Timeout = 10000;
request.ReadWriteTimeout = 30000;
request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) (Prevx 3.0.5)";
request.CookieContainer = container;
ASCIIEncoding encoding = new ASCIIEncoding();
string postData = "username=user&password=password&processlogin=1&return=";
byte[] data = encoding.GetBytes(postData);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = data.Length;
Stream newStream = request.GetRequestStream();
newStream.Write(data, 0, data.Length);
newStream.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string htmldoc = reader.ReadToEnd();
response.Close();
Console.Write(htmldoc);
}
Many thanks,

Use http://www.fiddler2.com/fiddler2/ to view the http request sent went you login in using a browser and ensure that the request you build in code is the same.

PHP logins use PHPSESSID cookie. You'll need to capture this and pass it back in the CookieContainer. This is how the server will recognise you as an authenticated user.
The cookie is set in the Set-Cookie header in the initial response. You'll need to parse it to recreate the cookie in your container (don't forget the path (and domain?)
var setCookie = response.GetResponseHeader("Set-Cookie");
response.Close();
container = new CookieContainer();
foreach (var cookie in setCookie.Split(','))
{
var split = cookie.Split(';');
string name = split[0].Split('=')[0];
string value = split[0].Split('=')[1];
var c = new Cookie(name, value);
if (cookie.Contains(" Domain="))
c.Domain = split.Where(x => x.StartsWith(" Domain")).First().Split('=')[1];
else
{
c.Domain = ".pagehype.com";
}
if (cookie.Contains(" Path="))
c.Path = split.Where(x => x.StartsWith(" Path")).First().Split('=')[1];
container.Add(c);
}
Then add this container to your request.

getResponse in c# not working. No response coming back

I have this code in C#:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.ContentType = "application/x-www-form-urlencoded";
request.Timeout = 30000;
request.Method = "POST";
request.KeepAlive = true;
request.AllowAutoRedirect = false;
Stream newStream = request.GetRequestStream();
newStream.Write(bPostData, 0, bPostData.Length);
byte[] buf = new byte[1025]; int read = 0; string sResp = "";
HttpWebResponse wResp = (HttpWebResponse)request.GetResponse();
Stream resp = wResp.GetResponseStream();
The line HttpWebResponse wResp =... just hangs (as in no response from the URL). I'm not sure where exactly its crashing (cause i dont even get an exception error). I tested the URL in IE and it works fine. I also checked the bPostData and that one has data in it.
Where is it going wrong?

Try closing the request stream in variable newStream. Maybe the API waits for it to be done.

You have to increase the limit:
ServicePointManager.DefaultConnectionLimit = 10; // Max number of requests

Try simplifying your code and faking a user agent. Maybe the site is blocking/throttling scrapers/bots. Also ensure your application/x-www-form-urlencoded HTTP POST values are properly encoded. For this I would recommend you WebClient:
using (var client = new WebClient())
{
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0) Gecko/20100101 Firefox/4.0";
var values = new NameValueCollection
{
{ "param1", "value1" },
{ "param2", "value2" },
};
byte[] result = client.UploadValues(url, values);
}

When I commented earlier, I had run your code at my office (heavily firewalled) I got the same result you did. Came home, tried again (less firewalled) it worked fine... I'm guessing you have a barrier there. I believe you are facing a firewall issue.

Use a content-length=0
Example:
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(URL);
request.Method = "POST";
request.ContentLength = 0;
var requestStream = request.GetRequestStream();
HttpWebResponse res = (HttpWebResponse)request.GetResponse();
res.Close();

Programmatically login to google app engine c#

I've tried to login to my google app engine application from ASP.NET for a few days, but no luck. I've read the following articles and got the basic ideas. But nothing works for me.
http://code.activestate.com/recipes/577217-routines-for-programmatically-authenticating-with-/
http://dalelane.co.uk/blog/?p=303
http://dalelane.co.uk/blog/?p=894
http://krasserm.blogspot.com/2010/01/accessing-security-enabled-google-app.html
http://blog.notdot.net/2010/05/Authenticating-against-App-Engine-from-an-Android-app
I know what to do. 1) Get an auth token from ClientLogin. 2) Get a cookie from Google App Engine. 3) Post data to my app with the cookie (Yes, I want to post data, not redirect after the second part). But the third part doesn't work for me at all. It give me 403 error. Here is my code:
void PostToGAE()
{
var auth = GetAuth(); // I can get the authtoken
var cookies = GetCookies(auth); // I can get the ACSID cookie
var url = string.Format("http://test.appspot.com/do/something/");
var content = "testvalue=test";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.KeepAlive = false;
request.CookieContainer = cookies;
byte[] byteArray = Encoding.UTF8.GetBytes(content);
request.ContentLength = byteArray.Length;
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
Stream dataStream = request.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse(); // This gives me 403
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream);
string result = reader.ReadToEnd();
reader.Close();
}
CookieContainer GetCookies(string auth)
{
CookieContainer cookies = new CookieContainer();
var url = string.Format("http://test.appspot.com/_ah/login?auth={0}",
System.Web.HttpUtility.UrlEncode(auth));
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.AllowAutoRedirect = false;
request.CookieContainer = cookies;
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "GET";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream);
string result = reader.ReadToEnd();
reader.Close();
return cookies;
}
string GetAuth()
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.google.com/accounts/ClientLogin");
var content = "Email=test#gmail.com&Passwd=testpass&service=ah&accountType=HOSTED_OR_GOOGLE";
byte[] byteArray = Encoding.UTF8.GetBytes(content);
request.ContentLength = byteArray.Length;
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
Stream dataStream = request.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader reader = new StreamReader(stream);
string loginStuff = reader.ReadToEnd();
reader.Close();
var auth = loginStuff.Substring(loginStuff.IndexOf("Auth")).Replace("Auth=", "").TrimEnd('\n');
return auth;
}
My app.yaml looks like this:
- url: /do/something/
script: something.py
login: admin
If I change the method POST to GET, that works. Could anyone tell me how I can post data?
Thanks.
EDITED:
Still no luck. I've tried several ways such as changing to [login: required] in app.yaml, adding [secure: always] to app.yaml and changing the request protocol to https, appending continue parameter to /_ah/login, but all of them don't work :(
I totally have no idea why POST doesn't work at all but GET. Any ideas?

I made it. I was on the wrong track. That was not the problem of app engine but Django. I am using Django-nonrel on google app engine, and I totally forgot to put #csrf_exempt decorator to my handler. I had the same problem before, but again. Anyway, the code above has been apparently working correctly since at the beginning. What a smart boy :)