Develop Reference

Get file list from owncloud

I want to get file list from owncloud on my ASP site. I was succeed using
curl -X PROPFIND -u user:password "http://yourserver.com/owncloud/remote.php/webdav/" from linux but I can't get the same result using default http request with propfind type in order to use it then in c# https://user:password#host/owncloud/remote.php/webdav. I get 400 code as a result on my request. Also I tried webdavclient from nuget but received method not allowed exception.
IClient c = new Client(new NetworkCredential { UserName = "user", Password = "password" });
var client = new WebDAVClient.Client(new NetworkCredential());
c.Server = "xxx.com/owncloud/remote.php/webdav/";
var isfolderCreated = c.CreateDir("/", "lalala").Result;
Could anybody say to me how to send http request to owncloud to get the file list? I tried webdav protocol that is used by clients but maybe I should try anything else?

I found the issue that prevented me. I just didn't use basic authorization correctly in http request. Since I add correct credentials I could send the http request and get the response. And here is the code I use in c#:
var request = (HttpWebRequest)WebRequest.Create("xxx.com/owncloud/remote.php/webdav/");
request.Credentials = new NetworkCredential("user", "password");
request.PreAuthenticate = true;
request.Method = #"PROPFIND";
request.Headers.Add(#"Translate", "F");
var httpGetResponse = (HttpWebResponse)request.GetResponse();
using (var responseStream = httpGetResponse.GetResponseStream())
{
long responseLength = httpGetResponse.ContentLength;
using (var streamReader = new StreamReader(responseStream, Encoding.UTF8))
{
var files = XElement.Parse(streamReader.ReadToEnd());
streamReader.Close();
}
responseStream.Close();
}

OneLogin Create Session Login Token API returns status 400 with message: Bad Request

I am developing a C# application which needs to use the onelogin API to retrieve a session token. I am able to authenticate and and create a token with the following code:
WebRequest Authrequest = WebRequest.Create("https://api.us.onelogin.com/auth/oauth2/token");
Authrequest.Method = "POST";
Authrequest.ContentType = "application/json";
Authrequest.Headers.Add("cache-control", "no-cache");
Authrequest.Headers.Add("Authorization: client_id:XXXXXXX7bbf2c50200d8175206f664dc28ffd3ec66eef0bfedb68c3366420dc, client_secret:XXXXXXXXXX6ba2802187feb23f6450c6812b8e6639361d24aa83f12010f ");
using (var streamWriter = new StreamWriter(Authrequest.GetRequestStream()))
{
string Authjson = new JavaScriptSerializer().Serialize(new
{
grant_type = "client_credentials"
});
streamWriter.Write(Authjson);
}
WebResponse AuthReponse;
AuthReponse = Authrequest.GetResponse();
Stream receiveStream = AuthReponse.GetResponseStream ();
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader (receiveStream);
JObject incdata = JObject.Parse(readStream.ReadToEnd());
string sToken = incdata["data"][0]["access_token"].Value<string>();
AuthReponse.Close();
However, when running the Create Session Login Token with the following code, it only returns a 400 error, and the message has no detail. Just Bad Request:
//Get the session token for the specified user, using the token recieved from previous web request
WebRequest request = WebRequest.Create("https://api.us.onelogin.com/api/1/login/auth");
request.Method = "POST";
request.ContentType = "application/json";
request.Headers.Add("authorization", "bearer:" + sToken);
using (var streamWriter2 = new StreamWriter(request.GetRequestStream()))
{
string json = JsonConvert.SerializeObject(new
{
username_or_email = sUsername,
password = sPassword,
subdomain = "comp-alt-dev"
});
streamWriter2.Write(json);
}
WebResponse response;
response = request.GetResponse();
string streamText = "";
var responseStream = response.GetResponseStream();
using (responseStream)
{
var streamReader = new StreamReader(responseStream);
using (streamReader)
{
streamText = streamReader.ReadToEnd();
streamReader.Close();
//
}
responseStream.Close();
}
Any ideas?
-Thank you

Also for anyone who may be getting this error. in C# the email is case sensitive. I tried User.email.com. In onelogin it was saved as user#email.com. changing the c# to lower case fixed it.

Can you let us know what payload you're sending across the wire to the .../1/login/auth endpoint as well as the response (either as others have suggested as packet snoop, or just as a debug output from the code)
400 means either bad json or the endpoint requires MFA, so this will narrow it down.
~thanks!

Just joining the troubleshooting effort =) -- I can replicate a 400 Bad Request status code with a "bad request" message when the request body contains a username_or_email and/or subdomain value that does not exist, or if the request body is empty.

Can you post what goes over the wire to the OneLogin endpoint...
OK Thanks. So it appears your subdomain does not exist. If you give me an email in the account I can find the correct subdomain value for you.

Problems authenticating to website from code

I am trying to write code that will authenticate to the website wallbase.cc. I've looked at what it does using Firfebug/Chrome Developer tools and it seems fairly easy:
Post "usrname=$USER&pass=$PASS&nopass_email=Type+in+your+e-mail+and+press+enter&nopass=0" to the webpage "http://wallbase.cc/user/login", store the returned cookies and use them on all future requests.
Here is my code:
private CookieContainer _cookies = new CookieContainer();
//......
HttpPost("http://wallbase.cc/user/login", string.Format("usrname={0}&pass={1}&nopass_email=Type+in+your+e-mail+and+press+enter&nopass=0", Username, assword));
//......
private string HttpPost(string url, string parameters)
{
try
{
System.Net.WebRequest req = System.Net.WebRequest.Create(url);
//Add these, as we're doing a POST
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
((HttpWebRequest)req).Referer = "http://wallbase.cc/home/";
((HttpWebRequest)req).CookieContainer = _cookies;
//We need to count how many bytes we're sending. Post'ed Faked Forms should be name=value&
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(parameters);
req.ContentLength = bytes.Length;
System.IO.Stream os = req.GetRequestStream();
os.Write(bytes, 0, bytes.Length); //Push it out there
os.Close();
//get response
using (System.Net.WebResponse resp = req.GetResponse())
{
if (resp == null) return null;
using (Stream st = resp.GetResponseStream())
{
System.IO.StreamReader sr = new System.IO.StreamReader(st);
return sr.ReadToEnd().Trim();
}
}
}
catch (Exception)
{
return null;
}
}
After calling HttpPost with my login parameters I would expect all future calls using this same method to be authenticated (assuming a valid username/password). I do get a session cookie in my cookie collection but for some reason I'm not authenticated. I get a session cookie in my cookie collection regardless of which page I visit so I tried loading the home page first to get the initial session cookie and then logging in but there was no change.
To my knowledge this Python version works: https://github.com/sevensins/Wallbase-Downloader/blob/master/wallbase.sh (line 336)
Any ideas on how to get authentication working?
Update #1
When using a correct user/password pair the response automatically redirects to the referrer but when an incorrect user/pass pair is received it does not redirect and returns a bad user/pass pair. Based on this it seems as though authentication is happening, but maybe not all the key pieces of information are being saved??
Update #2
I am using .NET 3.5. When I tried the above code in .NET 4, with the added line of System.Net.ServicePointManager.Expect100Continue = false (which was in my code, just not shown here) it works, no changes necessary. The problem seems to stem directly from some pre-.Net 4 issue.

This is based on code from one of my projects, as well as code found from various answers here on stackoverflow.
First we need to set up a Cookie aware WebClient that is going to use HTML 1.0.
public class CookieAwareWebClient : WebClient
{
private CookieContainer cookie = new CookieContainer();
protected override WebRequest GetWebRequest(Uri address)
{
HttpWebRequest request = (HttpWebRequest)base.GetWebRequest(address);
request.ProtocolVersion = HttpVersion.Version10;
if (request is HttpWebRequest)
{
(request as HttpWebRequest).CookieContainer = cookie;
}
return request;
}
}
Next we set up the code that handles the Authentication and then finally loads the response.
var client = new CookieAwareWebClient();
client.UseDefaultCredentials = true;
client.BaseAddress = #"http://wallbase.cc";
var loginData = new NameValueCollection();
loginData.Add("usrname", "test");
loginData.Add("pass", "123");
loginData.Add("nopass_email", "Type in your e-mail and press enter");
loginData.Add("nopass", "0");
var result = client.UploadValues(#"http://wallbase.cc/user/login", "POST", loginData);
string response = System.Text.Encoding.UTF8.GetString(result);
We can try this out using the HTML Visualizer inbuilt into Visual Studio while staying in debug mode and use that to confirm that we were able to authenticate and load the Home page while staying authenticated.
The key here is to set up a CookieContainer and use HTTP 1.0, instead of 1.1. I am not entirely sure why forcing it to use 1.0 allows you to authenticate and load the page successfully, but part of the solution is based on this answer.
https://stackoverflow.com/a/10916014/408182
I used Fiddler to make sure that the response sent by the C# Client was the same as with my web browser Chrome. It also allows me to confirm if the C# client is being redirect correctly. In this case we can see that with HTML 1.0 we are getting the HTTP/1.0 302 Found and then redirects us to the home page as intended. If we switch back to HTML 1.1 we will get an HTTP/1.1 417 Expectation Failed message instead.
There is some information on this error message available in this stackoverflow thread.
HTTP POST Returns Error: 417 "Expectation Failed."
Edit: Hack/Fix for .NET 3.5
I have spent a lot of time trying to figure out the difference between 3.5 and 4.0, but I seriously have no clue. It looks like 3.5 is creating a new cookie after the authentication and the only way I found around this was to authenticate the user twice.
I also had to make some changes on the WebClient based on information from this post.
http://dot-net-expertise.blogspot.fr/2009/10/cookiecontainer-domain-handling-bug-fix.html
public class CookieAwareWebClient : WebClient
{
public CookieContainer cookies = new CookieContainer();
protected override WebRequest GetWebRequest(Uri address)
{
var request = base.GetWebRequest(address);
var httpRequest = request as HttpWebRequest;
if (httpRequest != null)
{
httpRequest.ProtocolVersion = HttpVersion.Version10;
httpRequest.CookieContainer = cookies;
var table = (Hashtable)cookies.GetType().InvokeMember("m_domainTable", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.GetField | System.Reflection.BindingFlags.Instance, null, cookies, new object[] { });
var keys = new ArrayList(table.Keys);
foreach (var key in keys)
{
var newKey = (key as string).Substring(1);
table[newKey] = table[key];
}
}
return request;
}
}
var client = new CookieAwareWebClient();
var loginData = new NameValueCollection();
loginData.Add("usrname", "test");
loginData.Add("pass", "123");
loginData.Add("nopass_email", "Type in your e-mail and press enter");
loginData.Add("nopass", "0");
// Hack: Authenticate the user twice!
client.UploadValues(#"http://wallbase.cc/user/login", "POST", loginData);
var result = client.UploadValues(#"http://wallbase.cc/user/login", "POST", loginData);
string response = System.Text.Encoding.UTF8.GetString(result);

You may need to add the following:
//get response
using (System.Net.WebResponse resp = req.GetResponse())
{
foreach (Cookie c in resp.Cookies)
_cookies.Add(c);
// Do other stuff with response....
}
Another thing that you might have to do is, if the server responds with a 302 (redirect) the .Net web request will automatically follow it and in the process you might lose the cookie you're after. You can turn off this behavior with the following code:
req.AllowAutoRedirect = false;

The Python you reference uses a different referrer (http://wallbase.cc/start/). It is also followed by another post to (http://wallbase.cc/user/adult_confirm/1). Try the other referrer and followup with this POST.
I think you are authenticating correctly, but that the site needs more info/assertions from you before proceeding.

Login and retrieve data from embedded form page using C#

I'm a bit confused on how to go about this as I'm not really conversant with web stuff. I'm using a console application in C# to try and retrieve value from a page link inside a password protected homepage. I'm using the following details
Here's the code I'm trying:
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("");
req.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705;)";
req.Method = "POST";
req.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
req.Headers.Add("Accept-Language: en-us,en;q=0.5");
req.Headers.Add("Accept-Encoding: gzip,deflate");
req.Headers.Add("Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7");
req.KeepAlive = true;
req.Headers.Add("Keep-Alive: 300");
req.Referer = "copy from url";
req.ContentType = "application/x-www-form-urlencoded";
String Username = copy from url;
String PassWord = copy from url;
StreamWriter sw = new StreamWriter(req.GetRequestStream());
sw.Write(string.Format("&loginname={0}&password={1}&btnSubmit=Log In&institutioncode=H4V9KLUT45AV&version=2", Username, PassWord));
sw.Close();
HttpWebResponse response = (HttpWebResponse)req.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string tmp = reader.ReadToEnd();
However, when I inspect the data retrieved from the web page it shows something like this:
'...Your Session has timed out due to inactivity.Please logout and
relogin.return to login page>'
I'm guessing this is due to some VIEWSTATE stuff in ASP.NET
I'm also guessing I might have a problem with retrieving the data from the link I'll extract from the homepage, coz it seems the link simply loads data into a frame rather than reload the webpage.
Anyone please?

Your form data is incorrect. After removing the & at the beginning it worked for me:
sw.Write(string.Format("loginname={0}&password={1}&btnSubmit=Log In&institutioncode=H4V9KLUT45AV&version=2", Username, PassWord));
Additionally, as already mentioned in the other answer, you need to add the returned ASPSESSIONIDSSRRDRST cookie in further requests to the site.

Ok... the website is using Cookies, so, after you logged in you need to retrieve the cookies first, to make another WebRequest:
CookieCollection cookiesResponse = new CookieCollection();
if (response != null)
{
foreach (string cookie in response.Headers["Set-Cookie"].Split(';'))
{
string name = cookie.Split('=')[0];
string value = cookie.Substring(name.Length + 1);
cookiesResponse.Add(new Cookie(name.Trim(), value.Trim(), path, domain));
}
}
In you example the cookie contains: ASPSESSIONIDSSRRDRST=FEKODBMDBEIPCLLENCFLFBEA
You must use that CookieCollection for any request to the web, in your request you can set the cookies:
request.CookieContainer = cookiesResponse;
And finaly, you can parse the response. You can use an html tag parse, or parse the plain text.
I hope this is helpful.

HttpWebRequest has empty response requesting a search from Bing

I have the following code that sends a HttpWebRequest to Bing. When I request the url below though it returns what appears to be an empty response when it should be returning a list of results.
var response = string.Empty;
var httpWebRequest = WebRequest.Create("http://www.bing.com/search?q=stackoverflow&count=100") as HttpWebRequest;
httpWebRequest.Method = WebRequestMethods.Http.Get;
httpWebRequest.Headers.Add("Accept-Language", "en-US");
httpWebRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Win32)";
httpWebRequest.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate");
using (var httpWebResponse = httpWebRequest.GetResponse() as HttpWebResponse)
{
Stream stream = null;
using (stream = httpWebResponse.GetResponseStream())
{
if (httpWebResponse.ContentEncoding.ToLower().Contains("gzip"))
stream = new GZipStream(stream, CompressionMode.Decompress);
else if (httpWebResponse.ContentEncoding.ToLower().Contains("deflate"))
stream = new DeflateStream(stream, CompressionMode.Decompress);
var streamReader = new StreamReader(stream, Encoding.UTF8);
response = streamReader.ReadToEnd();
}
}
Its pretty standard code for requesting and receiving a web page. Any ideas why the response is empty? Thanks in advance.
EDIT I left off a query string parameter in the url. I also had &count=100 which I have now corrected. It seems to work for values of 50 and below but returns nothing when larger. This works ok when in the browser, but not for this web request.
It makes me think the issue is that the response is large and HttpWebResponse is not handling that for me the way I have it set up. Just a guess though.

This works just fine on my machine. Perhaps you are IP banned from Bing?

Your code works fine on my machine.
I suggest you get yourself a copy of Fiddler and examine the actual HTTP sesssion occuring. May be a proxy or firewall thing.