Been looking for a solution, but I havent been able to get a result that answers my questions.
I'm looking for a way to go through sites (basically read the html), using the HttpClient. I'm making an app for windows phone, so some options may be disabled.
I want to make a program that goes to a site, logs in, and then is able to retrieve the access html source code.
So when I log in, a session id is saved in a CookieContainer, so I'll be able to access the sites that require login. How would I do this using the HttpClient :)?
HttpClient manages authentication cookies automatically for you. Just make sure you re-use the same HttpClient instance for multiple requests. Under the covers, HttpClient creates an instance of HttpClientHandler which has a CookieContainer.
Here is an example that logs into the NerdDinner site and retrieves a secured page.
var httpClient = new HttpClient();
// Create login payload
var body = new Dictionary<string, string>()
{
{"UserName", "bob"},
{"Password", "xyz"},
{"RememberMe", "false"}
};
var content = new FormUrlEncodedContent(body);
// POST to login form
var response = await httpClient.PostAsync("http://www.nerddinner.com/Account/LogOn?returnUrl=%2F", content);
// Make new request to secured resource
var myresponse = await httpClient.GetAsync("http://www.nerddinner.com/Dinners/My");
var stringContent = await myresponse.Content.ReadAsStringAsync();
Related
I'm trying to upload a photo using the Picasa Web Albums Data API in C# and I'm using RestSharp to help me with the HTTP requests.
I have the client ID and the client secret, but I don't know how to put that on the http request to make the authentication.
I'm getting the error:
Modification only allowed with api authentication.
The StatusCode that is returning is Forbidden
Here is my code where is my user ID and is my album ID.
The method "Authenticantion()" returns a token that I already have.
[HttpPost]
public string PostImage(){
var restclient = new RestClient(BaseUrl);
RestRequest request = new RestRequest("https://picasaweb.google.com/data/feed/api/user/<userID>/albumid/<albumID>") {Method = Method.POST};
string imageBase64 = "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkGBwgHBgkIBwgKCgkLDRYPDQwMDRsUFRAWIB0iIiAdHx8kKDQsJCYxJx8fLT0tMTU3Ojo6Iys/RD84QzQ5OjcBCgoKDQwNGg8PGjclHyU3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3N//AABEIAIUAyAMBEQACEQEDEQH/xAAcAAABBQEBAQAAAAAAAAAAAAAAAQIDBAUGBwj/xAA6EAABAwIDBgMFBgYDAQAAAAABAAIDBBEFITESE0FRYXEGIpEUMkJSgSMzYqGx0QdDgpLB4TVTchX/xAAaAQEAAwEBAQAAAAAAAAAAAAAAAQIDBQQG/8QAMREAAgIBAwMDAgMIAwAAAAAAAAECAxEEEjETIUEFMlEiYRRCkSMzUoGhsdHhNHHx/9oADAMBAAIRAxEAPwD3FACAEAhNkA3eM29jbbt2vs3zsoyuAOupAqAEAIAQAgBACAEAIAQAgBACAEAIAQAgBACAEAIAQHmuNfxbw6mdUx4dRzVIieWNqLjdvIyJGdyL8eKxnbjsuT1Vafd3lweTYx4sbisrq+eB7nSEh0r7B3qNOnJYKpuXd9z2SuShiKwj1Dwx4nw6WppW4dLNE6Vzd5Bsm0Y47V8lza650XdSUsL+hWVfUi8dz1FrgRdpuOi7yaayjltY5FUgVACAEAIAQAgBACAEAIAQAgBACAEAIAQAgBACA+WMQpGYfUzYdUl0LaV7omGUZua0kA+i8UstvB168OKJcOo4a2WKjpm3iik3jnvGVwf04WWVk3FOTNopP6T0ajw+Cima+NgjEoIDguDZfOyDT8HpUYo3KXFp6c7hz3MeORycOYVY6i2lfRLseedMJvg0I8YqOEzvqVovUblxIyelh8FuHHKgZOLHjsvRX6tcvdhmMtHHwX4MbiebSsc3qMwuhV6tXLtNYPNLSyXBoQ1MU33cjXdL5roV6iqz2SyeeUJR5RMtioIAQAgBACAEAIAQAgBACAEAIAQAgBAeK/xHwanxfGhK8ENjndtluV22zHqFw43OF1qXydiFalXHPhGRhrGU+J7IY1jZQAABYDkpt71myWGdjFEKuhdBtBr2+aN54EaLjb1XZ34Zq35K1PWMqmmmrWbM0brFpNnNPQq1lU6frqeUThM04qcAACZ3dwzXkzGfddiHLHKJN1PHYtAcPw5qHBruhvhIfHUcDe6je1yVdZZZOrK15MZQL1PidRFk2Qub8rs17qvU76+Hkwnp4S8GjBjLTlPER1bmunV61F/vI4PNLStcMux4hSyaSgf+sl0Ia/Tz/N+vYxdM14JmzRu917T2K9KthLhme1/A8EHRWyiAupAIAugAuaNSAo3JDDAOB0IKKSfAwwupAXQCoAQAgBAVcRq20VHLUPt5BkOZ4LDU3qip2PwaVVuyaijyrFyZWukc67r3J5kr5ume6XfydvG1YOfmLWP27jJtwV0EsrAZ2eHwyFokAIBANlxrI7m0XTwT1eHU1e5rpBu52izZG5ZcjzCtVa61tfBTLi8kFqmjLWT+ZugeNCsLaYvvE3i4zXYtwzg6HNYqbj2ZnKD8k5cyUfagO68Qrb0+zM+8eBjqZ4zhdtjiDqqurPeJZWL8xEJSw2NwRzWWJRL7U+CRtQeao7JIzdZIKm+pRXMr0x2+B4qeqV2D2zn5leN0/kjYPFS8fG5bLUWLyV6aFNZIB9471V/xdq/MOkn4GGtedXOP1Vfxc3yy3SQ11TfUrN6hkqsYZwq9b7k7Bpq5Izdkj29nFFqbE/pb/Ut0ovlFSHxXiVH4soqWd28wuZgjffZ2myONmkHW3Puvq/SdROyn9o8vJ4NXQovMUeijRdc8AqAEAIDnMdf7bLuAfsozc24uXz/qM+vPYn2X9/8AR0dKumt3lnN1uFxPa/XMWXMblU+zPepbuTk6/D2xzNa4eRxsujTfvjnyS4ncxEBthwOS5tc/pyUku4Scx7ymXdExHsex7Cx4DgciCFRPHJSUWnlGbV0z6S8sV3Q8ebVEq4yPTXap9pciwVYyDl5pVSjwTKousnGVlVTweeUCVzmSi0rQR+a06ifZoot0e6ZXdSDWKT6OUOMXwzVXfKGmlnA0a7sVR0PwSrIMaYphrE/6BUenl8E5g+GJaT/rf/aVXpteB9PyOG+Okcn9pU7LPgj6fkeIah38p31VlVY/BG6C8i+zTH3nMb9bqypl5ZHUiOFMB78p+gU9KvyyHY/CF3EI+Y93Jsq+CN8hRBAdR+alQgRvmUqvC8EfUMqqqmifNEQ5j3EktINxbrde/T6qdXaLIlCVnKLPhPxlPX+JJ8Jrjk9hkpy5lnC2oNsuoX1NFvUjlnMvqUH2O9XoPOCAhqpNzTySD4WkrK6eytyLQjukkcq59ma3XyykddRKkrrleeyXc3ijBxyn3lO4MyeR5TyPBX0dm2Zo+DQoKts8TJWnJ7Q63JYNOqTi/BEllFouvxV9xVITatmCqSfwTjJaglDhb1UxsXkxlHBn12HFhMtILt1MY4dv2Wrxg9FV/wCWZUhmIzaVhOtPk9EoJlyOrys5YOtrgwlT8E7Khp4rJyceUZOtkrZRwJUq37mbiSNlWsbXjko4jzMeDj6rTq/cbRu8cfiNk6n3J2oa6QNGbh6qrn9wo58ED6uFurgT0Ue7hGqpk/BWkxBvwNv3V1S33ZtHT/LKzq2VxysB0WnRWDTpQQ0zSO1cVCqihiK4InXdqtFiPBGfBueE8ABxRmKyR7Lo2lrXfNdd70uNrTlL2+Dka6yDe1cndLsHOBAU8X/4+bt/leTXf8eRtp/3iOScciF8pux2OwkQOuVjOXc0KlXHtxFUpntmW8GJhdR7JVS4fIciTLD2JzH0P6hdPU1dWKsX8ysX4NyOXJc5pouSNeLqNwaHA7OipIjBcgqBo7Iq9d2O0jCVb5RHV4fFVXfE7dy87ZHuvQseCa75Q7Pgy5opqV2zMy3JwzBUNZPdCcbFmI0PHBVwWwOEjho4qjhF8oq4pj9+/wCb1VHTAr00G/f8wRUxI6aGmZ/zFXVUfgtsRE5xOpK0jHHBbjgiLraLVINiHNTgqOaFVsq2TsjJsAsZTKs6LA8C3pE1SLMB0PFdnQemueLLl2+Dm6rWL2QOtYxrGhrRYDIAL6BHKHKQCAr1zN5SStAudk2WGpjvpkvsXrltmmcbLkvjLHhncRAVhJlsDHC+SybLGHjeHPlaJaYtbURO24nHS/I9DoupotUvbLhhxzwJhmICqhuGljgdl8bveYeIK01Gm2MJmkx99F4ZRLkrX81m0B97rNoholildGcjkoUnHgpKKkXGVMb27MgB6ELeN6fZmDra7ohlwuknF4iYnfh09F6IyUuGXjqbIc9ypJg9Uz7t8cg9CrbTeOsrfuWCu+jrI/egf/SLquEaq6qXDIXCRps6N47tKYRonF+Rm0eRTCJwIS46NJ+issEDDtX913orJogGi7w0uFzoCcyrN9smbwa9BhlTVEbqM2PxHRVq012of7OPb58Hntvrr9zOowvAIqbZkn87+XBd3Sel10vdPvI5V+snZ2j2RuNFhYaLqnjFQAgBAIUBx+LU/s9VJGB5b3b2K+P9Qp6VridnT2b4JmcuVk9ItlVgY5oKopbXknJgYrhksUxraIHefzIx8Y/ddrSayM49OwnHlDaGvbM0WOfda3adxJTNNkgdoV4ZQZYla/qspRBIHhZODK4H3BVHFkYHMkcw3Y4hFuXBDinyTtrZW62PdaRusRR0xHCvePgH0Wn4iXwV6CYv/wBF/BqdeXwPw6GOr5fwqOrJk9CI32+XoE6kiehAa6vmdq4eiOUnySqIIqzOpZaiOonpYJJ4zdkroxtNPQrWq2+HsYlp4SLjfEdRTQhkRyGnFdOv1LVpY7GD0NWe5Vq/4jT4U6GSsonT0xfaV8Vg5g5gcV1dJq7LMqw8t+kjHvE9FgmZUQsmicHRvaHNcDkQV0jnvsSIAQAgBAZHiCk30G+YPNHr1C5Xqmm6te9cr+x69JbtltfDOUeNk5L5Ca2s66AFUZIEXVJIqNLcrWVFJplsmRieC715npCGTHUH3X9+R6rq6X1FRW2zgnOTNhrZYJDDUNdG8atcM/8AfddJ1QsW6DyiU/k0IqsOAIddeWVDRYssmBzWDrYJBL1WbrA7eqvTAu96qOmQG9TpAN6mwCbxTsAhkUqAGmRXVYyRudtZK6WAyNzNpaKWCuCtUYU7ER7O2Mvc/Jo5Femi2W9KHJlYo7cy4PVsHpBQYZS0gNxDE1l+wX1COC3kuKSAQAgBAI5oc0hwuCjBx2MUTqSoLQPs3ZtP+F8h6lo+hZ2X0vj/AAdnTXdSHfkzNCuPhnrHNcoIY5UcSoiyaBFVUdPWM2J4w5vC+o7LanU2UvMGMsxpvD88Xmopw4D4Zcj6hdar1WEli2P81/gspEQjrqf76lksOLfMPyXoVuns9kl/YumhzKi+RuO4sjrJRIJ+qp0y2B296hRsGBRIeahxGBwe7RRhENAXuU7UVAbR1JTsVHtaSqOSBMyIHusZWElyloXzyBsbddTyW+l0l+qf09l8mF19dS78nVYPhUNGA8jblOrivqNLoq9MsR5+fJyLtRO55fBtNGS9ZgKgBACAEAICriFHHWU7o368DyKxvohdW4T4L12OuW5HFVdNJTTGKVtnD0K+L1Wknp7Nk/8A07lVsbY5RBdeRrJqK11lRxaIaHhwVH9yMAqbRgLkaJgYF2uKrhkYAuB4XRJrgnDEuPlCt9XyO41xYAfs2+i0ipfxE4fyQmVrdI2jsFuqpPmTL4+5z3ip9e9sE9BLI0w7RLGNvckakceX1XY9OhUsxn5+Tz3qa7wLuFyGqoKadzTvJImucNmxvxyWV0GrHFLz/wBmkbE4KT7GhHTSvOTD6KY6TUz4gykr6o8yLsOGTvObLd8l6q/R75e9pf1PPLX1L29zSpsHtYyuHYBdCj0iivvL6meSzW2S9vY2KWBkQ2WNsF1UsLCPHk0IhZATDRAKgBACAEAIAQFDFMOir4rOFnj3XDgvPqNPXqIbJo0qtlXLMTi62lmo5THM2x4Hgey+S1egt00vq7r5OzTfG1duSDaXjwbjmvVHAD9pZusBtKNhGAumwBtAIoNkjHSALSNIwROmvkt41YLELnDhqt1FkixRue8C13HQLaqieolsrRlbdCqOZHUYfh7ImXLBtHVfWU1RpioxODZZKbyy+2BrdAAtTMkbGgJGMQFiONAWGCyAegBACAEAIAQAgBAVK6hhrIiyZlwqygpLbLglNxeUcbimD1NDd8bXTQ9PeH7rhar0fDcqP0OlTrVxZ+pltkBzByXFnU4vbJYZ74yUu6Hh6z2Fhd51TYBDKp2AY6Q2V1WgRF7itFFEgGucjaROSSKMvkEcTduQ8OXUrbT6WzVPC4+TC7URqXfk6XCcJENnyeaQ6lfU0aeGnhsgv9nEstlZLMjabFbQLYzHiK6AkbEgJWxhAStaAgHIAQAgBACAEAIAQAgBAMfG14sQgMTEvDlLVkuDNiT52GxWdlMLViayXhZKHtZz9X4ar4M4JWSjk8WPqFzbPSapextHshrpL3LJmS0mIQ3EtFJ3YQ7/AH+S8c/Sblw0z0R11b5IDIWe/FK3vGV55en6hflNVqqn+Ybvmng8/wBBUfg9R/Cy3Xr/AIkSxMmlNoaaZ/Ztv1WkPTdRJ8YM5ayqPk06PAa+ozm2YWHgDdy99PpMI97Xn7Hls17faCwdNhuCw0bfI3Pi46ldaMYxWIrCPA5NvLNQRADJWIHCNAODEA4NQC2CAVACAEAIAQAgBACAEAIAQAgBAIQCgI3QxnVqAidRQO1YEA32Cn+QICRlLE3RgQEzWNboEAtkAWQCoAQAgBACAEAIAQAgP//Z";
request.AddParameter("client_id", this._clientID);
request.AddParameter("client_secret", this._clientSecret);
request.AddParameter("grant_type", "client_credentials");
request.AddHeader("GData-Version", "3");
request.AddHeader("Content-Type", "image/jpeg");
request.AddHeader("accessToken", Authenticantion());
request.AddBody(#"Content-Length: 5951
Slug: banana.jpeg " + imageBase64);
var tResponse = restclient.Execute(request);
var responseJson = tResponse.Content;
var token = Authenticantion();
return responseJson;
}
The OAuth2 authentication normally happens inside a user flow, where the user makes the login with the user and allow (or not) some scopes to the app (e.g. access Picasa, read only for Google Drive).
You can check the complete flow through this link: OAuthPlayground. And get your own token at the end of the flow to use with your HTTP requests. One thing to take care is about refresh the tokens, inside the playground it's possible to set to never expire. But on the real world you would need to implement this refresh.
The Picasa API is a little outdated, I'd recommend you to use GoogleDrive instead, it's much more flexible and up to date.
GoogleDrive_API_Docs
It's also possible to make this flow via OAuth2_ Server to server, but I never tried before.
I am working on a project to build a web spider that crawls our internal web pages. The security is built using the Identity Framework. The non-protected pages is easy by using this code:
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("http://www.yoursite.com/resource/file.htm");
using (StreamWriter streamWriter = new StreamWriter(webRequest.GetRequestStream(), Encoding.UTF8))
{
streamWriter.Write(requestData);
}
string responseData = string.Empty;
HttpWebResponse httpResponse = (HttpWebResponse)webRequest.GetResponse();
using (StreamReader responseReader = new StreamReader(httpResponse.GetResponseStream()))
{
responseData = responseReader.ReadToEnd();
}
But this code isn't using the OWIN authentication so when it hits a protected page it can't access it.
If I am logged in and try HttpContext.Request then I see through the debugger the Request is authenticated. How do I use this request object to get a new protected page to parse? I am missing a very simple method I am sure.
Update
I am still struggling with this. Maybe I should ask this another way. I am calling a page from the same web application. From the calling controller this returns true which is good:
HttpContext.Current.User.Identity.IsAuthenticated
But through the debugger and check this in the receiving controller and now this is false, which is bad:
HttpContext.Current.User.Identity.IsAuthenticated
How can get the receiving controller to be authenticated?
It really depends on what authentication method you implementation is being used on server side. Asp.net Identity is essentially a membership system.
For example if it's using bearer authentication tokens then you'd need to set a bearer authentication token in the Authorization header of the request you are sending. And I'd rather suggest using HttpClient for these purposes.
As you mentioned OWIN, Im assuming this is an bearer token scenario.
var httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", "Your Oauth token");
Update
If you're using the default project template of Asp.net MVC with Asp.net Identity as a added dependency later, you can hook up a httpClient like the following one here:
class Program
{
static void Main()
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://yourapp");
var content = new FormUrlEncodedContent(new[]
{
// Push your stuff here, your username, password fields
// as you coded in your server
new KeyValuePair<string, string>("", "login")
});
var result = client.PostAsync("login", content).Result;
string resultContent = result.Content.ReadAsStringAsync().Result;
Console.WriteLine(resultContent);
}
}
}
And you'd of course need to save the auth cookie and you can see how here and to attach it when you do the request to get the secured page, you have to do something like this
Looks like the permissions are based on roles. Check to see if the user you are logged in as has the permission to access the controller method getting hit when the user is redirected to that page.
If you have Attribute parameter for the method that specifies the role for the user required in order to hit then the logged in user should have those permissions to access the page.
As you are getting authenticated may be the issue is with authorization. The user might not have enough permissions to access the page.
I have the following set up:
JS client -> Web Api -> Web Api
I need to send the auth cookie all the way down. My problem is sending it from one web api to another. Because of integration with an older system, that uses FormsAuthentication, I have to pass on the auth cookie.
For performance reasons I share a list of HttpClients (one for each web api) in the following dictionary:
private static ConcurrentDictionary<ApiIdentifier, HttpClient> _clients = new ConcurrentDictionary<ApiIdentifier, HttpClient>();
So given an identifier I can grab the corresponding HttpClient.
The following works, but I'm pretty sure this is bad code:
HttpClient client = _clients[identifier];
var callerRequest = HttpContext.Current.Items["MS_HttpRequestMessage"] as HttpRequestMessage;
string authCookieValue = GetAuthCookieValue(callerRequest);
if (authCookieValue != null)
{
client.DefaultRequestHeaders.Remove("Cookie");
client.DefaultRequestHeaders.Add("Cookie", ".ASPXAUTH=" + authCookieValue);
}
HttpResponseMessage response = await client.PutAsJsonAsync(methodName, dataToSend);
// Handle response...
Whats wrong about this is that 1) it seems wrong to manipulate DefaultRequestHeaders in a request and 2) potentially two simultanious requests may mess up the cookies, as the HttpClient is shared.
I've been searching for a while without finding a solution, as most having a matching problem instantiates the HttpClient for every request, hence being able to set the required headers, which I'm trying to avoid.
At one point I had get requests working using a HttpResponseMessage. Perhaps that can be of inspiration to a solution.
So my question is: is there a way to set cookies for a single request using a HttpClient, that will be safe from other clients using the same instance?
Instead of calling PutAsJsonAsync() you can use HttpRequestMessage and SendAsync():
Uri requestUri = ...;
HttpMethod method = HttpMethod.Get /*Put, Post, Delete, etc.*/;
var request = new HttpRequestMessage(method, requestUri);
request.Headers.TryAddWithoutValidation("Cookie", ".ASPXAUTH=" + authCookieValue);
request.Content = new StringContent(jsonDataToSend, Encoding.UTF8, "application/json");
var response = await client.SendAsync(request);
UPDATE:
To make sure that your HTTP client does not store any cookies from a response you need to do this:
var httpClient = new HttpClient(new HttpClientHandler() { UseCookies = false; });
Otherwise you might get unexpected behavior by using one client and sharing other cookies.
I'm trying to fix an issue with a legacy asp.net WebForms application written about 6 years ago. I'll try to explain and hope that someone can see a fix.
Our application supports what we call an "API". Really it's just a couple of pages intended to be displayed within the pages of our customers' websites. We also provice a sample application that shows how to use it.
The sample app has some form fields for the caller to provide values. When submitted, we create a string of HTML containing a elelement with the tag that
submit()'s the form on postgback.
Example:
...
form1.submit();
This method works successfully as long as the end user puts our domain in thier trusted sites. Otherwise the user gets permission denied errors. However, recently, we've had a couple customers refuse to add us to trusted sites and
want the issue fixed another way.
One approach I've tries is to use HttpClient.PostAsync() to do what the form/script above does but from the server side.
public HttpResponseMessage Post(string address, MediaTypeWithQualityHeaderValue acceptType, List<KeyValuePair<string, string>> data)
{
using (HttpClient client = new HttpClient())
{
//client.BaseAddress = new Uri(address);
client.DefaultRequestHeaders.Accept.Add(acceptType);
HttpContent content = new FormUrlEncodedContent(data);
HttpResponseMessage response = client.PostAsync(address, content).Result;
return response.EnsureSuccessStatusCode();
}
}
It's called like this:
string hash = GenerateHashValue();
List<KeyValuePair<string, string>> data = this.BuildPostData(hash);
MediaTypeWithQualityHeaderValue acceptType = new MediaTypeWithQualityHeaderValue("text/html");
HttpResponseMessage msg = Post(this.GetPostTargetUrl(), acceptType, data);
var task = msg.Content.ReadAsStringAsync();
task.Wait();
string result = task.Result;
Response.Write(result);
Response.End();
The response HTML is successfully written into the but it's essentally "disconnected" - nothing works after that. Normally, using the / method about uses can navigate in the iframe to other pages on our
site.Looking at it in fiddler, I see that some of the posts and redirects that I see when using the current / method is happeing and I get 404s for many of the javascript files coming from our site.
Can anyone suggest another approach in which I would not have XSS errors but a "connected" iframe?
Thanks,
Dan
I'm currently using HttpWebRequest to get a website. I'd like to use the await pattern, which is not given for HttpWebRequests. I found the class HttpClient, which seems to be the new Http worker class. I'm using HttpClient.GetAsync(...) to query my webpage. But I'm missing the option to add ClientCredentials like HttpWebRequest.Credentials. Is there any way to give the HttpClient authentication information?
You can pass an instance of the HttpClientHandler Class with the credentials to the HttpClient Constructor:
using (var handler = new HttpClientHandler { Credentials = ... })
using (var client = new HttpClient(handler))
{
var result = await client.GetAsync(...);
}
You shouldn't dispose of the HttpClient every time, but use it (or a small pool of clients) for a longer period (lifetime of application. You also don't need the handler for it, but instead you can change the default headers.
After creating the client, you can set its Default Request Headers for Authentication. Here is an example for Basic authentication:
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", "username:password".ToBase64());
ToBase64() represents a helper function that transforms the string to a base64 encoding.