Similar to the CSV file which can be downloaded from http://download.finance.yahoo.com/d/quotes.csv?s=RHT . How can I downloada file which requires authentication?
I can simply use
My.Computer.Network.DownloadFile("http://download.finance.yahoo.com/d/quotes.csv?s=RHT,MSFT,NOVL&f=sl1c1d1&e=.csv", "h:\s.csv")
To download the file which is available in public. I tried setting the username and password as per the MSDN documentation but all I get is the HTML content of the login page.
If the site uses Cookie based authentication, you'll need to post the login details to the server, collect the cookies, then pass them up on the request for the file.
That's not as easy as it sounds...
There's an example here
http://blogs.msdn.com/dgorti/archive/2005/08/16/452347.aspx
Phil
It all depends on whether or not the web site will interpret your username/password when you provide it in your HTTP request. Some sites/protocols will, and others won't.
Here's an article that shows how you can download with the WebRequest class.
Downloading Files with the WebRequest and WebResponse Classes
All you need in addition to this article is adding your username/password to the WebRequest. You can do it like this:
// Create a request for the specified remote file name
WebRequest request = WebRequest.Create(remoteFilename);
if (request != null)
{
string username = "your username";
string password = "your password";
request.Credentials = new System.Net.NetworkCredential(username, password);
...
}
Unfortunately, if using this method doesn't work, then Yahoo's finance page doesn't allow providing a username/password automatically, and you'd have to login "the old fashioned way" to be able to download your file.
It sounds like the site you are working with is using forms authentication. Unless you have access to a version that uses realm authentication you will probably need to use HttpWebRequest and fake POST request to the site while you have a CookieContainer so you can retain the token. Then you would be able to include that token in a get request to download the CSV file.
Related
I've been trying to log into Gmail using:
WebClient Web = new WebClient();
Web.Credentials = new NetworkCredential(Email, Password, "https://mail.google.com");
and then browse the email using IHTMLDocument2 interface by getting the id associated to each email, building up the url, navigating to that url and using DownloadString method, pass the HTML content to the IHTMLDocument2 interface.
The problem is that google does not accept the credentials that I send so I can never log in. However, after a few attempts I saw that this activity has been monitored by Google as I saw a notification about that on the actual email account.
I would like to know how this problem can be solved or is it possible at all to log into Gmail using WebClient? if yes, how exactly ?
Use Fiddler to observe what the browser does. Fiddler can capture HTTPS traffic. A Google login is more complex than you might think.
The Credentials property has no effect on web forms. There is no standardized way to have a login form in HTML. WebClient cannot possibly know how the Google login form works. It cannot possibly automate this for you.
I am writing an app that will talk with Salesforce. Salesforce provides access to APIs via OAuth. I've been attempting to go through the OAuth authentication process described here. Currently, I'm attempting to authorize my app. I have the following code.
// Ask Salesforce for a request token
var request = (HttpWebRequest)(WebRequest.Create(String.Format("https://login.salesforce.com/services/oauth2/authorize?response_type=code&client_id={0}&redirect_uri=http://localhost:5004/home/AuthCallback", CONSUMER_KEY)));
request.Method = "POST";
request.ContentType = "application/json";
// Retrieve the request token from the response
var response = (HttpWebResponse)(request.GetResponse());
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
string accessCodeData = accessCodeReader.ReadToEnd();
}
This code is triggered when a user clicks a button in my ASP.NET MVC view. When executed, this code calls to Salesforce. I see a request in fiddler. The request header looks like this:
POST /services/oauth2/authorize?response_type=code&client_id={consumerKey}&redirect_uri=http://localhost:5004/home/AuthCallback HTTP/1.1
I am in fact passing my consumer key, I'm just removing it from the example. Regardless, this request returns a 302, with a body size of 0. I might be misunderstanding something. However, I was expecting to get a request token. I was then going to use the request token to get the access token.
What am I doing wrong?
You are misusing the API.
Take a closer look at the sequence diagram at their page (under Obtaining an Access Token): in the auhorization_code flow you are supposed to redirect the browser to their page so that the user sees the login page, provides his/her credentials and you get the token back. Instead, you are trying to POST there using a web request from your server.
This particular flow belongs then to the passive flows group, this group is intended to be used in browser apps, your server redirects the browser to their server and you basically get the response to the uri passed in the redirect_uri parameter and this should point back to your application at your server.
There are other flows, of them one is suited for non-browser apps, it is called resource owner password flow. In this flow it is your application that hosts the login UI and you send the username/password to the authorization server and you get the token back. It is to be read in their docs however whether this flow is supported.
Read more here: http://aaronparecki.com/articles/2012/07/29/1/oauth2-simplified
Take a look how I handle the OAuth2 flow to Google, using the DotNetOpenAuth library. This is a direct solution, applying it to any other provider should be straightforward.
http://www.wiktorzychla.com/2014/11/simple-oauth2-federated-authentication.html
I have developed an application to download user names and Ids from face book graph API. But I cannot get their public information like birthdays.
So my approach is to download HTML of about pages and read Birthday from that.
my code follows; I developed my application in MVC4:
public ActionResult ExternalLoginCallback(string returnUrl)
{
AuthenticationResult result = OAuthWebSecurity.VerifyAuthentication(Url.Action("ExternalLoginCallback", new { ReturnUrl = returnUrl }));
if (!result.IsSuccessful)
{
return RedirectToAction("ExternalLoginFailure");
}
if (result.ExtraData.Keys.Contains("accesstoken"))
{
Session["facebooktoken1"] = result.ExtraData["accesstoken"];
string fbToken = result.ExtraData["accesstoken"];
ViewBag.TokenToview = fbToken;
ViewBag.Id = 5;
WebRequest myWebRequest = WebRequest.Create("https://www.facebook.com/amila.u.abeyrathne/about");
WebResponse myWebResponse = myWebRequest.GetResponse();
Stream ReceiveStream = myWebResponse.GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
StreamReader readStream = new StreamReader(ReceiveStream, encode);
string strResponse = readStream.ReadToEnd();
}
When I use this I only get html of LOG IN page of facebook. How can I use my facebook login session in this application?
To get information like this, you'd need to hit Facebook Graph API endpoints like mentioned here
. You'd need an access token gneerated with the necessary permissions as specified here in order to submit a working request. I'm assuming the access token you're using in this code is generated by querying OAuth in the specified manner.
It is not possible to get those information from facebook graph Search.The solution is to develop application that can log in to facebook and scrap the HTML of the facebook wall of each user whom userids have being downloaded. After scraping the HTML read the text from htlm is the solution. But it is not that much accurate. Birthdays and other information in the about page is not possible to read on that way either because there is no way to navigate to the about page in that way.
Ur application session data cannot be read by facebook API, so need to send the token in subsequent requests.
Getting FB Page data from facebook using C#
I think your approach is wrong. when you log in to a website it is created session cookie in client browser. Then it append to each request header when send request to that web site again. session cookie created on client browser not the server.
What you are doing is you send request to facebook from your server not from the client. Then there is no valid session cookie append to your request.
If you what to go this way you have to read the facebook page using javascript. But it is not the recommended way.
You need to crate facebook app for your website.
Then need to Select permissions to request from users. (for your case use user_birthday )
Then authenticate your website using that app. Then you can get what ever the details that you want using graph api if users gave permission.
I have a webpage which is windows forms authenticated ,and i want to download a copy of this page's HTML in to my server, when user request this page. I have tried something like this
using (WebClient client = new WebClient())
{
string htmlCode = client.DownloadString("http://aksphases:200/lynliste.aspx");
}
which doesn't gives me correct result because of the URL I had passes to system creates new session.And in that case i need to authenticate this web-request,which I can't do.Only way to authenticate this webpage is that user log in manually(I know ways to authenticate werequests by code,but I can't try that here for some special reasons). Is there any other way for me to download current page's HTML which is running in in browser with out authenticating the URL.
You could send the current forms authentication cookie along with the request:
using (WebClient client = new WebClient())
{
client.Headers[HttpRequestHeader.Cookie] =
System.Web.HttpContext.Current.Request.Headers["Cookie"];
string htmlCode = client.DownloadString("http://aksphases:200/lynliste.aspx");
}
This way we are basically transferring the current HTTP request cookies to the remote HTTP call.
If the web server does not allow anonymous access then there is no way around - you must authenticate yourself with the web site.
However, contrary to your belief that log on operation has to be done manually, it can be done via code also. In case of windows authentication, pass credentials via Credentials property. For Forms authentication, you need to POST log-on credentials to login page and then use the authentication cookie from the response in subsequent request (Use tool such fiddler to inspect request/responses from browser to replicate same within your code).
I'm trying to login directly to Google Analytics. To explain, I have an account system and I'd like when you select an ASP.NET button for instance it re-directs you - via a silent login - to a specified Google Analytics account.
I've looked long and hard at Dave Cullen's ASP.NET library and although I can login 'silently' using HttpWebRequest, I can't then stick the user on that page. I'm having allsorts of dramas with a 'Cannot send a content-body with this verb-type' error too.
Here is the very basic code I have currently based on Dave's library;
string token = GoogleAnalytics.getSessionTokenClientLogin(username, password);
NameValueCollection profiles = GoogleAnalytics.getAccountInfo(token, GoogleAnalytics.mode.ClientLogin);
HttpWebRequest theRequest = (HttpWebRequest)WebRequest.Create("https://www.google.com/analytics/settings/?et=reset&hl=en_uk&et=reset&hl=en-US&et=reset&hl=en-GB");
theRequest.Headers.Add("Authorization: GoogleLogin auth=" + token);
Stream responseBody = theRequest.GetRequestStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
StreamReader readStream = new StreamReader(responseBody, encode);
My question therefore is; 1. can this be done? and 2. is this even the right way to do this?
Advice welcomed!
I'm not sure what the overall goal of signing someone in to Google Analytics automatically is, but if its just to display some of the data that is in Google Analytics you might want to consider using the Google Data API to pull the information that you want from Google Analytics. You could create a simple dashboard of what they really need to see without giving access to other things in Google Analytics, by logging them in you are most likely giving them access to data and tools that they just don't need?
Check out the API if it doesn't fit your needs maybe provide some more information on the overall goal is of this functionality.
http://code.google.com/apis/analytics/
Unless you're willing to implement a proxy server to proxy google analytics, I don't think you're going to be able to do this because you can't assign cookies to the client for another domain.
If the auth tokens are stored in cookies you can add the cookies to your ASP.NET response - then host the google page in an IFRAME just by setting the src (no inlining). That IFRAME will "inherit" the cookies from your parent page and the page will think it's authenticated.