I use following code to post querystring
string URI = "http://somewebsite.com/default.aspx";
string myParameters = "param1=value1¶m2=value2¶m3=value3";
using (WebClient wc = new WebClient())
{
wc.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
string HtmlResult = wc.UploadString(URI, myParameters);
}
But somehow default.aspx does not accept that post call.
The point is when I manually in browser go to http://somewebsite.com/default.aspx all code there is working fine.
My questions is following what do I am missing here to archive the same result when I open page manually as I do it with WebClient?
Thank you in advance!
P.S. 1
I just tried to use GET method to that URL and it has no effect also. How is it possible?
What is difference between manual navigation to page and sending GET/POST?
P.S. 2
I even tried this
wc.Headers["Accept"] = "application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*";
wc.Headers["User-Agent"] = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; MDDC)";
and and Load event of Default.aspx is not hiting. :(
From your description of what you want to achieve, I think you may have chosen the wrong WebClient method. Instead of UploadString, try DownloadString:
using (WebClient wc = new WebClient())
{
string HtmlResult = wc.DownloadString("http://somewebsite.com/default.aspx?param1=value1¶m2=value2¶m3=value3");
}
So that comment is correct one
"What is difference between manual navigation to page and sending
GET/POST?" - see for yourself, for example using Fiddler. –
CodeCaster
I checked all requests with Fiddler and found that there is code of base page class that redirects to Index page. So Load event is never happened.
Related
in my C# application below method it is hitting service and returning data. At the return statement there was a fortify scan issue raised under the category named Server side request forgery. There is no parameter adding to URL in the method. uri as is used to fetch data. I am completely new.
using (var wc = new WebClient())
{
wc.Encoding = System.Text.Encoding.ASCII;
wc.Headers["User-Agent"] = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)";
return JsonObject.GetDynamicJsonObject(wc.DownloadString(uri.ToString())); //here issue raised
}
I have a webclient that calls to a URL that works fine when i view it in a browser, which led me to believe i would need to add headers in to my call
I have done this, but am still getting the error.
I do have other calls to the same API that work fine, and have checked that all the parameters I am passing across are exactly the same as expected(case, spelling)
using (var wb = new WebClient())
{
wb.Proxy = proxy;
wb.Headers.Add("Accept-Language", " en-US");
wb.Headers.Add("Accept", " text/html, application/xhtml+xml, */*");
wb.Headers.Add("User-Agent", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)");
byte[] response = wb.UploadValues("http://myserver/api/account/GetUser",
new NameValueCollection()
{
{ "email", register.Email },
});
userDetails = Encoding.UTF8.GetString(response);
}
Does anyone have an idea why I am still getting the protocol error on a call that works perfectly fine in a browser?
UploadValue uses a HTTP POST. Are you sure that it what you want? If you are viewing it in a browser it is likely a GET, unless you are filling out some sort of web form.
One might surmise that what you are trying to do is GET this response "http://myserver/api/account/GetUser?email=blah#blah.com"
in which case you would just formulate that url, with query parameters, and execute a GET using one of the DownloadString overloads.
using (var wb = new WebClient())
{
wb.Proxy = proxy;
userDetails = wb.DownloadString("http://myserver/api/account/GetUser?email=" + register.Email);
}
The Wikipedia article on REST has a nice table that outlines the semantics of each HTTP verb, which may help choosing the appropriate WebClient method to use for your use cases.
My code is :
string result = new WebClient().DownloadString("https://www.facebook.com/profile.php?id=123456789");
the result giving me supported browser.
body class=\"unsupportedBrowser
What i intend to do:
Download a source code from facebook for the particular page.
Problem encounter :
I get the stream from facebook, facebook block me since i access from the apps due to this is not a valid browser.
Expectation : How i can submit the browser type like chrome to cheat the facebook as this is a valid browser.
You can add headers to the webclient object. However, I prefer to go the HttpWebRequest/HttpWebResponse method of scraping since I believe it gives more options.
Headers.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
Headers.Add("Accept-Encoding", "gzip, deflate");
Headers.Add("Accept-Language", "en-US,en;q=0.5");
Headers.Add("Cookie", "has_js=1");
Headers.Add("DNT", "1");
Headers.Add("Host", host);
Headers.Add("Referer", url);
Headers.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0");
This page uses frames, but your browser doesn't support them.
This error occurs when I am trying to get the information from our smsgatwaye site.
The code is as follows:
WebClient client = new WebClient();
client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR1.0.3705;)");
string baseurl = "http://smsoutbox.in/?user=test&password=test#123";
Stream data = client.OpenRead(baseurl);
StreamReader reader = new StreamReader(data);
string s = reader.ReadToEnd();
data.Close();
reader.Close();
I am requesting to http://smsoutbox.in page which ask for username & password if it is valid than it shows my gateway balance on same page in frame.
But when I get the response, I found this error:
This page uses frames, but your browser doesn't support. (Line) instead of balance in response stream.
How can solve this?
View the source of the page yourself, and look at the frames being used. Open each one separately to determine which URL you need to retreive.
The problem might be that WebClient already submits a UserAgent and by adding another "user-agent"-header you're not replacing the original header.
Use this modified WebClient that internally uses HttpWebRequest's UserAgent property:
http://codehelp.smartdev.eu/2009/05/08/improve-webclient-by-adding-useragent-and-cookies-to-your-requests/
Alternatively it should work to correctly modify the UserAgent like this:
client.Headers["user-agent"] = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR1.0.3705;)");
I'd like my application to query a csv file from a secure website. I have no experience with web programming so I'd appreciate detailed instructions. Currently I have the user login to the site, manually query the csv, and have my application load the file locally. I'd like to automate this by having the user enter his login information, authenticating him on the website, and querying the data. The application is written in C# .NET.
I've tested the following code already and am able to access the file once the user has already authenticated himself and created a manual query.
System.Net.WebClient Client = new WebClient();
Stream strm = Client.OpenRead("https://<URL>/file.csv");
Here is the request sent to the site for authentication. I've angle bracketed the real userid and password.
POST /pwdVal.asp HTTP/1.1
Accept: image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/x-shockwave-flash, */*
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; Tablet PC 2.0; OfficeLiveConnector.1.4; OfficeLivePatch.1.3; .NET4.0C; .NET4.0E)
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
Cookie: ASPSESSIONID<unsure if this data contained password info so removed>; ClientId=<username>
Host: www3.emidas.com
Content-Length: 36
Connection: Keep-Alive
Cache-Control: no-cache
Accept-Language: en-US
client_id=<username>&password=<password>
Most likely the server sends a cookie once login is performed. You need to submit the same values as the login form. (this can be done using UploadValues()) However, you need to save the resulting cookies in a CookieContainer.
When I did this, I did it using HttpWebRequest, however per http://couldbedone.blogspot.com/2007/08/webclient-handling-cookies.html you can subclass WebClient and override the GetWebRequest() method to make it support cookies.
Oh, also, I found it useful to use Fiddler while manually accessing the web site to see what actually gets sent back and forth to the web site, so I knew what I was trying to reproduce.
edit, elaboration requested: I can only elaborate how to do it using HttpWebRequest, I have not done it using WebClient. Below is the code snippet I used for login.
private CookieContainer _jar = new CookieContainer();
private string _password;
private string _userid;
private string _url;
private string _userAgent;
...
string responseData;
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(_url);
webRequest.CookieContainer = _jar;
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.UserAgent = _userAgent;
string requestBody = String.Format(
"client_id={0}&password={1}", _userid, _password);
try
{
using (StreamWriter requestWriter = new StreamWriter(webRequest.GetRequestStream()))
{
requestWriter.Write(requestBody);
requestWriter.Close();
using (HttpWebResponse res = (HttpWebResponse)webRequest.GetResponse())
{
using (StreamReader responseReader = new StreamReader(res.GetResponseStream()))
{
responseData = responseReader.ReadToEnd();
responseReader.Close();
if (res.StatusCode != HttpStatusCode.OK)
throw new WebException("Logon failed", null, WebExceptionStatus.Success, res);
}
}
}
Before you go down this rabbit hole, contact the web site and ask them if they provide a web service to query user account info from. The simulated login method you are proposing should be a last resort only.
Another way you can do it is to automate IE, e.g. use a WebBrowser control. That will more accurately simulate all the clever stuff that IE does like running Javascript, which might be necessary. Although if Javascript or other clever stuff isn't necessary then using IE is a little heavy-handed and possibly prone to other problems.