i using HttpWebRequest/HttpWebResponse to get html document, the code follow was running but i can not encode received stream to html string:
string uri = "https://myfavoritesite.come";
HttpWebRequest webrequest = (HttpWebRequest)WebRequest.Create(uri);
webrequest.KeepAlive = true;
webrequest.Method = "GET";
webrequest.ContentType = "text/html";
webrequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
//webrequest.Connection = "keep-alive";
webrequest.Host = "cat.sabresonicweb.com";
webrequest.Headers.Add("Accept-Encoding", "gzip, deflate");
webrequest.Headers.Add("Accept-Language", "en-US,en;q=0.5");
webrequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1; rv:18.0) Gecko/20100101 Firefox/18.0";
HttpWebResponse webresponse = (HttpWebResponse)webrequest.GetResponse();
Console.Write(webresponse.StatusCode);
Stream receiveStream = webresponse.GetResponseStream();
Encoding enc = System.Text.Encoding.GetEncoding(1252);//1252
StreamReader loResponseStream = new
StreamReader(receiveStream, enc);
string Response = loResponseStream.ReadToEnd();
loResponseStream.Close();
webresponse.Close();
Console.Write(Response);
So, i use below code line to test is there successful request.
Console.Write(webresponse.StatusCode);
The result on the screen was OK, it's mean the request was sent but the Response string expose on screen was not html format, it's something so strange like this: #32u%&$&(#*#Eeeuw
By using webrequest.Headers.Add("Accept-Encoding", "gzip, deflate"); you are telling the server that you understand compressed responses. Remove that header and use a normal UTF8 encoding instead of 1252 that you are using. You should then get the proper string. You can just use System.Text.Encoding.UTF8.
Related
I haven't posted here in a long time so please forgive me if I am not formatting this question properly. I am trying to login to a website(omitted) via the .NET objects HttpWebRequest and HttpWebResponse. Using Wireshark, I can see that my POST request is identical to Chrome's POST request when I login to this website through my application. I am having issues getting the full response back though.
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
This response object has all the appropriate response headers, but there is still data I would like to see that is being sent through HTTP chunked responses. I can verify this in Wireshark as well. My understanding is that I need to instantiate a StreamReader object to read this remaining data. My code is blowing up at this line:
using (StreamReader sr = new StreamReader(response.GetResponseStream(), Encoding.UTF8))
The stack trace is showing this error:
System.ArgumentException: Stream was not readable.
How can I use this StreamReader object to read the entire response after my POST request is sent? Below is my code that sends the POST request for logging into the website. Please let me know if you have any questions and I will be happy to clarify any confusion.
public bool Login()
{
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(#"http://WEBSITE-REMOVED/CheckAccess");
request.CookieContainer = CookieContainer;
//Set POST data.
string postData = "institution=AAA";
postData += "&ismobile=false";
postData += "&id=BBB";
postData += "&password=CCC";
byte[] data = Encoding.ASCII.GetBytes(postData);
//Configure HTTP POST request.
request.Headers.Clear();
request.Method = "POST";
request.Accept = #"text/html, application/xhtml+xml, image/jxr, */*";
request.Referer = #"http://WEBSITE-REMOVED/entry.html";
request.Headers.Add("Accept-Language", "en-US");
request.UserAgent = #"Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko";
request.ContentType = #"application/x-www-form-urlencoded";
request.Headers.Add("Accept-Encoding", "gzip, deflate");
request.Host = "op.responsive.net";
request.ContentLength = data.Length;
request.KeepAlive = true;
request.Headers.Add("Cache-Control", "no-cache");
using (Stream stream = request.GetRequestStream())
{
stream.Write(data, 0, data.Length);
}//using
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
using (StreamReader sr = new StreamReader(response.GetResponseStream(), Encoding.UTF8))
{
//TO-DO
}
}//try
catch(Exception ex)
{
File.AppendAllText(ErrorLogDirectory + "Errors.txt", "Login() Exception\r\n" + System.DateTime.Now + "\r\n" + ex.ToString() + Environment.NewLine);
}//catch
return false;
}//Login
how can i set enc from this photo on my app
my codes :
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("my privet url");
req.Method = "get";
req.UserAgent = "Dalvik/1.6.0 (Linux; U; Android 4.4.2; vivo Y28L Build/KOT49H";
WebResponse res;
res = req.GetResponse();
StreamReader reader = new StreamReader(res.GetResponseStream());
textBox2.Text = reader.ReadToEnd();
If you mean Accept-Encoding, you can use Headers property of your req object just like that:
req.Headers["Accept-Encoding"] = "gzip, deflate";
But when you receive response, you need to wrap res.GetResponseStream() in GZipStream or DeflateStream if you want to read response content.
I'm having a problem when sending HttpWebRequest to a particular url.
The response is generated but it is not that I want.
It is giving a response like "Session Timeout! Please Login to continue".
Whereas,
When I'm sending the same request through Python 3.4 it is giving me the required response.
The response is like "--some html is here--" which I want.
I don't know why this is happening because I wan't it to do throught C#.
The CodeBehind (giving the wrong response) is:
string postData = "somequerystring";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("someurl");
request.KeepAlive = true;
request.ProtocolVersion = HttpVersion.Version10;
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36";
request.Method = "POST";
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = byteArray.Length;
request.Headers.Add(HttpRequestHeader.CacheControl, "no-cache");
using (var dataStream = request.GetRequestStream())
{
dataStream.Write(byteArray, 0, byteArray.Length);
}
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string responseFromServer = new StreamReader(response.GetResponseStream()).ReadToEnd();
response.Close();
Context.Response.Write(responseFromServer);
The Python Script (giving the correct response) is:
import requests
url = "someurl"
payload = "somequerystring"
headers = {
'content-type': "application/x-www-form-urlencoded"
}
response = requests.request("POST", url, data=payload, headers=headers)
print(response.text)
Help me out?
I'm writing simple application which connects to webpage and receives specific data from document's body.
I'm sending HttpWebRequest in this way:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://webpage.com/page1.php");
request.Method = "POST";
string postData = "some_post_data";
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
request.ContentType = "application/x-www-form-urlencoded";
request.Referer = "http://webpage.com/";
request.ContentLength = byteArray.Length;
request.CookieContainer = new CookieContainer();
request.UserAgent = "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31";
request.Headers.Add("Cookie", string.Format("sess_id={0}", UserInfo.SessionId));
request.Headers.Add("Accept-Charset", "ISO-8859-2,utf-8;q=0.7,*;q=0.3");
request.Headers.Add("Accept-Language", "pl-PL,pl;q=0.8,en-US;q=0.6,en;q=0.4");
request.Headers.Add("Accept-Encoding", "gzip");
request.AutomaticDecompression = DecompressionMethods.GZip;
Stream dataStream = request.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close(); // *** [1] ***
HttpWebResponse response = (HttpWebResponse)request.GetResponse(); // *** [2] ***
dataStream = response.GetResponseStream();
response.Close();
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = reader.ReadToEnd();
Fine! The request is ok, there is nothing magic. The point is, after dataStream.Close(); (line [1]) I receive 303 status code, which means there will be redirect to another page. When I read response at point [2] I see new (redirected) page.
All I want is to read headers before redirect. Is it possible anyway?
Set the AllowAutoRedirect property of request to false prior to calling GetResponse.
request.AllowAutoRedirect = false;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
More information.
I have problems with characters encoding received from http web response, I receive ? instead é.
I set the encoding to according Content-Type of web page that's text/javascript; charset=ISO-8859;
My code is:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(..);
request.Method = "GET";
request.AllowAutoRedirect = false;
request.Referer = "Mozilla/5.0 (Windows NT 6.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1";
request.Headers.Add("DNT", "1");
request.Accept = "text/html,application/xhtml+xml,application/xml";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader sr = new StreamReader(stream, Encoding.GetEncoding("iso-8859-1"));
char[] buf = new char[256];
int count;
StringBuilder buffer = new StringBuilder();
while ((count = sr.Read(buf, 0, 256)) > 0)
{
buffer.Append(buf, 0, count);
}
string responseStr = buffer.ToString();
Console.WriteLine(responseStr);
response.Close();
stream.Close();
sr.Close();
Can you tell me what is wrong with it?
Try adding the following before you make your request:
request.Headers.Add(HttpRequestHeader.AcceptCharset, "ISO-8859-1");
Btw, you should keep your StreamReader with ISO-8859-1 (instead of UTF8) if you want to try my proposed solution. Good luck!
Have you tried setting it at UTF-8?
Further more you send a referrer which I think you tried to set the UserAgent. The code below is the same as yours, but then does not go over the byte array and sets the useragent and utf8 encoding.
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
request.AllowAutoRedirect = false;
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1";
request.Headers.Add("DNT", "1");
request.Accept = "text/html,application/xhtml+xml,application/xml";
using(var response = (HttpWebResponse)request.GetResponse())
using(var stream = response.GetResponseStream())
using (var sr = new StreamReader(stream, Encoding.UTF8))
{
string responseStr = sr.ReadToEnd();
Console.WriteLine(responseStr);
response.Close();
if (stream != null)
stream.Close();
sr.Close();
}