With the following code, I'm trying to post a html form and expecting a complete html page-
string reqString = "destination=&calldate_from=2015-08-01&calldate_to=2015-08-02&call_status=ANSWERED&search=Search";
byte[] requestData = Encoding.UTF8.GetBytes(reqString);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://192.168.100.70:83/report/outgoing/search");
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = requestData.Length;
request.Host = "192.168.100.70:83";
request.Method = "POST";
Cookie idcookie = new Cookie("PHPSESSID", "19fv94lhgtdr80st6ig366q461") { Domain = "192.168.100.70" };
Cookie bdcookie = new Cookie("ciIPPBXOBd", "nMXEiUJiSU%3D") { Domain = "192.168.100.70" };
request.CookieContainer = request.CookieContainer ?? new CookieContainer();
request.CookieContainer.Add(idcookie);
request.CookieContainer.Add(bdcookie);
// Get the request stream.
Stream requestStream = request.GetRequestStream();
requestStream.Write(requestData, 0, requestData.Length);
// requestStream.Flush();
requestStream.Close();
WebResponse response = request.GetResponse();
var res = ((HttpWebResponse)response).StatusDescription; //gets OK here.
StreamReader reader = new StreamReader(response.GetResponseStream());
string returnValue = reader.ReadToEnd(); //found empty
reader.Close();
Code is running without throwing any exception. But I can not retrieve the full html. How can I do that?
Related
I want write a program in Windows Forms.I want log in in Instagram by HttpWebResponse in c#.
I use following code to log in:
HttpWebResponse Response;
HttpWebRequest Request;
CookieContainer cookieContainer = new CookieContainer();
Uri url = new Uri("https://i.instagram.com/api/v1/si/fetch_headers/?challenge_type=signup&guid=");
Request = (HttpWebRequest)WebRequest.Create(url);
Request.Method = "GET";
Request.CookieContainer = cookieContainer;
// Get the first response to obtain the cookie where you will find the "csrfmiddlewaretoken" value
Response = (HttpWebResponse)Request.GetResponse();
textBox1.Text = cookieContainer.GetCookies(url)["csrftoken"].Value;
string PostData = string.Format("csrfmiddlewaretoken={0}&username=seyed&password=13859", cookieContainer.GetCookies(url)["csrftoken"].Value);
//textBox1.Text = PostData;
HttpWebRequest req;
req = (HttpWebRequest)WebRequest.Create("https://i.instagram.com/api/v1/accounts/login/");
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.KeepAlive = true;
req.AllowAutoRedirect = true;
req.CookieContainer = cookieContainer;
byte[] byteArray = Encoding.ASCII.GetBytes(PostData);
req.ContentLength = byteArray.Length;
textBox1.Text = req.ContentLength.ToString();
Stream dataStream = req.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Flush();
dataStream.Close();
HttpWebResponse webResp = (HttpWebResponse)req.GetResponse();
Stream datastream = webResp.GetResponseStream();
StreamReader reader = new StreamReader(datastream);
string s = reader.ReadToEnd();
MessageBox.Show(s);
if (s.Contains("seyed"))
{
MessageBox.Show("Loggedin");
}
else
{
MessageBox.Show("Not");
}
}
My username and password is correct but can not log in in it and print "Not" Message.
How fix it?
I am using HttpWebRequest to get webpage source code with POST method.
The page needs to be accessed for multiple times with different parameters.
When getting response in the second time in a short period, it always return the same response object to me. After debugging, if the second call is after around 30 seconds, the request can get the correct response object(source code).
public HtmlAgilityPack.HtmlDocument getHtmlData(string url, string cname)
{
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
try
{
HttpRequestCachePolicy policy = new HttpRequestCachePolicy(HttpRequestCacheLevel.Default);
HttpWebRequest.DefaultCachePolicy = policy;
HttpRequestCachePolicy noCachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.NoCacheNoStore);
HttpWebRequest req = WebRequest.Create(url) as HttpWebRequest;
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.KeepAlive = false;
req.Headers[HttpRequestHeader.CacheControl] = "no-cache";
req.Headers[HttpRequestHeader.Pragma] = "no-cache";
req.IfModifiedSince = DateTime.Now;
req.CachePolicy = noCachePolicy;
//add post data
string postData = "cname=" + cname;
byte[] byteArray = Encoding.GetEncoding("shift-jis").GetBytes(postData);
req.ContentLength = byteArray.Length;
using (Stream stream = req.GetRequestStream())
{
stream.Write(byteArray, 0, byteArray.Length);
}
// get response
string data = "";
HttpWebResponse response = req.GetResponse() as HttpWebResponse;
using (var stream = response.GetResponseStream())
{
StreamReader reader = new StreamReader(stream, Encoding.GetEncoding("shift-jis"));
data = reader.ReadToEnd();
reader.Close();
}
response.Close();
doc.LoadHtml(data);
//System.Threading.Thread.Sleep(30000);
}
catch(Exception ex)
{
Log.log.Debug(ex.ToString());
}
return doc;
}
Function call block:
getHtmlData(#"http://www.jra.go.jp/JRADB/accessS.html", "pw01sli00/AF");
getHtmlData(#"http://www.jra.go.jp/JRADB/accessS.html", "pw01skl00999999/B3");
getHtmlData(#"http://www.jra.go.jp/JRADB/accessS.html", "pw01skl00201604/E3");
I have stuck with the problem for a whole day. hope someone can give me a clue.
Many thanks!
I need to call a method from a webservice, so I've written this code:
private string urlPath = "http://xxx.xxx.xxx/manager/";
string request = urlPath + "index.php/org/get_org_form";
WebRequest webRequest = WebRequest.Create(request);
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.
webRequest.ContentLength = 0;
WebResponse webResponse = webRequest.GetResponse();
But this method requires some parameters, as following:
Post data:
_username:'API USER', // api authentication username
_password:'API PASSWORD', // api authentication password
How can I add these parameters into this Webrequest?
Use stream to write content to webrequest
string data = "username=<value>&password=<value>"; //replace <value>
byte[] dataStream = Encoding.UTF8.GetBytes(data);
private string urlPath = "http://xxx.xxx.xxx/manager/";
string request = urlPath + "index.php/org/get_org_form";
WebRequest webRequest = WebRequest.Create(request);
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.ContentLength = dataStream.Length;
Stream newStream=webRequest.GetRequestStream();
// Send the data.
newStream.Write(dataStream,0,dataStream.Length);
newStream.Close();
WebResponse webResponse = webRequest.GetResponse();
If these are the parameters of url-string then you need to add them through '?' and '&' chars, for example http://example.com/index.aspx?username=Api_user&password=Api_password.
If these are the parameters of POST request, then you need to create POST data and write it to request stream. Here is sample method:
private static string doRequestWithBytesPostData(string requestUri, string method, byte[] postData,
CookieContainer cookieContainer,
string userAgent, string acceptHeaderString,
string referer,
string contentType, out string responseUri)
{
var result = "";
if (!string.IsNullOrEmpty(requestUri))
{
var request = WebRequest.Create(requestUri) as HttpWebRequest;
if (request != null)
{
request.KeepAlive = true;
var cachePolicy = new RequestCachePolicy(RequestCacheLevel.BypassCache);
request.CachePolicy = cachePolicy;
request.Expect = null;
if (!string.IsNullOrEmpty(method))
request.Method = method;
if (!string.IsNullOrEmpty(acceptHeaderString))
request.Accept = acceptHeaderString;
if (!string.IsNullOrEmpty(referer))
request.Referer = referer;
if (!string.IsNullOrEmpty(contentType))
request.ContentType = contentType;
if (!string.IsNullOrEmpty(userAgent))
request.UserAgent = userAgent;
if (cookieContainer != null)
request.CookieContainer = cookieContainer;
request.Timeout = Constants.RequestTimeOut;
if (request.Method == "POST")
{
if (postData != null)
{
request.ContentLength = postData.Length;
using (var dataStream = request.GetRequestStream())
{
dataStream.Write(postData, 0, postData.Length);
}
}
}
using (var httpWebResponse = request.GetResponse() as HttpWebResponse)
{
if (httpWebResponse != null)
{
responseUri = httpWebResponse.ResponseUri.AbsoluteUri;
cookieContainer.Add(httpWebResponse.Cookies);
using (var streamReader = new StreamReader(httpWebResponse.GetResponseStream()))
{
result = streamReader.ReadToEnd();
}
return result;
}
}
}
}
responseUri = null;
return null;
}
For doing FORM posts, the best way is to use WebClient.UploadValues() with a POST method.
Hope this works
webRequest.Credentials= new NetworkCredential("API_User","API_Password");
I have a feeling that the username and password that you are sending should be part of the Authorization Header. So the code below shows you how to create the Base64 string of the username and password. I also included an example of sending the POST data. In my case it was a phone_number parameter.
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(_username + ":" + _password));
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(Request);
webRequest.Headers.Add("Authorization", string.Format("Basic {0}", credentials));
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.Method = WebRequestMethods.Http.Post;
webRequest.AllowAutoRedirect = true;
webRequest.Proxy = null;
string data = "phone_number=19735559042";
byte[] dataStream = Encoding.UTF8.GetBytes(data);
request.ContentLength = dataStream.Length;
Stream newStream = webRequest.GetRequestStream();
newStream.Write(dataStream, 0, dataStream.Length);
newStream.Close();
HttpWebResponse response = (HttpWebResponse)webRequest.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader streamreader = new StreamReader(stream);
string s = streamreader.ReadToEnd();
The code below differs from all other code because at the end it prints the response string in the console that the request returns. I learned in previous posts that the user doesn't get the response Stream and displays it.
//Visual Basic Implementation Request and Response String
Dim params = "key1=value1&key2=value2"
Dim byteArray = UTF8.GetBytes(params)
Dim url = "https://okay.com"
Dim client = WebRequest.Create(url)
client.Method = "POST"
client.ContentType = "application/x-www-form-urlencoded"
client.ContentLength = byteArray.Length
Dim stream = client.GetRequestStream()
//sending the data
stream.Write(byteArray, 0, byteArray.Length)
stream.Close()
//getting the full response in a stream
Dim response = client.GetResponse().GetResponseStream()
//reading the response
Dim result = New StreamReader(response)
//Writes response string to Console
Console.WriteLine(result.ReadToEnd())
Console.ReadKey()
I am trying to log in to instagram using web requests. I am having a bad time understanding what's going on. Getting this: The remote server returned an error: (403) Forbidden. What I have so far:
public static string csrf;
CookieContainer c1 = new CookieContainer();
private void button1_Click(object sender, EventArgs e)
{
string PostData = String.Format("csrfmiddlewaretoken={0}&username=ra123&password=ra12345678",getToken());
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("https://instagram.com/accounts/login/");
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.KeepAlive = true;
req.AllowAutoRedirect = true;
req.CookieContainer = c1;
byte[] byteArray = Encoding.ASCII.GetBytes(PostData);
req.ContentLength = byteArray.Length;
Stream dataStream = req.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Flush();
dataStream.Close();
HttpWebResponse webResp = (HttpWebResponse)req.GetResponse();
Stream datastream = webResp.GetResponseStream();
StreamReader reader = new StreamReader(datastream);
string s = reader.ReadToEnd();
MessageBox.Show(s);
if (s.Contains("ra123"))
{
MessageBox.Show("Loggedin");
}
else
{
MessageBox.Show("Not");
}
}
string getToken()
{
string p = "<input type=\"hidden\" name=\"csrfmiddlewaretoken\" value=\"(.*)\"/>";
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("https://instagram.com/accounts/login/");
req.Method = "GET";
req.CookieContainer = c1;
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
Stream data = resp.GetResponseStream();
StreamReader sr = new StreamReader(data);
string src = sr.ReadToEnd();
Match m = Regex.Match(src, p);
if (m.Success)
{
return (m.Groups[1].Value.ToString());
}
return false.ToString();
}
The problem with the login is that the request needs to set the cookie at the header, and the container is not setting it since is changes at every login when you access from an unknown explorer. Here is what you can do:
WebResponse Response;
HttpWebRequest Request;
Uri url = new Uri("http://thewebpage.com:port/login/");
CookieContainer cookieContainer = new CookieContainer();
Request = (HttpWebRequest)WebRequest.Create(url);
Request.Method = "GET";
Request.CookieContainer = cookieContainer;
// Get the first response to obtain the cookie where you will find the "csrfmiddlewaretoken" value
Response = Request.GetResponse();
string Parametros = "csrfmiddlewaretoken=" + cookieContainer.GetCookies(url)["csrftoken"].Value + "&username=USER&password=PASSWORD&next="; // This whill set the correct url to access
Request = (HttpWebRequest)WebRequest.Create(url); // it is important to use the same url used for the first request
Request.Method = "POST";
Request.ContentType = "application/x-www-form-urlencoded";
Request.UserAgent = "Other";
// Place the cookie container to obtain the new cookies for further access
Request.CookieContainer = cookieContainer;
Request.Headers.Add("Cookie",Response.Headers.Get("Set-Cookie")); // This is the most important step, you have to place the cookies at the header (without this line you will get the 403 Forbidden exception
byte[] byteArray = Encoding.UTF8.GetBytes(Parametros);
Request.ContentLength = byteArray.Length;
Stream dataStream = Request.GetRequestStream();
dataStream.Responseite(byteArray, 0, byteArray.Length);
dataStream.Close();
Response = Request.GetResponse();
FYI, this won't solve your problem, but you need to learn to place your Stream and other objects that implement IDisposable into using blocks:
public static string csrf;
CookieContainer c1 = new CookieContainer();
private void button1_Click(object sender, EventArgs e)
{
string PostData = String.Format("csrfmiddlewaretoken={0}&username=ra123&password=ra12345678", getToken());
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("https://instagram.com/accounts/login/");
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.KeepAlive = true;
req.AllowAutoRedirect = true;
req.CookieContainer = c1;
byte[] byteArray = Encoding.ASCII.GetBytes(PostData);
req.ContentLength = byteArray.Length;
using (Stream dataStream = req.GetRequestStream())
{
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Flush();
dataStream.Close();
}
string s;
using (HttpWebResponse webResp = (HttpWebResponse)req.GetResponse())
{
using (Stream datastream = webResp.GetResponseStream())
{
using (StreamReader reader = new StreamReader(datastream))
{
s = reader.ReadToEnd();
}
}
}
MessageBox.Show(s);
if (s.Contains("ra123"))
{
MessageBox.Show("Loggedin");
}
else
{
MessageBox.Show("Not");
}
}
string getToken()
{
string p = "<input type=\"hidden\" name=\"csrfmiddlewaretoken\" value=\"(.*)\"/>";
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("https://instagram.com/accounts/login/");
req.Method = "GET";
req.CookieContainer = c1;
string src;
using (HttpWebResponse resp = (HttpWebResponse)req.GetResponse())
{
using (Stream data = resp.GetResponseStream())
{
using (StreamReader sr = new StreamReader(data))
{
src = sr.ReadToEnd();
}
}
}
Match m = Regex.Match(src, p);
if (m.Success)
{
return (m.Groups[1].Value.ToString());
}
return false.ToString();
}
I need to call a method from a webservice, so I've written this code:
private string urlPath = "http://xxx.xxx.xxx/manager/";
string request = urlPath + "index.php/org/get_org_form";
WebRequest webRequest = WebRequest.Create(request);
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.
webRequest.ContentLength = 0;
WebResponse webResponse = webRequest.GetResponse();
But this method requires some parameters, as following:
Post data:
_username:'API USER', // api authentication username
_password:'API PASSWORD', // api authentication password
How can I add these parameters into this Webrequest?
Use stream to write content to webrequest
string data = "username=<value>&password=<value>"; //replace <value>
byte[] dataStream = Encoding.UTF8.GetBytes(data);
private string urlPath = "http://xxx.xxx.xxx/manager/";
string request = urlPath + "index.php/org/get_org_form";
WebRequest webRequest = WebRequest.Create(request);
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.ContentLength = dataStream.Length;
Stream newStream=webRequest.GetRequestStream();
// Send the data.
newStream.Write(dataStream,0,dataStream.Length);
newStream.Close();
WebResponse webResponse = webRequest.GetResponse();
If these are the parameters of url-string then you need to add them through '?' and '&' chars, for example http://example.com/index.aspx?username=Api_user&password=Api_password.
If these are the parameters of POST request, then you need to create POST data and write it to request stream. Here is sample method:
private static string doRequestWithBytesPostData(string requestUri, string method, byte[] postData,
CookieContainer cookieContainer,
string userAgent, string acceptHeaderString,
string referer,
string contentType, out string responseUri)
{
var result = "";
if (!string.IsNullOrEmpty(requestUri))
{
var request = WebRequest.Create(requestUri) as HttpWebRequest;
if (request != null)
{
request.KeepAlive = true;
var cachePolicy = new RequestCachePolicy(RequestCacheLevel.BypassCache);
request.CachePolicy = cachePolicy;
request.Expect = null;
if (!string.IsNullOrEmpty(method))
request.Method = method;
if (!string.IsNullOrEmpty(acceptHeaderString))
request.Accept = acceptHeaderString;
if (!string.IsNullOrEmpty(referer))
request.Referer = referer;
if (!string.IsNullOrEmpty(contentType))
request.ContentType = contentType;
if (!string.IsNullOrEmpty(userAgent))
request.UserAgent = userAgent;
if (cookieContainer != null)
request.CookieContainer = cookieContainer;
request.Timeout = Constants.RequestTimeOut;
if (request.Method == "POST")
{
if (postData != null)
{
request.ContentLength = postData.Length;
using (var dataStream = request.GetRequestStream())
{
dataStream.Write(postData, 0, postData.Length);
}
}
}
using (var httpWebResponse = request.GetResponse() as HttpWebResponse)
{
if (httpWebResponse != null)
{
responseUri = httpWebResponse.ResponseUri.AbsoluteUri;
cookieContainer.Add(httpWebResponse.Cookies);
using (var streamReader = new StreamReader(httpWebResponse.GetResponseStream()))
{
result = streamReader.ReadToEnd();
}
return result;
}
}
}
}
responseUri = null;
return null;
}
For doing FORM posts, the best way is to use WebClient.UploadValues() with a POST method.
Hope this works
webRequest.Credentials= new NetworkCredential("API_User","API_Password");
I have a feeling that the username and password that you are sending should be part of the Authorization Header. So the code below shows you how to create the Base64 string of the username and password. I also included an example of sending the POST data. In my case it was a phone_number parameter.
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(_username + ":" + _password));
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(Request);
webRequest.Headers.Add("Authorization", string.Format("Basic {0}", credentials));
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.Method = WebRequestMethods.Http.Post;
webRequest.AllowAutoRedirect = true;
webRequest.Proxy = null;
string data = "phone_number=19735559042";
byte[] dataStream = Encoding.UTF8.GetBytes(data);
request.ContentLength = dataStream.Length;
Stream newStream = webRequest.GetRequestStream();
newStream.Write(dataStream, 0, dataStream.Length);
newStream.Close();
HttpWebResponse response = (HttpWebResponse)webRequest.GetResponse();
Stream stream = response.GetResponseStream();
StreamReader streamreader = new StreamReader(stream);
string s = streamreader.ReadToEnd();
The code below differs from all other code because at the end it prints the response string in the console that the request returns. I learned in previous posts that the user doesn't get the response Stream and displays it.
//Visual Basic Implementation Request and Response String
Dim params = "key1=value1&key2=value2"
Dim byteArray = UTF8.GetBytes(params)
Dim url = "https://okay.com"
Dim client = WebRequest.Create(url)
client.Method = "POST"
client.ContentType = "application/x-www-form-urlencoded"
client.ContentLength = byteArray.Length
Dim stream = client.GetRequestStream()
//sending the data
stream.Write(byteArray, 0, byteArray.Length)
stream.Close()
//getting the full response in a stream
Dim response = client.GetResponse().GetResponseStream()
//reading the response
Dim result = New StreamReader(response)
//Writes response string to Console
Console.WriteLine(result.ReadToEnd())
Console.ReadKey()