I made a Function for a program, which does work when the Request Type is GET, if it is POST, it always produces a Timeout Exception(and the timeout of 50s wasnt reached) on the Line HttpWebResponse response = (HttpWebResponse)request.GetResponse();
I tried many things, but I doesnt found out why, may someone here know it.
Edit: Got it to work, if someone is interested: https://gist.github.com/4347248
Any help will be greatly appreciated.
My Code is:
public ResRequest request(string URL, RequestType typ, CookieCollection cookies, string postdata ="", int timeout= 50000)
{
byte[] data;
Stream req;
Stream resp;
HttpWebRequest request = WebRequest.Create(URL) as HttpWebRequest;
request.Timeout = timeout;
request.ContinueTimeout = timeout;
request.ReadWriteTimeout = timeout;
request.Proxy = new WebProxy("127.0.0.1", 8118);
request.Headers.Add(HttpRequestHeader.AcceptLanguage, "de");
request.Headers.Add("UA-CPU", "x86");
request.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip, deflate");
request.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; WOW64; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.21022; .NET CLR 3.5.30729; .NET CLR 3.0.30618) ";
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(cookies);
if (typ == RequestType.POST)
{
data = System.Text.Encoding.Default.GetBytes(postdata);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = data.Length;
req = request.GetRequestStream();//after a few tries this produced a Timeout error
req.Write(data, 0, data.Length);
req.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();//This line produces a Timeout Exception
resp = response.GetResponseStream();
if ((response.ContentEncoding.ToLower().Contains("gzip")))
{
resp = new System.IO.Compression.GZipStream(resp, System.IO.Compression.CompressionMode.Decompress);
} else if ((response.ContentEncoding.ToLower().Contains("deflate"))) {
resp = new System.IO.Compression.DeflateStream(resp, System.IO.Compression.CompressionMode.Decompress);
}
return new ResRequest() { result = new System.IO.StreamReader(resp, System.Text.Encoding.UTF8).ReadToEnd(), cookies = response.Cookies, cstring = cookiestring(response.Cookies) };
}
else
{
request.Method = "GET";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
resp = response.GetResponseStream();
if ((response.ContentEncoding.ToLower().Contains("gzip")))
{
resp = new System.IO.Compression.GZipStream(resp, System.IO.Compression.CompressionMode.Decompress);
}
else if ((response.ContentEncoding.ToLower().Contains("deflate")))
{
resp = new System.IO.Compression.DeflateStream(resp, System.IO.Compression.CompressionMode.Decompress);
}
return new ResRequest() { result = new System.IO.StreamReader(resp, System.Text.Encoding.UTF8).ReadToEnd(), cookies = response.Cookies, cstring = cookiestring(response.Cookies) };
}
}
So does it hang on req.GetRequestStream() every time, or does it work "a few tries" and then hang?
If it works a few times and then hangs, it's possible that you're not closing the requests properly, which is causing you to run out of connections. Make sure to Close() and/or Dispose() the HttpWebResponse objects and all of the Streams and Readers that you're creating.
You have to use
response.Dispose();
end of the method
Related
I am working on simple web application where I need to get access token from outlook API in my app to use employee name and its image.I have written code and and able to login through outlook but my access token is coming as null.Please find my code below:
public void GetResponseCallback(IAsyncResult asynchronousResult)
{
HttpWebRequest request = (HttpWebRequest)asynchronousResult.AsyncState;
// End the operation
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asynchronousResult);
Stream streamResponse = response.GetResponseStream();
StreamReader streamRead = new StreamReader(streamResponse);
string responseString = streamRead.ReadToEnd();
access_token = responseString;
// Close the stream object
streamResponse.Close();
streamRead.Close();
// Release the HttpWebResponse
response.Close();
allDone.Set();
}
HttpWebRequest request =(HttpWebRequest)WebRequest.Create("some url");
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.UserAgent = "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 7.1; Trident/5.0)";
request.Accept = "/";
request.UseDefaultCredentials = true;
request.Proxy.Credentials = System.Net.CredentialCache.DefaultCredentials;
doc.Save(request.GetRequestStream());
HttpWebResponse resp = request.GetResponse() as HttpWebResponse;
Hope it helps
I'm trying to connect to the Lighthouse api using C# code. This is the php example https://support.lighthouserocks.com/hc/en-gb/articles/201319732-API-The-Basics which it describe how to do this. But I'm fails with it. I was try both NetworkCredentials & send in the Header but still have 401 Unauthorized access to it, here is the code:
public string RequestResponse()
{
HttpWebRequest webRequest = WebRequest.Create(HomeUrl) as HttpWebRequest;
webRequest.Method = "GET";
webRequest.ContentType = "application/json";
webRequest.ServicePoint.Expect100Continue = false;
webRequest.Timeout = 20000;
string auth = CreateAuthorization("domain.lhlive.com", "user", "token");
webRequest.Headers["auth"] = "Basic " + auth;
//webRequest.Credentials = new NetworkCredential("user", "token");
//webRequest.PreAuthenticate = true;
//webRequest.Headers.Add("auth", "user, token");
webRequest.Accept = "application/vnd.lighthouse.v1.hal+json";
Stream responseStream = null;
StreamReader responseReader = null;
string responseData = "";
try
{
WebResponse webResponse = webRequest.GetResponse();
responseStream = webResponse.GetResponseStream();
responseReader = new StreamReader(responseStream);
responseData = responseReader.ReadToEnd();
}
finally
{
if (responseStream != null)
{
responseStream.Close();
responseReader.Close();
}
}
return responseData;
}
public void Test()
{
using (var client = new WebClient())
{
client.Headers["User-Agent"] = "Mozilla/4.0 (Compatible; Windows NT 5.1; MSIE 6.0) (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)";
client.Headers["ContentType"] = "application/json";
client.Headers["Accept"] = "application/vnd.lighthouse.v1.hal+json";
//client.Headers["Lighthouse Username"] = "user";
//client.Headers["API Key"] = "token";
client.Headers["WWW-Authenticate"] = "user, token";
byte[] arr = client.DownloadData("https://domain.lhlive.com/contacts");
Console.WriteLine("--- WebClient result ---");
Console.WriteLine(arr.Length);
}
}
anybody know what I should to do?
Hard to say because I can't access Lighthouse but try the following (notice how auth header is set).
var webRequest = WebRequest.Create("http://some.endpoint.com/") as HttpWebRequest;
webRequest.Method = "GET";
webRequest.Accept = "application/vnd.lighthouse.v1.hal+json";
webRequest.ContentType = "application/json";
webRequest.Headers["Authorization"] = string.Format("Basic {0}",
Convert.ToBase64String(Encoding.Default.GetBytes(
string.Format("{0}:{1}", "your username", "your API key"))));
var response = webRequest.GetResponse();
var stream = response.GetResponseStream();
var data = (new StreamReader(stream)).ReadToEnd();
In this case your Authorization header looks like "Basic eW91ciB1c2VybmFtZTp5b3VyIEFQSSBrZXk=".
I am hitting a url as sample below:
http://mobile.example.com/ip/someProduct-fl-oz/productID
And my Request is :
public static string getMobileHtml(string url)
{
string responseData = "";
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Accept = "text/html, application/xhtml+xml, */*";
request.KeepAlive = true;
request.UserAgent = "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)";
request.Timeout = 10000;
request.Host = "mobile.example.com";
request.Method = "GET";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
Stream responseStream = response.GetResponseStream();
StreamReader myStreamReader = new StreamReader(responseStream);
responseData = myStreamReader.ReadToEnd();
}
response.Close();
}
catch (Exception e)
{
responseData = "An error occurred: " + e.Message;
}
return responseData;
}
And the code crashes on line:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
Without any exception just breaks off, same thing was happening in Curl Lib,WebClient but i changed it to HttpWebRequest assuming it would be a fix. Any suggestions?
I tested your posted code ahd the HttpWebRequest was created succesfully, it then failed at the response line, but in my case is because I am behind a firewall and I did not provided any credentials.
The problem was diagnosed as Fiddler affecting your tests, so closing it solve the problem
I'm trying to write a bit of code to login to a website. But it's not working. Please can you give me some advice. This is my a bit of code:
static void Main(string[] args)
{
CookieContainer container = new CookieContainer();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://pagehype.com/login.php");
request.Method = "POST";
request.Timeout = 10000;
request.ReadWriteTimeout = 30000;
request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) (Prevx 3.0.5)";
request.CookieContainer = container;
ASCIIEncoding encoding = new ASCIIEncoding();
string postData = "username=user&password=password&processlogin=1&return=";
byte[] data = encoding.GetBytes(postData);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = data.Length;
Stream newStream = request.GetRequestStream();
newStream.Write(data, 0, data.Length);
newStream.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string htmldoc = reader.ReadToEnd();
response.Close();
Console.Write(htmldoc);
}
Many thanks,
Use http://www.fiddler2.com/fiddler2/ to view the http request sent went you login in using a browser and ensure that the request you build in code is the same.
PHP logins use PHPSESSID cookie. You'll need to capture this and pass it back in the CookieContainer. This is how the server will recognise you as an authenticated user.
The cookie is set in the Set-Cookie header in the initial response. You'll need to parse it to recreate the cookie in your container (don't forget the path (and domain?)
var setCookie = response.GetResponseHeader("Set-Cookie");
response.Close();
container = new CookieContainer();
foreach (var cookie in setCookie.Split(','))
{
var split = cookie.Split(';');
string name = split[0].Split('=')[0];
string value = split[0].Split('=')[1];
var c = new Cookie(name, value);
if (cookie.Contains(" Domain="))
c.Domain = split.Where(x => x.StartsWith(" Domain")).First().Split('=')[1];
else
{
c.Domain = ".pagehype.com";
}
if (cookie.Contains(" Path="))
c.Path = split.Where(x => x.StartsWith(" Path")).First().Split('=')[1];
container.Add(c);
}
Then add this container to your request.
I have this code in C#:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.ContentType = "application/x-www-form-urlencoded";
request.Timeout = 30000;
request.Method = "POST";
request.KeepAlive = true;
request.AllowAutoRedirect = false;
Stream newStream = request.GetRequestStream();
newStream.Write(bPostData, 0, bPostData.Length);
byte[] buf = new byte[1025]; int read = 0; string sResp = "";
HttpWebResponse wResp = (HttpWebResponse)request.GetResponse();
Stream resp = wResp.GetResponseStream();
The line HttpWebResponse wResp =... just hangs (as in no response from the URL). I'm not sure where exactly its crashing (cause i dont even get an exception error). I tested the URL in IE and it works fine. I also checked the bPostData and that one has data in it.
Where is it going wrong?
Try closing the request stream in variable newStream. Maybe the API waits for it to be done.
You have to increase the limit:
ServicePointManager.DefaultConnectionLimit = 10; // Max number of requests
Try simplifying your code and faking a user agent. Maybe the site is blocking/throttling scrapers/bots. Also ensure your application/x-www-form-urlencoded HTTP POST values are properly encoded. For this I would recommend you WebClient:
using (var client = new WebClient())
{
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0) Gecko/20100101 Firefox/4.0";
var values = new NameValueCollection
{
{ "param1", "value1" },
{ "param2", "value2" },
};
byte[] result = client.UploadValues(url, values);
}
When I commented earlier, I had run your code at my office (heavily firewalled) I got the same result you did. Came home, tried again (less firewalled) it worked fine... I'm guessing you have a barrier there. I believe you are facing a firewall issue.
Use a content-length=0
Example:
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(URL);
request.Method = "POST";
request.ContentLength = 0;
var requestStream = request.GetRequestStream();
HttpWebResponse res = (HttpWebResponse)request.GetResponse();
res.Close();