Long running, but finite length HttpWebRequest Callout gets only partial data - c#

I am trying to work with a 3rd party service api . One of the methods they have brings in all records which takes a lot of time , about 9 mins ( i tried this using chrome app "Advanced Rest Client").
I have tried setting the webRequest.Timeout = 3600000;// Timeout.Infinite;
But it always comes back after maybe 2 minutes and the result contains 135 records (whereas the chrome app gets back 1050 records, which is correct #)
I am using the same parameters in both cases (i send it as POST data); so why is there difference in the results ?
I am using this code in a class library which will be used in a WPF application.
Is there a max value for timeout ?
Timeout.Infinite is setting it to '-1' , is this right ?
What are the other workarounds to get all data ?
Any help/suggestion is greatly appreciated.
Update: Adding code
HttpWebRequest webRequest = (HttpWebRequest)HttpWebRequest.Create(url);
AllDevicesList devInfo = null;
try
{
string postData = "";
foreach (string key in postParameters.Keys)
{
postData += HttpUtility.UrlEncode(key) + "="
+ HttpUtility.UrlEncode(postParameters[key]) + "&";
}
postData = postData.TrimEnd('&');
if (cookie == null)
webRequest.CookieContainer = new CookieContainer();
else
webRequest.CookieContainer = cookie;
webRequest.Timeout = 3600000;// Timeout.Infinite; // 1000000;
webRequest.KeepAlive = true;
webRequest.Method = "POST";
byte[] data = Encoding.ASCII.GetBytes(postData);
webRequest.ContentType = "application/x-www-form-urlencoded";
webRequest.ContentLength = data.Length;
Stream requestStream = webRequest.GetRequestStream();
requestStream.Write(data, 0, data.Length);
WebResponse WebResp = webRequest.GetResponse();
DataContractJsonSerializer jsonSerializer = new DataContractJsonSerializer(typeof(AllDevicesList));
object objResponse = jsonSerializer.ReadObject(WebResp.GetResponseStream());
devInfo = objResponse as AllDevicesList;
requestStream.Close();
WebResp.Close();

I have changed the implementation to Asynchronous and now i get all the records.
Thanks for all your responses.

Related

Google API response time very high

I am using google RESTApis to manage google calendar
One particular api call to check current "access token is valid or not" is taking extra time(2 minutes) only in azure environment and the same call is only taking few seconds in localhost and other non azure environments.
string url = "https://www.googleapis.com/oauth2/v2/tokeninfo?access_token=" + AccessToken;
var request = (HttpWebRequest)WebRequest.Create(url);
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
byte[] bytes = Encoding.UTF8.GetBytes(string.Empty);
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls11;
using (var stream = request.GetRequestStream())
{
stream.Write(bytes, 0, bytes.Length); //Push it out there
}
var resp = request.GetResponse();
Any help would be appreciated. Thanks in advance.

HttpRequest.GetResponse() taking too much time

I have an outlook addin application which I'm connecting with Salesforce.
I'm trying to request some data from SF API which normally returns result in less than 1 second, but with my code It takes from 5-15 seconds to complete.
I have also tried to set proxy to null.
Here is my code:
System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls12;
System.Net.WebRequest req = System.Net.WebRequest.Create(URI);
if (includeCustomHeader == true)
{
req.Headers.Add("Authorization: OAuth " + ServiceManager.Token.access_token);
req.Headers.Add("X-PrettyPrint:1");
req.ContentType = "application/json";
}
else
{
req.ContentType = "application/x-www-form-urlencoded";
}
req.Method = "POST";
req.Proxy = null;
byte[] data = System.Text.Encoding.ASCII.GetBytes(payload);
req.ContentLength = data.Length;
using (var responseStream = req.GetRequestStream())
{
responseStream.Write(data, 0, data.Length);
}
//here it's taking from 5-15 seconds, each request gets a batch of 200 records
using (var response = req.GetResponse())
{
return new System.IO.StreamReader(response.GetResponseStream()).ReadToEnd();
}
Don't know what am I missing , or could there be any other reason?
Suggestions?

Setup time Metrics for Webservice Call

I am calling a webservice to send json data by post method. Here is the code for that:
string sample_url = "mywebservice/createsample;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(sample_url);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.CookieContainer = new CookieContainer();
CookieCollection cookcol = new CookieCollection();
cookcol.Add(cookie);
string body= json data
byte[] postBytes = Encoding.UTF8.GetBytes(body);
request.ContentLength = postBytes.Length;
Stream postStream = request.GetRequestStream();
postStream.Write(postBytes, 0, postBytes.Length);
postStream.Close();
WebResponse response = request.GetResponse();
string satus = ((HttpWebResponse)response).StatusDescription;
postStream = response.GetResponseStream();
StreamReader reader = new StreamReader(postStream);
I need to get some timestamp around this webservice request and response as the service seems quite slow. How do I put the countdown timer around this? Thanks for your response.
You can use the Stopwatch class:
Stopwatch sw = new Stopwatch();
sw.Start();
// your code to measure here
sw.Stop();
// print sw.Elapsed
Beware that you aren't testing your own network latency as well. If you are interested specifically in how long each service call takes, you should probably use a Stopwatch as per #nvoigt answer, but put this round your service implementation method.

How to get access token for google oauth?

I am using C# (ASP.NET). I want to use Google OAuth for accessing the user profile detail in my app. I successfully got the authorization code but having a problem in getting the access token.
I prefer the Google tutorials. In tutorial, I read that I have to send the request and get the response from google. For that, I use System.Net.HttpWebRequest/HttpWebResponse (am I going in the right way). I have used this code...
byte[] buffer = Encoding.ASCII.GetBytes("?code=" + code + "&client_id=xxx&client_secret=xxx&redirect_uri=xxxx&grant_type=authorization_code");
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://accounts.google.com");
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = buffer.Length;
Stream strm = req.GetRequestStream();
strm.Write(buffer, 0, buffer.Length);
strm.Close();
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
Response.Write(((HttpWebResponse)resp).StatusDescription);
But, I got the error:
The remote server returned an error: (405) Method Not Allowed.
Update: Here variable code is authorization code.
I think you are sending the POST request to the wrong endpoint, the correct one is https://accounts.google.com/o/oauth2/token
As I had similar problems in the process of implementing Google auth, I will post the code that works.. The last mentioned problem: error (400) Bad request could be caused by leading '?' in the above code..
string codeClient = "code="+ t +"&client_id=number.apps.googleusercontent.com&";
string secretUri = "client_secret=yoursecret&" + "redirect_uri=path&"
+ "grant_type=authorization_code";
postString = codeClient + secretUri;
string url = "https://accounts.google.com/o/oauth2/token";
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(url.ToString());
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
UTF8Encoding utfenc = new UTF8Encoding();
byte[] bytes = utfenc.GetBytes(postString);
Stream os = null;
try
{
request.ContentLength = bytes.Length;
os = request.GetRequestStream();
os.Write(bytes, 0, bytes.Length);
}
catch
{ }
try
{
HttpWebResponse webResponse = (HttpWebResponse) request.GetResponse();
Stream responseStream = webResponse.GetResponseStream();
StreamReader responseStreamReader = new StreamReader(responseStream);
result = responseStreamReader.ReadToEnd();//parse token from result
My code is working, I have done mistakes in above two lines. It should be like this
byte[] buffer = Encoding.ASCII.GetBytes("code=" + code + "&client_id=xxx&client_secret=xxx&redirect_uri=xxxx&grant_type=authorization_code");
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("https://accounts.google.com/o/oauth2/token");
Remaining code is correct.
The original request seems to be somewhat outdated. But I found that the Google's code examples contain lots of "Best Practices" housekeeping code that's hard to separate from the essential operations.
I recently published a document that represents all the REST operations as curl commands. It's hard to be conversant in every language, but curl seems universal. Most people know it- otherwise, it's pretty easy to grasp. In my curl examples, the -d flag indicates a POST operation. Otherwise, the parameters are appended to the URL.
http://www.tqis.com/eloquency/googlecalendar.htm
public string ReceiveTokenGmail(string code, string GoogleWebAppClientID, string GoogleWebAppClientSecret, string RedirectUrl)
{
string postString = "code=" + code + "&client_id=" + GoogleWebAppClientID + #"&client_secret=" + GoogleWebAppClientSecret + "&redirect_uri=" + RedirectUrl;
string url = "https://accounts.google.com/o/oauth2/token";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url.ToString());
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
UTF8Encoding utfenc = new UTF8Encoding();
byte[] bytes = utfenc.GetBytes(postString);
Stream os = null;
try
{
request.ContentLength = bytes.Length;
os = request.GetRequestStream();
os.Write(bytes, 0, bytes.Length);
}
catch
{ }
string result = "";
HttpWebResponse webResponse = (HttpWebResponse)request.GetResponse();
Stream responseStream = webResponse.GetResponseStream();
StreamReader responseStreamReader = new StreamReader(responseStream);
result = responseStreamReader.ReadToEnd();
return result;
}
It was surprisingly difficult to find the correct and simple way of getting access token by auth code. (Especially because it has taken some time for me and then even with the correct code I got “invalid_grant” error because my auth code expired while searching :) )
So here is the code:
GoogleAuthorizationCodeFlow flow = new GoogleAuthorizationCodeFlow(
new GoogleAuthorizationCodeFlow.Initializer
{
ClientSecrets = new ClientSecrets()
{
// Use ones from "Web SDK configuration" section if you created your app in Firebase.
ClientId = "…",
ClientSecret = "…"
},
Scopes = new[] { "email" },
}
);
TokenResponse token = await flow.ExchangeCodeForTokenAsync(string.Empty, "4/…", string.Empty, CancellationToken.None);
As you can see, userId can be just empty, as well as redirectUri.
Don’t forget to add the Google.Apis.Auth Nuget package reference.

getResponse in c# not working. No response coming back

I have this code in C#:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.ContentType = "application/x-www-form-urlencoded";
request.Timeout = 30000;
request.Method = "POST";
request.KeepAlive = true;
request.AllowAutoRedirect = false;
Stream newStream = request.GetRequestStream();
newStream.Write(bPostData, 0, bPostData.Length);
byte[] buf = new byte[1025]; int read = 0; string sResp = "";
HttpWebResponse wResp = (HttpWebResponse)request.GetResponse();
Stream resp = wResp.GetResponseStream();
The line HttpWebResponse wResp =... just hangs (as in no response from the URL). I'm not sure where exactly its crashing (cause i dont even get an exception error). I tested the URL in IE and it works fine. I also checked the bPostData and that one has data in it.
Where is it going wrong?
Try closing the request stream in variable newStream. Maybe the API waits for it to be done.
You have to increase the limit:
ServicePointManager.DefaultConnectionLimit = 10; // Max number of requests
Try simplifying your code and faking a user agent. Maybe the site is blocking/throttling scrapers/bots. Also ensure your application/x-www-form-urlencoded HTTP POST values are properly encoded. For this I would recommend you WebClient:
using (var client = new WebClient())
{
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0) Gecko/20100101 Firefox/4.0";
var values = new NameValueCollection
{
{ "param1", "value1" },
{ "param2", "value2" },
};
byte[] result = client.UploadValues(url, values);
}
When I commented earlier, I had run your code at my office (heavily firewalled) I got the same result you did. Came home, tried again (less firewalled) it worked fine... I'm guessing you have a barrier there. I believe you are facing a firewall issue.
Use a content-length=0
Example:
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(URL);
request.Method = "POST";
request.ContentLength = 0;
var requestStream = request.GetRequestStream();
HttpWebResponse res = (HttpWebResponse)request.GetResponse();
res.Close();

Categories

Resources