I am working on a client that uses a webservice to get some events pushed its way - the webservice is designed so, that upon the client POST'ing a subscribe command, it will send back some events of interest and keep doing so as long as the client stay connected.
When POSTing the command, the service responds (immediately) with an initial answer with these headers
Keep-Alive: timeout=5, max=98
Connection: Keep-Alive
Transfer-Encoding: chunked
and then keeps the connection open until it times out (after 30s, if the client does not send some keep-alive data)
Since it is a mix of POST + having to read the response + keeping the connection open until endOFStream, it appears I have to use HttpWebRequest with BeginGetRequestStream (to POST) and BeginGetResponse to read and act on the response.
My problem is that the BeginGetResponse callback is not called until the input stream is actually closed by the server/service (after 30s), despite AllowReadStreamBuffering being set to false.
The doc have this to say on AllowReadStreamBuffering:
The AllowReadStreamBuffering property affects when the callback from BeginGetResponse method is called. When the AllowReadStreamBuffering property is true, the callback is raised once the entire stream has been downloaded into memory. When the AllowReadStreamBuffering property is false, the callback is raised as soon as the stream is available for reading which may be before all data has arrived.
I've seen a few suggestions that no matter what AllowReadStreamBuffering is set to, HttpWebRequest will not call BeginGetResponse until it's buffer is filled up - but I have not been able to find anything on that in the docs.
Does any one have an idea on how to control this buffering behaviour or maybe suggestion to another approach I should try when dealing with this kind of webservice?
The relevant snippets of the code I currently use, look like this:
public void open()
{
string url = "http://funplaceontheinternet/webservice";
HttpWebRequest request = WebRequest.CreateHttp(url);
request.Method = "POST";
request.Credentials = new NetworkCredential("username", "password");
request.CookieContainer = new CookieContainer();
request.AllowReadStreamBuffering = false;
request.BeginGetRequestStream(new AsyncCallback(GetRequestStreamCallback), request);
}
void GetRequestStreamCallback(IAsyncResult result)
{
Debug.WriteLine("open.GetRequestStreamCallback");
HttpWebRequest webRequest = (HttpWebRequest)result.AsyncState;
// End the stream request operation
Stream postStream = webRequest.EndGetRequestStream(result);
// Create the post data
byte[] byteArray = Encoding.UTF8.GetBytes(_xmlEncodedSubscribeCommand);
// Add the post data to the web request
postStream.Write(byteArray, 0, byteArray.Length);
postStream.Close();
// Start the web request
webRequest.BeginGetResponse(new AsyncCallback(BeginGetResponseCallback), webRequest);
}
void BeginGetResponseCallback(IAsyncResult result)
{
HttpWebRequest request = (HttpWebRequest)result.AsyncState;
HttpWebResponse response = null;
if (request != null)
response = (HttpWebResponse)request.EndGetResponse(result);
else
Debug.WriteLine("request==null :-(");
if (response != null)
{
using (var reader = new StreamReader(response.GetResponseStream()))
{
while (!reader.EndOfStream)
{
string line = reader.ReadLine();
Debug.WriteLine("BeginGetResponseCallback - received: " + line);
}
Debug.WriteLine("BeginGetResponseCallback - reader.EndOfStream");
}
}
else
Debug.WriteLine("response==null :-(");
}
You've mentioned that the service is a web service, but not which platform.
If this is a "normal" web service, then I assume that XML is the transport format.
If so, I suspect the problem may be that this style of communication does not really lend itself to streaming. The web service infrastructure at the server end might not be creating the SOAP envelope and payload until all the data is available. If you wanted to stream like this, you might be better using some custom service at the server end, rather than a web service.
Do you know for sure that the server is really streaming the response? (e.g confirmed with something like wireshark?)
If you really want to use a web service, then I would suggest you complete the request when the first event(s) are available, and don't wait for the timeout. This will still achieve the latency reduction that I assume you are trying to get.
Related
I'm trying to do a little "performance Test" for SAP WebUI. Therefore I have to sign in several times with different users from a C# programm. The HTTP-traffic will be tracked by fidllercore.dll.
Now, my problem is even after two days of research I am not able to automatically sign in to the SAP WebUI. If I open a link via HttpWebRequest and submit my username and password, the response stream contains only the LogOn-Html.
I remember there should be way to sign in via SSO2-Cookies, but I couldn't find nothing in particular about this. That's pretty close to what i've done yet: Login to website, via C#
Request.BeginGetRequestStream(new AsyncCallback(GetRequestStreamCallback), Request);
private static void GetRequestStreamCallback(IAsyncResult result)
{
if (result != null)
{
//Removed password from code!
String data = "&sap-user=******&sap-password=**********";
HttpWebRequest request = (HttpWebRequest)result.AsyncState;
// End the operation
System.IO.Stream newStream = request.EndGetRequestStream(result);
// Convert the string into a byte array.
byte[] dataStream = Encoding.UTF8.GetBytes(data);
// Write to the request stream.
newStream.Write(dataStream, 0, dataStream.Length);
newStream.Close();
//Thread.Sleep(5000);
request.BeginGetResponse(new AsyncCallback(FinishWebRequest), request);
}
}
Finally fount this.
I had to perform an GET-Request with the parameters &sap-user=****&sap-password=***. It is important to set them in the right order, as you can see following the link above.
Thanks!
Edit: Due to SAP-CRM-WEB-UI sends redirect and Cookies all in same request and HttpWebRequest ignores these Cookies for redirect, you have to handle this nanually (like explained here).
how would I be able to do something like http://myanimelist.net/modules.php?go=api#verifycred
aka "curl -u user:password http://myanimelist.net/api/account/verify_credentials.xml"
I wish to option the id
my code so far is
string url = "http://myanimelist.net/api/account/verify_credentials.xml";
WebRequest request = WebRequest.Create(url);
request.ContentType = "xml";
request.Method = "GET";
request.Credentials = new NetworkCredential(username, password);
byte[] buffer = Encoding.GetEncoding("UTF-8").GetBytes("xml/user/id"); // i think this line?
Stream reqstr = request.GetRequestStream();
reqstr.Write(buffer, 0, buffer.Length);
reqstr.Close();
but I get a error on "reqstr.Write(buffer, 0, buffer.Length)"
Cannot send a content-body with this verb-type, I have tried googling but with no prevail, I am using c#
Your snippet is trying to send a GET request with request data (you're calling GetRequestStream and then writing some data to the request stream). The HTTP protocol does not allow this - you can only send data with POST request.
However, the API that you are trying to call is actually doing something different - you do not need to send it the XML data. The XML data (with user ID and user name) is the response that you get when you successfully login.
So, instead of calling GetRequestStream and writing the XML data, you need to call GetResponse and then GetResponseStream to read the XML data!
I am requesting a handler file from another handler file that returns an image, when I request my HttpWebRequest it takes more time to get the response. Here is my code, please help.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
HttpCookie cookie = context.Request.Cookies["ASP.NET_SessionID"];
Cookie myCookie = new Cookie(cookie.Name, cookie.Value);
myCookie.Domain = url.Host;
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(myCookie);
request.Timeout = 200000;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
responseStream = response.GetResponseStream();
First make sure that your request is getting to the handler as quickly as possible. If not then it is some issue with your network. You can diagnose this with logs or debugging or whatever way you need. Use Fiddler to re-issue requests so you know exactly when it is fired and received.
If it is getting to the server and no processing on there is taking too much time then make sure that you are flushing and closing the response stream when you are finished writing to it. Also probably best to dispose of the responseStream object.
using(var responseStream = response.GetResponseStream()){
// write to the sucker
responseStream.Flush();
responseStream.Close();
}
NB If this is only on the first request (your question/answers are a little confusing) then try work out what exactly is happening at startup of the appdomain. Is there something big in global.asax - or is it doing a lot of DB work?
I'm having problems with sending POST request in C# and it seems I misunderstood some HTTP basics. So basically I'm implementing RESTfull service client, which work as follows:
Make POST request with username/password and get token
Use this token in header (Authorization:TOKEN) while making other GET/POST/PUT requests
I use WebRequest to make GET requests (with Authorization header) and it's working. But when I use following code to make PUT requests, service is giving "Authentication failed - not logged in" message back:
String url = String.Format("{0}/{1}", AN_SERVER, app);
WebRequest theRequest = WebRequest.Create(url);
theRequest.Method = "POST";
theRequest.ContentType = "text/x-json";
theRequest.ContentLength = json.Length;
Stream requestStream = theRequest.GetRequestStream();
requestStream.Write(Encoding.ASCII.GetBytes(json), 0, json.Length);
requestStream.Close();
theRequest.Headers.Add("Authorization", authToken);
HttpWebResponse response = (HttpWebResponse)theRequest.GetResponse();
I must be making minor mistake (at least I hope so) while sending POST request. So what am I doing wrong?
Thanks.
Moving Headers before the request steam works (as per AI W's comment), because the request stream is adding the body.
The way webrequest is implemented internally, you need to finish the header before writing body, and once its in stream format, its ready to send.
If you look at the implementation of webrequest in reflector or some such decompiling tool, you'll be able to see the logic.
Hope this helps
I have a big problem: I need to send 200 objects at once and avoid timeouts.
while (true)
{
NameValueCollection data = new NameValueCollection();
data.Add("mode", nat);
using (var client = new WebClient())
{
byte[] response = client.UploadValues(serverA, data);
responseData = Encoding.ASCII.GetString(response);
string[] split = Javab.Split(new[] { '!' }, StringSplitOptions.RemoveEmptyEntries);
string command = split[0];
string server = split[1];
string requestCountStr = split[2];
switch (command)
{
case "check":
int requestCount = Convert.ToInt32(requestCountStr);
for (int i = 0; i < requestCount; i++)
{
Uri myUri = new Uri(server);
WebRequest request = WebRequest.Create(myUri);
request.Timeout = 200000;
WebResponse myWebResponse = request.GetResponse();
}
break;
}
}
}
This produces the error:
Unhandled Exception: System.Net.WebException: The operation has timed out
at System.Net.HttpWebRequest.GetResponse()
at vir_fu.Program.Main(String[] args)
The requestCount loop works fine outside my base code but when I add it to my project I get this error. I have tried setting request.Timeout = 200; but it didn't help.
It means what it says. The operation took too long to complete.
BTW, look at WebRequest.Timeout and you'll see that you've set your timeout for 1/5 second.
Close/dispose your WebResponse object.
I'm not sure about your first code sample where you use WebClient.UploadValues, it's not really enough to go on, could you paste more of your surrounding code? Regarding your WebRequest code, there are two things at play here:
You're only requesting the headers of the response**, you never read the body of the response by opening and reading (to its end) the ResponseStream. Because of this, the WebRequest client helpfully leaves the connection open, expecting you to request the body at any moment. Until you either read the response body to completion (which will automatically close the stream for you), clean up and close the stream (or the WebRequest instance) or wait for the GC to do its thing, your connection will remain open.
You have a default maximum amount of active connections to the same host of 2. This means you use up your first two connections and then never dispose of them so your client isn't given the chance to complete the next request before it reaches its timeout (which is milliseconds, btw, so you've set it to 0.2 seconds - the default should be fine).
If you don't want the body of the response (or you've just uploaded or POSTed something and aren't expecting a response), simply close the stream, or the client, which will close the stream for you.
The easiest way to fix this is to make sure you use using blocks on disposable objects:
for (int i = 0; i < ops1; i++)
{
Uri myUri = new Uri(site);
WebRequest myWebRequest = WebRequest.Create(myUri);
//myWebRequest.Timeout = 200;
using (WebResponse myWebResponse = myWebRequest.GetResponse())
{
// Do what you want with myWebResponse.Headers.
} // Your response will be disposed of here
}
Another solution is to allow 200 concurrent connections to the same host. However, unless you're planning to multi-thread this operation so you'd need multiple, concurrent connections, this won't really help you:
ServicePointManager.DefaultConnectionLimit = 200;
When you're getting timeouts within code, the best thing to do is try to recreate that timeout outside of your code. If you can't, the problem probably lies with your code. I usually use cURL for that, or just a web browser if it's a simple GET request.
** In reality, you're actually requesting the first chunk of data from the response, which contains the HTTP headers, and also the start of the body. This is why it's possible to read HTTP header info (such as Content-Encoding, Set-Cookie etc) before reading from the output stream. As you read the stream, further data is retrieved from the server. WebRequest's connection to the server is kept open until you reach the end of this stream (effectively closing it as it's not seekable), manually close it yourself or it is disposed of. There's more about this here.
proxy issue can cause this. IIS webconfig put this in
<defaultProxy useDefaultCredentials="true" enabled="true">
<proxy usesystemdefault="True" />
</defaultProxy>
I remember I had the same problem a while back using WCF due the quantity of the data I was passing. I remember I changed timeouts everywhere but the problem persisted. What I finally did was open the connection as stream request, I needed to change the client and the server side, but it work that way. Since it was a stream connection, the server kept reading until the stream ended.
I encountered the same error than adding
Task.Delay(2000);
in each request solved the problem