I've been using REST calls with my app to get and post data. I want to implement REST streaming now though so that I can immediately get any changes to the data.
I have no clue how to implement REST streaming though. How do I go about setting connection in my app. Also, do you know have any tips or best practices when using REST streaming in an app. Any help is appreciated, thanks!
I've never has a chance to try that but here is some clues that might help.
If you want to get a streamed content from your server, you can have a look at HttpClient.GetAsync(Uri, HttpCompletionOption) | getAsync(Uri, HttpCompletionOption). The available options allow you to control how the response is handled. You can for example wait for the response headers and then once available read the response content stream from the HttpResponseMessage.Content property.
You can also have a look at the IHttpFilter interface. It will allow you to "hook up" the HTTP stack and extract the response as it arrives.
Related
I'm building an bot that auto replies to tweets that contain specific words. I am using HttpWebRequest to make a request to the streaming API and posting the tweet. Only problem is that the twitter API limits the ammount of requests you can make in a specific ammount of time. This makes the stream request successful but the request to post a tweet fails with the 403 error. How can i avoid this? (Not really sure if this is actually the problem, but it seems so.)
I used this as an layout on how the stream request thingy works http://www.emoticode.net/c-sharp/twitter-stream-api-client-example.html
If you know anything about this, let me know.
According to the Twitter docs(error codes, limits) you should get an error message explaining what limit you hit (if you hit it). That should clarify if that is truly your problem.
If it is I suggest using a message queue or a response schedule, to allow you to track how many tweets you are sending. Maybe you could also assign priorities to those tweets you want to respond to, even filter out some low priority ones eventually. That depends really on the load of tweets you will be processing.
What would be the best approach to make a POST request to a web service url, in a Console App?
I tried using WebClient.uploadstring but it fails whenever the POSTed data is slightly bigger.
I tried HTTPClient but it's an async call.. so I had to additionally use ManualResetEvent to keep it alive till the server response is received..
I'd like to know what's the best way to do this. Please let me know if mpre info is required.
What do you mean by "huge"?
If you are posting large amounts of data to the URL, then you could be exceeding the maximum request size, in which case no method of requesting the URL with that amount of post data will work.
I ended up using HTTPClient's PostAsync(url, content).Result which waits till it gets a response! neat stuff..
I am attempting to scrape a website which has some kind of flash plugin which is loading data after i retrieve the html. The following object is received in the page
<OBJECT classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" WIDTH="250" HEIGHT="20" id="Preloader"><PARAM NAME="movie" VALUE="/images/preloader.swf">
<PARAM NAME="quality" VALUE="high">
<PARAM NAME**strong text**="bgcolor" VALUE="#FFFFFF"><EMBED src="/images/preloader.swf" quality="high" bgcolor="#FFFFFF" WIDTH="250" HEIGHT="20" NAME="Preloader" ALIGN="" TYPE="application/x-shockwave-flash" PLUGINSPAGE="http://www.macromedia.com/go/getflashplayer"></EMBED></OBJECT>
Ive attempted to locate the data being received on wireshark but no luck. My knowledge of this flash plugin or how it works is nil. Im guessing the worst case scenario that I will not be able to do this.
HttpWebRequest mainRequest = (HttpWebRequest)(WebRequest.Create(URL));
mainRequest.Method = "GET";
mainRequest.Proxy = null;
WebResponse mainResponse = mainRequest.GetResponse();
StreamReader dataReader = new StreamReader(mainResponse.GetResponseStream(), System.Text.Encoding.UTF8);
string data = dataReader.ReadToEnd();
dataReader.Close();
mainResponse.Close();
return data;
Does anyone know a way I can receive this data or make the webresponse wait for the data to be injected to the html before it is received. Any help would be greatly appreciated.
UPDATE:
It seems I may have jumped the gun a little with the flash object. I think this is just a loading animation while the table populates. I've been using fiddler to see what is going on. The page is returned after a request with a loading div and the flash object contained inside. A few seconds later when the data is ready another page is returned with the data. From what I can rememebr (im not at home so cannot confirm right now) the new page has the same request header as the original. Theres no json or ajax data in fiddler. Theres no script on the client to cause a refresh that I can see. I do not understand what is causing this to update.
Ive briefly looked at the web browser object but I imagine this will be quite a performance hit when im scraping about 200 pages, currently taking a minute or so. I will try the amf viewer later to confirm that the flash object is not the source of the update.
Im guessing that the server is causing this page to be resent when it has the table ready.
If the server is finding the loading div and replacing this with the table of data, would this cause the whole page to be resent? Or wouldnt this show up in ajax/json data? If it is the server resending the data, how can I keep the response open until it is ready to send the new page?
Thanks. JM.
If the content is being loaded dynamically into the Flash movie it's very likely occurring over a standard HTTP request. Wire Shark may be a little overkill for detecting something like this. I'd recommend using a utility that will capture HTTP, such as Charles, HttpFox, or screen-scraper. Using one of those tools, watch the HTTP requests that occur while the content is loading. Once you determine which request it is it's likely you can just replicate it in your code.
That said, I've also seen cases (though not very common) where the data loaded into the Flash movie is done with a binary protocol, which makes things a little more difficult. AMF is often the protocol used in these cases. Charles proxy will detect this protocol, so that may be the tool to use in this case. A while back I wrote a blog post on extracting data that's delivered via AMF. It deals with a Java library, but you may be able to find something equivalent in .NET.
You won't be able to do that with a plain HttpWebRequest because the Flash content isn't running. The response you get back is just the HTML. It requires a browser (or a browser-like object) to actually execute, load that object, and pull down the content. I know there are libraries for executing Javascript, but I don't know of anything that will let you run a Flash plugin outside of a browser.
You might be better off using a WebBrowser object. But even if it will execute the Flash content (I honestly don't know if it will), you might not be able to access it. You'll have to look at the DOM and see.
Use Firebug and / or TamperData, load your page with flash as usual, and wait until Flash makes the HTTP POST/GET for getting the data.
Flash has three options to get data:
Sockets
HTTP GET
HTTP POST
You can fool this thing any day. Just have to make sure your request contains all this little things:
Method (GET or POST)
Cookies
Form Values (why? session state, for example)
URL Referrer
User Agent
Custom HTTP-Headers? (some guys might put this in the HTTP request so no one can "fool" the server)
This could make difference of having a response with data a default html error page.
One last thing:
If the content is delivered via HTTPS, then, don't worry, it's just an extra layer somewhere but still possible.
If the content is delivered via sockets, then forget it.
I am busy working on a mobile application that retrieves data from a web service.
Ofcourse everything is working perfectly, I get everything I need to and I can consume the services without much effort... on the emulator.
However, when I move over to testing this application on the device, instead of getting back the data that I am expecting I am getting a website returned. How am I supposed to handle this?
Currently I am using this to call my service: (using system.net - I dont know if this is what I should be using on windows phone 7 either)
WebClient proxy = new WebClient();
string strURI = "http://www.google.co.za";
proxy.OpenReadCompleted +=
new OpenReadCompletedEventHandler(proxy_openreadcompleted);
proxy.OpenReadAsync(new Uri(strURI));
Please note: I am not really calling google, it is just an example. So anyways I am expecting my JSON to be returned, instead I am getting a message from the service provider to change mobile options... I can put this into isolated storage and render it with a browser, however I do not know what the source of the message is so when you click on a button, the forms use relative URLs, so instead of it doing what it is intened to do I just see what it is trying to call.
Is there anyway to get the source of the response? I am looking for a source like http://vodafonelive.mobi/ or something like that. If someone can tell me what to do I would greatly appreciate it, my current thinking is that if I can identify the source I can create a webbrowser task so that my application does not need to handle this, however... since I am calling a specific source I don't know how to identify where the response is comming from.
Any help is appreciated.
This is most likely due to differences in the user agent of the emulator and the device. Check what is sent or set this explicitly to ensure that the 2 behave in the same way. To ensure the server doesn't try and redirect to a different location.
Alternatively, it could be a mobile operator proxy being "helpful" and adjusting the request that goes over their network.
Ok so after spending some time on this, I finally found a way that I could get a response uri (where the response was comming from) by using another method to do the call to the service:
So basically this is the call:
WebRequest request = HttpWebRequest.Create(strURI);
var result = (IAsyncResult)request.BeginGetResponse(ResponseCallback, request);
So in the function ResponseCallBack function I do something like this:
WebRequest request = (HttpWebRequest)result.AsyncState;
WebResponse response = request.EndGetResponse(result);
which then allows me to check the response uri (the source of the intercepted message) and have the native browser handle the html that I was not expecting.
WebBrowserTask webBrowserTask = new WebBrowserTask();
webBrowserTask.Uri = response.ResponseUri;
Thanks for the help though, hopefully this will help someone with a similar issue.
As part of learning node.js, I just created a very basic chat server with node.js and socket.io. The server basically adds everyone who visits the chat.html wep page into a real time chat and everything seems to be working!
Now, I'd like to have a C# desktop application take part in the chat (without using a web browser control :)).
What's the best way to go about this?
I created a socket server in nodejs, and connected to it using TcpClient.
using (var client = new TcpClient())
{
client.Connect(serverIp, port));
using (var w = new StreamWriter(client.GetStream()))
w.Write("Here comes the message");
}
Try using the HttpWebRequest class. It is pretty easy to use and doesn't have any dependencies on things like System.Web or any specific web browser. I use it simulating browser requests and analyzing responses in testing applications. It is flexible enough to allow you to set your own per request headers (in case you are working with a restful service, or some other service with expectations of specific headers). Additionally, it will follow redirects for you by default, but this behavior easy to turn off.
Creating a new request is simple:
HttpWebRequest my_request = (HttpWebRequest)WebRequest.Create("http://some.url/and/resource");
To submit the request:
HttpWebResponse my_response = my_request.GetResponse();
Now you can make sure you got the right status code, look at response headers, and you have access to the response body through a stream object. In order to do things like add post data (like HTML form data) to the request, you just write a UTF8 encoded string to the request object's stream.
This library should be pretty easy to include into any WinForms or WPF application. The docs on MSDN are pretty good.
One gotcha though, if the response isn't in the 200-402 range, HttpWebRequest throws an exception that you have to catch. Fortunately you can still access the response object, but it is kind of annoying that you have to handle it as an exception (especially since the exception is on the server side and not in your client code).