I am trying to use the .NET WebRequest/WebResponse classes to access the Twitter streaming API here "http://stream.twitter.com/spritzer.json".
I need to be able to open the connection and read data incrementally from the open connection.
Currently, when I call WebRequest.GetResponse method, it blocks until the entire response is downloaded. I know there is a BeginGetResponse method, but this will just do the same thing on a background thread. I need to get access to the response stream while the download is still happening. This just does not seem possible to me with these classes.
There is a specific comment about this in the Twitter documentation:
"Please note that some HTTP client libraries only return the response body after the connection has been closed by the server. These clients will not work for accessing the Streaming API. You must use an HTTP client that will return response data incrementally. Most robust HTTP client libraries will provide this functionality. The Apache HttpClient will handle this use case, for example."
They point to the Appache HttpClient, but that doesn't help much because I need to use .NET.
Any ideas whether this is possible with WebRequest/WebResponse, or do I have to go for lower level networking classes? Maybe there are other libraries that will allow me to do this?
Thx
Allen
I ended up using a TcpClient, which works fine. Would still be interested to know if this is possible with WebRequest/WebResponse though. Here is my code in case anybody is interested:
using (TcpClient client = new TcpClient())
{
string requestString = "GET /spritzer.json HTTP/1.1\r\n";
requestString += "Authorization: " + token + "\r\n";
requestString += "Host: stream.twitter.com\r\n";
requestString += "Connection: keep-alive\r\n";
requestString += "\r\n";
client.Connect("stream.twitter.com", 80);
using (NetworkStream stream = client.GetStream())
{
// Send the request.
StreamWriter writer = new StreamWriter(stream);
writer.Write(requestString);
writer.Flush();
// Process the response.
StreamReader rdr = new StreamReader(stream);
while (!rdr.EndOfStream)
{
Console.WriteLine(rdr.ReadLine());
}
}
}
BeginGetResponse is the method you need. It allows you to read the response stream incrementally:
class Program
{
static void Main(string[] args)
{
WebRequest request = WebRequest.Create("http://stream.twitter.com/spritzer.json");
request.Credentials = new NetworkCredential("username", "password");
request.BeginGetResponse(ar =>
{
var req = (WebRequest)ar.AsyncState;
// TODO: Add exception handling: EndGetResponse could throw
using (var response = req.EndGetResponse(ar))
using (var reader = new StreamReader(response.GetResponseStream()))
{
// This loop goes as long as twitter is streaming
while (!reader.EndOfStream)
{
Console.WriteLine(reader.ReadLine());
}
}
}, request);
// Press Enter to stop program
Console.ReadLine();
}
}
Or if you feel more comfortable with WebClient (I personnally prefer it over WebRequest):
using (var client = new WebClient())
{
client.Credentials = new NetworkCredential("username", "password");
client.OpenReadCompleted += (sender, e) =>
{
using (var reader = new StreamReader(e.Result))
{
while (!reader.EndOfStream)
{
Console.WriteLine(reader.ReadLine());
}
}
};
client.OpenReadAsync(new Uri("http://stream.twitter.com/spritzer.json"));
}
Console.ReadLine();
Have you tried WebRequest.BeginGetRequestStream() ?
Or something like this:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create (http://www.twitter.com );
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string str = reader.ReadLine();
while(str != null)
{
Console.WriteLine(str);
str = reader.ReadLine();
}
Related
I am trying to parse the following atom XML feed:
<dealer version="1.12" xmlns:atom="http://www.w3.org/2005/Atom"><atom:link rel="self" href="http://Blah.com/dealers/1234"/><atom:link rel="http://Blah.com/rels/dealer_notification_prefs" href="http://Blah.com/dealers/1234/notification_prefs"/><atom:link rel="http://Blah.com/rels/dealer_systems" href="http://Blah.com/dealers/1234/systems"/><atom:link rel="http://Blah.com/rels/dealer_logo" href="http://Blah.com/dealers/1234/logo"/><pid>1234</pid><name>ABC Heating & Air Conditioning</name><first>X</first><last>X</last><street1>PO Box 321</street1><street2/><city>Orson</city><state>IN</state><country>United States</country><postal>46142</postal><phone>317-555-5555</phone><phoneExt/><url></url><email>someone#noemail.com</email></dealer>
The C# code I am using is:
using (var client = new HttpClient()) // Using block for disposing of HttpClient when finished
{
client.DefaultRequestHeaders.Accept.Clear();
client.BaseAddress = new Uri(_baseUriPart); // Set to core base Uri for whole Api
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("OAuth", _builtParamsString);
// Send HTTP Requests Async
try
{
bool respSuccess = false;
HttpResponseMessage response = await client.GetAsync(_resourceUriPart);
//HttpResponseMessage response = await client.SendAsync(myRequestTest);
response.EnsureSuccessStatusCode(); // Throw Exception if not a success code. // ...}
Stream stream = await response.Content.ReadAsStreamAsync();
var prereader = new StreamReader(stream);
string readContent = prereader.ReadToEnd();
string readOut = string.Empty;
TextReader tr = new StringReader(readContent);
XmlReader reader = XmlReader.Create(tr);
SyndicationFeed feed = SyndicationFeed.Load(reader);
if(null!=feed)
{
foreach(var item in feed.Items)
{
//readOut = readOut + item.Title.Text + ":" + ((TextSyndicationContent)item.Content).Text+ Environment.NewLine;
}
}
respSuccess = response.IsSuccessStatusCode;
TextBox1.Text = respSuccess.ToString();
TextBox2.Text = response.StatusCode.ToString();
TextBox3.Text = readOut;
}
catch (HttpRequestException e)
{
TextBox1.Text = "False";
TextBox2.Text = "See Content Message";
TextBox3.Text = e.Message;
}
} // End using block
I can connect to the web service, and request the dealer info as you can see. But the error I get when the SyndicationFeed begins reading the XML is:
"The element with name 'dealer' and namespace '' is not an allowed feed format. "
Can someone please shed some light on this for me? Thanks!!!
dealer isn't a valid tag for the atom feed root. See the Atom Syndication Format RFC for details. It should be atom:feed.
It's unfortunately pretty common to find invalid RSS/Atom feeds. SyndicationFeed is strict so you have to do some massaging of the input data to get it working.
It's ugly but the simple approach is to do a String.Replace for the dealer tags.
// ...
readContent = readContent.Replace("<dealer ", "<atom:feed ").Replace("</dealer>", "</atom:feed>");
TextReader tr = new StringReader(readContent);
// ...
I've also fixed feeds in the past by deriving from XmlTextReader and fixing the bad elements as they are read.
In my code I am sending a GET request to a server. In response of this I will get one URL which I want to store in a variable.
Can anyone help me regarding this? My code so far:
private void CreatePublicUrl()
{
String CreatePublicUrl = String.Format("{0}/DataObjectServer/data/do/getproperties?cat=do&key=baseXmlPath&t={1}", base_url.Value, token);
Debug.WriteLine("CreatePublicUrl==>" + CreatePublicUrl);
HttpSyncRequest pub_url = new HttpSyncRequest();
pub_url.sendGet(CreatePublicUrl, (urlResp) =>
{
var url = new Uri(urlResp);
// String urlresponse = JsonConvert.urlResp;
});
}
It seem you're trying to achieve it using some proprietary library.
You can achieve the same using HttpWebRequestand WebResponse.
You can also do it using WebClient class as follows
WebClient client = new WebClient();
String CreatePublicUrl = String.Format("{0}/DataObjectServer/data/do/getproperties?cat=do&key=baseXmlPath&t={1}", base_url.Value, token);
using (Stream data = client.OpenRead(CreatePublicUrl))
{
using (StreamReader reader = new StreamReader(data))
{
string content = reader.ReadToEnd();
}
}
I have website A which is done in ASP.NET and it has in default.aspx
[System.Web.Services.WebMethod]
public string GetCurrentTime(string name)
{
return "Hello " + name + Environment.NewLine + "The Current Time is: "
+ DateTime.Now.ToString();
}
May we call that method somehow from another website B using C#?
Thank you!
May we call that method somehow from another website B using C#?
Yes, you can make REQUESTS to the endpoint using C#. Either GET or POST
Simple GET request
var endPoint = "http://domain.com/default.aspx";
var webReq = (HttpWebRequest)WebRequest.Create(endPoint);
using (var response = webReq.GetResponse()) {
using (var responseStream = response.GetResponseStream()) {
var reader = new StreamReader(responseStream);
var responseString = reader.ReadToEnd();
//Do whatever with responseString
}
}
Simple POST request
var endPoint = "http://domain.com/default.aspx"
var data = "param1=hello¶m2=world"
var webReq = (HttpWebRequest)WebRequest.Create(endPoint);
webReq.Method = "POST";
var bytes = Encoding.UTF8.GetBytes(data);
webReq.ContentLength = bytes.Length;
webReq.ContentType = "application/x-www-form-urlencoded";
using (var requestStream = webReq.GetRequestStream()) {
requestStream.Write(bytes, 0, bytes.Length);
}
using (var response = webReq.GetResponse()) {
using (var responseStream = response.GetResponseStream()) {
var reader = new StreamReader(responseStream);
var responseString = reader.ReadToEnd();
//Do whatever with responseString
}
}
This is a simple way of doing it. More info at MSDN.
You can use WebClient or HttpClient on the other hand. You can find example in this post also.
Yes of course, webapi is created intentionally to be called from inside the same website, another website, and from a whatever client (console, winform, wpf, mobile apps, and so on) using c# or another language.
.Net framework has avalaible various classes for calling webapi ex. HttpWebRequest, HttpClient or external libraries ex. RestSharp.
I decided that my question here isn't really what I want to do - the XML I need to send is a lot longer, potentially, than I really want to send in a URI.
It didn't "feel" right doing that, and this unsealed the deal.
I need to send both a couple of args AND a file to my Web API app from a client (handheld/CF) app.
I may have found the code for receiving that, from here [
http://www.asp.net/web-api/overview/working-with-http/sending-html-form-data,-part-2]
Specifically, Wasson's Controller code here looks like it very well might work:
public async Task<HttpResponseMessage> PostFile()
{
// Check if the request contains multipart/form-data.
if (!Request.Content.IsMimeMultipartContent())
{
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
string root = HttpContext.Current.Server.MapPath("~/App_Data");
var provider = new MultipartFormDataStreamProvider(root);
try
{
StringBuilder sb = new StringBuilder(); // Holds the response body
// Read the form data and return an async task.
await Request.Content.ReadAsMultipartAsync(provider);
// This illustrates how to get the form data.
foreach (var key in provider.FormData.AllKeys)
{
foreach (var val in provider.FormData.GetValues(key))
{
sb.Append(string.Format("{0}: {1}\n", key, val));
}
}
// This illustrates how to get the file names for uploaded files.
foreach (var file in provider.FileData)
{
FileInfo fileInfo = new FileInfo(file.LocalFileName);
sb.Append(string.Format("Uploaded file: {0} ({1} bytes)\n", fileInfo.Name, fileInfo.Length));
}
return new HttpResponseMessage()
{
Content = new StringContent(sb.ToString())
};
}
catch (System.Exception e)
{
return Request.CreateErrorResponse(HttpStatusCode.InternalServerError, e);
}
}
...but now I need to know how to send it; other calls from the client are of the form:
http://<IPAddress>:<portNum>/api/<ControllerName>?arg1=Bla&arg2=Blee
but how does the file I need to send/attach get passed along? It is a XML file, but I don't want to append the whole thing to the URI, as it can be quite large, and doing so would be horrendously weird.
Does anybody know how to accomplish this?
UPDATE
Following the crumbs tvanfosson dropped below, I found code here which I think I can adapt to work on the client:
var message = new HttpRequestMessage();
var content = new MultipartFormDataContent();
foreach (var file in files)
{
var filestream = new FileStream(file, FileMode.Open);
var fileName = System.IO.Path.GetFileName(file);
content.Add(new StreamContent(filestream), "file", fileName);
}
message.Method = HttpMethod.Post;
message.Content = content;
message.RequestUri = new Uri("http://localhost:3128/api/uploading/");
var client = new HttpClient();
client.SendAsync(message).ContinueWith(task =>
{
if (task.Result.IsSuccessStatusCode)
{
//do something with response
}
});
...but that depends on whether the Compact Framework supports MultipartFormDataContent
UPDATE 2
Which it doesn't, according to How can i determine which .Net features the compact framework has?
UPDATE 3
Using the Bing Search Code for C# extension, I mashed "h", chose "How do I", entered "send file via http" and got this:
WebRequest request = WebRequest.Create("http://www.contoso.com/PostAccepter.aspx ");
request.Method = "POST";
string postData = "This is a test that posts this string to a Web server.";
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = byteArray.Length;
Stream dataStream = request.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
WebResponse response = request.GetResponse();
Console.WriteLine(((HttpWebResponse)response).StatusDescription);
dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = reader.ReadToEnd();
Console.WriteLine(responseFromServer);
reader.Close();
dataStream.Close();
response.Close();
As I need to add a couple of string args in addition to the file (which I assume I can add via the postData byte array), can I do that by adding more calls to dataStream.Write()? IOW, is this sensible (first and third lines differ):
WebRequest request = WebRequest.Create("http://MachineName:NNNN/api/Bla?str1=Blee&str2=Bloo");
request.Method = "POST";
string postData = //open the HTML file and assign its contents to this, or make it File postData instead of string postData?
// the rest is the same
?
UPDATE 4
Progress: This, such as it is, is working:
Server code:
public string PostArgsAndFile([FromBody] string value, string serialNum, string siteNum)
{
string s = string.Format("{0}-{1}-{2}", value, serialNum, siteNum);
return s;
}
Client code (from Darin Dimitrov in this post):
private void ProcessRESTPostFileData(string uri)
{
using (var client = new WebClient())
{
client.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
var data = "=Short test...";
var result = client.UploadString(uri, "POST", data);
//try this: var result = client.UploadFile(uri, "bla.txt");
//var result = client.UploadData()
MessageBox.Show(result);
}
}
Now I need to get it sending a file instead of a string in the [FromBody] arg.
You should look into using multipart/form-data with a custom media type formatter that will extract both the string properties and the uploaded XML file.
http://lonetechie.com/2012/09/23/web-api-generic-mediatypeformatter-for-file-upload/
Problem:
I have a Java spring rest service to upload a file (large size).
I want use a .NET httpClient (or other .net client) to call upload service.
Questions:
It seems that best option to send large file is multi-part file, what's about interoperability ?
If it weren't possible, what is the best alternative ?
Thank you!
This is the answer:
I can send a file with multipart attachment from c# client to Java JAX Rest Webservice.
try
{
using (
var client = new HttpClient())
using (var form = new MultipartFormDataContent())
{
using (var stream = new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)) {
using (var fileContent = new StreamContent(stream)) {
fileContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment") {FileName = fileName, DispositionType = DispositionTypeNames.Attachment, Name = "fileData"};
form.Add(fileContent);
// only for test purposes, for stable environment, use ApiRequest class.
response = client.PostAsync(url, form).Result;
}
}
}
return response.RequestMessage != null ? response.ReasonPhrase : null;
}
catch (Exception ex)
{
TraceManager.TraceError("Post Asyn Request to " + url + " \n" + ex.Message, ex);
throw;
}
HTTP is a standard that is independent of OS platforms and programming languages, so you shouldn't have any problems with interoperability in case your .net client complies with the standards.
java spring boot
#RequestMapping(value="/upload", method=RequestMethod.POST)
public #ResponseBody String upload(#RequestParam("FileParam") MultipartFile file){
InputStream fromClient=file.getInputStream();
...do stuff with the database/ process the input file...
c#
HttpClient client = new HttpClient();
MultipartFormDataContent form = new MultipartFormDataContent();
FileInfo file = new FileInfo(#"<file path>");
form.Add(new StreamContent(file.OpenRead()),"FileParam",file.Name);
HttpResponseMessage response = await client.PostAsync("http://<host>:<port>/upload", form);
Console.WriteLine(response.StatusCode);
Console.WriteLine(response.ReasonPhrase);
Console.WriteLine(response.ToString());
Console.WriteLine(Encoding.ASCII.GetString(await response.Content.ReadAsByteArrayAsync()));