I want to generate html content based on a result returned by http url.
http://www.zillow.com/webservice/GetDeepSearchResults.htm?zws-id=X1-ZWz1c239bjatxn_5taq0&address=2114+Bigelow+Ave&citystatezip=Seattle%2C+WA
This page will give you some XML results. I want to convert to use that XML to generate HTML. I am not getting any idea where to start? Would someone offer any guidelines or sample code for asp.net?
For details: http://www.zillow.com/howto/api/GetDeepSearchResults.htm
To fetch the data you can use the HttpWebRequest class, this is an example I have to hand but it may be slightly overdone for your needs (and you need to make sure you're doing the right thing - I suspect the above to be a GET rather than a POST).
Uri baseUri = new Uri(this.RemoteServer);
HttpWebRequest rq = (HttpWebRequest)HttpWebRequest.Create(new Uri(baseUri, action));
rq.Method = "POST";
rq.ContentType = "application/x-www-form-urlencoded";
rq.Accept = "text/xml";
rq.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
Encoding encoding = Encoding.GetEncoding("UTF-8");
byte[] chars = encoding.GetBytes(body);
rq.ContentLength = chars.Length;
using (Stream stream = rq.GetRequestStream())
{
stream.Write(chars, 0, chars.Length);
stream.Close();
}
XDocument doc;
WebResponse rs = rq.GetResponse();
using (Stream stream = rs.GetResponseStream())
{
using (XmlTextReader tr = new XmlTextReader(stream))
{
doc = XDocument.Load(tr);
responseXml = doc.Root;
}
if (responseXml == null)
{
throw new Exception("No response");
}
}
return responseXml;
Once you've got the data back you need to render HTML, lots and lots of choices - if you just want to convert what you've got into HTML with minimal further processing then you can use XSLT - which is a question all on its own. If you need to do stuff with it then the question is too vague and you'll need to be more specific.
Create a xsl stylesheet, and inject the stylesheet element into the resulting xml from teh page
Related
I originally asked a question regarding a WCF web service that I was trying to write and then found that the ASP.net web API was more appropriate to my needs, due to some feedback on here.
I've now found a good tutorial that tells me how to create a simple REST service using Web API which works well pretty much out of the box.
My question
I have a POST method in my REST service server:
// POST api/values/5
public string Post([FromBody]string value)
{
return "Putting value: " + value;
}
I can POST to this using POSTER and also my C# client code.
However the bit I don't understand is why I have to prepend an '=' sign to the POST data so that it reads: "=Here is my data which is actually a JSON string"; rather than just sending: "Here is my data which is actually a JSON string";
My C# Client that talks to the REST service is written as follows:
public string SendPOSTRequest(string sFunction, string sData)
{
string sResponse = string.Empty;
// Create the request string using the data provided
Uri uriRequest = GetFormRequest(m_sWebServiceURL, sFunction, string.Empty);
// Data to post
string sPostData = "=" + sData;
// The Http Request obj
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uriRequest);
request.Method = m_VERB_POST;
System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
Byte[] byteArray = encoding.GetBytes(sPostData);
request.ContentLength = byteArray.Length;
request.ContentType = m_APPLICATION_FORM_URLENCODED;
try
{
using (Stream dataStream = request.GetRequestStream())
{
dataStream.Write(byteArray, 0, byteArray.Length);
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
StreamReader reader = new StreamReader(stream, Encoding.UTF8);
sResponse = reader.ReadToEnd();
}
}
}
catch (WebException ex)
{
//Log exception
}
return sResponse;
}
private static Uri GetFormRequest(string sURL, string sFunction, string sParam)
{
StringBuilder sbRequest = new StringBuilder();
sbRequest.Append(sURL);
if ((!sURL.EndsWith("/") &&
(!string.IsNullOrEmpty(sFunction))))
{
sbRequest.Append("/");
}
sbRequest.Append(sFunction);
if ((!sFunction.EndsWith("/") &&
(!string.IsNullOrEmpty(sParam))))
{
sbRequest.Append("/");
}
sbRequest.Append(sParam);
return new Uri(sbRequest.ToString());
}
Is anybody able to explain why I have to prepend the '=' sign as in the above code (string sPostData = "=" + sData;)?
Many thanks in advance!
The content type x-www-form-urlencoded is a key-value format. With form bodies you are only able to read a single simple type from a request body. As a name is expected, but in this case not allowed, you have to prefix the equal sign to indicate that there is no name with the followed value.
However, you should lean away from accepting simple types within the body of your web-api controller actions.
You are limited to only a single simple type if you attempt to pass data in the body of an HttpPost/HttpPut without directly implementing your own MediaTypeFormatter, which is unlikely to be reliable. Creating a light-weight complex type is generally much more preferable, and will make interoperating with other content-types, like application/json, much easier.
I am working on a c# console application where I am making a Http Post request to a web api by using xml file and I'm kind of new to XML and web services but I figured out the following code for the request but failed to pass xml data to the method
static void Main(string[] args)
{
string desturl=#"https://xyz.abcgroup.com/abcapi/";
Program p = new Program();
System.Console.WriteLine(p.WebRequestPostData(desturl, #"C:\Applications\TestService\FixmlSub.xml"));
}
public string WebRequestPostData(string url, string postData)
{
System.Net.WebRequest req = System.Net.WebRequest.Create(url);
req.ContentType = "text/xml";
req.Method = "POST";
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(postData);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
using (System.Net.WebResponse resp = req.GetResponse())
{
if (resp == null) return null;
using (System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream()))
{
return sr.ReadToEnd().Trim();
}
}
}
For obvious reasons the above code throws 404 error as I think I am not passing the xml data properly
May I know how I can fix this?
You're not posting xml, your posting the string C:\Applications\TestService\FixmlSub.xml
Change your method call from:
System.Console.WriteLine(p.WebRequestPostData(desturl, #"C:\Applications\TestService\FixmlSub.xml"));
to
var xml = XElement.Load(#"C:\Applications\TestService\FixmlSub.xml");
System.Console.WriteLine(p.WebRequestPostData(desturl, xml.ToString(SaveOptions.DisableFormatting));
If you are trying to learn post / receive, go for it. But there are open source libraries that are already well tested to use if you want to use them.
The non-free version of Servicestack.
And their older free-version. I think the older free version is great. I've never tried the newer one. You deal with objects, like say an Employee and pass that to the library and it does the translation to xml or whatever the web-service wants.
You can post whole strings if you want. They have great extension methods to help you with that too.
I have tried both WebClient's DownloadSring and WebRequest+Stream to try and scrape a page (This one) and get the Raw Paste data from it. I have scoured the net but have found no answers.
I have this code:
WebRequest request = WebRequest.Create("http://pastebin.com/raw.php?i=" + textBox1.Text);
WebResponse response = request.GetResponse();
Stream data = response.GetResponseStream();
string pasteContent = "";
using (StreamReader sr = new StreamReader(data))
{
pasteContent = sr.ReadToEnd();
}
new Note().txtMain.Text += pasteContent;
new Note().txtMain.Refresh();
and I have multiple forms so I am editing Note's txtMain textbox to add the paste content but it seems to return nothing, no matter which function I use. I know cross-form editing works because I have multiple things that can return to it.
How can I scrape the raw data?
Thank you VERY much,
P
There is no problem in downloading the content of your site. You simply doesn't use the instance of the Node class you created.
var note = new Note();
note.txtMain.Text += pasteContent;
note.Show();
I am doing task of loading the live xml file (from live url) to XmlDataDocument, but every time I am getting error:
The operation has timed out
The code is as follows, The url containing the xml feeds , I want to load it into xmlDoc.
XmlDataDocument xmlDoc = new XmlDataDocument();
xmlDoc.Load("http://www.globalgear.com.au/productfeed.xml");
Please suggest any solution.
Don't use the Load method of the XmlDataDocument class directly; you have little to no way of influencing the behaviour when it comes to long running HTTP requests.
Instead, use the HttpWebRequest and HttpWebResponse classes to do the work for you, and then load the subsequent response into your document.
For example:
HttpWebRequest rq = WebRequest.Create("http://www.globalgear.com.au/productfeed.xml") as HttpWebRequest;
//60 Second Timeout
rq.Timeout = 60000;
//Also note you can set the Proxy property here if required; sometimes it is, especially if you are behind a firewall - rq.Proxy = new WebProxy("proxy_address");
HttpWebResponse response = rq.GetResponse() as HttpWebResponse;
XmlTextReader reader = new XmlTextReader(response.GetResponseStream());
XmlDocument doc = new XmlDocument();
doc.Load(reader);
I've tested this code in a local app instance and the XmlDocument is populated with the data from your URL.
You can also substitute in XmlDataDocument for XmlDocument in the example above - I prefer to use XmlDocument as it's not (yet) marked as obsolete.
I've wrapped this in a function for you:
public XmlDocument GetDataFromUrl(string url)
{
XmlDocument urlData = new XmlDocument();
HttpWebRequest rq = (HttpWebRequest)WebRequest.Create(url);
rq.Timeout = 60000;
HttpWebResponse response = rq.GetResponse() as HttpWebResponse;
using (Stream responseStream = response.GetResponseStream())
{
XmlTextReader reader = new XmlTextReader(responseStream);
urlData.Load(reader);
}
return urlData;
}
Simply call using:
XmlDocument document = GetDataFromUrl("http://www.globalgear.com.au/productfeed.xml");
To my knowledge there is no easy way to adjust the timeout with the method you are using.
The easiest change would be to use the webclient class and set the timeout property. This is described here http://w3ka.blogspot.co.uk/2009/12/how-to-fix-webclient-timeout-issue.html. Then use downloadfile on the webclient. Then load the saved file in the XMLDocument.
Set a timeout for your web request:
using System;
using System.Net;
using System.Xml;
namespace Shelver
{
class Program
{
static void Main(string[] args)
{
WebRequest requ = WebRequest.Create("http://www.globalgear.com.au/productfeed.xml");
requ.Timeout = 10 * 60 * 1000; // 10 minutes timeout and not 100s as the default.
var resp = requ.GetResponse();
Console.WriteLine("Will download {0:N0}bytes", resp.ContentLength);
var stream = resp.GetResponseStream();
XmlDocument doc = new XmlDocument();
doc.Load(stream);
}
}
}
This example will set it to 10 minutes.
In addition to the previous answers, which should be the first step towards fixing this, I continued to get this exception despite having already loaded the response and closing the connections.
The solution for me: the Load() and LoadXml() methods would throw their own Timeout exception if the value provided wasn't actually XML. Checking to verify that the response content was XML worked in our case (this will require that the host you are getting your response from actually sets content types).
Building upon dash's answer:
public XmlDocument GetDataFromUrl(string url)
{
XmlDocument urlData = new XmlDocument();
HttpWebRequest rq = (HttpWebRequest)WebRequest.Create(url);
rq.Timeout = 60000;
HttpWebResponse response = rq.GetResponse() as HttpWebResponse;
// New check added to dash's answer.
if (response.ContentType.Contains("text/xml")
{
using (Stream responseStream = response.GetResponseStream())
{
XmlTextReader reader = new XmlTextReader(responseStream);
urlData.Load(reader);
}
}
return urlData;
}
I am writting a winforms c# 2.0 application that needs to put an XML file into a document library on SharePoint.
I want to use a WebService instead of using the object model (no sharepoint.dll to reference here)
I am currently using the http://webserver/site/_vti_bin/copy.asmx webservice.
Here is some code:
byte[] xmlByteArray;
using (MemoryStream memoryStream = new MemoryStream())
{
xmlDocument.Save(memoryStream);
xmlBytes = memoryStream.ToArray();
}
string[] destinationUrlArray = new string[] {"http://webserver/site/Doclib/UploadedDocument.xml"};
FieldInformation fieldInfo = new FieldInformation();
FieldInformation[] fields = { fieldInfo };
CopyResult[] resultsArray;
using (Copy copyService = new Copy())
{
copyService.Credentials = CredentialCache.DefaultCredentials;
copyService.Url = "http://webserver/site/_vti_bin/copy.asmx";
copyService.Timeout = 600000;
uint documentId = copyService.CopyIntoItems("", destinationUrlArray, fields, xmlByteArray, out resultsArray);
}
When this code runs, I get a single result in the resultsArray out parameter:
DestinationURL: "http://webserver/site/Doclib/UploadedDocument.xml"
ErrorCode: UnKnown
ErrorMessage: "Object reference not set to an instance of an object."
From my searching, I have found a couple of possible helps.
Microsoft TechNet -- "The copy.asmx copyintoitems will only work if the source and destination urls are in the same SPWebApplication (Site Collection)."
Microsoft Social -- "Object reference not set to an instance of an object
error occurs because of SharePoint not able to identified that particular property."
This leads me to believe my source url should be set to something, but what? This is originating from a client workstation and does not have a source URL.
Any help would be appricated.
hank you,
Keith
I know this is an old thread but it kept coming up as I was searching for a solution to the same problem.
Check Steve Curran's answer on this thread http://social.msdn.microsoft.com/Forums/en-SG/sharepointdevelopment/thread/833e38a8-f13c-490d-8ba7-b889b6b25e38. Looks like Basically the request fails because the destination url can't be resolved.
(Limitations of a new stackflow user - can't post more than one link. See my comment for the rest)
pat
SharePoint responds to a plain old HTTP PUT
Here is what is currently working:
WebRequest request = WebRequest.Create(“http://webserver/site/Doclib/UploadedDocument.xml”);
request.Credentials = CredentialCache.DefaultCredentials;
request.Method = "PUT";
byte[] buffer = new byte[1024];
using (Stream stream = request.GetRequestStream())
{
using (MemoryStream memoryStream = new MemoryStream())
{
dataFile.MMRXmlData.Save(memoryStream);
memoryStream.Seek(0, SeekOrigin.Begin);
for (int i = memoryStream.Read(buffer, 0, buffer.Length); i > 0;
i = memoryStream.Read(buffer, 0, buffer.Length))
{
stream.Write(buffer, 0, i);
}
}
}
WebResponse response = request.GetResponse();
response.Close();
So... Does anyone have an opinion as to if this "PUT" method is better in the SharePoint environment than using a built-in webservice?
Right now I would have to say the "PUT" method is better since it works and I could not get the WebService to work.
Keith
your code is fine, just use the destination url instead of an empty string. See below:
byte[] xmlByteArray;
using (MemoryStream memoryStream = new MemoryStream())
{
xmlDocument.Save(memoryStream);
xmlBytes = memoryStream.ToArray();
}
string destinationUrl = “http://webserver/site/Doclib/UploadedDocument.xml”
string[] destinationUrlArray = new string[] { destinationUrl };
FieldInformation fieldInfo = new FieldInformation();
FieldInformation[] fields = { fieldInfo };
CopyResult[] resultsArray;
using (Copy copyService = new Copy())
{
copyService.Credentials = CredentialCache.DefaultCredentials;
copyService.Url = "http://webserver/site/_vti_bin/copy.asmx";
copyService.Timeout = 600000;
uint documentId = copyService.CopyIntoItems(destinationUrl , destinationUrlArray, fields, xmlByteArray, out resultsArray);
}
I get the same message when I use the default credentials.
Try replacing them with this:
copyWebService.Credentials
= new NetworkCredential("Administrator", "pass", "MyDomain");
Here's some code I wrote awhile (i apologize, i've had to piece meal it together, but hopefully you get the point of it)
// Create a request using a URL that can receive a post.
WebRequest request = WebRequest.Create("http://sharepointsite/somefile.txt");
// Set the Method property of the request to POST.
request.Method = "PUT"
Stream dataStream;
// Set the ContentType property of the WebRequest.
request.ContentType = "multipart/form-data; charset=ISO-8859-1";
byte[] byteArray = File.ReadAllBytes(#"c:\somefile.txt");
// Set the ContentLength property of the WebRequest.
request.ContentLength = byteArray.Length;
// Get the request stream.
dataStream = request.GetRequestStream();
// Write the data to the request stream.
dataStream.Write(byteArray, 0, byteArray.Length);
// Close the Stream object.
dataStream.Close();
// Get the response.
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
HttpStatusCode statCode = response.StatusCode;
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd();
// Clean up the streams.
reader.Close();
dataStream.Close();
response.Close();
I'm not sure if it will solve your problem but, when you reference the webservice, don't use the [site] part of the URL.
Try instead: http://[server]/_vti_bin/[webservice].
I'm not an expert in SP but I'm pretty sure the webservices belongs to the main server, not to an especific site.
Hope it helps.
I had a similiar problem, it turned out that the the client was configured to use NTLM security, but no NTLM header was attached.
I my case, becuase of the fact that I was using this code on the server-side of an ASP.NET applicaton, was to enable Windows authentication and set
identity impersonate="true"
in the server.web section.
if your sharepoint server is built on a farm,
Check your "Alternate Access Mapping" see if there is an entry:
yourwebserverurl intranet yourwebserverurl
if not, add it.
for my case, after adding this, the Copy service start working.
It probably due to farm load balance address resolve related.
I don't get it, why are you using Copy rather then UpdateListItems. Perhaps UpdateListItems will be a better match?