Develop Reference

XML from URL - Data at the root level is invalid. Line 1, position 1 Why it works with one URL and not the other?

As far as I can tell, these two end points are both valid XML output. However when I use the same code on the second end point I get the error:
Data at the root level is invalid. Line 1, position 1
Here is my code:
//Works
XmlDocument testDocument = new XmlDocument();
testDocument.Load("https://www.w3schools.com/xml/note.xml");
//Fails
XmlDocument testDocumentTwo = new XmlDocument();
testDocumentTwo.Load("https://www.domainNameHere.com/direct/umbraco/api/productsearch/NameSearch?countryCode=en-gb");

I opened Fiddler and watched the request and its response, and lo and behold your endpoint is returning JSON, not XML:
If I use HttpClient to set an explicit Accept header, then I get XML back and everything works:
using var client = new HttpClient();
var requestMessage = new HttpRequestMessage(HttpMethod.Get, "https://www.clinigengroup.com/direct/umbraco/api/productsearch/NameSearch?countryCode=en-gb");
requestMessage.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/xml"));
var response = await client.SendAsync(requestMessage);
var xml = await response.Content.ReadAsStringAsync();
XmlDocument testDocumentTwo = new XmlDocument();
testDocumentTwo.LoadXml(xml);

XDocument Load - cannot open

I'm trying to load rss feed by XDocument.
The url is:
http://www.ft.com/rss/home/uk
XDocument doc = XDocument.Load(url);
But I'm getting an error:
Cannot open 'http://www.ft.com/rss/home/uk'. The Uri parameter must be a file system relative or absolute path.

XDocument.Load does not take URL's, only files as stated in the documentation.
Try something like the following code which I totally did not test:
using(var httpclient = new HttpClient())
{
var response = await httpclient.GetAsync("http://www.ft.com/rss/home/uk");
var xDoc = XDocument.Load(await response.Content.ReadAsStreamAsync());
}

C# HTMLAgilityPack getting src. xpath is not valid

I am not able to get the xpath right. I am trying to get the image of any IMDB movie but it just seems not to work. This is my code of it.
// Getting the node
HtmlNode node = doc.DocumentNode.SelectSingleNode("//*[#id=\"title - overview - widget\"]/div[2]/div[3]/div[1]/a/img");
// Getting the attribute data
HtmlAttributeCollection attr = node.Attributes;
the attribute is null. every time but. the xpath does not work and i dont know why. it seems good to me.

You can use a simpler xpath
var url = "http://www.imdb.com/title/tt0816692/";
using (var client = new HttpClient())
{
var html = await client.GetStringAsync(url);
var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(html);
var img = doc.DocumentNode.SelectSingleNode("//img[#title='Trailer']")
?.Attributes["src"]?.Value;
//or
var poster = doc.DocumentNode.SelectSingleNode("//div[#class='poster']//img")
?.Attributes["src"]?.Value;
}

Run a script with API call from a webpage using C#

I want to run a script using API calls in C#. I don't want the webpage to open and just the script should run. I am trying this:
HttpWebRequest request = WebRequest.Create(URL) as HttpWebRequest;
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
HtmlDocument doc; //I have tried HtmlDocument = new HtmlDocument();, didn't work.
var resultStream = response.GetResponseStream();
doc.LoadHtml(resultStream); // I have tried using Load instead of LoadHtml,didn't work out.
doc.InvokeScript("Submit");
I get an error, use of unassigned variable doc. and doc doesn't contain function name LoadHtml. I have adding the Microsoft.VisualStudio.TestTools.UITesting.HtmlControls; , didn't help.
I have checked th questions HtmlDocument.LoadHtml from WebResponse? and Get HTML code from website in C# but they didn't get an error on doc.
Any solutions.

You will have to change the way you load the HtmlDocument
string html = new WebClient().DownloadString(URL);
WebBrowser browser = new WebBrowser()
{
ScriptErrorsSuppressed = true,
DocumentText = string.Empty
};
HtmlDocument doc = browser.Document.OpenNew(true);
doc.Write(html);
doc.InvokeScript("Submit");
Hope it works.

HtmlDocument doc; //I have tried HtmlDocument = new HtmlDocument();, didn't work.
This won't work unless you do:
HtmlDocument doc = new HtmlDocument(/*someUri*/, /*documentLocation*/);
You need to initialize doc. This is why you are seeing
use of unassigned variable doc
Check out the documentation here for details:
https://msdn.microsoft.com/en-us/library/microsoft.visualstudio.testtools.webtesting.htmldocument.aspx
According to it, the constructor signature is:
HtmlDocument(Uri, String)
with the description:
Initializes a new instance of the HtmlDocument class. This constructor takes a >string and uses it as the document.
Also, by looking at the documentation, it doesn't have a method LoadHtml()

Timeout error when loading Xml from URL

I am doing task of loading the live xml file (from live url) to XmlDataDocument, but every time I am getting error:
The operation has timed out
The code is as follows, The url containing the xml feeds , I want to load it into xmlDoc.
XmlDataDocument xmlDoc = new XmlDataDocument();
xmlDoc.Load("http://www.globalgear.com.au/productfeed.xml");
Please suggest any solution.

Don't use the Load method of the XmlDataDocument class directly; you have little to no way of influencing the behaviour when it comes to long running HTTP requests.
Instead, use the HttpWebRequest and HttpWebResponse classes to do the work for you, and then load the subsequent response into your document.
For example:
HttpWebRequest rq = WebRequest.Create("http://www.globalgear.com.au/productfeed.xml") as HttpWebRequest;
//60 Second Timeout
rq.Timeout = 60000;
//Also note you can set the Proxy property here if required; sometimes it is, especially if you are behind a firewall - rq.Proxy = new WebProxy("proxy_address");
HttpWebResponse response = rq.GetResponse() as HttpWebResponse;
XmlTextReader reader = new XmlTextReader(response.GetResponseStream());
XmlDocument doc = new XmlDocument();
doc.Load(reader);
I've tested this code in a local app instance and the XmlDocument is populated with the data from your URL.
You can also substitute in XmlDataDocument for XmlDocument in the example above - I prefer to use XmlDocument as it's not (yet) marked as obsolete.
I've wrapped this in a function for you:
public XmlDocument GetDataFromUrl(string url)
{
XmlDocument urlData = new XmlDocument();
HttpWebRequest rq = (HttpWebRequest)WebRequest.Create(url);
rq.Timeout = 60000;
HttpWebResponse response = rq.GetResponse() as HttpWebResponse;
using (Stream responseStream = response.GetResponseStream())
{
XmlTextReader reader = new XmlTextReader(responseStream);
urlData.Load(reader);
}
return urlData;
}
Simply call using:
XmlDocument document = GetDataFromUrl("http://www.globalgear.com.au/productfeed.xml");

To my knowledge there is no easy way to adjust the timeout with the method you are using.
The easiest change would be to use the webclient class and set the timeout property. This is described here http://w3ka.blogspot.co.uk/2009/12/how-to-fix-webclient-timeout-issue.html. Then use downloadfile on the webclient. Then load the saved file in the XMLDocument.

Set a timeout for your web request:
using System;
using System.Net;
using System.Xml;
namespace Shelver
{
class Program
{
static void Main(string[] args)
{
WebRequest requ = WebRequest.Create("http://www.globalgear.com.au/productfeed.xml");
requ.Timeout = 10 * 60 * 1000; // 10 minutes timeout and not 100s as the default.
var resp = requ.GetResponse();
Console.WriteLine("Will download {0:N0}bytes", resp.ContentLength);
var stream = resp.GetResponseStream();
XmlDocument doc = new XmlDocument();
doc.Load(stream);
}
}
}
This example will set it to 10 minutes.

In addition to the previous answers, which should be the first step towards fixing this, I continued to get this exception despite having already loaded the response and closing the connections.
The solution for me: the Load() and LoadXml() methods would throw their own Timeout exception if the value provided wasn't actually XML. Checking to verify that the response content was XML worked in our case (this will require that the host you are getting your response from actually sets content types).
Building upon dash's answer:
public XmlDocument GetDataFromUrl(string url)
{
XmlDocument urlData = new XmlDocument();
HttpWebRequest rq = (HttpWebRequest)WebRequest.Create(url);
rq.Timeout = 60000;
HttpWebResponse response = rq.GetResponse() as HttpWebResponse;
// New check added to dash's answer.
if (response.ContentType.Contains("text/xml")
{
using (Stream responseStream = response.GetResponseStream())
{
XmlTextReader reader = new XmlTextReader(responseStream);
urlData.Load(reader);
}
}
return urlData;
}