When trying to consume an RDF feed from craigslist, I'm running into a (401) Unauthorized WebException. I'm able to read the two commented out URLs directly below it with no issues. If I'm able to directly navigate to the craigslist URL using Internet Explorer with no problem, why does it fail when trying to load the data using an XmlReader?
http://portland.craigslist.org/search/sss?query=mac&srchType=A&format=rss
static void Main(string[] args)
{
XmlReader reader = XmlReader.Create("http://portland.craigslist.org/search/sss?query=mac&srchType=A&format=rss");
//XmlReader reader = XmlReader.Create("http://wdfw.wa.gov/news/newsrss.php");
//XmlReader reader = XmlReader.Create("http://rss.slashdot.org/Slashdot/slashdot");
Rss10FeedFormatter rf = new Rss10FeedFormatter();
rf.ReadFrom(reader);
Console.ReadLine();
}
}
Use XmlResolver
XmlUrlResolver resolver = new XmlUrlResolver();
resolver.Credentials = System.Net.CredentialCache.DefaultCredentials;
XmlReaderSettings settings = new XmlReaderSettings();
settings.XmlResolver = resolver;
// create a reader and populate the document
XmlReader reader = XmlReader.Create(rssFeedUrl, settings); //
doc = new XmlDocument();
doc.Load(reader);
Related
Rule:
Do not allow Dtd on XmlTextReader
Category:
Microsoft.Security.Xml
Check Id:
CA3054
Rule Description:
Prohibit DTD processing when using XmlTextReader on untrusted sources. Enabling DTD processing on the XML reader and using UrlResolver for resolving external XML entities may lead to information disclosure. Content from file system or network shares for
the machine processing the XML can be exposed to attacker. In addition, an attacker can use this as a DoS vector.
Rule File:
securityxmlrules.dll [14.0.0.0]
Help:
http://go.microsoft.com/fwlink/?LinkId=282614&clcid=0x409
XmlReaderSettings settings = new XmlReaderSettings();
settings.DtdProcessing = DtdProcessing.Prohibit;
StringReader stringReader = new StringReader(xml);
XmlTextReader reader = new XmlTextReader(stringReader);
XmlDocument doc = new XmlDocument();
doc.Load(reader);
XmlAttributeCollection attrs = doc.DocumentElement.Attributes;
if (!string.IsNullOrWhiteSpace(cookie))
{
string pagingcookie = GetPagingCookie(cookie);
if (!string.IsNullOrWhiteSpace(pagingcookie))
{
XmlAttribute pagingAttr = doc.CreateAttribute("paging-cookie");
pagingAttr.Value = pagingcookie;
attrs.Append(pagingAttr);
}
}
How to resolve this FxCop for SDL
You created a settings object but your XmlReader doesn't actually use it. You need to create an XmlReader with the settings.
XmlReaderSettings settings = new XmlReaderSettings();
settings.DtdProcessing = DtdProcessing.Prohibit;
StringReader stringReader = new StringReader(xml);
XmlReader reader = XmlReader.Create(stringReader, settings);
I have changed the code when trying to fix Veracode error for Improper Restriction of XML External Entity Reference, but it did not fix it.
Here is the code I have now:
XmlDocument xmlDoc=new XmlDocument();
using (System.IO.MemoryStream xmlstream = new System.IO.MemoryStream
(Encoding.Default.GetBytes(dsEQ.GetXml().ToString())))
{
XmlReaderSettings settings = new XmlReaderSettings();
settings.DtdProcessing = DtdProcessing.Prohibit;
using (XmlReader xmlreader = XmlReader.Create(xmlstream, settings))
{
try
{
xmlDoc.Load(xmlreader);
}
catch(XmlException e)
{
Connection.LogError(e.ToString(), e.Message);
}
}
}
However, Veracode still point out on this section of code with the same error message.
Is there anything else that I should do to fix it? We do not have any external references, everything is through intranet.
Set XmlResolver to null:
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.XmlResolver = null;
Set XmlResolver = null will fix the issue.
static void LoadXML()
{
string xml = "<?xml version=\"1.0\" ?><!DOCTYPE doc
[<!ENTITY win SYSTEM \"file:///C:/Users/user/Documents/testdata2.txt\">]
><doc>&win;</doc>";
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.XmlResolver = null; // Setting this to NULL disables DTDs - Its NOT null by default.
xmlDoc.LoadXml(xml);
Console.WriteLine(xmlDoc.InnerText);
Console.ReadLine();
}
Please go through below link for more information.
XML External Entity (XXE) Prevention Cheat Sheet
The original answer works for xmlDoc.Load(xmlreader).
The second question is a different context and requires different technology.
using (System.IO.StringReader rxml = new System.IO.StringReader(myxmltext))
{
XmlSerializer serializer = new XmlSerializer(typeof(MenuConfigBase));
using (XmlTextReader xr = new XmlTextReader(rxml))
{
xr.XmlResolver = null;
var cfgBase = (MenuConfigBase)serializer.Deserialize(xr);
}
}
I did generate C# class form scham using xsd.exe (VS 2010 command prompt),
but when I serialize class to xml file, in the out file I don't have entry for schema.
Serialized xml:
<?xml version="1.0" encoding="utf-16"?>
<Dokumenty xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" idSystemuLokalnego="ASD" dataUtworzenia="0001-01-01T00:00:00">
<Wniosek>
<Beneficjent />
</Wniosek>
When I try to validate with schema using code:
//Serilalize xml to string
StringWriter sw = new StringWriter();
XmlTextWriter xw = new XmlTextWriter(sw);
x.Serialize(xw, doc);
String xml = sw.ToString();
StringReader sr = new StringReader(xml);
XmlTextReader xtr = new XmlTextReader(sr);
XmlReaderSettings settings = new XmlReaderSettings();
settings.Schemas.Add("", "schemas\\SimWniosekApl_v2.0.xsd");
settings.ValidationType = ValidationType.Schema;
//XmlReader reader = XmlReader.Create(xtr);
XmlDocument document = new XmlDocument();
document.Load(xtr);
ValidationEventHandler eventHandler = new ValidationEventHandler(ValidationHandler);
// the following call to Validate succeeds.
document.Validate(eventHandler);
It fails with exception:
Additional information: The XmlSchemaSet on the document is either null or has no schemas in it. Provide schema information before calling Validate.
What do I do wrong?
you need to apply the XmlReaderSettings when you create the reader.
That code does nothing with the settings. The reader has been created already, before the settings are created. The code simply creates settings and then forgets them.
StringReader sr = new StringReader(xml);
//XmlTextReader xtr = new XmlTextReader(sr);
XmlReaderSettings settings = new XmlReaderSettings();
settings.Schemas.Add("", "schemas\\SimWniosekApl_v2.0.xsd");
settings.ValidationType = ValidationType.Schema;
XmlReader reader = XmlReader.Create(xtr,settings);
XmlDocument document = new XmlDocument();
document.Load(reader);
ValidationEventHandler eventHandler = new ValidationEventHandler(ValidationHandler);
document.Validate(eventHandler);
There is a full example here:
http://msdn.microsoft.com/en-us/library/ms162371.aspx
I'm trying to figure out how to correct his deprecated xml schema validation code.
public static bool ValidateXml(string xmlFilename, string schemaFilename)
{
⁞
//Forward stream reading access to data
XmlTextReader forwardStream = new XmlTextReader(xmlFilename);
//deprecated way of checking agaisnt a schema -- update.
//xmlreader class.
XmlValidatingReader validation = new XmlValidatingReader(forwardStream);
validation.ValidationType = ValidationType.Schema;
//XmlReader validator = new XmlReader.Create(
XmlSchemaCollection schemas = new XmlSchemaCollection();
schemas.Add(null, schemaFilename);
validation.Schemas.Add(schemas);
⁞
you need to use XmlReader and XmlReaderSettings instead of deprecated classes. Below is an example:
// Create the XmlSchemaSet class.
XmlSchemaSet sc = new XmlSchemaSet();
// Add the schema to the collection.
sc.Add("urn:bookstore-schema", "books.xsd");
// Set the validation settings.
XmlReaderSettings settings = new XmlReaderSettings();
settings.ValidationType = ValidationType.Schema;
settings.Schemas = sc;
settings.ValidationEventHandler += new ValidationEventHandler (ValidationCallBack);
// Create the XmlReader object.
XmlReader reader = XmlReader.Create("booksSchemaFail.xml", settings);
// Parse the file.
while (reader.Read());
more details here: Validating XML Data with XmlReader
I'm getting an "Illegal characters in path error" while using XMLTextReader method. Basically, I'm sending a long URL to tr.im, and tr.im sends the response as an XML stream, which I'm trying to parse but I get the above mentioned error. Can you guys guide me as to why I'm getting this error and where I'm going wrong? Here's the code:
WebRequest wrURL;
Stream objStream;
string strURL;
wrURL = WebRequest.Create("http://api.tr.im/api/trim_url.xml?url=" + HttpUtility.UrlEncode(txtURL.Text));
objStream = wrURL.GetResponse().GetResponseStream();
StreamReader objSReader = new StreamReader(objStream);
strURL = objSReader.ReadToEnd().ToString();
XmlTextReader reader = new XmlTextReader(strURL); //getting the error at this point
I'm using Visual Studio 2008, Express Edition
The reason why is you are using the constructor of XmlTextReader which takes a file path as the parameter but you're passing XML content instead.
Try the following code
XmlTextReader reader = new XmlTextReader(new StringReader(strURL));
XmlTextReader constructor accepts a string that points to the URL where an XML file is stored. You are passing it the XML itself which of course is an invalid path. Try this instead:
using (var client = new WebClient())
{
var xml = client.DownloadString("http://api.tr.im/api/trim_url.xml?url=" + HttpUtility.UrlEncode(txtURL.Text));
using (var strReader = new StringReader(xml))
using (var reader = XmlReader.Create(strReader))
{
}
}
The XmlTextReader(string) constructor expects a file path, not the actual XML data.
You can create an XML reader directly from the stream. The recommended way to do this is using the XmlReader.Create method:
XmlReader reader = XmlReader.Create(objStream);
You should print or otherwise display strUrl. Once you can actually see the path that you're passing to the test reader, it should be obvious what the path error is.
Also, just looking at the code, it seems like the response itself might be XML, in which case you should pass objSReader directly to the XmlTextReader constructor.
private void csv2_Load(object sender, EventArgs e)
{
DataSet ds = new DataSet();
DataSet dsSchema = new DataSet();
dsSchema.ReadXml(#"C:\Working\Teradata\ssis\Sample.xml");
StringReader sreader = new StringReader(ToXml(dsSchema));
ds.ReadXmlSchema(sreader);
ds.ReadXml(#"C:\Working\Teradata\ssis\Sample.xml");
ExportTableToCsvString(ds.Tables["session"], true, #"C:\Working\Teradata\ssis\op\session.csv");
BuildDynamicTable(ds, #"C:\Working\Teradata\ssis\op\");
}
public string ToXml(DataSet ds)
{
using (var memoryStream = new MemoryStream())
{
using
(
TextWriter streamWriter = new StreamWriter(memoryStream))
{
var xmlSerializer = new XmlSerializer(typeof(DataSet));
xmlSerializer.Serialize(streamWriter, ds);
return Encoding.UTF8.GetString(memoryStream.ToArray());
}
}
}