Develop Reference

.NET won't download full XML response from REST API

When downloading an XML response from a REST API, I cannot get .NET to download the full XML document on many requests. In each case, I'm missing the last several characters of the XML file which means I can't parse it. The requests work fine in a browser.
I have tried WebResponse.GetResponseStream() using a StreamReader. Within the StreamReader I have tried Read(...) with a buffer, ReadLine(), and ReadToEnd() to build a string for the response. Wondering if there was a bug in my code, I also tried WebClient.DownloadString(url) with the same result and XmlDocument.Load(url) which just throws an exception (unexpected end of file while parsing ____).
I know for a fact that this API has had some encoding issues in the past, so I've tried specifying multiple different encodings (e.g., UTF-8, iso-8859-1) for the StreamReader as well as letting .NET detect the encoding. Changing the encoding seems to result in a different number of characters that get left off the end.
Is there any way I can detect the proper encoding myself? How does a browser do it? Is there somewhere in any browser to see the actual encoding the response is using (not what the HTTP headers say it's returning)? Any other methods of getting a string response from a web site with an unknown encoding?
StreamReader sample code
StringBuilder sb = new StringBuilder();
using (resp = (HttpWebResponse)req.GetResponse())
{
using (Stream stream = resp.GetResponseStream())
{
using (StreamReader sr = new StreamReader(stream))
{
int charsRead = 1;
char[] buffer = new char[4096];
while (charsRead > 0)
{
charsRead = sr.Read(buffer, 0, buffer.Length);
sb.Append(buffer, 0, charsRead);
}
}
}
}
WebClient sample code
WebClient wc = new WebClient();
string text = wc.DownloadString(url);
XmlDocument sample code
XmlDocument doc = new XmlDocument();
doc.Load(url)

Serving Binary files using HttpLIstener

I am trying to make a httplistener server in c# that sends files to the client (who is on a browser). This is my code:
static void SendFile(HttpListenerResponse response, string FileName, string ContentType) {
response.ContentType = ContentType;
// Read contents of file
var reader = new StreamReader(FileName);
var contents = reader.ReadToEnd();
reader.Close();
// Write to output stream
var writer = new StreamWriter(output);
writer.Write(contents);
// Wrap up.
writer.Close();
stream.Close();
response.Close();
}
Unfortunately, this code cannot send binary files, such as images, PDFs, and lots of other file types. How can I make this SendFile function binary-safe?

Thank you for all the comments and the gist link! The solution where you read from the file as a byte[] and write those bytes to the output stream I looked up worked, but is was kind of confusing, so I made a really short SendFile function.
static void SendFile(HttpListenerResponse response, string FileName, string ContentType) {
response.AddHeader("Content-Type", ContentType);
var output = response.OutputStream;
// Open the file
var file = new FileStream(FileName, FileMode.Open, FileAccess.Read);
// Write to output stream
file.CopyTo(output);
// Wrap up.
file.Close();
stream.Close();
response.Close();
}
This code just copies the file to the output stream.

Modifying Word Document XML using Packages

I am attempting to modify a simple MS word templates XML. I realize there are SDK's available that could make this process easier but what I am tasked with maintaining uses packages and I was told to do the same.
I have a basic test document with two placeholders mapped to the following XML:
<root>
<element>
Fubar
</element>
<second>
This is the second placeholder
</second>
</root>
What I am doing is creating a stream using the word doc, removing the existing XML, getting some hard coded test XML and trying to write that to the stream.
Here is the code I am using:
string strRelRoot = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument";
byte[] buffer = File.ReadAllBytes("dev.docx");
//stream with the template
MemoryStream stream = new MemoryStream(buffer, true);
//create a package using the stream
Package package = Package.Open(stream, FileMode.Open, FileAccess.ReadWrite);
PackageRelationshipCollection pkgrcOfficeDocument = package.GetRelationshipsByType(strRelRoot);
foreach (PackageRelationship pkgr in pkgrcOfficeDocument)
{
if (pkgr.SourceUri.OriginalString == "/")
{
Uri uriData = new Uri("/customXML/item1.xml", UriKind.Relative);
//remove the existing part
if (package.PartExists(uriData))
{
// Delete template "/customXML/item1.xml" part
package.DeletePart(uriData);
}
//create a new part
PackagePart pkgprtData = package.CreatePart(uriData, "application/xml");
//test data
string xml = #"<root>
<element>
Changed
</element>
<second>
The second placeholder changed
</second>
</root>";
//stream created from the xml string
MemoryStream fromStream = new MemoryStream();
UnicodeEncoding uniEncoding = new UnicodeEncoding();
byte[] fromBuffer = uniEncoding.GetBytes(xml);
fromStream.Write(fromBuffer, 0, fromBuffer.Length);
fromStream.Seek(0L, SeekOrigin.Begin);
Stream toStream = pkgprtData.GetStream();
//copy the xml to the part stream
fromStream.CopyTo(toStream);
//copy part stream to the byte stream
toStream.CopyTo(stream);
}
}
This is currently not modifying the document although I feel like I am close to a solution. Any advice would be very much appreciated. Thanks!
Edit: To clairify, the result I am getting is the document is unchanged. I get no exceptions or the like, but the documents XML is not modified.

OK, so not quite the timely response I promised, but here goes!
There are several aspects to the problem. Sample code is from memory and documentation, not necessarily compiled and tested.
Read the template XML
Before you delete the package part containing the template XML, you need to open its stream and read the XML. How you get the XML if the part doesn't exist to begin with is up to you.
My example code uses classes from the LINQ to XML API, though you could use whichever set of XML APIs you prefer.
XElement templateXml = null;
using (Stream stream = package.GetPart(uriData))
templateXml = XElement.Load(stream);
// Now you can delete the part.
At this point you have an in-memory representation of the template XML in templateXml.
Substitute values into the placeholders
templateXml.SetElementValue("element", "Replacement value of first placeholder");
templateXml.SetElementValue("second", "Replacement value of second placeholder");
Check out the methods on XElement if you need to do anything more advanced than this, e.g. read the original content in order to determine the replacement value.
Save the document
This is your original code, modified and annotated.
// The very first thing to do is create the Package in a using statement.
// This makes sure it's saved and closed when you're done.
using (Package package = Package.Open(...))
{
// XML reading, substituting etc. goes here.
// Eventually...
//create a new part
PackagePart pkgprtData = package.CreatePart(uriData, "application/xml");
// Don't need the test data anymore.
// Assuming you need UnicodeEncoding, set it up like this.
var writerSettings = new XmlWriterSettings
{
Encoding = Encoding.Unicode,
};
// Shouldn't need a MemoryStream at all; write straight to the part stream.
// Note using statements to ensure streams are flushed and closed.
using (Stream toStream = pkgprtData.GetStream())
using (XmlWriter writer = XmlWriter.Create(toStream, writerSettings))
templateXml.Save(writer);
// No other copying should be necessary.
// In particular, your toStream.CopyTo(stream) appeared
// to be appending the part's data to the package's stream
// (the physical file), which is a bug.
} // This closes the using statement for the package, which saves the file.

How to attach an XElement to a SMTP message as an attachment in C#

I'm trying to attach an XElement to an SMTP message i'm sending.
My code looks like this:
XElement xmlMsg = new XElement("Test",new XElement("TestSon", "DummyValue"),new XElement("TestSon2","DummyValue"));
using (MemoryStream memoryStream = new MemoryStream())
{
byte[] contentAsBytes = Encoding.Default.GetBytes(xmlMsg.ToString());
memoryStream.Write(contentAsBytes, 0, contentAsBytes.Length);
// Set the position to the beginning of the stream.
memoryStream.Seek(0, SeekOrigin.Begin);
// Create attachment
ContentType contentType = new ContentType();
contentType.MediaType = MediaTypeNames.Text.Plain;
contentType.Name = "Conversation.xml";
Attachment attachment = new Attachment(memoryStream, contentType);
mail.Attachments.Add(attachment);
Server.Send(mail);
}
However, my email is received with the XML file clipped at the end, without the last 2 chars...
Am i missing something here?
Thanks

What encoding is Encoding.Default on your system?
If it is UTF-16 I expect the two bytes are a BOM which is (for some reason) not being included in the byte count.
Suggestions:
Put xmlMsg.ToString() into a local variable, and check it in the debugger.
Have a look at the bytes in memoryStream
Look at the content of the email message in the raw, possibly copying the unencoded attachment so you can manually decode as a binary file.
Ie. check each step with as little automatic re-interpretation (eg. by an XML viewer) as possible.

StreamReader and reading an XML file

I get a response from a web-server using StreamReader... now I want to parse this response (it's an XML document file) to get its values, but every time I try to do it I get a error: Root element is missing.
If I read the same XML file directly, the file is well formatted and I can read it.
This is the stream:
WebResponse response = webRequest.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
string responseString = responseReader.ReadToEnd();
And this is how I try to read the XML file:
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.Load(responseReader);
XmlNodeList address = xmlDoc.GetElementsByTagName("original");

You have called ReadToEnd(), hence consumed all the data (into a string). This means the reader has nothing more to give. Just: don't do that. Or, do that and use LoadXml(reaponseString).

The Load method is capable of fetching XML documents from remote resources. So you could simplify your code like this:
var xmlDoc = new XmlDocument();
xmlDoc.Load("http://example.com/foo.xml");
var address = xmlDoc.GetElementsByTagName("original");
No need of any WebRequests, WebResponses, StreamReaders, ... (which by the way you didn't properly dispose). If this doesn't work it's probably because the remote XML document is not a real XML document and it is broken.

If you do it with the exact code you pasted in your question, then the problem is that you first read the whole stream into string, and then try to read the stream again when calling
xmlDoc.Load(responseReader)
If you have already read the whole stream to the string, use that string to create the xml document
xmlDoc.Load(responseString)

Check what's the content of responseString: probably it contains some additional headers that makes the xmlparser unhappy.

The error you are getting means, that XML you receive lacks first element that wraps the whole content. Try wrapping the answer you receive with some element, for example:
WebResponse response = webRequest.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
string responseString = responseReader.ReadToEnd();
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.LoadXML( "<root>" + responseString + "</root>" );
XmlNodeList address = xmlDoc.GetElementsByTagName("original")
Hope this helped