Develop Reference

Json deserialization error when mapping object to array within JSON

Good Afternoon all,
I am writing a program that calls and API, maps the json response to a corresponding data model. then will ultimately write those maps to disk.
The following code snippet is the main entry point for the program.
public static async Task Main(string[] args)
{
try
{
await WriteFile(2010);
}
catch (Exception ex)
{
Console.WriteLine($"There was an exception: {ex.ToString()}");
}
}
public static async Task WriteFile(int year)
{
ApiHelper.InitializeClient();
var data = await DataLoader.LoadData(year);
StreamWriter writer = new StreamWriter(#"Data.txt");
writer.WriteLine(data.Id);
writer.Close();
}
The DataLoader is as follows
public static async Task<BigDataModel> LoadData(int year)
{
using (HttpResponseMessage response = await ApiHelper.ApiClient.GetAsync(GetUri(year)))
{
if (response.IsSuccessStatusCode)
{
DataModel data = await response.Content.ReadAsAsync<DataModel>();
return data.Meta;
}
else
{
throw new Exception(response.ReasonPhrase);
}
}
}
Within the if statement the program returns an exception regarding my json format being incorrect and needing to be deserialized. I have tested this system without attempting to map and array and it works correctly.
My Data Models are as followed
public class DataModel
{
public BigDataModel Meta { get; set; }
}
public class BigDataModel
{
public int Id { get; set; }
}
expected Json response
{
"projected":"value",
"actual":"value",
"meta":["data":"1","id":"data"]
}
What are the best practices when mapping a json object(array) to an object.
Is deserialization a must or can I create a work around to write the data to disk.

Is it possible to return a response DTO that matches what is expected in the Contract using IErrorHandler

I want to be able to catch all unhandled exceptions and return the expected DTO but with some error information filled in. For example
public class CreateFooRequest
{
public string Name { get; set; }
}
public class CreateFooResponse
{
public Foo Created { get; set; }
public string Error { get; set; } // If call was successful then this will be null
public string Detail { get; set; }
}
public interface IFooService
{
CreateFooResponse Create(CreateFooRequest request);
}
public ErrorHandler: IErrorHandler
{
public bool Handle(Exception ex)
{
return true;
}
public void ProvideFault(Exception error, MessageVersion version, ref Message fault)
{
// Some how figure out that IFooService.Create was called.
// Inspect the method signature and see that there is an input called CreateFooRequest
// Use reflection to initialize response objects that will replace the "Request" with "Response"
var response = new CreateFooResponse();
response.Error = error.GetType().Name;
// I think i need one of the following overloads
fault = Message.CreateMessage(version, action, response);
}
}
Is doing something like this even possible? We are using NetTCP as our binding if that makes a difference.

IErrorHandler is designed to generate fault contracts. If you don't want to return a fault, you are better off taking an interceptor approach and using a the IOperationInvoker extension point instead.
The operation invoker is the part of the WCF framework that actually calls the service method. When you extend it, you can effectively "intercept" the call to the service. Note the Invoker does have responsibilities. You can't simply replace the WCF invoker implementation. Instead you chain them (see below).
At a high level the steps are:
Create an IOperationInvoker.Invoke() that implements a try/catch block. Catch the exceptions you want to be Response messages instead of FaultExceptions.
Create an IOperationBehavior (that's optionally also an attribute) to apply the behavior to your service.
The approach has a couple of advantages:
The exception is caught before WCF sees it.
In a IOperationBehavior.ApplyDispatchBehavior() you have access to the OperationDescription at the time of service start up. If you save this in the Invoker, you don't need to use reflection to capture the method's return type.
The IOperationBehavior.Validate() allows robust checks to ensure that the return type can actually be processed.
Below is a complete windows console application that demonstrates the approach. Paste it into Visual Studio, add the obvious assembly references, and run it.
My apologizes for the large amount of code and excessive use of base classes. I'm cutting down from actual production code.
If you want a better understanding of the IOperationInvoker extension point and what the example InvokerBase class is doing see Carlos Figueira's blog.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Description;
using System.ServiceModel.Dispatcher;
using System.Text;
using System.Threading.Tasks;
namespace WcfErrorResponse
{
/// <summary>
/// Provides a base IOperationInvoker implementation that stores and passes through calls to the exisiting (old) invoker
/// </summary>
public abstract class InvokerBase : IOperationInvoker
{
private readonly IOperationInvoker m_OldInvoker;
protected IOperationInvoker OldInvoker
{
get { return m_OldInvoker; }
}
public InvokerBase(IOperationInvoker oldInvoker)
{
m_OldInvoker = oldInvoker;
}
public virtual object[] AllocateInputs()
{
return OldInvoker.AllocateInputs();
}
public virtual object Invoke(object instance, object[] inputs, out object[] outputs)
{
return OldInvoker.Invoke(instance, inputs, out outputs);
}
public virtual IAsyncResult InvokeBegin(object instance, object[] inputs, AsyncCallback callback, object state)
{
return OldInvoker.InvokeBegin(instance, inputs, callback, state);
}
public virtual object InvokeEnd(object instance, out object[] outputs, IAsyncResult result)
{
return OldInvoker.InvokeEnd(instance, out outputs, result);
}
public virtual bool IsSynchronous
{
get { return OldInvoker.IsSynchronous; }
}
}
/// <summary>
/// Base implementation for a Method level attribte that applies a <see cref="InvokerBase"/> inherited behavior.
/// </summary>
[AttributeUsage(AttributeTargets.Method)]
public abstract class InvokerOperationBehaviorAttribute : Attribute, IOperationBehavior
{
protected abstract InvokerBase CreateInvoker(IOperationInvoker oldInvoker, OperationDescription operationDescription, DispatchOperation dispatchOperation);
public void AddBindingParameters(OperationDescription operationDescription, System.ServiceModel.Channels.BindingParameterCollection bindingParameters)
{ }
public void ApplyClientBehavior(OperationDescription operationDescription, ClientOperation clientOperation)
{ }
public virtual void ApplyDispatchBehavior(OperationDescription operationDescription, DispatchOperation dispatchOperation)
{
// chain invokers.
IOperationInvoker oldInvoker = dispatchOperation.Invoker;
dispatchOperation.Invoker = CreateInvoker(oldInvoker, operationDescription, dispatchOperation);
}
public virtual void Validate(OperationDescription operationDescription)
{
return;
}
}
public class ResponseExceptionInvoker : InvokerBase
{
private Type returnType;
public ResponseExceptionInvoker(IOperationInvoker oldInvoker, OperationDescription operationDescription)
: base(oldInvoker)
{
// save the return type for creating response messages
this.returnType = operationDescription.GetReturnType();
if (this.returnType == null)
{
throw new InvalidOperationException("The operation '" + operationDescription.SyncMethod.DeclaringType.Name + "' does not define a return type.");
}
}
public override object Invoke(object instance, object[] inputs, out object[] outputs)
{
object returnedValue = null;
object[] outputParams = new object[] { };
outputs = new object[] { };
try
{
returnedValue = OldInvoker.Invoke(instance, inputs, out outputParams);
outputs = outputParams;
return returnedValue;
}
catch (Exception ex)
{
Logger.Debug("ResponseExceptionInvoker() - Caught Exception. A Response Message will be returned. Message='" + ex.Message + "'");
// there was an excpetion. Do not assign output params... their state is undefined.
//outputs = outputParams;
try
{
// assumes the behavior only used for return types that inherit from Response, as verified by ResponseExceptionOperationBehaviorAttribute.Validate()
Response response = (Response)Activator.CreateInstance(this.returnType);
response.Success = false;
response.ErrorMessage = ex.Message;
return response;
}
catch (Exception exCreateResponse)
{
// Log that the Response couldn't be created and throw the original exception.
// Probably preferable to wrap and throw.
Logger.Error("Caught ResponseException, but unable to create the Response object. Likely indicates a bug or misconfiguration. Exception will be rethrown." + exCreateResponse.Message);
}
throw;
}
}
}
public class ResponseExceptionOperationBehaviorAttribute : InvokerOperationBehaviorAttribute
{
protected override InvokerBase CreateInvoker(IOperationInvoker oldInvoker, OperationDescription operationDescription, DispatchOperation dispatchOperation)
{
return new ResponseExceptionInvoker(oldInvoker, operationDescription);
}
public override void Validate(OperationDescription operationDescription)
{
// validate that this attribute can be applied to the service behavior.
Type returnType = operationDescription.GetReturnType();
if (!typeof(Response).IsAssignableFrom(returnType))
{
throw new InvalidOperationException("'" + returnType.FullName + "' does not inherit from '" + typeof(Response).FullName +
"'. ImplicitResponse behavior applied to '" + operationDescription.SyncMethod.DeclaringType.Name + "." + operationDescription.Name +
"' requires the method return type inherit from '" + typeof(Response).FullName);
}
}
}
static class OperationDescriptionExtensions
{
public static Type GetReturnType(this OperationDescription operationDescription)
{
if (operationDescription.SyncMethod == null)
throw new InvalidOperationException("These behaviors have only been tested with Sychronous methods.");
// !! Warning: This does NOT work for Asynch or Task based implementations.
System.Reflection.MethodInfo method = operationDescription.SyncMethod ?? operationDescription.EndMethod;
return method.ReturnType;
}
}
// When not using FaultContracts, return success/fail as a part of all responses via some base class properties.
[DataContract]
public class Response
{
[DataMember]
public bool Success { get; set; }
[DataMember]
public string ErrorMessage { get; set; }
}
public class ChildResponse : Response
{
[DataMember]
public string Foo { get; set; }
}
[DataContract]
public class Request
{
[DataMember]
public string Name { get; set; }
}
[ServiceContract]
public interface ISimple
{
[OperationContract]
ChildResponse Work(Request request);
[OperationContract]
ChildResponse Fail(Request request);
}
public class SimpleService : ISimple
{
public ChildResponse Work(Request request) {
return new ChildResponse() { Success = true };
}
[ResponseExceptionOperationBehavior]
public ChildResponse Fail(Request request)
{
throw new NotImplementedException("This method isn't done");
}
}
class Program
{
static void Main(string[] args)
{
ServiceHost simpleHost = new ServiceHost(typeof(SimpleService), new Uri("http://localhost/Simple"));
simpleHost.Open();
ChannelFactory<ISimple> factory = new ChannelFactory<ISimple>(simpleHost.Description.Endpoints[0]);
ISimple proxy = factory.CreateChannel();
Logger.Debug("Calling Work...");
var response1 = proxy.Work(new Request() { Name = "Foo" });
Logger.Debug("Work() returned Success=" + response1.Success + " message='" + response1.ErrorMessage + "'");
Logger.Debug("Calling Fail...");
var response2 = proxy.Fail(new Request() { Name = "FooBar" });
Logger.Debug("Fail() returned Success=" + response2.Success + " message='" + response2.ErrorMessage + "'");
Console.WriteLine("Press ENTER to close the host.");
Console.ReadLine();
((ICommunicationObject)proxy).Shutdown();
simpleHost.Shutdown();
}
}
public static class CommunicationObjectExtensions
{
static public void Shutdown(this ICommunicationObject obj)
{
try
{
obj.Close();
}
catch (Exception ex)
{
Console.WriteLine("Shutdown exception: {0}", ex.Message);
obj.Abort();
}
}
}
public static class Logger
{
public static void Debug(string message) { Console.WriteLine(message); }
public static void Error(string message) { Console.WriteLine(message); }
}
}

In your place, I would implement this as simple as possible by using general try/catch block. Using custom exception handler to observe the invoked service method and create a corresponding response using reflection looks like an overkill for me.
Make your life easier:
public CreateFooResponse Create(CreateFooRequest request)
{
try
{
// Create Foo
var foo = CreateFoo();
// Return successful CreateFooResponse
return new CreateFooResponse
{
Created = foo,
Error = null,
Detail = "Created successfully"
};
}
catch (Exception ex)
{
// Return CreateFooResponse with an error
return new CreateFooResponse
{
Created = null,
Error = CreateError(ex),
Detail = "Unable to create Foo."
};
}
}
A good example of using custom WCF error handler is to log the error, convert it to the FaultContract and return to the caller. You have a different scenario and I would suggest a different approach.

First of all, I would like to say that it is much better to use faultcontracts. Below I will give an example for service GetDataUsingDataContract:
[OperationContract]
CompositeType GetDataUsingDataContract(CompositeType composite);
[DataContract]
public class CompositeType
{
[DataMember]
public bool BoolValue { get; set; }
[DataMember]
public string StringValue { get; set; }
}
Then, you create a bodyWriter equivalent to a normal response:
public class MyBodyWriter : BodyWriter
{
public CompositeType CompositeType { get; private set; }
public MyBodyWriter(CompositeType composite)
: base(false)
{
CompositeType = composite;
}
protected override void OnWriteBodyContents(XmlDictionaryWriter writer)
{
writer.WriteStartElement("GetDataUsingDataContractResponse", "http://tempuri.org/");
writer.WriteStartElement("GetDataUsingDataContractResult");
writer.WriteAttributeString("xmlns", "a", null, "http://schemas.datacontract.org/2004/07/WcfService1");
writer.WriteAttributeString("xmlns", "i", null, "http://www.w3.org/2001/XMLSchema-instance");
writer.WriteStartElement("a:BoolValue");
writer.WriteString(CompositeType.BoolValue.ToString().ToLower());
writer.WriteEndElement();
writer.WriteStartElement("a:StringValue");
writer.WriteString(CompositeType.StringValue);
writer.WriteEndElement();
writer.WriteEndElement();
}
}
And finally, you use it in your IErrorHandler:
public void ProvideFault(Exception error, MessageVersion version, ref Message fault)
{
// TODO: parse error and gets response
var response = new CompositeType {BoolValue = true, StringValue = "a"};
fault = Message.CreateMessage(version, "http://tempuri.org/", new MyBodyWriter(response));
}
I did a test and I can get a correct answer if the service throws an exception or if the service responds normally:
var response1 = client.GetDataUsingDataContract(null);
var response2 = client.GetDataUsingDataContract(new CompositeType { StringValue = "a", BoolValue = true });

Passing object messages in Azure Queue Storage

I'm trying to find a way to pass objects to the Azure Queue. I couldn't find a way to do this.
As I've seen I can pass string or byte array, which is not very comfortable for passing objects.
Is there anyway to pass custom objects to the Queue?
Thanks!

You can use the following classes as example:
[Serializable]
public abstract class BaseMessage
{
public byte[] ToBinary()
{
BinaryFormatter bf = new BinaryFormatter();
byte[] output = null;
using (MemoryStream ms = new MemoryStream())
{
ms.Position = 0;
bf.Serialize(ms, this);
output = ms.GetBuffer();
}
return output;
}
public static T FromMessage<T>(CloudQueueMessage m)
{
byte[] buffer = m.AsBytes;
T returnValue = default(T);
using (MemoryStream ms = new MemoryStream(buffer))
{
ms.Position = 0;
BinaryFormatter bf = new BinaryFormatter();
returnValue = (T)bf.Deserialize(ms);
}
return returnValue;
}
}
Then a StdQueue (a Queue that is strongly typed):
public class StdQueue<T> where T : BaseMessage, new()
{
protected CloudQueue queue;
public StdQueue(CloudQueue queue)
{
this.queue = queue;
}
public void AddMessage(T message)
{
CloudQueueMessage msg =
new CloudQueueMessage(message.ToBinary());
queue.AddMessage(msg);
}
public void DeleteMessage(CloudQueueMessage msg)
{
queue.DeleteMessage(msg);
}
public CloudQueueMessage GetMessage()
{
return queue.GetMessage(TimeSpan.FromSeconds(120));
}
}
Then, all you have to do is to inherit the BaseMessage:
[Serializable]
public class ParseTaskMessage : BaseMessage
{
public Guid TaskId { get; set; }
public string BlobReferenceString { get; set; }
public DateTime TimeRequested { get; set; }
}
And make a queue that works with that message:
CloudStorageAccount acc;
if (!CloudStorageAccount.TryParse(connectionString, out acc))
{
throw new ArgumentOutOfRangeException("connectionString", "Invalid connection string was introduced!");
}
CloudQueueClient clnt = acc.CreateCloudQueueClient();
CloudQueue queue = clnt.GetQueueReference(processQueue);
queue.CreateIfNotExist();
this._queue = new StdQueue<ParseTaskMessage>(queue);
Hope this helps!

Extension method that uses Newtonsoft.Json and async
public static async Task AddMessageAsJsonAsync<T>(this CloudQueue cloudQueue, T objectToAdd)
{
var messageAsJson = JsonConvert.SerializeObject(objectToAdd);
var cloudQueueMessage = new CloudQueueMessage(messageAsJson);
await cloudQueue.AddMessageAsync(cloudQueueMessage);
}

I like this generalization approach but I don't like having to put Serialize attribute on all the classes I might want to put in a message and derived them from a base (I might already have a base class too) so I used...
using System;
using System.Text;
using Microsoft.WindowsAzure.Storage.Queue;
using Newtonsoft.Json;
namespace Example.Queue
{
public static class CloudQueueMessageExtensions
{
public static CloudQueueMessage Serialize(Object o)
{
var stringBuilder = new StringBuilder();
stringBuilder.Append(o.GetType().FullName);
stringBuilder.Append(':');
stringBuilder.Append(JsonConvert.SerializeObject(o));
return new CloudQueueMessage(stringBuilder.ToString());
}
public static T Deserialize<T>(this CloudQueueMessage m)
{
int indexOf = m.AsString.IndexOf(':');
if (indexOf <= 0)
throw new Exception(string.Format("Cannot deserialize into object of type {0}",
typeof (T).FullName));
string typeName = m.AsString.Substring(0, indexOf);
string json = m.AsString.Substring(indexOf + 1);
if (typeName != typeof (T).FullName)
{
throw new Exception(string.Format("Cannot deserialize object of type {0} into one of type {1}",
typeName,
typeof (T).FullName));
}
return JsonConvert.DeserializeObject<T>(json);
}
}
}
e.g.
var myobject = new MyObject();
_queue.AddMessage( CloudQueueMessageExtensions.Serialize(myobject));
var myobject = _queue.GetMessage().Deserialize<MyObject>();

In case the storage queue is used with WebJob or Azure function (quite common scenario) then the current Azure SDK allows to use POCO object directly. See examples here:
https://learn.microsoft.com/en-us/sandbox/functions-recipes/queue-storage
https://github.com/Azure/azure-webjobs-sdk/wiki/Queues#trigger
Note: The SDK will automatically use Newtonsoft.Json for serialization/deserialization under the hood.

I liked #Akodo_Shado's approach to serialize with Newtonsoft.Json. I updated it for Azure.Storage.Queues and also added a "Retrieve and Delete" method that deserializes the object from the queue.
public static class CloudQueueExtensions
{
public static async Task AddMessageAsJsonAsync<T>(this QueueClient queueClient, T objectToAdd) where T : class
{
string messageAsJson = JsonConvert.SerializeObject(objectToAdd);
BinaryData cloudQueueMessage = new BinaryData(messageAsJson);
await queueClient.SendMessageAsync(cloudQueueMessage);
}
public static async Task<T> RetreiveAndDeleteMessageAsObjectAsync<T>(this QueueClient queueClient) where T : class
{
QueueMessage[] retrievedMessage = await queueClient.ReceiveMessagesAsync(1);
if (retrievedMessage.Length == 0) return null;
string theMessage = retrievedMessage[0].MessageText;
T instanceOfT = JsonConvert.DeserializeObject<T>(theMessage);
await queueClient.DeleteMessageAsync(retrievedMessage[0].MessageId, retrievedMessage[0].PopReceipt);
return instanceOfT;
}
}
The RetreiveAndDeleteMessageAsObjectAsync is designed to process 1 message at time, but you could obviously rewrite to deserialize the full array of messages and return a ICollection<T> or similar.

That is not right way to do it. queues are not ment for storing object. you need to put object in blob or table (serialized).
I believe queue messgae body has 64kb size limit with sdk1.5 and 8kb wih lower versions.
Messgae body is ment to transfer crutial data for workera that pick it up only.

How to make second test passing in these C# Xml unit tests? What do I miss in initialization of XmlReaderSettings?

Issue description:
I need to fix an issue with resolving of standard HTML entitities.
I've implemented HtmlEntityReader - implementation of XmlReader which has a code to resolve entities
Public API of our system provides a methods with usage of XmlReader, so user can pass XmlReader created using one of the XmlReader.Create methods
Current code of my xml unit tests is below:
using System.Xml;
using NUnit.Framework;
namespace Tests
{
[TestFixture]
public class XmlTests
{
// this test works
[Test]
public void TestEntitiesResolving1()
{
var path = QA.ResolvePath(#"html\bugs\317.html");
using (var reader = new XmlTextReader(path, new NameTable()))
{
reader.XmlResolver = null; //to prevent DTD downloading
var wrapper = new HtmlEntityReader(reader, XmlUtils.HtmlEntities);
while (wrapper.Read()) { }
}
}
// this test does not work - why?
// what's the difference in initialization of internal XmlTextReaderImpl?
[Test]
public void TestEntitiesResolving2()
{
var path = QA.ResolvePath(#"html\bugs\317.html");
var settings = new XmlReaderSettings
{
XmlResolver = null, //to prevent DTD downloading
NameTable = new NameTable(),
ProhibitDtd = false,
CheckCharacters = false,
};
using (var reader = XmlReader.Create(path, settings))
{
var wrapper = new HtmlEntityReader(reader, XmlUtils.HtmlEntities);
while (wrapper.Read()) { }
}
}
}
}
Partial code of HtmlEntityReader is below:
internal sealed class HtmlEntityReader : XmlReader
{
readonly XmlReader _impl;
readonly Hashtable _entitySet;
string _entityValue;
public HtmlEntityReader(XmlReader reader, Hashtable entitySet)
{
if (reader == null) throw new ArgumentNullException("reader");
if (entitySet == null) throw new ArgumentNullException("entitySet");
_impl = reader;
_entitySet = entitySet;
}
public override XmlNodeType NodeType
{
get { return _entityValue != null ? XmlNodeType.Text : _impl.NodeType; }
}
public override string LocalName
{
get { return _entityValue != null ? string.Empty : _impl.LocalName; }
}
public override string Prefix
{
get { return _entityValue != null ? string.Empty : _impl.Prefix; }
}
public override string Name
{
get { return _entityValue != null ? string.Empty : _impl.Name; }
}
public override bool HasValue
{
get { return _entityValue != null || _impl.HasValue; }
}
public override string Value
{
get { return _entityValue ?? _impl.Value; }
}
public override bool CanResolveEntity
{
get { return true; }
}
public override void ResolveEntity()
{
//it seems this does not call - why?
}
public override bool Read()
{
_entityValue = null;
if (!_impl.Read()) return false;
if (NodeType == XmlNodeType.EntityReference)
{
//resolving of entity reference
_entityValue = (string)_entitySet[Name];
}
return true;
}
// ... delegation of XmlReader abstract methods to _impl
}
I've got the exception:
System.Xml.XmlException: Reference to undeclared entity 'nbsp'. Line 4, position 5.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.Throw(String res, String arg, Int32 lineNo, Int32 linePos)
at System.Xml.XmlTextReaderImpl.HandleGeneralEntityReference(String name, Boolean isInAttributeValue, Boolean pushFakeEntityIfNullResolver, Int32 entityStartLinePos)
at System.Xml.XmlTextReaderImpl.HandleEntityReference(Boolean isInAttributeValue, EntityExpandType expandType, ref Int32 charRefEndPos)
at System.Xml.XmlTextReaderImpl.ParseText(ref Int32 startPos, ref Int32 endPos, ref Int32 outOrChars)
at System.Xml.XmlTextReaderImpl.ParseText()
at System.Xml.XmlTextReaderImpl.ParseElementContent()
at System.Xml.XmlTextReaderImpl.Read()
... private staff
Could you provide a quick advice or link to a solution while I am fixing / investigating / searching this issue through my own efforts?

I've done some research on your question and as best I can tell the only way to ensure that character entities are resolved is to declare them in a DTD. You can resolve the DTD contents yourself (e.g. for caching) by deriving an implementation from the Systm.Xml.XmlResolver base class and responding to GetEntity calls with a stream containing the DTD data.
I wrote an article some time back that explains how to push a default DTD onto the XmlParserContext if there is no DTD declared in your input document. This article is a little dated, but the same concept continues to work with XmlReaderSettings & XmlReader.Create by using an XmlReader.Create overload that accepts an XmlParserContext object as an argument.
Finally, it also looks like .NET 4 will help us out a little with a new XmlResolver derivative named XmlPreloadedResolver which seems to have the XHTML1 and RSS DTDs built in.

The funny thing is that, as sergeyt noted, XmlTextReader doesn't care about undefined entities when processing a xml fragment, while XmlReader does!
So a solution in many cases would be to try with an XmlTextRader.

How do I get access to SOAP response

(If anything here needs clarification/ more detail please let me know.)
I have an application (C#, 2.* framework) that interfaces with a third-party webservice using SOAP. I used thinktecture's WSCF add-in against a supplied WSDL to create the client-side implementation. For reasons beyond my control the SOAP message exchange uses WSE2.0 for security (the thinctecture implementation had to be modified to include the WSE2.0 reference). In addition to the 'normal' data package I attach a stored X509 cert and a binary security token from a previous call to a different web service. We are using SSL encryption of some sort - I don't know the details.
All the necessary serialization/deserialization is contained in the web service client - meaning when control is returned to me after calling the client the entire XML string contained in the SOAP response is not available to me - just the deserialized components. Don't get me wrong - I think that's good because it means I don't have to do it myself.
However, in order for me to have something worth storing/archiving I am having to re-serialize the data at the root element. This seems like a waste of resources since my result was in the SOAP response.
Now for my question:
How can I get access to a 'clear' version of the SOAP response so that I don't have to re-serialize everything for storage/archiving?
Edit- My application is a 'formless' windows app running as a network service - triggered by a WebsphereMQ client trigger monitor. I don't think ASP.NET solutions will apply.
Edit - Since the consensus so far is that it doesn't matter whether my app is ASP.NET or not then I will give CodeMelt's (and by extension Chris's) solution a shot.

You can utilize SoapExtension from existing WSE2.0 framework to intercept the responses from the server.
public class MyClientSOAPExtension : SoapExtension
{
Stream oldStream;
Stream newStream;
// Save the Stream representing the SOAP request or SOAP response into
// a local memory buffer.
public override Stream ChainStream( Stream stream )
{
oldStream = stream;
newStream = new MemoryStream();
return newStream;
}
public override void ProcessMessage(SoapMessage message)
{
switch (message.Stage)
{
case SoapMessageStage.BeforeDeserialize:
// before the XML deserialized into object.
break;
case SoapMessageStage.AfterDeserialize:
break;
case SoapMessageStage.BeforeSerialize:
break;
case SoapMessageStage.AfterSerialize:
break;
default:
throw new Exception("Invalid stage...");
}
}
}
At stage of SoapMessageStage.BeforeDeserialize,
You can read the expected data you want from oldstream (e.g. use XmlReader).
Then store the expected data somewhere for yourself to use and also you need
forward the old stream data to the newstream for web service later stage to use the data, e.g. deserialize XML into objects.
The sample of logging all the traffic for the web service from MSDN

Here is an example you can setup using Visual studio web reference to http://footballpool.dataaccess.eu/data/info.wso?WSDL
Basically, you must insert in the webservice call chain a XmlReader spyer that will reconstruct the raw XML.
I believe this way is somehow simpler that using SoapExtensions.
Solution solution was inspired by http://orbinary.com/blog/2010/01/getting-the-raw-soap-xml-sent-via-soaphttpclientprotocol/
using System;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.IO;
using System.Reflection;
using System.Xml;
namespace ConsoleApplication1 {
public class XmlReaderSpy : XmlReader {
XmlReader _me;
public XmlReaderSpy(XmlReader parent) {
_me = parent;
}
/// <summary>
/// Extracted XML.
/// </summary>
public string Xml;
#region Abstract method that must be implemented
public override XmlNodeType NodeType {
get {
return _me.NodeType;
}
}
public override string LocalName {
get {
return _me.LocalName;
}
}
public override string NamespaceURI {
get {
return _me.NamespaceURI;
}
}
public override string Prefix {
get {
return _me.Prefix;
}
}
public override bool HasValue {
get { return _me.HasValue; }
}
public override string Value {
get { return _me.Value; }
}
public override int Depth {
get { return _me.Depth; }
}
public override string BaseURI {
get { return _me.BaseURI; }
}
public override bool IsEmptyElement {
get { return _me.IsEmptyElement; }
}
public override int AttributeCount {
get { return _me.AttributeCount; }
}
public override string GetAttribute(int i) {
return _me.GetAttribute(i);
}
public override string GetAttribute(string name) {
return _me.GetAttribute(name);
}
public override string GetAttribute(string name, string namespaceURI) {
return _me.GetAttribute(name, namespaceURI);
}
public override void MoveToAttribute(int i) {
_me.MoveToAttribute(i);
}
public override bool MoveToAttribute(string name) {
return _me.MoveToAttribute(name);
}
public override bool MoveToAttribute(string name, string ns) {
return _me.MoveToAttribute(name, ns);
}
public override bool MoveToFirstAttribute() {
return _me.MoveToFirstAttribute();
}
public override bool MoveToNextAttribute() {
return _me.MoveToNextAttribute();
}
public override bool MoveToElement() {
return _me.MoveToElement();
}
public override bool ReadAttributeValue() {
return _me.ReadAttributeValue();
}
public override bool Read() {
bool res = _me.Read();
Xml += StringView();
return res;
}
public override bool EOF {
get { return _me.EOF; }
}
public override void Close() {
_me.Close();
}
public override ReadState ReadState {
get { return _me.ReadState; }
}
public override XmlNameTable NameTable {
get { return _me.NameTable; }
}
public override string LookupNamespace(string prefix) {
return _me.LookupNamespace(prefix);
}
public override void ResolveEntity() {
_me.ResolveEntity();
}
#endregion
protected string StringView() {
string result = "";
if (_me.NodeType == XmlNodeType.Element) {
result = "<" + _me.Name;
if (_me.HasAttributes) {
_me.MoveToFirstAttribute();
do {
result += " " + _me.Name + "=\"" + _me.Value + "\"";
} while (_me.MoveToNextAttribute());
//Let's put cursor back to Element to avoid messing up reader state.
_me.MoveToElement();
}
if (_me.IsEmptyElement) {
result += "/";
}
result += ">";
}
if (_me.NodeType == XmlNodeType.EndElement) {
result = "</" + _me.Name + ">";
}
if (_me.NodeType == XmlNodeType.Text || _me.NodeType == XmlNodeType.Whitespace) {
result = _me.Value;
}
if (_me.NodeType == XmlNodeType.XmlDeclaration) {
result = "<?" + _me.Name + " " + _me.Value + "?>";
}
return result;
}
}
public class MyInfo : ConsoleApplication1.eu.dataaccess.footballpool.Info {
protected XmlReaderSpy _xmlReaderSpy;
public string Xml {
get {
if (_xmlReaderSpy != null) {
return _xmlReaderSpy.Xml;
}
else {
return "";
}
}
}
protected override XmlReader GetReaderForMessage(System.Web.Services.Protocols.SoapClientMessage message, int bufferSize) {
XmlReader rdr = base.GetReaderForMessage(message, bufferSize);
_xmlReaderSpy = new XmlReaderSpy((XmlReader)rdr);
return _xmlReaderSpy;
}
}
class Program {
static void Main(string[] args) {
MyInfo info = new MyInfo();
string[] rest = info.Cities();
System.Console.WriteLine("RAW Soap XML response :\n"+info.Xml);
System.Console.ReadLine();
}
}
}

Old thread, but in case others are looking to do this today: these ideas of leveraging SoapExtension or creating 'spy' classes are great, but don't work in .NET Core.
#mting923's suggestion to use IClientMessageInspector approach works in .NET Core 3.1; see here: Get SOAP Message before sending it to the WebService in .NET.
A generated SOAP proxy class is still just a WCF client under the hood, and so the IClientMessageInspector approach works a treat, even for an .NET Core Azure Function calling an older SOAP web service. The following works for me in a .NET Core 3.1 Azure Function:
public class SoapMessageInspector : IClientMessageInspector
{
public string LastRequestXml { get; private set; }
public string LastResponseXml { get; private set; }
public object BeforeSendRequest(ref Message request, IClientChannel channel)
{
LastRequestXml = request.ToString();
return request;
}
public void AfterReceiveReply(ref Message reply, object correlationState)
{
LastResponseXml = reply.ToString();
}
}
public class SoapInspectorBehavior : IEndpointBehavior
{
private readonly SoapMessageInspector inspector_ = new SoapMessageInspector();
public string LastRequestXml => inspector_.LastRequestXml;
public string LastResponseXml => inspector_.LastResponseXml;
public void AddBindingParameters(ServiceEndpoint endpoint, BindingParameterCollection bindingParameters)
{
}
public void ApplyDispatchBehavior(ServiceEndpoint endpoint, EndpointDispatcher endpointDispatcher)
{
}
public void Validate(ServiceEndpoint endpoint)
{
}
public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime)
{
clientRuntime.ClientMessageInspectors.Add(inspector_);
}
}
And then it can be set up like this:
var client = new ServiceClient();
var soapInspector = new SoapInspectorBehavior();
client.Endpoint.EndpointBehaviors.Add(soapInspector);
After invoking a web service call on the client proxy, soapInspector.LastRequestXml and soapInspector.LastResponseXml will contain the raw SOAP request and response (as strings).

Inspired by jfburdet, I wanted to see if it was possible to directly intercept at stream/byte level rather than reconstructing XML. And it is! See code below:
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Web.Services.Protocols;
using System.Xml;
using Test.MyWebReference;
namespace Test {
/// <summary>
/// Adds the ability to retrieve the SOAP request/response.
/// </summary>
public class ServiceSpy : OriginalService {
private StreamSpy writerStreamSpy;
private XmlTextWriter xmlWriter;
private StreamSpy readerStreamSpy;
private XmlTextReader xmlReader;
public MemoryStream WriterStream {
get { return writerStreamSpy == null ? null : writerStreamSpy.ClonedStream; }
}
public XmlTextWriter XmlWriter {
get { return xmlWriter; }
}
public MemoryStream ReaderStream {
get { return readerStreamSpy == null ? null : readerStreamSpy.ClonedStream; }
}
public XmlTextReader XmlReader {
get { return xmlReader; }
}
protected override void Dispose(bool disposing) {
base.Dispose(disposing);
DisposeWriterStreamSpy();
DisposeReaderStreamSpy();
}
protected override XmlWriter GetWriterForMessage(SoapClientMessage message, int bufferSize) {
// Dispose previous writer stream spy.
DisposeWriterStreamSpy();
writerStreamSpy = new StreamSpy(message.Stream);
// XML should always support UTF8.
xmlWriter = new XmlTextWriter(writerStreamSpy, Encoding.UTF8);
return xmlWriter;
}
protected override XmlReader GetReaderForMessage(SoapClientMessage message, int bufferSize) {
// Dispose previous reader stream spy.
DisposeReaderStreamSpy();
readerStreamSpy = new StreamSpy(message.Stream);
xmlReader = new XmlTextReader(readerStreamSpy);
return xmlReader;
}
private void DisposeWriterStreamSpy() {
if (writerStreamSpy != null) {
writerStreamSpy.Dispose();
writerStreamSpy.ClonedStream.Dispose();
writerStreamSpy = null;
}
}
private void DisposeReaderStreamSpy() {
if (readerStreamSpy != null) {
readerStreamSpy.Dispose();
readerStreamSpy.ClonedStream.Dispose();
readerStreamSpy = null;
}
}
/// <summary>
/// Wrapper class to clone read/write bytes.
/// </summary>
public class StreamSpy : Stream {
private Stream wrappedStream;
private long startPosition;
private MemoryStream clonedStream = new MemoryStream();
public StreamSpy(Stream wrappedStream) {
this.wrappedStream = wrappedStream;
startPosition = wrappedStream.Position;
}
public MemoryStream ClonedStream {
get { return clonedStream; }
}
public override bool CanRead {
get { return wrappedStream.CanRead; }
}
public override bool CanSeek {
get { return wrappedStream.CanSeek; }
}
public override bool CanWrite {
get { return wrappedStream.CanWrite; }
}
public override void Flush() {
wrappedStream.Flush();
}
public override long Length {
get { return wrappedStream.Length; }
}
public override long Position {
get { return wrappedStream.Position; }
set { wrappedStream.Position = value; }
}
public override int Read(byte[] buffer, int offset, int count) {
long relativeOffset = wrappedStream.Position - startPosition;
int result = wrappedStream.Read(buffer, offset, count);
if (clonedStream.Position != relativeOffset) {
clonedStream.Position = relativeOffset;
}
clonedStream.Write(buffer, offset, result);
return result;
}
public override long Seek(long offset, SeekOrigin origin) {
return wrappedStream.Seek(offset, origin);
}
public override void SetLength(long value) {
wrappedStream.SetLength(value);
}
public override void Write(byte[] buffer, int offset, int count) {
long relativeOffset = wrappedStream.Position - startPosition;
wrappedStream.Write(buffer, offset, count);
if (clonedStream.Position != relativeOffset) {
clonedStream.Position = relativeOffset;
}
clonedStream.Write(buffer, offset, count);
}
public override void Close() {
wrappedStream.Close();
base.Close();
}
protected override void Dispose(bool disposing) {
if (wrappedStream != null) {
wrappedStream.Dispose();
wrappedStream = null;
}
base.Dispose(disposing);
}
}
}
}

The MSDN Library includes example code for obtaining the XML of both the request and the response that you can use to archive it. Obviously you'll have to make some changes since the example stores data in a text file, but it isn't too complicated.