This question has been asked before in a few forms but I cannot get any of the answers to work, I'm losing my hair and unsure if the problem is just that the solutions were from 2 years ago and things have changed.
How can I safely intercept the Response stream in a custom Owin Middleware - I based my code on this, it looks like it should work, but it doesn't
OWIN OnSendingHeaders Callback - Reading Response Body - seems to be a different OWIN version, because method signature doesn't work
What I want to do is write an OMC that can inspect the response stream from MVC.
What I did (amongst several other attempts), is to add an OMC that sets context.Response.Body to a MemoryStream, so I can rewind it and inspect what was written by downstream components:
public async Task Invoke(IDictionary<string, object> env)
{
IOwinContext context = new OwinContext(env);
// Buffer the response
var stream = context.Response.Body;
var buffer = new MemoryStream();
context.Response.Body = buffer;
.......
What I find is that the MemoryStream is always empty, unless I write to it from another OMC. So it seems that downstream OMCs are using my MemoryStream, but MVC responses are not, as if the OWIN pipeline completes before the request goes to MVC, but that's not right is it?
Complete code:
public partial class Startup
{
public void Configuration(IAppBuilder app)
{
ConfigureAuth(app);
app.Use(new ResponseExaminerMiddleware());
// Specify the stage for the OMC
//app.UseStageMarker(PipelineStage.Authenticate);
}
}
public class ResponseExaminerMiddleware
{
private AppFunc next;
public void Initialize(AppFunc next)
{
this.next = next;
}
public async Task Invoke(IDictionary<string, object> env)
{
IOwinContext context = new OwinContext(env);
// Buffer the response
var stream = context.Response.Body;
var buffer = new MemoryStream();
context.Response.Body = buffer;
await this.next(env);
buffer.Seek(0, SeekOrigin.Begin);
var reader = new StreamReader(buffer);
string responseBody = await reader.ReadToEndAsync();
// Now, you can access response body.
System.Diagnostics.Debug.WriteLine(responseBody);
// You need to do this so that the response we buffered
// is flushed out to the client application.
buffer.Seek(0, SeekOrigin.Begin);
await buffer.CopyToAsync(stream);
}
}
For what it's worth I also tried a suggestion where the response.Body stream is set to a Stream subclass, just so I could monitor what is written to the stream and bizarrely the Stream.Write method is called, but with an empty byte array, never any actual content...
MVC does not pass its request through OWIN pipeline. To capture MVC response we need to make custom response filter that captures response data
/// <summary>
/// Stream capturing the data going to another stream
/// </summary>
internal class OutputCaptureStream : Stream
{
private Stream InnerStream;
public MemoryStream CapturedData { get; private set; }
public OutputCaptureStream(Stream inner)
{
InnerStream = inner;
CapturedData = new MemoryStream();
}
public override bool CanRead
{
get { return InnerStream.CanRead; }
}
public override bool CanSeek
{
get { return InnerStream.CanSeek; }
}
public override bool CanWrite
{
get { return InnerStream.CanWrite; }
}
public override void Flush()
{
InnerStream.Flush();
}
public override long Length
{
get { return InnerStream.Length; }
}
public override long Position
{
get { return InnerStream.Position; }
set { CapturedData.Position = InnerStream.Position = value; }
}
public override int Read(byte[] buffer, int offset, int count)
{
return InnerStream.Read(buffer, offset, count);
}
public override long Seek(long offset, SeekOrigin origin)
{
CapturedData.Seek(offset, origin);
return InnerStream.Seek(offset, origin);
}
public override void SetLength(long value)
{
CapturedData.SetLength(value);
InnerStream.SetLength(value);
}
public override void Write(byte[] buffer, int offset, int count)
{
CapturedData.Write(buffer, offset, count);
InnerStream.Write(buffer, offset, count);
}
}
And then we make a logging middleware that can log both kinds of responses properly
public class LoggerMiddleware : OwinMiddleware
{
public LoggerMiddleware(OwinMiddleware next): base(next)
{
}
public async override Task Invoke(IOwinContext context)
{
//to intercept MVC responses, because they don't go through OWIN
HttpResponse httpResponse = HttpContext.Current.Response;
OutputCaptureStream outputCapture = new OutputCaptureStream(httpResponse.Filter);
httpResponse.Filter = outputCapture;
IOwinResponse owinResponse = context.Response;
//buffer the response stream in order to intercept downstream writes
Stream owinResponseStream = owinResponse.Body;
owinResponse.Body = new MemoryStream();
await Next.Invoke(context);
if (outputCapture.CapturedData.Length == 0) {
//response is formed by OWIN
//make sure the response we buffered is flushed to the client
owinResponse.Body.Position = 0;
await owinResponse.Body.CopyToAsync(owinResponseStream);
} else {
//response by MVC
//write captured data to response body as if it was written by OWIN
outputCapture.CapturedData.Position = 0;
outputCapture.CapturedData.CopyTo(owinResponse.Body);
}
LogResponse(owinResponse);
}
}
Related
I have a method that take Stream parameter and pass it to server
public async Task<string> Execute(Stream archive)
{
archive.Seek(0, SeekOrigin.Begin);
using var content = new MultipartFormDataContent();
content.Add(new StreamContent(archive), "file1", "file1");
var result = "";
using (var response = await _client.PostAsync(_uri, content))
{
if (response.IsSuccessStatusCode)
{
var stringResult = await response.Content.ReadAsStringAsync();
result = stringResult;
}
}
// here archive is already disposed
return result;
}
Now I implement the retry policy of this method.
If outside code calling this method gets "" as result, then it tries to call this method againg.
But the archive is disposed to that moment.
I see that archive stream is disposed immediately after disposing of response.
Why?
What should I do if I need stream parameter outside after this method?
It's the StreamContent that will dispose the Stream, as states in it's source. And that will be disposed by the MultipartContent. And that will be disposed in PostAsync... all though the chain.
A solution is to subclass the Stream and remove the dispose method, like proposed here. but you'll have to make sure the original stream gets disposed yourself!.
Edit: update. Stream is an abstract class, so it would be easier if you know the specific stream type, e.g.
public sealed class NoDisposeMemoryStream : MemoryStream
{
protected override void Dispose(bool disposing) { }
}
Else you will need to write your own complete Stream wrapper, see bottom.
Another solution is to implement the retry mechanism in the innermost using block, resetting the archive seek origin every fail. That's likely safer.
public sealed class NoDisposeStream : Stream
{
private Stream _original;
public NoDisposeStream(Stream original) => _original = original
?? throw new ArgumentNullException(nameof(original));
public override bool CanRead => _original.CanRead;
public override bool CanSeek => _original.CanSeek;
public override bool CanWrite => _original.CanWrite;
public override long Length => _original.Length;
public override long Position { get => _original.Position; set { _original.Position = value; } }
public override void Flush() => _original.Flush();
public override int Read(byte[] buffer, int offset, int count) => _original.Read(buffer, offset, count);
public override long Seek(long offset, SeekOrigin origin) => _original.Seek(offset, origin);
public override void SetLength(long value) => _original.SetLength(value);
public override void Write(byte[] buffer, int offset, int count) => _original.Write(buffer, offset, count);
protected override void Dispose(bool disposing) { }
}
This happens because HttpClient PostAsync Disposes of Content that you pass.
https://github.com/microsoft/referencesource/blob/master/System/net/System/Net/Http/HttpClient.cs
The relevant code:
public Task<HttpResponseMessage> PostAsync(Uri requestUri, HttpContent content,
CancellationToken cancellationToken)
{
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, requestUri);
request.Content = content;
return SendAsync(request, cancellationToken);
}
And then SendAsync calls DisposeRequestContent which is implemented like:
private void DisposeRequestContent(HttpRequestMessage request)
{
Contract.Requires(request != null);
// When a request completes, HttpClient disposes the request content so the user doesn't have to. This also
// ensures that a HttpContent object is only sent once using HttpClient (similar to HttpRequestMessages
// that can also be sent only once).
HttpContent content = request.Content;
if (content != null)
{
content.Dispose();
}
}
The comments say why
I would like to know how many bytes were actually transmitted when using Post or PostAsync. I'm using code similar to the following. I could look at the bytes of the filePath, but in my real code, I'm doing some manipulation to the file stream between being read and sent. If you pull out the MyFilteredContent line, how would you do it?
async Task<bool> SendFile(string filePath)
{
using (HttpContent fileContent = new FileContent(filePath))
using (MyFilteredContent filteredContent = new MyFilteredContent(fileContent))
{
var t = await MyAppSettings.TargetUrl
.AllowAnyHttpStatus()
.PostAsync(filteredContent);
if (t.IsSuccessStatusCode)
{
return true;
}
throw new Exception("blah blah");
}
}
Here's a code sample of what I described in the comment - using DelegatingHandler, overriding SendAsync to get the bytes of the request being sent and then configuring FlurlHttp settings to use the handler:
public class HttpFactory : DefaultHttpClientFactory
{
public override HttpMessageHandler CreateMessageHandler()
{
return new CustomMessageHandler();
}
}
public class CustomMessageHandler : DelegatingHandler
{
protected async override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var content = await request.Content.ReadAsByteArrayAsync();
return await base.SendAsync(request, cancellationToken);
}
}
FlurlHttp.Configure(settings =>
{
settings.HttpClientFactory = new HttpFactory();
});
How do you read the content of an HttpResponse object in C# / ASP.net?
I need to be able to read the body content as a JSON object, modify it, and then write it back to the response output stream. I want to make sure I don't lose what's already in the stream, thus I need to read from it first.
How do I do this?
You can use a Delegating handler
public class ContentHandler : DelegatingHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
return base.SendAsync(request, cancellationToken).ContinueWith<HttpResponseMessage>((responseToCompleteTask) =>
{
HttpResponseMessage response = responseToCompleteTask.Result;
var YourContent = response.Content.ReadAsStreamAsync().Result;
response.Content = new CompressedContent(response.Content, acceptedEncoding);
return response;
},
TaskContinuationOptions.OnlyOnRanToCompletion);
}
}
Register your handler at WebApiConfig
GlobalConfiguration.Configuration.MessageHandlers.Add(new ContentHandler());
You can edit your reponse content by extending the HttpContent Class.
For example to compress content
public class CompressedContent : HttpContent
{
private HttpContent originalContent;
protected override bool TryComputeLength(out long length)
{
length = -1;
return false;
}
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
Stream editedStream = null;
if (encodingType == "gzip")
{
editedStream = new GZipStream(stream, CompressionMode.Compress, leaveOpen: true);
}
else if (encodingType == "deflate")
{
editedStream = new DeflateStream(stream, CompressionMode.Compress, leaveOpen: true);
}
return originalContent.CopyToAsync(editedStream).ContinueWith(tsk =>
{
if (editedStream != null)
{
editedStream.Dispose();
}
});
}
}
The Problem
I am trying to send audio recorded by my android device to a MVC.NET Web Api. I confirmed the connection by passing simple string parameters. However, when I try to pass the byte array generated from the audio, I get a 500 from the server every time. I have tried multiple configurations, but here's what I currently have:
MVC Web API
public class PostParms
{
public string AudioSource { get; set; }
public string ExtraInfo { get; set; }
}
public class MediaController : ApiController
{
[HttpPost]
public string Post([FromBody]PostParms parms)
{
byte[] bytes = System.Text.Encoding.UTF8.GetBytes(parms.AudioSource);
return "success";
}
}
Android Code
public class WebServiceTask extends AsyncTask<String, Void, Long>
{
#Override
protected Long doInBackground(String... parms)
{
long totalsize = 0;
String filepath = parms[0];
byte[] fileByte = convertAudioFileToByte(new File(filepath));
//Tried base64 below with Convert.FromBase64 on the C# side
//String bytesstring = Base64.encodeToString(fileByte, Base64.DEFAULT);
String bytesstring = "";
try
{
String t = new String(fileByte,"UTF-8");
bytesstring = t;
} catch (UnsupportedEncodingException e1)
{
// TODO Auto-generated catch block
e1.printStackTrace();
}
URL url;
HttpURLConnection urlConnection = null;
try {
url = new URL("http://my.webservice.com:8185/api/media");
//setup for connection
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setUseCaches(false);
urlConnection.setRequestMethod("POST");
urlConnection.setDoOutput(true);
urlConnection.setRequestProperty("Content-Type","application/json");
urlConnection.setRequestProperty("Accept","application/json");
urlConnection.connect();
JSONObject json = new JSONObject();
json.put("AudioSource", bytesstring);
json.put("ExtraInfo", "none");
DataOutputStream output = new DataOutputStream(urlConnection.getOutputStream());
output.writeBytes(URLEncoder.encode(json.toString(),"UTF-8"));
output.flush();
output.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
finally {
urlConnection.disconnect();
}
return totalsize;
}
protected void onPreExecute(){
}
protected void onPostExecute(){
}
private byte[] convertAudioFileToByte(File file) {
byte[] bFile = new byte[(int) file.length()];
try {
// convert file into array of bytes
FileInputStream fileInputStream = new FileInputStream(file);
fileInputStream.read(bFile);
fileInputStream.close();
} catch (Exception e) {
e.printStackTrace();
}
return bFile;
}
}
I can get a .NET application to work with the same API sending an audio stream, but not Android. Like I said, I've tried a number of configurations found on the internet with encoding and the output stream, but continue to get 500. I need help, as I am at a loss.
Thanks in advance
I used a ByteArrayEntity instead of a StringEntity.
// ByteArrayEntity
HttpPost request = new HttpPost(getString(R.string.server_url) + "SendMessage");
ByteArrayEntity be = new ByteArrayEntity(msg);
be.setContentType(new BasicHeader(HTTP.CONTENT_TYPE, "application/x-www-form-urlencoded"));
request.setEntity(be);
HttpResponse response = client.execute(request);
On the server side, I used this web API code to get the byte array:
[HttpPost]
public async Task<string> SendMessage()
{
byte[] arrBytes = await Request.Content.ReadAsByteArrayAsync();
return "";
}
I just posted a question on Stack Overflow at Uploading/Downloading Byte Arrays with AngularJS and ASP.NET Web API that is relevant (in part) to your question here. You might want to read that post to set the context for my response. It is about sending and receiving byte[]’s via Web API. I have one issue with the approach, but it has to do with the JavaScript client incorrectly handling the server response.
I would make the following changes to your Server-Side code as follows:
public class PostParms
{
public string ExtraInfo { get; set; }
}
public class MediaController : ApiController
{
[HttpPost]// byte[] is sent in body and parms sent in url
public string Post([FromBody] byte[] audioSource, PostParms parms)
{
byte[] bytes = audioSource;
return "success";
}
}
Read the post at http://byterot.blogspot.com/2012/04/aspnet-web-api-series-part-5.html
Use/add Byte Rot’s asynchronous version of the Media Formatter for byte[]’s.
I had to tweak the formatter as shown below for a newer version of ASP.NET.
using System;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Net.Http.Formatting;
using System.Net.Http.Headers;
using System.Threading.Tasks;
namespace YOUR_NAMESPACE
{
public class BinaryMediaTypeFormatter : MediaTypeFormatter
{
/****************** Asynchronous Version ************************/
private static Type _supportedType = typeof(byte[]);
private bool _isAsync = true;
public BinaryMediaTypeFormatter() : this(false){
}
public BinaryMediaTypeFormatter(bool isAsync) {
SupportedMediaTypes.Add(new MediaTypeHeaderValue("application/octet-stream"));
IsAsync = isAsync;
}
public bool IsAsync {
get { return _isAsync; }
set { _isAsync = value; }
}
public override bool CanReadType(Type type){
return type == _supportedType;
}
public override bool CanWriteType(Type type){
return type == _supportedType;
}
public override Task<object> ReadFromStreamAsync(Type type, Stream stream,
HttpContent contentHeaders, IFormatterLogger formatterLogger){// Changed to HttpContent
Task<object> readTask = GetReadTask(stream);
if (_isAsync){
readTask.Start();
}
else{
readTask.RunSynchronously();
}
return readTask;
}
private Task<object> GetReadTask(Stream stream){
return new Task<object>(() =>{
var ms = new MemoryStream();
stream.CopyTo(ms);
return ms.ToArray();
});
}
private Task GetWriteTask(Stream stream, byte[] data){
return new Task(() => {
var ms = new MemoryStream(data);
ms.CopyTo(stream);
});
}
public override Task WriteToStreamAsync(Type type, object value, Stream stream,
HttpContent contentHeaders, TransportContext transportContext){ // Changed to HttpContent
if (value == null)
value = new byte[0];
Task writeTask = GetWriteTask(stream, (byte[])value);
if (_isAsync){
writeTask.Start();
}
else{
writeTask.RunSynchronously();
}
return writeTask;
}
}
}
I also had to add the following line to the config file.
public static class WebApiConfig
{
public static void Register(HttpConfiguration config)
{
// This line added to activate your binary formatter.
config.Formatters.Add(new BinaryMediaTypeFormatter());
/*********************************** Client-Code *************************************/
You will need to use 'Content-Type': 'application/octet-stream' on the client.
You can send the byte[] as binary, but it has been too long since I have coded in C# on the client.
I think that the line: output.writeBytes(URLEncoder.encode(json.toString(),"UTF-8")); will have to be changed as a minimum.
I will have a look and see if I can find some C# client code snippets that might be relevant.
Edit You might have a look at the C# client code of the following post. It may help you to tweak your code to work with the approach that I suggested.
How to Get byte array properly from an Web Api Method in C#?
I am currently working on migrating few of my MVC3 Controllers to MVC4 Api Controllers.
I have implemented Compression mechanism for MVC3 controller Get Method Responses by inherting ActionFilterAttribute and overriding OnActionExecutiong method. After some Research I found that I need to use ActionFilterMethod from System.Web.HttpFilters. It would be great if somebody can share piece of sample code to get me started for this compressing HTTP response using GZip
The easiest is to enable compression directly at IIS level.
If you want to do it at the application level you could write a custom delegating message handler as shown in the following post:
public class CompressHandler : DelegatingHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
return base.SendAsync(request, cancellationToken).ContinueWith<HttpResponseMessage>((responseToCompleteTask) =>
{
HttpResponseMessage response = responseToCompleteTask.Result;
if (response.RequestMessage.Headers.AcceptEncoding != null)
{
string encodingType = response.RequestMessage.Headers.AcceptEncoding.First().Value;
response.Content = new CompressedContent(response.Content, encodingType);
}
return response;
},
TaskContinuationOptions.OnlyOnRanToCompletion);
}
}
public class CompressedContent : HttpContent
{
private HttpContent originalContent;
private string encodingType;
public CompressedContent(HttpContent content, string encodingType)
{
if (content == null)
{
throw new ArgumentNullException("content");
}
if (encodingType == null)
{
throw new ArgumentNullException("encodingType");
}
originalContent = content;
this.encodingType = encodingType.ToLowerInvariant();
if (this.encodingType != "gzip" && this.encodingType != "deflate")
{
throw new InvalidOperationException(string.Format("Encoding '{0}' is not supported. Only supports gzip or deflate encoding.", this.encodingType));
}
// copy the headers from the original content
foreach (KeyValuePair<string, IEnumerable<string>> header in originalContent.Headers)
{
this.Headers.AddWithoutValidation(header.Key, header.Value);
}
this.Headers.ContentEncoding.Add(encodingType);
}
protected override bool TryComputeLength(out long length)
{
length = -1;
return false;
}
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
Stream compressedStream = null;
if (encodingType == "gzip")
{
compressedStream = new GZipStream(stream, CompressionMode.Compress, leaveOpen: true);
}
else if (encodingType == "deflate")
{
compressedStream = new DeflateStream(stream, CompressionMode.Compress, leaveOpen: true);
}
return originalContent.CopyToAsync(compressedStream).ContinueWith(tsk =>
{
if (compressedStream != null)
{
compressedStream.Dispose();
}
});
}
}
All that's left now is to register the handler in Application_Start:
GlobalConfiguration.Configuration.MessageHandlers.Add(new CompressHandler());
If you are using IIS 7+, I would say leave the compression to IIS as it supports GZIP compression. Just turn it on.
On the other hand, compression is too close to the metal for the controller. Ideally controller should work in much higher level than bytes and streams.
Use a class and write the following code
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method)]
public class CompressFilter : ActionFilterAttribute
{
public override void OnActionExecuted(HttpActionExecutedContext context)
{
var acceptedEncoding = context.Response.RequestMessage.Headers.AcceptEncoding.First().Value;
if (!acceptedEncoding.Equals("gzip", StringComparison.InvariantCultureIgnoreCase)
&& !acceptedEncoding.Equals("deflate", StringComparison.InvariantCultureIgnoreCase))
{
return;
}
context.Response.Content = new CompressedContent(context.Response.Content, acceptedEncoding);
}
}
Now create another class and write the following code.
public class CompressedContent : HttpContent
{
private readonly string _encodingType;
private readonly HttpContent _originalContent;
public CompressedContent(HttpContent content, string encodingType = "gzip")
{
if (content == null)
{
throw new ArgumentNullException("content");
}
_originalContent = content;
_encodingType = encodingType.ToLowerInvariant();
foreach (var header in _originalContent.Headers)
{
Headers.TryAddWithoutValidation(header.Key, header.Value);
}
Headers.ContentEncoding.Add(encodingType);
}
protected override bool TryComputeLength(out long length)
{
length = -1;
return false;
}
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
Stream compressedStream = null;
switch (_encodingType)
{
case "gzip":
compressedStream = new GZipStream(stream, CompressionMode.Compress, true);
break;
case "deflate":
compressedStream = new DeflateStream(stream, CompressionMode.Compress, true);
break;
default:
compressedStream = stream;
break;
}
return _originalContent.CopyToAsync(compressedStream).ContinueWith(tsk =>
{
if (compressedStream != null)
{
compressedStream.Dispose();
}
});
}
}
Now use the following attribute in Controller or in any api action method like this
[Route("GetData")]
[CompressFilter]
public HttpResponseMessage GetData()
{
}