Develop Reference

Forward multipart/x-mixed-replace live stream ASP.NET Core

I've an ASP.NET Core server that, upon client request, start fetching a stream from an AXIS camera and returns it to the client for displaying. If the http request to server is made directly by the browser by means of img src attribute, it works fine. But if I use HttpClient (which I need to do since I need CancellationToken), the httpClient.GetStreamAsync instruction get stuck and I cannot parse the returned data.
Server side Controller (take from here):
[ApiController]
[Route("[controller]")]
public class CameraSystemController : ControllerBase
{
private string _contentTypeStreaming = "multipart/x-mixed-replace;boundary=myboundary";
private HttpClient _httpClient = new HttpClient(new HttpClientHandler { Credentials = ...});
[HttpGet("getStream")]
public async Task<IActionResult> GetStream(CancellationToken token)
{
Stream stream = await _httpClient.GetStreamAsync("http://.../axis-cgi/mjpg/video.cgi?resolution=800x600&compression=50", token);
if (stream != null) {
FileStreamResult result = new FileStreamResult(stream, _contentTypeStreaming) {
EnableRangeProcessing = true
};
return result;
} else {
return new StatusCodeResult((int)HttpStatusCode.ServiceUnavailable);
}
}
}
Now, as I said, as long as I make the browser perform the http request by means of
// LiveCamera.razor
<img src="CameraSystem/getStream" onerror="#OnImgLoadingError" alt="">
Data acquired in live mode by the camera is correctly displayed in browser.
If instead I make the request in the client this way:
//LiveCamera.razor.cs
[Inject] public HttpClient Http { get; private set; }
private CancellationTokenSource _cancellationTokenSource = new CancellationTokenSource();
private async void StartStreamingRequest(object sender, ElapsedEventArgs e) {
Console.WriteLine("I'm about to request live stream");
Stream responseStream = await Http.GetStreamAsync("CameraSystem/getStream", _cancellationTokenSource.Token);
Console.WriteLine("Header found!");
string boundary = "myboundary";
for (MultipartReader streamReader = new MultipartReader(boundary, responseStream); ;) {
MultipartSection nextFrame = await streamReader.ReadNextSectionAsync(_cancellationTokenSource.Token);
DisplayOneFrameCallback(nextFrame.Body);
}
}
private void DisplayOneFrameCallback(Stream body)
{
StreamReader reader = new StreamReader(body);
string frameData = reader.ReadToEnd();
_imgSrc = "data:image/jpeg;base64," + frameData;
StateHasChanged();
}
In this seconds case the request is performed correctly (server-side I can see the code executed and from task manager I can see the bandwidth usage increasing) but the client got stuck on await instruction, and subsequent code is never executed. Now, in Microsoft documentation under GetStreamAsync it is possible to read
This operation will not block. The returned Task object will complete after the response headers are read. This method does not read nor buffer the response body.
so I would expect it not to block. Thus I suppose there could be a problem in the header I'm producing server-side, even though browser request works just fine.
Just out of curiosity, I've captured with Wireshark the data between the camera and the server. The header and the initial part of the body are like this:
..-...HTTP/1.0 200 OK..Cache-Control: no-cache..Pragma: no-cache..Expires: Thu, 01 Dec 1994 16:00:00 GMT..Connection: close..Content-Type: multipart/x-mixed-replace;boundary=myboundary....--myboundary..Content-Type: image/jpeg..Content-Length: 30146......
I've double-checked with browser developer tools and I can confirm that sending the request from browser or from httpClient yield the same exact request. Furthermore right-clicking on the request url and issueing "open in new tab" opens a tab where I can see the camera live stream in both cases.
Could you please help me in this?

Gzip MultipartFormDataContent Time Out When Posted To The Server

I'm trying to POST a MultipartFormDataContent in a real case scenario, a data content object could contain anything from a simple string to a video file I'm using a serialized object down there, just a proof of concept.
Also I would like to note that using JSON objects wont serve my real life scenarios
public class GzipMultipartContent : MultipartFormDataContent
{
public GzipMultipartContent()
{
Headers.ContentEncoding.Add("gzip");
}
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
return Task.Factory.StartNew(() =>
{
using (var gzip = new GZipStream(stream, CompressionMode.Compress, true))
base.SerializeToStreamAsync(gzip, context);
});
}
}
and here's how I call it
var gzipped = new GzipMultipartContent();
var test = new TestClass();
gzipped.Add(new StringContent(JsonConvert.SerializeObject(test)), "value");
var client = new HttpClient();
var result = client.PostAsync("http://localhost:60001/api/Home/", gzipped).Result;
and here's the post action in the controller
// POST: api/Home
[HttpPost]
public void Post([FromForm] object value)
{
}
I have added a break point at the server side and made sure it doesn't even reach the Post method, also I have tried with a normal POST request to make sure that it's not a server configuration problem or a URL mistyping

Client side
If the code in question is your real code, then there are at least two issues:
Did not wait on base.SerializeToStreamAsync
You created a new task, but you did not wait until the base class completed writing to the compressed stream in the task. So you could send unpredictable content to server.
Did not override Content-Length
MultipartFormDataContent calculates length of content based on data not compressed, since you have compressed data, you must re-compute length for the compressed data.
Frankly, I don't think you need to inherit from MultipartFormDataContent to make it compressed. Instead, you could compress the entire MultipartFormDataContent in a wrapper HttpContent:
public class GzipCompressedContent : HttpContent
{
private readonly HttpContent _content;
public GzipCompressedContent(HttpContent content)
{
// Copy original headers
foreach (KeyValuePair<string, IEnumerable<string>> header in content.Headers)
{
Headers.TryAddWithoutValidation(header.Key, header.Value);
}
Headers.ContentEncoding.Add("gzip");
_content = content;
}
protected override async Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
using (var gzip = new GZipStream(stream, CompressionMode.Compress, true))
{
// Compress the entire original content
await _content.CopyToAsync(gzip);
}
}
protected override bool TryComputeLength(out long length)
{
// Content-Lenght is optional, so set to -1
length = -1;
return false;
}
}
And use it:
var test = new TestClass();
using (var client = new HttpClient())
{
var form = new MultipartFormDataContent();
form.Add(new StringContent(JsonConvert.SerializeObject(test)), "value");
var compressed = new GzipCompressedContent(form);
var result = await client.PostAsync(..., compressed);
}
Server side
Your server needs to support compressed stream.
For example, by default, ASP.NET Core does not support compressed request, if you send GZip compressed request to an ASP.NET Core application, you will see exception:
System.IO.IOException: Unexpected end of Stream, the content may have already been read by another component.
at Microsoft.AspNetCore.WebUtilities.MultipartReaderStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
The above exception happens in action invocation pipeline before any controller action takes place. So controller actions in this case could not be reached.
To fix such, you will need to enable server side request decompression support.
If you are using ASP.NET Core, check out this nuget package.

I am not sure if I understood the issue, but if it is that your request isn't getting to the server, while your "normal" POST requests are, then I think that I found your problem.
I think that the issue is that your server doesn't know what Content-Type is coming to it. I literally copy-pasted your code, but added
Headers.ContentType = new MediaTypeHeaderValue("application/x-gzip");
to GzipMultipartContent.cs ctor.
After I added the type, I was hitting my breakpoint in the localhost server.
Source: Content-Type
In requests, (such as POST or PUT), the client tells the server what type of data is actually sent.

Copy HttpContext request content data to new request

I am building a APIGateway proxy for our dotnet core microservices platform.
I used https://medium.com/#mirceaoprea/api-gateway-aspnet-core-a46ef259dc54 as a starting place, this picks up all requests by using
app.Run(async (context) =>
{
// Do things with context
});
You have the context for the request to the gateway, but how do I copy over the content data from the gateway request to a new request I am going to make to my API?
I see the ability to set the request content to a HttpContent object:
newRequest.Content = new StringContent(requestContent, Encoding.UTF8, "application/json");
But I want my application to take file uploads through the gateway, the only way I found to do it is to create a MultipartFormDataContent, but all examples on how to create a MultipartFormDataContent use a IFormFile instead of a HttpContext.
Is there a way to just copy the content on the initial apigateway request to my internal request:
using (var newRequest = new HttpRequestMessage(new HttpMethod(request.Method), serviceUrl))
{
// Add headers, etc
newRequest.Content = // TODO: how to get content from HttpContext
using (var serviceResponse = await new HttpClient().SendAsync(newRequest))
{
// handle response
}
}

You can use StreamContent for this, passing in the HttpContext.Request.Body Stream as the actual content to use. Here's what that looks like in your example:
newRequest.Content = new StreamContent(context.Request.Body);
As an aside, make sure that you use a shared instance of HttpClient.

How to Post a web request and receive a file from web API response?

I have an Infopath Form Template on Sharepoint, I want to add a button there so when the user clicks on it, it will POST an string to the following Web API. The following web API is tested and returns an excel file as shown:
I want to Post the FileName of the excel file using post request and it is important for me the request method to be POST type. and then the user will download a file with the specified 'FileName'.
Actually i want to use post method because at the next stage i will send the content of the excel file too.
Important Note: I only can use .Net FrameWork 3.5 because this is the only framework supported in InfoPath Form Templates.
[HttpPost]
public HttpResponseMessage Post([FromBody]string FileName)
{
string reqBook = "c:\somefile.xlsx";
//converting Excel(xlsx) file into bytes array
var dataBytes = File.ReadAllBytes(reqBook);
//adding bytes to memory stream
var dataStream = new MemoryStream(dataBytes);
HttpResponseMessage httpResponseMessage = Request.CreateResponse(HttpStatusCode.OK);
httpResponseMessage.Content = new StreamContent(dataStream);
httpResponseMessage.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment");
httpResponseMessage.Content.Headers.ContentDisposition.FileName = FileName;
httpResponseMessage.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/octet-stream");
return httpResponseMessage;
}

When you perform the HttpPost on the client side, you will want to read the HttpResponseStream to get the byte data of the response stream.
Once you have the response stream data, you can then deserialize it to the type of object in C# you want, or you could alternatively just write it to the disk as
File.WriteAllBytes("someexcel.xlsx",data);
An easy way to do it would be with the HttpClient class.
HttpClient client = new HttpClient();
var response = client.PostAsync("", null).Result;
var content = response.Content.ReadAsByteArrayAsync().Result;
File.WriteAllBytes("excel.xlsx", content);
Just fill in the PostAsync bit with the Url and the content you wish to post.
I am using .Result to keep everything synchronous - but you can use 'await' if you prefer.
If you are working with HttpWebRequests - then the process becomes more complicated, as you need to manage the streams yourself.
The HttpClient will manage and handle it all for you - so I recommend it, unless there is something special it needs to do that it currently does not.
Due to your .Net 3.5 requirement:
private static HttpWebResponse MakeRequest(string url, string postArgument)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "POST";
request.ContentType = "multipart/form-data;";
Stream stream = request.GetRequestStream();
string result = string.Format("arg1={0}", postArgument);
byte[] value = Encoding.UTF8.GetBytes(result);
stream.Write(value, 0, value.Length);
stream.Close();
return (HttpWebResponse)request.GetResponse();
}
You can then do:
var response = MakeRequest("http://mywebsite.com/ProcessExcel", "accounts.xlsx");
And then do
Stream objStream = response .GetResponseStream();
BinaryReader breader = new BinaryReader(objStream);
byte[] data= breader.ReadBytes((int)webresponse.ContentLength);
File.WriteAllBytes("excel.xlsx",data);

Streaming Content constantly updating 8x8 streaming service

I have tried to create a simple console application.
We have a call system from 8x8 that provide a web streaming API but their documentation is very limited and nothing in C#.
The api service streams call statuses in near real time and I would like to get that 'stream' and be able to read and process it in realtime if possible. The response or Content Type is 'text/html'. But the actual body of the response can be declared as json - sample below:
{"Interaction":{"attachedData":{"attachedDatum":[{"attachedDataKey":"#pri","attachedDataValue":100},{"attachedDataKey":"callingName","attachedDataValue":999999999999},{"attachedDataKey":"cha","attachedDataValue":99999999999},{"attachedDataKey":"cnt","attachedDataValue":0},{"attachedDataKey":"con","attachedDataValue":0},{"attachedDataKey":"med","attachedDataValue":"T"},{"attachedDataKey":"pho","attachedDataValue":9999999999},{"attachedDataKey":"phoneNum","attachedDataValue":9999999999},{"attachedDataKey":"tok","attachedDataValue":999999999}]},"event":"InteractionCreated","inboundChannelid":9999999999,"interactionEventTS":9999999,"interactionGUID":"int-15b875d0da2-DJOJkDhDsrh3AIaFP8VkICv9t-phone-01-testist","resourceType":0}}
I have seen several posts concerning httpClient and the GetAsync methods but none of these appear to work as they appear to be for calls when a response is made, not something that constantly has a response.
Using fiddler for the call it does not appear to close so the stream is constantly running, so fiddler does not display any data until a separate user or instance connects.
When I use a browser the content is 'streamed' to the page and updates automatically and shows all the content (as above).
The api contains authentication so when another client connects and retrieves data the connected client closes and finally I am able to see the data that was gathering.
This is the code so and does return the big stream when another client connects but ideally I want a real time response and appears to just get stuck in the GETASYNC method:
var response = await client.GetAsync(address, HttpCompletionOption.ResponseHeadersRead);
if (response.IsSuccessStatusCode)
{
var responseContent = response.Content;
string responseString = await responseContent.ReadAsStringAsync();
Console.WriteLine(responseString);
}
Hopefully that's enough information for one of you clever people to help me in my predicament.

I was also having an issue consuming their streaming API and the examples I found that worked with the Twitter and CouchBase streaming API's did not work with 8x8. Both Twitter and CouchBase send line terminators in their pushes so the solution relied on ReadLine to pull in the feed. Since 8x8 does not send terminators you'll need to use ReadBlock or better ReadBlockAsync.
The following code shows how to connect using credentials and consume their feed:
private static async Task StreamAsync(string url, string username, string password)
{
var handler = new HttpClientHandler()
{
Credentials = new NetworkCredential {UserName = username, Password = password},
PreAuthenticate = true
};
// Client can also be singleton
using (var client = new HttpClient(handler))
{
client.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite);
var request = new HttpRequestMessage(HttpMethod.Get, url);
request.Headers.Connection.Add("keep-alive");
using (var response = await client.SendAsync(
request,
HttpCompletionOption.ResponseHeadersRead))
{
using (var body = await response.Content.ReadAsStreamAsync())
{
using (var reader = new StreamReader(body))
{
while (!reader.EndOfStream)
{
var buffer = new char[1024];
await reader.ReadBlockAsync(buffer, 0, buffer.Length);
Console.WriteLine(new string(buffer));
}
}
}
}
}
}