I am building a APIGateway proxy for our dotnet core microservices platform.
I used https://medium.com/#mirceaoprea/api-gateway-aspnet-core-a46ef259dc54 as a starting place, this picks up all requests by using
app.Run(async (context) =>
{
// Do things with context
});
You have the context for the request to the gateway, but how do I copy over the content data from the gateway request to a new request I am going to make to my API?
I see the ability to set the request content to a HttpContent object:
newRequest.Content = new StringContent(requestContent, Encoding.UTF8, "application/json");
But I want my application to take file uploads through the gateway, the only way I found to do it is to create a MultipartFormDataContent, but all examples on how to create a MultipartFormDataContent use a IFormFile instead of a HttpContext.
Is there a way to just copy the content on the initial apigateway request to my internal request:
using (var newRequest = new HttpRequestMessage(new HttpMethod(request.Method), serviceUrl))
{
// Add headers, etc
newRequest.Content = // TODO: how to get content from HttpContext
using (var serviceResponse = await new HttpClient().SendAsync(newRequest))
{
// handle response
}
}
You can use StreamContent for this, passing in the HttpContext.Request.Body Stream as the actual content to use. Here's what that looks like in your example:
newRequest.Content = new StreamContent(context.Request.Body);
As an aside, make sure that you use a shared instance of HttpClient.
Related
Task:
I am trying to send the image from the C# client at the C# Http Server. The image is stored in the memory only and should not be stored on any hard drive.
State:
I have C# client (.NET FW 4.5) sending an Http request on C# MVC API Controller, using MultiPartFormDataContent.
The client minimal code:
HttpContent content = new StreamContent(new FileStream("R:\\a.png", FileMode.Open));
using (var client = new HttpClient())
using (var form = new MultipartFormDataContent())
{
form.Add(content, "img");
var post = client.PostAsync("http://localhost:58652/api/bubla", form);
var resp = post.Result.Content.ReadAsStringAsync().Result;
}
I am trying to consume the stream on the server-side using:
[Route("api/bubla")]
[ApiController]
public class ValuesController : ControllerBase
{
[HttpPost]
public async Task<ActionResult<string>> Post([FromForm] MultipartFormDataContent content)
{
// ???
// e.g. :
// var provider = new MultipartFormDataStreamProvider("R:\\temp");
// await content.ReadAsMultipartAsync(provider);
// not working
}
}
Issue:
I have no idea how to consume the data at the server-side. I have read many tutorials, most of them requiring Request.Content which I do not have. The server-side code above will provide 'something' in the content variable, but I have no idea how to read it out.
Additional notes:
For the final solution, the image cannot be stored on the hard drive - only memory streams. (The reading from the disc in the code above is for the illustration only.
Maybe I am doing something generally wrong
I have the following code to set content-md5 in my GET method request using HttpClient
httpClient.DefaultRequestHeaders.TryAddWithoutValidation("content-md5", "value");
I cannot use HttpRequestMessage content to set it because it's not a POST method. When using Postman it works like a charm but fails when using HttpClient.GetAsync.
Client request a hmac to the server as follows
{
"content_to_hash": "my content"
}
The server will give response like this
{
"content_md5": "88af7ceab9fdafb76xxxxx",
"date": "Sat, 02 May 2020 00:13:16 +0700",
"hmac_value": "WfHgFyT792IENmK8Mqz9LysmP8ftOP00qA="
}
Now I have to access a GET request using that hmac where it's the problem because I cannot set in httpClient GET request header.
Here's the image
From reading the HttpClient and related source code, there's no way you can get around this and add the header to the actual request object headers. There is an internal list of invalid headers, which includes any Content-* headers. It has to be on a content object.
Therefore, my suggest solution is to create your own content object:
public class NoContentMd5 : HttpContent
{
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
return Task.CompletedTask;
}
protected override bool TryComputeLength(out long length)
{
length = 0;
return false;
}
public NoContentMd5(byte[] contentMd5)
{
this.Headers.ContentMD5 = contentMd5;
}
public NoContentMd5(string contentMd5)
{
this.Headers.TryAddWithoutValidation("Content-MD5", contentMd5);
}
}
This will add the Content-MD5 header with a value of your choosing, but the request won't contain a body.
The next problem you'll encounter is that you're trying to make a GET request with content, which isn't supported by the helper client.GetAsync(...) method. You'll have to make your own request object and use client.SendAsync(...) instead:
HttpClient client = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Get, "https://localhost/my/test/uri");
request.Content = new NoContentMd5("d41d8cd98f00b204e9800998ecf8427e ");
var result = await client.SendAsync(request);
Note that if you have your Content-MD5 hash as bytes, I've also added a constructor to NoContentMd5 for byte[] too.
The only potential issue with this is that it includes a Content-Length: 0 header. Hopefully that's OK with the API you're working with.
There's an alternative solution described in this answer to question with a similar issue. I'd argue against using it since is vulnerable to changes in the implementation details of HttpRequestHeaders (because it uses reflection, so if MS change the code, it might break) .
Aside from the fact that it's not considered good practice to send a body with GET request (see HTTP GET with request body), you can try this:
using (var content = new StringContent(string.Empty))
using (var request = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = new Uri("http://localhost"),
Content = content
})
{
request.Headers.TryAddWithoutValidation("content-md5", "value");;
using (var response = await httpClient.SendAsync(request))
{
response.EnsureSuccessStatusCode();
}
}
UPDATE:
The proper way would be to set the ContentMD5 property on the HttpContentHeaders, for example:
content.Headers.ContentMD5 = Convert.FromBase64String(hashString);
But as you pointed out in the comments, trying to send content in a GET request causes an error.
I have an API which has to do the following:
Incoming HttpRequest -> Logic to decide which endpoint -> Send call to specific endpoint (another API)
HttpResult -> Logic to manipulate response -> Send response back
I was wondering whether there is a clean way to do so? Currently using something like this:
var httpRequestMessage = new HttpRequestMessage()
{
Method = new HttpMethod(Request.Method),
RequestUri = new Uri(endpoint)
};
using (HttpClient httpClient = new HttpClient())
{
httpClient.DefaultRequestHeaders.Add("Authorization", Request.Headers["Authorization"][0]);
using (var response = await httpClient.SendAsync(httpRequestMessage))
{
using (HttpContent content = response.Content)
{
string data = await content.ReadAsStringAsync();
}
}
}
But I'm not really happy about the conversion from HttpRequest to HttpRequestMessage. I'd just like to pass the HttpRequest with another Uri but still be able to put logic in between the requesting and receiving.
When working with proxy middleware are you still able to put logic in between the request and result?
I have tried to create a simple console application.
We have a call system from 8x8 that provide a web streaming API but their documentation is very limited and nothing in C#.
The api service streams call statuses in near real time and I would like to get that 'stream' and be able to read and process it in realtime if possible. The response or Content Type is 'text/html'. But the actual body of the response can be declared as json - sample below:
{"Interaction":{"attachedData":{"attachedDatum":[{"attachedDataKey":"#pri","attachedDataValue":100},{"attachedDataKey":"callingName","attachedDataValue":999999999999},{"attachedDataKey":"cha","attachedDataValue":99999999999},{"attachedDataKey":"cnt","attachedDataValue":0},{"attachedDataKey":"con","attachedDataValue":0},{"attachedDataKey":"med","attachedDataValue":"T"},{"attachedDataKey":"pho","attachedDataValue":9999999999},{"attachedDataKey":"phoneNum","attachedDataValue":9999999999},{"attachedDataKey":"tok","attachedDataValue":999999999}]},"event":"InteractionCreated","inboundChannelid":9999999999,"interactionEventTS":9999999,"interactionGUID":"int-15b875d0da2-DJOJkDhDsrh3AIaFP8VkICv9t-phone-01-testist","resourceType":0}}
I have seen several posts concerning httpClient and the GetAsync methods but none of these appear to work as they appear to be for calls when a response is made, not something that constantly has a response.
Using fiddler for the call it does not appear to close so the stream is constantly running, so fiddler does not display any data until a separate user or instance connects.
When I use a browser the content is 'streamed' to the page and updates automatically and shows all the content (as above).
The api contains authentication so when another client connects and retrieves data the connected client closes and finally I am able to see the data that was gathering.
This is the code so and does return the big stream when another client connects but ideally I want a real time response and appears to just get stuck in the GETASYNC method:
var response = await client.GetAsync(address, HttpCompletionOption.ResponseHeadersRead);
if (response.IsSuccessStatusCode)
{
var responseContent = response.Content;
string responseString = await responseContent.ReadAsStringAsync();
Console.WriteLine(responseString);
}
Hopefully that's enough information for one of you clever people to help me in my predicament.
I was also having an issue consuming their streaming API and the examples I found that worked with the Twitter and CouchBase streaming API's did not work with 8x8. Both Twitter and CouchBase send line terminators in their pushes so the solution relied on ReadLine to pull in the feed. Since 8x8 does not send terminators you'll need to use ReadBlock or better ReadBlockAsync.
The following code shows how to connect using credentials and consume their feed:
private static async Task StreamAsync(string url, string username, string password)
{
var handler = new HttpClientHandler()
{
Credentials = new NetworkCredential {UserName = username, Password = password},
PreAuthenticate = true
};
// Client can also be singleton
using (var client = new HttpClient(handler))
{
client.Timeout = TimeSpan.FromMilliseconds(Timeout.Infinite);
var request = new HttpRequestMessage(HttpMethod.Get, url);
request.Headers.Connection.Add("keep-alive");
using (var response = await client.SendAsync(
request,
HttpCompletionOption.ResponseHeadersRead))
{
using (var body = await response.Content.ReadAsStreamAsync())
{
using (var reader = new StreamReader(body))
{
while (!reader.EndOfStream)
{
var buffer = new char[1024];
await reader.ReadBlockAsync(buffer, 0, buffer.Length);
Console.WriteLine(new string(buffer));
}
}
}
}
}
}
I have a HttpResponse object as a result of HttpClient.SendAsync() call. The response has a chunked transfer encoding and results in 1.5 GB of data.
I want to pass this data through OWIN pipeline. To do this I need to convert it to a stream. Simplified code to do this is:
public async Task Invoke(IDictionary<string, object> environment)
{
var httpContent = GetHttpContent();
var responseStream = (Stream)environment["owin.ResponseBody"];
await httpContent.CopyToAsync(responseStream);
}
However, the last line results in copying the entire stream to the memory. And when I use wget to download the data directly from the backend server, it is downloaded successfully and shows a progress bar (although it doesn't know the overall size since it is chunked). But when I use wget to download data from my OWIN-hosted application it sticks on sending the request.
How should I stream this data through an OWIN pipeline to prevent copying it to memory?
EDIT
This is how I get the HttpResponse:
var client = new HttpClient(new HttpClientHandler());
// …and then:
using (var request = new HttpRequestMessage { RequestUri = uri, Method = HttpMethod.Get })
{
return client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead).Result;
}
I assume this is in IIS? System.Web also buffers responses: https://msdn.microsoft.com/en-us/library/system.web.httpresponse.bufferoutput(v=vs.110).aspx
See server.DisableResponseBuffering in
https://katanaproject.codeplex.com/wikipage?title=OWIN%20Keys&referringTitle=Documentation