In case we search for a failure in our application, we want to add the possibility to log all request and responses given by our Nancy service. In order to accomplish this, we added some pipeline hooks.
But unfortunately I'm not able to get the body of our responses. When I execute the delegate that renders the contents to the response stream (Contents), my response stream (ms) is disposed.
public void LogResponse(NancyContext context) {
string CreateResponseMessage() {
return $"Response (Client: {context.Request.UserHostAddress}) [{context.Request.Method}] {context.Request.Url}, " +
$"Status code: {(int) context.Response.StatusCode} {context.Response.StatusCode}";
}
string CreateContentMessage() {
using (var ms = new MemoryStream())
{
// For some reason, when executing this line, my ms is disposed
context.Response.Contents(ms);
ms.Seek(0, SeekOrigin.Begin);
using (var sr = new StreamReader(ms)) {
var headers = FormatHeaders(context.Response.Headers);
var content = sr.ReadToEnd();
return
$"Headers:{NewLine}{headers}{NewLine}" +
$"Body:{NewLine} {content}";
}
}
}
Log.Info(CreateResponseMessage());
Log.Verbose(CreateContentMessage);
}
Is this the wrong way to get the content of the body? Is there a way at all?
Related
I have a .NET Core 3 MVC app that needs to read a file from one location over HTTP and then re-deliver it back out to the response. Some of these files will be ~200MB in size.
What I have works, but it reads the whole file into memory before sending the File result out to the client. Is there a way to make it essentially a passthrough where the read stream flows into the response stream so that very little memory is required on the server?
This is what I have now but I do not think will perform well with large files:
if (requestedFile != null)
{
using (var client = new System.Net.Http.HttpClient())
{
using (var result = await client.GetAsync(requestedFile.DownloadUrl))
{
if (result.IsSuccessStatusCode)
{
var bytes = await result.Content.ReadAsByteArrayAsync();
return File(bytes, "application/zip", "largefile.zip");
}
}
}
}
I have also tried this which results in a runtime error of "Cannot access a closed Stream":
using (var client = new System.Net.Http.HttpClient())
{
using (var httpResponseMessage = await client.GetAsync(requestedFile.DownloadUrl))
{
return File(await httpResponseMessage.Content.ReadAsStreamAsync(), "application/zip", "largefile.zip");
}
}
Edit:
Solution after some trial and error was remocing all using statements and letting the FileStreamResult close the stream on its own. So I ended up with:
var client = new HttpClient();
var result = await client.GetAsync(requestedFile.DownloadUrl);
var stream = await result.Content.ReadAsStreamAsync();
return new FileStreamResult(stream, "application/zip")
{
FileDownloadName = "largefile.zip"
};
One of the overloads for File is a Stream. Just get that URL as a Stream or read the response body as a stream and immediately return that in the overload:
var client = new System.Net.Http.HttpClient();
var result = await client.GetAsync(requestedFile.DownloadUrl);
var stream = await result.Content.ReadAsStreamAsync();
return File(stream,"application/pdf", "Invoice.pdf");
Note: this will fail if you wrap the Stream in a using block as the FileResult already closes the Stream.
For Export Excel
var client = new System.Net.Http.HttpClient();
var comp = client.GetAsync($"RewardEmployee/ExportExcelCalculate?rewardConfigId={id}").Result;
`var stream = await result.Content.ReadAsStreamAsync();
return File(stream, "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"ExportExcelCalculateRewardEmployee.xlsx");
I'm trying my hand at .NET Core but I'm stuck trying to convert multipart/form-data to an application/octet-stream to send via a PUT request. Anybody have any expertise I could borrow?
[HttpPost("fooBar"), ActionName("FooBar")]
public async Task<IActionResult> PostFooBar() {
HttpResponseMessage putResponse = await _httpClient.PutAsync(url, HttpContext.Request.Body);
}
Update: I think I might have two issues here:
My input format is multipart/form-data so I need to split out the file from the form data.
My output format must be application-octet stream but PutAsync expects HttpContent.
I had been trying to do something similar and having issues. I needed to PUT large files (>1.5GB) to a bucket on Amazon S3 using a pre-signed URL. The implementation on Amazon for .NET would fail for large files.
Here was my solution:
static HttpClient client = new HttpClient();
client.Timeout = TimeSpan.FromMinutes(60);
static async Task<bool> UploadLargeObjectAsync(string presignedUrl, string file)
{
Console.WriteLine("Uploading " + file + " to bucket...");
try
{
StreamContent strm = new StreamContent(new FileStream(file, FileMode.Open, FileAccess.Read));
strm.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/octet-stream");
HttpResponseMessage putRespMsg = await client.PutAsync(presignedUrl, strm);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
return false;
}
return true;
}
Turns out Request has a Form property that contains a Files property that has an OpenReadStream() function on it to convert it into a stream. How exactly I was supposed to know that, I'm not sure.
Either way, here's the solution:
StreamContent stream = new StreamContent(HttpContext.Request.Form.Files[0].OpenReadStream());
HttpResponseMessage putResponse = await _httpClient.PutAsync(url, stream);
I am uploading video to Twitter in chunks using Twitter Media API and then trying to call FINALIZE.
Keep on getting:
"Large file can not be finalized synchronously."
result = new RequestBuilder(oauth, "POST", "https://upload.twitter.com/1.1/media/upload.json")
.AddParameter("command", "FINALIZE")
.AddParameter("media_id", media_id)
.Execute().Result;
//code from Execute
try
{
response = await request.GetResponseAsync().ConfigureAwait(false);
using (var stream = response.GetResponseStream())
{
using (var reader = new StreamReader(stream))
{
content = reader.ReadToEnd();
}
}
}
catch (WebException ex)
{
using (var stream = ex.Response.GetResponseStream())
{
using (var reader = new StreamReader(stream))
{
content = reader.ReadToEnd();
}
}
throw;
}
finally
{
if (response != null)
{
((IDisposable)response).Dispose();
}
}
I never received this exception. Have you tried using tweetinvi and see if it works for the same file?
Please note that Twitter only supports 15MB video.
Wiki for video upload : https://github.com/linvi/tweetinvi/wiki/Upload#upload-videos. The video are automatically chunked to satisfy Twitter requirements (5MB per chunk for video).
var video = File.ReadAllBytes("path");
var media = Upload.UploadVideo(video);
var tweet = Tweet.PublishTweet("hello", new PublishTweetOptionalParameters
{
Medias = { media }
});
If you really want to do this manually you can still use the ChunkUploader:
https://github.com/linvi/tweetinvi/wiki/Upload#chunked-uploads
Twitter does support larger videos up to 512Mb, requirements can be found on this page: https://developer.twitter.com/en/docs/media/upload-media/uploading-media/media-best-practices
In order to upload larger files you have to set the media_category parameter to "tweet_video" in the INIT call. The FINALIZE call will then give you processing_info that you need to follow up with the STATUS command call.
.post("media/upload", {
command: "INIT",
total_bytes: mediaSize,
media_type: mediaType,
media_category: 'tweet_video'
})
I have recently started working with web api's.
I need to download a file in C# project from a web api, which works fine when I hit the web api using postman's send and download option. Refer to the image, also please check the response in header's tab. This way, I am able to directly download the file to my computer.
I want to do the same from my C# project, I found following two links which shows how to download a file from web api.
https://code.msdn.microsoft.com/HttpClient-Downloading-to-4cc138fd
http://blogs.msdn.com/b/henrikn/archive/2012/02/16/downloading-a-google-map-to-local-file.aspx
I am using the following code in C# project to get the response:
private static async Task FileDownloadAsync()
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Add("Accept", "text/html");
try
{
// _address is exactly same which I use from postman
HttpResponseMessage response = await client.GetAsync(_address);
if (response.IsSuccessStatusCode)
{
}
else
{
}
}
catch (Exception)
{
throw;
}
}
}
However I am not getting the response at all (before I can start to convert the response to a file), please check the error message coming:
What am I missing here, any help would be appreciated.
As the (500s) error says - it's the Server that rejects the request. The only thing I see that could cause an issues is the charset encoding. Yours is the default UTF-8. You could try with other encodings.
Below method uses:
SSL certificate (comment out code for cert, if you don't use it)
Custom api header for additional layer of security (comment out Custom_Header_If_You_Need code, if you don't need that)
EnsureSuccessStatusCode will throw an error, when response is not 200. This error will be caught in and converted to a human readable string format to show on your screen (if you need to). Again, comment it out if you don't need that.
private byte[] DownloadMediaMethod(string mediaId)
{
var cert = new X509Certificate2("Keystore/p12_keystore.p12", "p12_keystore_password");
var handler = new WebRequestHandler { ClientCertificates = { cert } };
using (var client = new HttpClient(handler))
{
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Add("Custom_Header_If_You_Need", "Value_For_Custom_Header");
var httpResponse = client.GetAsync(new Uri($"https://my_api.my_company.com/api/v1/my_media_controller/{mediaId}")).Result;
//Below one line is most relevant to this question, and will address your problem. Other code in this example if just to show the solution as a whole.
var result = httpResponse.Content.ReadAsByteArrayAsync().Result;
try
{
httpResponse.EnsureSuccessStatusCode();
}
catch (Exception ex)
{
if (result == null || result.Length == 0) throw;
using (var ms = new MemoryStream(result))
{
var sr = new StreamReader(ms);
throw new Exception(sr.ReadToEnd(), ex);
}
}
return result;
}
}
Once you have your http response 200, you can use the received bytes[] as under to save them in a file:
using (var fs = new FileStream(fileName, FileMode.Create, FileAccess.Write))
{
fs.Write(content, 0, content.Length);
}
Your request header says that you accept HTML responses only. That could be a problem.
Context:
I am improving my .dll to execute WebRequests (Get and Post), adding a new feature, that is : automatic encoding detecting after the response
How will it work:
User of the library configures it's default parameters for requests, including Encoding
Library executes the request for a certain url
Library checks if the encoding of the page is the same as the pre-configured one (via checking the value of the meta tag)
If the encodes are not the same, i re-encode the response using the right encoding (the one found on the page)
Code Fragment:
// Executes web request and wait for response
using (HttpWebResponse resp = (HttpWebResponse) m_HttpWebRequest.GetResponse())
{
using (var stream = resp.GetResponseStream ())
{
using (var reader = new StreamReader (stream))
{
// Reading Stream to the end
response = reader.ReadToEnd ();
// Identifying the encode used on the page
// I will not paste method here, but it works
m_PageEncoding = IdentifyEncoding (response);
}
// Checking if the page encode is not the same as the preconfigured one
if (m_PageEncoding != System.Text.Encoding.GetEncoding (m_encoding))
{
using (var encodedReader = new StreamReader (stream, m_PageEncoding))
{
response = encodedReader.ReadToEnd();
}
}
}
}
Problem:
Once i create another Reader, which is the EncodedReaderwith the Encoding argument, an exception is thrown: Stream was not readable.
If, i nest the readers within the responseStream using block, the response value after the second reading is always "empty"
using (var stream = resp.GetResponseStream ())
{
using (var reader = new StreamReader (stream))
{
// Reading Stream to the end
response = reader.ReadToEnd ();
// Identifying the encode used on the page
m_PageEncoding = IdentifyEncoding (response);
// Checking if the page encode is not the one i've used as argument
if (m_PageEncoding != System.Text.Encoding.GetEncoding(m_encoding))
{
using (var encodedReader = new StreamReader(stream, m_PageEncoding))
{
response = encodedReader.ReadToEnd();
}
}
}
}
Question:
How can i execute the ReadToEnd method twice, on the same WebResponse, without executing the request twice, which would be lousy.