Develop Reference

Asp.Net Core 2 + Google Cloud Storage download Memory Stream

I'm working on an Asp.Net Core 2 Web Api and I have to make an endpoint to download the file. This file is not public, so I cannot use the MediaLink property of the google storage object. I'm using their C# library.
In the piece of code you will see bellow _storageClient was created like this: _storageClient = StorageClient.Create(cred);. The client is working, just showing which class it is.
[HttpGet("DownloadFile/{clientId}/{fileId}")]
public async Task<IActionResult> DownloadFile([FromRoute] long fileId, long clientId)
{
// here there are a bunch of logic and permissions. Not relevant to the quest
var stream = new MemoryStream();
try
{
stream.Position = 0;
var obj = _storageClient.GetObject("bucket name here", "file.png");
_storageClient.DownloadObject(obj, stream);
var response = File(stream, obj.ContentType, "file.png"); // FileStreamResult
return response;
}
catch (Exception ex)
{
throw;
}
}
The variable obj comes OK. with all properties filled as expected. The stream seems to be filled properly. it has length and everything, but it returns me a 500 error that I cannot even catch.
I cannot see what I'm doing wrong, maybe how I'm using memory stream but I can;t even catch the error.
Thanks for any help

You're rewinding the stream, but before you've written anything to it - but you're not rewinding it afterwards. I'd expect that to result in an empty response rather than a 500 error, but I'd at least move the stream.Position call to after the download:
var obj = _storageClient.GetObject("bucket name here", "file.png");
_storageClient.DownloadObject(obj, stream);
stream.Position = 0;
Note that you don't need to fetch the object metadata before downloading it. You can just use:
_storageClient.DownloadObject("bucket name here", "file.png", stream);
stream.Position = 0;

Solution can be like below.
[HttpGet("get-file")]
public ActionResult GetFile()
{
var storageClient = ...;
byte[] buffer;
using (var memoryStream = new MemoryStream())
{
storageClient.DownloadObject("bucket name here"+"/my-file.jpg", memoryStream);
buffer = memoryStream.ToArray();
}
return File(buffer, "image/jpeg", "my-file.jpg");
}

Stream was not readable

I have code below that read ftp response stream and write data to two different files (test1.html & test2.html). The 2nd StreamReader throw a stream was not readable error. The response stream should be readable because it's not out of scope yet and the dispose shouldn't be called. Can someone explain why?
static void Main(string[] args)
{
// Make sure it is ftp
if (Properties.Settings.Default.FtpEndpoint.Split(':')[0] != Uri.UriSchemeFtp) return;
// Intitalize object to used to communicuate to the ftp server
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(Properties.Settings.Default.FtpEndpoint + "/test.html");
// Credentials
request.Credentials = new NetworkCredential(Properties.Settings.Default.FtpUser, Properties.Settings.Default.FtpPassword);
// Set command method to download
request.Method = WebRequestMethods.Ftp.DownloadFile;
// Get response
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
using (Stream output = File.OpenWrite(#"C:\Sandbox\vs_projects\FTP\FTP_Download\test1.html"))
using (Stream responseStream = response.GetResponseStream())
{
responseStream.CopyTo(output);
Console.WriteLine("Successfully wrote stream to test.html");
try
{
using (StreamReader reader = new StreamReader(responseStream))
{
string file = reader.ReadToEnd();
File.WriteAllText(#"C:\Sandbox\vs_projects\FTP\FTP_Download\test2.html", file);
Console.WriteLine("Successfully wrote stream to test2.html");
}
}
catch (Exception ex)
{
Console.WriteLine($"Exception: {ex}");
}
}
}

You can't read from the stream twice. After this call:
responseStream.CopyTo(output);
... you've already read all the data in the stream. There's nothing left to read, and you can't "rewind" the stream (e.g. seeking to the beginning) because it's a network stream. Admittedly I'd expect it to just be empty rather than throwing an error, but the details don't really matter much as it's not a useful thing to try to do.
If you want to make two copies of the same data, the best option is to copy it to disk as you're already doing, then read the file that you've just written.
(Alternatively, you could just read it into memory by copying to a MemoryStream, then you can rewind that stream and read from it repeatedly. But if you're already going to save it to disk, you might as well do that first.)

convert pdf to stream from url

I have a PDF which is hosted in say http://test.com/mypdf.pdf.
How can I convert the PDF to Stream and then using this Stream convert it back to PDF.
I tried the following but got an exception(see image):
private static Stream ConvertToStream(string fileUrl)
{
HttpWebResponse aResponse = null;
try
{
HttpWebRequest aRequest = (HttpWebRequest)WebRequest.Create(fileUrl);
aResponse = (HttpWebResponse)aRequest.GetResponse();
}
catch (Exception ex)
{
}
return aResponse.GetResponseStream();
}

This will work:
private static Stream ConvertToStream(string fileUrl)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(fileUrl);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
try {
MemoryStream mem = new MemoryStream();
Stream stream = response.GetResponseStream();
stream.CopyTo(mem,4096);
return mem;
} finally {
response.Close();
}
}
However you are entirely responsible for the lifetime of the returned memory stream.
A better approach is:
private static void ConvertToStream(string fileUrl, Stream stream)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(fileUrl);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
try {
Stream response_stream = response.GetResponseStream();
response_stream.CopyTo(stream,4096);
} finally {
response.Close();
}
}
You can then do something like:
using (MemoryStream mem = new MemoryStream()) {
ConvertToStream('http://www.example.com/',mem);
mem.Seek(0,SeekOrigin.Begin);
... Do something else ...
}
You may also be able to return the response stream directly but you'd have to check on the lifetime of that, releasing the response may release the stream, hence the mem copy.

You may want to take a look at WebClient.DownloadFile.
You give it a URL and local file name and it saves the file straight to disk. Might save you a step or two.
You could also try WebClient.DownloadData which saves the file to an in-memory byte[].
EDIT
You did not specify the protocol of the web-service you are posting the file to. The simplest form (RESTful) would be just to POST the file to data to another URL. Here is how you would do that.
using (WebClient wc = new WebClient())
{
// copy data to byte[]
byte[] data = wc.DownloadData("http://somesite.com/your.pdf");
// POST data to another URL
wc.Headers.Add("Content-Type","application/pdf");
wc.UploadData("http://anothersite.com/your.pdf", data);
}
If you are using SOAP, you would have to convert the file to a Base64 string, but hopefully you are using a generated client which takes care of that for you. If you could elaborate on the type of web-service you are sending the file to, I could probably provide some more information..

how can be possible to programmatically upload and retrieve photos from PicasaWeb in C#

Is it possible to use PicasaWeb to host photos for my website? My requirement to upload and access within my ASP.NET Website.

Yes, technically. You can use the Picasa Web Albums Data API to access the metadata about your images and then display them from Picasa. For an album, this is not a horrible idea, but I would not use this method for your site graphics.

public string UploadImage(byte[] imageBytes_, string imageName_)
{
string url = string.Empty;
try
{
PicasaEntry newPhoto = null;
MemoryStream ms = new MemoryStream();
ms.Write(imageBytes_, 0, imageBytes_.Length);
if (_albumFeed != null)
{
PicasaEntry photoEntry = new PhotoEntry();
photoEntry.MediaSource = new Google.GData.Client.MediaFileSource(ms, imageName_, "image/jpeg");
newPhoto = this._service.Insert<PicasaEntry>(new Uri(this._albumFeed.Post), photoEntry);
}
url = newPhoto.FeedUri;
}
catch (Exception ex)
{
throw new DneException("Error while uploading photo", ex );
}
return url ;
}
I am uploading a zipped images and extracting using SharpZipLib library. I am getting each file in chunk of bytes and which I am converting into MemoryStream later to pass the MediaSource object. I am getting 400 Bad Request error each time. When I am passing FileStream, it works fine but doesn't work with MemoryStream

Get filename while downloading it

We are providing files that are saved in our database and the only way to retrieve them is by going by their id as in:
www.AwesomeURL.com/AwesomeSite.aspx?requestedFileId=23
Everything is working file as I am using the WebClient Class.
There's only one issue that I am facing:
How can I get the real filename?
My code looks like this atm:
WebClient client = new WebClient ();
string url = "www.AwesomeURL.com/AwesomeSite.aspx?requestedFileId=23";
client.DownloadFile(url, "IDontKnowHowToGetTheRealFileNameHere.txt");
All I know is the id.
This does not happen when I try accessing url from the browser where it get's the proper name => DownloadedFile.xls.
What's the proper way to get the correct response?

I had the same problem, and I found this class: System.Net.Mime.ContentDisposition.
using (WebClient client = new WebClient()){
client.OpenRead(url);
string header_contentDisposition = client.ResponseHeaders["content-disposition"];
string filename = new ContentDisposition(header_contentDisposition).FileName;
...do stuff...
}
The class documentation suggests it's intended for email attachments, but it works fine on the server I used to test, and it's really nice to avoid the parsing.

Here is the full code required, assuming the server has applied content-disposition header:
using (WebClient client = new WebClient())
{
using (Stream rawStream = client.OpenRead(url))
{
string fileName = string.Empty;
string contentDisposition = client.ResponseHeaders["content-disposition"];
if (!string.IsNullOrEmpty(contentDisposition))
{
string lookFor = "filename=";
int index = contentDisposition.IndexOf(lookFor, StringComparison.CurrentCultureIgnoreCase);
if (index >= 0)
fileName = contentDisposition.Substring(index + lookFor.Length);
}
if (fileName.Length > 0)
{
using (StreamReader reader = new StreamReader(rawStream))
{
File.WriteAllText(Server.MapPath(fileName), reader.ReadToEnd());
reader.Close();
}
}
rawStream.Close();
}
}
If the server did not set up this header, try debugging and see what ResponseHeaders you do have, one of them will probably contain the name you desire. If the browser show the name, it must come from somewhere.. :)

You need to look at the content-disposition header, via:
string disposition = client.ResponseHeaders["content-disposition"];
a typical example would be:
"attachment; filename=IDontKnowHowToGetTheRealFileNameHere.txt"

I achieve this with the code of wst.
Here is the full code to download the url file in c:\temp folder
public static void DownloadFile(string url)
{
using (WebClient client = new WebClient())
{
client.OpenRead(url);
string header_contentDisposition = client.ResponseHeaders["content-disposition"];
string filename = new ContentDisposition(header_contentDisposition).FileName;
//Start the download and copy the file to the destinationFolder
client.DownloadFile(new Uri(url), #"c:\temp\" + filename);
}
}

You can use HTTP content-disposition header to suggest filenames for the content you are providing:
Content-Disposition: attachment; filename=downloadedfile.xls;
So, in your AwesomeSite.aspx script, you would set the content-disposition header. In your WebClient class you would retrieve that header to save the file as suggested by your AwesomeSite site.

Although the solution proposed by Shadow Wizard works well for text files, I needed to support downloading binary files, such as pictures and executables, in my application.
Here is a small extension to WebClient that does the trick. Download is asynchronous. Also default value for file name is required, because we don't really know if the server would send all the right headers.
static class WebClientExtensions
{
public static async Task<string> DownloadFileToDirectory(this WebClient client, string address, string directory, string defaultFileName)
{
if (!Directory.Exists(directory))
throw new DirectoryNotFoundException("Downloads directory must exist");
string filePath = null;
using (var stream = await client.OpenReadTaskAsync(address))
{
var fileName = TryGetFileNameFromHeaders(client);
if (string.IsNullOrWhiteSpace(fileName))
fileName = defaultFileName;
filePath = Path.Combine(directory, fileName);
await WriteStreamToFile(stream, filePath);
}
return filePath;
}
private static string TryGetFileNameFromHeaders(WebClient client)
{
// content-disposition might contain the suggested file name, typically same as origiinal name on the server
// Originally content-disposition is for email attachments, but web servers also use it.
string contentDisposition = client.ResponseHeaders["content-disposition"];
return string.IsNullOrWhiteSpace(contentDisposition) ?
null :
new ContentDisposition(contentDisposition).FileName;
}
private static async Task WriteStreamToFile(Stream stream, string filePath)
{
// Code below will throw generously, e. g. when we don't have write access, or run out of disk space
using (var outStream = new FileStream(filePath, FileMode.CreateNew))
{
var buffer = new byte[8192];
while (true)
{
int bytesRead = await stream.ReadAsync(buffer, 0, buffer.Length);
if (bytesRead == 0)
break;
// Could use async variant here as well. Probably helpful when downloading to a slow network share or tape. Not my use case.
outStream.Write(buffer, 0, bytesRead);
}
}
}
}

Ok, my turn.
I had a few things in mind when I tried to "download the file":
Use only HttpClient. I had a couple of extension methods over it, and it wasn't desirable to create other extensions for WebClient.
It was mandatory for me also to get a File name.
I had to write the result to MemoryStream but not FileStream.
Solution
So, for me, it turned out to be this code:
// assuming that httpClient created already (including the Authentication cumbersome)
var response = await httpClient.GetAsync(absoluteURL); // call the external API
// reading file name from HTTP headers
var fileName = response.Content.Headers.ContentDisposition.FileNameStar; // also available to read from ".FileName"
// reading file as a byte array
var fileBiteArr = await response.Content
.ReadAsByteArrayAsync()
.ConfigureAwait(false);
var memoryStream = new MemoryStream(fileBiteArr); // memory streamed file
Test
To test that the Stream contains what we have, we can check it by converting it to file:
// getting the "Downloads" folder location, can be anything else
string pathUser = Environment.GetFolderPath(Environment.SpecialFolder.UserProfile);
string downloadPath = Path.Combine(pathUser, "Downloads\\");
using (FileStream file =
new FileStream(
$"{downloadPath}/file.pdf",
FileMode.Create,
FileAccess.Write))
{
byte[] bytes = new byte[memoryStream .Length];
memoryStream.Read(bytes, 0, (int)memoryStream.Length);
file.Write(bytes, 0, bytes.Length);
memoryStream.Close();
}