Downloading an entire firebase storage folder's content in Unity - c#

Hello im trying to download an entire folder's content from firebase into an android device.
The firebase hierarchy looks like the following:
So far I can only download a single file using the following code:
// ------------------------- FILE DOWNLOADING ------------------------------------- //
Debug.Log("Download Attempt...");
if (Permission.HasUserAuthorizedPermission(Permission.ExternalStorageWrite))
{
Debug.Log("STEP1...");
//Firestore Reference
storage = FirebaseStorage.DefaultInstance;
storageReference = storage.GetReferenceFromUrl("gs://houdini-ac884.appspot.com");
StorageReference riversRef = storageReference.Child("uploads/3895d968-65bf-4e2d-a964-763e22742fdf.meta");
//StorageReference
//pathReference = storage.GetReference("uploads/3895d968-65bf-4e2d-a964-763e22742fdf.meta");
// Create local filesystem URL
Debug.Log("STEP2...");
var Directory_path = ("SparseSpatialMap/" + "3895d968-65bf-4e2d-a964-763e22742fdf.meta");
var path = (Application.persistentDataPath + "/" + Directory_path);
Debug.Log("STEP3...");
// Download to the local filesystem
//pathReference.GetFileAsync(path).ContinueWithOnMainThread(task =>
//{
riversRef.GetFileAsync(path).ContinueWithOnMainThread(task =>
{
if (!task.IsFaulted && !task.IsCanceled)
{
Debug.Log("Finished downloading...");
easyar.GUIPopup.EnqueueMessage("Download Completed", 5);
}
else
{
Debug.Log("DOWNLOAD FAILURE !!!!!!!!!!!!!");
Debug.Log(task.Exception.ToString());
easyar.GUIPopup.EnqueueMessage("FAIL EXCEPTION", 5);
}
Debug.Log("STEP4...");
});
}
else
{
Debug.Log("No Permissions");
easyar.GUIPopup.EnqueueMessage("FAIL, No permissions", 5);
Permission.RequestUserPermission(Permission.ExternalStorageWrite);
}
Debug.Log("End of Download Attempt...");
// ------------------------- FILE DOWNLOADING END ------------------------------------- //
From what I understand there isnt a firebase function to download all files in folder and I would have to use something else.
Any help would be apreciated thanks

There's a REST API to get the metadata information of all the files present inside a folder.
https://firebasestorage.googleapis.com/v0/b/YOUR_PROJECT_ID/o?preifx=path/to/folder
The param passed to the above API is the path to the folder.
First you need to get a list of files present in a folder. The below method gets a list of file:
async Task<List<FileMetadata>> GetFilesInFolder(string path)
{
const string baseUrl = "https://firebasestorage.googleapis.com/v0/b/";
const string projectId = "PROJECT_ID";
// Build the REST API URL
string url = $"{baseUrl}{projectId}/o?prefix={path}";
// Send a GET request to the URL
using (HttpClient client = new HttpClient())
using (HttpResponseMessage response = await client.GetAsync(url))
using (HttpContent content = response.Content)
{
// Read the response body
string responseText = await content.ReadAsStringAsync();
// Deserialize the JSON response
ListFilesResponse responseData = JsonConvert.DeserializeObject<ListFilesResponse>(responseText);
// Return the list of files
return responseData.Files;
}
}
After you get the list of metadata of the files, start downloading each of them.
List<FileMetadata> files = await GetFilesInFolder(folderPath);
// Download each file
foreach (FileMetadata file in files)
{
// Get the file's download URL
string downloadUrl = file.DownloadUrl;
// Download the file using the URL
using (HttpClient client = new HttpClient())
using (HttpResponseMessage response = await client.GetAsync(downloadUrl))
using (HttpContent content = response.Content)
{
byte[] fileData = await content.ReadAsByteArrayAsync();
// Save the file to the device
}
}
Also do add the response objects from Firebase's endpoints
class ListFilesResponse
{
public List<FileMetadata> Files { get; set; }
}
// Class that represents metadata for a file in Firebase Storage
class FileMetadata
{
public string Name { get; set; }
public string DownloadUrl { get; set; }
}

Related

How can I extract Metadata from StreamContent object that comes from Web API?

I have an application which you can upload Image. It sends the streamcontent via web api.
I'm using metadataextractor library and it works when I only use Stream object. but when it is streamcontent, reading metadata doesn't work.
This is my controller and client service code
[HttpPost]
public void UploadFile(IEnumerable<IFormFile> files)
{
if (!files.Any())
{
Response.StatusCode = (int)System.Net.HttpStatusCode.NoContent;
return;
}
foreach (var file in files)
{
using (Stream stream = file.OpenReadStream())
{
_clientService.UploadStip(stream);
}
}
}
public void UploadStip(Stream stream)
{
//I have removed some codes because sending data into the end point already works
var response = _client.PostAsync("uploadphoto", new StreamContent(stream));
response.Wait();
if (response.Result.StatusCode != System.Net.HttpStatusCode.OK)
{
throw new Exception("Error when uploading photo");
}
}
Now the challenge here is that I get an error when receiving and reading Metadata of the StreamContent object.
Here's my code for the end-point:
public void UploadPhoto(Stream content)
{
// Read all metadata from the image
var directories = ImageMetadataReader.ReadMetadata(content); //this part always throw
//System.ArgumentException: 'Must support seek
//Parameter name: stream'
// Find the so-called Exif "SubIFD" (which may be null)
var subIfdDirectory = directories.OfType<ExifSubIfdDirectory>().FirstOrDefault();
// Read the DateTime tag valueWW
var datetime = subIfdDirectory?.GetDateTime(ExifDirectoryBase.TagDateTimeOriginal);
var gps = directories.OfType<GpsDirectory>().FirstOrDefault();
var location = gps.GetGeoLocation();
var latitude = Math.Round(location.Latitude, 6);
var longtitude = Math.Round(location.Longitude, 6);
}
I'm not sure how to get metadata out of StreamContent.

C# MVC/API - Return an image from Amazon S3 for my API Call

I'm using .net core to upload and retrieve an image from a private Amazon S3 bucket.
I'm able to upload it successfully, and even view it after I download it from S3, however when I'm a bit unsure about how to return the stream/response back to the client for the actual API call (for example right now I'm just trying to use Postman/Fiddler proxy tools to get back the image from my API)
My code for S3 to retrieve the stream:
///Retrieve my image from my bucket
public async Task<string> ReadObjectData(MediaFolder key, String fileName)
{
string responseBody = "";
IAmazonS3 client;
using (client = new AmazonS3Client(accessKey, accessSecret, endpoint))
{
Amazon.S3.Model.GetObjectRequest request = new Amazon.S3.Model.GetObjectRequest
{
BucketName = bucket,
Key = key + "/" + fileName,
};
using (GetObjectResponse response = await client.GetObjectAsync(request))
using (Stream responseStream = response.ResponseStream)
using (StreamReader reader = new StreamReader(responseStream))
{
string title = response.Metadata["x-amz-meta-title"];
responseBody = reader.ReadToEnd();
}
}
return responseBody;
}
So now in my controller, I have the following action:
[HttpGet("ProfilePic")]
public async Task<IActionResult> GetProfilePicture()
{
var user = await GetUserFromBearerToken();
//Retrieve
var utf8ImageResponse = await _fileService.ReadObjectData(MediaFolder.Profiles, user.ProfileImageFileName);
//To return a file as a stream
var imageBytes = System.Text.Encoding.UTF8.GetBytes(utf8ImageResponse);
//Return the image, which I'll hardcode as jpeg for a test
return File(imageBytes, "image/jpeg");
}
When I make the call using Postman, it returns a little blank box (the box you'd see if you tried to return an image, but it wasn't a valid image or null in some way).
Right now I'm using Postman but ideally I'd want an app to present this image.
Any ideas what I'm doing wrong? I tried messing around with base64 encoding and other things but nothing seems to work.
Thanks!
This way you can retrieve the file as stream from S3 storage
public async Task<Stream> ReadObjectData(MediaFolder key, String fileName)
{
try
{
using (var client = new AmazonS3Client(accessKey, accessSecret, endpoint))
{
var request = new GetObjectRequest
{
BucketName = bucket,
Key = key + "/" + fileName
};
using (var getObjectResponse = await client.GetObjectAsync(request))
{
using (var responseStream = getObjectResponse.ResponseStream)
{
var stream = new MemoryStream();
await responseStream.CopyToAsync(stream);
stream.Position = 0;
return stream;
}
}
}
}
catch (Exception exception)
{
throw new Exception("Read object operation failed.", exception);
}
}
And then - return this stream as FileStreamResult:
[HttpGet("ProfilePic")]
public async Task<IActionResult> GetProfilePicture()
{
var user = await GetUserFromBearerToken();
Stream imageStream = await _fileService.ReadObjectData(MediaFolder.Profiles, user.ProfileImageFileName);
Response.Headers.Add("Content-Disposition", new ContentDisposition
{
FileName = "Image.jpg",
Inline = true // false = prompt the user for downloading; true = browser to try to show the file inline
}.ToString());
return File(imageStream, "image/jpeg");
}

How can I post image from UWP to .NET core web api?

Now I have configured for UWP photo post to web api part which is using HttpClient.
Uri uri = new Uri("http://localhost:50040/api/Upload");
IInputStream inputStream = await photoFile.OpenAsync(FileAccessMode.Read);
HttpMultipartFormDataContent multipartContent = new HttpMultipartFormDataContent();
multipartContent.Add(new HttpStreamContent(inputStream), "myFile", photoFile.Name);
Windows.Web.Http.HttpClient newclient = new Windows.Web.Http.HttpClient();
Windows.Web.Http.HttpResponseMessage response = await client.PostAsync(uri, multipartContent);
But I don't know how to set for the server side which is my .NET core web api to get the image which post from my UWP application.Please Help me, thank you.
But I don't know how to set for the server side which is my .NET core web api
Please reference the File uploads official tutorial to create your server side. For example, add POST method as following sample code showed to receive the UWP client sent file with the client code you showed above.
// POST api/values
[HttpPost]
public async Task<IActionResult> Post(IFormFile myFile)
{
// full path to file in temp location, you could change this
var filePath = Path.GetTempFileName();
if (myFile.Length > 0)
{
using (var stream = new FileStream(filePath, FileMode.Create))
{
await myFile.CopyToAsync(stream);
}
}
// process uploaded files
// Don't rely on or trust the FileName property without validation.
return Ok(new { filePath, myFile.Length });
}
More details you could also reference the official sample.
In Web API Controller
public IHostingEnvironment _environment;
public UploadFilesController(IHostingEnvironment environment) // Create Constructor
{
_environment = environment;
}
[HttpPost("UploadImages")]
public Task<ActionResult<string>> UploadImages([FromForm]List<IFormFile> allfiles)
{
string filepath = "";
foreach (var file in allfiles)
{
string extension = Path.GetExtension(file.FileName);
var upload = Path.Combine(_environment.ContentRootPath, "ImageFolderName");
if (!Directory.Exists(upload))
{
Directory.CreateDirectory(upload);
}
string FileName = Guid.NewGuid() + extension;
if (file.Length > 0)
{
using (var fileStream = new FileStream(Path.Combine(upload, FileName), FileMode.Create))
{
file.CopyTo(fileStream);
}
}
filepath = Path.Combine("ImageFolderName", FileName);
}
return Task.FromResult<ActionResult<string>>(filepath);
}
In yourpage.xaml.cs
using Windows.Storage;
using Windows.Storage.Pickers;
.....
StorageFile file;
......
private async void btnFileUpload_Click(object sender, RoutedEventArgs e) // Like Browse button
{
try
{
FileOpenPicker openPicker = new FileOpenPicker();
openPicker.ViewMode = PickerViewMode.Thumbnail;
openPicker.SuggestedStartLocation = PickerLocationId.PicturesLibrary;
openPicker.FileTypeFilter.Add(".jpg");
openPicker.FileTypeFilter.Add(".png");
file = await openPicker.PickSingleFileAsync();
if (file != null)
{
//fetch file details
}
}
catch (Exception ex)
{
}
}
//When upload file
var http = new HttpClient();
var formContent = new HttpMultipartFormDataContent();
var fileContent = new HttpStreamContent(await file.OpenReadAsync());
formContent.Add(fileContent, "allfiles", file.Name);
var response = await http.PostAsync(new Uri("Give API Path" + "UploadImages", formContent);
string filepath = Convert.ToString(response.Content); //Give path in which file is uploaded
Hope this code helps you...
But remember formContent.Add(fileContent, "allfiles", file.Name); line is important and allfiles is that name of parameter to fetch files in web api method "public Task<ActionResult<string>> UploadImages([FromForm]List<IFormFile> **allfiles**)"
Thanks!!!

Using Google Drive V3 API and service account auth, WebViewLink is null

I am using google drive v3 api to upload a file and then preview it in browser using web view link in the response. But web view link is coming null. When i was using v2, I was able to do it using alternate link.
I have not set the parent ref so I am assuming as per the documentation, the file is stored in my drive folder(root) of service account. As I couldn't login to service account, so I shared the file with my existing test gmail account and it was shared.
My question is how can I open the file in browser using System.Diagnostics.Process.Start(newFile.WebViewLink);
here is my code:
{
File fileInGoogleDrive = Utils.uploadToDrive(service, pathOfTheFileToBeUploaded, "root");
Permission toShare = new Permission();
toShare.EmailAddress = "xyz#gmail.com";
toShare.Type = "user";
toShare.Role = "reader";
PermissionsResource.CreateRequest createRequest = service.Permissions.Create(toShare, fileInGoogleDrive.Id);
createRequest.Execute();
return fileInGoogleDrive.WebViewLink; //THIS IS NULL
}
here is the upload code:
public static File uploadToDrive(DriveService _service, string _uploadFile, string _parent = "root")
{
if (!String.IsNullOrEmpty(_uploadFile))
{
File fileMetadata = new File();
fileMetadata.Name = System.IO.Path.GetFileName(_uploadFile);
fileMetadata.MimeType = GetMimeType(_uploadFile);
//fileMetadata.Parents = new List<FilesResource>() { new FilesResource() {}};
try
{
byte[] byteArray = System.IO.File.ReadAllBytes(_uploadFile);
System.IO.MemoryStream stream = new System.IO.MemoryStream(byteArray);
FilesResource.CreateMediaUpload request = _service.Files.Create(fileMetadata, stream, GetMimeType(_uploadFile));
request.Upload();
return request.ResponseBody;
}
catch (System.IO.IOException iox)
{
// Log
return null;
}
catch (Exception e) // any special google drive exceptions??
{
//Log
return null;
}
}
else
{
//Log file does not exist
return null;
}
}
Could anyone please guide me here?
Just wanted to post the syntax in C# for the above. From the google documentation, it says we have to do a get on files and then request using Fields property.
"Getting the fields in google drive v3 api for .net"
File resultFile = null;
FilesResource.ListRequest listRequest = _service.Files.List();
/* Specify camelCase format to specify fields. You can also check in debug mode the files properties before requesting which will be null. All properties will be capitalized so make th efirst letter as small(camel case standard)*/
listRequest.Fields = "files(id, webViewLink, size)";
var files = listRequest.Execute().Files;
if (files != null && files.Count > 0)
{
foreach (var file in files)
{
if (file.Id == _fileId)
{
Console.WriteLine("{0}, {1}, {2}", file.Id, file.WebViewLink, file.Size);
resultFile = file;
}
}
}

Http MultipartFormDataContent

I have been asked to do the following in C#:
/**
* 1. Create a MultipartPostMethod
* 2. Construct the web URL to connect to the SDP Server
* 3. Add the filename to be attached as a parameter to the MultipartPostMethod with parameter name "filename"
* 4. Execute the MultipartPostMethod
* 5. Receive and process the response as required
* /
I have written some code that has no errors, however, the file is not attached.
Can someone have a look at my C# code to see if I have written the code incorrectly?
Here is my code:
var client = new HttpClient();
const string weblinkUrl = "http://testserver.com/attach?";
var method = new MultipartFormDataContent();
const string fileName = "C:\file.txt";
var streamContent = new StreamContent(File.Open(fileName, FileMode.Open));
method.Add(streamContent, "filename");
var result = client.PostAsync(weblinkUrl, method);
MessageBox.Show(result.Result.ToString());
Posting MultipartFormDataContent in C# is simple but may be confusing the first time.
Here is the code that works for me when posting a .png .txt etc.
// 2. Create the url
string url = "https://myurl.com/api/...";
string filename = "myFile.png";
// In my case this is the JSON that will be returned from the post
string result = "";
// 1. Create a MultipartPostMethod
// "NKdKd9Yk" is the boundary parameter
using (var formContent = new MultipartFormDataContent("NKdKd9Yk"))
{
formContent.Headers.ContentType.MediaType = "multipart/form-data";
// 3. Add the filename C:\\... + fileName is the path your file
Stream fileStream = System.IO.File.OpenRead("C:\\Users\\username\\Pictures\\" + fileName);
formContent.Add(new StreamContent(fileStream), fileName, fileName);
using (var client = new HttpClient())
{
// Bearer Token header if needed
client.DefaultRequestHeaders.Add("Authorization", "Bearer " + _bearerToken);
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("multipart/form-data"));
try
{
// 4.. Execute the MultipartPostMethod
var message = await client.PostAsync(url, formContent);
// 5.a Receive the response
result = await message.Content.ReadAsStringAsync();
}
catch (Exception ex)
{
// Do what you want if it fails.
throw ex;
}
}
}
// 5.b Process the reponse Get a usable object from the JSON that is returned
MyObject myObject = JsonConvert.DeserializeObject<MyObject>(result);
In my case I need to do something with the object after it posts so I convert it to that object with JsonConvert.
I debugged this the problem is here:
method.Add(streamContent, "filename");
This 'Add' doesn't actually put the file in the BODY of Multipart Content.
I know this is an old post But to those searching for a solution, to provide a more direct answer, here's what I've found:
using System.Diagnostics;
using System.Net;
using System.Net.Http;
using System.Threading.Tasks;
using System.Web;
using System.Web.Http;
public class UploadController : ApiController
{
public async Task<HttpResponseMessage> PostFormData()
{
// Check if the request contains multipart/form-data.
if (!Request.Content.IsMimeMultipartContent())
{
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
string root = HttpContext.Current.Server.MapPath("~/App_Data");
var provider = new MultipartFormDataStreamProvider(root);
try
{
// Read the form data.
await Request.Content.ReadAsMultipartAsync(provider);
// This illustrates how to get the file names.
foreach (MultipartFileData file in provider.FileData)
{
Trace.WriteLine(file.Headers.ContentDisposition.FileName);
Trace.WriteLine("Server file path: " + file.LocalFileName);
}
return Request.CreateResponse(HttpStatusCode.OK);
}
catch (System.Exception e)
{
return Request.CreateErrorResponse(HttpStatusCode.InternalServerError, e);
}
}
}
Here's where I found it:
http://www.asp.net/web-api/overview/advanced/sending-html-form-data,-part-2
For a more Elaborate implementation:
http://galratner.com/blogs/net/archive/2013/03/22/using-html-5-and-the-web-api-for-ajax-file-uploads-with-image-preview-and-a-progress-bar.aspx
Specify the third parameter which is a fileName.
Something like this:
method.Add(streamContent, "filename", "filename.pdf");

Categories

Resources