Develop Reference

Downloading an entire firebase storage folder's content in Unity

Hello im trying to download an entire folder's content from firebase into an android device.
The firebase hierarchy looks like the following:
So far I can only download a single file using the following code:
// ------------------------- FILE DOWNLOADING ------------------------------------- //
Debug.Log("Download Attempt...");
if (Permission.HasUserAuthorizedPermission(Permission.ExternalStorageWrite))
{
Debug.Log("STEP1...");
//Firestore Reference
storage = FirebaseStorage.DefaultInstance;
storageReference = storage.GetReferenceFromUrl("gs://houdini-ac884.appspot.com");
StorageReference riversRef = storageReference.Child("uploads/3895d968-65bf-4e2d-a964-763e22742fdf.meta");
//StorageReference
//pathReference = storage.GetReference("uploads/3895d968-65bf-4e2d-a964-763e22742fdf.meta");
// Create local filesystem URL
Debug.Log("STEP2...");
var Directory_path = ("SparseSpatialMap/" + "3895d968-65bf-4e2d-a964-763e22742fdf.meta");
var path = (Application.persistentDataPath + "/" + Directory_path);
Debug.Log("STEP3...");
// Download to the local filesystem
//pathReference.GetFileAsync(path).ContinueWithOnMainThread(task =>
//{
riversRef.GetFileAsync(path).ContinueWithOnMainThread(task =>
{
if (!task.IsFaulted && !task.IsCanceled)
{
Debug.Log("Finished downloading...");
easyar.GUIPopup.EnqueueMessage("Download Completed", 5);
}
else
{
Debug.Log("DOWNLOAD FAILURE !!!!!!!!!!!!!");
Debug.Log(task.Exception.ToString());
easyar.GUIPopup.EnqueueMessage("FAIL EXCEPTION", 5);
}
Debug.Log("STEP4...");
});
}
else
{
Debug.Log("No Permissions");
easyar.GUIPopup.EnqueueMessage("FAIL, No permissions", 5);
Permission.RequestUserPermission(Permission.ExternalStorageWrite);
}
Debug.Log("End of Download Attempt...");
// ------------------------- FILE DOWNLOADING END ------------------------------------- //
From what I understand there isnt a firebase function to download all files in folder and I would have to use something else.
Any help would be apreciated thanks

There's a REST API to get the metadata information of all the files present inside a folder.
https://firebasestorage.googleapis.com/v0/b/YOUR_PROJECT_ID/o?preifx=path/to/folder
The param passed to the above API is the path to the folder.
First you need to get a list of files present in a folder. The below method gets a list of file:
async Task<List<FileMetadata>> GetFilesInFolder(string path)
{
const string baseUrl = "https://firebasestorage.googleapis.com/v0/b/";
const string projectId = "PROJECT_ID";
// Build the REST API URL
string url = $"{baseUrl}{projectId}/o?prefix={path}";
// Send a GET request to the URL
using (HttpClient client = new HttpClient())
using (HttpResponseMessage response = await client.GetAsync(url))
using (HttpContent content = response.Content)
{
// Read the response body
string responseText = await content.ReadAsStringAsync();
// Deserialize the JSON response
ListFilesResponse responseData = JsonConvert.DeserializeObject<ListFilesResponse>(responseText);
// Return the list of files
return responseData.Files;
}
}
After you get the list of metadata of the files, start downloading each of them.
List<FileMetadata> files = await GetFilesInFolder(folderPath);
// Download each file
foreach (FileMetadata file in files)
{
// Get the file's download URL
string downloadUrl = file.DownloadUrl;
// Download the file using the URL
using (HttpClient client = new HttpClient())
using (HttpResponseMessage response = await client.GetAsync(downloadUrl))
using (HttpContent content = response.Content)
{
byte[] fileData = await content.ReadAsByteArrayAsync();
// Save the file to the device
}
}
Also do add the response objects from Firebase's endpoints
class ListFilesResponse
{
public List<FileMetadata> Files { get; set; }
}
// Class that represents metadata for a file in Firebase Storage
class FileMetadata
{
public string Name { get; set; }
public string DownloadUrl { get; set; }
}

How to pass on the content of a HttpInputStream and send it to another Rest Api in c# .net

I have a REST API Web Service that acts as middleware that redirects calls onto another REST API Service.
For example in this case I upload a file on the webpage. And this Web Service receives the files and sends them to another API that actually processes the files.
It is something like this:
public async Task<IHttpActionResult> ExecuteFileUpload()
{
IHttpActionResult res;
try
{
var httpRequest = HttpContext.Current.Request;
var requestedFiles = new List<System.IO.Stream>();
var url = "http://localhost:2288" + "/api/v1/templates/upload";
if (httpRequest.Files.Count > 0)
{
HttpFileCollection files = httpRequest.Files;
using (var content = new MultipartFormDataContent())
{
int index = 0;
foreach (var file in httpRequest.Files)
{
var postedFile = httpRequest.Files[index];
var fileName = postedFile.FileName;
var fileInMemory = postedFile.InputStream;
content.Add(new StreamContent(fileInMemory), "f" + index, fileName);
index++;
}
res = await ForwardPost(url, content);
}
}
else
res = BadRequest();
}
catch (Exception ex)
{
res = InternalServerError(ex);
}
return res;
}
The forward post function is simple just like this:
protected async Task<IHttpActionResult> ForwardPost(string url, MultipartFormDataContent forwardContent)
{
IHttpActionResult res;
using (var client = CreateClient())
{
using (var response = await client.PostAsync(url, forwardContent))
{
if (response.StatusCode == HttpStatusCode.OK)
{
var content = await response.Content.ReadAsAsync<JToken>();
res = Ok(content);
}
else
{
res = InternalServerError();
}
}
return res;
}
}
As you can see I just want to forward whatever is passed to me from the webpage and forward it to the actual REST API that handles this.
However it throws an exception on this line:
response = await client.PostAsync(url, forwardContent)
It throws a System.IO.IOException
Cannot close stream until all bytes are written.
Why is this the case?
Is there a way to solve this problem?

The using in the ForwardPost function will dispose the forwardedContent from the calling method. This dispose will attempt to dispose the "postedFile.InputStream" reference from the request object of the origin method. This is likely deeply tied to the httprequest object.
using (var response = await client.PostAsync(url, forwardContent))
The solution is to copy the postedFile.InputStream to a new memorystream such that it can be disposed separately.

Upload new file to onedrive using microsoft graph c# asp.net

Trying to upload a file to onedrive that does not already exist. I have managed to get it to update an existing file. But can't seem to figure out how to create a brand new file. I have done this useing the Microsoft.Graph library.
Here is the code that works to update an existing file:
public async Task<ActionResult> OneDriveUpload()
{
string token = await GetAccessToken();
if (string.IsNullOrEmpty(token))
{
// If there's no token in the session, redirect to Home
return Redirect("/");
}
GraphServiceClient client = new GraphServiceClient(
new DelegateAuthenticationProvider(
(requestMessage) =>
{
requestMessage.Headers.Authorization =
new AuthenticationHeaderValue("Bearer", token);
return Task.FromResult(0);
}));
try
{
string path = #"C:/Users/user/Desktop/testUpload.xlsx";
byte[] data = System.IO.File.ReadAllBytes(path);
Stream stream = new MemoryStream(data);
// Line that updates the existing file
await client.Me.Drive.Items["55BBAC51A4E4017D!104"].Content.Request().PutAsync<DriveItem>(stream);
return View("Index");
}
catch (ServiceException ex)
{
return RedirectToAction("Error", "Home", new { message = "ERROR retrieving messages", debug = ex.Message });
}
}

I'd suggest using the ChunkedUploadProvider utility that is included in the SDK. Aside from being a little easier to work with, it will allow you to upload files of any side rather than being limited to files under 4MB.
You can find a sample of how to use ChunkedUploadProvider in the OneDriveUploadLargeFile unit test.
To answer your direct question, uploading works the same for both replacing and creating files. You do however need to specify the file name rather than just an existing Item number:
await graphClient.Me
.Drive
.Root
.ItemWithPath("fileName")
.Content
.Request()
.PutAsync<DriveItem>(stream);

This code will help you all to upload small and large files using Microsoft graph Api Sdk in ASP .NEt Core
Upload or replace the contents of a DriveItem
*Controller code : -*
[BindProperty]
public IFormFile UploadedFile { get; set; }
public IDriveItemChildrenCollectionPage Files { get; private set; }
public FilesController(ILogger<FilesModel> logger, GraphFilesClient graphFilesClient, GraphServiceClient graphServiceClient, ITokenAcquisition tokenAcquisition)
{
_graphFilesClient = graphFilesClient;
_logger = logger;
_graphServiceClient = graphServiceClient;
_tokenAcquisition = tokenAcquisition;
}
[EnableCors]
[HttpPost]
[Route("upload-file")]
[RequestFormLimits(MultipartBodyLengthLimit = 100000000)]
[RequestSizeLimit(100000000)]
public async Task<IActionResult> uploadFiles(string itemId, string folderName, [FromHeader] string accessToken)
{
_logger.LogInformation("into controller");
if (UploadedFile == null || UploadedFile.Length == 0)
{
return BadRequest();
}
_logger.LogInformation($"Uploading {UploadedFile.FileName}.");
var filePath = Path.Combine(System.IO.Directory.GetCurrentDirectory(), UploadedFile.FileName);
_logger.LogInformation($"Uploaded file {filePath}");
using (var stream = new MemoryStream())
{
UploadedFile.CopyTo(stream);
var bytes = stream.ToArray();
_logger.LogInformation($"Stream {stream}.");
stream.Flush();
await _graphFilesClient.UploadFile(
UploadedFile.FileName, new MemoryStream(bytes), itemId, folderName, accessToken);
}
return Ok("Upload Successful!");
}
*Service code :-*
[EnableCors]
public async Task UploadFile(string fileName, Stream stream,string itemId,string folderName,string accessToken)
{
GraphClients graphClients = new GraphClients(accessToken);
GraphServiceClient _graphServiceClient = graphClients.getGraphClient();
_logger.LogInformation("Into Service");
var filePath = Path.Combine(System.IO.Directory.GetCurrentDirectory(),fileName);
_logger.LogInformation($"filepath : {filePath}");
Console.WriteLine("Uploading file: " + fileName);
var size = stream.Length / 1000;
_logger.LogInformation($"Stream size: {size} KB");
if (size/1000 > 4)
{
// Allows slices of a large file to be uploaded
// Optional but supports progress and resume capabilities if needed
await UploadLargeFile(filePath, stream,accessToken);
}
else
{
try
{
_logger.LogInformation("Try block");
String test = folderName + "/" + fileName;
// Uploads entire file all at once. No support for reporting progress.
// for getting your sharepoint site open graph explorer > sharepoint sites > get my organization's default sharepoint site.
var driveItem = await _graphServiceClient
.Sites["Your share point site"]
.Drive
.Root.ItemWithPath(test)
.Content
.Request()
.PutAsync<DriveItem>(stream);
_logger.LogInformation($"Upload complete: {driveItem.Name}");
}
catch (ServiceException ex)
{
_logger.LogError($"Error uploading: {ex.ToString()}");
throw;
}
}
}
private async Task UploadLargeFile(string itemPath, Stream stream,string accessToken)
{
GraphClients graphClients = new GraphClients(accessToken);
GraphServiceClient _graphServiceClient = graphClients.getGraphClient();
// Allows "slices" of a file to be uploaded.
// This technique provides a way to capture the progress of the upload
// and makes it possible to resume an upload using fileUploadTask.ResumeAsync(progress);
// Based on https://docs.microsoft.com/en-us/graph/sdks/large-file-upload
// Use uploadable properties to specify the conflict behavior (replace in this case).
var uploadProps = new DriveItemUploadableProperties
{
ODataType = null,
AdditionalData = new Dictionary<string, object>
{
{ "#microsoft.graph.conflictBehavior", "replace" }
}
};
// Create the upload session
var uploadSession = await _graphServiceClient.Me.Drive.Root
.ItemWithPath(itemPath)
.CreateUploadSession(uploadProps)
.Request()
.PostAsync();
// Max slice size must be a multiple of 320 KiB
int maxSliceSize = 320 * 1024;
var fileUploadTask = new LargeFileUploadTask<DriveItem>(uploadSession, stream, maxSliceSize);
// Create a callback that is invoked after
// each slice is uploaded
IProgress<long> progress = new Progress<long>(prog =>
{
_logger.LogInformation($"Uploaded {prog} bytes of {stream.Length} bytes");
});
try
{
// Upload the file
var uploadResult = await fileUploadTask.UploadAsync(progress);
if (uploadResult.UploadSucceeded)
{
_logger.LogInformation($"Upload complete, item ID: {uploadResult.ItemResponse.Id}");
}
else
{
_logger.LogInformation("Upload failed");
}
}
catch (ServiceException ex)
{
_logger.LogError($"Error uploading: {ex.ToString()}");
throw;
}
}

Post files from ASP.NET Core web api to another ASP.NET Core web api

We are building a web application that consist of an Angular2 frontend, a ASP.NET Core web api public backend, and a ASP.NET Core web api private backend.
Uploading files from Angular2 to the public backend works. But we would prefer to post them forward to the private backend.
Current working code
[HttpPost]
public StatusCodeResult Post(IFormFile file)
{
...
}
From there I can save the file to disk using file.CopyTo(fileStream);
However, I want to re-send that file, or those files, or, ideally, the whole request to my second web api core.
I am not sure how to achieve this with the HttpClient class of asp.net core.
I've tried all kinds of things such as
StreamContent ss = new StreamContent(HttpContext.Request.Body);
var result = client.PostAsync("api/Values", ss).Result;
But my second backend gets an empty IFormFile.
I have a feeling it is possible to send the file(s) as a stream and reconstruct them on the other side, but can't get it to work.
The solution must use two web api core.

Solution
Public backend in DMZ
[HttpPost]
public StatusCodeResult Post(IFormFile file)
{
try
{
if (file != null && file.Length > 0)
{
using (var client = new HttpClient())
{
try
{
client.BaseAddress = new Uri(currentPrivateBackendAddress);
byte[] data;
using (var br = new BinaryReader(file.OpenReadStream()))
data = br.ReadBytes((int)file.OpenReadStream().Length);
ByteArrayContent bytes = new ByteArrayContent(data);
MultipartFormDataContent multiContent = new MultipartFormDataContent();
multiContent.Add(bytes, "file", file.FileName);
var result = client.PostAsync("api/Values", multiContent).Result;
return StatusCode((int)result.StatusCode); //201 Created the request has been fulfilled, resulting in the creation of a new resource.
}
catch (Exception)
{
return StatusCode(500); // 500 is generic server error
}
}
}
return StatusCode(400); // 400 is bad request
}
catch (Exception)
{
return StatusCode(500); // 500 is generic server error
}
}
Private backend
[HttpPost]
public void Post()
{
//Stream bodyStream = HttpContext.Request.Body;
if (Request.HasFormContentType)
{
var form = Request.Form;
foreach (var formFile in form.Files)
{
var targetDirectory = Path.Combine(_appEnvironment.WebRootPath, "uploads");
var fileName = GetFileName(formFile);
var savePath = Path.Combine(targetDirectory, fileName);
using (var fileStream = new FileStream(savePath, FileMode.Create))
{
formFile.CopyTo(fileStream);
}
}
}
}

Hi i had the same issue and this is what worked for me :
My setup is netCore MVC netCoreApi.
My MVC Controller looks like :
[HttpPost("UploadFiles")]
public async Task<IActionResult> Post(List<IFormFile> files)
{
Sp4RestClient dataPovider = new Sp4RestClient("http://localhost:60077/");
long size = files.Sum(f => f.Length);
foreach (var file in files)
{
await dataPovider.ImportFile(file);
}
return Ok();
}
DataProvider Method :
public async Task ImportFile(IFormFile file)
{
RestClient restClient = new RestClient(_queryBulder.BuildImportFileRequest());
using (var content = new MultipartFormDataContent())
{
content.Add(new StreamContent(file.OpenReadStream())
{
Headers =
{
ContentLength = file.Length,
ContentType = new MediaTypeHeaderValue(file.ContentType)
}
}, "File", "FileImport");
var response = await restClient.Post<IFormFile>(content);
}
}
And least my WebApi Controller :
[HttpPost]
[Route("ImportData")]
public IActionResult Import(IFormFile file)
{
return Ok();
}
To see the complete code here is my RestClient Post method :
public async Task<RestResult<T>> Post<T>(HttpContent content)
{
using (HttpClient httpClient = new HttpClient())
{
HttpResponseMessage response = await httpClient.PostAsync(Endpoint, content);
if (response.StatusCode == HttpStatusCode.Created)
{
T result = JsonConvert.DeserializeObject<T>(await response.Content.ReadAsStringAsync());
return new RestResult<T> { Result = result, ResultCode = HttpStatusCode.OK };
}
RestResult<T> nonOkResult =
new RestResult<T> { Result = default(T), ResultCode = response.StatusCode, Message = await response.Content.ReadAsStringAsync() };
return nonOkResult;
}
}
// Yeah i know im not getting HttpStatusCode.Created back ;)
happy coding ;)

API Code
[Route("api/upload/{id}")]
[HttpPost]
public async Task<IActionResult> Post(string id)
{
var filePath = #"D:\" + id; //+ Guid.NewGuid() + ".png";
if (Request.HasFormContentType)
{
var form = Request.Form;
foreach (var formFile in form.Files)
{
if (formFile.Length > 0)
{
using (var stream = new FileStream(filePath, FileMode.Create))
{
await formFile.CopyToAsync(stream);
}
}
}
}
return Ok(new { Path = filePath });
}
Back End
[Route("home/UploadFile")]
[HttpPost]
public IActionResult UploadFile(IFormFile file)
{
if (file == null || file.Length == 0)
return Content("file not selected");
var client = new HttpClient();
byte[] data;
using (var br = new BinaryReader(file.OpenReadStream()))
data = br.ReadBytes((int)file.OpenReadStream().Length);
ByteArrayContent bytes = new ByteArrayContent(data);
MultipartFormDataContent multiContent = new MultipartFormDataContent
{
{ bytes, "file", file.FileName }
};
var result = client.PostAsync("http://localhost:2821/api/upload/" + file.FileName, multiContent).Result;
return RedirectToAction("file");
}
Download Source

I was in a similar situation - I needed a proxy method for forwarding not only files but also JSON data and whatnot. I did not want to do any analysis of the data in my proxy to let the final receiver deal with it.
So with some help from #Anton Tykhyy I came to the following working solution:
byte[] arr = null;
using (var mems = new MemoryStream())
{
// read entire body into memory first because it might be chunked with unknown length
await request.Body.CopyToAsync(mems);
await mems.FlushAsync(); // not sure if needed after CopyToAsync - better safe then sorry
arr = mems.ToArray();
}
msg.Content = new ByteArrayContent(arr);
msg.Content.Headers.ContentLength = arr.Length;
// keep content-type header "as is" to preserve multipart boundaries etc.
msg.Content.Headers.TryAddWithoutValidation("Content-Type", request.ContentType);
var response = await _httpClient.SendAsync(msg);
I tested it with complex request that contained multipart form data with JSON field and multiple attached files, and all the data reached my backend server without any issues.

Ignoring the HttpClient when you call the private backend API, can you reference the private Core API project from the public Core API project and call the controller directly from the Core API project? See the request is still null/empty. If the request comes out with a value then the issue is with the use of the HttpClient.
Ideally, you want to create a package library(kind of SDK) for your private Core API that you want to distribute to consuming clients. This acts like a wrapper/proxy. This way you can isolate the private backend system and you can troubleshoot it in isolation. So you public Core API project(which is the private backend client) can reference it as nuget package.

Using Google Drive V3 API and service account auth, WebViewLink is null

I am using google drive v3 api to upload a file and then preview it in browser using web view link in the response. But web view link is coming null. When i was using v2, I was able to do it using alternate link.
I have not set the parent ref so I am assuming as per the documentation, the file is stored in my drive folder(root) of service account. As I couldn't login to service account, so I shared the file with my existing test gmail account and it was shared.
My question is how can I open the file in browser using System.Diagnostics.Process.Start(newFile.WebViewLink);
here is my code:
{
File fileInGoogleDrive = Utils.uploadToDrive(service, pathOfTheFileToBeUploaded, "root");
Permission toShare = new Permission();
toShare.EmailAddress = "xyz#gmail.com";
toShare.Type = "user";
toShare.Role = "reader";
PermissionsResource.CreateRequest createRequest = service.Permissions.Create(toShare, fileInGoogleDrive.Id);
createRequest.Execute();
return fileInGoogleDrive.WebViewLink; //THIS IS NULL
}
here is the upload code:
public static File uploadToDrive(DriveService _service, string _uploadFile, string _parent = "root")
{
if (!String.IsNullOrEmpty(_uploadFile))
{
File fileMetadata = new File();
fileMetadata.Name = System.IO.Path.GetFileName(_uploadFile);
fileMetadata.MimeType = GetMimeType(_uploadFile);
//fileMetadata.Parents = new List<FilesResource>() { new FilesResource() {}};
try
{
byte[] byteArray = System.IO.File.ReadAllBytes(_uploadFile);
System.IO.MemoryStream stream = new System.IO.MemoryStream(byteArray);
FilesResource.CreateMediaUpload request = _service.Files.Create(fileMetadata, stream, GetMimeType(_uploadFile));
request.Upload();
return request.ResponseBody;
}
catch (System.IO.IOException iox)
{
// Log
return null;
}
catch (Exception e) // any special google drive exceptions??
{
//Log
return null;
}
}
else
{
//Log file does not exist
return null;
}
}
Could anyone please guide me here?

Just wanted to post the syntax in C# for the above. From the google documentation, it says we have to do a get on files and then request using Fields property.
"Getting the fields in google drive v3 api for .net"
File resultFile = null;
FilesResource.ListRequest listRequest = _service.Files.List();
/* Specify camelCase format to specify fields. You can also check in debug mode the files properties before requesting which will be null. All properties will be capitalized so make th efirst letter as small(camel case standard)*/
listRequest.Fields = "files(id, webViewLink, size)";
var files = listRequest.Execute().Files;
if (files != null && files.Count > 0)
{
foreach (var file in files)
{
if (file.Id == _fileId)
{
Console.WriteLine("{0}, {1}, {2}", file.Id, file.WebViewLink, file.Size);
resultFile = file;
}
}
}