Lets say, I upload really important files such a contracts via an API with HttpClient in .Net with the following code:
using (var content = new MultipartFormDataContent())
{
foreach (FileInfo fi in inputFiles)
{
content.Add(CreateFileContent(fi));
}
AwaitRateLimit();
var response = await _Client.PostAsync("upload/", content);
response.EnsureSuccessStatusCode();
// deserialize
string responseJson = await response.Content.ReadAsStringAsync();
ClientResponse.Response decodedResponse =
JsonSerializer.Deserialize<ClientResponse.Response>(responseJson);
}
private StreamContent CreateFileContent(FileInfo fileInfo)
{
var fileContent = new StreamContent(fileInfo.OpenRead());
fileContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("form-data")
{
Name = "\"file\"",
FileName = "\"" + fileInfo.Name + "\""
};
return fileContent;
}
Currently I do the following:
Send multiple files via post in one request
Download each file into ram, build sha256checksum of it and compare it against local files sha256checksum
step 1 is really quick (~a couple of seconds) for a large quantity of files
step 2 takes at least 15 minutes, because each file can only be downloaded individually.
Therefore I would know if you consider step 2 nessesary or if the HttpClient will handle that automatically.
Related
I have a script that uploads a video to an API I built, and after it processes on the API side, a text file is returned to the client. The strange thing is, this only works with one type of file, a .QT file extension. Any other video type I try to send sends and empty video. I have tried .mov, .mp4, and .qt and only the .qt uploads properly. I'll post my code below. Would anyone know what cause only the one file type to work? Nothing on the API side singles out the qt file. I believe this is an issue with this script.
public async void Function() {
Debug.Log("works1");
string filePath = "IMG_0491.mov";
//string filePath = ProcessMode.theFilePath;
var client = new HttpClient();
using (var multipartFormContent = new MultipartFormDataContent()) {
//Add the file
Debug.Log("works2");
var fileStreamContent = new StreamContent(File.OpenRead(filePath));
Debug.Log("works3");
fileStreamContent.Headers.ContentType = new MediaTypeHeaderValue("video/mov");
multipartFormContent.Add(fileStreamContent, name: "file", fileName: filePath); //Originally Actual "Name`
//Send it
var response = await client.PostAsync("http://127.0.0.1:5000/", multipartFormContent); //Enter IP and Port of API when set up
Debug.Log("works4");
//Ensure it was successful.
response.EnsureSuccessStatusCode();
//Grab the animation data from the content.
var animation_data = await response.Content.ReadAsStringAsync();
Debug.Log(animation_data);
//Save to file.
//File.WriteAllTextAsync("AnimationFile.txt", animation_data);
await File.WriteAllTextAsync("AnimationFile.txt", animation_data);
Debug.Log("works5");
}
I've been trying to get uploading an image anonymously onto Imgur using the Imgur API to work, however I've been facing an issue with unauthorized path access.
I've tried search around other similar articles on Microsoft Docs and posts here on stack overflow but couldn't find a solution. I've even given my application "broadFileSystemAccess" as rescap capability in my Package.appxmanifest which I found from reading the Microsoft UWP documentations.
The error I receive is:
System.UnauthorizedAccessException: 'Access to the path 'C:\Users\lysyr\Pictures\ROG Logo.png' is denied.'
The error occurs at the var filecon = File.ReadAllBytes(imgpath); line.
My File Picker code is:
public static string imgpath = "";
public static string finalimg = "";
private async void FileNameButton_Click(object sender, RoutedEventArgs e)
{
var picker = new Windows.Storage.Pickers.FileOpenPicker();
picker.ViewMode = Windows.Storage.Pickers.PickerViewMode.Thumbnail;
picker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.PicturesLibrary;
picker.FileTypeFilter.Add(".jpg");
picker.FileTypeFilter.Add(".jpeg");
picker.FileTypeFilter.Add(".png");
Windows.Storage.StorageFile file = await picker.PickSingleFileAsync();
if (file != null)
{
// Application now has read/write access to the picked file
imgpath = file.Path;
var filecon = File.ReadAllBytes(imgpath); #Error drops here <---
finalimg = Convert.ToBase64String(filecon);
await ImgurUploadAPI();
Debug.WriteLine("Picked Image: " + file.Name);
uploadedimage_text.Text = "Picked Image: " + file.Name;
}
else
{
Debug.WriteLine("Image uploading has been cancelled.");
}
}
And the ImgurUpload task code is:
public static string imgurlink = "";
public async Task ImgurUploadAPI()
{
try
{
if (imgpath != null)
{
// Construct the HttpClient and Uri
HttpClient httpClient = new HttpClient();
Uri uri = new Uri("https://api.imgur.com/3/upload");
httpClient.DefaultRequestHeaders.Add("Authorization", "Client-ID IMGUR-CLIENTIDHERE");
//Debug.WriteLine("Request Headers: ");
// Construct the JSON to post
HttpStringContent content = new HttpStringContent("image=\"{finalimg}\"");
Debug.WriteLine("Request Upload: " + content);
// Post the JSON and wait for a response
HttpResponseMessage httpResponseMessage = await httpClient.PostAsync(
uri,
content);
// Make sure the post succeeded, and write out the response
httpResponseMessage.EnsureSuccessStatusCode();
var httpResponseBody = await httpResponseMessage.Content.ReadAsStringAsync();
imgurlink = httpResponseBody;
Debug.WriteLine("Request Response: " + httpResponseBody);
}
}
catch (Exception ex)
{
Debug.WriteLine(ex);
}
}
I feel like it might be something to do with the way I access and convert the image to be ready for upload. Any tips would be very appreciated as I've been stuck on this for awhile now and it's one of the last things I need to complete for my project. Cheers!
Besides #Julian Silden Langlo answer. there is another option for your scenario. The reason for this behavior is that you could not directly access the file using File.ReadAllBytes(String) Method.
I noticed that you've already added the broadFileSystemAccess capability, what you need to note is that this capability only works for the Windows.Storage APIs. So you could only use it like
StorageFile file = StorageFile.GetFileFromPathAsync(filepath)
When you use UWP your access to the file system is limited. This means that you can't simply read a file at an arbitrary path like you're trying to do here:
var filecon = File.ReadAllBytes(imgpath);
What you need to do instead is to ask the StorageFile object you received from the FilePicker for a read Stream. Like so:
var buffer = await FileIO.ReadBufferAsync(file);
var filecon = buffer.ToArray();
finalimg = Convert.ToBase64String(filecon);
You can find more information about file access for UWP at Microsoft Docs.
I am currently trying to make a post request with a file attached to the form but I found out that it is not the file I have attached but just the path to the file.
My question is how do I get this file and attach it to the form?
Here is my code so far:
string altPath = Path.Combine(Application.persistentDataPath, "nice-work.wav");
List<IMultipartFormSection> formData = new List<IMultipartFormSection>();
formData.Add(new MultipartFormFileSection("wavfile", altPath));
UnityWebRequest uwr = UnityWebRequest.Post(url, formData);
yield return uwr.SendWebRequest();
if (uwr.isNetworkError)
{
Debug.Log("Error While Sending: " + uwr.error);
}
else
{
Debug.Log("Received: " + uwr.downloadHandler.text);
}
the variable altPath is the path but not the file and this leads to failed post request.
If you look at MultipartFormFileSection the constructor you are currently using is
MultipartFormFileSection(string data, string fileName)
which is of course not what you want to do.
You rather have to actually read the according file content e.g. simply using File.ReadAllBytes
...
var multiPartSectionName = "wavfile";
var fileName = "nice-work.wav";
var altPath = Path.Combine(Application.persistentDataPath, fileName);
var data = File.ReadAllBytes(altPath);
var formData = new List<IMultipartFormSection>
{
new MultipartFormFileSection(multiPartSectionName, data, fileName)
};
...
or depending on your server side needs
...
var formData = new List<IMultipartFormSection>
{
new MultipartFormFileSection(fileName, data)
};
...
Though have in mind that ReadAllBytes is a synchronous (blocking) call and for larger files you might rather want to use some asynchronous approach.
My text file is saved on my hard disk and I want to upload it in Visual Studio to this file management service: https://learn.microsoft.com/en-us/gaming/playfab/features/data/content-delivery-network/quickstart#fetching-assets
Before uploading, I want to create the file headers, because I want to add a version number of the file to the headers. For example, I want to add "1" to the headers, if it is the first version. How can I do that?
After that, I want to upload the text file. But I don't know how to upload the file with httpClient. How can I upload the file?
I have tried it with this code, but it's not working and I don't know what I'm doing wrong. What is wrong?
private async Task UpoadNewContent()
{
var result = await PlayFabAdminAPI.GetContentUploadUrlAsync(new GetContentUploadUrlRequest()
{
Key = "Levelfiles/Level2.txt",
ContentType = "binary/octet-stream"
});
if (result.Error != null)
{
Console.WriteLine("Error.");
}
else
{
Uri webService = new Uri(result.Result.URL + "Levelfiles/" + "Level2.txt");
await UploadFileAsync(webService, "/Users/myname/TESTFOLDER/Level2.txt");
}
}
async Task UploadFileAsync(Uri uri, string filename)
{
using (var stream = File.OpenRead(filename))
{
var client = new HttpClient();
var response = await client.PostAsync(uri, new StreamContent(stream));
response.EnsureSuccessStatusCode();
}
}
I get this response:
I upload my files in azure data lake. I try to download that file through asp.net mvc application.I have adl path for that file. I can download below 150 MB files. But i can't download the more then 150 MB files. Time out error came.
My Code in the bellow...
public ActionResult Download(string adlpath)
{
String header = adlpath;
Console.WriteLine(header);
string[] splitedStr = header.Split('/');
var path = GenerateDownloadPaths(adlpath);
string filename = path["fileName"];
HttpResponseMessage val = DataDownloadFile(path["fileSrcPath"]);
byte[] filedata = val.Content.ReadAsByteArrayAsync().Result;
string contentType = MimeMapping.GetMimeMapping(filename);
var cd = new System.Net.Mime.ContentDisposition
{
FileName = filename,
Inline = true,
};
Response.AppendHeader("Content-Disposition", cd.ToString());
return File(filedata, contentType);
}
public HttpResponseMessage DataDownloadFile(string srcFilePath)
{
string DownloadUrl = "https://{0}.azuredatalakestore.net/webhdfs/v1/{1}?op=OPEN&read=true";
var fullurl = string.Format(DownloadUrl, _datalakeAccountName, srcFilePath);
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", _accesstoken.access_token);
using (var formData = new MultipartFormDataContent())
{
resp = client.GetAsync(fullurl).Result;
}
}
return resp;
}
Image :
You should modify your code to use async and await. Your implementation blocks while retrieving the file and that is probably what times out:
public async Task<HttpResponseMessage> DataDownloadFile(string srcFilePath)
{
string DownloadUrl = "https://{0}.azuredatalakestore.net/webhdfs/v1/{1}?op=OPEN&read=true";
var fullurl = string.Format(DownloadUrl, _datalakeAccountName, srcFilePath);
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", _accesstoken.access_token);
using (var formData = new MultipartFormDataContent())
{
resp = await client.GetAsync(fullurl);
}
}
return resp;
}
The return value of the method is changed to Task<HttpResponseMessage> and the async modifier is added.
Calling client.GetAsync is changed to use await instead of blocking by retrieving the Result property.
Your code may still timeout. I believe that there is a configurable limit on how long a request can take before it is aborted and if you still get a timeout you should investigate this.
Per my understanding, you could try to increase the HttpClient.Timeout (100 seconds by default) for your HttpClient instance.
HttpClient.Timeout
Gets or sets the timespan to wait before the request times out.
The default value is 100,000 milliseconds (100 seconds).
Moreover, if you host your application via Azure Web App, you may encounter an idle timeout setting of 4 minutes from Azure Load Balancer. You could change the idle timeout setting in Azure VM and Azure Cloud Service. Details you could follow here.