Upload files in chunks - .Net core - c#

Need help on uploading a file in chunks in .Net core.
I got a stream of 1GB data, need to split in 64MB chucks and post to an API.
I got the below implementation of posting stream of data in single POST request, need suggestion on splitting the stream and POST in multiple API call
private async Task<string> Publish(string siteId, string datasourceName, string projectId, SkyConnection connectionInfo, MemoryStream file, string fileName)
{
_authInfo = await GetAccessToken(siteId);
var datasourceId = string.Empty;
var boundaryId = Guid.NewGuid().ToString();
using (var multipartcontent = new MultipartContent("mixed", boundaryId))
{
var stringContent = new StringContent(GetPublishRequest(datasourceName, projectId, connectionInfo));
stringContent.Headers.Remove("Content-Type");
stringContent.Headers.Remove("Content-Disposition");
stringContent.Headers.TryAddWithoutValidation("Content-Disposition", "name=\"request_payload\"");
stringContent.Headers.TryAddWithoutValidation("Content-Type", "text/xml");
multipartcontent.Add(stringContent);
file.Position = 0;
var streamContent = new StreamContent(file);
streamContent.Headers.TryAddWithoutValidation("Content-Disposition", $"name=\"tableau_datasource\"; filename=\"{fileName}\"");
streamContent.Headers.TryAddWithoutValidation("Content-Type", "application/octet-stream");
multipartcontent.Add(streamContent);
var req = new HttpRequestMessage(HttpMethod.Post, GetUrl($"sites/{_authInfo.SiteId}/datasources?datasourceType=tdsx"));
req.Content = multipartcontent;
req.Headers.Add("X-Tableau-Auth", _authInfo.Token);
req.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var res = await _httpHandler.SendAsync(req);
res.EnsureSuccessStatusCode();
var joResponse = JObject.Parse(await res.Content.ReadAsStringAsync());
datasourceId = joResponse["datasource"]["id"].ToString();
}
return datasourceId;
}

Related

How to split an HTTP Request Body in two parts?

I'm fairly new to use HTTPClient and sending REST requests to APIs, I'm currently practicing multipart upload using this Google Drive API endpoint:
POST https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart
There's an instruction that states there to split the request body into two parts, I tried to recreate this but was unable to do so.
https://developers.google.com/drive/api/guides/manage-uploads#multipart
Here's my current code:
async void UploadFile(StorageFile fileName)
{
using (HttpClient client = new HttpClient())
{
// Opens files and convert it to stream
var resultStream = await fileName.OpenReadAsync();
var fileStreamContent = new StreamContent(resultStream.AsStream());
// Create file MetaData
var fileMetaData = JsonConvert.SerializeObject(
new { name = fileName.Name, mimetype = fileName.ContentType });
// Create POST request
var requestMessage = new HttpRequestMessage(HttpMethod.Post, uploadFileEndpoint);
requestMessage.Headers.Authorization = new AuthenticationHeaderValue(tokenType, accessToken);
// Add request body
requestMessage.Content = new StringContent(fileMetaData, Encoding.UTF8, "application/json");
requestMessage.Content.Headers.ContentType = new MediaTypeHeaderValue("multipart/related");
var response = await client.SendAsync(requestMessage);
string responseString = await response.Content.ReadAsStringAsync();
output(responseString);
}
}
Any help would be greatly appreciated, thank you!
According to the documentation on Perform a multipart upload (HTTP tab), you need the MultipartFormDataContent as suggested by #Jeremy.
There are a few things needed to perform/migrate:
Add AuthenticationHeaderValue into client.DefaultRequestHeaders.Authorization.
Create a StreamContent instance, fileStreamContent (which you have done) and specify its Headers.ContentType.
Create a StringContent instance, stringContent (which you have done).
Append both StreamContent and StringContent into the MultipartFormDataContent instance, formData.
Specify the formData's Headers.ContentType as requested in API docs.
Post the formData with await client.PostAsync(/* API Url */, formData);
using (HttpClient client = new HttpClient())
{
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(tokenType, accessToken);
// Opens files and convert it to stream
var resultStream = await fileName.OpenReadAsync();
var fileStreamContent = new StreamContent(resultStream.AsStream());
fileStreamContent.Headers.ContentType = new MediaTypeHeaderValue(fileName.ContentType);
// Create file MetaData
var fileMetaData = JsonConvert.SerializeObject(new { name = fileName.Name, mimetype = fileName.ContentType });
var stringContent = new StringContent(fileMetaData, Encoding.UTF8, "application/json");
// Create POST request
MultipartFormDataContent formData = new MultipartFormDataContent();
formData.Add(stringContent, "metadata");
formData.Add(fileStreamContent, "media");
formData.Headers.ContentType = new MediaTypeHeaderValue("multipart/related");
var response = await client.PostAsync(uploadFileEndpoint, formData);
string responseString = await response.Content.ReadAsStringAsync();
}

How to return PDF content from another HttpResponseMessage to the browser?

I am in the process of creating a proxy server that makes a request to a PDF Blob link then takes the request to setup its HttpResponse Header which we sent to the client. This diagram should explain
As of now, I am successful at making the request to get the pdf content however I am not sure how to send that back to the user. I have followed other Stackoverflow post such as this one : https://stackoverflow.com/a/43232581/10541061
I turn the response message in step 3 of the diagram to a stream and sent it back in the new HttpResponseMessage content.But instead of PDF content , I get a json file
What I want to return to the client
What I am actually returning to the client
Here is the code I am using to create this proxy endpoint
[AllowAnonymous]
[HttpGet("openPDF")]
public async Task<HttpResponseMessage> OpenPDF([FromQuery] string url)
{
var _httpClient = _httpClientFactory.CreateClient();
var response = await _httpClient.GetAsync(url);
var stream = await response.Content.ReadAsStreamAsync();
HttpResponseMessage message = new HttpResponseMessage(HttpStatusCode.OK);
message.Content = new StreamContent(stream);
message.Content.Headers.ContentLength = stream.Length;
message.Content.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
return message;
}
EDIT
Ok so this actually sends back the PDF when I write the proxy like this
[AllowAnonymous]
[HttpGet("openPDF")]
public async Task<FileStreamResult> OpenPDF([FromQuery] string url)
{
var fileStream = new MemoryStream();
var _httpClient = _httpClientFactory.CreateClient();
var file = await _httpClient.GetStreamAsync(url).ConfigureAwait(false);
await file.CopyToAsync(fileStream);
fileStream.Position = 0;
return File(fileStream, "application/pdf", "filename.pdf");
}
The problem is I want to update the content-disposition to inline so I can force this to open in the browser instead of downloading.So I decided to take the filestream and injecting that in the httpResponseMessage.content instead but that still didn't work. It would continue to send me a json file
[AllowAnonymous]
[HttpGet("openPDF")]
public async Task<HttpResponseMessage> OpenPDF([FromQuery] string url)
{
var fileStream = new MemoryStream();
var _httpClient = _httpClientFactory.CreateClient();
var file = await _httpClient.GetStreamAsync(url).ConfigureAwait(false);
await file.CopyToAsync(fileStream);
fileStream.Position = 0;
HttpResponseMessage message = new HttpResponseMessage(HttpStatusCode.OK);
message.Content = new StreamContent(fileStream);
message.Content.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
return message;
}
To be honest, I thought defining the content-type should suffice but guess not
This is pretty straight forward for .NET 6... suspect it should be roughly the same for .NET 4x... This uses the NuGet package Azure.Storage.Blobs
https://github.com/Azure/azure-sdk-for-net/blob/Azure.Storage.Blobs_12.13.1/sdk/storage/Azure.Storage.Blobs/README.md
[HttpGet("stream")]
public async Task GetBlobAsync()
{
var url = new Uri("https://path.to.blob.content/xxx");
var blobClient = new BlobClient(url);
Response.Headers.Add("Content-Type", "application/pdf");
Response.Headers.Add("Content-Disposition", #"attachment;filename=""intended file name.pdf""");
await blobClient.DownloadToAsync(Response.Body);
}
for .NET 4x.
try to add:
result.Content.Headers.ContentDisposition =
new ContentDispositionHeaderValue("inline")
{
FileName = "filename.pdf"
};

How to implement WebClient.UploadFileAsync with HttpClient? [duplicate]

Does anyone know how to use the HttpClient in .Net 4.5 with multipart/form-data upload?
I couldn't find any examples on the internet.
my result looks like this:
public static async Task<string> Upload(byte[] image)
{
using (var client = new HttpClient())
{
using (var content =
new MultipartFormDataContent("Upload----" + DateTime.Now.ToString(CultureInfo.InvariantCulture)))
{
content.Add(new StreamContent(new MemoryStream(image)), "bilddatei", "upload.jpg");
using (
var message =
await client.PostAsync("http://www.directupload.net/index.php?mode=upload", content))
{
var input = await message.Content.ReadAsStringAsync();
return !string.IsNullOrWhiteSpace(input) ? Regex.Match(input, #"http://\w*\.directupload\.net/images/\d*/\w*\.[a-z]{3}").Value : null;
}
}
}
}
It works more or less like this (example using an image/jpg file):
async public Task<HttpResponseMessage> UploadImage(string url, byte[] ImageData)
{
var requestContent = new MultipartFormDataContent();
// here you can specify boundary if you need---^
var imageContent = new ByteArrayContent(ImageData);
imageContent.Headers.ContentType =
MediaTypeHeaderValue.Parse("image/jpeg");
requestContent.Add(imageContent, "image", "image.jpg");
return await client.PostAsync(url, requestContent);
}
(You can requestContent.Add() whatever you want, take a look at the HttpContent descendant to see available types to pass in)
When completed, you'll find the response content inside HttpResponseMessage.Content that you can consume with HttpContent.ReadAs*Async.
This is an example of how to post string and file stream with HTTPClient using MultipartFormDataContent. The Content-Disposition and Content-Type need to be specified for each HTTPContent:
Here's my example. Hope it helps:
private static void Upload()
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("User-Agent", "CBS Brightcove API Service");
using (var content = new MultipartFormDataContent())
{
var path = #"C:\B2BAssetRoot\files\596086\596086.1.mp4";
string assetName = Path.GetFileName(path);
var request = new HTTPBrightCoveRequest()
{
Method = "create_video",
Parameters = new Params()
{
CreateMultipleRenditions = "true",
EncodeTo = EncodeTo.Mp4.ToString().ToUpper(),
Token = "x8sLalfXacgn-4CzhTBm7uaCxVAPjvKqTf1oXpwLVYYoCkejZUsYtg..",
Video = new Video()
{
Name = assetName,
ReferenceId = Guid.NewGuid().ToString(),
ShortDescription = assetName
}
}
};
//Content-Disposition: form-data; name="json"
var stringContent = new StringContent(JsonConvert.SerializeObject(request));
stringContent.Headers.Add("Content-Disposition", "form-data; name=\"json\"");
content.Add(stringContent, "json");
FileStream fs = File.OpenRead(path);
var streamContent = new StreamContent(fs);
streamContent.Headers.Add("Content-Type", "application/octet-stream");
//Content-Disposition: form-data; name="file"; filename="C:\B2BAssetRoot\files\596090\596090.1.mp4";
streamContent.Headers.Add("Content-Disposition", "form-data; name=\"file\"; filename=\"" + Path.GetFileName(path) + "\"");
content.Add(streamContent, "file", Path.GetFileName(path));
//content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
Task<HttpResponseMessage> message = client.PostAsync("http://api.brightcove.com/services/post", content);
var input = message.Result.Content.ReadAsStringAsync();
Console.WriteLine(input.Result);
Console.Read();
}
}
}
Try this its working for me.
private static async Task<object> Upload(string actionUrl)
{
Image newImage = Image.FromFile(#"Absolute Path of image");
ImageConverter _imageConverter = new ImageConverter();
byte[] paramFileStream= (byte[])_imageConverter.ConvertTo(newImage, typeof(byte[]));
var formContent = new MultipartFormDataContent
{
// Send form text values here
{new StringContent("value1"),"key1"},
{new StringContent("value2"),"key2" },
// Send Image Here
{new StreamContent(new MemoryStream(paramFileStream)),"imagekey","filename.jpg"}
};
var myHttpClient = new HttpClient();
var response = await myHttpClient.PostAsync(actionUrl.ToString(), formContent);
string stringContent = await response.Content.ReadAsStringAsync();
return response;
}
Here is another example on how to use HttpClient to upload a multipart/form-data.
It uploads a file to a REST API and includes the file itself (e.g. a JPG) and additional API parameters. The file is directly uploaded from local disk via FileStream.
See here for the full example including additional API specific logic.
public static async Task UploadFileAsync(string token, string path, string channels)
{
// we need to send a request with multipart/form-data
var multiForm = new MultipartFormDataContent();
// add API method parameters
multiForm.Add(new StringContent(token), "token");
multiForm.Add(new StringContent(channels), "channels");
// add file and directly upload it
FileStream fs = File.OpenRead(path);
multiForm.Add(new StreamContent(fs), "file", Path.GetFileName(path));
// send request to API
var url = "https://slack.com/api/files.upload";
var response = await client.PostAsync(url, multiForm);
}
Here's a complete sample that worked for me. The boundary value in the request is added automatically by .NET.
var url = "http://localhost/api/v1/yourendpointhere";
var filePath = #"C:\path\to\image.jpg";
HttpClient httpClient = new HttpClient();
MultipartFormDataContent form = new MultipartFormDataContent();
FileStream fs = File.OpenRead(filePath);
var streamContent = new StreamContent(fs);
var imageContent = new ByteArrayContent(streamContent.ReadAsByteArrayAsync().Result);
imageContent.Headers.ContentType = MediaTypeHeaderValue.Parse("multipart/form-data");
form.Add(imageContent, "image", Path.GetFileName(filePath));
var response = httpClient.PostAsync(url, form).Result;
Example with preloader Dotnet 3.0 Core
ProgressMessageHandler processMessageHander = new ProgressMessageHandler();
processMessageHander.HttpSendProgress += (s, e) =>
{
if (e.ProgressPercentage > 0)
{
ProgressPercentage = e.ProgressPercentage;
TotalBytes = e.TotalBytes;
progressAction?.Invoke(progressFile);
}
};
using (var client = HttpClientFactory.Create(processMessageHander))
{
var uri = new Uri(transfer.BackEndUrl);
client.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Bearer", AccessToken);
using (MultipartFormDataContent multiForm = new MultipartFormDataContent())
{
multiForm.Add(new StringContent(FileId), "FileId");
multiForm.Add(new StringContent(FileName), "FileName");
string hash = "";
using (MD5 md5Hash = MD5.Create())
{
var sb = new StringBuilder();
foreach (var data in md5Hash.ComputeHash(File.ReadAllBytes(FullName)))
{
sb.Append(data.ToString("x2"));
}
hash = result.ToString();
}
multiForm.Add(new StringContent(hash), "Hash");
using (FileStream fs = File.OpenRead(FullName))
{
multiForm.Add(new StreamContent(fs), "file", Path.GetFileName(FullName));
var response = await client.PostAsync(uri, multiForm);
progressFile.Message = response.ToString();
if (response.IsSuccessStatusCode) {
progressAction?.Invoke(progressFile);
} else {
progressErrorAction?.Invoke(progressFile);
}
response.EnsureSuccessStatusCode();
}
}
}
I'm adding a code snippet which shows on how to post a file to an API which has been exposed over DELETE http verb. This is not a common case to upload a file with DELETE http verb but it is allowed. I've assumed Windows NTLM authentication for authorizing the call.
The problem that one might face is that all the overloads of HttpClient.DeleteAsync method have no parameters for HttpContent the way we get it in PostAsync method
var requestUri = new Uri("http://UrlOfTheApi");
using (var streamToPost = new MemoryStream("C:\temp.txt"))
using (var fileStreamContent = new StreamContent(streamToPost))
using (var httpClientHandler = new HttpClientHandler() { UseDefaultCredentials = true })
using (var httpClient = new HttpClient(httpClientHandler, true))
using (var requestMessage = new HttpRequestMessage(HttpMethod.Delete, requestUri))
using (var formDataContent = new MultipartFormDataContent())
{
formDataContent.Add(fileStreamContent, "myFile", "temp.txt");
requestMessage.Content = formDataContent;
var response = httpClient.SendAsync(requestMessage).GetAwaiter().GetResult();
if (response.IsSuccessStatusCode)
{
// File upload was successfull
}
else
{
var erroResult = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
throw new Exception("Error on the server : " + erroResult);
}
}
You need below namespaces at the top of your C# file:
using System;
using System.Net;
using System.IO;
using System.Net.Http;
P.S. You are seeing a number of using blocks(IDisposable pattern) in the above code snippet which doesn't look very clean. Unfortunately, the syntax of using construct doesn't support initializing multiple variables in single statement.
X509Certificate clientKey1 = null;
clientKey1 = new X509Certificate(AppSetting["certificatePath"],
AppSetting["pswd"]);
string url = "https://EndPointAddress";
FileStream fs = File.OpenRead(FilePath);
var streamContent = new StreamContent(fs);
var FileContent = new ByteArrayContent(streamContent.ReadAsByteArrayAsync().Result);
FileContent.Headers.ContentType = MediaTypeHeaderValue.Parse("ContentType");
var handler = new WebRequestHandler();
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.ClientCertificates.Add(clientKey1);
handler.ServerCertificateValidationCallback = (httpRequestMessage, cert, cetChain, policyErrors) =>
{
return true;
};
using (var client = new HttpClient(handler))
{
// Post it
HttpResponseMessage httpResponseMessage = client.PostAsync(url, FileContent).Result;
if (!httpResponseMessage.IsSuccessStatusCode)
{
string ss = httpResponseMessage.StatusCode.ToString();
}
}
public async Task<object> PassImageWithText(IFormFile files)
{
byte[] data;
string result = "";
ByteArrayContent bytes;
MultipartFormDataContent multiForm = new MultipartFormDataContent();
try
{
using (var client = new HttpClient())
{
using (var br = new BinaryReader(files.OpenReadStream()))
{
data = br.ReadBytes((int)files.OpenReadStream().Length);
}
bytes = new ByteArrayContent(data);
multiForm.Add(bytes, "files", files.FileName);
multiForm.Add(new StringContent("value1"), "key1");
multiForm.Add(new StringContent("value2"), "key2");
var res = await client.PostAsync(_MEDIA_ADD_IMG_URL, multiForm);
}
}
catch (Exception e)
{
throw new Exception(e.ToString());
}
return result;
}

Unable to pass file to web api in ASP.NET MVC Core

I am working on an angular and .NET Core application. I have to pass the file uploaded from angular to WEB API. My code is:
public async Task ImportDataScienceAnalytics(string authToken, IFormFile file)
{
var baseUrl = Import.GetBaseURL();
var client = new RestClientExtended(baseUrl + "algorithm/import");
var request = new RestRequest(Method.POST);
request.AddHeader("authorization", authToken);
string jsonBody = JsonConvert.SerializeObject(file);
request.AddJsonBody(jsonBody);
var response = await client.ExecutePostTaskAsync(request);
var result = response.Content;
}
Issue is that i get "No Attachment Found". I think the issue is because of IFormFile. How can i resolve this issue so that i can upload the file to web api.
It seems that you'd like to post uploaded file to an external API from your API action using RestClient, you can refer to the following code snippet.
var client = new RestClient(baseUrl + "algorithm/import");
var request = new RestRequest(Method.POST);
request.AddHeader("authorization", authToken);
using (var ms = new MemoryStream())
{
file.CopyTo(ms);
var fileBytes = ms.ToArray();
request.AddFile("file", fileBytes, file.FileName, "application/octet-stream");
}
//...
Testing code of Import action
public IActionResult Import(IFormFile file)
{
//...
//code logic here
You need to make following changes to the code.
var baseUrl = Import.GetBaseURL();
var client = new RestClientExtended(baseUrl + "algorithm/import");
var request = new RestRequest(Method.POST);
byte[] data;
using (var br = new BinaryReader(file.OpenReadStream()))
data = br.ReadBytes((int)file.OpenReadStream().Length);
ByteArrayContent bytes = new ByteArrayContent(data);
MultipartFormDataContent multiContent = new MultipartFormDataContent
{
{ bytes, "file", file.FileName }
};
//request.AddHeader("authorization", authToken);
//string jsonBody = JsonConvert.SerializeObject(file);
//request.AddJsonBody(jsonBody);
/// Pass the multiContent into below post
var response = await client.ExecutePostTaskAsync(request);
var result = response.Content;
Do not forget to pass the variable multiContent into the post call.

HttpClient Post photo WITH message to Facebook Graph

Issue
I am trying to upload a photo to the Facebook API WITH a message.
Code Snippet - Upload
requestUri = "https://graph.facebook.com/v2.0/me/photos?access_token=MyAccessToken"
var streamContent = new StreamContent(fileStream);
streamContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("form-data")
{
Name = "\"files\"",
FileName = "\"image.jpg\""
};
streamContent.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
var messageContent = new StringContent("message=HelloWorld");
var resultJson = webRequestClient.Post(requestUri, new MultipartFormDataContent()
{
messageContent,
streamContent,
});
Code - webRequestClient
public string Post(string uri, HttpContent postData)
{
return PostAsync(uri, postData).Result;
}
public async Task<string> PostAsync(string uri, HttpContent httpContent)
{
string resultStream;
using (var httpClient = new HttpClient())
{
var response = await httpClient.PostAsync(uri, httpContent);
response.EnsureSuccessStatusCode();
resultStream = await response.Content.ReadAsStringAsync();
}
return resultStream;
}
Notes
If I remove the "messageContent" : It uploads he picture
If I use MultipartContent : It uploads the picture but ignores my "message"
Don't bother for now why I don't do use the async functionality
When it fails I get "bad" requests
When I append the "message=helloworld" in the requestUri, it works, but that isn't the most flexible solution for in my architecture to deal with this issue.
Check this it will solve you problem, either you have to send the image via stream then you don't need to tell that the type is "image/jpeg" explicitly.
protected async void TakePictureAndUpload()
{
var ui = new CameraCaptureUI();
var file = await ui.CaptureFileAsync(CameraCaptureUIMode.Photo);
if (file != null)
{
byte[] myPicArray = await GetPhotoBytesAsync(file);
HttpClient httpClient = new HttpClient();
httpClient.BaseAddress = new Uri("http://yourdomainname.com");
MultipartFormDataContent form = new MultipartFormDataContent();
HttpContent content = new ByteArrayContent(myPicArray);
form.Add(content, "media", "filename.jpg");
content = new StringContent("my-username");
form.Add(content, "username");
HttpResponseMessage response = await httpClient.PostAsync("directory/my-site.php", form);
}
}
public async Task<byte[]> GetPhotoBytesAsync(StorageFile file)
{
IRandomAccessStream fileStream = await file.OpenAsync(FileAccessMode.Read);
var reader = new Windows.Storage.Streams.DataReader(fileStream.GetInputStreamAt(0));
await reader.LoadAsync((uint)fileStream.Size);
byte[] pixels = new byte[fileStream.Size];
reader.ReadBytes(pixels);
return pixels;
}

Categories

Resources