Post large files to REST Endpoint c# .net core - c#

I need to post large files in chunks to an external API. The files type is an MP4 that I have downloaded to my local system and they can be up to 4 gig in size. I need to chunk this data and send it out. Everything I looked at deals with the posting from a Web front end (Angular, JS, etc) and handling the chunked data on the controller. I need to take the file that I have saved local and chunk it up and send it off to an existing API that is expecting chunked data.
Thanks

I think these 2 links can help you to achieve what you need, normally the IFormFile has restrictions for big files, in this case, you need to stream it.
This is from MVC 2 and will help you to understand the HttpPostedFileBase approach
Same approach but wrapping it into a class
Asp.net core 2.2 has the correct example on the documentation in case you want to upload bigger files : See this section
The idea behind is to stream the content, for that, you need to disable the bindings that Asp.net core has and start streaming the content that was posted/uploaded.
After you receive that information, then you use the FormValueProvider to rebind all the key/value you received from the client.
Because you are using multipart content type, you need to be aware that all the content will not come in the same order, maybe you receive the file, later other parameters or vice-versa.
[HttpPost(Name = "CreateDocumentForApplication")]
[DisableFormValueModelBinding]
public async Task<IActionResult> CreateDocumentForApplication(Guid tenantId, Guid applicationId, DocumentForCreationDto docForCreationDto, [FromHeader(Name = "Accept")] string mediaType)
{
//use here the code of the asp.net core documentation on the Upload method to read the file and save it, also get your model correctly after the binding
}
you can notice that I am passing more parameters as part of the post like DocumentForCreationDto, but the approach is the same(disable the binding)
public class DocumentForCreationDto : IDto
{
//public string CreatedBy { get; set; }
public string DocumentName { get; set; }
public string MimeType { get; set; }
public ICollection<DocumentTagInfoForCreationDto> Tags { get; set; }
}
If you want to use the postman, see how I am passing the paremeters:
If you want to upload it via code here is the pseudocode:
void Upload()
{
var content = new MultipartFormDataContent();
// Add the file
var fileContent = new ByteArrayContent(file);
fileContent.Headers.ContentDisposition = new ContentDispositionHeaderValue("form-data")
{
FileName = fileName,
FileNameStar = "file"
};
content.Add(fileContent);
//this is the way to add more content to the post
content.Add(new StringContent(documentUploadDto.DocumentName), "DocumentName");
var url = "myapi.com/api/documents";
HttpResponseMessage response = null;
try
{
response = await _httpClient.PostAsync(url, content);
}
catch (Exception e)
{
Console.WriteLine(e);
}
}
Hope this helps

Related

Could not upload photo using IFormFile object in ASP.Net Core 2 with null reference expception

I tried to upload a photo using IFormFile using a postman plugin. But the API didn't get the file object from the body of the request. I tried with and without [FromBody].
[HttpPost]
public async Task<IActionResult> Upload(int vId, IFormFile fileStream)
{
var vehicle = await this.repository.GetVehicle(vId, hasAdditional: false);
if (vehicle == null)
return NotFound();
var uploadsFolderPath = Path.Combine(host.WebRootPath, "uploads");
if (!Directory.Exists(uploadsFolderPath))
Directory.CreateDirectory(uploadsFolderPath);
var fileName = Guid.NewGuid().ToString() + Path.GetExtension(fileStream.FileName);
var filePath = Path.Combine(uploadsFolderPath, fileName);
using (var stream = new FileStream(filePath, FileMode.Create))
{
await fileStream.CopyToAsync(stream);
}
The error shows on this line :
var fileName = Guid.NewGuid().ToString() + Path.GetExtension(fileStream.FileName);
I figured out that it is not getting the file, while I am sending an image.jpg with the same key "fileStream". By the way everything else works fine. I found no solution to fix this issue. If anybody can help me with this please let me know.
The FromBody attribute can only be used on one parameter in the signature.
One option to send the int vId is by a query string and read it with the FromQuery attribute.
Try it like this
[HttpPost]
public async Task<IActionResult> Upload([FromQuery]int vId, [FromBody]IFormFile fileStream)
Then make the POST to url api/yourController?vId=123456789 where the body contains the IFromFile
Update
As the form-data will be sent as key-value try and create a model containing the keys and read it from the body
public class RequestModel
{
public IFormFile fileStream { get; set; }
}
Then read the model from the body
[HttpPost]
public async Task<IActionResult> Upload([FromBody]RequestModel model)
Finally got the solution. Actually the problem was with the old version of postman Tabbed Postman - REST Client chrome extension. After trying with new postman app it worked perfectly fine. Thanks all of you who tried to solve this problem. Here is the result:enter image description here

WebAPI + OWIN: zipping files directly to the output stream does not work as expected

I'm developing a website with ASP.NET MVC 5 + Web API. One of the requirements is that users must be able to download a large zip file, which is created on the fly.
Because I immediately want to show progress of the user, my idea was to use a PushStreamContent with a callback in the resonse. The callback creates the zipfile and streams it to the response.
When I implement this as follows, starting from an empty ASP.NET MVC + Web API project, it works as expected. As soon as the result is returned to the client, the callback gets invoked and
the zipfile is streamed to the client. So the user can see progress as soon as the callback creates the zip archive and add files to it.
[RoutePrefix("api/download")]
public class DownloadController : ApiController
{
[HttpGet]
public HttpResponseMessage Get()
{
var files = new DirectoryInfo(#"c:\tempinput").GetFiles();
var pushStreamContent = new PushStreamContent(async (outputStream, httpContext, transportContext) =>
{
using (var zipOutputStream = new ZipOutputStream(outputStream))
{
zipOutputStream.CompressionLevel = CompressionLevel.BestCompression;
foreach (var file in files)
{
zipOutputStream.PutNextEntry(file.Name);
using (var stream = File.OpenRead(file.FullName))
{
await stream.CopyToAsync(zipOutputStream);
}
}
}
});
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = pushStreamContent
};
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
response.Content.Headers.ContentDisposition =
new ContentDispositionHeaderValue("attachment") {FileName = "MyZipfile.zip"};
return response;
}
}
Now, I have to integrate this in an existing website, which is configured to use Microsoft.Owin.OwinMiddleware. I used the same code as pasted above, but now the behavior is different: during the creation of the zipfile, it 's not streamed to the response, but only downloaded when the creation of the zip has finished. So the user doesn't see any progress during the creation of the file.
I also tried a different approach in my Web API + Owin project, as described here: (generate a Zip file from azure blob storage files).
In an empty Asp.NET MVC project (without OWIN middleware), this works exactly as expected, but when OWIN is involved, I get this HTTPException and stacktrace:
System.Web.HttpException: 'Server cannot set status after HTTP headers have been sent.'
System.Web.dll!System.Web.HttpResponse.StatusCode.set(int value) Unknown
System.Web.dll!System.Web.HttpResponseWrapper.StatusCode.set(int value) Unknown
Microsoft.Owin.Host.SystemWeb.dll!Microsoft.Owin.Host.SystemWeb.OwinCallContext.Microsoft.Owin.Host.SystemWeb.CallEnvironment.AspNetDictionary.IPropertySource.SetResponseStatusCode(int value) Unknown
It seems that OWIN wants to set a response status, although that was already done in my Get() method (HttpResponseMessage(HttpStatusCode.OK)).
Any suggestions how to fix this or ideas for a different approach?
Thanks a lot!

How upload file and FormData in one method without saving file on disk on server?

I want to upload file and some FormData to server. I'm did it like that and it is works:
On client side i use Angular 2 and logic looks follows:
1.In component
onLoadForeignLightCompanies(event: any) {
let fileList: FileList = event.target.files;
if (fileList.length > 0) {
let file: File = fileList[0];
let formData: FormData = new FormData();
formData.append('projectId', this.projectId);
formData.append('file', file);
this.companyService.uploadForeginLightCompany(formData).then(result => {});
this.isLoading = false;
window.location.reload();
}
}
2.In service
uploadForeginLightCompany(formData: FormData): Promise<any> {
return this.http.post(this.baseUrl + 'foreignLightCompanyImport', formData).toPromise();
}
On server side
public byte[] LoadUploadedFile(HttpPostedFile uploadedFile)
{
var buf = new byte[uploadedFile.InputStream.Length];
uploadedFile.InputStream.Read(buf, 0, (int)uploadedFile.InputStream.Length);
return buf;
}
[HttpPost, Route("foreignLightCompanyImport")]
public async void UploadForeignLigtCompanyCompanyFile()
{
string root = HttpContext.Current.Server.MapPath("~/App_Data");
var file = LoadUploadedFile(HttpContext.Current.Request.Files[0]);
var provider = new MultipartFormDataStreamProvider(root);
await Request.Content.ReadAsMultipartAsync(provider);
var projectId = provider.FormData.GetValues("projectId")[0];
((IForeginLightCompanyService) Service).UploadDataFromExcel(file, Convert.ToInt32(projectId));
}
The following construction is used on the server side. HttpContext.Current.Server.MapPath("~/App_Data");
But in this case, the files are stored on disk. I know how to upload a file without saving it to disk. But in this case I can not get other parameters. Like projectId e.t.c.
How this be made without saving file on disk?
May be i must using model on client side and send file like binary string in json with other parameters?
Is anybody knows a solution of this issue or maybe have thoughts on it?
A clean way would be to use a model:
public class UploadForeignLigtCompanyCompanyFile
{
public string ProjectId { get; set; }
public HttpPostedFileBase Attachment { get; set; }
}
And change your post action method to have a parameter of the model type:
public async void UploadForeignLigtCompanyCompanyFile(
UploadForeignLigtCompanyCompanyFile companyFile)
{
// code...
}
Of course you will need to change your view code to submit that info to the action method.

unable to configure Web API for content type multipart

I am working on Web APIs - Web API 2. My basic need is to create an API to update the profile of the user. In this, the ios and android will send me the request in multipart/form-data. They will send me a few parameters with an image. But whenever I try to create the API, my model comes to be null every time.
I have also added this line in WebApiConfig:
config.Formatters.JsonFormatter.SupportedMediaTypes
.Add(new MediaTypeHeaderValue("multipart/form-data"));
This is my class:
public class UpdateProfileModel
{
public HttpPostedFileBase ProfileImage { get; set; }
public string Name { get; set; }
}
This is my controller:
[Route("api/Account/UpdateProfile")]
[HttpPost]
public HttpResponseMessage UpdateProfile(UpdateProfileModel model)
{
}
I am even not getting parameter values in my Model. Am I doing something wrong?
None of the answers related to this were helpful for me. It's about 3rd day and I have tried almost everything and every method. but I am unable to achieve it.
Although I can use this but this as shown below but this doesn't seem to be a good approach. so I am avoiding it.
var httpRequest = HttpContext.Current.Request;
if (httpRequest.Form["ParameterName"] != null)
{
var parameterName = httpRequest.Form["ParameterName"];
}
and for files I can do this:
if (httpRequest.Files.Count > 0)
{
//I can access my files here and save them
}
Please help if you have any good approach for it Or Please explain to me why I am unable to get these values in the Model.
Thanks a lot in Advance
The answer provided by JPgrassi is what you would be doing to have MultiPart data. I think there are few more things that needs to be added, so I thought of writing my own answer.
MultiPart form data, as the name suggest, is not single type of data, but specifies that the form will be sent as a MultiPart MIME message, so you cannot have predefined formatter to read all the contents. You need to use ReadAsync function to read byte stream and get your different types of data, identify them and de-serialize them.
There are two ways to read the contents. First one is to read and keep everything in memory and the second way is to use a provider that will stream all the file contents into some randomly name files(with GUID) and providing handle in form of local path to access file (The example provided by jpgrassi is doing the second).
First Method: Keeping everything in-memory
//Async because this is asynchronous process and would read stream data in a buffer.
//If you don't make this async, you would be only reading a few KBs (buffer size)
//and you wont be able to know why it is not working
public async Task<HttpResponseMessage> Post()
{
if (!request.Content.IsMimeMultipartContent()) return null;
Dictionary<string, object> extractedMediaContents = new Dictionary<string, object>();
//Here I am going with assumption that I am sending data in two parts,
//JSON object, which will come to me as string and a file. You need to customize this in the way you want it to.
extractedMediaContents.Add(BASE64_FILE_CONTENTS, null);
extractedMediaContents.Add(SERIALIZED_JSON_CONTENTS, null);
request.Content.ReadAsMultipartAsync()
.ContinueWith(multiPart =>
{
if (multiPart.IsFaulted || multiPart.IsCanceled)
{
Request.CreateErrorResponse(HttpStatusCode.InternalServerError, multiPart.Exception);
}
foreach (var part in multiPart.Result.Contents)
{
using (var stream = part.ReadAsStreamAsync())
{
stream.Wait();
Stream requestStream = stream.Result;
using (var memoryStream = new MemoryStream())
{
requestStream.CopyTo(memoryStream);
//filename attribute is identifier for file vs other contents.
if (part.Headers.ToString().IndexOf("filename") > -1)
{
extractedMediaContents[BASE64_FILE_CONTENTS] = memoryStream.ToArray();
}
else
{
string jsonString = System.Text.Encoding.ASCII.GetString(memoryStream.ToArray());
//If you need just string, this is enough, otherwise you need to de-serialize based on the content type.
//Each content is identified by name in content headers.
extractedMediaContents[SERIALIZED_JSON_CONTENTS] = jsonString;
}
}
}
}
}).Wait();
//extractedMediaContents; This now has the contents of Request in-memory.
}
Second Method: Using a provider (as given by jpgrassi)
Point to note, this is only filename. If you want process file or store at different location, you need to stream read the file again.
public async Task<HttpResponseMessage> Post()
{
HttpResponseMessage response;
//Check if request is MultiPart
if (!Request.Content.IsMimeMultipartContent())
{
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
//This specifies local path on server where file will be created
string root = HttpContext.Current.Server.MapPath("~/App_Data");
var provider = new MultipartFormDataStreamProvider(root);
//This write the file in your App_Data with a random name
await Request.Content.ReadAsMultipartAsync(provider);
foreach (MultipartFileData file in provider.FileData)
{
//Here you can get the full file path on the server
//and other data regarding the file
//Point to note, this is only filename. If you want to keep / process file, you need to stream read the file again.
tempFileName = file.LocalFileName;
}
// You values are inside FormData. You can access them in this way
foreach (var key in provider.FormData.AllKeys)
{
foreach (var val in provider.FormData.GetValues(key))
{
Trace.WriteLine(string.Format("{0}: {1}", key, val));
}
}
//Or directly (not safe)
string name = provider.FormData.GetValues("name").FirstOrDefault();
response = Request.CreateResponse(HttpStatusCode.Ok);
return response;
}
By default there is not a media type formatter built into the api that can handle multipart/form-data and perform model binding. The built in media type formatters are :
JsonMediaTypeFormatter: application/json, text/json
XmlMediaTypeFormatter: application/xml, text/xml
FormUrlEncodedMediaTypeFormatter: application/x-www-form-urlencoded
JQueryMvcFormUrlEncodedFormatter: application/x-www-form-urlencoded
This is the reason why most answers involve taking over responsibility to read the data directly from the request inside the controller. However, the Web API 2 formatter collection is meant to be a starting point for developers and not meant to be the solution for all implementations. There are other solutions that have been created to create a MediaFormatter that will handle multipart form data. Once a MediaTypeFormatter class has been created it can be re-used across multiple implementations of Web API.
How create a MultipartFormFormatter for ASP.NET 4.5 Web API
You can download and build the full implementation of the web api 2 source code and see that the default implementations of media formatters do not natively process multi part data.
https://aspnetwebstack.codeplex.com/
You can't have parameters like that in your controller because there's no built-in media type formatter that handles Multipart/Formdata. Unless you create your own formatter, you can access the file and optional fields accessing via a MultipartFormDataStreamProvider :
Post Method
public async Task<HttpResponseMessage> Post()
{
HttpResponseMessage response;
//Check if request is MultiPart
if (!Request.Content.IsMimeMultipartContent())
{
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
string root = HttpContext.Current.Server.MapPath("~/App_Data");
var provider = new MultipartFormDataStreamProvider(root);
//This write the file in your App_Data with a random name
await Request.Content.ReadAsMultipartAsync(provider);
foreach (MultipartFileData file in provider.FileData)
{
//Here you can get the full file path on the server
//and other data regarding the file
tempFileName = file.LocalFileName;
}
// You values are inside FormData. You can access them in this way
foreach (var key in provider.FormData.AllKeys)
{
foreach (var val in provider.FormData.GetValues(key))
{
Trace.WriteLine(string.Format("{0}: {1}", key, val));
}
}
//Or directly (not safe)
string name = provider.FormData.GetValues("name").FirstOrDefault();
response = Request.CreateResponse(HttpStatusCode.Ok);
return response;
}
Here's a more detailed list of examples:
Sending HTML Form Data in ASP.NET Web API: File Upload and Multipart MIME
Not so sure this would be helpful in your case , have a look
mvc upload file with model - second parameter posted file is null
and
ASP.Net MVC - Read File from HttpPostedFileBase without save
So, what worked for me is -
[Route("api/Account/UpdateProfile")]
[HttpPost]
public Task<HttpResponseMessage> UpdateProfile(/* UpdateProfileModel model */)
{
string root = HttpContext.Current.Server.MapPath("~/App_Data");
var provider = new MultipartFormDataStreamProvider(root);
await Request.Content.ReadAsMultipartAsync(provider);
foreach (MultipartFileData file in provider.FileData)
{
}
}
Also -
config.Formatters.JsonFormatter.SupportedMediaTypes.Add(new MediaTypeHeaderValue("multipart/form-data"));
isn't required.
I guess the multipart/form-data is internally handled somewhere after the form is submitted.
Very clearly described here -
http://www.asp.net/web-api/overview/advanced/sending-html-form-data-part-2

Can I upload a file and a model to an MVC 4 action at the same time from a Winforms app?

I am required to integrate a signature pad into an intranet (MVC4) application allowing people to apply electronic signatures to system generated documents. Unfortunately, the signature pad I've been given only has a COM/ActiveX API, so I've written a short Windows Forms application that will allow the user to capture the signature and upload it to the server. When it is uploaded, I need the MVC4 action to associate the signature image with a specified document entity sent by the Windows Forms request. So, say I have this model:
public class DocumentToSign {
public int DocumentId { get; set; }
public int DocumentTypeId { get; set; }
}
Then I have this action to receive the uploaded image:
[HttpPost]
public ActionResult UploadSignature(DocumentToSign doc, HttpPostedFileBase signature)
{
//do stuff and catch errors to report back to winforms app
return Json(new {Success = true, Error = string.Empty});
}
Then, the code to upload the image:
var doc = new DocumentToSign{ DocumentId = _theId, DocumentTypeId = _theType };
var fileName = SaveTheSignature();
var url = GetTheUrl();
using(var request = new WebClient())
{
request.Headers.Add("enctype", "multipart/form-data");
foreach(var prop in doc.GetType().GetProperties())
{
request.QueryString.Add(prop.Name, Convert.ToString(prop.GetValue(doc, null)));
}
var responseBytes = request.UploadFile(url, fileName);
//deserialize resulting Json, etc.
}
The model binder seems to pick up the DocumentToSign class without any problems, but the HttpPostedFileBase is always null. I know that I need to somehow tell the model binder that the uploaded image is the signature parameter in the action, but I can't figure out how to do it. I tried using UploadValues with a NameValueCollection, but NameValueCollection only allows the value to be a string, so the image (even as a byte[]) can't be part of that.
Is it possible to upload a file as well as a model to the same action from outside of the actual MVC4 application? Should I be using something other than HttpPostedFileBase? Other than the WebClient? I am at a loss.
var responseBytes = request.UploadFile(url, fileName); is not sending your file in the format your controller expect.
HttpPostedFileBase works with multipart/form-data POST request. But WebClient.UploadFile is not sending a multipart request, it sends file content as a body of request with no other information.
You can save the file by calling Request.SaveAs(filename, false);
or you have to change the way you are sending the file. But I don't think WebClient support sending multipart requests.

Categories

Resources