Develop Reference

How can I send a File over a REST API?

I am trying to send a file to a server over a REST API. The file could potentially be of any type, though it can be limited in size and type to things that can be sent as email attachments.
I think my approach will be to send the file as a binary stream, and then save that back into a file when it arrives at the server. Is there a built in way to do this in .Net or will I need to manually turn the file contents into a data stream and send that?
For clarity, I have control over both the client and server code, so I am not restricted to any particular approach.

I'd recommend you look into RestSharp
http://restsharp.org/
The RestSharp library has methods for posting files to a REST service. (RestRequst.AddFile()). I believe on the server-side this will translate into an encoded string into the body, with the content-type in the header specifying the file type.
I've also seen it done by converting a stream to a base-64 string, and transferring that as one of the properties of the serialized json/xml object. Especially if you can set size limits and want to include file meta-data in the request as part of the same object, this works really well.
It really depends how large your files are though. If they are very large, you need to consider streaming, of which the nuances of that is covered in this SO post pretty thoroughly: How do streaming resources fit within the RESTful paradigm?

You could send it as a POST request to the server, passing file as a FormParam.
#POST
#Path("/upload")
//#Consumes(MediaType.MULTIPART_FORM_DATA)
#Consumes("application/x-www-form-urlencoded")
public Response uploadFile( #FormParam("uploadFile") String script, #HeaderParam("X-Auth-Token") String STtoken, #Context HttpHeaders hh) {
// local variables
String uploadFilePath = null;
InputStream fileInputStream = new ByteArrayInputStream(script.getBytes(StandardCharsets.UTF_8));
//System.out.println(script); //debugging
try {
uploadFilePath = writeToFileServer(fileInputStream, SCRIPT_FILENAME);
}
catch(IOException ioe){
ioe.printStackTrace();
}
return Response.ok("File successfully uploaded at " + uploadFilePath + "\n").build();
}
private String writeToFileServer(InputStream inputStream, String fileName) throws IOException {
OutputStream outputStream = null;
String qualifiedUploadFilePath = SIMULATION_RESULTS_PATH + fileName;
try {
outputStream = new FileOutputStream(new File(qualifiedUploadFilePath));
int read = 0;
byte[] bytes = new byte[1024];
while ((read = inputStream.read(bytes)) != -1) {
outputStream.write(bytes, 0, read);
}
outputStream.flush();
}
catch (IOException ioe) {
ioe.printStackTrace();
}
finally{
//release resource, if any
outputStream.close();
}
return qualifiedUploadFilePath;
}

Building on to #MutantNinjaCodeMonkey's suggestion of RestSharp. My use case was posting webform data from jquery's $.ajax method into a web api controller. The restful API service required the uploaded file to be added to the request Body. The default restsharp method of AddFile mentioned above caused an error of The request was aborted: The request was canceled. The following initialization worked:
// Stream comes from web api's HttpPostedFile.InputStream
(HttpContext.Current.Request.Files["fileUploadNameFromAjaxData"].InputStream)
using (var ms = new MemoryStream())
{
fileUploadStream.CopyTo(ms);
photoBytes = ms.ToArray();
}
var request = new RestRequest(Method.PUT)
{
AlwaysMultipartFormData = true,
Files = { FileParameter.Create("file", photoBytes, "file") }
};

Detect the file/s being transported with the request.
Decide on a path where the file will be uploaded (and make sure CHMOD 777 exists for this directory)
Accept the client connect
Use ready library for the actual upload
Review the following discussion:
REST file upload with HttpRequestMessage or Stream?

First, you should login to the server and get an access token.
Next, you should convert your file to stream and post the stream:
private void UploadFile(FileStream stream, string fileName)
{
string apiUrl = "http://example.com/api";
var formContent = new MultipartFormDataContent
{
{new StringContent(fileName),"FileName"},
{new StreamContent(stream),"formFile",fileName},
};
using HttpClient httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Add("Authorization", accessToken);
var response = httpClient.PostAsync(#$"{apiUrl}/FileUpload/save", formContent);
var result = response.Result.Content.ReadAsStringAsync().Result;
}
In this example, we upload the file to http://example.com/api/FileUpload/save and the controller has the following method in its FileUpload controller:
[HttpPost("Save")]
public ActionResult Save([FromForm] FileContent fileContent)
{
// ...
}
public class FileContent
{
public string FileName { get; set; }
public IFormFile formFile { get; set; }
}

How live video or audio streams work?

I coding the server that generates html pages, so users can view those in their browsers.
It has onGetRequest event and this is handler for it:
var req = e.Request;
var res = e.Response;
var path = req.RawUrl.Replace("%20", " ");
if (path == "/")
path += "index.html";
if (path.Contains("/../"))
{
res.StatusCode = (int)HttpStatusCode.Forbidden;
return;
}
var content = this.ServerToRun.GetFile(path); //getting file to read
if (content == null)
{
res.StatusCode = (int)HttpStatusCode.NotFound;
return;
}
string extension = path.Substring(path.LastIndexOf('.'));
string auto_mime = PageControls.MimeTypeDeterminer.GetMimeTypeFor(extension);
if (string.IsNullOrEmpty(auto_mime))
{
if (extension.Length > 1)
res.ContentType = "application/" + extension.Substring(1);
else
res.ContentType = "application/unknown";
}
else
res.ContentType = auto_mime;
if (path.EndsWith(".html") || path.EndsWith(".htm"))
res.ContentEncoding = Encoding.UTF8;
res.WriteContent(content); //sending content to client
I don't understand what is needed to do for supporting live streams.
For example, I can record audio from microphone, so file will increase it's size every second.
I can do this in html code:
<audio>
<source src = "live.wav" type = "audio/wav" />
</audio>
The server will receive query for this file, read it till the end and send it to client, but right after this live.wav will get more chunks of sound that server will not send to client anymore.
So, I am stuck, how live streams ever work and what I need to do?
I have WebSocket opened to every client, so I can call some scripts.

You should use the Transfer-Encoding: Chunked HTTP header. This header allows you to send data in chunks without the need to specifiy a Content-Length, thus the client will not close the socket until the server indicates the last chunk has been sent. See https://en.wikipedia.org/wiki/Chunked_transfer_encoding.

RestSharp AddFile Using Stream

I am using RestSharp (version 105.2.3.0 in Visual Studio 2013, .net 4.5) to call a NodeJS hosted webservice. One of the calls I need to make is to upload a file. Using a RESTSharp request, if I retrieve the stream from my end into a byte array and pass that to AddFile, it works fine. However, I would much rather stream the contents and not load up entire files in server memory (the files can be 100's of MB).
If I set up an Action to copy my stream (see below), I get an exception at the "MyStream.CopyTo" line of System.Net.ProtocolViolationException (Bytes to be written to the stream exceed the Content-Length bytes size specified). This exception is thrown within the Action block after client.Execute is called.
From what I read, I should not be manually adding a Content-Length header, and it doesn't help if I do. I have tried setting CopyTo buffer too small and large values, as well as omitting it entirely, to no avail. Can somebody give me a hint on what I've missed?
// Snippet...
protected T PostFile<T>(string Resource, string FieldName, string FileName,
string ContentType, Stream MyStream,
IEnumerable<Parameter> Parameters = null) where T : new()
{
RestRequest request = new RestRequest(Resource);
request.Method = Method.POST;
if (Parameters != null)
{
// Note: parameters are all UrlSegment values
request.Parameters.AddRange(Parameters);
}
// _url, _username and _password are defined configuration variables
RestClient client = new RestClient(_url);
if (!string.IsNullOrEmpty(_username))
{
client.Authenticator = new HttpBasicAuthenticator(_username, _password);
}
/*
// Does not work, throws System.Net.ProtocolViolationException,
// Bytes to be written to the stream exceed the
// Content-Length bytes size specified.
request.AddFile(FieldName, (s) =>
{
MyStream.CopyTo(s);
MyStream.Flush();
}, FileName, ContentType);
*/
// This works, but has to load the whole file in memory
byte[] data = new byte[MyStream.Length];
MyStream.Read(data, 0, (int) MyStream.Length);
request.AddFile(FieldName, data, FileName, ContentType);
var response = client.Execute<T>(request);
// check response and continue...
}

I had the same issue. I ended up using the .Add() on the Files collection. It has a FileParameter param which has the same params as AddFile(), you just have to add the ContentLength:
var req = GetRestRequest("Upload", Method.POST, null);
//req.AddFile("file",
// (s) => {
// var stream = input(imageObject);
// stream.CopyTo(s);
// stream.Dispose();
// },
// fileName, contentType);
req.Files.Add(new FileParameter {
Name = "file",
Writer = (s) => {
var stream = input(imageObject);
stream.CopyTo(s);
stream.Dispose();
},
FileName = fileName,
ContentType = contentType,
ContentLength = contentLength
});

The following code works for me for uploading a csv file using rest sharp. Web services API has been called.
var client = new RestClient(<YOUR API END URL >);
var request = new RestRequest(Method.POST) ;
request.AlwaysMultipartFormData = true;
request. AddHeader("Content-Type", "multipart/form-data");
request.AddHeader("X-API-TOKEN", <Your Unique Token - again not needed for certain calls>);
request.AddParameter(<Your parameters.....>);
request.AddFile("file", currentFileLocation, contentType);
request.AddParameter("multipart/form-data", fileName, ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
var response = client.Execute(request);

Retrieving website content and returning it in ASP.NET MVC 4

I have two servers. One is a private server and I don't want users to have direct access to it, and the other one is the server that public does have access to.
I can access my private server by URL like: http://xxx.xx.xxx.xxx/
What i want to do is create some kind of "proxy", only to work with my private server. My idea is to go to: http://www.domain.com/server/path/here/something
This page should show me the content of http://xxx.xx.xxx.xxx/path/here/something
I have this working, but the only way I could make it work was to return the content as a string, and then the browser would interpret the HTML.
This works fine for pages that return HTML content, but it doesn't work (of course) if I want to access a .gif or any kind of file directly.
Here's the code I currently have:
public string Index(string url)
{
string uri = "http://xxx.xx.xxx.xxx/" + url;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "GET";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader responseStream = new StreamReader(response.GetResponseStream());
string resultado = responseStream.ReadToEnd();
return resultado;
}
How can I change my code so that it works for any file ?

You can check the response content type and do what you need based on that.
You'll need to change your action to return ActionResult instead of string.
if(response.ContentType.Equals("text/html"))
{
//show html stuff
return Content(resultado);
}
else if(response.ContentType.Contains("image/"))
{
var ms = new MemoryStream();
responseStream.BaseStream.CopyTo(ms);
var imageBytes = ms.ToArray();
return File(imageBytes, response.ContentType);
}

you have to write a system which reads your html or images from resultado and do something according to that PLUS you need to control your Url as well.

How to get the file size from http headers

I want to get the size of an http:/.../file before I download it. The file can be a webpage, image, or a media file. Can this be done with HTTP headers? How do I download just the file HTTP header?

Yes, assuming the HTTP server you're talking to supports/allows this:
public long GetFileSize(string url)
{
long result = -1;
System.Net.WebRequest req = System.Net.WebRequest.Create(url);
req.Method = "HEAD";
using (System.Net.WebResponse resp = req.GetResponse())
{
if (long.TryParse(resp.Headers.Get("Content-Length"), out long ContentLength))
{
result = ContentLength;
}
}
return result;
}
If using the HEAD method is not allowed, or the Content-Length header is not present in the server reply, the only way to determine the size of the content on the server is to download it. Since this is not particularly reliable, most servers will include this information.

Can this be done with HTTP headers?
Yes, this is the way to go. If the information is provided, it's in the header as the Content-Length. Note, however, that this is not necessarily the case.
Downloading only the header can be done using a HEAD request instead of GET. Maybe the following code helps:
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://example.com/");
req.Method = "HEAD";
long len;
using(HttpWebResponse resp = (HttpWebResponse)(req.GetResponse()))
{
len = resp.ContentLength;
}
Notice the property for the content length on the HttpWebResponse object – no need to parse the Content-Length header manually.

Note that not every server accepts HTTP HEAD requests. One alternative approach to get the file size is to make an HTTP GET call to the server requesting only a portion of the file to keep the response small and retrieve the file size from the metadata that is returned as part of the response content header.
The standard System.Net.Http.HttpClient can be used to accomplish this. The partial content is requested by setting a byte range on the request message header as:
request.Headers.Range = new RangeHeaderValue(startByte, endByte)
The server responds with a message containing the requested range as well as the entire file size. This information is returned in the response content header (response.Content.Header) with the key "Content-Range".
Here's an example of the content range in the response message content header:
{
"Key": "Content-Range",
"Value": [
"bytes 0-15/2328372"
]
}
In this example the header value implies the response contains bytes 0 to 15 (i.e., 16 bytes total) and the file is 2,328,372 bytes in its entirety.
Here's a sample implementation of this method:
public static class HttpClientExtensions
{
public static async Task<long> GetContentSizeAsync(this System.Net.Http.HttpClient client, string url)
{
using (var request = new System.Net.Http.HttpRequestMessage(System.Net.Http.HttpMethod.Get, url))
{
// In order to keep the response as small as possible, set the requested byte range to [0,0] (i.e., only the first byte)
request.Headers.Range = new System.Net.Http.Headers.RangeHeaderValue(from: 0, to: 0);
using (var response = await client.SendAsync(request))
{
response.EnsureSuccessStatusCode();
if (response.StatusCode != System.Net.HttpStatusCode.PartialContent)
throw new System.Net.WebException($"expected partial content response ({System.Net.HttpStatusCode.PartialContent}), instead received: {response.StatusCode}");
var contentRange = response.Content.Headers.GetValues(#"Content-Range").Single();
var lengthString = System.Text.RegularExpressions.Regex.Match(contentRange, #"(?<=^bytes\s[0-9]+\-[0-9]+/)[0-9]+$").Value;
return long.Parse(lengthString);
}
}
}
}

WebClient webClient = new WebClient();
webClient.OpenRead("http://stackoverflow.com/robots.txt");
long totalSizeBytes= Convert.ToInt64(webClient.ResponseHeaders["Content-Length"]);
Console.WriteLine((totalSizeBytes));

HttpClient client = new HttpClient(
new HttpClientHandler() {
Proxy = null, UseProxy = false
} // removes the delay getting a response from the server, if you not use Proxy
);
public async Task<long?> GetContentSizeAsync(string url) {
using (HttpResponseMessage responce = await client.GetAsync(url))
return responce.Content.Headers.ContentLength;
}