I'm trying to send large files to Asp.net server from my Xamarin Android application. I found the best way to do it would be to send the file in chunks rather than sending it whole. I'm following the tutorial here which usesHttpWebRequest to send the request to the server. In the following lines of code in the tutorial, the file is sent in chunks to the server:
/ Send the file binaries over to the server, in 1024 bytes chunk
FileStream fileStream = new FileStream(fileUrl, FileMode.Open,
FileAccess.Read);
byte[] buffer = new byte[1024];
int bytesRead = 0;
while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
{
s.Write(buffer, 0, bytesRead);
} // end while
fileStream.Close();
I know that s.Write(buffer, 0, bytesRead) writes the byte on the network socket but how does the back and forth communication work? Does the client wait for the server to respond and send the next chunk to the server or all chunks are written on the network stream and sent together?
Right now the server code I have receives the request which is the full file rather than it's chunks. My server side code looks like this:
public async Task<HttpResponseMessage> UploadFile()
{
var httpRequest = HttpContext.Current.Request;
if (httpRequest.Files.Count > 0)
{
foreach (string file in httpRequest.Files)
{
var postedFile = httpRequest.Files[file];
var filePath = HttpContext.Current.Server.MapPath("~/" + postedFile.FileName);
postedFile.SaveAs(filePath);
// NOTE: To store in memory use postedFile.InputStream
}
return Request.CreateResponse(HttpStatusCode.Created);
}
return Request.CreateResponse(HttpStatusCode.BadRequest);
}
How can I make sure that the connection doesn't time out while the file is sending and is this the most efficient way of sending large file to the server? How can I resume the upload if their a blimp in the netowrk connection on the client side?
Related
I've implemented file uploading/downloading via gRPC, where the client is a console application and the server is an ASP.NET Core application. Uploading is implemented as a client-streaming RPC and downloading is implemented as a server-streaming RPC. The problem I'm having is that whenever I run both the client and the server on the same machine, it takes roughly 10 times as long to upload file x to the server file system as it does to download file x from the server file system (note that all of this is taking place on the hard drive of my machine). The client upload (download) code is symmetric to the server download (upload) code. Here is the relevant snippet of the client upload code:
var clientStreamingCall = client.Upload();
// Now open the file-to-upload for reading
using (FileStream source = File.OpenRead(Path.Combine(baseDir + #"\UploadFiles", fileName)))
{
byte[] buffer = new byte[1024];
int bytesRead;
// write the file chunks to the requestStream
while ((bytesRead = await source.ReadAsync(buffer, 0, buffer.Length)) > 0)
{
await clientStreamingCall.RequestStream.WriteAsync(new UploadRequest()
{
ChunkData = Google.Protobuf.ByteString.CopyFrom(buffer),
});
}
}
Here is the server download code, which mirrors the above client upload code:
public override async Task Download(DownloadRequest request, IServerStreamWriter<DownloadResponse> responseStream, ServerCallContext context)
{
using (FileStream source = File.OpenRead(request.FilePath))
{
byte[] buffer = new byte[1024];
int bytesRead;
// write the file chunks to the responseStream
while ((bytesRead = await source.ReadAsync(buffer, 0, buffer.Length)) > 0)
{
await responseStream.WriteAsync(new DownloadResponse()
{
ChunkData = Google.Protobuf.ByteString.CopyFrom(buffer),
});
}
}
}
Likewise, the client download code mirrors the server upload code; here is the client download code, which writes a chunk of the file-to-download in the response stream to the destination file (on the hard drive of my machine):
while (await serverStreamingCall.ResponseStream.MoveNext(new CancellationToken()))
{
dest.Write(serverStreamingCall.ResponseStream.Current.ChunkData.ToByteArray(), 0, serverStreamingCall.ResponseStream.Current.ChunkData.Length);
}
Here is the server upload code, which writes a chunk of the file-to-upload to the destination file (on the hard drive of my machine):
while (await requestStream.MoveNext())
{
dest.Write(requestStream.Current.ChunkData.ToByteArray(), 0, requestStream.Current.ChunkData.Length);
}
Any idea as to why uploading takes so much longer than downloading, in spite of this symmetry?
I have a endpoint which accepts an HttpPostedFileBase, which provides an input stream which can be read to download the file. I don't actually want to read it - I want to re-post it immediately to another endpoint on a different domain (this is because my API is built on top of another).
Can I do this without waiting for the whole stream to be read? I don't want users of my endpoint to be delayed unnecessarily.
public ActionResult Upload(HttpPostedFileBase image)
string url = "www.example.com";
using (var wb = new WebClient())
{
var stream = wb.OpenWrite(url, "POST");
int read;
byte[] buffer = new byte[0x1000];
while ((read = image.InputStream.Read(buffer, 0, buffer.Length)) > 0)
stream.Write(buffer, 0, read);
}
}
Unfortunately using OpenWrite doesn't really seem to provide any easy option to read the response, so I'll probably just buffer the entire stream and use UploadData :(.
I have a problem trying to stream files from amazon s3. Basically, I have files stored on amazom s3, I can't provide direct access to these files as users need to be authenticated. I'm trying to find a way to stream files without downloading each file from amazon onto my server and then from my server to the end client. I just want to be able to stream the file direct, but it seems most of the stream reader classes in .NET are not cable of doing this. Please correct me if I am wrong.
You can stream the file from Amazon S3 to the client through your server without downloading the file to your server, by opening a stream to the Amazon S3 file then read from it and write on the client stream (buffer by buffer).
Sample Code:
byte[] buffer = new byte[BUFFER_SIZE];
GetObjectRequest getObjRequest = new GetObjectRequest().WithBucketName(Bucket_Name).WithKey(Object_Key);
using (GetObjectResponse getObjRespone = amazonS3Client.GetObject(getObjRequest))
using (Stream amazonStream = getObjRespone.ResponseStream)
{
int bytesReaded = 0;
Response.AddHeader("Content-Length", getObjRespone.ContentLength.ToString());
while ((bytesReaded = amazonStream.Read(buffer, 0, buffer.Length)) > 0 && Response.IsClientConnected)
{
Response.OutputStream.Write(buffer, 0, bytesReaded);
Response.OutputStream.Flush();
buffer = new byte[BUFFER_SIZE];
}
}
I am working on a project that uses a python script to send an image taken with a webcam to a c# webserver using sockets. I am currently sending strings to the server from the python client using code like:
info = bytearray("Text to send", "UTF-8")
socket.send(info)
to send data to the c# server and it functions perfectly for text and numbers. I run into an issue when trying to encode the data read from the .bmp into "UTF-8", as trying this returns an error of not being able to encode certain characters into UTF-8.
I was wondering if anyone had any idea of a way to encode this that c# will be able to recognize, or, if there is a better way of trying to implement this process, I am all ears.
A couple of options I have come up with would be to 1 - use something like google drive to save the image to, or an FTP server and then have the c# server retrieve it from there or 2 - create a packet system containing the RGB values and recreating the image from those pixel values on the server.
Thanks for your help.
EDIT: I have tried sending the file this way
data = bytearray("123456789","UTF-8")
file = open("image.bmp", "rb")
data += file.read()
socket.send(data)
and was able to successfully retreive the string "123456789", but the data after this is garbage. I have also implemented sending the size of the file before sending the data and that size number is retrieved fine, but the img data saves as a black bmp.
Edit 2 :
Here is the server and client code I am using to try and recreate the image using a memory stream. I the client code is using the process mentioned by hcalves.
Client
if __name__ == "__main__":
sock = socket.socket()
sock.connect(("localhost", 50839))
with open("image.bmp", "rb") as fd:
buf = fd.read(1024)
while (buf):
sock.send(buf)
buf = fd.read(1024)
sock.close()
Server
Socket client = server.AcceptSocket();
NetworkStream stream = new NetworkStream(client);
byte[] imgData = new byte[1024];
MemoryStream memstream = new MemoryStream();
stream.Read(imgData,0, 1024);
int counter = 0;
while (stream.DataAvailable)
{
memstream.Write(imgData, 0, 1024);
stream.Read(imgData, 0, 1024);
counter = counter + 1024;
}
memstream.Seek(0, SeekOrigin.Begin);
using (Stream file = File.OpenWrite("img.bmp"))
{
byte[] buffer = new byte[8*1024];
int len;
while ((len = memstream.Read(buffer, 0, buffer.Length)) > 0)
{
file.Write(buffer,0,len);
}
}
You shouldn't need more than this recipe:
import socket
if __name__ == "__main__":
sock = socket.socket()
sock.connect(("localhost", 50839))
with open("data.bin", "rb") as fd:
buf = fd.read(1024)
while (buf):
sock.send(buf)
buf = fd.read(1024)
sock.close()
For practical reasons, you can treat str objects (the result of fd.read) as raw data, you don't need any further crazy encoding. Just iterate the file and send over the socket. Test by running this server which just echoes to stdout with python server.py > data2.bin:
import socket
import sys
if __name__ == "__main__":
sock = socket.socket()
sock.bind(("localhost", 50839))
sock.listen(1)
client, address = sock.accept()
buf = client.recv(1024)
while (buf):
sys.stdout.write(buf)
buf = client.recv(1024)
client.close()
sock.close()
A checksum shows the binary data is sent correctly:
% md5 data.bin data2.bin
MD5 (data.bin) = 8b3280072275badf3e53a6f7aae0b8be
MD5 (data2.bin) = 8b3280072275badf3e53a6f7aae0b8be
Your C# server should be able to accept this data as is. If it doesn't work, it's because your server is expecting something in particular, not just raw data.
How to send application dll from one system to another without File.Copy() in C#? May be by FileStream, any Idea ?
I m using web Service
Web service Code
public string fileUpdates(string filesPath)
{
//System.IO.DirectoryInfo dir = new System.IO.DirectoryInfo(filesPath);
//string[] fileNames = Directory.GetFiles(filesPath);
FileStream fileStream = new FileStream(filesPath, FileMode.Open, FileAccess.Read);
FileInfo fi = new FileInfo(filesPath);
string s=fi.Extension;
byte[] byteArr = new byte[fileStream.Length];
fileStream.Read(byteArr, 0, Convert.ToInt32(fileStream.Length));
string data=Encoding.ASCII.GetString(byteArr);
return data;
}
on Client Side
fileDa = wcf3.fileUpdates(listItem);
byteArray = Encoding.ASCII.GetBytes(fileDa);
fileData = new MemoryStream(byteArray);
int Length = 256;
Byte[] buffer = new Byte[Length];
int bytesRead = fileData.Read(buffer, 0, Length);
while (bytesRead > 0)
{
writeStream.Write(buffer, 0, bytesRead);
bytesRead = fileData.Read(buffer, 0, Length);
}
//fileData.Close();
//writeStream.Close();
You could use network sockets - TcpListener in conjunction with a TcpClient. Or if you want to get a level higher up you could use the HTTP protocol which would probably be easier. So on the remote machine you could have a web server running an ASP.NET application that will receive the file and on the client side simply send the file using an HTTP request.
To move files from one computer to the other you need to have setup a server, and a client.
The server can be a web server, or an ftp server.
And from the part of the client you use code to retrieve the file via http, or ftp.
One simple example: Download files using asp.net
I suggest http from the moment you say asp.net, and you probably have iss install and running, you only then need to place the file on the correct url.