This question already has an answer here:
Upload and download a file to/from FTP server in C#/.NET
(1 answer)
Closed 4 years ago.
The code below works when i upload files around 12kb and below but for some reason anything above that size results in the program freezing and getting an error:
"The underlying connection was closed: An unexpected error occurred on a receive"
Is there something I am doing wrong?
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using System.IO;
using System.Net;
namespace FTP3
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void uploadFile(string filePath)
{
FtpWebRequest request = (FtpWebRequest)FtpWebRequest.Create("ftp://175.137.158.136" + "/" + Path.GetFileName(filePath));
request.Method = WebRequestMethods.Ftp.UploadFile;
request.Credentials = new NetworkCredential(username, password);
request.UsePassive = true;
request.UseBinary = true;
request.KeepAlive = false;
FileStream stream = File.OpenRead(filePath);
byte[] buffer = new byte[stream.Length];
stream.Read(buffer, 0, buffer.Length);
stream.Close();
Stream reqStream = request.GetRequestStream();
reqStream.Write(buffer, 0, buffer.Length);
reqStream.Close();
MessageBox.Show("Upload complete!");
}
private void button1_Click(object sender, EventArgs e)
{
uploadFile(#"C:\Users\User\Desktop\test\data.xlsx");
}
}
}
you should upload the file in 'chunks', not completely. So please try the following:
using (FileStream fileStream = File.OpenRead(filePath))
{
using (Stream reqStream = request.GetRequestStream())
{
long curFileStreamPos = 0;
long chunkSize = 512;
while (curFileStreamPos < fileStream.Length)
{
if (fileStream.Length - curFileStreamPos < chunkSize)
chunkSize = fileStream.Length - curFileStreamPos;
byte[] buff = new byte[chunkSize];
fileStream.Read(buff, 0, buff.Length);
reqStream.Write(buff, 0, buff.Length);
curFileStreamPos += chunkSize;
}
reqStream.Close();
}
fileStream.Close();
}
I have not tested this snippet, so please verify indexes especially.
If you upload a file at once it may happen that the tcp/ip stack does get timeouts due to preparing the data to upload.
Further, this way you do not create big memory objects when you upload files with hundreds of mb (which would end up completely in memory at your solution)
Related
I'm using the following code in order to stream a video on the aspx page and showing to the user.
But I get the following error in running the code:
CS0029: Cannot implicitly convert type 'System.IO.Stream' to 'Stream'
May you please help me whats wrong with this code?
using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Web;
public partial class Stream : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
Stream stream = null;
int bytesToRead = 10000;
byte[] buffer = new Byte[bytesToRead];
try
{
HttpWebRequest fileReq = (HttpWebRequest)HttpWebRequest.Create("the url");
HttpWebResponse fileResp = (HttpWebResponse)fileReq.GetResponse();
if (fileReq.ContentLength > 0)
fileResp.ContentLength = fileReq.ContentLength;
stream = fileResp.GetResponseStream();
var resp = HttpContext.Current.Response;
resp.ContentType = "application/octet-stream";
resp.AddHeader("Content-Disposition", "attachment; filename=\"" + "ali" + "\"");
resp.AddHeader("Content-Length", fileResp.ContentLength.ToString());
int length;
do
{
if (resp.IsClientConnected)
{
length = stream.Read(buffer, 0, bytesToRead);
resp.OutputStream.Write(buffer, 0, length);
resp.Flush();
buffer = new Byte[bytesToRead];
}
else
{
length = -1;
}
} while (length > 0);
}
finally
{
if (stream != null)
{
stream.Close();
}
}
}
}
stream = fileResp.GetResponseStream();
This line is throwing compilation error I believe.
Your stream referance is of type Class stream. Please use
System.IO.Stream stream = null;
while initializing.. :)
Your class name is the same as the type returned by .GetResponseStream(). When you do Stream stream it defines stream as an object or your class not of System.IO.Stream. You have a couple of options. One is you can declare it as following:
System.IO.Stream stream = null;
Another is define your own for it as a using statement outside the namespace
using IOStream = System.IO.Stream;
Then you can use IOStream everywhere you need like this:
IOStream stream = null;
stream = fileResp.GetResponseStream(); is wrong
The declare of the function as follow
public override Stream GetResponseStream()
https://msdn.microsoft.com/en-us/library/system.net.httpwebresponse.getresponsestream(v=vs.110).aspx
you can change the code to (NetworkStream) fileResp.GetResponseStream();
To explain it in more detail, the Stream is an abstract class, it should initialize from its child class, in your case, you give an abstract object with abstract class type which violate this rule.
I'm using google speech recognition for speech to text of a audio file. Response i get as output shows only {"result":[]}. I don't see any output result.
i have picked my code from How to use google speech recognition api in c#? and Google speech to text API in C# . i have tried almost every answer in above links still i am getting the error.
My code is:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Net;
using System.IO;
using Newtonsoft.Json;
namespace google_api
{
class Program
{
static void Main(string[] args)
{
try
{
FileStream fileStream = File.OpenRead("good-morning-google.flac");
MemoryStream memoryStream = new MemoryStream();
memoryStream.SetLength(fileStream.Length);
fileStream.Read(memoryStream.GetBuffer(), 0, (int)fileStream.Length);
byte[] BA_AudioFile = memoryStream.GetBuffer();
HttpWebRequest _HWR_SpeechToText = null;
_HWR_SpeechToText =
(HttpWebRequest)HttpWebRequest.Create(
"https://www.google.com/speech-api/v2/recognize?output=json&lang=en-us&key=mykey");
_HWR_SpeechToText.Credentials = CredentialCache.DefaultCredentials;
_HWR_SpeechToText.Method = "POST";
_HWR_SpeechToText.ContentType = "audio/x-flac; rate=44100";
_HWR_SpeechToText.ContentLength = BA_AudioFile.Length;
Stream stream = _HWR_SpeechToText.GetRequestStream();
stream.Write(BA_AudioFile, 0, BA_AudioFile.Length);
stream.Close();
HttpWebResponse HWR_Response = (HttpWebResponse)_HWR_SpeechToText.GetResponse();
if (HWR_Response.StatusCode == HttpStatusCode.OK)
{
StreamReader SR_Response = new StreamReader(HWR_Response.GetResponseStream());
Console.WriteLine(SR_Response.ReadToEnd()+SR_Response.ToString());
string responseFromServer = (SR_Response.ReadToEnd());
String[] jsons = responseFromServer.Split('\n');
String text = "";
foreach (String j in jsons)
{
dynamic jsonObject = JsonConvert.DeserializeObject(j);
if (jsonObject == null || jsonObject.result.Count <= 0)
{
continue;
}
text = jsonObject.result[0].alternative[0].transcript;
}
Console.WriteLine("MESSAGE : "+text);
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
Console.ReadLine();
}
}
}
I am currently transferring a file from a c++ server using the following c# code
using System;
using System.Collections.Generic;
using System.Text;
using System.Threading.Tasks;
using System.Net;
using System.IO;
namespace WebRequestTest
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Making small binary request: 1MB");
makeRequest("http://server:8081/sm_binary", "sm_copied");
Console.WriteLine("Making medium binary request: 10MB");
makeRequest("http://server:8081/md_binary", "md_copied");
Console.WriteLine("Making large binary request: 100MB");
makeRequest("http://server:8081/lg_binary", "lg_copied");
Console.WriteLine("Making huge binary request: 2GB");
makeRequest("http://server:8081/hg_binary", "hg_copied");
while (true) { }
}
static void makeRequest(string url, string filename)
{
Console.WriteLine("Starting request: " + DateTime.Now.ToString("HH-mm-ss-fff"));
WebRequest request = WebRequest.Create(url);
WebResponse response = request.GetResponse();
Stream data = response.GetResponseStream();
Console.WriteLine("Starting file write: " + DateTime.Now.ToString("HH-mm-ss-fff"));
using (System.IO.FileStream fs = System.IO.File.Create("./Binaries/" + filename))
{
byte[] buffer = new byte[8 * 1024];
int len;
while ((len = data.Read(buffer, 0, buffer.Length)) > 0)
{
fs.Write(buffer, 0, len);
}
fs.Close();
}
Console.WriteLine("Process is done: " + DateTime.Now.ToString("HH-mm-ss-fff") + "\n");
}
}
}
I am getting reasonable times on the first three file transfers, with the 100MB transfer and write taking about 43 seconds. The 2GB file transfer is taking obnoxiously long at about 37 minutes compared to the expected ~15 minutes. I would like to make sure that the receiving side of this code is not causing the slowdown and I am wondering if there is a more efficient way to write these files to disk.
You may try using the WebClient class instead to obtain the raw file rather than a streamed block:
WebClient client = new WebClient();
client.DownloadFile(fileUri, filePathOnHardDrive);
I'm trying to download a .torrent file (not the contents of the torrent itself) in my .NET application.
Using the following code works for other files, but not .torrent. The resulting files is about 1-3kb smaller than if I download the file via a browser. When opening it in a torrent client, it says the torrent is corrupt.
WebClient web = new WebClient();
web.Headers.Add("Content-Type", "application/x-bittorrent");
web.DownloadFile("http://kat.ph/torrents/linux-mint-12-gnome-mate-dvd-64-bit-t6008958/", "test.torrent");
Opening the URL in a browser results in the file being downloaded correctly.
Any ideas as to why this would happen? Are there any good alternatives to WebClient that would download the file correctly?
EDIT: I've tried this as well as WebClient, and it results in the same thing:
private void DownloadFile(string url, string file)
{
byte[] result;
byte[] buffer = new byte[4096];
WebRequest wr = WebRequest.Create(url);
wr.ContentType = "application/x-bittorrent";
using (WebResponse response = wr.GetResponse())
{
using (Stream responseStream = response.GetResponseStream())
{
using (MemoryStream memoryStream = new MemoryStream())
{
int count = 0;
do
{
count = responseStream.Read(buffer, 0, buffer.Length);
memoryStream.Write(buffer, 0, count);
} while (count != 0);
result = memoryStream.ToArray();
using (BinaryWriter writer = new BinaryWriter(new FileStream(file, FileMode.Create)))
{
writer.Write(result);
}
}
}
}
}
The problem that server returns content compressed by gzip and you download this compressed content to file. For such cases you should check the "Content-Encoding" header and use proper stream reader to decompress the source.
I modified your function to handle gzipped content:
private void DownloadFile(string url, string file)
{
byte[] result;
byte[] buffer = new byte[4096];
WebRequest wr = WebRequest.Create(url);
wr.ContentType = "application/x-bittorrent";
using (WebResponse response = wr.GetResponse())
{
bool gzip = response.Headers["Content-Encoding"] == "gzip";
var responseStream = gzip
? new GZipStream(response.GetResponseStream(), CompressionMode.Decompress)
: response.GetResponseStream();
using (MemoryStream memoryStream = new MemoryStream())
{
int count = 0;
do
{
count = responseStream.Read(buffer, 0, buffer.Length);
memoryStream.Write(buffer, 0, count);
} while (count != 0);
result = memoryStream.ToArray();
using (BinaryWriter writer = new BinaryWriter(new FileStream(file, FileMode.Create)))
{
writer.Write(result);
}
}
}
}
As part of an upcoming project at my university, I need to write a client that downloads a media file from a server and writes it to the local disk. Since these files can be very large, I need to implement partial download and serialization in order to avoid excessive memory use.
What I came up with:
namespace PartialDownloadTester
{
using System;
using System.Diagnostics.Contracts;
using System.IO;
using System.Net;
using System.Text;
public class DownloadClient
{
public static void Main(string[] args)
{
var dlc = new DownloadClient(args[0], args[1], args[2]);
dlc.DownloadAndSaveToDisk();
Console.ReadLine();
}
private WebRequest request;
// directory of file
private string dir;
// full file identifier
private string filePath;
public DownloadClient(string uri, string fileName, string fileType)
{
this.request = WebRequest.Create(uri);
this.request.Method = "GET";
var sb = new StringBuilder();
sb.Append("C:\\testdata\\DownloadedData\\");
this.dir = sb.ToString();
sb.Append(fileName + "." + fileType);
this.filePath = sb.ToString();
}
public void DownloadAndSaveToDisk()
{
// make sure directory exists
this.CreateDir();
var response = (HttpWebResponse)request.GetResponse();
Console.WriteLine("Content length: " + response.ContentLength);
var rStream = response.GetResponseStream();
int bytesRead = -1;
do
{
var buf = new byte[2048];
bytesRead = rStream.Read(buf, 0, buf.Length);
rStream.Flush();
this.SerializeFileChunk(buf);
}
while (bytesRead != 0);
}
private void CreateDir()
{
if (!Directory.Exists(dir))
{
Directory.CreateDirectory(dir);
}
}
private void SerializeFileChunk(byte[] bytes)
{
Contract.Requires(!Object.ReferenceEquals(bytes, null));
FileStream fs = File.Open(filePath, FileMode.Append);
fs.Write(bytes, 0, bytes.Length);
fs.Flush();
fs.Close();
}
}
}
For testing purposes, I've used the following parameters:
"http://itu.dk/people/janv/mufc_abc.jpg" "mufc_abc" "jpg"
However, the picture is incomplete (only the first ~10% look right) even though the content length prints 63780 which is the actual size of the image.
So my questions are:
Is this the right way to go for partial download and serialization or is there a better/easier approach?
Is the full content of the response stream stored in client memory? If this is the case, do I need to use HttpWebRequest.AddRange to partially download data from the server in order to conserve my client's memory?
How come the serialization fails and I get a broken image?
Do I introduce a lot of overhead when I use the FileMode.Append? (msdn states that this option "seeks to the end of the file")
Thanks in advance
You could definitely simplify your code using a WebClient:
class Program
{
static void Main()
{
DownloadClient("http://itu.dk/people/janv/mufc_abc.jpg", "mufc_abc.jpg");
}
public static void DownloadClient(string uri, string fileName)
{
using (var client = new WebClient())
{
using (var stream = client.OpenRead(uri))
{
// work with chunks of 2KB => adjust if necessary
const int chunkSize = 2048;
var buffer = new byte[chunkSize];
using (var output = File.OpenWrite(fileName))
{
int bytesRead;
while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, bytesRead);
}
}
}
}
}
}
Notice how I am writing only the number of bytes I have actually read from the socket to the output file and not the entire 2KB buffer.
I don't know if this is the source of the problem, however I would change the loop like this
const int ChunkSize = 2048;
var buf = new byte[ChunkSize];
var rStream = response.GetResponseStream();
do {
int bytesRead = rStream.Read(buf, 0, ChunkSize);
if (bytesRead > 0) {
this.SerializeFileChunk(buf, bytesRead);
}
} while (bytesRead == ChunkSize);
The serialize method would get an additional argument
private void SerializeFileChunk(byte[] bytes, int numBytes)
and then write the right number of bytes
fs.Write(bytes, 0, numBytes);
UPDATE:
I do not see the need for closing and reopening the file each time. I also would use the using statement, which closes the resources, even if an exception should occur. The using statement calls the Dispose() method of the resource at the end, which in turn calls Close() in the case of file streams. using can be applied to all types implementing IDisposable.
var buf = new byte[2048];
using (var rStream = response.GetResponseStream()) {
using (FileStream fs = File.Open(filePath, FileMode.Append)) {
do {
bytesRead = rStream.Read(buf, 0, buf.Length);
fs.Write(bytes, 0, bytesRead);
} while (...);
}
}
The using statement does something like this
{
var rStream = response.GetResponseStream();
try
{
// do some work with rStream here.
} finally {
if (rStream != null) {
rStream.Dispose();
}
}
}
Here is the solution from Microsoft: http://support.microsoft.com/kb/812406
Updated 2021-03-16: seems the original article is not available now. Here is the archived one: https://mskb.pkisolutions.com/kb/812406