EDIT: Filezilla caused the problem, when i download files back from server it added new lines. I'm sorry for confusion.
This method upload files to ftp server and it's work fine, but in text files uploaded to server blank lines appear after every line("cr lf" appear), for example:
File:
First line
Second line
Third line
Uploaded file:
First line
Second line
Third line
Origin and uploaded files accordingly have different sizes, non-text files are the same.
Code:
private void sendFile(string In, string Out)
{
FtpWebRequest request = (FtpWebRequest) WebRequest.Create("ftp://domain//" + Out);
request.Method = WebRequestMethods.Ftp.UploadFile;
request.Credentials = new NetworkCredential("username", "password");
FileStream sourceStream = new FileStream(In, FileMode.Open, FileAccess.Read, FileShare.Read);
byte[] fileContents = new byte[sourceStream.Length];
sourceStream.Read(fileContents, 0, (int) sourceStream.Length);
sorceStream.Close();
request.ContentLength = fileContents.Length;
Stream requestStream = request.GetRequestStream();
requestStream.Write(fileContents, 0, fileContents.Length);
requestStream.Close();
}
How can i fix this?
EDIT: As the answer below doesn't seem to have helped (but I'm leaving it there for posterity as it shows better code) here are the next diagnostics steps I'd check:
How are you viewing the files? If at all possible, get onto the server directly rather than fetching the files again via a web browser or whatever.
What's the type of FTP server you're connecting to? Maybe there's a known issue.
Have you tried looking at what's actually being sent via Wireshark?
Have you tried sending the same files via a normal FTP client?
You should set FtpWebRequest.UseBinary to true in order to preserve the exact file contents. Otherwise the two systems will try to figure out line endings themselves, changing line terminators as they see fit. I very rarely think that's a good idea. (EDIT: UseBinary is actually true by default, but this sounds like the kind of problem introduced by using text mode... it certainly does no harm to make this explicit.)
Additionally:
You should be disposing of your FileStream via a using statement
You should be disposing of the request stream via a using statement
You should be taking note of the result of Stream.Read - it needn't always read the whole of the requested data in one go
You can either use File.ReadAllBytes to simply read the complete file data in one go, or use Stream.CopyTo (if you're using .NET 4) to copy the FileStream to the request stream (which won't set the content length, of course; I don't know whether this is a problem)
You're never calling GetResponse; it's unclear exactly what happens if you never fetch the response of an FtpWebRequest
Your parameter names don't match .NET naming conventions, and aren't very descriptive
So I would probably use:
private void SendFile(string inputFile, string outputPath)
{
FtpWebRequest request = (FtpWebRequest) WebRequest.Create
("ftp://domain//" + outputPath);
request.Method = WebRequestMethods.Ftp.UploadFile;
request.UseBinary = true;
request.Credentials = new NetworkCredential("username", "password");
byte[] fileContents = File.ReadAllBytes(inputFile);
request.ContentLength = fileContents.Length;
using (Stream requestStream = request.GetRequestStream())
{
requestStream.Write(fileContents, 0, fileContents.Length);
}
// This *may* be necessary in order to validate that everything has happened
using (WebResponse response = request.GetResponse())
{
}
}
Its strange. I face the same problem and I was unable to fix it until I did not provide an extension in file. For Example if my file name was
abcfile
then I make it abcfile.dat and after that it shows me the uploaded file as actual file. I again upload file with abcfile.txt but this time again empty line problem appear in my uploaded file.
I suggest that you must provide extension to your file any except .txt.
The system that you're sending to uses different line endings to what your system uses. I can assume, because you get an extra line, that you're on Windows, and it uses CRLF endings. The system you're sending to recognises CR and LF as separate endings, so you get the extra lines.
For text, truncate the LF or the CR, see what happens. I have no clue about the differing file sizes.
In the top menu of FileZilla, set:
Transfer menu > Transfer type > binary
In the top menu of FileZilla, set:
Transfer menu > Transfer type > binary
It's working for me.
Related
I'm downloading in image from web to save it locally. It works great with any other image formats but it this method below fails with an argument exception when I try to read a WebP image.
private static Image GetImage(string url)
{
try
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
return Image.FromStream(response.GetResponseStream());
}
catch
{
return null;
}
}
How do you read .webp images in C#?
I found this other question that allows for converting between types but I do not want to do that WebP library for C#
Reason I'm not wanting to do it is because I think it might lose some quality. Besides, I want to know why is this not working.
The base class libraries won't help you to deal with WebP images. However, if you only want to save the received file to the disk, you don't have to even know that you are dealing with a WebP images. You can simply treat the received data as a binary blob and dump it to a file, for example using Stream.CopyTo and a FileStream.
The Content-Type HTTP header will give you the mime type of the file you're downloading, and the Content-Disposition header can provide you with a filename and extension (though you might have to do some parsing). You can access those using HttpWebResponse.ContentType and HttpWebResponse.Headers["Content-Disposition"].
#Trillian nailed it. Here is a code snippet for what I did based on his suggestion. Wanted to add code so not posting this as a comment.
To get just the image file extension, you can do this
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string fileExt = response.ContentType.Replace("image/", string.Empty);
To get the file name with extension, you can do the following and the do parsing like I did above. It just has some more data in it.
response.Headers["Content-Disposition"];
Once you have you file name you want to save as, create a file stream and copy the response stream into it.
FileStream fs = new FileStream(targetPath + fileName, FileMode.Create);
response.GetResponseStream().CopyTo(fs);
Assuming you app has access to the destination, image should get saved. Make sure to add try catch and handle exceptions properly. Also note that FileMode.Create will overwrite if the file already exists!
I have a small console app that scrapes some data and uploads it to an FTP directory using the FtpWebRequest object - very typical.
I had it working successfully while I was stepping through it with the debugger. Files were being written to the remote server with the correct data. Then when I took the breakpoints out and let it rip, it created the file with no data, then hung until it finally timed out. No exception other than the timeout was returned. I put the breakpoint back in and sure enough it worked perfectly.
I figured that the act of debugging was causing a pause and put in a Thread.Sleep(10), and, again, worked perfectly.
I'd be happy with just an explanation for this (as long as it's not terrible practice), but suggestions on how to do it better would also be appreciated. I'll note that I only sparingly use 3rd Party controls.
EDIT: I should also mention that I don't want to have to create a physical file, I'd rather just write the Stream directly.
Here's the relevant code:
filePathAndName = new Uri(String.Format("{0}{1}_{2}.txt", Properties.Settings.Default.QTMFTPDirectory,
fileNamePrefix, DateTime.Now.ToString("MMddyyHHmmss")));
fileContents = Encoding.UTF8.GetBytes(csv);
request = (FtpWebRequest)WebRequest.Create(filePathAndName);
request.Method = WebRequestMethods.Ftp.UploadFile;
request.Credentials = new NetworkCredential(Properties.Settings.Default.FTPUserName, Properties.Settings.Default.FTPPassword);
request.ContentLength = fileContents.Length;
request.KeepAlive = false;
try
{
using (Stream reqStream = request.GetRequestStream())
{
reqStream.Write(fileContents, 0, fileContents.Length);
System.Threading.Thread.Sleep(10); // Won't work without this.
}
}
catch (Exception ex)
{
LogError(ex);
}
Thanks!
Try manually closing it.
reqStream.Close(); // Close
and if that does not help, you can try getting the response from the server.
GetResponse();
Source:
http://msdn.microsoft.com/en-us/library/system.net.ftpwebrequest.getrequeststream(v=vs.110).aspx
Edit:
I know, the using keyword sometimes does not close a connection correctly.
My issue is this:
I am generating and uploading a SQL file using ASP.NET, but after the file is saved to the FTP server, characters like ü are changed to &uul;, ø to ø and so on... How can I prevent this from happening? I don't want the file to be formatted with ASCII code, but with UTF-8.
The code that generates and uploads the file looks like this:
//request = the object to be made an request out of.
Stream requestStream = request.GetReguestStream();
var encoding = new UTF8Encoding();
//fileContent is the string to be saved in the file
byte[] buffer = encoding.GetBytes(fileContent);
requestStream.Write(buffer, 0, buffer.Length);
requestStream.Close();
As you can see I've tried to use the System.Text.UTF8Encoding, but it doesn't work.
Remember, with streams you can almost always wrap the streams as necessary. If you want to write UTF-8 encoded content you wrap the request stream in a StreamWriter with the correct encoding:
using (Stream requestStream = request.GetRequestStream())
using (StreamWriter writer = new StreamWriter(requestStream, Encoding.UTF8)) {
writer.Write(fileContent);
}
Since you say you're uploading to a web service be sure to set your content encoding as well. Since you haven't posted where the request object comes from, I'll assume it's a normal HttpWebRequest.
With a HttpWebRequest you would tell the server what the content encoding is by using the ContentType property.
request.ContentType = "text/plain;charset=utf-8";
As others have mentioned, though, the FTP transfer itself may be breaking it too. If you can, make sure it's transferred in binary mode, not ASCII mode.
Put it in debug and look at what gets put in 'buffer' after encoding.GetBytes() is called. This will verify if it's the rx side causing it.
I'm trying to download remote file using C# (ASP.NET). The problem is when I browse to the file download URL - it downloads perfectly. When I try the WebClient.DownloadData(url) I get "no data to show" response.
If I browse using the built-in VS2010 browser I still get this "error" message.
The file link is: http://www.tase.co.il/TASE/Pages/Export.aspx?sn=he-IL_ds&enumTblType=AllSecurities&Columns=he-IL_Columns&Titles=he-IL_Titles&TblId=0&ExportType=3
(CSV file)
How can the file be downloaded? any ideas?
Many Thanks
This isn't a download problem on your side, that page is referencing something, probably from session...so you would need to login to the site, get a session, basically go through whatever steps you normally would in route to getting the file so it's generated correctly/available.
If I had to take a guess, I'm thinking they do the export to the file system when you ask for it, so it's a temp file somewhere...not something you can just grab, so unless you went through the step that created it (doesn't seem this Export.aspx page does it), it won't be there, and their error message for this is "No data to show".
There's also the possibility they're explicitly blocking you to prevent any kind of bots/leeching from happening...basically preventing exactly what you're trying to do.
Many sites will block downloads based on the Referer or User-Agent header.
Watch a "working" download with
Fiddler
Watch your code's
download with Fiddler
Compare the
two requests
Update your request
to match the "working" case.
I solved the problem by obtaining the remote file as a stream. From that point I was able to read/write the stream to the response.
//The absolute path to your file
string remoteFile = "http://my.cloudfront.net/videos/vehicle_english_v3.mp4";
//The name of the file you want the user to see when they download the file
int pos = remoteFile.LastIndexOf("/") + 1;
string fileName = remoteFile.Substring(pos, remoteFile.Length - pos);
//Obtain your file as a stream
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(remoteFile);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream fileStream = response.GetResponseStream();
//Write the stream to the response
if (fileStream != null && fileStream.CanRead) {
context.Response.AddHeader("Content-Disposition", "attachment; filename=\"" + fileName + "\"");
context.Response.ContentType = "application/octet-stream";
context.Response.ClearContent();
fileStream.CopyTo(context.Response.OutputStream);
}
I have to interface with a slightly archaic system that doesn't use webservices. In order to send data to this system, I need to post an XML document into a form on the other system's website. This XML document can get very large so I would like to compress it.
The other system sits on IIS and I use C# my end. I could of course implement something that compresses the data before posting it, but that requires the other system to change so it can decompress the data. I would like to avoid changing the other system as I don't own it.
I have heard vague things about enabling compression / http 1.1 in IIS and the browser but I have no idea how to translate that to my program. Basically, is there some property I can set in my program that will make my program automatically compress the data that it is sending to IIS and for IIS to seamlessly decompress it so the receiving app doesn't even know the difference?
Here is some sample code to show roughly what I am doing;
private static void demo()
{
Stream myRequestStream = null;
Stream myResponseStream = null;
HttpWebRequest myWebRequest = (HttpWebRequest)System.Net
.WebRequest.Create("http://example.com");
byte[] bytMessage = null;
bytMessage = Encoding.ASCII.GetBytes("data=xyz");
myWebRequest.ContentLength = bytMessage.Length;
myWebRequest.Method = "POST";
// Set the content type as form so that the data
// will be posted as form
myWebRequest.ContentType = "application/x-www-form-urlencoded";
//Get Stream object
myRequestStream = myWebRequest.GetRequestStream();
//Writes a sequence of bytes to the current stream
myRequestStream.Write(bytMessage, 0, bytMessage.Length);
//Close stream
myRequestStream.Close();
WebResponse myWebResponse = myWebRequest.GetResponse();
myResponseStream = myWebResponse.GetResponseStream();
}
"data=xyz" will actually be "data=[a several MB XML document]".
I am aware that this question may ultimately fall under the non-programming banner if this is achievable through non-programmatic means so apologies in advance.
I see no way to compress the data on one side and receiving them uncompressed on the other side without actively uncompressing the data..
No idea if this will work since all of the examples I could find were for download, but you could try using gzip to compress the data, then set the Content-Encoding header on the outgoing message to gzip. I believe that the Length should be the length of the zipped message, although you may want to play with making it the length of the unencoded message if that doesn't work.
Good luck.
EDIT I think the issue is whether the ISAPI filter that supports compression is ever/always/configurably invoked on upload. I couldn't find an answer to that so I suspect that the answer is never, but you won't know until you try (or find the answer that eluded me).