General Info
I'm still in the process of learning C#. To help myself out, I'm trying to create a program that will automatically synchronise all of my local projects with a folder on my FTP server. This so that whether I'm at school or at home, I always have the same projects available to me.
I know there are programs like Dropbox that already do this for me, but I figured creating something like that myself will teach me a lot along the way.
The problem
My first step towards my goal was to just download all files, subdirectories and subfiles from my FTP server. I've managed to download all files from a directory with the code below. However, my code only lists the folder names and the files in the main directory. Subfolders and subfiles are never returned and never downloaded. Aside from that, the server returns a 550 error because I'm trying to download the folders as if they are files. I've been on this for 4+ hours now, but I just can't find anything on how to fix these problems and make it work. Therefor I'm hoping you guys will help me out :)
Code
public string[] GetFileList()
{
string[] downloadFiles;
StringBuilder result = new StringBuilder();
WebResponse response = null;
StreamReader reader = null;
try
{
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(url);
request.UseBinary = true;
request.Method = WebRequestMethods.Ftp.ListDirectory;
request.Credentials = new NetworkCredential(ftpUserName, ftpPassWord);
request.KeepAlive = false;
request.UsePassive = false;
response = request.GetResponse();
reader = new StreamReader(response.GetResponseStream());
string line = reader.ReadLine();
while (line != null)
{
result.Append(line);
result.Append("\n");
line = reader.ReadLine();
}
result.Remove(result.ToString().LastIndexOf('\n'), 1);
return result.ToString().Split('\n');
}
catch (Exception ex)
{
if (reader != null)
{
reader.Close();
}
if (response != null)
{
response.Close();
}
downloadFiles = null;
return downloadFiles;
}
}
private void Download(string file)
{
try
{
string uri = url + "/" + file;
Uri serverUri = new Uri(uri);
if (serverUri.Scheme != Uri.UriSchemeFtp)
{
return;
}
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(url + "/" + file);
request.UseBinary = true;
request.Method = WebRequestMethods.Ftp.DownloadFile;
request.Credentials = new NetworkCredential(ftpUserName, ftpPassWord);
request.KeepAlive = false;
request.UsePassive = false;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
FileStream writeStream = new FileStream(localDestnDir + "\\" + file, FileMode.Create);
int Length = 2048;
Byte[] buffer = new Byte[Length];
int bytesRead = responseStream.Read(buffer, 0, Length);
while (bytesRead > 0)
{
writeStream.Write(buffer, 0, bytesRead);
bytesRead = responseStream.Read(buffer, 0, Length);
}
writeStream.Close();
response.Close();
}
catch (WebException wEx)
{
MessageBox.Show(wEx.Message, "Download Error");
}
catch (Exception ex)
{
MessageBox.Show(ex.Message, "Download Error");
}
}
The FtpWebRequest does not have any explicit support for recursive file operations (including downloads). You have to implement the recursion yourself:
List the remote directory
Iterate the entries, downloading files and recursing into subdirectories (listing them again, etc.)
Tricky part is to identify files from subdirectories. There's no way to do that in a portable way with the FtpWebRequest. The FtpWebRequest unfortunately does not support the MLSD command, which is the only portable way to retrieve directory listing with file attributes in FTP protocol. See also Checking if object on FTP server is file or directory.
Your options are:
Do an operation on a file name that is certain to fail for file and succeeds for directories (or vice versa). I.e. you can try to download the "name". If that succeeds, it's a file, if that fails, it's a directory.
You may be lucky and in your specific case, you can tell a file from a directory by a file name (i.e. all your files have an extension, while subdirectories do not)
You use a long directory listing (LIST command = ListDirectoryDetails method) and try to parse a server-specific listing. Many FTP servers use *nix-style listing, where you identify a directory by the d at the very beginning of the entry. But many servers use a different format. The following example uses this approach (assuming the *nix format)
void DownloadFtpDirectory(
string url, NetworkCredential credentials, string localPath)
{
FtpWebRequest listRequest = (FtpWebRequest)WebRequest.Create(url);
listRequest.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
listRequest.Credentials = credentials;
List<string> lines = new List<string>();
using (var listResponse = (FtpWebResponse)listRequest.GetResponse())
using (Stream listStream = listResponse.GetResponseStream())
using (var listReader = new StreamReader(listStream))
{
while (!listReader.EndOfStream)
{
lines.Add(listReader.ReadLine());
}
}
foreach (string line in lines)
{
string[] tokens =
line.Split(new[] { ' ' }, 9, StringSplitOptions.RemoveEmptyEntries);
string name = tokens[8];
string permissions = tokens[0];
string localFilePath = Path.Combine(localPath, name);
string fileUrl = url + name;
if (permissions[0] == 'd')
{
if (!Directory.Exists(localFilePath))
{
Directory.CreateDirectory(localFilePath);
}
DownloadFtpDirectory(fileUrl + "/", credentials, localFilePath);
}
else
{
FtpWebRequest downloadRequest =
(FtpWebRequest)WebRequest.Create(fileUrl);
downloadRequest.Method = WebRequestMethods.Ftp.DownloadFile;
downloadRequest.Credentials = credentials;
using (FtpWebResponse downloadResponse =
(FtpWebResponse)downloadRequest.GetResponse())
using (Stream sourceStream = downloadResponse.GetResponseStream())
using (Stream targetStream = File.Create(localFilePath))
{
byte[] buffer = new byte[10240];
int read;
while ((read = sourceStream.Read(buffer, 0, buffer.Length)) > 0)
{
targetStream.Write(buffer, 0, read);
}
}
}
}
}
Use the function like:
NetworkCredential credentials = new NetworkCredential("user", "mypassword");
string url = "ftp://ftp.example.com/directory/to/download/";
DownloadFtpDirectory(url, credentials, #"C:\target\directory");
If you want to avoid troubles with parsing the server-specific directory listing formats, use a 3rd party library that supports the MLSD command and/or parsing various LIST listing formats; and recursive downloads.
For example with WinSCP .NET assembly you can download whole directory with a single call to the Session.GetFiles:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "ftp.example.com",
UserName = "user",
Password = "mypassword",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// Download files
session.GetFiles("/directory/to/download/*", #"C:\target\directory\*").Check();
}
Internally, WinSCP uses the MLSD command, if supported by the server. If not, it uses the LIST command and supports dozens of different listing formats.
The Session.GetFiles method is recursive by default.
In most cases, Session.GetFilesToDirectory is a more straightforward equivalent to Session.GetFiles:
session.GetFiles("/directory/to/download", #"C:\target\directory").Check();
(I'm the author of WinSCP)
Related
General Info
I'm still in the process of learning C#. To help myself out, I'm trying to create a program that will automatically synchronise all of my local projects with a folder on my FTP server. This so that whether I'm at school or at home, I always have the same projects available to me.
I know there are programs like Dropbox that already do this for me, but I figured creating something like that myself will teach me a lot along the way.
The problem
My first step towards my goal was to just download all files, subdirectories and subfiles from my FTP server. I've managed to download all files from a directory with the code below. However, my code only lists the folder names and the files in the main directory. Subfolders and subfiles are never returned and never downloaded. Aside from that, the server returns a 550 error because I'm trying to download the folders as if they are files. I've been on this for 4+ hours now, but I just can't find anything on how to fix these problems and make it work. Therefor I'm hoping you guys will help me out :)
Code
public string[] GetFileList()
{
string[] downloadFiles;
StringBuilder result = new StringBuilder();
WebResponse response = null;
StreamReader reader = null;
try
{
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(url);
request.UseBinary = true;
request.Method = WebRequestMethods.Ftp.ListDirectory;
request.Credentials = new NetworkCredential(ftpUserName, ftpPassWord);
request.KeepAlive = false;
request.UsePassive = false;
response = request.GetResponse();
reader = new StreamReader(response.GetResponseStream());
string line = reader.ReadLine();
while (line != null)
{
result.Append(line);
result.Append("\n");
line = reader.ReadLine();
}
result.Remove(result.ToString().LastIndexOf('\n'), 1);
return result.ToString().Split('\n');
}
catch (Exception ex)
{
if (reader != null)
{
reader.Close();
}
if (response != null)
{
response.Close();
}
downloadFiles = null;
return downloadFiles;
}
}
private void Download(string file)
{
try
{
string uri = url + "/" + file;
Uri serverUri = new Uri(uri);
if (serverUri.Scheme != Uri.UriSchemeFtp)
{
return;
}
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(url + "/" + file);
request.UseBinary = true;
request.Method = WebRequestMethods.Ftp.DownloadFile;
request.Credentials = new NetworkCredential(ftpUserName, ftpPassWord);
request.KeepAlive = false;
request.UsePassive = false;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
FileStream writeStream = new FileStream(localDestnDir + "\\" + file, FileMode.Create);
int Length = 2048;
Byte[] buffer = new Byte[Length];
int bytesRead = responseStream.Read(buffer, 0, Length);
while (bytesRead > 0)
{
writeStream.Write(buffer, 0, bytesRead);
bytesRead = responseStream.Read(buffer, 0, Length);
}
writeStream.Close();
response.Close();
}
catch (WebException wEx)
{
MessageBox.Show(wEx.Message, "Download Error");
}
catch (Exception ex)
{
MessageBox.Show(ex.Message, "Download Error");
}
}
The FtpWebRequest does not have any explicit support for recursive file operations (including downloads). You have to implement the recursion yourself:
List the remote directory
Iterate the entries, downloading files and recursing into subdirectories (listing them again, etc.)
Tricky part is to identify files from subdirectories. There's no way to do that in a portable way with the FtpWebRequest. The FtpWebRequest unfortunately does not support the MLSD command, which is the only portable way to retrieve directory listing with file attributes in FTP protocol. See also Checking if object on FTP server is file or directory.
Your options are:
Do an operation on a file name that is certain to fail for file and succeeds for directories (or vice versa). I.e. you can try to download the "name". If that succeeds, it's a file, if that fails, it's a directory.
You may be lucky and in your specific case, you can tell a file from a directory by a file name (i.e. all your files have an extension, while subdirectories do not)
You use a long directory listing (LIST command = ListDirectoryDetails method) and try to parse a server-specific listing. Many FTP servers use *nix-style listing, where you identify a directory by the d at the very beginning of the entry. But many servers use a different format. The following example uses this approach (assuming the *nix format)
void DownloadFtpDirectory(
string url, NetworkCredential credentials, string localPath)
{
FtpWebRequest listRequest = (FtpWebRequest)WebRequest.Create(url);
listRequest.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
listRequest.Credentials = credentials;
List<string> lines = new List<string>();
using (var listResponse = (FtpWebResponse)listRequest.GetResponse())
using (Stream listStream = listResponse.GetResponseStream())
using (var listReader = new StreamReader(listStream))
{
while (!listReader.EndOfStream)
{
lines.Add(listReader.ReadLine());
}
}
foreach (string line in lines)
{
string[] tokens =
line.Split(new[] { ' ' }, 9, StringSplitOptions.RemoveEmptyEntries);
string name = tokens[8];
string permissions = tokens[0];
string localFilePath = Path.Combine(localPath, name);
string fileUrl = url + name;
if (permissions[0] == 'd')
{
if (!Directory.Exists(localFilePath))
{
Directory.CreateDirectory(localFilePath);
}
DownloadFtpDirectory(fileUrl + "/", credentials, localFilePath);
}
else
{
FtpWebRequest downloadRequest =
(FtpWebRequest)WebRequest.Create(fileUrl);
downloadRequest.Method = WebRequestMethods.Ftp.DownloadFile;
downloadRequest.Credentials = credentials;
using (FtpWebResponse downloadResponse =
(FtpWebResponse)downloadRequest.GetResponse())
using (Stream sourceStream = downloadResponse.GetResponseStream())
using (Stream targetStream = File.Create(localFilePath))
{
byte[] buffer = new byte[10240];
int read;
while ((read = sourceStream.Read(buffer, 0, buffer.Length)) > 0)
{
targetStream.Write(buffer, 0, read);
}
}
}
}
}
Use the function like:
NetworkCredential credentials = new NetworkCredential("user", "mypassword");
string url = "ftp://ftp.example.com/directory/to/download/";
DownloadFtpDirectory(url, credentials, #"C:\target\directory");
If you want to avoid troubles with parsing the server-specific directory listing formats, use a 3rd party library that supports the MLSD command and/or parsing various LIST listing formats; and recursive downloads.
For example with WinSCP .NET assembly you can download whole directory with a single call to the Session.GetFiles:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "ftp.example.com",
UserName = "user",
Password = "mypassword",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// Download files
session.GetFiles("/directory/to/download/*", #"C:\target\directory\*").Check();
}
Internally, WinSCP uses the MLSD command, if supported by the server. If not, it uses the LIST command and supports dozens of different listing formats.
The Session.GetFiles method is recursive by default.
In most cases, Session.GetFilesToDirectory is a more straightforward equivalent to Session.GetFiles:
session.GetFiles("/directory/to/download", #"C:\target\directory").Check();
(I'm the author of WinSCP)
General Info
I'm still in the process of learning C#. To help myself out, I'm trying to create a program that will automatically synchronise all of my local projects with a folder on my FTP server. This so that whether I'm at school or at home, I always have the same projects available to me.
I know there are programs like Dropbox that already do this for me, but I figured creating something like that myself will teach me a lot along the way.
The problem
My first step towards my goal was to just download all files, subdirectories and subfiles from my FTP server. I've managed to download all files from a directory with the code below. However, my code only lists the folder names and the files in the main directory. Subfolders and subfiles are never returned and never downloaded. Aside from that, the server returns a 550 error because I'm trying to download the folders as if they are files. I've been on this for 4+ hours now, but I just can't find anything on how to fix these problems and make it work. Therefor I'm hoping you guys will help me out :)
Code
public string[] GetFileList()
{
string[] downloadFiles;
StringBuilder result = new StringBuilder();
WebResponse response = null;
StreamReader reader = null;
try
{
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(url);
request.UseBinary = true;
request.Method = WebRequestMethods.Ftp.ListDirectory;
request.Credentials = new NetworkCredential(ftpUserName, ftpPassWord);
request.KeepAlive = false;
request.UsePassive = false;
response = request.GetResponse();
reader = new StreamReader(response.GetResponseStream());
string line = reader.ReadLine();
while (line != null)
{
result.Append(line);
result.Append("\n");
line = reader.ReadLine();
}
result.Remove(result.ToString().LastIndexOf('\n'), 1);
return result.ToString().Split('\n');
}
catch (Exception ex)
{
if (reader != null)
{
reader.Close();
}
if (response != null)
{
response.Close();
}
downloadFiles = null;
return downloadFiles;
}
}
private void Download(string file)
{
try
{
string uri = url + "/" + file;
Uri serverUri = new Uri(uri);
if (serverUri.Scheme != Uri.UriSchemeFtp)
{
return;
}
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(url + "/" + file);
request.UseBinary = true;
request.Method = WebRequestMethods.Ftp.DownloadFile;
request.Credentials = new NetworkCredential(ftpUserName, ftpPassWord);
request.KeepAlive = false;
request.UsePassive = false;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
FileStream writeStream = new FileStream(localDestnDir + "\\" + file, FileMode.Create);
int Length = 2048;
Byte[] buffer = new Byte[Length];
int bytesRead = responseStream.Read(buffer, 0, Length);
while (bytesRead > 0)
{
writeStream.Write(buffer, 0, bytesRead);
bytesRead = responseStream.Read(buffer, 0, Length);
}
writeStream.Close();
response.Close();
}
catch (WebException wEx)
{
MessageBox.Show(wEx.Message, "Download Error");
}
catch (Exception ex)
{
MessageBox.Show(ex.Message, "Download Error");
}
}
The FtpWebRequest does not have any explicit support for recursive file operations (including downloads). You have to implement the recursion yourself:
List the remote directory
Iterate the entries, downloading files and recursing into subdirectories (listing them again, etc.)
Tricky part is to identify files from subdirectories. There's no way to do that in a portable way with the FtpWebRequest. The FtpWebRequest unfortunately does not support the MLSD command, which is the only portable way to retrieve directory listing with file attributes in FTP protocol. See also Checking if object on FTP server is file or directory.
Your options are:
Do an operation on a file name that is certain to fail for file and succeeds for directories (or vice versa). I.e. you can try to download the "name". If that succeeds, it's a file, if that fails, it's a directory.
You may be lucky and in your specific case, you can tell a file from a directory by a file name (i.e. all your files have an extension, while subdirectories do not)
You use a long directory listing (LIST command = ListDirectoryDetails method) and try to parse a server-specific listing. Many FTP servers use *nix-style listing, where you identify a directory by the d at the very beginning of the entry. But many servers use a different format. The following example uses this approach (assuming the *nix format)
void DownloadFtpDirectory(
string url, NetworkCredential credentials, string localPath)
{
FtpWebRequest listRequest = (FtpWebRequest)WebRequest.Create(url);
listRequest.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
listRequest.Credentials = credentials;
List<string> lines = new List<string>();
using (var listResponse = (FtpWebResponse)listRequest.GetResponse())
using (Stream listStream = listResponse.GetResponseStream())
using (var listReader = new StreamReader(listStream))
{
while (!listReader.EndOfStream)
{
lines.Add(listReader.ReadLine());
}
}
foreach (string line in lines)
{
string[] tokens =
line.Split(new[] { ' ' }, 9, StringSplitOptions.RemoveEmptyEntries);
string name = tokens[8];
string permissions = tokens[0];
string localFilePath = Path.Combine(localPath, name);
string fileUrl = url + name;
if (permissions[0] == 'd')
{
if (!Directory.Exists(localFilePath))
{
Directory.CreateDirectory(localFilePath);
}
DownloadFtpDirectory(fileUrl + "/", credentials, localFilePath);
}
else
{
FtpWebRequest downloadRequest =
(FtpWebRequest)WebRequest.Create(fileUrl);
downloadRequest.Method = WebRequestMethods.Ftp.DownloadFile;
downloadRequest.Credentials = credentials;
using (FtpWebResponse downloadResponse =
(FtpWebResponse)downloadRequest.GetResponse())
using (Stream sourceStream = downloadResponse.GetResponseStream())
using (Stream targetStream = File.Create(localFilePath))
{
byte[] buffer = new byte[10240];
int read;
while ((read = sourceStream.Read(buffer, 0, buffer.Length)) > 0)
{
targetStream.Write(buffer, 0, read);
}
}
}
}
}
Use the function like:
NetworkCredential credentials = new NetworkCredential("user", "mypassword");
string url = "ftp://ftp.example.com/directory/to/download/";
DownloadFtpDirectory(url, credentials, #"C:\target\directory");
If you want to avoid troubles with parsing the server-specific directory listing formats, use a 3rd party library that supports the MLSD command and/or parsing various LIST listing formats; and recursive downloads.
For example with WinSCP .NET assembly you can download whole directory with a single call to the Session.GetFiles:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "ftp.example.com",
UserName = "user",
Password = "mypassword",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// Download files
session.GetFiles("/directory/to/download/*", #"C:\target\directory\*").Check();
}
Internally, WinSCP uses the MLSD command, if supported by the server. If not, it uses the LIST command and supports dozens of different listing formats.
The Session.GetFiles method is recursive by default.
In most cases, Session.GetFilesToDirectory is a more straightforward equivalent to Session.GetFiles:
session.GetFiles("/directory/to/download", #"C:\target\directory").Check();
(I'm the author of WinSCP)
I want to write a C# that:
Access the FTP with username and password
Goes to a certain folder
Sees what files aren't there from my PC folder and upload them.
My idea:
I have a folder called "mods" in my PC and another folder called "mods" in the FTP, so instead of opening the "FileZilla" software I want to write a C# that connects to the FTP and check what files aren't there.
Thank you so much!
you can list your file inftp server like this and do a comparaiosn with list file of mod dir
FtpWebRequest ftpRequest = (FtpWebRequest)WebRequest.Create(uri);
ftpRequest.Credentials =new NetworkCredential("anonymous","janeDoe#contoso.com");
ftpRequest.Method = WebRequestMethods.Ftp.ListDirectory;
FtpWebResponse response = (FtpWebResponse)ftpRequest.GetResponse();
StreamReader streamReader = new StreamReader(response.GetResponseStream());
List<string> directories = new List<string>();
string line = streamReader.ReadLine();
while (!string.IsNullOrEmpty(line))
{
directories.Add(line);
line = streamReader.ReadLine();
}
streamReader.Close();
There's no magic way, with a pure .NET framework (its FtpWebRequest class). You have to code it.
List the FTP folder
List the local folder
Find the missing files
Upload them one by one
void SynchronizeLocalAndFtpDirectory(
string localPath, string remoteUri, NetworkCredential credentials)
{
List<string> remoteFiles = new List<string>();
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(remoteUri);
request.Credentials = credentials;
request.Method = WebRequestMethods.Ftp.ListDirectory;
using (FtpWebResponse response = (FtpWebResponse)request.GetResponse())
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
while (!reader.EndOfStream)
{
remoteFiles.Add(reader.ReadLine());
}
}
IEnumerable<string> localFiles =
Directory.GetFiles(localPath).Select(path => Path.GetFileName(path));
IEnumerable<string> missingFiles = localFiles.Except(remoteFiles);
foreach (string filename in missingFiles)
{
Console.WriteLine("Uploading missing file {0}", filename);
string remoteFileUri = remoteUri + filename;
string localFilePath = Path.Combine(localPath, filename);
FtpWebRequest uploadRequest = (FtpWebRequest)WebRequest.Create(remoteFileUri);
uploadRequest.Method = WebRequestMethods.Ftp.UploadFile;
uploadRequest.Credentials = credentials;
using (Stream targetStream = uploadRequest.GetRequestStream())
using (Stream sourceStream = File.OpenRead(localFilePath))
{
byte[] buffer = new byte[10240];
int read;
while ((read = sourceStream.Read(buffer, 0, buffer.Length)) > 0)
{
targetStream.Write(buffer, 0, read);
}
}
}
}
Use the function like:
SynchronizeLocalAndFtpDirectory(
#"C:\local\mods", "ftp://ftp.example.com/remote/mods/",
new NetworkCredential("username", "password"));
(Do not forget the trailing slash in the URI).
Or use 3rd party FTP library that supports synchronization.
For example with the WinSCP .NET assembly, this is as easy as a single call to Session.SynchronizeDirectories.
And it will not only upload missing file, but also update out-of-the-date files and optionally delete orphan files.
void SynchronizeLocalAndFtpDirectory(
string localPath, string remotePath, SessionOptions sessionOptions)
{
using (Session session = new Session())
{
session.Open(sessionOptions);
session.SynchronizeDirectories(
SynchronizationMode.Remote, localPath, remotePath, false).Check();
}
}
Use it like:
SessionOptions sessionOptions = new SessionOptions()
{
Protocol = Protocol.Ftp,
HostName = "ftp.example.com",
UserName = "username",
Password = "password",
};
SynchronizeLocalAndFtpDirectory(#"C:\local\mods", "/remote/mods", sessionOptions);
(I'm the author of WinSCP)
General Info
I'm still in the process of learning C#. To help myself out, I'm trying to create a program that will automatically synchronise all of my local projects with a folder on my FTP server. This so that whether I'm at school or at home, I always have the same projects available to me.
I know there are programs like Dropbox that already do this for me, but I figured creating something like that myself will teach me a lot along the way.
The problem
My first step towards my goal was to just download all files, subdirectories and subfiles from my FTP server. I've managed to download all files from a directory with the code below. However, my code only lists the folder names and the files in the main directory. Subfolders and subfiles are never returned and never downloaded. Aside from that, the server returns a 550 error because I'm trying to download the folders as if they are files. I've been on this for 4+ hours now, but I just can't find anything on how to fix these problems and make it work. Therefor I'm hoping you guys will help me out :)
Code
public string[] GetFileList()
{
string[] downloadFiles;
StringBuilder result = new StringBuilder();
WebResponse response = null;
StreamReader reader = null;
try
{
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(url);
request.UseBinary = true;
request.Method = WebRequestMethods.Ftp.ListDirectory;
request.Credentials = new NetworkCredential(ftpUserName, ftpPassWord);
request.KeepAlive = false;
request.UsePassive = false;
response = request.GetResponse();
reader = new StreamReader(response.GetResponseStream());
string line = reader.ReadLine();
while (line != null)
{
result.Append(line);
result.Append("\n");
line = reader.ReadLine();
}
result.Remove(result.ToString().LastIndexOf('\n'), 1);
return result.ToString().Split('\n');
}
catch (Exception ex)
{
if (reader != null)
{
reader.Close();
}
if (response != null)
{
response.Close();
}
downloadFiles = null;
return downloadFiles;
}
}
private void Download(string file)
{
try
{
string uri = url + "/" + file;
Uri serverUri = new Uri(uri);
if (serverUri.Scheme != Uri.UriSchemeFtp)
{
return;
}
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(url + "/" + file);
request.UseBinary = true;
request.Method = WebRequestMethods.Ftp.DownloadFile;
request.Credentials = new NetworkCredential(ftpUserName, ftpPassWord);
request.KeepAlive = false;
request.UsePassive = false;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
FileStream writeStream = new FileStream(localDestnDir + "\\" + file, FileMode.Create);
int Length = 2048;
Byte[] buffer = new Byte[Length];
int bytesRead = responseStream.Read(buffer, 0, Length);
while (bytesRead > 0)
{
writeStream.Write(buffer, 0, bytesRead);
bytesRead = responseStream.Read(buffer, 0, Length);
}
writeStream.Close();
response.Close();
}
catch (WebException wEx)
{
MessageBox.Show(wEx.Message, "Download Error");
}
catch (Exception ex)
{
MessageBox.Show(ex.Message, "Download Error");
}
}
The FtpWebRequest does not have any explicit support for recursive file operations (including downloads). You have to implement the recursion yourself:
List the remote directory
Iterate the entries, downloading files and recursing into subdirectories (listing them again, etc.)
Tricky part is to identify files from subdirectories. There's no way to do that in a portable way with the FtpWebRequest. The FtpWebRequest unfortunately does not support the MLSD command, which is the only portable way to retrieve directory listing with file attributes in FTP protocol. See also Checking if object on FTP server is file or directory.
Your options are:
Do an operation on a file name that is certain to fail for file and succeeds for directories (or vice versa). I.e. you can try to download the "name". If that succeeds, it's a file, if that fails, it's a directory.
You may be lucky and in your specific case, you can tell a file from a directory by a file name (i.e. all your files have an extension, while subdirectories do not)
You use a long directory listing (LIST command = ListDirectoryDetails method) and try to parse a server-specific listing. Many FTP servers use *nix-style listing, where you identify a directory by the d at the very beginning of the entry. But many servers use a different format. The following example uses this approach (assuming the *nix format)
void DownloadFtpDirectory(
string url, NetworkCredential credentials, string localPath)
{
FtpWebRequest listRequest = (FtpWebRequest)WebRequest.Create(url);
listRequest.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
listRequest.Credentials = credentials;
List<string> lines = new List<string>();
using (var listResponse = (FtpWebResponse)listRequest.GetResponse())
using (Stream listStream = listResponse.GetResponseStream())
using (var listReader = new StreamReader(listStream))
{
while (!listReader.EndOfStream)
{
lines.Add(listReader.ReadLine());
}
}
foreach (string line in lines)
{
string[] tokens =
line.Split(new[] { ' ' }, 9, StringSplitOptions.RemoveEmptyEntries);
string name = tokens[8];
string permissions = tokens[0];
string localFilePath = Path.Combine(localPath, name);
string fileUrl = url + name;
if (permissions[0] == 'd')
{
if (!Directory.Exists(localFilePath))
{
Directory.CreateDirectory(localFilePath);
}
DownloadFtpDirectory(fileUrl + "/", credentials, localFilePath);
}
else
{
FtpWebRequest downloadRequest =
(FtpWebRequest)WebRequest.Create(fileUrl);
downloadRequest.Method = WebRequestMethods.Ftp.DownloadFile;
downloadRequest.Credentials = credentials;
using (FtpWebResponse downloadResponse =
(FtpWebResponse)downloadRequest.GetResponse())
using (Stream sourceStream = downloadResponse.GetResponseStream())
using (Stream targetStream = File.Create(localFilePath))
{
byte[] buffer = new byte[10240];
int read;
while ((read = sourceStream.Read(buffer, 0, buffer.Length)) > 0)
{
targetStream.Write(buffer, 0, read);
}
}
}
}
}
Use the function like:
NetworkCredential credentials = new NetworkCredential("user", "mypassword");
string url = "ftp://ftp.example.com/directory/to/download/";
DownloadFtpDirectory(url, credentials, #"C:\target\directory");
If you want to avoid troubles with parsing the server-specific directory listing formats, use a 3rd party library that supports the MLSD command and/or parsing various LIST listing formats; and recursive downloads.
For example with WinSCP .NET assembly you can download whole directory with a single call to the Session.GetFiles:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "ftp.example.com",
UserName = "user",
Password = "mypassword",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// Download files
session.GetFiles("/directory/to/download/*", #"C:\target\directory\*").Check();
}
Internally, WinSCP uses the MLSD command, if supported by the server. If not, it uses the LIST command and supports dozens of different listing formats.
The Session.GetFiles method is recursive by default.
In most cases, Session.GetFilesToDirectory is a more straightforward equivalent to Session.GetFiles:
session.GetFiles("/directory/to/download", #"C:\target\directory").Check();
(I'm the author of WinSCP)
I need to use FtpWebRequest to put a file in a FTP directory. Before the upload, I would first like to know if this file exists.
What method or property should I use to check if this file exists?
var request = (FtpWebRequest)WebRequest.Create
("ftp://ftp.domain.com/doesntexist.txt");
request.Credentials = new NetworkCredential("user", "pass");
request.Method = WebRequestMethods.Ftp.GetFileSize;
try
{
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
FtpWebResponse response = (FtpWebResponse)ex.Response;
if (response.StatusCode ==
FtpStatusCode.ActionNotTakenFileUnavailable)
{
//Does not exist
}
}
As a general rule it's a bad idea to use Exceptions for functionality in your code like this, however in this instance I believe it's a win for pragmatism. Calling list on the directory has the potential to be FAR more inefficient than using exceptions in this way.
If you're not, just be aware it's not good practice!
EDIT: "It works for me!"
This appears to work on most ftp servers but not all. Some servers require sending "TYPE I" before the SIZE command will work. One would have thought that the problem should be solved as follows:
request.UseBinary = true;
Unfortunately it is a by design limitation (big fat bug!) that unless FtpWebRequest is either downloading or uploading a file it will NOT send "TYPE I". See discussion and Microsoft response here.
I'd recommend using the following WebRequestMethod instead, this works for me on all servers I tested, even ones which would not return a file size.
WebRequestMethods.Ftp.GetDateTimestamp
Because
request.Method = WebRequestMethods.Ftp.GetFileSize
may fails in some case (550: SIZE not allowed in ASCII mode), you can just check Timestamp instead.
reqFTP.Credentials = new NetworkCredential(inf.LogOn, inf.Password);
reqFTP.UseBinary = true;
reqFTP.Method = WebRequestMethods.Ftp.GetDateTimestamp;
FtpWebRequest (nor any other class in .NET) does not have any explicit method to check a file existence on FTP server. You need to abuse a request like GetFileSize or GetDateTimestamp.
string url = "ftp://ftp.example.com/remote/path/file.txt";
WebRequest request = WebRequest.Create(url);
request.Credentials = new NetworkCredential("username", "password");
request.Method = WebRequestMethods.Ftp.GetFileSize;
try
{
request.GetResponse();
Console.WriteLine("Exists");
}
catch (WebException e)
{
FtpWebResponse response = (FtpWebResponse)e.Response;
if (response.StatusCode == FtpStatusCode.ActionNotTakenFileUnavailable)
{
Console.WriteLine("Does not exist");
}
else
{
Console.WriteLine("Error: " + e.Message);
}
}
If you want a more straightforward code, use some 3rd party FTP library.
For example with WinSCP .NET assembly, you can use its Session.FileExists method:
SessionOptions sessionOptions = new SessionOptions {
Protocol = Protocol.Ftp,
HostName = "ftp.example.com",
UserName = "username",
Password = "password",
};
Session session = new Session();
session.Open(sessionOptions);
if (session.FileExists("/remote/path/file.txt"))
{
Console.WriteLine("Exists");
}
else
{
Console.WriteLine("Does not exist");
}
(I'm the author of WinSCP)
You can use WebRequestMethods.Ftp.ListDirectory to check if a file exist, no need for nasty try catch mechanism.
private static bool ExistFile(string remoteAddress)
{
int pos = remoteAddress.LastIndexOf('/');
string dirPath = remoteAddress.Substring(0, pos); // skip the filename only get the directory
NetworkCredential credentials = new NetworkCredential(FtpUser, FtpPass);
FtpWebRequest listRequest = (FtpWebRequest)WebRequest.Create(dirPath);
listRequest.Method = WebRequestMethods.Ftp.ListDirectory;
listRequest.Credentials = credentials;
using (FtpWebResponse listResponse = (FtpWebResponse)listRequest.GetResponse())
using (Stream listStream = listResponse.GetResponseStream())
using (StreamReader listReader = new StreamReader(listStream))
{
string fileToTest = Path.GetFileName(remoteAddress);
while (!listReader.EndOfStream)
{
string fileName = listReader.ReadLine();
fileName = Path.GetFileName(fileName);
if (fileToTest == fileName)
{
return true;
}
}
}
return false;
}
static void Main(string[] args)
{
bool existFile = ExistFile("ftp://123.456.789.12/test/config.json");
}
I use FTPStatusCode.FileActionOK to check if file exists...
then, in the "else" section, return false.