I would use this instruction:
System.IO.DirectoryInfo dir = new System.IO.DirectoryInfo("ftp://192.168.47.1/DocXML");
But I can´t.
How can I use ("ftp://192.168.47.1/DocXML"); with new System.IO.DirectoryInfo("");?
This is the code
System.IO.DirectoryInfo dir = new System.IO.DirectoryInfo(#"\\192.168.47.1\DocXML");`
IEnumerable<System.IO.FileInfo> fileList = dir.GetFiles("*.*", System.IO.SearchOption.AllDirectories);
I'm afraid you can't.
Try this instead:
FtpWebRequest req = (FtpWebRequest)WebRequest.Create("ftp://192.168.47.1/DocXML");
req.Credentials = new NetworkCredential("foo", "foo#foo.com");
req.Method = WebRequestMethods.Ftp.ListDirectory;
FtpWebResponse res = (FtpWebResponse)req.GetResponse();
using (StreamReader streamReader = new StreamReader(res.GetResponseStream()))
{
...
}
If you a need structured information about files in an FTP directory, you have to use a 3rd party library. The .NET framework does not offer such functionality.
Particularly because it does not support an MLSD FTP command, what is the only reliable way to retrieve a machine-readable listing of remote files with their attributes.
There are many 3rd party libraries that allow this.
For example with WinSCP .NET assembly:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "example.com",
UserName = "username",
Password = "password",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// Get list of files in the directory
string remotePath = "/remote/path/";
RemoteDirectoryInfo directoryInfo = session.ListDirectory(remotePath);
foreach (RemoteFileInfo fileInfo in directoryInfo.Files)
{
Console.WriteLine("{0} with size {1}, permissions {2} and last modification at {3}",
fileInfo.Name, fileInfo.Length, fileInfo.FilePermissions,
fileInfo.LastWriteTime);
}
}
References:
https://winscp.net/eng/docs/library_session_listdirectory
https://winscp.net/eng/docs/library_remotefileinfo
From your comment and your other question, you seem to actually need to retrieve the oldest file in FTP directory. For that see:
Download the latest file from an FTP server (C#)
Downloading the most recent file (PowerShell, but translates easily to C#)
Both are for the newest, not oldest, file. Just replace the .OrderByDescending with the .Order in the C# code to get the oldest file.
(I'm the author of WinSCP)
Not working in this way.
I recommend using SFTP instead of FTP. For this I'm using the 3rd party library "SharpSSH".
The following example seems to work:
using System.IO;
using Tamir.SharpSsh;
using Tamir.SharpSsh.jsch;
string ip = "DestinationIp";
string user = "JohnDoe";
string password = "YourPassword";
Sftp sftp = new Tamir.SharpSsh.Sftp(ip, user, password);
sftp.Connect();
FileInfo yourFileInfo = new FileInfo("path");
There's also the possibility to add a primary key with
sftp.AddIdentityFile();
Related
My app requires copying file using SFTP from a location directly to Azure storage.
Our app is using C# with .NET 4.6 and our WinSCP version is 5.21.1.
My old code works using Session.GetFileToDirectory() method, but the problem is it need to store the file on temp folder inside our hosting.
using (Session session = new Session())
{
session.Open(sessionOptions);
TransferOptions transferOptions = new TransferOptions();
transferOptions.TransferMode = TransferMode.Binary;
var transfer = session.GetFileToDirectory(FilePath, fullPath);
using (Stream stream = File.OpenRead(transfer.Destination))
{
UploadToAzure(stream, Filename, Foldername);
}
}
As we planned to entirely use Azure storage, I change my code like this
using (Session session = new Session())
{
session.Open(sessionOptions);
TransferOptions transferOptions = new TransferOptions();
transferOptions.TransferMode = TransferMode.Binary;
using (Stream stream = session.GetFile(FilePath, transferOptions))
{
UploadToAzure(stream, Filename, Foldername);
}
}
Here my library that uploads the file using Stream to Azure.
This code is working fine using my old code that still save to temp folder before send to Azure.
public static string UploadToAzure(Stream attachment, string Filename, string Foldername)
{
System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls12;
var connectionString = $"{ConfigurationManager.AppSettings["AzureFileShareConnectionString"]}";
string shareName = $"{ConfigurationManager.AppSettings["AzureFileShareFolderName"]}";
string dirName = $"files\\{Foldername}";
string fileName = Filename;
try
{
ShareClient share = new ShareClient(connectionString, shareName);
share.CreateIfNotExists();
ShareDirectoryClient directory = share.GetDirectoryClient(dirName);
directory.CreateIfNotExists();
// Get a reference to a file and upload it
ShareFileClient file = directory.GetFileClient(fileName);
file.Create(attachment.Length);
file.UploadRange(
new HttpRange(0, attachment.Length), attachment);
}
catch (Exception e)
{
return $"Uploaded {Filename} failed : {e.ToString()}";
}
return $"{Filename} Uploaded";
}
But currently my new code not working with error message
'((WinSCP.PipeStream)stream).Length' threw an exception of type 'System.NotSupportedException'.
This is the object description on creating stream using Session.GetFile method
This is 'exception stacktrace' on sending the empty-stream to Azure
The Stream returned by WinSCP Session.GetFile does not implement the Stream.Length property, because WinSCP cannot guarantee that the size of the file is fixed. The remote file might be changing while you are downloading the file. Not to mention ASCII transfer mode, when the file is converted while being transferred, with unpredictable impact on the final size.
You use the size (Stream.Length) in two places:
When creating the file:
file.Create(attachment.Length);
The parameter of ShareFileClient.Create is maxSize. So it does not look like it's a real size. You can possibly just put an arbitrary large number here.
Or if you prefer (and know that the file is not changing), read the current size of the remote file using Session.GetFileInfo and RemoteFileInfo.Length:
file.Create(session.GetFileInfo(FilePath).Length);
When uploading the contents:
file.UploadRange(new HttpRange(0, attachment.Length), attachment);
The above can be replaced with simple ShareFileClient.Upload:
file.Upload(attachment);
I'm currently working on a .NET 4.6 console application. I need to parse a couple of XML files from different directories on my FTP server. I thought the best approach would be, to read all file paths and store them into an IEnumerable, to process them further (Serialize XML Files to objects).
The root FTP path looks like this:
string urlFtpServer = #"ftp://128.0.1.70";
File paths look like this:
string file1 = #"ftp://128.0.1.70/MyFolder1/Mainfile.xml";
string file2 = #"ftp://128.0.1.70/MyFolder1/Subfile.xml";
string file3 = #"ftp://128.0.1.70/MyFolder2/Mainfile.xml";
string file4 = #"ftp://128.0.1.70/MyFolder2/Subfile.xml";
string file5 = #"ftp://128.0.1.70/MyFolder3/Mainfile.xml";
My question is, do you know how I can get those specific file paths?
I currently can read the folders of my FTP directory with this coding:
static void Main(string[] args)
{
string url = #"ftp://128.0.1.70";
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(url);
request.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
request.Credentials = new NetworkCredential("My-User", "mypassword");
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader reader = new StreamReader(responseStream);
Console.WriteLine(reader.ReadToEnd());
Console.WriteLine("Directory List Complete, status {0}", response.StatusDescription);
reader.Close();
response.Close();
Console.ReadKey();
}
Do you know how I can read all file paths from the FTP main directory and possibly store them into a List<string>?
Thank you very much!!
Using FtpWebRequest
The FtpWebRequest does not have any explicit support for recursive file operations (including listing). You have to implement the recursion yourself:
List the remote directory
Iterate the entries, recursing into subdirectories (listing them again, etc.)
Tricky part is to identify files from subdirectories. There's no way to do that in a portable way with the FtpWebRequest. The FtpWebRequest unfortunately does not support the MLSD command, which is the only portable way to retrieve directory listing with file attributes in FTP protocol. See also Checking if object on FTP server is file or directory.
Your options are:
Do an operation on a file name that is certain to fail for file and succeeds for directories (or vice versa). I.e. you can try to download the "name". If that succeeds, it's a file, if that fails, it's a directory.
You may be lucky and in your specific case, you can tell a file from a directory by a file name (i.e. all your files have an extension, while subdirectories do not)
You use a long directory listing (LIST command = ListDirectoryDetails method) and try to parse a server-specific listing. Many FTP servers use *nix-style listing, where you identify a directory by the d at the very beginning of the entry. But many servers use a different format. The following example uses this approach (assuming the *nix format)
void ListFtpDirectory(
string url, string rootPath, NetworkCredential credentials, List<string> list)
{
FtpWebRequest listRequest = (FtpWebRequest)WebRequest.Create(url + rootPath);
listRequest.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
listRequest.Credentials = credentials;
List<string> lines = new List<string>();
using (FtpWebResponse listResponse = (FtpWebResponse)listRequest.GetResponse())
using (Stream listStream = listResponse.GetResponseStream())
using (StreamReader listReader = new StreamReader(listStream))
{
while (!listReader.EndOfStream)
{
lines.Add(listReader.ReadLine());
}
}
foreach (string line in lines)
{
string[] tokens =
line.Split(new[] { ' ' }, 9, StringSplitOptions.RemoveEmptyEntries);
string name = tokens[8];
string permissions = tokens[0];
string filePath = rootPath + name;
if (permissions[0] == 'd')
{
ListFtpDirectory(url, filePath + "/", credentials, list);
}
else
{
list.Add(filePath);
}
}
}
Use the function like:
List<string> list = new List<string>();
NetworkCredential credentials = new NetworkCredential("user", "mypassword");
string url = "ftp://ftp.example.com/";
ListFtpDirectory(url, "", credentials, list);
Using 3rd party library
If you want to avoid troubles with parsing the server-specific directory listing formats, use a 3rd party library that supports the MLSD command and/or parsing various LIST listing formats; and recursive downloads.
For example with WinSCP .NET assembly you can list whole directory with a single call to the Session.EnumerateRemoteFiles:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "ftp.example.com",
UserName = "user",
Password = "mypassword",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// List files
IEnumerable<string> list =
session.EnumerateRemoteFiles("/", null, EnumerationOptions.AllDirectories).
Select(fileInfo => fileInfo.FullName);
}
Internally, WinSCP uses the MLSD command, if supported by the server. If not, it uses the LIST command and supports dozens of different listing formats.
(I'm the author of WinSCP)
I went through many posts on Stack Overflow, but I have not found what I needed.
I have a file suppose temp.csv
I want to upload this file to an SFTP server which is working fine.
I need to save this file with different extension like temp.csv.ready
Can you please suggest something on this.
Here is the code, which I tried and working fine. But I'm not not able to change file extension.
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Sftp,
HostName = System.Configuration.ConfigurationManager.AppSettings["puthost"].ToString(),
UserName = System.Configuration.ConfigurationManager.AppSettings["putusername"].ToString(),
Password = System.Configuration.ConfigurationManager.AppSettings["putpassword"].ToString(),
PortNumber = Convert.ToInt32(System.Configuration.ConfigurationManager.AppSettings["putport"].ToString()),
SshHostKeyFingerprint = ... //followed by your 16 bit key
};
using (Session session = new Session())
{
session.SessionLogPath = "log.txt";
session.Open(sessionOptions); //Attempts to connect to your sFtp site
//Get Ftp File
TransferOptions transferOptions = new TransferOptions();
transferOptions.TransferMode = TransferMode.Binary; //The Transfer Mode -
//<em style="font-size: 9pt;">Automatic, Binary, or Ascii
transferOptions.FilePermissions = null; //Permissions applied to remote files;
//null for default permissions. Can set user,
//Group, or other Read/Write/Execute permissions.
transferOptions.PreserveTimestamp = false; //Set last write time of
//destination file to that of source file - basically change the timestamp
//to match destination and source files.
transferOptions.ResumeSupport.State = TransferResumeSupportState.Off;
WinSCP.TransferOperationResult transferResult;
//the parameter list is: local Path, Remote Path, Delete source file?, transfer Options
transferResult = session.PutFiles(#System.Configuration.ConfigurationManager.AppSettings["sendfilesource"].ToString(), System.Configuration.ConfigurationManager.AppSettings["sendfiletarget"].ToString(), false, transferOptions);
}
The remotePath argument of the Session.PutFiles method is:
Full path to upload the file to.
So, all you need to do, is to specify the full path like:
/remote/path/temp.csv.ready
If you are willing to use a library, you may want to use a 3rd library like componenentpro sftp library. You can accomplish the job with client.UploadFile(#"xxx\temp.csv", "/temp.new.csv"). That line of code is excerpt from the following code example:
// Create a new class instance.
Sftp client = new Sftp();
// Connect to the SFTP server.
client.Connect("localhost");
// Authenticate.
client.Authenticate("test", "test");
// ...
// Upload local file 'c:\test.dat' to '/test.dat'.
client.UploadFile(#"xxx\temp.csv", "/temp.new.csv");
// ...
// Disconnect.
client.Disconnect();
I am using Tamir SharpSSH for transferring files from remote to local and vice versa with no issues.
But, when trying to upload multiple XML files via SFTP but I am receiving an error:
Illegal characters in path.
If I try to upload using the exact file name it transfers the file without any issues.
Every time I try to upload two XML files:
KDO_E2D_A21_AA769_20170124_143123.xml
KDO_E2D_A21_AA776_20170130_143010.xml
string ftpURL = "11.11.11.1";
string userName = "Aaaaaa"; //User Name of the SFTP server
string password = "hah4444"; //Password of the SFTP server
int port = 22; //Port No of the SFTP server (if any)
//The directory in SFTP server where the files will be uploaded
string ftpDirectory = "/home/A21sftp/kadoe/";
//Local directory from where the files will be uploaded
string localDirectory = "E:\\Zatpark\\*.xml";
Sftp Connection = new Sftp(ftpURL, userName, password);
Connection.Connect(port);
Connection.Put(localDirectory, ftpDirectory);
Connection.Close();
Do not use Tamir.SharpSSH, it's a dead project. Use some up to date SSH/SFTP implementation.
If you switch to library like SSH.NET that does not support wildcards, you have to use the Directory.GetFiles method to find files to upload:
SftpClient client = new SftpClient("example.com", "username", "password");
client.Connect();
string localDirectory = #"E:\Zatpark upload";
string localPattern = "*.xml";
string ftpDirectory = "/dv/inbound/";
string[] files = Directory.GetFiles(localDirectory, localPattern);
foreach (string file in files)
{
using (Stream inputStream = new FileStream(file, FileMode.Open))
{
client.UploadFile(inputStream, ftpDirectory + Path.GetFileName(file));
}
}
Or use a library that does support wildcards.
For example with WinSCP .NET assembly (what is not a pure .NET assembly though), you can do:
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Sftp,
HostName = "example.com",
UserName = "username",
Password = "password",
SshHostKeyFingerprint = "ssh-dss ...",
};
using (Session session = new Session())
{
session.Open(sessionOptions);
string ftpDirectory = "/dv/inbound/";
string localDirectory = #"E:\Zatpark upload\*.xml";
session.PutFiles(localDirectory, ftpDirectory).Check();
}
You can have a code template generated in WinSCP GUI.
(I'm the author of WinSCP)
To explain, why your code does not work: Check the ChannelSftp.glob_local method. Its implementation is strange, if not broken. It basically supports only masks consisting completely of * and ?'s.
I am facing a strange issue, I want to download a list of files from FTP. I preferred to go with Parallel Task. Below is my code. The issue is, all the list of files are getting downloaded, but duplicate files with different name are being generated. I am very new to Parallel task concept. Please help me to find out the issue.
Note: I am using SSH.Net for sftp connection and download.
private void ConcurrentDownload()
{
// Declaring Connection Information
PasswordAuthenticationMethod pm = new PasswordAuthenticationMethod("FTPUserName", "Password");
ConnectionInfo connectionInfo = new ConnectionInfo("FTPHost", 22, "FTPUserName", ProxyTypes.Socks5, "127.0.0.1", 8080, string.Empty, string.Empty, pm);
using (SftpClient sfc = new SftpClient(connectionInfo))
{
// Establish the remote connection
sfc.Connect();
// Getting Remote Directory Contents
IEnumerable<SftpFile> sFiles = new List<SftpFile>();
sFiles = sfc.ListDirectory(".\\");
// Building the File List
List<string> remotefiles = new List<string>();
foreach (SftpFile sfile in sFiles)
{
if (!sfile.IsDirectory)
{
string ss = sfile.Name;
remotefiles.Add(ss);
}
}
// Parallel Download
Parallel.ForEach(remotefiles.Distinct(), file => DownloadFile(sfc, file));
sfc.Disconnect();
}
}
private void DownloadFile(SftpClient sf, string RemoteFileName)
{
using (Stream ms = File.OpenWrite(RemoteFileName))
{
sf.DownloadFile(RemoteFileName, ms);
}
}
You better use Distinct like below
Parallel.ForEach(remotefiles.Distinct(), file => DownloadFile(sfc, file));
if you have duplicate file names and when parallel processing start on same file you will get exception on those duplicate files.
And also you are not downloading to another location, what you are doing is download to same ftp source location. is that correct?
I would give diferent download directory and get file name from source file and then download to that location as below
private void DownloadFile(SftpClient sf, string RemoteFileName)
{
string downloadTo = Path.Combine(DownloadDirectoryPath, Path.GetFileName(RemoteFileName));
using (Stream ms = File.OpenWrite(downloadTo))
{
sf.DownloadFile(RemoteFileName, ms);
}
}
Related Reference : SFTP Async Upload in Parallel