Develop Reference

Streaming from SFTP using WinSCP Session.GetFile fails – ((WinSCP.PipeStream)stream).Length' threw an exception of type 'System.NotSupportedException'

My app requires copying file using SFTP from a location directly to Azure storage.
Our app is using C# with .NET 4.6 and our WinSCP version is 5.21.1.
My old code works using Session.GetFileToDirectory() method, but the problem is it need to store the file on temp folder inside our hosting.
using (Session session = new Session())
{
session.Open(sessionOptions);
TransferOptions transferOptions = new TransferOptions();
transferOptions.TransferMode = TransferMode.Binary;
var transfer = session.GetFileToDirectory(FilePath, fullPath);
using (Stream stream = File.OpenRead(transfer.Destination))
{
UploadToAzure(stream, Filename, Foldername);
}
}
As we planned to entirely use Azure storage, I change my code like this
using (Session session = new Session())
{
session.Open(sessionOptions);
TransferOptions transferOptions = new TransferOptions();
transferOptions.TransferMode = TransferMode.Binary;
using (Stream stream = session.GetFile(FilePath, transferOptions))
{
UploadToAzure(stream, Filename, Foldername);
}
}
Here my library that uploads the file using Stream to Azure.
This code is working fine using my old code that still save to temp folder before send to Azure.
public static string UploadToAzure(Stream attachment, string Filename, string Foldername)
{
System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls12;
var connectionString = $"{ConfigurationManager.AppSettings["AzureFileShareConnectionString"]}";
string shareName = $"{ConfigurationManager.AppSettings["AzureFileShareFolderName"]}";
string dirName = $"files\\{Foldername}";
string fileName = Filename;
try
{
ShareClient share = new ShareClient(connectionString, shareName);
share.CreateIfNotExists();
ShareDirectoryClient directory = share.GetDirectoryClient(dirName);
directory.CreateIfNotExists();
// Get a reference to a file and upload it
ShareFileClient file = directory.GetFileClient(fileName);
file.Create(attachment.Length);
file.UploadRange(
new HttpRange(0, attachment.Length), attachment);
}
catch (Exception e)
{
return $"Uploaded {Filename} failed : {e.ToString()}";
}
return $"{Filename} Uploaded";
}
But currently my new code not working with error message
'((WinSCP.PipeStream)stream).Length' threw an exception of type 'System.NotSupportedException'.
This is the object description on creating stream using Session.GetFile method
This is 'exception stacktrace' on sending the empty-stream to Azure

The Stream returned by WinSCP Session.GetFile does not implement the Stream.Length property, because WinSCP cannot guarantee that the size of the file is fixed. The remote file might be changing while you are downloading the file. Not to mention ASCII transfer mode, when the file is converted while being transferred, with unpredictable impact on the final size.
You use the size (Stream.Length) in two places:
When creating the file:
file.Create(attachment.Length);
The parameter of ShareFileClient.Create is maxSize. So it does not look like it's a real size. You can possibly just put an arbitrary large number here.
Or if you prefer (and know that the file is not changing), read the current size of the remote file using Session.GetFileInfo and RemoteFileInfo.Length:
file.Create(session.GetFileInfo(FilePath).Length);
When uploading the contents:
file.UploadRange(new HttpRange(0, attachment.Length), attachment);
The above can be replaced with simple ShareFileClient.Upload:
file.Upload(attachment);

Read folder paths from FTP directory to IEnumerable

I'm currently working on a .NET 4.6 console application. I need to parse a couple of XML files from different directories on my FTP server. I thought the best approach would be, to read all file paths and store them into an IEnumerable, to process them further (Serialize XML Files to objects).
The root FTP path looks like this:
string urlFtpServer = #"ftp://128.0.1.70";
File paths look like this:
string file1 = #"ftp://128.0.1.70/MyFolder1/Mainfile.xml";
string file2 = #"ftp://128.0.1.70/MyFolder1/Subfile.xml";
string file3 = #"ftp://128.0.1.70/MyFolder2/Mainfile.xml";
string file4 = #"ftp://128.0.1.70/MyFolder2/Subfile.xml";
string file5 = #"ftp://128.0.1.70/MyFolder3/Mainfile.xml";
My question is, do you know how I can get those specific file paths?
I currently can read the folders of my FTP directory with this coding:
static void Main(string[] args)
{
string url = #"ftp://128.0.1.70";
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(url);
request.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
request.Credentials = new NetworkCredential("My-User", "mypassword");
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader reader = new StreamReader(responseStream);
Console.WriteLine(reader.ReadToEnd());
Console.WriteLine("Directory List Complete, status {0}", response.StatusDescription);
reader.Close();
response.Close();
Console.ReadKey();
}
Do you know how I can read all file paths from the FTP main directory and possibly store them into a List<string>?
Thank you very much!!

Using FtpWebRequest
The FtpWebRequest does not have any explicit support for recursive file operations (including listing). You have to implement the recursion yourself:
List the remote directory
Iterate the entries, recursing into subdirectories (listing them again, etc.)
Tricky part is to identify files from subdirectories. There's no way to do that in a portable way with the FtpWebRequest. The FtpWebRequest unfortunately does not support the MLSD command, which is the only portable way to retrieve directory listing with file attributes in FTP protocol. See also Checking if object on FTP server is file or directory.
Your options are:
Do an operation on a file name that is certain to fail for file and succeeds for directories (or vice versa). I.e. you can try to download the "name". If that succeeds, it's a file, if that fails, it's a directory.
You may be lucky and in your specific case, you can tell a file from a directory by a file name (i.e. all your files have an extension, while subdirectories do not)
You use a long directory listing (LIST command = ListDirectoryDetails method) and try to parse a server-specific listing. Many FTP servers use *nix-style listing, where you identify a directory by the d at the very beginning of the entry. But many servers use a different format. The following example uses this approach (assuming the *nix format)
void ListFtpDirectory(
string url, string rootPath, NetworkCredential credentials, List<string> list)
{
FtpWebRequest listRequest = (FtpWebRequest)WebRequest.Create(url + rootPath);
listRequest.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
listRequest.Credentials = credentials;
List<string> lines = new List<string>();
using (FtpWebResponse listResponse = (FtpWebResponse)listRequest.GetResponse())
using (Stream listStream = listResponse.GetResponseStream())
using (StreamReader listReader = new StreamReader(listStream))
{
while (!listReader.EndOfStream)
{
lines.Add(listReader.ReadLine());
}
}
foreach (string line in lines)
{
string[] tokens =
line.Split(new[] { ' ' }, 9, StringSplitOptions.RemoveEmptyEntries);
string name = tokens[8];
string permissions = tokens[0];
string filePath = rootPath + name;
if (permissions[0] == 'd')
{
ListFtpDirectory(url, filePath + "/", credentials, list);
}
else
{
list.Add(filePath);
}
}
}
Use the function like:
List<string> list = new List<string>();
NetworkCredential credentials = new NetworkCredential("user", "mypassword");
string url = "ftp://ftp.example.com/";
ListFtpDirectory(url, "", credentials, list);
Using 3rd party library
If you want to avoid troubles with parsing the server-specific directory listing formats, use a 3rd party library that supports the MLSD command and/or parsing various LIST listing formats; and recursive downloads.
For example with WinSCP .NET assembly you can list whole directory with a single call to the Session.EnumerateRemoteFiles:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "ftp.example.com",
UserName = "user",
Password = "mypassword",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// List files
IEnumerable<string> list =
session.EnumerateRemoteFiles("/", null, EnumerationOptions.AllDirectories).
Select(fileInfo => fileInfo.FullName);
}
Internally, WinSCP uses the MLSD command, if supported by the server. If not, it uses the LIST command and supports dozens of different listing formats.
(I'm the author of WinSCP)

Upload all files matching wildcard to SFTP

I am using Tamir SharpSSH for transferring files from remote to local and vice versa with no issues.
But, when trying to upload multiple XML files via SFTP but I am receiving an error:
Illegal characters in path.
If I try to upload using the exact file name it transfers the file without any issues.
Every time I try to upload two XML files:
KDO_E2D_A21_AA769_20170124_143123.xml
KDO_E2D_A21_AA776_20170130_143010.xml
string ftpURL = "11.11.11.1";
string userName = "Aaaaaa"; //User Name of the SFTP server
string password = "hah4444"; //Password of the SFTP server
int port = 22; //Port No of the SFTP server (if any)
//The directory in SFTP server where the files will be uploaded
string ftpDirectory = "/home/A21sftp/kadoe/";
//Local directory from where the files will be uploaded
string localDirectory = "E:\\Zatpark\\*.xml";
Sftp Connection = new Sftp(ftpURL, userName, password);
Connection.Connect(port);
Connection.Put(localDirectory, ftpDirectory);
Connection.Close();

Do not use Tamir.SharpSSH, it's a dead project. Use some up to date SSH/SFTP implementation.
If you switch to library like SSH.NET that does not support wildcards, you have to use the Directory.GetFiles method to find files to upload:
SftpClient client = new SftpClient("example.com", "username", "password");
client.Connect();
string localDirectory = #"E:\Zatpark upload";
string localPattern = "*.xml";
string ftpDirectory = "/dv/inbound/";
string[] files = Directory.GetFiles(localDirectory, localPattern);
foreach (string file in files)
{
using (Stream inputStream = new FileStream(file, FileMode.Open))
{
client.UploadFile(inputStream, ftpDirectory + Path.GetFileName(file));
}
}
Or use a library that does support wildcards.
For example with WinSCP .NET assembly (what is not a pure .NET assembly though), you can do:
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Sftp,
HostName = "example.com",
UserName = "username",
Password = "password",
SshHostKeyFingerprint = "ssh-dss ...",
};
using (Session session = new Session())
{
session.Open(sessionOptions);
string ftpDirectory = "/dv/inbound/";
string localDirectory = #"E:\Zatpark upload\*.xml";
session.PutFiles(localDirectory, ftpDirectory).Check();
}
You can have a code template generated in WinSCP GUI.
(I'm the author of WinSCP)
To explain, why your code does not work: Check the ChannelSftp.glob_local method. Its implementation is strange, if not broken. It basically supports only masks consisting completely of * and ?'s.

Use "DirectoryInfo" with FTP server

I would use this instruction:
System.IO.DirectoryInfo dir = new System.IO.DirectoryInfo("ftp://192.168.47.1/DocXML");
But I can´t.
How can I use ("ftp://192.168.47.1/DocXML"); with new System.IO.DirectoryInfo("");?
This is the code
System.IO.DirectoryInfo dir = new System.IO.DirectoryInfo(#"\\192.168.47.1\DocXML");`
IEnumerable<System.IO.FileInfo> fileList = dir.GetFiles("*.*", System.IO.SearchOption.AllDirectories);

I'm afraid you can't.
Try this instead:
FtpWebRequest req = (FtpWebRequest)WebRequest.Create("ftp://192.168.47.1/DocXML");
req.Credentials = new NetworkCredential("foo", "foo#foo.com");
req.Method = WebRequestMethods.Ftp.ListDirectory;
FtpWebResponse res = (FtpWebResponse)req.GetResponse();
using (StreamReader streamReader = new StreamReader(res.GetResponseStream()))
{
...
}

If you a need structured information about files in an FTP directory, you have to use a 3rd party library. The .NET framework does not offer such functionality.
Particularly because it does not support an MLSD FTP command, what is the only reliable way to retrieve a machine-readable listing of remote files with their attributes.
There are many 3rd party libraries that allow this.
For example with WinSCP .NET assembly:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "example.com",
UserName = "username",
Password = "password",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// Get list of files in the directory
string remotePath = "/remote/path/";
RemoteDirectoryInfo directoryInfo = session.ListDirectory(remotePath);
foreach (RemoteFileInfo fileInfo in directoryInfo.Files)
{
Console.WriteLine("{0} with size {1}, permissions {2} and last modification at {3}",
fileInfo.Name, fileInfo.Length, fileInfo.FilePermissions,
fileInfo.LastWriteTime);
}
}
References:
https://winscp.net/eng/docs/library_session_listdirectory
https://winscp.net/eng/docs/library_remotefileinfo
From your comment and your other question, you seem to actually need to retrieve the oldest file in FTP directory. For that see:
Download the latest file from an FTP server (C#)
Downloading the most recent file (PowerShell, but translates easily to C#)
Both are for the newest, not oldest, file. Just replace the .OrderByDescending with the .Order in the C# code to get the oldest file.
(I'm the author of WinSCP)

Not working in this way.
I recommend using SFTP instead of FTP. For this I'm using the 3rd party library "SharpSSH".
The following example seems to work:
using System.IO;
using Tamir.SharpSsh;
using Tamir.SharpSsh.jsch;
string ip = "DestinationIp";
string user = "JohnDoe";
string password = "YourPassword";
Sftp sftp = new Tamir.SharpSsh.Sftp(ip, user, password);
sftp.Connect();
FileInfo yourFileInfo = new FileInfo("path");
There's also the possibility to add a primary key with
sftp.AddIdentityFile();

How to upload an image file to Active Directory user profile in C#?

I need a method which will take an *.jpg image file and upload it to a user profile in the Active Directory of Windows AD 2003.
Also a method to retrieve the photo as stream or expose it as secure web service to be called by cross platform apps in java etc (Damn! am I asking too much!!!)
The file being uploaded will be a *.jpg which is basically a visual signature file created by a user.
Does anyone having any experience working with Active Directory in C# provide some information as to how this can be done with minimum implication related to security.
From the point of view of the Windows Active Directory Administrator what does he have to
do to make this possible.Changes/provisions to schema of user profile etc.
The image is being uploaded so that it can be later retrieved from the AD to be inserted into PDF document for signature purposes.
Can this be done in C#? Or is there any done libraries etc?

Here's a series of blog postings with code that shows how to do it:
(The first shows how to get a photo in, the second shows how to get it out)
Using the jpegPhoto attribute in AD - Part I
Using the jpegPhoto attribute in AD - Part II
EDIT: Here's a generic function implementing the code from Part I:
void AddPictureToUser(
string strDN, // User Distinguished Name, in the form "CN=Joe User,OU=Employees,DC=company,DC=local"
string strDCName, // Domain Controller, ie: "DC-01"
string strFileName // Picture file to open and import into AD
)
{
// Open file
System.IO.FileStream inFile = new System.IO.FileStream(strFileName, System.IO.FileMode.Open, System.IO.FileAccess.Read);
// Retrive Data into a byte array variable
byte[] binaryData = new byte[inFile.Length];
int bytesRead = inFile.Read(binaryData, 0, (int)inFile.Length);
inFile.Close();
// Connect to AD
System.DirectoryServices.DirectoryEntry myUser = new System.DirectoryServices.DirectoryEntry(#"LDAP://" + strDCName + #"/" + strDN);
// Clear existing picture if exists
myUser.Properties["jpegPhoto"].Clear();
// Update attribute with binary data from file
myUser.Properties["jpegPhoto"].Add(binaryData);
myUser.CommitChanges();
}
EDIT: I found that in my organisation, the correct attribute to set was "thumbnailPhoto" like this:
myUser.Properties["thumbnailPhoto"].Add(binaryData);
This also seems to tbe the one that the commercial product Exclaimer is setting (but it might be only doing that in my organization)

The common AD attribute for a user photo is jpegPhoto but you can use what ever name you want
This sample shows the basic AD way to get and set an image stream. You need to flesh these methods out to be a useful class
Consider making your web service to just return the URL of the image. The request handler for that URL should then return the image with the correct content type etc. Much more useful in a web environment
using System;
using System.DirectoryServices;
using System.Collections;
using System.IO;
public class ADPhoto {
public void Set() {
try {
var de = new DirectoryEntry("LDAP://cn=username,cn=users,DC=domain, DC=com");
de.Username = "username";
de.Password = "password";
var forceAuth = de.NativeObject;
var fs = new FileStream("path\\photo.jpg", FileMode.Open);
var br = new BinaryReader(fs);
br.BaseStream.Seek(0, SeekOrigin.Begin);
byte[] ba = new byte[br.BaseStream.Length];
ba = br.ReadBytes((int)br.BaseStream.Length);
de.Properties["jpegPhoto"].Insert(0, ba);
de.CommitChanges();
}
catch(Exception ex) {
Console.WriteLine(ex.Message);
}
}
public Stream Get() {
var fs = new MemoryStream();
try {
var de = new DirectoryEntry("LDAP://cn=username,cn=users,DC=domain, DC=com");
de.Username = "username";
de.Password = "password";
var forceAuth = de.NativeObject;
var wr = new BinaryWriter(fs);
byte[] bb = (byte[])de.Properties["jpegPhoto"][0];
wr.Write(bb);
wr.Close();
}
catch (Exception e) {
Console.WriteLine(e.Message);
}
return fs;
}
}

Found an article that describes how to upload pictures to Active Directory and how to get them to show on the end-users computers.
http://blog.jocha.se/tech/ad-user-pictures-in-windows-10

Each Active Directory User Profile will have a home folder.
If you are not sure about this please checkout the below article
http://support.microsoft.com/kb/816313
I believe that you have to upload the image file to this directory.
Also if this doesn't solve your problem, please update if you find something else.
MNK...