Develop Reference

FTP Error The remote server returned an error: 150 Opening data channel for file download

I have an FTPS site I need to connect to and get a file from. A vendor will be dropping a csv file there daily and I have to retrieve it and process it. My problem is no matter what I try and I can't connect to this site. I realize FTPS is different than SFTP and according to my research my normal method of getting files from FTP should work simply by adding an EnableSsl flag as seen below (ip, port, credentials have been changed obviously):
string uri = "ftp://127.0.0.1:123/";
string filename = "remoteFile.txt";
uri += filename;
var request = (FtpWebRequest)WebRequest.Create(uri);
request.Credentials = new NetworkCredential("user1", "secure-password1");
request.EnableSsl = true;
request.KeepAlive = false;
request.UseBinary = true;
request.UsePassive = true;
ServicePointManager.ServerCertificateValidationCallback = (s, certificate, chain, sslPolicyErrors) => true;
request.Method = WebRequestMethods.Ftp.DownloadFile;
var response = (FtpWebResponse)request.GetResponse(); //<-- error here
Stream responseStream = response.GetResponseStream();
var reader = new StreamReader(responseStream);
var fileContents = reader.ReadToEnd();
reader.Close();
response.Close();
string filePath = #"C:\Temp\localFile.txt";
using var stream = new StreamWriter(filePath);
{
stream.Write(fileContents);
}
I've tried variations of the 4 booleans I set on the request object. In this configuration I get the error in the title. If I switch passive to false I get a timeout. I can connect to this FTP site using WinSCP. There is a certificate on the site and I imported my connection configuration from a co-worker. There is an SHA-1 fingerprint.
I have also tried creating a connection with the WinSCP Nuget package and followed their example, I just can't seem to get the fingerprint correct:
var options = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "ftp://127.0.0.1:21/",
UserName = "user1",
Password = "secure-password1",
SshHostKeyFingerprint = "???",
};
using var session = new WinSCP.Session();
session.Open(options);
No matter what I've tried in that finger print property it doesn't match the pattern they want and I can't find a good example of what it should look like. On the WinSCP page it says to obtain the fingerprint from your administrator, ours provided a certificate file that has an RSA section and a Certificate section. I've tried assigning the whole file to that field, the RSA section, certificate, nothing works. I tried the fingerprint displayed in my working session from WinSCP and that doesn't work.
I've found a few questions on this site with this error but all seem to point to server issues. I figure if I can connect and get files using WinSCP then I should be able to do it through code as well.
thanks

As Martin suggested I opened WinSCP, logged into my FTP site using the app. Then I clicked the Session menu and chose Generate URL/Code... Click the .NET assembly code tab at the top and pasted the code into my project
// Set up session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "127.0.0.1",
PortNumber = 21,
UserName = "user1",
Password = "secure-password1",
FtpSecure = FtpSecure.Explicit,
TlsHostCertificateFingerprint = "2f:f5:ab:e5:f7:27:65:12:30:73:3d:9a:b7:12:88:11:62:0e:6f:a1",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// Your code
}
I hope this helps whoever is stuck on this like I was.

Recursive upload to FTP server in C#

I would need to upload a folder (which contains sub folders and files) from one server to another from C# code. I have done few research and found that we can achieve this using FTP. But with that I am able to move only files and not the entire folder. Any help here is appreciated.

The FtpWebRequest (nor any other FTP client in .NET framework) indeed does not have any explicit support for recursive file operations (including uploads). You have to implement the recursion yourself:
List the local directory
Iterate the entries, uploading files and recursing into subdirectories (listing them again, etc.)
void UploadFtpDirectory(
string sourcePath, string url, NetworkCredential credentials)
{
IEnumerable<string> files = Directory.EnumerateFiles(sourcePath);
foreach (string file in files)
{
using (WebClient client = new WebClient())
{
Console.WriteLine($"Uploading {file}");
client.Credentials = credentials;
client.UploadFile(url + Path.GetFileName(file), file);
}
}
IEnumerable<string> directories = Directory.EnumerateDirectories(sourcePath);
foreach (string directory in directories)
{
string name = Path.GetFileName(directory);
string directoryUrl = url + name;
try
{
Console.WriteLine($"Creating {name}");
FtpWebRequest requestDir =
(FtpWebRequest)WebRequest.Create(directoryUrl);
requestDir.Method = WebRequestMethods.Ftp.MakeDirectory;
requestDir.Credentials = credentials;
requestDir.GetResponse().Close();
}
catch (WebException ex)
{
FtpWebResponse response = (FtpWebResponse)ex.Response;
if (response.StatusCode ==
FtpStatusCode.ActionNotTakenFileUnavailable)
{
// probably exists already
}
else
{
throw;
}
}
UploadFtpDirectory(directory, directoryUrl + "/", credentials);
}
}
For the background of complicated code around creating the folders, see:
How to check if an FTP directory exists
Use the function like:
string sourcePath = #"C:\source\local\path";
// root path must exist
string url = "ftp://ftp.example.com/target/remote/path/";
NetworkCredential credentials = new NetworkCredential("username", "password");
UploadFtpDirectory(sourcePath, url, credentials);
A simpler variant, if you do not need a recursive upload:
Upload directory of files to FTP server using WebClient
Or use FTP library that can do the recursion on its own.
For example with WinSCP .NET assembly you can upload whole directory with a single call to the Session.PutFilesToDirectory:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "ftp.example.com",
UserName = "username",
Password = "password",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// Download files
session.PutFilesToDirectory(
#"C:\source\local\path", "/target/remote/path").Check();
}
The Session.PutFilesToDirectory method is recursive by default.
(I'm the author of WinSCP)

Create Excel file directly on FTP with C# using EPPlus library

I want to create an Excel file on FTP server. I've tried creating a file locally and it works so to the problem: How do I create it in FTP instead of local drive?
ExcelPkg.SaveAs(
new FileInfo(#"C:\ExcelTest\" + DateTime.Now.ToString("yyyy-MM-dd--hh-mm-ss") + ".xlsx"))
And I want to create this file directly in FTP and NOT locally and then move it.

Use ExcelPackage.SaveAs overload that accepts a Stream and pass it an FTP request stream, as for example shown in:
Upload a streamable in-memory document (.docx) to FTP with C#?
I cannot test EPPlus right now, but this should work:
WebRequest request = WebRequest.Create("ftp://ftp.example.com/remote/path/sheet.xlsx");
request.Method = WebRequestMethods.Ftp.UploadFile;
request.Credentials = new NetworkCredential(username, password);
using (Stream ftpStream = request.GetRequestStream())
{
ExcelPkg.SaveAs(ftpStream);
}

I had to make it 2 different operations first one creates the Excel file and then this, uploads existing file and remove it afterward.
Using a library called WinSCP
I couldn't find any library that supports 90s Encryptions like Implicit FTP over TLS and streaming API at the same time. So there was no way for me to directly create the excel file on the server but to temporary create it locally to copy it over and remove it at the same time, it is almost the same thing I get the same result. And here's solution for doing it
try
{
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = #"xx.xx.x.x",
UserName = "xxxx",
Password = "xxxxxx",
PortNumber = xxxx,
FtpSecure = FtpSecure.Implicit, //Encryption protocol
GiveUpSecurityAndAcceptAnyTlsHostCertificate = true // Accepts any certificate
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// Upload files
TransferOptions transferOptions = new TransferOptions();
transferOptions.TransferMode = TransferMode.Binary;
TransferOperationResult transferResult;
// Copy's Existing file to Connected server and delets the old one.
// change by replace "true" with "false".
transferResult =
session.PutFiles(
ExistingPath, "/ScheduleJobs/" + RemotePath, true, transferOptions);
// Throw on any error
transferResult.Check();
// Print results
foreach (TransferEventArgs transfer in transferResult.Transfers)
{
Console.WriteLine("Upload of {0} succeeded", transfer.FileName);
Console.ReadLine();
}
}
}
catch (Exception e)
{
Console.WriteLine("Error: {0}", e);
}

AWS S3 bucket Temporary Credentials download file .Net

I have a bucket on Amazon S3 and I have created IAM user Now I want to download private bucket file using temporary credential.
This is my bucket policy
{
"Id": "Policy1509026195925",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1509026179419",
"Action": [
"s3:GetObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::test-folder/*",
"Principal": {
"AWS": [
"arn:aws:iam::461567291450:user/john"
]
}
}
]
}
this is my c# .Net code
ServicePointManager.Expect100Continue = false;
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
try
{
// In real applications, the following code is part of your trusted code. It has
// your security credentials you use to obtain temporary security credentials.
AmazonSecurityTokenServiceConfig config = new AmazonSecurityTokenServiceConfig();
AmazonSecurityTokenServiceClient stsClient =
new AmazonSecurityTokenServiceClient(config);
GetFederationTokenRequest federationTokenRequest =
new GetFederationTokenRequest();
federationTokenRequest.Name = "testuser";
// federationTokenRequest.Policy = "Policy1509026195925";
federationTokenRequest.DurationSeconds = 7200;
GetFederationTokenResponse federationTokenResponse = stsClient.GetFederationToken(federationTokenRequest);
//FederatedUser federationTokenResult = federationTokenResponse.;
Credentials credentials = federationTokenResponse.Credentials;
SessionAWSCredentials sessionCredentials =
new SessionAWSCredentials(credentials.AccessKeyId,
credentials.SecretAccessKey,
credentials.SessionToken);
// The following will be part of your less trusted code. You provide temporary security
// credentials so it can send authenticated requests to Amazon S3.
// Create Amazon S3 client by passing in the basicSessionCredentials object.
AmazonS3Client s3Client = new AmazonS3Client(sessionCredentials, Amazon.RegionEndpoint.USEast1);
// Test. For example, send list object keys in a bucket.
ListObjectsRequest listObjectRequest = new ListObjectsRequest();
listObjectRequest.BucketName = bucketName;
ListObjectsResponse response = s3Client.ListObjects(listObjectRequest);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
Every time when I run the code I got Access denied message. Why? How to download the bucket file using Temporary credential?

You can try something like :
assumeRoleResult = AssumeRole(role-arn);
tempCredentials = new SessionAWSCredentials(
assumeRoleResult.AccessKeyId,
assumeRoleResult.SecretAccessKey,
assumeRoleResult.SessionToken);
s3Request = CreateAmazonS3Client(tempCredentials);
You need to to call AssumeRole to get temporary security credentials, and then use those credentials to make a call to Amazon S3, see Switching to an IAM Role (API).
Refer : Using Temporary Security Credentials with the AWS SDKs

Error in uploading file on S3 when bucket name containing periods(dots) through c# SDK

I have created two bucket on S3, named like "demobucket" and "demo.bucket".
When I am uploading any file on "demobucket" it works fine. But when I upload file on "demo.bucket", it gives me an error "Maximum number of retry attempts reached : 3"
My concern is that what is the problem in uploading file when bucket name contain periods(dots).
My code is:
public static bool UploadResumeFileToS3(string uploadAsFileName, Stream ImageStream, S3CannedACL filePermission, S3StorageClass storageType)
{
try
{
AmazonS3 client = Amazon.AWSClientFactory.CreateAmazonS3Client(MY_AWS_ACCESS_KEY_ID, MY_AWS_SECRET_KEY);
PutObjectRequest request = new PutObjectRequest();
request.WithKey(uploadAsFileName);
request.WithInputStream(ImageStream);
request.WithBucketName("demo.bucket");
request.CannedACL = filePermission;
request.StorageClass = storageType;
client.PutObject(request);
client.Dispose();
}
catch
{
return false;
}
return true;
}

There is a problem establishing a secure connection to S3 when the bucket name contains a period. The issue is explained well here: http://shlomoswidler.com/2009/08/amazon-s3-gotcha-using-virtual-host.html.
One solution is to create your S3 client passing a third argument which causes it to use HTTP instead of HTTPS. See Amazon S3 C# SDK "The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel." Error.
AmazonS3Config S3Config = new AmazonS3Config()
{
ServiceURL = "s3.amazonaws.com",
CommunicationProtocol = Protocol.HTTP,
RegionEndpoint = region
};
However, be aware that this is not secure and Amazon does not recommend it. Your secret access key could possibly be intercepted. I have not yet found a secure way to upload a file to a bucket with a period in the name.
If you are using a recent version of AWSSDK.dll (at least 2.0.13.0), you can do this instead:
AmazonS3Config S3Config = new AmazonS3Config()
{
ServiceURL = "s3.amazonaws.com",
ForcePathStyle = true,
RegionEndpoint = region
};
This forces the S3Client to use the version of the path to your bucket which avoids the problem.