I would need to upload a folder (which contains sub folders and files) from one server to another from C# code. I have done few research and found that we can achieve this using FTP. But with that I am able to move only files and not the entire folder. Any help here is appreciated.
The FtpWebRequest (nor any other FTP client in .NET framework) indeed does not have any explicit support for recursive file operations (including uploads). You have to implement the recursion yourself:
List the local directory
Iterate the entries, uploading files and recursing into subdirectories (listing them again, etc.)
void UploadFtpDirectory(
string sourcePath, string url, NetworkCredential credentials)
{
IEnumerable<string> files = Directory.EnumerateFiles(sourcePath);
foreach (string file in files)
{
using (WebClient client = new WebClient())
{
Console.WriteLine($"Uploading {file}");
client.Credentials = credentials;
client.UploadFile(url + Path.GetFileName(file), file);
}
}
IEnumerable<string> directories = Directory.EnumerateDirectories(sourcePath);
foreach (string directory in directories)
{
string name = Path.GetFileName(directory);
string directoryUrl = url + name;
try
{
Console.WriteLine($"Creating {name}");
FtpWebRequest requestDir =
(FtpWebRequest)WebRequest.Create(directoryUrl);
requestDir.Method = WebRequestMethods.Ftp.MakeDirectory;
requestDir.Credentials = credentials;
requestDir.GetResponse().Close();
}
catch (WebException ex)
{
FtpWebResponse response = (FtpWebResponse)ex.Response;
if (response.StatusCode ==
FtpStatusCode.ActionNotTakenFileUnavailable)
{
// probably exists already
}
else
{
throw;
}
}
UploadFtpDirectory(directory, directoryUrl + "/", credentials);
}
}
For the background of complicated code around creating the folders, see:
How to check if an FTP directory exists
Use the function like:
string sourcePath = #"C:\source\local\path";
// root path must exist
string url = "ftp://ftp.example.com/target/remote/path/";
NetworkCredential credentials = new NetworkCredential("username", "password");
UploadFtpDirectory(sourcePath, url, credentials);
A simpler variant, if you do not need a recursive upload:
Upload directory of files to FTP server using WebClient
Or use FTP library that can do the recursion on its own.
For example with WinSCP .NET assembly you can upload whole directory with a single call to the Session.PutFilesToDirectory:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "ftp.example.com",
UserName = "username",
Password = "password",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// Download files
session.PutFilesToDirectory(
#"C:\source\local\path", "/target/remote/path").Check();
}
The Session.PutFilesToDirectory method is recursive by default.
(I'm the author of WinSCP)
Related
Uploading some txt files from a local folder to a specific FTP address (I'm using this, ftp://ftpint/sales/to_system/) is one of my daily routines. I'm using ZappySys for automate this routine, but my company doesn't want to use it anymore, so i think WinSCP could be a good option.
I've installed WinSCP 5.19 & .NET assembly and followed the instructions from this link, https://winscp.net/eng/docs/library_ssis. But I think WinSCP can't recognize my FTP link. Here's my C# code, any suggestions? Thank you.
using System;
using WinSCP;
class Example
{
public static int Main()
{
try
{
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Sftp,
HostName = "xxx",
UserName = "xxx",
Password = "xxx",
SshHostKeyFingerprint = "SHA-256 xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx"
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// Upload files
TransferOptions transferOptions = new TransferOptions();
transferOptions.TransferMode = TransferMode.Binary;
TransferOperationResult transferResult =
session.PutFiles(#"C:\Users\Diomedas\test\*", "ftp://ftpint/sales/to_system/", false, transferOptions);
// Throw on any error
transferResult.Check();
// Print results
foreach (TransferEventArgs transfer in transferResult.Transfers)
{
Console.WriteLine("Upload of {0} succeeded", transfer.FileName);
}
}
return 0;
}
catch (Exception e)
{
Console.WriteLine("Error: {0}", e);
return 1;
}
}
}
The remotePath argument of Session.PutFiles is a remote path. Not any URL. So it should be like:
session.PutFiles(#"C:\Users\Diomedas\test\*", "/sales/to_system/", false, transferOptions);
You have already specified the hostname in SessionOptions.HostName. No point trying to repeat that information.
Your protocols do not match. You have specified Protocol.Sftp in SessionOptions.Protocol, while your URL has ftp://. Make sure you know what is the actual protocol of your server.
WinSCP GUI can generate full working code (including the upload part) for you.
I need to connect to AWS without using C# library i.e. using only HTTP Rest endpoint so is it possible?
The reason I want to do this because I want to give flexibility to customers to connect to any service, in the case of the library I have to utilize the library code to connect to the relevant services.
And can we create an instance of AWS connection once and use that throughout the session instead of passing token or user name & password in headers again and again?
Here is what I tried using C# AWS library and I need to achieve the same using Rest endpoints.
public bool sendMyFileToS3(System.IO.Stream localFilePath, string bucketName, string subDirectoryInBucket, string fileNameInS3)
{
IAmazonS3 client = new AmazonS3Client(RegionEndpoint.USEast1);
TransferUtility utility = new TransferUtility(client);
TransferUtilityUploadRequest request = new TransferUtilityUploadRequest();
if (subDirectoryInBucket == "" || subDirectoryInBucket == null)
{
request.BucketName = bucketName; //no subdirectory just bucket name
}
else
{
// subdirectory and bucket name
request.BucketName = bucketName + #"/" + subDirectoryInBucket;
}
request.Key = fileNameInS3; //file name up in S3
request.InputStream = localFilePath;
request.ContentType = "";
utility.Upload(request); //commensing the transfer
return true; //indicate that the file was sent
}
I want to create an Excel file on FTP server. I've tried creating a file locally and it works so to the problem: How do I create it in FTP instead of local drive?
ExcelPkg.SaveAs(
new FileInfo(#"C:\ExcelTest\" + DateTime.Now.ToString("yyyy-MM-dd--hh-mm-ss") + ".xlsx"))
And I want to create this file directly in FTP and NOT locally and then move it.
Use ExcelPackage.SaveAs overload that accepts a Stream and pass it an FTP request stream, as for example shown in:
Upload a streamable in-memory document (.docx) to FTP with C#?
I cannot test EPPlus right now, but this should work:
WebRequest request = WebRequest.Create("ftp://ftp.example.com/remote/path/sheet.xlsx");
request.Method = WebRequestMethods.Ftp.UploadFile;
request.Credentials = new NetworkCredential(username, password);
using (Stream ftpStream = request.GetRequestStream())
{
ExcelPkg.SaveAs(ftpStream);
}
I had to make it 2 different operations first one creates the Excel file and then this, uploads existing file and remove it afterward.
Using a library called WinSCP
I couldn't find any library that supports 90s Encryptions like Implicit FTP over TLS and streaming API at the same time. So there was no way for me to directly create the excel file on the server but to temporary create it locally to copy it over and remove it at the same time, it is almost the same thing I get the same result. And here's solution for doing it
try
{
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = #"xx.xx.x.x",
UserName = "xxxx",
Password = "xxxxxx",
PortNumber = xxxx,
FtpSecure = FtpSecure.Implicit, //Encryption protocol
GiveUpSecurityAndAcceptAnyTlsHostCertificate = true // Accepts any certificate
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// Upload files
TransferOptions transferOptions = new TransferOptions();
transferOptions.TransferMode = TransferMode.Binary;
TransferOperationResult transferResult;
// Copy's Existing file to Connected server and delets the old one.
// change by replace "true" with "false".
transferResult =
session.PutFiles(
ExistingPath, "/ScheduleJobs/" + RemotePath, true, transferOptions);
// Throw on any error
transferResult.Check();
// Print results
foreach (TransferEventArgs transfer in transferResult.Transfers)
{
Console.WriteLine("Upload of {0} succeeded", transfer.FileName);
Console.ReadLine();
}
}
}
catch (Exception e)
{
Console.WriteLine("Error: {0}", e);
}
Our system has been used to upload millions of files over several years. The clients use the following code to send an authentication token and zip file to our WEB API on Windows Server 2008 R2. On our Windows 7 devices, the system works great. As we are attempting to move to Windows 10 devices, we have suddenly encountered an issue where the received file has blocks of data in a different order than the source file. The problem only occurs about half of the time, which makes it very difficult to track down.
client code (.NET 4.5)
private static void UploadFile(string srcFile, string username, string password)
{
if (File.Exists(srcFile))
{
ConnectionUtilities connUtil = new ConnectionUtilities();
string authToken = connUtil.GetAuthToken(username, password);
using (HttpContent authContent = new StringContent(authToken))
{
using (HttpContent fileStreamContent = new ByteArrayContent(File.ReadAllBytes(srcFile)))
{
FileInfo fi = new FileInfo(srcFile);
using (HttpClient client = new HttpClient())
using (MultipartFormDataContent formData = new MultipartFormDataContent())
{
client.DefaultRequestHeaders.ExpectContinue = false;
formData.Add(authContent, "auth");
formData.Add(fileStreamContent, "data", fi.Name);
var response = client.PostAsync(ConfigItems.hostName + "UploadData", formData).Result;
if (response.IsSuccessStatusCode)
{
File.Delete(srcFile);
}
}
}
}
}
}
WEB API code (.NET 4.5.2)
public async Task<HttpResponseMessage> PostUploadData()
{
if (Request.Content.IsMimeMultipartContent())
{
MultipartFormDataStreamProvider streamProvider =
MultipartFormDataStreamProvider(HttpContext.Current.Server.MapPath("~/app_data"));
await Request.Content.ReadAsMultipartAsync(streamProvider);
string auth = streamProvider.FormData["auth"];
if (auth != null)
{
auth = HttpUtility.UrlDecode(auth);
}
if (Util.IsValidUsernameAndPassword(auth))
{
string username = Util.GetUsername(auth);
foreach (var file in streamProvider.FileData)
{
DirectoryInfo di = new DirectoryInfo(ConfigurationManager.AppSettings["DataRoot"]);
di = di.CreateSubdirectory(username);
string contentFileName = file.Headers.ContentDisposition.FileName;
di = di.CreateSubdirectory("storage");
FileInfo fi = new FileInfo(file.LocalFileName);
string destFileName = Path.Combine(di.FullName, contentFileName);
File.Move(fi.FullName, destFileName);
}
return new HttpResponseMessage(HttpStatusCode.OK);
}
}
return new HttpResponseMessage(HttpStatusCode.ServiceUnavailable);
}
The problem initially manifests as a zipped file that can't open in Windows. Only by doing a hexadecimal compare did we determine that the file was all there, just not in the same order as the original.
Any thoughts on what might be causing the blocks of data to be reordered?
P.S. I know the HttpClient is not being used as effectively as possible.
After some long and tedious testing (Yay, scientific method) we determined that our web content filter software was causing the issue.
I have a simple console application that upload a file from a local folder to a library in sharepoint, and also have a method that download that folder, but using the url that is activated manually in the web site. But I need to download the same file that I upload a second later, it is for a test, AND WHAT I NEED IS TO ACTIVATE THE "VIEW LINK" FOR DOWNLOAD LATER. Here is my upload method:
static void o365SaveBinaryDirect(ClientContext o365Context, string o365LibraryName, string o365FilePath, string o365FileName) {
Web o365Web = o365Context.Web;
if (!LibraryExist(o365Context, o365Web, o365LibraryName)) {
CreateLibrary(o365Context, o365Web, o365LibraryName);
}
using (FileStream o365FileStream = new FileStream(o365FilePath, FileMode.Open)) {
Microsoft.SharePoint.Client.File.SaveBinaryDirect(o365Context, string.Format("/{0}/{1}", o365LibraryName, o365FileName), o365FileStream, true);
}
}
Now I have this method that download:
private static void DownloadFile(string webUrl, ICredentials credentials, string fileRelativeUrl) {
using (var client = new WebClient()) {
client.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f");
client.Headers.Add("User-Agent: Other");
client.Credentials = credentials;
client.DownloadFile(webUrl, fileRelativeUrl);
}
}
I need to generate the url for download the file later.
Some suggestions:
avoid generating file url based on library title since library url is
not the same as library title
since Microsoft.SharePoint.Client.File.SaveBinaryDirect method from CSOM
API is used for uploafding a file, consider to utilize Microsoft.SharePoint.Client.File.OpenBinaryDirect method for downloading a file
The following example demonstrates how to upload a file into a library and then downloading it:
var sourceFilePath = #"c:\in\UserGuide.pdf"; //local file path;
var listTitle = "Documents"; //target library;
var list = ctx.Web.Lists.GetByTitle(listTitle);
ctx.Load(list.RootFolder);
ctx.ExecuteQuery();
var targetFileUrl = string.Format("{0}/{1}", list.RootFolder.ServerRelativeUrl, Path.GetFileName(sourceFilePath));
//upload a file
using (var fs = new FileStream(sourceFilePath, FileMode.Open))
{
Microsoft.SharePoint.Client.File.SaveBinaryDirect(ctx, targetFileUrl, fs, true);
}
//download a file
var downloadPath = #"c:\out\";
var fileInfo = Microsoft.SharePoint.Client.File.OpenBinaryDirect(ctx, targetFileUrl);
var fileName = Path.Combine(downloadPath, Path.GetFileName(targetFileUrl));
using (var fileStream = System.IO.File.Create(fileName))
{
fileInfo.Stream.CopyTo(fileStream);
}