Description
Download multiple files using webclient's DownloadFileAsync and utilizing a text file for URL input for download.
Problem
The approach that I have used won't download files at all. Just runs and does nothing. It fills the list array then quits the program without downloading a single file. I have googled for solutions but come up shorthanded. Then attempted to search for a solution in the database here with same results. Any help is appreciated.
Questions
Why does this approach not work?
What can I do to improve this and learn from this.
Code
DownloadClass.cs
using System;
using System.ComponentModel;
using System.Collections.Generic;
using System.Net;
using System.Threading;
using System.Windows.Forms;
namespace ThreadTest
{
class DownloadClass
{
public struct download
{
public static string URL { get; set; }
public static string file { get; set; }
public static string[] link;
public static int downloadcount;
}
public static List<string> list = new List<string>();
public static WebClient wc = new WebClient();
public static void Download()
{
int count = 0;
download.URL = list[0];
Uri URI = new Uri(download.URL);
UriBuilder uri = new UriBuilder(URI);
download.link = uri.Path.ToLower().Split(new char[] { '/' });
count = 0;
// Find file
foreach (string abs in download.link)
{
count++;
if (abs.ToLower().Contains(".html") || abs.ToLower().Contains(".exe") || abs.ToLower().Contains(".txt"))
{
try
{
download.file = download.link[count];
wc.Proxy = null;
wc.DownloadFileCompleted += new AsyncCompletedEventHandler(wc_DownloadFileCompleted);
wc.DownloadFileAsync(URI, Application.StartupPath + "\\" + download.file);
break;
}
catch (Exception)
{ }
}
}
}
public static void BeginDownload()
{
new Thread(Download).Start();
}
public static void wc_DownloadFileCompleted(object sender, AsyncCompletedEventArgs e)
{
int count = 0;
download.downloadcount++;
download.URL = list[0];
Uri URI = new Uri(download.URL);
UriBuilder uri = new UriBuilder(URI);
download.link = uri.Path.ToLower().Split(new char[] { '/' });
count = 0;
// Find file
foreach (string abs in download.link)
{
count++;
if (abs.ToLower().Contains(".html") || abs.ToLower().Contains(".exe") || abs.ToLower().Contains(".txt"))
{
try
{
download.file = download.link[count];
}
catch (Exception)
{ }
}
}
list.RemoveAt(0);
if (list.Count > 0)
{
wc.DownloadFileAsync(URI, list[download.downloadcount], Application.StartupPath + "\\" + download.file);
}
else
{
Console.WriteLine("Downloading is done.");
Environment.Exit(0);
}
}
}
}
Program.cs (Main Class)
using System;
using System.IO;
using System.Collections.Generic;
using System.Windows.Forms;
namespace ThreadTest
{
class Program
{
static void Main(string[] args)
{
if (args.Length < 1)
{
Console.WriteLine("Usage: {0} <download txtfile>", Environment.GetCommandLineArgs()[0]);
Environment.Exit(0);
}
int counter = 0;
string line;
string format = string.Format("{0}\\{1}", Application.StartupPath, args[0]);
// Read the file line by line.
using(StreamReader file = new StreamReader(format))
{
while ((line = file.ReadLine())!= null)
{
// Store urls in a list.
DownloadClass.list.Add(line);
counter++;
}
}
DownloadClass.BeginDownload();
}
}
}
Besides being bad design there are lots of issues that lead to your code not (or nor correctly working).
You need to make sure that you application lives while it downloads something. Your current app quits right away (you have to wait for the downloading to complete in your main).
You application may download the same file multiple times but not download others at all (You need to completely lock object when they are used in an async=multithreading way like here when accessing static objects) BTW: Don't use static objects at all to avoid that in the first place.
Even if 2 is corrected it may still download the same file multiple times into the same filename and thus fail.
As long as you have no knowledge about multithreading I'd recommend you use the synchoneous methods to avoid all those problems.
Related
I am trying to save data locally to my device app folder.
When I try to save collected data on an actual Android smartphone, it doesn't work. It is limited by name and filetype, as I cannot change it from test.txt and it is limited in string length, as a maximum of twelve characters get saved.
I have the acquired the following permissions:
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
This is my code:
MainPage.xaml.cs
private async void ExportData(object sender, EventArgs e)
{
var items = await App.Database.GetDataAsync();
DependencyService.Get<IFileService>().CreateFile(items);
}
Interface
using System;
using System.Collections.Generic;
using System.Text;
namespace LocationApp.Interface
{
public interface IFileService
{
void CreateFile(List<LocationData> items);
}
}
Service
using Android.App;
using LocationApp.Droid;
using LocationApp.Interface;
using System.Collections.Generic;
using System.IO;
[assembly:Xamarin.Forms.Dependency(typeof(FileService))]
namespace LocationApp.Droid
{
public class FileService : IFileService
{
public string GetRootPath()
{
return Application.Context.GetExternalFilesDir(null).ToString();
}
public void CreateFile(List<LocationData> items)
{
var fileName = "test-file.txt";
var destination = Path.Combine(GetRootPath(), fileName);
string[] text = new string[items.Count];
for (int i = 0; i < text.Length; i++)
{
text[i] = $"{items[i].Latitude},{items[i].Longitude},{items[i].Day},{items[i].Time}";
}
File.WriteAllLines(destination, text);
}
}
}
I also attempted to see what would happen to an emulator, I used a Pixel 2 with Android 9.0, API 28 where I got the following error:
[ContextImpl] Failed to ensure /storage/120E-0B1B/Android/data/com.companyname.locationapp/files: java.lang.IllegalStateException: Failed to prepare /storage/120E-0B1B/Android/data/com.companyname.locationapp/files/: android.os.ServiceSpecificException: (code -13)
In the end, I only care about putting all my data in a single file. The filename or the error on my emulator I provided in case the error is based on that. If not, I do not care if they are fixed/fixable.
Based on your code, I created a simple demo, and it works on my android emulator(android 11) .
You can test on your side.
The code is:
public void CreateFile(List<LocationData> items)
{
var fileName = "test-file.txt";
var documentsPath = Environment.GetFolderPath(Environment.SpecialFolder.Personal);
var destination = Path.Combine(documentsPath, fileName);
string[] text = new string[items.Count];
for (int i = 0; i < text.Length; i++)
{
text[i] = $"{items[i].Latitude},{items[i].Longitude}";
}
File.WriteAllLines(destination, text);
}
And after I saved the data,I could get the saved data by the following code(the filename is test-file.txt):
public string ReadData(string filename)
{
var documentsPath = Environment.GetFolderPath(Environment.SpecialFolder.Personal);
var filePath = Path.Combine(documentsPath, filename);
return File.ReadAllText(filePath);
}
I'm not so expert on C# but i tried to do my best ,
this is a small console application that should get 2 arg2 ( by passing args to the exe or by console input) i have 2 issue with it and i can't find any other solution
Merging the files is not in the correct order, if the files name has
letters included. ex.
(1.docx , 2.docx , 3.docx ) it work => result.docx(1,2,3)
(1test.docx , 2rice.docx , 3john.docx ) => result.docx(3,1,2)
Can't get WINWORD.exe to close after its the appliaction is
completed
PS: This exe is being called by PHP line CMD.EXE
i tried all possible commands to release com objects then close application ,
what is my mistake here? how to optimize the code correctly?
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Office.Interop.Word;
using System.Reflection;
using System.Runtime.InteropServices;
using System.IO;
namespace DocEngine
{
public class Parameter
{
public string Name;
public string Value;
// Note we need to give a default constructor when override it
public Parameter()
{
}
}
class Program
{
public static void MergeDocuments(string fileName, List<string> documentFiles)
{
_Application oWord = new Microsoft.Office.Interop.Word.Application();
_Document oDoc = oWord.Documents.Add();
Selection oSelection = oWord.Selection;
foreach (string documentFile in documentFiles)
{
_Document oCurrentDocument = oWord.Documents.Add(documentFile);
CopyPageSetup(oCurrentDocument.PageSetup, oDoc.Sections.Last.PageSetup);
oCurrentDocument.Range().Copy();
oSelection.PasteAndFormat(WdRecoveryType.wdFormatOriginalFormatting);
if (!Object.ReferenceEquals(documentFile, documentFiles.Last()))
oSelection.InsertBreak(WdBreakType.wdSectionBreakNextPage);
}
oDoc.SaveAs(fileName, WdSaveFormat.wdFormatDocumentDefault);
oDoc.Close();
//TODO: release objects, close word application
//Marshal.ReleaseComObject(oSelection);
//Marshal.ReleaseComObject(oDoc);
//Marshal.ReleaseComObject(oWord);
Marshal.FinalReleaseComObject(oSelection);
Marshal.FinalReleaseComObject(oDoc);
Marshal.FinalReleaseComObject(oWord);
System.Runtime.InteropServices.Marshal.FinalReleaseComObject(oWord);
System.Runtime.InteropServices.Marshal.ReleaseComObject(oSelection);
System.Runtime.InteropServices.Marshal.ReleaseComObject(oDoc);
System.Runtime.InteropServices.Marshal.ReleaseComObject(oWord);
oSelection = null;
oDoc = null;
oWord = null;
// A good idea depending on who you talk to...
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
GC.WaitForPendingFinalizers();
}
public static void CopyPageSetup(PageSetup source, PageSetup target)
{
target.PaperSize = source.PaperSize;
//target.Orientation = source.Orientation; //not working in word 2003, so here is another way
if (!source.Orientation.Equals(target.Orientation))
target.TogglePortrait();
target.TopMargin = source.TopMargin;
target.BottomMargin = source.BottomMargin;
target.RightMargin = source.RightMargin;
target.LeftMargin = source.LeftMargin;
target.FooterDistance = source.FooterDistance;
target.HeaderDistance = source.HeaderDistance;
target.LayoutMode = source.LayoutMode;
}
static void Main(string[] args)
{
if (args == null || args.Length == 0)
{
Console.WriteLine("Enter Path:");
Parameter parameter1 = new Parameter
{
Name = "dir",
Value = Console.ReadLine()
};
Console.WriteLine("FileName without ext:");
Parameter parameter2 = new Parameter
{
Name = "fileName",
Value = Console.ReadLine()
};
Console.WriteLine("Thank you! ");
Console.WriteLine("Test parameter1: [{0}] = [{1}]", parameter1.Name, parameter1.Value);
Console.WriteLine("Test parameter2: [{0}] = [{1}]", parameter2.Name, parameter2.Value);
try
{
List<string> result = Directory.EnumerateFiles(parameter1.Value, "*.doc", SearchOption.AllDirectories).Union(Directory.EnumerateFiles(parameter1.Value, "*.docx", SearchOption.AllDirectories)).ToList();
var filename = Path.Combine(parameter1.Value, parameter2.Value);
MergeDocuments(filename, result);
}
catch (UnauthorizedAccessException UAEx)
{
Console.WriteLine(UAEx.Message);
}
catch (PathTooLongException PathEx)
{
Console.WriteLine(PathEx.Message);
}
}
else
{
//args = new string[2];
string sourceDirectory = args[0];
string filename1 = args[1];
try
{
List<string> result = Directory.EnumerateFiles(sourceDirectory, "*.doc", SearchOption.AllDirectories).Union(Directory.EnumerateFiles(sourceDirectory, "*.docx", SearchOption.AllDirectories)).ToList();
var filename = Path.Combine(sourceDirectory, filename1);
MergeDocuments(filename, result);
}
catch (UnauthorizedAccessException UAEx)
{
Console.WriteLine(UAEx.Message);
}
catch (PathTooLongException PathEx)
{
Console.WriteLine(PathEx.Message);
}
}
}
}
}
You don't need to make any of those COM calls or explicit GC calls, and you don't need to explicitly set things to null. You just need to call Application.Quit.
I'm trying to transfer data from a treenode (at least I think that's what it is) which contains much more data than I need. It would be very difficult for me to manipulate the data within the treenode. I would much rather have an array which provides me with only the necessary data for data manipulation.
I would like higher rates have following variables:
1. BookmarkNumber (integer)
2. Date (string)
3. DocumentType (string)
4. BookmarkPageNumberString (string)
5. BookmarkPageNumberInteger (integer)
I would like to the above defined rate from the data from variable book_mark (as can be seen in my code).
I've been wrestling with this for two days. Any help would be much appreciated. I'm probably sure that the question wasn't phrased correctly so please ask questions so that I may explain further if needed.
Thanks so much
BTW what I'm trying to do is create a Windows Form program which parses a PDF file which has multiple bookmarks into discrete PDF files for each bookmark/chapter while saving the bookmark in the correct folder with the correct naming convention, the folder and naming convention dependent upon the PDF name and title name of the bookmark/chapter being parsed.
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using System.IO;
using itextsharp.pdfa;
using iTextSharp.awt;
using iTextSharp.testutils;
using iTextSharp.text;
using iTextSharp.xmp;
using iTextSharp.xtra;
namespace WindowsFormsApplication1
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void ChooseImageFileWrapper_Click(object sender, EventArgs e)
{
OpenFileDialog openFileDialog1 = new OpenFileDialog();
openFileDialog1.InitialDirectory = GlobalVariables.InitialDirectory;
openFileDialog1.Filter = "Pdf Files|*.pdf";
openFileDialog1.RestoreDirectory = true;
openFileDialog1.Title = "Image File Wrapper Chooser";
if (openFileDialog1.ShowDialog() == DialogResult.OK)
{
try
{
GlobalVariables.ImageFileWrapperPath = openFileDialog1.FileName;
}
catch (Exception ex)
{
MessageBox.Show("Error: Could not read file from disk. Original error: " + ex.Message);
}
}
ImageFileWrapperPath.Text = GlobalVariables.ImageFileWrapperPath;
}
private void ImageFileWrapperPath_TextChanged(object sender, EventArgs e)
{
}
private void button2_Click(object sender, EventArgs e)
{
iTextSharp.text.pdf.PdfReader pdfReader = new iTextSharp.text.pdf.PdfReader(GlobalVariables.ImageFileWrapperPath);
IList<Dictionary<string, object>> book_mark = iTextSharp.text.pdf.SimpleBookmark.GetBookmark(pdfReader);
List<ImageFileWrapperBookmarks> IFWBookmarks = new List<ImageFileWrapperBookmarks>();
foreach (Dictionary<string, object> bk in book_mark) // bk is a single instance of book_mark
{
ImageFileWrapperBookmarks.BookmarkNumber = ImageFileWrapperBookmarks.BookmarkNumber + 1;
foreach (KeyValuePair<string, object> kvr in bk) // kvr is the key/value in bk
{
if (kvr.Key == "Kids" || kvr.Key == "kids")
{
//create recursive program for children
}
else if (kvr.Key == "Title" || kvr.Key == "title")
{
}
else if (kvr.Key == "Page" || kvr.Key == "page")
{
}
}
}
MessageBox.Show(GlobalVariables.ImageFileWrapperPath);
}
}
}
Here's one way to parse a PDF and create a data structure similar to what you describe. First the data structure:
public class BookMark
{
static int _number;
public BookMark() { Number = ++_number; }
public int Number { get; private set; }
public string Title { get; set; }
public string PageNumberString { get; set; }
public int PageNumberInteger { get; set; }
public static void ResetNumber() { _number = 0; }
// bookmarks title may have illegal filename character(s)
public string GetFileName()
{
var fileTitle = Regex.Replace(
Regex.Replace(Title, #"\s+", "-"),
#"[^-\w]", ""
);
return string.Format("{0:D4}-{1}.pdf", Number, fileTitle);
}
}
A method to create a list of Bookmark (above):
List<BookMark> ParseBookMarks(IList<Dictionary<string, object>> bookmarks)
{
int page;
var result = new List<BookMark>();
foreach (var bookmark in bookmarks)
{
// add top-level bookmarks
var stringPage = bookmark["Page"].ToString();
if (Int32.TryParse(stringPage.Split()[0], out page))
{
result.Add(new BookMark() {
Title = bookmark["Title"].ToString(),
PageNumberString = stringPage,
PageNumberInteger = page
});
}
// recurse
if (bookmark.ContainsKey("Kids"))
{
var kids = bookmark["Kids"] as IList<Dictionary<string, object>>;
if (kids != null && kids.Count > 0)
{
result.AddRange(ParseBookMarks(kids));
}
}
}
return result;
}
Call method above like this to dump the results to a text file:
void DumpResults(string path)
{
using (var reader = new PdfReader(path))
{
// need this call to parse page numbers
reader.ConsolidateNamedDestinations();
var bookmarks = ParseBookMarks(SimpleBookmark.GetBookmark(reader));
var sb = new StringBuilder();
foreach (var bookmark in bookmarks)
{
sb.AppendLine(string.Format(
"{0, -4}{1, -100}{2, -25}{3}",
bookmark.Number, bookmark.Title,
bookmark.PageNumberString, bookmark.PageNumberInteger
));
}
File.WriteAllText(outputTextFile, sb.ToString());
}
}
The bigger problem is how to extract each Bookmark into a separate file. If every Bookmark starts a new page it's easy:
Iterate over the return value of ParseBookMarks()
Select a page range that begins with the current BookMark.Number, and ends with the next BookMark.Number - 1
Use that page range to create separate files.
Something like this:
void ProcessPdf(string path)
{
using (var reader = new PdfReader(path))
{
// need this call to parse page numbers
reader.ConsolidateNamedDestinations();
var bookmarks = ParseBookMarks(SimpleBookmark.GetBookmark(reader));
for (int i = 0; i < bookmarks.Count; ++i)
{
int page = bookmarks[i].PageNumberInteger;
int nextPage = i + 1 < bookmarks.Count
// if not top of page will be missing content
? bookmarks[i + 1].PageNumberInteger - 1
/* alternative is to potentially add redundant content:
? bookmarks[i + 1].PageNumberInteger
*/
: reader.NumberOfPages;
string range = string.Format("{0}-{1}", page, nextPage);
// DEMO!
if (i < 10)
{
var outputPath = Path.Combine(OUTPUT_DIR, bookmarks[i].GetFileName());
using (var readerCopy = new PdfReader(reader))
{
var number = bookmarks[i].Number;
readerCopy.SelectPages(range);
using (FileStream stream = new FileStream(outputPath, FileMode.Create))
{
using (var document = new Document())
{
using (var copy = new PdfCopy(document, stream))
{
document.Open();
int n = readerCopy.NumberOfPages;
for (int j = 0; j < n; )
{
copy.AddPage(copy.GetImportedPage(readerCopy, ++j));
}
}
}
}
}
}
}
}
}
The problem is that it's highly unlikely all bookmarks are going to be at the top of every page of the PDF. To see what I mean, experiment with commenting / uncommenting the bookmarks[i + 1].PageNumberInteger lines.
I'm running a windows application(x86) platform in a windows server 2012(x64) and I'm using a SubSonic to get data from the SQLserver.
And I got this error from try catch Exception
at SubSonic.SqlQuery.ExecuteAsCollectionListType at
CIS.Server.Automailer.Core.ReportConfig.getReportConfig() at
CIS.Server.Automailer.Automailer.processDownload() at
CIS.Server.Automailer.Program.Main()
Here's my source code:
Program.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Windows.Forms;
using System.IO;
namespace CIS.Server.Automailer
{
static class Program
{
[STAThread]
static void Main()
{
try
{
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
Automailer mailer = new Automailer();
mailer.processDownload();
Application.Run();
}
catch(Exception ex)
{
Log(ex);
}
}
static void Log(Exception ex)
{
string stackTrace = ex.StackTrace;
File.WriteAllText("trace.txt", stackTrace); // path of file where stack trace will be stored.
}
}
}
Automailer.cs
public void processDownload()
{
var data = ReportConfig.getReportConfig();
var machineCenterIds = data[0].MachineCenterId;
var reportIds = data[0].ReportId;
var email = data[0].Email;
recipient = email;
string[] splitRepId = reportIds.Split(',');
int[] repIds = new int[splitRepId.Length];
int c=0;
foreach (string repId in splitRepId)
{
repIds[c] = Convert.ToInt32(repId);
c++;
}
var reportNames = ReportConfig.getReportName(repIds);
string[] splitMcIds = machineCenterIds.Split(',');
int ctr = 0;
filePaths = new string[splitMcIds.Length * reportNames.Count()];
ei.processFinished = false;
foreach (string mcId in splitMcIds)
{
for (int i = 0; i < reportNames.Count(); i++)
{
string reportName = Convert.ToString(reportNames[i]);
string url = Utility.GetTemporaryURL(mcId, reportName);
string fileName = reportName.Replace(" ", "");// + "_" + j + "_00" + i
downloadPath = string.Format(configPath, mcId, fileName);
filePaths[ctr] = downloadPath;
GenerateMails(url);
Console.WriteLine(downloadPath);
ctr++;
}
}
processEmail();
ei.processFinished = true;
}
ReportConfig.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace CIS.Server.Automailer.Core
{
class ReportConfig
{
public static ReportAutomationCollection getReportConfig()
{
return DB.Select().From(ReportAutomation.Schema)
.Where(ReportAutomation.Columns.UserId).IsEqualTo("001111d6-cc2a-469a-a1bc-1ccd64e60a08")
.ExecuteAsCollection<ReportAutomationCollection>();
}
public static ReportTypeCollection getReportName(int[] reportId)
{
return DB.Select(ReportType.Columns.ReportName).From(ReportType.Schema)
.Where(ReportType.ReportIdColumn).In(reportId.ToString())
.ExecuteAsCollection<ReportTypeCollection>();
}
}
}
Seems like you have a problem with your database. since your access now is on the server.
check your data if both exist on your local and on the Server.
The following code always shows path manipulation problem. How to resolve it ?
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Text.RegularExpressions;
namespace PathManipulation
{
class Program
{
public string dir = null;
public void someFunction(string fileName)
{
// File.Delete(Regex.Replace(dir + fileName, #"\..\", String.Empty));
if (!(dir.IndexOf("//") >= 0) || !Regex.IsMatch(dir, "System32"))
{
String p = Regex.Replace(dir, #"..\", string.Empty);
DirectoryInfo di = new DirectoryInfo(p);
FileInfo[] fi = di.GetFiles();
if (fi.Length > 0)
{
for (int i = 0; i < fi.Length; i++)
{
if (fi[i].ToString().Equals(fileName))
{
Console.WriteLine(fi[i].ToString());
fi[i].Delete();
}
}
File.Delete(dir + fileName);
}
}
else
{
return;
}
}
static void Main(string[] args)
{
Program p = new Program();
p.dir = args[0];
p.someFunction(args[1]);
}
}
}
Yes, you break the flow of data so that the end user is not able to specify the file to be deleted.
For instance:
public void someFunction(int fileIndex){
...
if (fileIndex == 0){
File.Delete( "puppies.txt" );
}
else if (fileIndex == 1){
File.Delete( "kittens.txt" );
}
else {
throw new IllegalArgumentException( "Invalid delete index" );
}
}
That's an extreme way to solve the problem, but it does not allow the end user to delete ANYTHING the developer didn't intend.
Your data validation check:
if (!(dir.IndexOf("//") >= 0) || !Regex.IsMatch(dir, "System32"))
is weak. This is referred to as "blacklisting" and the attacker only has to figure out a pattern that is missed by your checks. So, #"C:\My Documents" for instance.
Instead, you should consider a "whitelisting" approach. Take a look at https://www.owasp.org/index.php/Data_Validation#Accept_known_good for a pretty thorough example. It doesn't address path injection directly. You simply have to think hard about what files/directories you expect to receive. Throw an error if the input deviates from that. With a little testing you will create a good whitelist.