I have Listbox full with links like this: http://example.com/sorted/Avicii/Avicii+-+Wake+Me+Up.mp3 and I wanna download them all on same time any suggestions?
this code I use to download a single file
private void txtUrl_TextChanged(object sender, EventArgs e)
{
try
{
// function that enter the file name automatically in the savefiledialog.
Uri uri = new Uri(txtUrl.Text);
// Save the file name to the string.
filename = Path.GetFileName(uri.LocalPath);
}
catch
{
// no need need an exception message.
}
}
private void DownloadFile(string url, string save)
{
using (var client = new WebClient())
{
// Run code every time the download changes.
client.DownloadProgressChanged += new DownloadProgressChangedEventHandler(Changed);
// Run codes when file download has been completed.
client.DownloadFileCompleted += new AsyncCompletedEventHandler(Completed);
client.DownloadFileAsync(new Uri(url), save);
The solution could look like this:
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
namespace SillyCSharpNameSpace
{
public class VerboseDownloaderClassName
{
private string downloadFile(string url, string localDir)
{
try
{
var localPath = Path.Combine(localDir, Path.GetFileName(url));
using (var wc = new WebClient())
{
wc.DownloadFile(url, localPath);
return localPath;
}
}
catch (Exception e)
{
Console.WriteLine(e);
return null;
}
}
public void DownloadAll(List<string> urls)
{
urls.AsParallel()
.Where(url => !string.IsNullOrWhiteSpace(url))
.WithDegreeOfParallelism(20)
.Select(url => downloadFile(url, "."));
}
}
}
I'm sure if you work with WinForms or ASP.NET you can figure out how to take an url string from a list box item. And note, that Path.GetFileName() works only for the form you have provided - with the file name at the end of URL, without any URL parameters. AsParallel method parallelizes the downloading work into 20 "threads". I think it should be enough for your purpose.
Bonus. This is the same in F#, just because I can ;o)
open System.IO
open System.Net
open FSharp.Collections.ParallelSeq // from NuGet FSharp.Collections.ParallelSeq
let downloadFile dir url =
let localPath = Path.Combine(dir, Path.GetFileName url)
try
use wc = new WebClient()
wc.DownloadFile(url, localPath)
Some localPath
with ex ->
printfn "Can't download file from %s: %A" url ex
None
let downloadAll (urls: string list) =
urls
|> PSeq.withDegreeOfParallelism 20 // 20 threads
|> PSeq.choose (downloadFile ".")
Related
My requirement is the following:
I have various bits of network hardware, from which we regularly export configurations (this is done using another c#/.NET app I created, this works really well - the tool logs in via SSH to the various devices and downloads the latest configuration to the local hard disk of the server from which the app runs).
We have various SharePoint online sites (provided via our M365 subscription).
I want to have these configs uploaded to a particular folder, in a particular document library, in a particular SharePoint Online site, basically so that there is an off-site backup of the config files.
I was following the steps in this blog: https://daoudisamir.com/from-c-to-sharepoint-file-upload-using-csom-with-subfolders-structure/.
It looked like it did what I wanted... but when I try to build the code, it tells me that I'm missing references to assemblies etc. I know that this means that I'm missing "using" directives, but I've tried all sorts of variations to reference the various NUGET packages that are instructed to be installed, but I can't get this to work and couldn't see anywhere on the blog itself to email the owner.
Anyone got any ideas... at the moment, my code is more or less a copy/paste of what's on the blog page, but I've included my code here for convenience:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Configuration;
using System.Security;
using System.IO;
using Microsoft.SharePoint;
using Microsoft.SharePoint.Client;
namespace UploadToSharePoint_Net48
{
internal class Program
{
static void Main(string[] args)
{
string SiteUrl = "https://You.sharepoint.com/sites/Upload";
string DocumentLibrary = "UploadLibrary";
string FileName = #"C:\testupload.pdf";
string CustomerFolder = "1564_dsfgsst";
string UserName = "samir.daoudi#******.co.uk";
string Password = "*****";
UploadFileToSharePoint(SiteUrl, DocumentLibrary, CustomerFolder, FileName, UserName, Password);
}
private static void UploadFileToSharePoint(string SiteUrl, string DocLibrary, string ClientSubFolder, string FileName, string Login, string Password)
{
try
{
#region ConnectToSharePoint
var securePassword = new SecureString();
foreach (char c in Password)
{ securePassword.AppendChar(c); }
var onlineCredentials = new SP.SharePointOnlineCredentials(Login, securePassword);
#endregion
#region Insert the data
using (SP.ClientContext CContext = new SP.ClientContext(SiteUrl))
{
CContext.Credentials = onlineCredentials;
SP.Web web = CContext.Web;
SP.FileCreationInformation newFile = new SP.FileCreationInformation();
byte[] FileContent = System.IO.File.ReadAllBytes(FileName);
newFile.ContentStream = new MemoryStream(FileContent);
newFile.Url = Path.GetFileName(FileName);
SP.List DocumentLibrary = web.Lists.GetByTitle(DocLibrary);
//SP.Folder folder = DocumentLibrary.RootFolder.Folders.GetByUrl(ClientSubFolder);
SP.Folder Clientfolder = DocumentLibrary.RootFolder.Folders.Add(ClientSubFolder);
Clientfolder.Update();
SP.File uploadFile = Clientfolder.Files.Add(newFile);
CContext.Load(DocumentLibrary);
CContext.Load(uploadFile);
CContext.ExecuteQuery();
Console.ForegroundColor = ConsoleColor.Green;
Console.WriteLine("The File has been uploaded" + Environment.NewLine + "FileUrl -->" + SiteUrl + "/" + DocLibrary + "/" + ClientSubFolder + "/" + Path.GetFileName(FileName));
}
#endregion
}
catch (Exception exp)
{
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine(exp.Message + Environment.NewLine + exp.StackTrace);
}
finally
{
Console.ReadLine();
}
}
}
}
The errors are with all the references to SP.xxxxx
I can see that there is no definition for anything called "SP", but I'm assuming at this stage that this is in one of the external libraries or whatever is in the NUGETS that are required.
Any help gratefully received.
Thanks
Colin
We installed nuget titanium web proxy, created a window service and initiated titanium web proxy. The windows service works, runs, and start and stop times are written to a log file. But the web proxy is supposed to catch internet request and afford them, though no such events happens and nothing is logged, when i open some page with different browsers.
Here is our code:
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.ServiceProcess;
using System.Text;
using System.Threading.Tasks;
using Titanium.Web.Proxy;
using Titanium.Web.Proxy.EventArguments;
namespace WebProxy1 {
public partial class MyNewService : ServiceBase {
public ProxyServer proxyServer;
public MyNewService() {
InitializeComponent();
}
protected override void OnStart(string[] args) {
proxyServer = new ProxyServer(true, true, true);
proxyServer.BeforeRequest += OnRequest;
proxyServer.Start();
WriteToFile("Service is started at " + DateTime.Now);
}
protected override void OnStop() {
proxyServer.Stop();
WriteToFile("Service is stopped at " + DateTime.Now);
}
public void WriteToFile(string Message) {
string path = "E:\\Downloads\\Logs";
if (!Directory.Exists(path)) {
Directory.CreateDirectory(path);
}
string filepath = "E:\\Downloads\\Logs\\ServiceLog_" + DateTime.Now.Date.ToShortDateString().Replace('/', '_') + ".txt";
if (!File.Exists(filepath)) {
// Create a file to write to.
using (StreamWriter sw = File.CreateText(filepath)) {
sw.WriteLine(Message);
}
} else {
using (StreamWriter sw = File.AppendText(filepath)) {
sw.WriteLine(Message);
}
}
}
public async Task OnRequest(object sender, SessionEventArgs e) {
WriteToFile(e.HttpClient.Request.Url);
// To cancel a request with a custom HTML content
// Filter URL
if (e.HttpClient.Request.Method.ToUpper() == "GET" && e.HttpClient.Request.RequestUri.AbsoluteUri.Contains("google.com")) {
e.Ok("<!DOCTYPE html>" +
"<html><body><h1>" +
"Website Blocked" +
"</h1>" +
"<p>Blocked by titanium web proxy.</p>" +
"</body>" +
"</html>");
}
// Redirect example
if (e.HttpClient.Request.RequestUri.AbsoluteUri.Contains("wikipedia.org")) {
e.Redirect("https://www.paypal.com");
}
}
}
}
I think you did not set the titanium proxy properly.
Before starting the proxy, you have to set endpoint.
There is titanium proxy using example here.
This is my sample source.
var explicitEndPoint = new ExplicitProxyEndPoint(IPAddress.Any, 8000, true)
{
// Use self-issued generic certificate on all https requests
// Optimizes performance by not creating a certificate for each https-enabled domain
// Useful when certificate trust is not required by proxy clients
//GenericCertificate = new X509Certificate2(Path.Combine(System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location), "genericcert.pfx"), "password")
};
// Fired when a CONNECT request is received
explicitEndPoint.BeforeTunnelConnect += OnBeforeTunnelConnect;
// An explicit endpoint is where the client knows about the existence of a proxy
// So client sends request in a proxy friendly manner
proxyServer.AddEndPoint(explicitEndPoint);
proxyServer.Start();
I've read the tutorial and I'm able to generate the .cs file but it doesn't include any of my service or rpc definitions.
I've added protoc to my PATH and from inside the project directory.
protoc project1.proto --csharp_out="C:\output" --plugin=protoc-gen-grpc="c:\Users\me\.nuget\packages\grpc.tools\1.8.0\tools\windows_x64\grpc_csharp_plugin.exe"
No errors output in console
You need to add the --grpc_out command line option, e.g. add
--grpc_out="C:\output\"
Note that it won't write any files if you don't have any services.
Here's a complete example. From a root directory, create:
An empty output directory
A tools directory with protoc.exe and grpc_csharp_plugin.exe
A protos directory with test.proto as shown below:
test.proto:
syntax = "proto3";
service StackOverflowService {
rpc GetAnswer(Question) returns (Answer);
}
message Question {
string text = 1;
string user = 2;
repeated string tags = 3;
}
message Answer {
string text = 1;
string user = 2;
}
Then run (all on one line; I've broken it just for readability here):
tools\protoc.exe -I protos protos\test.proto --csharp_out=output
--grpc_out=output --plugin=protoc-gen-grpc=tools\grpc_csharp_plugin.exe
In the output directory, you'll find Test.cs and TestGrpc.cs
Just an idle comment here for other that find this, the documentation about this is terribly out of date and just flat out wrong.
Installing Grpc.Tools does not install anything in a packages folder; that is legacy behaviour which is no longer true even on windows.
When you install Grpc.Tools it will be hidden away in your local package cache, which you can see by calling:
$ dotnet nuget locals all --list
info : http-cache: /Users/doug/.local/share/NuGet/v3-cache
info : global-packages: /Users/doug/.nuget/packages/
info : temp: /var/folders/xx/s2hnzbrj3yn4hp1bg8q9gb_m0000gn/T/NuGetScratch
The binaries you want will be in one of these folders.
The easiest way to do this is to download the Grpc.Tools package directly from nuget, and install it locally.
I've hacked up this little helper script to do that, which works on windows/mac/linux, which may ease the difficulty of getting starting with this for others:
using System;
using System.Diagnostics;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Net.Http;
using System.Reflection;
using System.Runtime.InteropServices;
using System.Threading.Tasks;
using Mono.Unix;
namespace BuildProtocol
{
public class Program
{
private const string ToolsUrl = "https://www.nuget.org/api/v2/package/Grpc.Tools/";
private const string Service = "Greeter";
private static string ProtocolPath = Path.Combine("..", "protos");
private static string Protocol = Path.Combine(ProtocolPath, "helloworld.proto");
private static string Output = Path.Combine("..", "Greeter");
public static void Main(string[] args)
{
RequireTools().Wait();
var protoc = ProtocPath();
var plugin = ProtocPluginPath();
Console.WriteLine($"Using: {protoc}");
Console.WriteLine($"Using: {plugin}");
var command = new string[]
{
$"-I{ProtocolPath}",
$"--csharp_out={Output}",
$"--grpc_out={Output}",
$"--plugin=protoc-gen-grpc=\"{plugin}\"",
Protocol,
};
Console.WriteLine($"Exec: {protoc} {string.Join(' ', command)}");
var process = new Process
{
StartInfo = new ProcessStartInfo
{
UseShellExecute = false,
FileName = protoc,
Arguments = string.Join(' ', command)
}
};
process.Start();
process.WaitForExit();
Console.WriteLine($"Completed status: {process.ExitCode}");
}
public static async Task RequireTools()
{
if (!Directory.Exists("Tools"))
{
Console.WriteLine("No local tools found, downloading binaries from nuget...");
Directory.CreateDirectory("Tools");
await DownloadTools();
ExtractTools();
}
}
private static void ExtractTools()
{
ZipFile.ExtractToDirectory(Path.Combine("Tools", "tools.zip"), Path.Combine("Tools", "bin"));
}
private static async Task DownloadTools()
{
using (var client = new HttpClient())
{
Console.WriteLine($"Fetching: {ToolsUrl}");
using (var result = await client.GetAsync(ToolsUrl))
{
if (!result.IsSuccessStatusCode) throw new Exception($"Unable to download tools ({result.StatusCode}), check URL");
var localArchive = Path.Combine("Tools", "tools.zip");
Console.WriteLine($"Saving to: {localArchive}");
File.WriteAllBytes(localArchive, await result.Content.ReadAsByteArrayAsync());
}
}
}
private static string ProtocPath()
{
var path = Path.Combine("Tools", "bin", "tools", DetermineArch(), "protoc");
RequireExecutablePermission(path);
return WithExeExtensionIfRequired(path);
}
private static string ProtocPluginPath()
{
var path = Path.Combine("Tools", "bin", "tools", DetermineArch(), "grpc_csharp_plugin");
RequireExecutablePermission(path);
return WithExeExtensionIfRequired(path);
}
private static void RequireExecutablePermission(string path)
{
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows)) return;
Console.WriteLine($"Ensuring +x on {path}");
var unixFileInfo = new UnixFileInfo(path);
unixFileInfo.FileAccessPermissions = FileAccessPermissions.UserRead | FileAccessPermissions.UserWrite | FileAccessPermissions.UserExecute;
}
private static string WithExeExtensionIfRequired(string path)
{
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
path += ".exe";
}
return path;
}
private static string DetermineArch()
{
var arch = RuntimeInformation.OSArchitecture;
if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
return WithArch("windows_", arch);
}
if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX))
{
return WithArch("macosx_", arch);
}
if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
{
return WithArch("linux_", arch);
}
throw new Exception("Unable to determine runtime");
}
private static string WithArch(string platform, Architecture arch)
{
switch (arch)
{
case Architecture.X64:
return $"{platform}x86";
case Architecture.X86:
return $"{platform}x64";
default:
throw new ArgumentOutOfRangeException(nameof(arch), arch, null);
}
}
}
}
the following approach helped me :
Create a gRPC client and server in ASP.NET Core
in project, where .proto file located, edit the .csproj file
<ItemGroup>
....
<Protobuf Include="Shipping.proto" GrpcServices="Server" />
</ItemGroup>
rebuild the project, the all necessary .cs files will be added automaticaly
\obj\Debug\[TARGET_FRAMEWORK]\Shipping.cs
\obj\Debug\[TARGET_FRAMEWORK]\ShippingGrpc.cs
I am downloading a large file from a web site.
Size: = 599 MB (629,113,799 bytes).
The program works fine.
However I run into some errors:
not able to validate that the file was completely donwloaded.
I did not see any of the messages in the screen that says
Console.WriteLine("..............File succesfully downloaded......This mesage comes from -- wc_DownloadFileCompleted..........");
So the program does download the file completely.
(I did download the file manually using the web browser and compared the
file size between the manual download and the file that was downloaded
via the program. The size was the same. So that is a good reason for me
to believe that the program did download the file completely)
However at the very end (after it displays the percentage as 99)
it throws an error.. Here is the message
An exception occured during a WebClient request.
The error message gets fired inside the wc_DownloadFileCompleted
method.
So the help I need is... How do we validate that the file was downloaded
completely (without any errors)? Also how do we eliminate the error.
Is there some other code that I can use instead of using the code
I presented.
I did see the File Download percentage 99 appear in the screen.
However I did not see the "100" appear. Any comments?
using System.Threading.Tasks;
using System;
using System.Data;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Collections.Generic;
using System.Text;
using System.Diagnostics;
using System.Reflection; //Need this to get the DEBUG path
using MHPUtil;
using System.Net;
using System.Globalization;
namespace CmsNpiFileLoad
{
public static class FileDownLoadPercentage
{
//We need a global variable that will remain the same throughout the run.
//The unique value for this variable will get set in the DownloadFile() method.
public static string Value { get; set; }
}
class CMsNPIFileDownLoad
{
//599 MB (629,113,799 bytes) is the size of the file.
public void DownloadCMSNPIFile()
{
string Destinationfile = "S:\\MIS\\Provider NPI file\\" + "NPI.zip";
string CmsDownLoadSite = "http://download.cms.gov/nppes/NPPES_Data_Dissemination_January_2018.zip";
//Is the INTERNET AVAILABLE
if (System.Net.NetworkInformation.NetworkInterface.GetIsNetworkAvailable())
{
Console.WriteLine("Internet available, proceed with the download");
}
else
{
Console.WriteLine("Internet not available, proceed with the download");
return;
}
//So at this point we dont have that file with us locally.. so lets download
Console.WriteLine("Start Downloading....");
try
{
using (WebClient client = new WebClient())
{
client.DownloadProgressChanged += wc_DownloadProgressChanged; // This works well
client.DownloadFileCompleted += wc_DownloadFileCompleted;
client.DownloadFileAsync(new System.Uri(CmsDownLoadSite), Destinationfile);
}
}
catch (WebException we)
{
Console.WriteLine(we.ToString());
}
Console.ReadLine(); // We dont want the black screen to just disapper from us. so we put a readline so that it will keep displaying all the messsages
}
private static void wc_DownloadFileCompleted(object sender, System.ComponentModel.AsyncCompletedEventArgs e)
{
if (e.Cancelled)
{
Console.WriteLine("The download has been cancelled");
return;
}
if (e.Error != null)
{
//I tested the the program 2 times.. each time an error occured.
//The error message is "An exceptipon occured during a WebClient request.
Console.WriteLine("An error ocurred while trying to download file");
Console.WriteLine(e.Error.Message.ToString());
return;
}
// I did not see this on the screen.. ????
Console.WriteLine("..............File succesfully downloaded......This mesage comes from -- wc_DownloadFileCompleted..........");
}
private static void wc_DownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e)
{
if (FileDownLoadPercentage.Value != e.ProgressPercentage.ToString())
{
FileDownLoadPercentage.Value = e.ProgressPercentage.ToString();
//Display File Donwload Percentage ( increments of 10 )
if (Convert.ToInt16(e.ProgressPercentage) % 10 == 0)
Console.WriteLine(e.ProgressPercentage.ToString());
//Show the percentage when it is 99
if (Convert.ToInt16(e.ProgressPercentage) == 99)
Console.WriteLine(e.ProgressPercentage.ToString());
}
if (e.BytesReceived == e.TotalBytesToReceive)
{
Console.WriteLine("File DownLoad Complete...This message comes from -- wc_DownloadProgressChanged "); // I did not see this on the screen.. ????
}
}
}
}
Iam trying to get the dht implementation of monotorrent to work but i just cant seem to find any peers.
ive tried most of the examplecode code availeble on the net like the testclient and dhttest.
I have tried with several diffrent infohashes.
Anyone here got it working? or do you know where i can find the devs?
This is how my code looks atm:
using System;
using System.Collections.Generic;
using System.Text;
using MonoTorrent.Dht;
using MonoTorrent.Dht.Listeners;
using System.Net;
using System.IO;
using MonoTorrent.Common;
using MonoTorrent.Tracker.Listeners;
namespace SampleClient
{
class Program
{
static void Main(string[] args)
{
string basePath = Environment.CurrentDirectory;
string torrentsPath = Path.Combine(basePath, "Torrents");
Torrent torrent = null;
// If the torrentsPath does not exist, we want to create it
if (!Directory.Exists(torrentsPath))
Directory.CreateDirectory(torrentsPath);
// For each file in the torrents path that is a .torrent file, load it into the engine.
foreach (string file in Directory.GetFiles(torrentsPath))
{
if (file.EndsWith(".torrent"))
{
try
{
// Load the .torrent from the file into a Torrent instance
// You can use this to do preprocessing should you need to
torrent = Torrent.Load(file);
Console.WriteLine(torrent.InfoHash.ToString());
}
catch (Exception e)
{
Console.Write("Couldn't decode {0}: ", file);
Console.WriteLine(e.Message);
continue;
}
}
}
DhtListener listener = new DhtListener(new IPEndPoint(IPAddress.Parse("192.168.2.3"), 10000));
DhtEngine engine = new DhtEngine(listener);
//engine.RegisterDht(dht);
byte[] nodes = null;
if (File.Exists("mynodes"))
nodes = File.ReadAllBytes("mynodes");
listener.Start();
int i = 0;
bool running = true;
StringBuilder sb = new StringBuilder(1024);
while (running)
{
engine.Start(nodes);
while (Console.ReadLine() != "q")
{
engine.GetPeers(torrent.InfoHash);
}
File.WriteAllBytes("mynodes", engine.SaveNodes());
}
}
}
}
I know it's very old question, I'm not sure why it's still noone has answer it, anyway. The problem seem to be this line:
DhtListener listener = new DhtListener(new IPEndPoint(IPAddress.Parse("192.168.2.3"), 10000));
This ip is not the real ip, so you actually asl peers to send the respone to unkonw adress.
What to do? register your own adress.