Develop Reference

Post request to not accessible domain [duplicate]

I want to download a file using Tor. Most solutions I found require that additional software (e.g. privoxy) is installed and running, but I don't want to have an additional software running all the time even when I don't use my program.
So I tried the Tor.NET library, but I can't get it using Tor. This example shouldn't return my IP-address, but it does:
ClientCreateParams createParams = new ClientCreateParams(#"D:\tor.exe", 9051);
Client client = Client.Create(createParams);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.icanhazip.com/");
request.Proxy = client.Proxy.WebProxy;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
var reader = new StreamReader(response.GetResponseStream());
Console.WriteLine(reader.ReadToEnd());
}
Console.WriteLine("Press enter to exit...");
Console.ReadLine();
There are already multiple comments about this but unfortunally the author of the library isn't active anymore.
Maybe you know what I'm doing wrong (is more configuration neccessary?) or have an idea for an alternative way to download a file using tor.

You follow the Tor project manual, command line HTTPTunnelPort, that you find here: first you must start an HTTP tunnel with
Tor.exe --HTTPTunnelPort 4711
It supplies you with a HTTP tunnel at 127.0.0.1:4711 (see also here). Now you can connect to this proxy:
WebProxy oWebProxy = new WebProxy (IPAddress.Loopback.ToString (), 4711);
WebClient oWebClient = new WebClient ();
oWebClient.Proxy = oWebProxy;
oWebClient.DownloadFile ("https://myUri", "myFilename");
By now, Microsoft recommends the use of the HttpClient for new developments. Here is the code:
// we must configure the HttpClient
HttpClientHandler oHttpClientHandler = new HttpClientHandler ();
oHttpClientHandler.UseProxy = true;
oHttpClientHandler.Proxy =
new WebProxy (IPAddress.Loopback.ToString (), 4711);
// we start an HttpClient with the handler; Microsoft recommends to start
// one HttpClient per application
HttpClient oHttpClient = new HttpClient (oHttpClientHandler);
// we request the resource by the GET method
HttpRequestMessage oHttpRequestMessage =
new HttpRequestMessage (HttpMethod.Get, "https://myUri");
// we make the request and we do only wait for the headers and not for
// content
Task<HttpResponseMessage>
oTaskSendAsync =
oHttpClient.SendAsync
(oHttpRequestMessage, HttpCompletionOption.ResponseHeadersRead);
// we wait for the arrival of the headers
oTaskSendAsync.Wait ();
// the function throws an exception if something went wrong
oTaskSendAsync.Result.EnsureSuccessStatusCode ();
// we can act on the returned headers
HttpResponseHeaders oResponseHeaders = oTaskSendAsync.Result.Headers;
HttpContentHeaders oContentHeaders = oTaskSendAsync.Result.Content.Headers;
// we fetch the content stream
Task<Stream> oTaskResponseStream =
oTaskSendAsync.Result.Content.ReadAsStreamAsync ();
// we open a file for the download data
FileStream oFileStream = File.OpenWrite ("myFilename");
// we delegate the copying of the content to the file
Task oTaskCopyTo = oTaskResponseStream.Result.CopyToAsync (oFileStream);
// and wait for its completion
oTaskCopyTo.Wait ();
// now we can close the file
oFileStream.Close ();
Please heed the following in using Tor.exe:
If the port is already in use Tor.exe will not be able to supply a proxy. It does not even necessarily inform you about this failure.
Make sure that nobody spoofs your program Tor.exe so that it is Tor that supplies you with this proxy. Hence, Tor.exe should be at a secure place in your file system.
Inform yourself about other precautions with respect to using Tor.
At least, you might want to check that your proxy has a different IP address from your local internet connection.

In the end I used https://github.com/Ogglas/SocksWebProxy by #Ogglas to download a file using Tor.
The project has an example which is not working (the first time you start it waits infinitely for Tor to exit, but when you start the program again it can use the Tor process startet by your first try), so I changed it.
I made a Start() method to start Tor:
public async void Start(IProgress<int> progress)
{
torProcess = new Process();
torProcess.StartInfo.FileName = #"D:\...\tor.exe";
torProcess.StartInfo.Arguments = #"-f ""D:\...\torrc""";
torProcess.StartInfo.UseShellExecute = false;
torProcess.StartInfo.RedirectStandardOutput = true;
torProcess.StartInfo.CreateNoWindow = true;
torProcess.Start();
var reader = torProcess.StandardOutput;
while (true)
{
var line = await reader.ReadLineAsync();
if (line == null)
{
// EOF
Environment.Exit(0);
}
// Get loading status
foreach (Match m in Regex.Matches(line, #"Bootstrapped (\d+)%"))
{
progress.Report(Convert.ToInt32(m.Groups[1].Value));
}
if (line.Contains("100%: Done"))
{
// Tor loaded
break;
}
if (line.Contains("Is Tor already running?"))
{
// Tor already running
break;
}
}
proxy = new SocksWebProxy(new ProxyConfig(
//This is an internal http->socks proxy that runs in process
IPAddress.Parse("127.0.0.1"),
//This is the port your in process http->socks proxy will run on
12345,
//This could be an address to a local socks proxy (ex: Tor / Tor Browser, If Tor is running it will be on 127.0.0.1)
IPAddress.Parse("127.0.0.1"),
//This is the port that the socks proxy lives on (ex: Tor / Tor Browser, Tor is 9150)
9150,
//This Can be Socks4 or Socks5
ProxyConfig.SocksVersion.Five
));
progress.Report(100);
}
Afterwards you can use something like this to download something:
public static string DownloadString(string url)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Proxy = proxy;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
return new StreamReader(response.GetResponseStream()).ReadToEnd();
}
}
And when you exit the program you should also kill the Tor process.

Download file using Tor in C#

I want to download a file using Tor. Most solutions I found require that additional software (e.g. privoxy) is installed and running, but I don't want to have an additional software running all the time even when I don't use my program.
So I tried the Tor.NET library, but I can't get it using Tor. This example shouldn't return my IP-address, but it does:
ClientCreateParams createParams = new ClientCreateParams(#"D:\tor.exe", 9051);
Client client = Client.Create(createParams);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.icanhazip.com/");
request.Proxy = client.Proxy.WebProxy;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
var reader = new StreamReader(response.GetResponseStream());
Console.WriteLine(reader.ReadToEnd());
}
Console.WriteLine("Press enter to exit...");
Console.ReadLine();
There are already multiple comments about this but unfortunally the author of the library isn't active anymore.
Maybe you know what I'm doing wrong (is more configuration neccessary?) or have an idea for an alternative way to download a file using tor.

You follow the Tor project manual, command line HTTPTunnelPort, that you find here: first you must start an HTTP tunnel with
Tor.exe --HTTPTunnelPort 4711
It supplies you with a HTTP tunnel at 127.0.0.1:4711 (see also here). Now you can connect to this proxy:
WebProxy oWebProxy = new WebProxy (IPAddress.Loopback.ToString (), 4711);
WebClient oWebClient = new WebClient ();
oWebClient.Proxy = oWebProxy;
oWebClient.DownloadFile ("https://myUri", "myFilename");
By now, Microsoft recommends the use of the HttpClient for new developments. Here is the code:
// we must configure the HttpClient
HttpClientHandler oHttpClientHandler = new HttpClientHandler ();
oHttpClientHandler.UseProxy = true;
oHttpClientHandler.Proxy =
new WebProxy (IPAddress.Loopback.ToString (), 4711);
// we start an HttpClient with the handler; Microsoft recommends to start
// one HttpClient per application
HttpClient oHttpClient = new HttpClient (oHttpClientHandler);
// we request the resource by the GET method
HttpRequestMessage oHttpRequestMessage =
new HttpRequestMessage (HttpMethod.Get, "https://myUri");
// we make the request and we do only wait for the headers and not for
// content
Task<HttpResponseMessage>
oTaskSendAsync =
oHttpClient.SendAsync
(oHttpRequestMessage, HttpCompletionOption.ResponseHeadersRead);
// we wait for the arrival of the headers
oTaskSendAsync.Wait ();
// the function throws an exception if something went wrong
oTaskSendAsync.Result.EnsureSuccessStatusCode ();
// we can act on the returned headers
HttpResponseHeaders oResponseHeaders = oTaskSendAsync.Result.Headers;
HttpContentHeaders oContentHeaders = oTaskSendAsync.Result.Content.Headers;
// we fetch the content stream
Task<Stream> oTaskResponseStream =
oTaskSendAsync.Result.Content.ReadAsStreamAsync ();
// we open a file for the download data
FileStream oFileStream = File.OpenWrite ("myFilename");
// we delegate the copying of the content to the file
Task oTaskCopyTo = oTaskResponseStream.Result.CopyToAsync (oFileStream);
// and wait for its completion
oTaskCopyTo.Wait ();
// now we can close the file
oFileStream.Close ();
Please heed the following in using Tor.exe:
If the port is already in use Tor.exe will not be able to supply a proxy. It does not even necessarily inform you about this failure.
Make sure that nobody spoofs your program Tor.exe so that it is Tor that supplies you with this proxy. Hence, Tor.exe should be at a secure place in your file system.
Inform yourself about other precautions with respect to using Tor.
At least, you might want to check that your proxy has a different IP address from your local internet connection.

In the end I used https://github.com/Ogglas/SocksWebProxy by #Ogglas to download a file using Tor.
The project has an example which is not working (the first time you start it waits infinitely for Tor to exit, but when you start the program again it can use the Tor process startet by your first try), so I changed it.
I made a Start() method to start Tor:
public async void Start(IProgress<int> progress)
{
torProcess = new Process();
torProcess.StartInfo.FileName = #"D:\...\tor.exe";
torProcess.StartInfo.Arguments = #"-f ""D:\...\torrc""";
torProcess.StartInfo.UseShellExecute = false;
torProcess.StartInfo.RedirectStandardOutput = true;
torProcess.StartInfo.CreateNoWindow = true;
torProcess.Start();
var reader = torProcess.StandardOutput;
while (true)
{
var line = await reader.ReadLineAsync();
if (line == null)
{
// EOF
Environment.Exit(0);
}
// Get loading status
foreach (Match m in Regex.Matches(line, #"Bootstrapped (\d+)%"))
{
progress.Report(Convert.ToInt32(m.Groups[1].Value));
}
if (line.Contains("100%: Done"))
{
// Tor loaded
break;
}
if (line.Contains("Is Tor already running?"))
{
// Tor already running
break;
}
}
proxy = new SocksWebProxy(new ProxyConfig(
//This is an internal http->socks proxy that runs in process
IPAddress.Parse("127.0.0.1"),
//This is the port your in process http->socks proxy will run on
12345,
//This could be an address to a local socks proxy (ex: Tor / Tor Browser, If Tor is running it will be on 127.0.0.1)
IPAddress.Parse("127.0.0.1"),
//This is the port that the socks proxy lives on (ex: Tor / Tor Browser, Tor is 9150)
9150,
//This Can be Socks4 or Socks5
ProxyConfig.SocksVersion.Five
));
progress.Report(100);
}
Afterwards you can use something like this to download something:
public static string DownloadString(string url)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Proxy = proxy;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
return new StreamReader(response.GetResponseStream()).ReadToEnd();
}
}
And when you exit the program you should also kill the Tor process.

"The remote server returned an error: (401) Unauthorized" error when trying to upload a file

I found a lot of question about (401) Unauthorized error, but none of them explained to me how to diagnose the issue.
I created a new MVC ASP.net app to learn how to upload a file to a sharepoint folder. As of right now, I can't seem to be able to copy a file from
C:\testfolder\file.txt to
C:\testfolder\UploadedFiles
Here is the method that I made where I send in the source file path and the target folder:
//Flag to indicate whether file was uploaded successfuly or not
bool isUploaded = true;
try
{
// Create a PUT Web request to upload the file.
WebRequest request = WebRequest.Create(targetDocumentLibraryPath);
//Set credentials of the current security context
request.PreAuthenticate = true;
request.UseDefaultCredentials = true;
ICredentials credentials = new NetworkCredential( "Username", "password", "Domain"); //I used my username and password here
request.Credentials = credentials;
//request.Credentials = CredentialCache.DefaultCredentials;
request.Method = "PUT";
// Create buffer to transfer file
byte[] fileBuffer = new byte[1024];
// Write the contents of the local file to the request stream.
using (Stream stream = request.GetRequestStream())
{
//Load the content from local file to stream
using (FileStream fsWorkbook = File.Open(sourceFilePath, FileMode.Open, FileAccess.Read))
{
//Get the start point
int startBuffer = fsWorkbook.Read(fileBuffer, 0, fileBuffer.Length);
for (int i = startBuffer; i > 0; i = fsWorkbook.Read(fileBuffer, 0, fileBuffer.Length))
{
stream.Write(fileBuffer, 0, i);
}
}
}
// Perform the PUT request
WebResponse response = request.GetResponse();
//Close response
response.Close();
}
catch (Exception ex)
{
//Set the flag to indiacte failure in uploading
isUploaded = false; //The remote server returned an error: (401) Unauthorized.
}
//Return the final upload status
return isUploaded;
}
I tried the following:
Changing the permission of the folders to be able to write/read from them for all users.
Use other folder locations (SharePoint, local drive, network drive) to upload the file to.
Any idea on how to get this problem to work? Any prospective on how to debug the problem is also appreciated.
Update: It is probably being my account is set as the app pool account in IIS. I am still not sure what the issue is.

This wouldn't be something like the proxy settings would it?
Do you need to add the default proxy setting to your
WebRequest
Also do you need to add
<system.net>
<defaultProxy useDefaultCredentials="true" />
</system.net>
To your web config?
This may help.
How should I set the default proxy to use default credentials?

I decided to try another approach and use httpposted file and .saveAs() described here

Download PDFs through proxy

I have a list of URLs linking directly to PDFs on a database website. It would be very easy to automate the download process, except for the fact that I have to access the website through a proxy server. The code I've been trying to use has been this:
public void Download()
{
WebClient wb2 = new WebClient();
WebProxy proxy = new WebProxy("PROXY_URL:port", true);
proxy.Credentials = new NetworkCredential("USERNAME", "PASSWORD");
GlobalProxySelection.Select = proxy;
try
{
for(int i = 0; i < URLList.Length; i++)
{
byte[] Data = DownloadData(URLList[i]);
FileStream fs = new FileStream(#"D:\Files\" + i.toString() + ".pdf", FileMode.Create)
fs.Write(Data, 0, Data.Length);
fs.Close();
}
}
catch(WebException WebEx)
{
MessageBox.Show(WebEx.Message);
}
}
public byte[] DownloadData(string path)
{
WebClient wb2 = new WebClient();
wb2.Credentials = new NetworkCredential("USERNAME","PASSWORD");
return wb2.DownloadData(path);
}
For some reason, it returns the error "(400): Bad Request" every time. I'm obviously able to get to these PDFs just fine through Firefox, so I'm wondering what I'm doing wrong here. I'm fairly new to programming in general, and very new to web protocols through C#. Any help would be appreciated.

use fiddler to work out the difference between the request your code is sending vs the one via your browser.
the 400 error is due to a malformed request; opposed to the proxy denying you (407) or the site requiring authentication (401).
Incidently, the line "wb2.Credentials = ..." is providing your username/password to the target server. is this intended?
Haven't used WebClient for a while, but you can use var request = HttpWebRequest.Create();
request.Proxy = proxy; request.GetResponse().GetResponseStream() and read the bytes using BinaryReader().
That will give you the byte array that you can write to a file using File.WriteAllBytes() rather than having to use a FileStream.
hth

upload file from window application to Remote Web Server

string filePath = "C:\\test\\564.flv";
try
{
WebClient client = new WebClient();
NetworkCredential nc = new NetworkCredential(uName, password);
Uri addy = new Uri("\\\\192.168.1.28\\Files\\test.flv");
client.Credentials = nc;
byte[] arrReturn = client.UploadFile(addy, filePath);
Console.WriteLine(arrReturn.ToString());
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
i am get this error "The remote server returned an error: (405) Method Not Allowed." what should i doo ??

WebClient uses POST method to upload a file in an HTTP server or uses STOR command to upload a file in an FTP server. It can't upload files in a network folder.

I do not believe the UploadFile() method can have a UNC address for the URI. See the documentation for the method: http://msdn.microsoft.com/en-us/library/ms144229.aspx. It needs to either be an FTP site or an HTTP site.