Develop Reference

Starting a web server using C# and .net in Visual Studio

I am using the below code to start a web server that should be open to listen always but it is executing only once, I am deploying this in AKS - the container is not staying in running state, it is exiting after completing the job i.e., printing the response.
Please help me understand where I am doing wrong or my understanding itself is not right.
May be this is not the right piece of code to use if I want the container to be always running as a web server that listens to requests from other apps in the project.
string baseAddress = "http://localhost:9000/";
// Start OWIN host
using (WebApp.Start<Startup>(url: baseAddress))
{
// Create HttpClient and make a request to api/values
HttpClient client = new HttpClient();
var response = client.GetAsync(baseAddress + "api/values").Result;
System.Console.WriteLine(response);
System.Console.WriteLine(response.Content.ReadAsStringAsync().Result);
System.Console.ReadLine();
}

If your code is already working for once. Thats should be help for loop
while(true){
var response = client.GetAsync(baseAddress + "api/values").Result;
System.Console.WriteLine(response.Content.ReadAsStringAsync().Result);
}

If you want to listen incoming http requests, try something like that with HttpListener:
var listener = new HttpListener();
listener.Prefixes.Add("http://*:9000/api/values");
listener.Start();
while (true)
{
HttpListenerContext context = listener.GetContext();
HttpListenerRequest request = context.Request;
// do something here with request
// default OK response
context.Response.StatusCode = 200;
context.Response.OutputStream.Write(Array.Empty<byte>(), 0, 0);
context.Response.Headers.Add("Access-Control-Allow-Origin", "*");
context.Response.Close();
}

Post request to not accessible domain [duplicate]

I want to download a file using Tor. Most solutions I found require that additional software (e.g. privoxy) is installed and running, but I don't want to have an additional software running all the time even when I don't use my program.
So I tried the Tor.NET library, but I can't get it using Tor. This example shouldn't return my IP-address, but it does:
ClientCreateParams createParams = new ClientCreateParams(#"D:\tor.exe", 9051);
Client client = Client.Create(createParams);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://www.icanhazip.com/");
request.Proxy = client.Proxy.WebProxy;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
var reader = new StreamReader(response.GetResponseStream());
Console.WriteLine(reader.ReadToEnd());
}
Console.WriteLine("Press enter to exit...");
Console.ReadLine();
There are already multiple comments about this but unfortunally the author of the library isn't active anymore.
Maybe you know what I'm doing wrong (is more configuration neccessary?) or have an idea for an alternative way to download a file using tor.

You follow the Tor project manual, command line HTTPTunnelPort, that you find here: first you must start an HTTP tunnel with
Tor.exe --HTTPTunnelPort 4711
It supplies you with a HTTP tunnel at 127.0.0.1:4711 (see also here). Now you can connect to this proxy:
WebProxy oWebProxy = new WebProxy (IPAddress.Loopback.ToString (), 4711);
WebClient oWebClient = new WebClient ();
oWebClient.Proxy = oWebProxy;
oWebClient.DownloadFile ("https://myUri", "myFilename");
By now, Microsoft recommends the use of the HttpClient for new developments. Here is the code:
// we must configure the HttpClient
HttpClientHandler oHttpClientHandler = new HttpClientHandler ();
oHttpClientHandler.UseProxy = true;
oHttpClientHandler.Proxy =
new WebProxy (IPAddress.Loopback.ToString (), 4711);
// we start an HttpClient with the handler; Microsoft recommends to start
// one HttpClient per application
HttpClient oHttpClient = new HttpClient (oHttpClientHandler);
// we request the resource by the GET method
HttpRequestMessage oHttpRequestMessage =
new HttpRequestMessage (HttpMethod.Get, "https://myUri");
// we make the request and we do only wait for the headers and not for
// content
Task<HttpResponseMessage>
oTaskSendAsync =
oHttpClient.SendAsync
(oHttpRequestMessage, HttpCompletionOption.ResponseHeadersRead);
// we wait for the arrival of the headers
oTaskSendAsync.Wait ();
// the function throws an exception if something went wrong
oTaskSendAsync.Result.EnsureSuccessStatusCode ();
// we can act on the returned headers
HttpResponseHeaders oResponseHeaders = oTaskSendAsync.Result.Headers;
HttpContentHeaders oContentHeaders = oTaskSendAsync.Result.Content.Headers;
// we fetch the content stream
Task<Stream> oTaskResponseStream =
oTaskSendAsync.Result.Content.ReadAsStreamAsync ();
// we open a file for the download data
FileStream oFileStream = File.OpenWrite ("myFilename");
// we delegate the copying of the content to the file
Task oTaskCopyTo = oTaskResponseStream.Result.CopyToAsync (oFileStream);
// and wait for its completion
oTaskCopyTo.Wait ();
// now we can close the file
oFileStream.Close ();
Please heed the following in using Tor.exe:
If the port is already in use Tor.exe will not be able to supply a proxy. It does not even necessarily inform you about this failure.
Make sure that nobody spoofs your program Tor.exe so that it is Tor that supplies you with this proxy. Hence, Tor.exe should be at a secure place in your file system.
Inform yourself about other precautions with respect to using Tor.
At least, you might want to check that your proxy has a different IP address from your local internet connection.

In the end I used https://github.com/Ogglas/SocksWebProxy by #Ogglas to download a file using Tor.
The project has an example which is not working (the first time you start it waits infinitely for Tor to exit, but when you start the program again it can use the Tor process startet by your first try), so I changed it.
I made a Start() method to start Tor:
public async void Start(IProgress<int> progress)
{
torProcess = new Process();
torProcess.StartInfo.FileName = #"D:\...\tor.exe";
torProcess.StartInfo.Arguments = #"-f ""D:\...\torrc""";
torProcess.StartInfo.UseShellExecute = false;
torProcess.StartInfo.RedirectStandardOutput = true;
torProcess.StartInfo.CreateNoWindow = true;
torProcess.Start();
var reader = torProcess.StandardOutput;
while (true)
{
var line = await reader.ReadLineAsync();
if (line == null)
{
// EOF
Environment.Exit(0);
}
// Get loading status
foreach (Match m in Regex.Matches(line, #"Bootstrapped (\d+)%"))
{
progress.Report(Convert.ToInt32(m.Groups[1].Value));
}
if (line.Contains("100%: Done"))
{
// Tor loaded
break;
}
if (line.Contains("Is Tor already running?"))
{
// Tor already running
break;
}
}
proxy = new SocksWebProxy(new ProxyConfig(
//This is an internal http->socks proxy that runs in process
IPAddress.Parse("127.0.0.1"),
//This is the port your in process http->socks proxy will run on
12345,
//This could be an address to a local socks proxy (ex: Tor / Tor Browser, If Tor is running it will be on 127.0.0.1)
IPAddress.Parse("127.0.0.1"),
//This is the port that the socks proxy lives on (ex: Tor / Tor Browser, Tor is 9150)
9150,
//This Can be Socks4 or Socks5
ProxyConfig.SocksVersion.Five
));
progress.Report(100);
}
Afterwards you can use something like this to download something:
public static string DownloadString(string url)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Proxy = proxy;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
return new StreamReader(response.GetResponseStream()).ReadToEnd();
}
}
And when you exit the program you should also kill the Tor process.

HttpWebRequest one url, two target ip

I must individually test each server of a cluster. Each server have one IP address but I must use same URL. I am using a console project.
static string RequestGet(string requestUrl, string ipspecify)
{
HttpWebRequest request = HttpWebRequest.CreateHttp(requestUrl);
// Submit the request, and get the response body.
string responseBodyFromRemoteServer;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
responseBodyFromRemoteServer = reader.ReadToEnd();
}
}
return responseBodyFromRemoteServer;
}
Locally, I can modify my host file, but I want to do this automatically through my program.
RequestGet("https://toto.org/myservice", "172.2.240.16")
RequestGet("https://toto.org/myservice", "172.2.240.17")
IP address is server address.
What is the solution?

Turns out, as of .NET 5, this is possible using HttpClient. SocketsHttpHandler, the default handler used by HttpClient, gained a ConnectCallback property which lets you override how a connection to the remote machine is established.
Here's where it's called, which you can use as inspiration for writing your own. The following seems to work fine:
public static async Task Main()
{
using var client = new HttpClient(new SocketsHttpHandler() { ConnectCallback = ConnectCallback });
await client.GetStringAsync("http://example.com");
}
private static async ValueTask<Stream> ConnectCallback(SocketsHttpConnectionContext context, CancellationToken cancellationToken)
{
var endPoint = new DnsEndPoint("1.2.3.4", context.DnsEndPoint.Port);
var socket = new Socket(SocketType.Stream, ProtocolType.Tcp) { NoDelay = true };
await socket.ConnectAsync(endPoint, cancellationToken);
return new NetworkStream(socket, ownsSocket: true);
}
The request is made to 1.2.3.4 (or whatever IP you specify), but everything above the level of the TCP socket itself, including TLS, the Host header, etc, is still set to example.com, meaning that you shouldn't get certificate errors.

cURL request with HttpClient - how to set the timeout on the server connection (WinInet)

I am sending cURL request using HttpClient through the method described here under.
The parameter used for this method are:
SelectedProxy = a custom class that stores my proxy's parameters
Parameters.WcTimeout = the timeout
url, header, content = the cURL request (based on this tool to convert to C# https://curl.olsh.me/).
const SslProtocols _Tls12 = (SslProtocols)0x00000C00;
const SecurityProtocolType Tls12 = (SecurityProtocolType)_Tls12;
ServicePointManager.SecurityProtocol = Tls12;
string source = "";
using (var handler = new HttpClientHandler())
{
handler.UseCookies = usecookies;
WebProxy wp = new WebProxy(SelectedProxy.Address);
handler.Proxy = wp;
using (var httpClient = new HttpClient(handler))
{
httpClient.Timeout = Parameters.WcTimeout;
using (var request = new HttpRequestMessage(new HttpMethod(HttpMethod), url))
{
if (headers != null)
{
foreach (var h in headers)
{
request.Headers.TryAddWithoutValidation(h.Item1, h.Item2);
}
}
if (content != "")
{
request.Content = new StringContent(content, Encoding.UTF8, "application/x-www-form-urlencoded");
}
HttpResponseMessage response = new HttpResponseMessage();
try
{
response = await httpClient.SendAsync(request);
}
catch (Exception e)
{
//Here the exception happens
}
source = await response.Content.ReadAsStringAsync();
}
}
}
return source;
If I am running this without proxy, it works like a charm.
When I send a request using a proxy which I tested first from Chrome, I have the following error on my try {} catch {}. Here is the error tree
{"An error occurred while sending the request."}
InnerException {"Unable to connect to the remote server"}
InnerException {"A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond [ProxyAdress]"}
SocketErrorCode: TimedOut
By using a Stopwatch I see that the TimedOut occurred after around 30 sec.
I tried a few different handler based on the following links What's the difference between HttpClient.Timeout and using the WebRequestHandler timeout properties?, HttpClient Timeout confusion or with the WinHttpHandler.
It's worth noting that WinHttpHandler allow for a different error code, i.e. Error 12002 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'The operation timed out'. The underlying reason is the same though it helped to target where it bugs (i.e. WinInet) which confirms also what #DavidWright was saying regarding that timeouts from HttpClient manages a different part of the request sending.
Hence my issue is coming from the time it takes to establish a connection to the server, which triggers the 30sec timeout from WinInet.
My question is then How to change those timeout?
On a side note, it's worth noting that Chrome, which uses WinInet, does not seem to suffer from this timeout, nor Cefsharp on which a big part of my app is based, and through which the same proxies can properly send requests.

So thanks to #DavidWright I understand a few things:
Before that the HttpRequestMessage is sent and the timeout from HttpClient starts, a TCP connection to the server is initiated
The TCP connection has its own timeout, defined at OS level, and we do not identified a way to change it at run time from C# (question pending if anyone want to contribute)
Insisting on trying to connect works as each try benefits from previous tries, though proper exception management & manual timeout counter needs to be implemented (I actually considered a number of tries in my code, assuming each try is around 30sec)
All this together ended up in the following code:
const SslProtocols _Tls12 = (SslProtocols)0x00000C00;
const SecurityProtocolType Tls12 = (SecurityProtocolType)_Tls12;
ServicePointManager.SecurityProtocol = Tls12;
var sp = ServicePointManager.FindServicePoint(endpoint);
sp.ConnectionLeaseTimeout = (int)Parameters.ConnectionLeaseTimeout.TotalMilliseconds;
string source = "";
using (var handler = new HttpClientHandler())
{
handler.UseCookies = usecookies;
WebProxy wp = new WebProxy(SelectedProxy.Address);
handler.Proxy = wp;
using (var client = new HttpClient(handler))
{
client.Timeout = Parameters.WcTimeout;
int n = 0;
back:
using (var request = new HttpRequestMessage(new HttpMethod(HttpMethod), endpoint))
{
if (headers != null)
{
foreach (var h in headers)
{
request.Headers.TryAddWithoutValidation(h.Item1, h.Item2);
}
}
if (content != "")
{
request.Content = new StringContent(content, Encoding.UTF8, "application/x-www-form-urlencoded");
}
HttpResponseMessage response = new HttpResponseMessage();
try
{
response = await client.SendAsync(request);
}
catch (Exception e)
{
if(e.InnerException != null)
{
if(e.InnerException.InnerException != null)
{
if (e.InnerException.InnerException.Message.Contains("A connection attempt failed because the connected party did not properly respond after"))
{
if (n <= Parameters.TCPMaxTries)
{
n++;
goto back;
}
}
}
}
// Manage here other exceptions
}
source = await response.Content.ReadAsStringAsync();
}
}
}
return source;
On a side note, my current implementation of HttpClient may be problematic in the future. Though being disposable, HttpClient should be defined at App level through a static, and not within a using statement. To read more about this go here or there.
My issue is that I want to renew the proxy at each request and that it is not set on a per request basis. While it explains the reasdon of the new ConnectionLeaseTimeout parameter (to minimize the time the lease remains open) it is a different topic

I have had the same problem with HttpClient. Two things need to happen for SendAsync to return: first, setting up the TCP channel over which the communication occurs (the SYN, SYN/ACK, ACK handshake, if you're familiar with that) and second getting back the data that constitutes the HTTP response over that TCP channel. HttpClient's timeout only applies to the second part. The timeout for the first part is governed by the OS's network subsystem, and it's quite difficult to change that timeout in .NET code.
(Here's how you can reproduce this effect. Set up a working client/server connection between two machines, so you know that name resolution, port access, listening, and client and server logic all works. Then unplug the network cable on the server and re-run the client request. It will time out with the OS's default network timeout, regardless of what timeout you set on your HttpClient.)
The only way I know around this is to start your own delay timer on a different thread and cancel the SendAsync task if the timer finishes first. You can do this using Task.Delay and Task.WaitAny or by creating a CancellationTokenSource with your desired timeone (which essentially just does the first way under the hood). In either case you will need to be careful about cancelling and reading exceptions from the task that loses the race.

HttpListner not listening

I am trying to check if there is an ajax request to a particular url and if yes, then I am trying to add a cookie to the request.
But my HttpListner is listening only for localhost. If I try to give any other url, it is not listening.
I have turned off my firewall also but still the same issue.
Please tell me where I am making a mistake.
Below is my code:
class Program
{
static void Main(string[] args)
{
HttpListener listner = new HttpListener();
listner.Prefixes.Add("<my url>");//For example, https://www.google.co.in/
listner.Start();
HttpListenerContext context = listner.GetContext();
HttpListenerRequest request = context.Request;
WebRequest webReq = WebRequest.Create(request.Url);
//Copying the headers
System.Collections.Specialized.NameValueCollection headers = request.Headers;
for (int i = 0; i < headers.AllKeys.Length; i++)
{
if (!WebHeaderCollection.IsRestricted(headers.AllKeys[i]))
{
webReq.Headers.Add(headers.AllKeys[i], headers[headers.AllKeys[i]]);
}
}
webReq.ContentLength = request.ContentLength64;
webReq.ContentType = request.ContentType;
webReq.Headers.Add(HttpRequestHeader.Cookie, "<My Cookie>");
HttpListenerResponse response = context.Response;
WebResponse webResponse = webReq.GetResponse();
webResponse.GetResponseStream().CopyTo(response.OutputStream);
}
}

Whatever prefixes you add to the listener HAVE to resolve to your localhost machine. Make sure that your DNS server correctly resolves the host to the machine where you run your code.
If you are testing this in a development environment and/or don't have control over the DNS server, add an entry to the hosts file that maps whatever host name you want to use to your local machine.
The other thing to watch out for is the port. Make sure nothing else is listening on the port you intend to use.