Code is in F# but also tagging C# in case for any suggestions. It's an SSL based request for which it couldn't resolve the proxy name, I've tried both http://localhost and https://localhost besides 127.0.0.1
Code:
let request = WebRequest.Create("https://foo.example.com") :?> HttpWebRequest
let myproxy = WebProxy("http://127.0.0.1", 60103);
myproxy.BypassProxyOnLocal <- false;
request.Proxy <- myproxy;
Error:
System.Net.WebException: An error occurred while sending the request. Couldn't resolve proxy name ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.CurlException: Couldn't resolve proxy name
My configurations
Reverse Proxy Settings
Local port: 60103
Remote Hot: foo.example.com
Remote Port: 443
(Check box is enabled for both "Enable Reverse Proxies" and the above record itself)
SSL Proxy Settings
Host: foo.example.com
Port: 443
Both checkboxes are checked for "Enable SSL Proxying" and the record itself.
Under Help -> Proxying -> Install Charles Root Certificate, the certificate was installed and marked as trusted in keychain
It has been painful and I realize the mistake and here's the answer with the solution.
The cause of problem was a solution that worked for me in nodejs and it caused a confusion in here while not realizing that C#/F# provides a WebProxy class for that. nodejs doesn't have a WebProxy class (as per my knowledge) it so the concept is to use reverse proxy in nodejs, that is to send the request to localhost and a specific port which maps to the remote url.
So keep your URL same as where it should be pointed to and use WebProxy class here to point to localhost and port (8888 in my case) where Charles Proxy is listening and intercepting requests.
Thanks Fyodor Soikin for point out not to use the url (http:// etc.) and just use hostname.
// let url = "http://localhost:60103"; // don't do this, reverse proxy settings
let url = "https://example.com"; // keep the url intact
let request = WebRequest.Create(url) :?> HttpWebRequest
let myproxy = WebProxy("localhost", 8888); // port where Charles proxy is running
myproxy.BypassProxyOnLocal <- false;
request.Proxy <- myproxy;
Related
I have gRPC client / server code happily working on my local machine with the client using ChannelCredentials.Insecure but need to switch to a secure mode. I dont need any certificate checks just encrypted traffic.
Ive been unable to find any configuration of client server that talks to each other.
Running c# core server (gRPC 2.27.0), & c# .net framework client (gRPC 2.28.1).
The server says it publishing on http & https as below:
[20:12:58 DBG] Using development certificate: CN=localhost (Thumbprint: 3EDA2E5BD559D75C9DCF058E0A6994EED859CD34)
[20:12:58 INF] Now listening on: https://localhost:5001
[20:12:58 INF] Now listening on: http://localhost:5000
and client works with:
ChannelBase channel = new Channel("localhost:5000", ChannelCredentials.Insecure);
var client = new MyApp.MyAppClient(channel);
var response = client.Test(request)
If I switch client to SslCredentials as below
ChannelBase channel = new Channel("localhost:5001", new SslCredentials());
var client = new MyApp.MyAppClient(channel);
var response = client.Test(request)
I get the following errors.
server error
[19:32:53 DBG] Failed to authenticate HTTPS connection.
System.IO.IOException: Authentication failed because the remote party has closed the transport stream.
at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
client error
Grpc.Core.RpcException: 'Status(StatusCode=Unavailable, Detail="failed to connect to all addresses")'
I've also tried adding (server) with no change in errors.
services.AddAuthentication(CertificateAuthenticationDefaults.AuthenticationScheme)
.AddCertificate(options =>
{
options.RevocationMode = X509RevocationMode.NoCheck;
options.ValidateCertificateUse = false;
options.AllowedCertificateTypes = CertificateTypes.SelfSigned;
});
Any suggestions as to what i need to do to make them talk.
So you need to specify roots.pem that correspond to the dev certificate you're using on your server when creating SslCredentials.
If you use parameterless SslCredentials(), it will use the default trust roots which only work for certificates that have been signed by publicly trusted certificate authorities (e.g. if you're talking "official" servers such as googleapis.com etc, definitely not the case for your development certificates that you are using on your server). So you need to use SslCredentials(yourCustomTrustRootsPem).
Btw, with SSL/TLS, there's no such thing as "no certificate checks". The client will ALWAYS check that it connected to the right server (server's certificate are verifiable by their trust roots) - without that the secure communication would make no sense because it would be too susceptible to man-in-the-middle attack. (the server checking authenticity of client is optional though).
Feel free to refer to https://github.com/jtattermusch/grpc-authentication-kubernetes-examples for extra resources.
I had a similar problem and finally found a solution to establish HTTPs connection between
.NET Framework 4.7.2 client (WPF app) and
ASP .NET Core 3.1 gRPC Server (Console).
Indeed the answer of Jan Tattermusch is correct. You need to supply the server side certificate as PEM to the constructor of SslCredentials and in addition, the certificate has to contain the DNS name or IP of the url you are contacting the server through. In your case, I suggest to manually create a self-signed certificate for each server instead of using the developer certificate. This one can be downloaded by your client and then be passed as SslCredentials. Works like a charm.
See my more details on how to do all the details in my answer to a similar question here:
https://stackoverflow.com/a/63565090/378415
Try to check on which port Grpc Service is running after deployment in any way either windows service or other , then give the same in client to consume the service. Eg: If your service is running on https://localhost:5000 then give the same in client.
It will work for sure.
Mostly you will get the above issue when you make a mistake of mismatching the address of service to client.
I am setting up a simple server based on HttpListener in .net core that needs to run on MacOS. Works great on http, but i need it to run on https (port 443. Tried a custom port but still no success)
On windows, it works with "netsh add iplisten" and binding the cert with "netsh add sslcert". And it works great for both http and https.
On Mac OS however, first thing i found out was to set up an alias for localhost "ifconfig lo0 alias 127.0.0.9 up" so i'm allowed to start the httplistener.
That worked, and whenever i make a HTTP request to 127.0.0.9 i get the response i hard-coded in the browser.
Https still doesn't work. So i tried binding the certificate.
I used openssl to create a certificate => got a .p12 cert self-signed.
Used "httpcfg -add -port 443 -p12 server.p12 -pwd mypassword"
That worked, as i could see 2 files appeared in "~/.config/.mono/httplistener/" called 443.cer and 443.pvk.
However, if i navigate to https://127.0.0.9 i get an error connection reset in the browser.
I have no idea what else to try, and i can't find any answer.
Did anybody tried to do this on Mac OS?
var ipAddress = "127.0.0.9";
var httpListener = new HttpListener();
var httpPrefix = string.Format("http://{0}/", ipAddress);
var httpsPrefix = string.Format("https://{0}/", ipAddress);
httpListener.Prefixes.Add(httpPrefix);
httpListener.Prefixes.Add(httpsPrefix);
httpListener.Start();
WaitForNewRequest(httpListener);
In the "WaitForNewRequest(httpListener)" method i just write a hard-coded "hello world" response and wait for the next request. Nothing complicated.
Thank you!
Ports below 1024 are privileged, you need to run your app as root to open them.
If you need to use 443 in production, I highly recommend using apache or nginx to listen on the SSL port and proxy the traffic to your .net code.
This is the issue.
I have an https request. The request is is being sent as an SSL / TLS request (Not the CONNECT .... that comes from a browser with the proxy setup).
I need to write a proxy in c# that blocks a specific https://foo.com/foo.htm request but lets through https://foo.com/anything_else.htm.
I can do this fine creating a MITM attack with a new certificate etc etc.
But Im now wondering if there is an easy way to do this Im missing without using a MITM attack as I have no need to decrypt the data. I only need to know the URI/file.
I can easily just transfer streams but I want to know if there is a simple way to transfer the streams after I have read the URI and file.
I can write some fancy code to pull apart the tcp request and thats what I may have to do.
Anybody any ideas before I go down this path. Remember there is no CONNECT request. Just direct SSL / TLS.
The main reason for this is it just makes things simpler not creating self signed certificates etc.
Maybe its even possible to use the real certificate somehow from the server end as I dont need to decrypt any of the no header data.
I find the networking side of c# is not very well documented and a little all over the place.
Just for reference i can get the URI from the TcpClient using:
IPEndPoint ipEndPoint = (IPEndPoint)clientTcpClient.Client.RemoteEndPoint;
IPAddress ipAddress = ipEndPoint.Address;
// Get the hostname.
IPHostEntry ipHostEntry = Dns.GetHostEntry(ipAddress);
String hostName = ipHostEntry.HostName;
// Get the port.
Int32 port = ipEndPoint.Port;
But not the requested page.
While the target hostname might be visible in the TLS handshake as SNI extension or by analyzing the certificate returned by the server the path component of the URL is only contained in the HTTP request. Since this HTTP request is only done after TLS handshake and the request is thus already encrypted you cannot get to the full path without decrypting the request. This means that blocking access to a specific path is not possible without SSL man in the middle and thus requires a certificate for the target site owned by the man in the middle and trusted by the client.
Not that this is true for CONNECT requests too since these requests only contain the target hostname but the path component is again only contained in the encrypted HTTP request sent inside the tunnel created by CONNECT.
I've seen a lot of discussion about "TF400324: Team Foundation services are not available from server", but everything I've read relates to DNS or proxies, and reflect being unable to connect to TFS at all, through any channel. My case is different: I cannot reach my TFS server when using the .NET library, but it works fine using Visual Studio's workflow tools, and I can reach the same URL just fine in a browser.
Zee code, it is here:
private TfsConfigurationServer _server;
...
Uri url = new Uri(serverName + rootFolder);
var creds1 = new NetworkCredential(username, password, Environment.UserDomainName);
var creds2 = new BasicAuthCredential(creds1);
var creds3 = new TfsClientCredentials(creds2) { AllowInteractive = false };
_server = new TfsConfigurationServer(url, creds3);
// Throws Microsoft.TeamFoundation.TeamFoundationServiceUnavailableException
_server.Authenticate();
The creds are needlessly elaborate because I've tried various suggested solutions, but I don't think that's relevant; I get the same behavior with any other creds I've tried, or no creds at all.
If I copy the exact URL to a browser, I can authenticate and proceed. Within Visual Studio I can connect to TFS using Source Control Explorer and other tools, without explicitly authenticating. What might the library want handled differently?
Additional detail from the error:
Team Foundation services are not available from server https://tfs.imahufflepuff.com:8080/tfs/Root/Project.
Technical information (for administrator):
The underlying connection was closed: An unexpected error occurred on a send.
---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send.
---> System.IO.IOException: The handshake failed due to an unexpected packet format.
We don't have an in-house TFS admin, otherwise I'd kick this issue over to him. I've tried to use Fiddler to get additional detail, but VS consistently refuses to show up there. I can reach an externally-hosted API while debugging, so I don't think there's a network or proxy problem locking down VS.
Have you tried using TfsTeamProjectCollection class instead of TfsConfigurationServer ?
E.g.
Uri url = new Uri(serverName + rootFolder);
var creds = new NetworkCredential(username, password, Environment.UserDomainName);
var server = new TfsTeamProjectCollection(url, creds);
server.Authenticate();
You can also try debugging this issue using Fiddler. You'll have to change VS proxy settings before starting Fiddler:
Either set the registry key reg add hkcu\SOFTWARE\Microsoft\VisualStudio\12.0\TeamFoundation\RequestSettings
/v BypassProxyOnLocal /t REG_SZ /d False
or
Set environment variable TFS_BYPASS_PROXY_ON_LOCAL=0
Seems like an SSL handshake issue at its root, which has nothing to do with the TFS and HTTP protocol and authentication, they sit above SSL. So make sure you have a valid certificate, matching hostname, good validity, accessible CRL where the cert is not revoked, etc. Also check in other browsers or openssl.exe ("openssl.exe s_client -connect servername:8080") for more diagnostic info. You didn't mention but a proxy or an SSL-level certificate authentication could also cause problems, should be easily debuggable by the methods I mentioned above.
Is there a way to get a System.Net.WebRequest or System.Net.WebClient to respect the hosts or lmhosts file?
For example: in my hosts file I have:
10.0.0.1 www.bing.com
When I try to load Bing in a browser (both IE and FF) it fails to load as expected.
Dns.GetHostAddresses("www.bing.com")[0]; // 10.0.0.1
WebRequest.Create("http://10.0.0.1").GetResponse(); // throws exception (expected)
WebRequest.Create("http://www.bing.com/").GetResponse(); // unexpectedly succeeds
Similarly:
WebClient wc = new WebClient();
wc.DownloadString("http://www.bing.com"); //succeeds
Why would System.Net.Dns respect the hosts file but System.Net.WebRequest ignore it? What do I need to change to make the WebRequest respect the hosts file?
Additional Info:
If I disable IPv6 and set my IPv4 DNS Server to 127.0.0.1, the above code works (fails) as expected. However if I add my normal DNS servers back as alternates, the unexpected behavior resumes.
I've reproduced this on 3 Win7 and 2 Vista boxes. The only constant is my company's network.
I'm using .NET 3.5 SP1 and VS2008
Edit
Per #Richard Beier's suggestion, I tried out System.Net tracing. With tracing ON the WebRequest fails as it should. However as soon as I turn tracing OFF the behavior reverts to the unexpected success. I have reproduced this on the same machines as before in both debug and release mode.
Edit 2
This turned out to be the company proxy giving us issues. Our solution was a custom proxy config script for our test machines that had "bing.com" point to DIRECT instead of the default proxy.
I think that #Hans Passant has spotted the issue here. It looks like you have a proxy setup in IE.
Dns.GetHostAddresses("www.bing.com")[0]; // 10.0.0.1
This works because you are asking the OS to get the IP addresses for www.bing.com
WebRequest.Create("http://www.bing.com/").GetResponse(); // unexpectedly succeeds
This works because you are asking the framework to fetch a path from a server name. The framework uses the same engine and settings that IE frontend uses and hence if your company has specified by a GPO that you use a company proxy server, it is that proxy server that resolves the IP address for www.bing.com rather than you.
WebRequest.Create("http://10.0.0.1").GetResponse(); // throws exception (expected)
This works/fails because you have asked the framework to fetch you a webpage from a specific server (by IP). Even if you do have a proxy set, this proxy will still not be able to connect to this IP address.
I hope that this helps.
Jonathan
I'm using VS 2010 on Windows 7, and I can't reproduce this. I made the same hosts-file change and ran the following code:
Console.WriteLine(Dns.GetHostAddresses("www.bing.com")[0]); // 10.0.0.1
var response = WebRequest.Create("http://www.bing.com/").GetResponse(); // * * *
Console.WriteLine(new StreamReader(response.GetResponseStream()).ReadToEnd());
I got an exception on the line marked "* * *". Here's the exception detail:
System.Net.WebException was unhandled
Message=Unable to connect to the remote server
Source=System
StackTrace:
at System.Net.HttpWebRequest.GetResponse()
at ConsoleApplication2.Program.Main(String[] args) in c:\Data\Projects\ConsoleApplication2\ConsoleApplication2\Program.cs:line 17
InnerException: System.Net.Sockets.SocketException
Message=A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.0.0.1:80
Source=System
ErrorCode=10060
Maybe it's an issue with an earlier .NET version, that's now fixed in .NET 4 / VS 2010? Which version of .NET are you using?
I also found this thread from 2007, where someone else ran into the same problem. There are some good suggestions there, including the following:
Turn on system.net tracing
Work around the problem by using Dns.GetHostAddresses() to resolve it to an IP. Then put the IP in the URL - e.g. "http://10.0.0.1/". That may not be an option for you though.
In the above thread, mariyaatanasova_msft also says: "HttpWebRequest uses Dns.GetHostEntry to resolve the host, so you may get a different result from Dns.GetHostAddresses".
You should overwrite the default proxy.
HttpWebRequest & WebRequest will set a default proxy if present in Internet Explorer and your file hosts will be bypassed.
request.Proxy = new WebProxy();
The following is just an example of code:
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("www.bing.com");
request.Proxy = new WebProxy();
request.Method = "POST";
request.AllowAutoRedirect = false;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
//some code here
}
}
catch (exception e)
{
//Some other code here
}