I am writing a proxy with TcpListener in C#.
This proxy listens a port that users send request to. And when accepted a user request, it will parse the request header and find the host name. Then it creates a TcpClient to the host server.
Here comes the problem. When http request comes, it should connect the port 80 of the server; while https request comes, it should connect the port 443 of the server. But I have no idea of how to distinguish http request and https request.
Question in one sentence: how to know it is a http request or https request that TcpListener accepted?
Many thanks!
You've stepped in a problem that has flustered web server administrators for a long time.
Here's the process:
Web browser establishes a TCP connection to a particular IP on the web server.
The web server knows what IP it's getting a connection from, knows that that IP is only ever used for secure.example.com, and so loads the SSL certificate for secure.example.com.
The web server and web browser negotiate a SSL connection.
The web browser sends vanilla HTTP headers down the SSL pipe, which include the "HOST: secure.example.com" line that indicates the virtual host to use.
The web server processes the request and sends the response using vanilla HTTP headers sent down the SSL pipe.
The web server has to decide which virtual host to use before it has any HTTP headers. This is because it has to negotiate an SSL connection first, and it has to know which certificate to use first. The vanilla solution is to use IP-based virtual hosts - run the web server on IP address X; whenever the server gets a request sent to address X, it knows the request belongs to the configured vhost for that address.
The problem with that scheme is that the server has to have separate IP addresses for each secure website it runs. That might be many, many IP addresses, and is either costly or impractical.
Step in Server Name Indication. When the web browser is negotiating the SSL connection to the web server, the web browser includes the hostname it wants to connect to in the SSL negotiation information. Now the web server can use that information to do normal name-based virtual hosts, and so the web server can run a thousand different secure websites each with their own SSL certificates all on exactly one IP address. Everything is right in the world again.
You want to get in the middle of this, which means that you have to understand the SSL/TLS negotiation phase, parse the server name information, and forward the request down to the right web server.
Your new flow looks something like this:
Web browser establishes a TCP connection to the proxy.
Proxy begins recording the SSL exchange.
Web browser starts to do SSL negotiation, and as part of such, sends the Server Name Information down.
The proxy parses the Server Name Information, decides which web server should handle the request, and forwards the SSL negotiation information to the web server.
The proxy does not otherwise participate in the negotiation; it reads the SNI, but otherwise is completely "pass-through".
The web browser and server complete the SSL negotiation, the server picks the right vhost, and the browser sends vanilla http headers for a request.
The web server reads the vanilla headers via the SSL connection, and processes the request.
Now that that's been said, you might realize that sticking your nose in the SSL connection negotiation might be more trouble than it's worth. Turns out a few other people have already had the same idea as you and have implemented a few programs that seem to do exactly what you're trying to do - do a search for "http sni proxy" - I came up with this: https://github.com/dlundquist/sniproxy
The headers are entirely encrypted. The only information going over the network 'in the clear' is related to the SSL setup and D/H key exchange. This exchange is carefully designed not to yield any useful information to eavesdroppers, and once it has taken place, all data is encrypted.
Update By the way After the SSL negotiation, normal HTTP headers will travel inside the encrypted stream, so there is really no difference between the two.
Related
I have written a winforms client, that connects to a Windows service establishing a connection with XSockets.Net. This is working fine for a direct connection to the internet.
But when there is a proxy server, it will fail.
As I checked the XSockets API I have not found any settings, that allows me to use a proxy server.
Also for the websockets protocol I have not found a sufficient answer.
Any ideas?
Use WSS:// for connection, that is the equivalent to HTTPS in WebSocket.
The WebSocket protocol handshake sends the HTTP headers "Upgrade:websocket" and "Connection:Upgrade", meaning that the proxy will probably remove the "Upgrade" header because is set as a "Connection" header. By using a secure protocol, the proxy won't be able of intercept the request and will just let it pass.
Cheers.
I have an asp.net application working in https (SSL). This is working well in my local computer and Amazon AWS(production environment).
But when I host this application in office (for testing) some strange things happens.
I can see the https in browser and the lock sign.
Fiddler also showing that the output is encrypted and shows port 443.
But HttpContext.Current.Request.IsSecureConnection returns false
And HttpContext.Current.Request.Url.Scheme returns http.
In the office we are using Juniper SSG firewall and TMG 2010 (Forefront Threat Management Gateway 2010). So server receive request through Juniper and TMG 2010. Thanks in advance.
To reduce costs I suspect that the SSL certificate is installed on the TMG Gateway and that this gateway is simply rewriting the request to standard HTTP when passing it to the actual web server. So by the time the request hits IIS and your web application it is a standard plain HTTP request.
This tripped my up after deploying to Amazon's Elastic Beanstalk environment. I couldn't see any way to get the load-balancer to allow the SSL request straight through to the server. Instead it was always terminating the SSL at the load-balancer and passing plain http back to the server.
I found this documentation: Elastic Load Balancing Concepts - X-Forwarded Headers.
Essentially the load-balancer injects a number of additional HTTP Headers into each request before forwarding it to the back-end server. The most relevant one is X-Forwarded-Proto which tracks the protocol used to connect from the client's browser to the load-balancer. This can be checked like so:
var loadbalancerReceivedSSLRequest = string.Equals(Request.Headers["X-Forwarded-Proto"], "https");
var serverReceivedSSLRequest = Request.IsSecureConnection;
if (loadbalancerReceivedSSLRequest || serverReceivedSSLRequest)
{
// SSL in use.
}
else
{
// SSL not in use.
}
Well another way to check is to check the port
if(context.Request.Url.Port == 443)
Note: check which port is used for secure connections, usually it is 443
In my corporate environment there is a transparent proxy that requires credentials for internet access (every four hours). In my application I successfully passed credentials like this:
var client = new WebClient();
client.Credientals = new NetworkCredential("username", "password");
string result = client.DownloadString("http://...");
// this works!
However, when my intial request is to a "https://" url, there is an Exception throw: "The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel."
Now my current work around is to:
catch the WebException thrown when accessing the "https://" url
add my Credentials to a new request to an arbitrary "http://" site
(this should "open up" the internet for a four hour window)
go back and re-try the "https://" request
I am wondering if there is a better/cleaner way to do this?
What you are using right now is an HTTP proxy with authentication. So far so good. But it won't work for HTTPS requests, and here's why:
SSL/TLS is endpoint security. This means that the data must be sent between the client and the server via the single encrypted channel.
When you connect to the HTTP proxy, you tell it "GET the remote resource and send it to me", which contradicts to endpoint security. Here you don't have direct connection to the remote server and you can't validate its credentials. Also the proxy can peep into your data.
In general, it's possible to connect to regular HTTP proxy using HTTPS OR it is possible to ask the HTTP proxy to access HTTPS resource, but this breaks security cause in both cases the client can't validate server's credentials and HTTP proxy can log or alter the data being transferred.
HTTPS proxy works in a different way. Here you tell the HTTPS proxy server "CONNECT to remote address and then only resend whatever is passed". This way the proxy creates an opaque secure channel between the client and the server thus preserving endpoint security. Actually, HTTPS proxy can be used to tunnel any traffic, not necessarily SSL.
Consequently you need to establish the tunnel by sending CONNECT request (with your authentication included), and then send regular HTTP GET (without host/address in the URL) over the same channel - this request will go to the destination server, not to the proxy.
I have serious doubts that your WebClient can be made to establish a tunnel before sending a request. As an option, you can use HTTPBlackbox package of our SecureBlackbox product which lets you access HTTP and HTTPS resources, and supports HTTPS proxies (called WebTunneling in SecureBlackbox) with authentication.
I have created a WCF server in a PC named PC1. I access the URI http://PC1:8000/ServiceModelSamples/Service using internet explorer from another PC named PC2.
When I debug the messages in Wireshark, I am confused that why there is no HTTP message, even though I specify "http://" in my URI. It only shows TCP protocol, there is no HTTP message or header.
Please advice
To wireshark, any browser request is just another TCP connection. As far as showing the HTTP protocol details is concerned, it might be guessing that traffic on a specific port (80) would be http. Since you are using a non-standard http port, it might not be able to do so.
To confirm this, try loading some other website/webpage (e.g. www.google.com) and see if it is able to show you http details. If it works, then next thing would be try and find some setting/configuration by which you can tell wireshark that it should treat traffic from another port (8000 in your case) as http traffic.
EDIT:
See this question as a guide to configure wireshark for http ports.
HTTP is an application layer protocol that sits on top of TCP, the transport layer protocol.
See http://en.wikipedia.org/wiki/TCP/IP_model for more information.
When you access the service using a browser, a "friendly" service responds to generate a web form to your browser. If you get a web page back in your browser, it is HTTP. That's not part of the "SOAP" spec, but it is part of the MS WCF stack supporting HTTP.
Then, if you fill it out, you might be POSTing or GETing the form, but POST is the default. That's also HTTP. GET is often disabled in WCF.
Then, you get back XMLish stuff in your browser, that also came by HTTP.
So you might just be missing an HTTP protocol decode in wireshark.
EDIT: I didn't see your URL included :8000. Wireshark won't decode that as HTTP unless you force it to, because it's not on the HTTP port. You can right-click on a port 8000 packet and say "follow conversation", and you'll see all the http goodness. You can also force wireshark to decode that stream as HTTP, which will let you "drill into" the packets past the TCP layer.
I am developing an application which has one web server and one C# client which posts XML to the web server. The web server needs to know the local IP address of client. I tried methods to retrieve the IP address at the server side, but these methods don't give the IP address of the client when there are proxy servers or NAT in between. So I need to find the local IP of the connection at the client application and send it with the request.
The problem is in HttpWebRequest. I don't see any method by which I can get the local IP address of the connection made while sending HTTP request.
CLARIFICATION:
The client in my case is not browser-based. It is a C# application. My server has specific rules based on local IP, that was are used to connect to the server while sending the HTTP request. In my case, local IPs are fixed. There could be multiple IPs on a local machine -- that's why I want the local IP associated with the Socket used to send the request. This can be solved if I use TcpClient in C# and implement the HTTP protocol on top of it, but I want to avoid that. So, is there any way I can get the socket associated with HttpWebRequest before posting the request?
It is not possible to get the local IPAddress of the connection that HttpWebRequest is going to use. However, you could use Dns or System.Net.NetworkInformation class to get all the local IPaddresses, and send them as a custom header with your request. On the server side you could parse this header, and see if any of the IP's match the one that you are expecting.
However, as Damien has indicated above, this solution is not foolproof, people could spoof the IPaddress to the one the server is expecting. Maybe you need to take a step back and think what security you are trying to achive by this? And see if you could accomplish your goals by a different method - for eg: user Authentication using a supported method like Basic(over HTTPS), Digest, NTLM, Kerberos, Negotiate, or use SSL with client certs?
One possible solution is to tell the client what ip it should use when connecting to the server, and include that ip in a custom header for the server to read.
var srcip = IPAddress.Parse("10.0.0.1");
var request = (HttpWebRequest)HttpWebRequest.Create ("http://example.com/");
request.Headers.Add("X-Original-Client-IP", srcip.ToString());
request.ServicePoint.BindIPEndPointDelegate = delegate { return new IPEndPoint(srcip, 0); };
request.GetResponse();
There's no guaranteed way of doing this. Javascript and server-side code can only see the public address, not the local one. There is a method for getting it using Java (mentioned here), but it relies on both Javascript and Java being available, and the user's browser supporting that particular call.
Given that many local IPs are assigned as needed (my home PC's is assigned using DHCP, my work PC's changes occasonally too), do you think knowing the local IP would actually be any use?
Maybe you should move from HttpWebRequest to SOAP requests/responses if XML is what you send over the line. Implement ASPX web service or WFC webservice. HttpWebRequest are always seen to the server as they are coming from the last IP that issued them so that would be NAT or proxy.
However, I too find it odd enough that you need to know local machine IP on the server side.