I've written a proxy in c#. It works by getting the http request from the browser sending the request to the site and sending back the site's response to the client. It works in firefox but some of the pages are cut like if it did not send all the response and in chrome it gives blank pages and for google.co.uk in chrome the browser gives "no data received". Can you see a mistake in my code which might be causing all this?
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.Threading;
using System.Windows.Forms;
namespace LexProxy
{
class ProxyServer
{
private TcpListener tcpListener;
public ProxyServer()
{
this.tcpListener = new TcpListener(IPAddress.Any, 3000);
this.tcpListener.Start();
while (true)
{
Console.Write("Waiting for a connection... ");
TcpClient client = tcpListener.AcceptTcpClient();
Thread thread = new Thread(delegate()
{
Serve(client);
});
thread.Start();
}
}
private void Serve(TcpClient client)
{
Console.WriteLine("Connected!");
NetworkStream stream = client.GetStream();
byte[] request = GetBytesFromStream(stream, client.ReceiveBufferSize);
if (request != null)
{
string requestString = System.Text.Encoding.UTF8.GetString(request);
string[] requestParts = requestString.Split(' ');
if (requestParts.Length >= 2)
{
string method = requestParts[0];
if (!requestParts[1].Contains("http://") && !requestParts[1].Contains("https://"))
requestParts[1] = "http://" + requestParts[1];
Uri uri = new Uri(requestParts[1], UriKind.RelativeOrAbsolute);
string host = StringUtils.ReplaceFirst(uri.Host, "www.", "");
int port = uri.Port;
byte[] response = getResponse(host, port, request);
if (response != null)
stream.Write(response, 0, response.Length);
client.Close();
}
}
}
private byte[] getResponse(string host, int port, byte[] request)
{
TcpClient client = new TcpClient(host, port);
NetworkStream stream = client.GetStream();
stream.Write(request, 0, request.Length);
byte[] response = GetBytesFromStream(stream, client.ReceiveBufferSize);
return response;
}
private byte[] GetBytesFromStream(NetworkStream stream, int bufferSize)
{
Byte[] bytes = new Byte[bufferSize];
int i;
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
return bytes;
}
return null;
}
}
}
One minor error is that you're using UTF8 to read first line of request. HTTP request line and headers are ASCII (and body may be UTF8, but doesn't have to be, and may not even be a string at all). The reason it works is b/c for english charset, UTF8 and ASCII happen to be encoded using the same bytes. But that's the minor issue.
Most likely primary issue is hiding in your GetBytesFromStream. You only call .Read once, but that doesn't guarantee that the whole message is returned. It may return as little as just 1 byte... so you need to continue getting more data. How do you know how much more? It's dictated by HTTP protocol which you'd need to properly parse and examine. (At a minimum you're reading until you reach \r\n\r\n byte sequence indicating the end of request headers.
However, this is not enough as there may be a request body, length of which will be specified via an HTTP header, Content-Length: (IIRC), or possibly using chunked encoding. I don't see any code beyond URI examination, so most likely you do not handle HTTP messaging protocol itself, and so it is unlikely to work properly (unless you somehow manage to force browser to use HTTP/0.9 or HTTP/1.0 since those do not reuse connection and send one message per connection, at which point you could just blindly read everything the socket has to give you until browser signals the end of stream by closing its write endpoint of the connection).
But your main problem is that GetBytesFromStream as written won't give you "all" bytes.
Related
I am using windivert to capture tcp packet. I am successful to capture the packets by WinDivertSharp.dll. Now i want to parse the packet for only http request. To parse packet i am using the flowing code.
var index = BinaryMatch(messageBody, Encoding.ASCII.GetBytes("\r\n\r\n")) + 4;
var headers = Encoding.ASCII.GetString(messageBody, 0, index);
var memory = new MemoryStream(messageBody);
memory.Position = index;
string body = "";
if (headers.IndexOf("Content-Encoding: gzip") > 0)
{
using (GZipStream decompressionStream = new GZipStream(memory, CompressionMode.Decompress))
using (var decompressedMemory = new MemoryStream())
{
decompressionStream.CopyTo(decompressedMemory);
decompressedMemory.Position = 0;
body = Encoding.UTF8.GetString(decompressedMemory.ToArray());
}
}
else
{
body = Encoding.UTF8.GetString(messageBody, index, messageBody.Length - index);
}
It working good for some webs. But for http request like from https://stackoverflow.com/ it is not working. Please help to decode
http request for all kinds of web.
Might be a long shot, but it might be related to WinDivert not decrypting responses from sites that use SSL/TLS (HTTPS) like https://stackoverflow.com/.
I would suggest you try and compare between sites that use HTTP and HTTPS.
Hope it helps!
Snapd has documentation on a REST API.
I'm able to connect to the socket from C# using the following
var snapSocket = new Socket(AddressFamily.Unix, SocketType.Stream, ProtocolType.IP);
var snapEndpoint = new UnixEndPoint("/run/snapd.socket");
snapSocket.Connect(snapEndpoint);
var req = Encoding.ASCII.GetBytes("GET /v2/system-info HTTP/1.1");
snapSocket.Send(req, req.Length, 0);
var bytesReceived = new byte[256];
var bytes = 0;
var response = "";
do
{
bytes = snapSocket.Receive(bytesReceived, bytesReceived.Length, 0);
response = response + Encoding.ASCII.GetString(bytesReceived, 0, bytes);
} while (bytes > 0);
Console.WriteLine(response);
But everything halts at snapSocket.Receive - a response is never sent. I suspect that there's something wrong with the message that I'm sending it.
It turns out that it expects a standard HTTP request, which means a Host: line, a Connection: Close line, and two \ns at the very end are required.
The documentation's following claim...
While it is expected to allow clients to connect using HTTPS over a TCP socket, at this point only a UNIX socket is supported.
... is meant only to imply that HTTPS and TCP do not work yet - HTTP is currently the valid request format even when using the UNIX Socket.
I am not fluent in C# at all, but maybe this python snippet can help lead into a solution:
import requests_unixsocket
session = requests_unixsocket.Session()
r = session.get('http+unix://%2Frun%2Fsnapd.socket/v2/snaps')
r.raise_for_status()
r.json()
I'm a beginner with the WebSocket API. I'm trying to connect to my server locally but I'm obtaining the connection closed message. Can anyone tell me what I am doing wrong?
That's my code:
Server
using System.Net.Sockets;
using System.Net;
using System.IO;
using System;
class Program
{
static void Main(string[] args)
{
var listener = new TcpListener(IPAddress.Loopback, 8181);
listener.Start();
while (true)
{
Console.WriteLine("Listening...");
using (var client = listener.AcceptTcpClient())
using (var stream = client.GetStream())
using (var reader = new StreamReader(stream))
using (var writer = new StreamWriter(stream))
{
string line = null, key = "", responseKey = "";
string MAGIC_STRING = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11";
while (line != "")
{
line = reader.ReadLine();
if (line.StartsWith("Sec-WebSocket-Key:"))
{
key = line.Split(':')[1].Trim();
}
}
if (key != "")
{
key += MAGIC_STRING;
using (var sha1 = SHA1.Create())
{
responseKey = Convert.ToBase64String(sha1.ComputeHash(Encoding.ASCII.GetBytes(key)));
}
}
// send handshake to the client
writer.WriteLine("HTTP/1.1 101 Web Socket Protocol Handshake");
writer.WriteLine("Upgrade: WebSocket");
writer.WriteLine("Connection: Upgrade");
writer.WriteLine("WebSocket-Origin: http://localhost:8080");
writer.WriteLine("WebSocket-Location: ws://localhost:8181/websession");
if (!String.IsNullOrEmpty(responseKey))
writer.WriteLine("Sec-WebSocket-Accept: " + responseKey);
writer.WriteLine("");
Console.ReadLine();
writer.Flush();
}//using
Console.WriteLine("Finished");
}//while
}
}
The Client
<!DOCTYPE HTML>
<html>
<head><title></title>
<script type="text/javascript">
function WebSocketTest() {
var msg = document.getElementById("msg");
if ("WebSocket" in window) {
msg.innerHTML="WebSocket is supported by your Browser!";
// Let us open a web socket
var ws = new WebSocket("ws://localhost:8181/websession");
ws.onopen = function () {
// Web Socket is connected, send data using send()
msg.innerHTML="connection open";
//ws.send("Message to send");
//msg.innerHTML="Message is sent...";
};
ws.onclose = function () {
// websocket is closed.
msg.innerHTML = "Connection is closed...";
};
ws.onerror = function(error){
console.log('Error detected: ' + error);
};
ws.onmessage = function (evt) {
var received_msg = evt.data;
msg.innerHTML="Message is received...";
};
}
else {
// The browser doesn't support WebSocket
msg.innerHTML="WebSocket NOT supported by your Browser!";
}
}
</script>
</head>
<body>
<div id="sse">
Run WebSocket<br />
<p id="msg"></p>
</div>
</body>
</html>
Any solution would be appreciated and thank you :)
That is not a valid web-socket response under any specification. The initial web-socket response always requires you to crunch some numbers as part of the response headers, to prove you're a web-socket server. Which headers to read and write, and what crunching to do, depends on the version of web-sockets (hibi/hixie 76/rfc). It actually looks like your server is using the headers of a client.
For example, a RFC6455 (13) response would start:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: {crunch some numbers}
Note that a Hixie-76 response is different, and there are bits in the above that I have omitted.
From the RFC6455 specification:
To prove that the handshake was received, the server has to take two
pieces of information and combine them to form a response. The first
piece of information comes from the |Sec-WebSocket-Key| header field
in the client handshake:
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
For this header field, the server has to take the value (as present in
the header field, e.g., the base64-encoded [RFC4648] version minus any
leading and trailing whitespace) and concatenate this with the
Globally Unique Identifier (GUID, [RFC4122]) "258EAFA5-E914-47DA-
95CA-C5AB0DC85B11" in string form, which is unlikely to be used by
network endpoints that do not understand the WebSocket Protocol. A
SHA-1 hash (160 bits) [FIPS.180-3], base64-encoded (see Section 4 of
[RFC4648]), of this concatenation is then returned in the server's
handshake.
After the javascript code connects to the webserver. C# runs through all using statements and prints all the lines to your file after which all using statements are closed and thus the generated client is disposed, which closes the connection.
A quick (but dirty) fix for this problem is to add "Console.Readline" at the end of the using statements. Be carefull with this though, your process will hang!
For more information about TcpClients accepting clients, go to msdn.
Notice that the given example there, only allows one connection at a time.
Last remark: there exist libraries for accepting websocket connections like Marc suggested.
I'm building a small HTTP proxy that runs between the browser and squid proxy. The browser sends the HTTP request to my proxy that redirects it to the squid proxy, then my application gets the response from the squid proxy and returns it back to the browser.
the problem is that i can't get the full response from the proxy, i get HTTP 200 OK ... (just the response header), but with out the body then i have to call receive method another time to get the body. but if i debug my code (which make the application slower) it get all the response (response header and body)
is there any propriety in the TCPClass that indicates to me that the remote server still have data to send to me ?
here is my code :
static void Main(string[] args)
{
int ServerPort = 8888;
IPAddress localHost = new IPAddress(0x0100007f);
TcpListener listener = new TcpListener(localHost,ServerPort);
listener.Start();
while(true)
{
string requestString = "";
String respenseString = "";
TcpClient application = listener.AcceptTcpClient();
string source = application.Client.RemoteEndPoint.ToString();
byte[] dataFromApp = new byte[application.ReceiveBufferSize];
application.Client.Receive(dataFromApp);
TcpClient tunnel = new TcpClient("127.0.0.1",8080);
tunnel.Client.Send(dataFromApp);
while (tunnel.Client.Connected ==true)
{
if(tunnel.Available != 0)
{
byte[] responseFromProxy = new byte[tunnel.ReceiveBufferSize];
tunnel.Client.Receive(responseFromProxy);
respenseString += Encoding.UTF8.GetString(responseFromProxy);
}
else
{
break;
}
}
application.Client.Send(Encoding.UTF8.GetBytes(respenseString));
}
You should check the return value of tunnel.Client.Receive and application.Client.Receive. Receive doesn't gurantee that it will read dataFromApp.Length bytes
REMARKS: The Receive method reads data into the buffer parameter and returns the number of bytes successfully read
PS: You may also want to try FiddlerCore to write an Http Proxy
There is no "there are N bytes remaining for this message" property on a socket, because a TCP socket is streaming: it sends and receives bytes, not messages.
HTTP defines messages, and if you are implementing an HTTP proxy, you should be familiar with the HTTP 1.1 RFC. There are various ways to determine the lenght of an HTTP message, all of which you have to implement to make sure you can successfully receive and send HTTP messages.
Thanks guys
I've done it :
while (tunnel.Client.Receive(oneByte) != 0)
{
byte[] responseFromProxy = new byte[tunnel.Available];
tunnel.Client.Receive(responseFromProxy);
application.Client.Send(oneByte);
application.Client.Send(responseFromProxy);
}
I have been putting together a little embedded HTTP server in a windows service app that listens for updates coming from other devices on the network that speak HTTP.
For each HTTP request, the code that processes the request/response is executed twice, I expect it to run only once. I tried the code using the AsyncGetContext method and using the synchronous version GetContext - the end result is the same.
Code
public void RunService()
{
var prefix = "http://*:4333/";
HttpListener listener = new HttpListener();
listener.Prefixes.Add(prefix);
try
{
listener.Start();
_logger.Debug(String.Format("Listening on http.sys prefix: {0}", prefix));
}
catch (HttpListenerException hlex)
{
_logger.Error(String.Format("HttpListener failed to start listening. Error Code: {0}", hlex.ErrorCode));
return;
}
while (listener.IsListening)
{
var context = listener.GetContext(); // This line returns a second time through the while loop for each request
ProcessRequest(context);
}
listener.Close();
}
private void ProcessRequest(HttpListenerContext context)
{
// Get the data from the HTTP stream
var body = new StreamReader(context.Request.InputStream).ReadToEnd();
_logger.Debug(body);
byte[] b = Encoding.UTF8.GetBytes("OK");
context.Response.StatusCode = 200;
context.Response.KeepAlive = false;
context.Response.ContentLength64 = b.Length;
var output = context.Response.OutputStream;
output.Write(b, 0, b.Length);
output.Close();
context.Response.Close();
}
Is there anything obvious that I am missing, I have run out of ideas to track down the issue.
Ok, the issue was I was using a web browser to test the HTTP connection and by default a web browser also sends a request for favicon.ico. So two requests were actually coming across. Thank you to #Inuyasha for suggesting I check things out with Wireshark.