Develop Reference

What sort of message does Snapd's API expect?

Snapd has documentation on a REST API.
I'm able to connect to the socket from C# using the following
var snapSocket = new Socket(AddressFamily.Unix, SocketType.Stream, ProtocolType.IP);
var snapEndpoint = new UnixEndPoint("/run/snapd.socket");
snapSocket.Connect(snapEndpoint);
var req = Encoding.ASCII.GetBytes("GET /v2/system-info HTTP/1.1");
snapSocket.Send(req, req.Length, 0);
var bytesReceived = new byte[256];
var bytes = 0;
var response = "";
do
{
bytes = snapSocket.Receive(bytesReceived, bytesReceived.Length, 0);
response = response + Encoding.ASCII.GetString(bytesReceived, 0, bytes);
} while (bytes > 0);
Console.WriteLine(response);
But everything halts at snapSocket.Receive - a response is never sent. I suspect that there's something wrong with the message that I'm sending it.

It turns out that it expects a standard HTTP request, which means a Host: line, a Connection: Close line, and two \ns at the very end are required.
The documentation's following claim...
While it is expected to allow clients to connect using HTTPS over a TCP socket, at this point only a UNIX socket is supported.
... is meant only to imply that HTTPS and TCP do not work yet - HTTP is currently the valid request format even when using the UNIX Socket.

I am not fluent in C# at all, but maybe this python snippet can help lead into a solution:
import requests_unixsocket
session = requests_unixsocket.Session()
r = session.get('http+unix://%2Frun%2Fsnapd.socket/v2/snaps')
r.raise_for_status()
r.json()

Proxy won't work in chrome and only partly in firefox

I've written a proxy in c#. It works by getting the http request from the browser sending the request to the site and sending back the site's response to the client. It works in firefox but some of the pages are cut like if it did not send all the response and in chrome it gives blank pages and for google.co.uk in chrome the browser gives "no data received". Can you see a mistake in my code which might be causing all this?
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.Threading;
using System.Windows.Forms;
namespace LexProxy
{
class ProxyServer
{
private TcpListener tcpListener;
public ProxyServer()
{
this.tcpListener = new TcpListener(IPAddress.Any, 3000);
this.tcpListener.Start();
while (true)
{
Console.Write("Waiting for a connection... ");
TcpClient client = tcpListener.AcceptTcpClient();
Thread thread = new Thread(delegate()
{
Serve(client);
});
thread.Start();
}
}
private void Serve(TcpClient client)
{
Console.WriteLine("Connected!");
NetworkStream stream = client.GetStream();
byte[] request = GetBytesFromStream(stream, client.ReceiveBufferSize);
if (request != null)
{
string requestString = System.Text.Encoding.UTF8.GetString(request);
string[] requestParts = requestString.Split(' ');
if (requestParts.Length >= 2)
{
string method = requestParts[0];
if (!requestParts[1].Contains("http://") && !requestParts[1].Contains("https://"))
requestParts[1] = "http://" + requestParts[1];
Uri uri = new Uri(requestParts[1], UriKind.RelativeOrAbsolute);
string host = StringUtils.ReplaceFirst(uri.Host, "www.", "");
int port = uri.Port;
byte[] response = getResponse(host, port, request);
if (response != null)
stream.Write(response, 0, response.Length);
client.Close();
}
}
}
private byte[] getResponse(string host, int port, byte[] request)
{
TcpClient client = new TcpClient(host, port);
NetworkStream stream = client.GetStream();
stream.Write(request, 0, request.Length);
byte[] response = GetBytesFromStream(stream, client.ReceiveBufferSize);
return response;
}
private byte[] GetBytesFromStream(NetworkStream stream, int bufferSize)
{
Byte[] bytes = new Byte[bufferSize];
int i;
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
return bytes;
}
return null;
}
}
}

One minor error is that you're using UTF8 to read first line of request. HTTP request line and headers are ASCII (and body may be UTF8, but doesn't have to be, and may not even be a string at all). The reason it works is b/c for english charset, UTF8 and ASCII happen to be encoded using the same bytes. But that's the minor issue.
Most likely primary issue is hiding in your GetBytesFromStream. You only call .Read once, but that doesn't guarantee that the whole message is returned. It may return as little as just 1 byte... so you need to continue getting more data. How do you know how much more? It's dictated by HTTP protocol which you'd need to properly parse and examine. (At a minimum you're reading until you reach \r\n\r\n byte sequence indicating the end of request headers.
However, this is not enough as there may be a request body, length of which will be specified via an HTTP header, Content-Length: (IIRC), or possibly using chunked encoding. I don't see any code beyond URI examination, so most likely you do not handle HTTP messaging protocol itself, and so it is unlikely to work properly (unless you somehow manage to force browser to use HTTP/0.9 or HTTP/1.0 since those do not reuse connection and send one message per connection, at which point you could just blindly read everything the socket has to give you until browser signals the end of stream by closing its write endpoint of the connection).
But your main problem is that GetBytesFromStream as written won't give you "all" bytes.

Sending custom WCF requests via Sockets

I've seen a few questions left half-answered regarding this topic.
I want to send a request to my self hosted WCF application, which uses NetTcpBinding.
The problem is, I have to use sockets.
I've written a transport binding element which opens a TcpListener(on the ChannelListener OnOpen)
This works fine, but in that case - I'll have to use my own message framing model.
Obviously, I'm not going to implement the net.tcp message framing model( http://msdn.microsoft.com/en-us/library/ff470920%28v=prot.10%29.aspx)
I've read a few posts(including http://blogs.msdn.com/b/carlosfigueira/archive/2008/01/13/writing-custom-requests-to-simple-wcf-services.aspx) which suggest I should send a request and interpret the bytes sent via a MessageEncoder/MessageInspector.
This way, I can basically create a message for each of my methods/operations(with minor
changes per request)
I tried this method and I got the bytes in question(through the use of a custom MessageEncoder) when I used a WCF channel(from a ChannelFactory).
I've saved those bytes, and sent them through a TcpClient - the MessageEncoder wouldn't fire up.
When I Open a normal channel:
ChannelFactory<ITestService> factory = new ChannelFactory<ITestService>(new NetTcpBinding(),
"net.tcp://localhost:76599");
factory.CreateChannel().DoSomething(string.Empty); //gets to MessageEncoder
When I try the same via Sockets:
TcpClient cli = new TcpClient("localhost", 76599);
byte[] fileArray = File.ReadAllBytes("c:\\fileFromMessageEncoder.bin");
cli.GetStream().Write(fileArray, 0, fileArray.Length); // Does not get to MessageEncoder
any ideas?

Finally got it to work.
I initiated the default handshake for a Net.Tcp connection prior to sending the bytes I got via the MessageEncoder.
It's important to note that the endpoint uri must be UTF-8 encoded.
The process is pretty easy - send the negotiation request, send an End Preamble message(0x0C) and wait for the server to return a Preamble Ack message(0x0B).
Then you can send the binary envelope and recieve the response from the WCF application.
I hope this post could help someone in the future.
NegotiationConsts.Via via = new NegotiationConsts.Via(uri);
int arraySize = via.EndpointString.Length +
NegotiationConsts.DefaultRequestLength;
byte[] request = new byte[arraySize];
int count = 0;
request[count++] = NegotiationConsts.Version.RECORD_TYPE;
request[count++] = NegotiationConsts.Version.MAJOR_VERSION;
request[count++] = NegotiationConsts.Version.MINOR_VERSION;
request[count++] = NegotiationConsts.Mode.RECORD_TYPE;
request[count++] = NegotiationConsts.Mode.DUPLEX;
request[count++] = NegotiationConsts.Via.RECORD_TYPE;
request[count++] = via.Length;
via.EndpointString.CopyTo(request,count);
count+=via.EndpointString.Length;
request[count++] = NegotiationConsts.WCFEncoding.RECORD_TYPE;
request[count++] = NegotiationConsts.WCFEncoding.BINARY_ENCODING;
return request;

HttpListener problems: Each HTTP request results in two contexts returned by HttpListener

I have been putting together a little embedded HTTP server in a windows service app that listens for updates coming from other devices on the network that speak HTTP.
For each HTTP request, the code that processes the request/response is executed twice, I expect it to run only once. I tried the code using the AsyncGetContext method and using the synchronous version GetContext - the end result is the same.
Code
public void RunService()
{
var prefix = "http://*:4333/";
HttpListener listener = new HttpListener();
listener.Prefixes.Add(prefix);
try
{
listener.Start();
_logger.Debug(String.Format("Listening on http.sys prefix: {0}", prefix));
}
catch (HttpListenerException hlex)
{
_logger.Error(String.Format("HttpListener failed to start listening. Error Code: {0}", hlex.ErrorCode));
return;
}
while (listener.IsListening)
{
var context = listener.GetContext(); // This line returns a second time through the while loop for each request
ProcessRequest(context);
}
listener.Close();
}
private void ProcessRequest(HttpListenerContext context)
{
// Get the data from the HTTP stream
var body = new StreamReader(context.Request.InputStream).ReadToEnd();
_logger.Debug(body);
byte[] b = Encoding.UTF8.GetBytes("OK");
context.Response.StatusCode = 200;
context.Response.KeepAlive = false;
context.Response.ContentLength64 = b.Length;
var output = context.Response.OutputStream;
output.Write(b, 0, b.Length);
output.Close();
context.Response.Close();
}
Is there anything obvious that I am missing, I have run out of ideas to track down the issue.

Ok, the issue was I was using a web browser to test the HTTP connection and by default a web browser also sends a request for favicon.ico. So two requests were actually coming across. Thank you to #Inuyasha for suggesting I check things out with Wireshark.

how to receive server push data in c#?

I am writing a program. my program receive data from a server through HTTP protocol. the data will be pushed by server to my program.
I tried to use WebRequest, but only received one session of data.
How can i keep the connection alive, to receive the data from server continuosly,
Any help is appreciated.
the following is the SDK document:
Under the authorization of GUEST or ADMIN, it is possible to get the series of live images
(Server push). To get the images, send the request to “/liveimg.cgi?serverpush=1” as shown
in the Figure. 2-1-1.
When the camera receives the above request from the client, it sends the return as shown
in the Figure. 2-2.
Each JPEG data is separated by “--myboundary”, and “image/jpeg” is returned as
“Content-Type” header, after “--myboundary”. For “Content-Length” header, it returns the
number of bytes in the --myboundary data (excluding “--myboundary”, each header, and
\r\n as delimiter). After the “Content-Length” header and “\r\n” (delimiter), the actual
data will be sent.
This data transmission will continue until the client stop the connection (disconnect), or
some network error occurs.
int len;
string uri = #"http://192.168.0.2/liveimg.cgi?serverpush=1";
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(uri);
req.Credentials = new NetworkCredential("admin", "admin");
req.KeepAlive = true;
string line = "";
HttpWebResponse reply = (HttpWebResponse)req.GetResponse();
Stream stream = reply.GetResponseStream();
System.Diagnostics.Debug.WriteLine(reply.ContentType);
StreamReader reader = new StreamReader(stream);
do
{
line = reader.ReadLine();
System.Diagnostics.Debug.WriteLine(line);
System.Threading.Thread.Sleep(300);
} while (line.Length>0);

You can keep an HTTP connection open for an extended period of time, if the server supports doing so. (As already mentioned, this will significantly limit the number of simultaneous users you can support.)
The server will need to be set Response.Buffer=false, and have an extended ScriptTimeout (I'm assuming your using ASP.NET on the server side). Once you do that, your page can keep sending Response.Write data as needed until whatever it is doing is done.
Your client will need to process the incoming Response before the connection is complete rather than blocking for the complete response.

You may want to take a look at StreamHub Push Server - its a popular Comet server and has an .NET Client SDK which allows you to receive real-time push updates in C# (or VB / C++).

If I'm understanding you correctly, your server is going to respond to some event by sending data to your client outside of the client making a request/response. Is this correct? If so, I wouldn't recommend trying to keep the connection open unless you have a very small number of clients -- there are a limited number of connections available, so keeping them open may rapidly result in an exception.
Probably the easiest solution would be to have the clients poll periodically for new data. This would allow you to use a simple server and you'd only have to code a thread on the client to request any changes or new work once every minute or thirty seconds or whatever your optimal time period is.
If you truly want to have the server notify the clients proactively, without them polling, then you'll have to do something other than a simple web server -- and you'll also have to code and configure the client to accept incoming requests. This may be difficult if your clients are running behind firewalls and such. If you go this route, WCF is probably your best choice, as it will allow you to configure server and client appropriately.

You need to get a cookie from IP cam and include that cookie in header of your next HttpWebRequest. Otherways it will always try to redirect you to "index.html".
Here is how you can do it...
BitmapObject is a class that serves as a container for Jpeg image, current date and eventual error text. Once a connection is established it will pool an image every 200 ms. Same should be applicable for continuous image stream obtained through "serverpush".
public void Connect()
{
try
{
request = (HttpWebRequest)WebRequest.Create("Http://192.168.0.2/index.html");
request.Credentials = new NetworkCredential(UserName,Password);
request.Method = "GET";
response = (HttpWebResponse)request.GetResponse();
WebHeaderCollection headers = response.Headers;
Cookie = headers["Set-Cookie"];//get cookie
GetImage(null);
}
catch (Exception ex)
{
BitmapObject bitmap = new BitmapObject(Properties.Resources.Off,DateTime.Now);
bitmap.Error = ex.Message;
onImageReady(bitmap);
}
}
private Stream GetStream()
{
Stream s = null;
try
{
request = (HttpWebRequest)WebRequest.Create("http://192.168.0.2/liveimg.cgi");
if (!Anonimous)
request.Credentials = new NetworkCredential(UserName, Password);
request.Method = "GET";
request.KeepAlive = KeepAlive;
request.Headers.Add(HttpRequestHeader.Cookie, Cookie);
response = (HttpWebResponse)request.GetResponse();
s = response.GetResponseStream();
}
catch (Exception ex)
{
BitmapObject bitmap = new BitmapObject(Properties.Resources.Off,DateTime.Now);
bitmap.Error = ex.Message;
onImageReady(bitmap);
}
return s;
}
public void GetImage(Object o)
{
BitmapObject bitmap = null;
stream = GetStream();
DateTime CurrTime = DateTime.Now;
try
{
bitmap = new BitmapObject(new Bitmap(stream),CurrTime);
if (timer == null)//System.Threading.Timer
timer = new Timer(new TimerCallback(GetImage), null, 200, 200);
}
catch (Exception ex)
{
bitmap = new BitmapObject(Properties.Resources.Off, CurrTime);
bitmap.Error = ex.Message;
}
finally
{
stream.Flush();
stream.Close();
}
onImageReady(bitmap);
}

If you are using a standard web server, it will never push anything to you - your client will have to periodically pull from it instead.
To really get server push data you have to build such server yourself.