I'm trying to get half-duplex ssh connection in C#/.NET 5 with SSH.NET for sending protobuf-formatted packets. (Although full-duplex would be nice, I can emulate that using two streams)
So far I'm able to get data back and forth, however with a few caveats:
I need to send a newline in order to actually commit any info.
I'm only able to send plain text, sending protobuf-formatted binary data creates weird deserialization errors, probably related to encoding-errors.
Currently I'm trying to use SSH.NETs ShellStream on the client-side and deserializing using ProtoBufs non-generic deserializer, mapping types to indexes with a dict:
using SshClient client = new(connectionInfo);
var stream = client.CreateShellStream("", 80, 24, 800, 600, 1024, new Dictionary<TerminalModes, uint>());
stream.WriteLine("./bin/myProcess");
stream.Expect("Running");
// The stream is now at the point where the packets are supposed to be sent
// Two dicts for mapping types to indicies and back
// Just send strings as packets for now
Dictionary<int, Type> typeByIndex = new { [0] = typeof(string) };
Dictionary<Type, int> indexByType = new { [typeof(string)] = 0 };
// Send a packet over
Serializer.SerializeWithLengthPrefix(
stream, "Hello World!",
PrefixStyle.Base128, indexByType[typeof(string)]);
stream.WriteLine(""); // Write newline to commit and use SSH.NET writing method to directly flush
while (true)
{
if (Serializer.NonGeneric.TryDeserializeWithLengthPrefix(
stream, PrefixStyle.Base128,
field => typeByIndex[field-1], out var packet)) Console.WriteLine(packet);
}
With the server (./bin/myProcess) just reading from Console.OpenStandardInput() with the same while-loop.
This produces deserialization errors (Unexpected wire-type usually). My guess is that this has to do with encoding problems.
Given the not to great documentation of SSH.NET I don't really know what to try here.
To sum up:
How would I configure SSH.NET to allow sending arbitrary protobuf-formatted binary data back and forth (without any kind of encoding conversions or control characters causing issues) and how would I get around having to send a newline with every packet?
It sounds like the Stream it is giving you here is really a wrapper over the text-based console, and doesn't really support arbitrary binary. This is something you should probably ask the package creator - there may be some toggle to enable "binary mode" or similar, but without that - honestly I'm a little surprised it doesn't give you a text-writer/text-reader API instead of a stream.
Worst case, you can base-64 encode the protobuf data, and send that down a text protocol, and reverse it at the other end.
Related
I have a pcap-ng file, and I'd like to extract the source/destination IPs.
According to the winpcap dump file format, the data I'm looking for is in the Packet Data section of the enhanced packet block.
I've been using this library in C# to parse through the pcap-ng file. And while I've been able to successfully get out the Enhanced Packet Block, I'm really not sure how to get into it.
The current Enhanced Packet Block Packet Data comes out as a byte array, using the following method.
private static void extractEnhancedPacketBlock()
{
var myFile = "\\path\\to\\my.pcapng"
using (StreamWriter file = new StreamWriter(myFile))
{
foreach (var enhancedPacketBlock in reader.EnhancedPacketBlocks)
{
byte[] packetData = enhancedPacketBlock.Data;
Console.WriteLine(BitConverter.ToString(packetData));
}
}
}
Which outputs what you would expect, similar to the following:
79-2C-C8-80-A8-65-00-00-BC-C4-2F-65-09-00-42-00-01-5E...etc
A good answer to this could be a few different things like, guidance on where to look to learn more about what I need to do next. A library that already does that that I could use (I've tried a lot of libraries, and none of them seem to go this deep). Or if you already have some code that does this, that would be awesome. I'm also open to moving to Python if necessary.
Additional info.
I know that I can parse the source IP and destination IP out of the Enhanced Packet Blocks, and I know that it will require a hexadecimal to IP conversion, but I do not know where the IP Hex exists in the Enhanced Packet Blocks. I know it's not in the same place every time, but I need to know how to calculate this.
Use https://github.com/chmorgan/packetnet for parsing the packet data
Example:
var packet = Packet.ParsePacket(LinkLayers.Ethernet, enhancedPacketBlock.Data);
var ip = packet.Extract<IPPacket>();
I have a problem to obtain the right buffer size of my application.
What i read from the site about specifying the buffer size is normally declared before reading.
byte[] buffer = new byte[2000];
And then using to get the result.
However, this method will stop once the received data contains '00', but my return code contains something like this... 5300000002000000EF0000000A00. and the length is not fixed, can be this short until 400 bytes
So the problems comes, if i define a prefixed length like above, eg 2000, the return value is
5300000002000000EF0000000A000000000000000000000000000000000000000000000000000..........
thus making me unable to split the bytes to the correct amount.
Can any1 show me how to obtain the actual received data size from networkstream or any method/cheat to get what i need?
Thanks in advance.
Network streams have no length.
Unfortunately, your question is light on detail, so it's hard to offer specific advice. But you have a couple of options:
If the high-level protocol being used here offers a way to know the length of the data that will be sent, use that. This could be as simple as the remote host sending the byte count before the rest of the data, or some command you could send to the remote host to query the length of the data. Without knowing what high-level protocol you're using, it's not possible to say whether this is even an option or not.
Write the incoming data into a MemoryStream object. This would always work, whether or not the high-level protocol offers a way to know in advance how much data to expect. Note that if it doesn't, then you will simply have to receive data until the end of the network stream.
The latter option looks something like this:
MemoryStream outputStream = new MemoryStream();
int readByteCount;
byte[] rgb = new byte[1024]; // can be any size
while ((readByteCount = inputStream.Read(rgb, 0, rgb.Length)) > 0)
{
outputStream.Write(rgb, 0, readByteCount);
}
return outputStream.ToArray();
This assumes you have a network stream named "inputStream".
I show the above mainly because it illustrates the more general practice of reading from a network stream in pieces and then storing the result elsewhere. Also, it is easily adapted to directly reading from a socket instance (you didn't mention what you're actually using for network I/O).
However, if you are actually using a Stream object for your network I/O, then as of .NET 4.0, there has been a more convenient way to write the above:
MemoryStream outputStream = new MemoryStream();
inputStream.CopyTo(outputStream);
return outputStream.ToArray();
The Client sends a 1481 bytes array.
The server can read all the 1481 bytes message without any problems but by parsing the given messsage from the received binary array i get this exeption:
com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero).
The binary data is the same. I checked that I am using the right version of the proto files. I am a bit at a loss tbh. Any help appreciated.
Code
byte [] data= IOUtils.toByteArray(br1, "ASCII");
System.out.println("SIZE:" + data.length);
AddressBook adb1 = AddressBook.parseFrom(data); System.out.println("Server: Addressbook:" + adb1.getPersonCount()); System.out.println("Server: Addressbook:" + adb1.getPerson(0).getName());
Question:
I need to find a way to correctly parse the received Adressbook msg from the read 1481 bytes arry.
Thanks.
This is the problem:
br1 = new InputStreamReader(s.getInputStream());
That's trying to treat opaque binary data as text. It's not text, it's binary data. So when you convert that Reader into a byte array, you've lost a load of the original data - no wonder it's an invalid protocol buffer.
Just use:
AddressBook adb1 = AddressBook.parseFrom(s.getInputStream());
and avoid the lossy text conversion. That's assuming you haven't got something equally broken on the C# side, of course.
If you must go via text, you should use base64 encoding on both sides.
Now it works I had same mistake by Serializing and Sending the Protocol Buffers Message
So i am trying to create something like a syncronized paint program by using sockets.I have a server side..and a client side and i am trying to send the inkCollection from the server to the client.This works for simple text, but i cant seem to send the inkCollection.Or it would be even cooler if you could help me send the last stroke so that the data transfers faster.Here is some code of what i've been trying:
sending strokes:
byte[] data;
using (MemoryStream ms = new MemoryStream())
{
inkcanvas.Strokes.Save(ms);
data = ms.ToArray();
}
svr.SendToAll("u getting some strokes");
svr.SendStrokes(data);
svr.SendStrokes(byte[] data):
public void SendStrokes(byte[] data)
{
for (int i = 0; i < no; i++)
{
byte[] dt = data;
accepted[i].Send(dt);
}
MessageBox.Show("dONE");
}
and this is on the clientside:
byte[] buffer=new byte[1024];
MessageBox.Show("receiving strokes");
int rec = conn.Receive(buffer, 0, buffer.Length, 0);
if (rec <= 0)
throw new SocketException();
MessageBox.Show("strokes received");
//Array.Resize(ref buffer, rec);
using (MemoryStream ms = new MemoryStream(buffer))
{
inkcanvas.Strokes = new System.Windows.Ink.StrokeCollection(ms);
ms.Close();
}
MessageBox.Show("strokes added");
these exact same methods work perfectly for string, but when i try to do it with the strokeCollection, it fails.Nothing shows up on the client and i get the following SocketException ont the serverside: An existing connection was forcibly closed by the remote host.
But if you guys got a better way on how to do this it would be great...is it something i am missing? i mean..if it works for text transformed into a byte array...why wouldint it work for this strokecollection?
Thanks!
EDIT: You think you could help me out with some sample code? i cant really seem to implement it;
You forgot to either design or implement a protocol! You can't just send a bunch of bytes over TCP and assume the receiver will be able to make sense out of it.
You have an application message that consists of a bunch of strokes which you are trying to send over TCP. But TCP is a byte stream service, not an application message service. So you need some kind of application message protocol to package the message for transport and unpackage it on receipt. But you have not written any such code. So you're basically expecting it to work by magic.
Define a protocol. For example, it might say:
Canvas strokes shall be sent by a '1' byte indicating canvas strokes, followed by 4 bytes indicating the number of bytes in the strokes object in network byte order, followed by the stroke data. The receiver will read the first byte and identify that it's a canvas strokes object. Then the receiver will read the next four bytes to determine the length. The receiver shall accumulate that number of bytes (using multiple reads if necessary) and then process the reconstructed canvas strokes object.
Do not skip the step of creating a written protocol definition.
Then, when you have problems, follow this handy troubleshooting guide:
Does the sender follow the specification? If not, stop, the sender is broken.
Does the receiver follow the specification? If not, stop, the receiver is broken.
Stop, the specification is broken.
If you want simple, you can convert the data into base64 and encode each message as a line of text. That will allow you to use a ReadLine function to grab exactly one message. Then you can just use a message format like an "S" (for "strokes") followed by the data in base64 format. Use a WriteLine function to send the text message followed by a newline.
i think you forgot to set the memorystream position. you should set the memory stream position to 0 before you send out the stream, cause after strokes.save, the position of the stream is at the end.
I have to interface with a slightly archaic system that doesn't use webservices. In order to send data to this system, I need to post an XML document into a form on the other system's website. This XML document can get very large so I would like to compress it.
The other system sits on IIS and I use C# my end. I could of course implement something that compresses the data before posting it, but that requires the other system to change so it can decompress the data. I would like to avoid changing the other system as I don't own it.
I have heard vague things about enabling compression / http 1.1 in IIS and the browser but I have no idea how to translate that to my program. Basically, is there some property I can set in my program that will make my program automatically compress the data that it is sending to IIS and for IIS to seamlessly decompress it so the receiving app doesn't even know the difference?
Here is some sample code to show roughly what I am doing;
private static void demo()
{
Stream myRequestStream = null;
Stream myResponseStream = null;
HttpWebRequest myWebRequest = (HttpWebRequest)System.Net
.WebRequest.Create("http://example.com");
byte[] bytMessage = null;
bytMessage = Encoding.ASCII.GetBytes("data=xyz");
myWebRequest.ContentLength = bytMessage.Length;
myWebRequest.Method = "POST";
// Set the content type as form so that the data
// will be posted as form
myWebRequest.ContentType = "application/x-www-form-urlencoded";
//Get Stream object
myRequestStream = myWebRequest.GetRequestStream();
//Writes a sequence of bytes to the current stream
myRequestStream.Write(bytMessage, 0, bytMessage.Length);
//Close stream
myRequestStream.Close();
WebResponse myWebResponse = myWebRequest.GetResponse();
myResponseStream = myWebResponse.GetResponseStream();
}
"data=xyz" will actually be "data=[a several MB XML document]".
I am aware that this question may ultimately fall under the non-programming banner if this is achievable through non-programmatic means so apologies in advance.
I see no way to compress the data on one side and receiving them uncompressed on the other side without actively uncompressing the data..
No idea if this will work since all of the examples I could find were for download, but you could try using gzip to compress the data, then set the Content-Encoding header on the outgoing message to gzip. I believe that the Length should be the length of the zipped message, although you may want to play with making it the length of the unencoded message if that doesn't work.
Good luck.
EDIT I think the issue is whether the ISAPI filter that supports compression is ever/always/configurably invoked on upload. I couldn't find an answer to that so I suspect that the answer is never, but you won't know until you try (or find the answer that eluded me).