I'm creating a socket application which is able to receive strings from a websocket within a web page. I have been able to successfully connect the Websocket to my C# program but when ever the webpage sends a string to the program it seems to be encrypted or hashed in some way.
For example if the webpage sends "Test" the program would then output "???9uu?\". I'm obviously missing a step here and I'm not sure what I should searching for to resolve this issue. I'm guessing the string has to be decrypted or put trough a specific function with the TCP key in order to get the actual string?
The code below is the section responsible for receiving the strings from the HTML, (Both "Data" and "MyWriter" output the same string):
while (true)
{
CollectedBytes = new byte[128];
stream.Read(CollectedBytes, 0, CollectedBytes.Length);
string Data = Encoding.ASCII.GetString(CollectedBytes, 0, CollectedBytes.Length);
Output.Speak("Message: " + Data);
StringWriter MyWriter = new StringWriter();
HttpUtility.HtmlDecode(Data, MyWriter);
Output.Speak("Message: " + MyWriter.ToString());
// The word "Test" should output here
// But instead "???9uu?\" is.
}
I'm assuming that I'm missing a simple step but I've looked everywhere and can't seem to find anything to help me!! If anyone can give me guidance on what I should do that would be great :)
Thanks in advance.
Are you trying to decode data manually without using any WebSocket library? If so, you must know that the payload part of WebSocket frames from clients is masked. See RFC 6455, 5.3. Client-to-Server Masking.
Related
I have a pcap-ng file, and I'd like to extract the source/destination IPs.
According to the winpcap dump file format, the data I'm looking for is in the Packet Data section of the enhanced packet block.
I've been using this library in C# to parse through the pcap-ng file. And while I've been able to successfully get out the Enhanced Packet Block, I'm really not sure how to get into it.
The current Enhanced Packet Block Packet Data comes out as a byte array, using the following method.
private static void extractEnhancedPacketBlock()
{
var myFile = "\\path\\to\\my.pcapng"
using (StreamWriter file = new StreamWriter(myFile))
{
foreach (var enhancedPacketBlock in reader.EnhancedPacketBlocks)
{
byte[] packetData = enhancedPacketBlock.Data;
Console.WriteLine(BitConverter.ToString(packetData));
}
}
}
Which outputs what you would expect, similar to the following:
79-2C-C8-80-A8-65-00-00-BC-C4-2F-65-09-00-42-00-01-5E...etc
A good answer to this could be a few different things like, guidance on where to look to learn more about what I need to do next. A library that already does that that I could use (I've tried a lot of libraries, and none of them seem to go this deep). Or if you already have some code that does this, that would be awesome. I'm also open to moving to Python if necessary.
Additional info.
I know that I can parse the source IP and destination IP out of the Enhanced Packet Blocks, and I know that it will require a hexadecimal to IP conversion, but I do not know where the IP Hex exists in the Enhanced Packet Blocks. I know it's not in the same place every time, but I need to know how to calculate this.
Use https://github.com/chmorgan/packetnet for parsing the packet data
Example:
var packet = Packet.ParsePacket(LinkLayers.Ethernet, enhancedPacketBlock.Data);
var ip = packet.Extract<IPPacket>();
I have created a simple server using socket programming in C# which will receive a file from the client side. My sample code segment is given below.
I want to add some restrictions. I want to make a limit on the file size (such as 4 KB or 2 KB) and allowable file formats (such as .doc, .txt, .cpp, etc.) which will be sent to the client as soon as the client connects to the server so that the client can send files accordingly. How will I do that?
Sample code segment:
using System;
using System.Collections.Generic;
using System.Net;
using System.Net.Sockets;
using System.IO;
using System.Text;
namespace FileTransfer
{
class Program
{
static void Main(string[] args)
{
// Listen on port 1234
TcpListener tcpListener = new TcpListener(IPAddress.Any, 1234);
tcpListener.Start();
Console.WriteLine("Server started");
//Infinite loop to connect to new clients
while (true)
{
// Accept a TcpClient
TcpClient tcpClient = tcpListener.AcceptTcpClient();
Console.WriteLine("Connected to client");
byte[] data = new byte[1024];
NetworkStream ns = tcpClient.GetStream();
int recv = ns.Read(data, 0, data.Length);
StreamReader reader = new StreamReader(tcpClient.GetStream());
//Will add some lines to add restrictions...
}
}
}
}
Which additional lines will I have to add to the code to send the restrictions to client?
Basically I think mainly you need two things:
define application protocol as suggested in other answer
and handle partial read/writes
For handling partial reads (not sure how much such function is needed for write) you may use function like below:
public static void ReadWholeArray (Stream stream, byte[] data)
{
int offset=0;
int remaining = data.Length;
while (remaining > 0)
{
int read = stream.Read(data, offset, remaining);
if (read <= 0)
throw new EndOfStreamException
(String.Format("End of stream reached with {0} bytes left to read", remaining));
remaining -= read;
offset += read;
}
}
Thing is traditional Stream.Read() doesn't guarantee to read as many bytes as you told it, this method on the other hand, will ensure to have read as many bytes as specified in data.Length parameter. So you can use such function to implement the desired application protocol instead.
Some relevant information about such application protocols you will find here too
Ok this is for example how the server could send file length limit and the file extension:
// Send string
string ext = ".txt";
byte [] textBytes = Encoding.ASCII.GetBytes(ext);
ns.Write(textBytes, 0, textBytes.Length);
// Now, send integer - the file length limit parameter
int limit = 333;
byte[] intBytes = BitConverter.GetBytes(limit);
ns.Write(intBytes, 0, intBytes.Length); // send integer - mind the endianness
But you will still need some kind of protocol otherwise you should let client read the "full" stream and parse these data later somehow, which isn't trivial if the data doesn't have fixed length etc - otherwise how will the client distinguish which part of the message is text, which integer?
You seem to be making the classical socket mistake. The given code and explanation seem to assume sockets handle in messages. They don't. When used this way, you're using streaming internet sockets, which provide a stream, not messages.
You don't show any code that does the actual sending, so I'm guessing that you just pump a file's data to the other side and close the connection. How else will you know you've successfully transferred an entire file?
This set of rules that client and server have to follow in order to usefully exchange data through sockets is called an application protocol. You will have to have one, otherwise you'll just be sending data to $deity knows where, and you'll have no control over it at all. This means server nor client will know what's going on, they'll just be sending and receiving data and hoping all goes well. So there's not "a few lines" you have to add to your code, you'll have to restructure it entirely.
There are many ways to define an application protocol and many options to choose from, so I'm going to show you an arbitrary one: a textual explanation of messages that are prefixed with an ID and a payload length (if applicable), both in unspecified numeric variables. You could choose little-endian four-byte unsigned integers, for example.
Messages in this format are known as "Type/Length/Value" or TLV.
So we define these messages:
ID Name Direction Description Payload
1 ServerHello Server -> Client The server sends this message None.
to every connecting client. Or maybe server or
protocol version.
2 MaxUpload Server -> Client Sent after the ServerHello. Maximum upload size
in bytes.
3 AllowedExts Server -> Client Allowed upload extensions, The allowed extensions.
comma-separated. Sent after
MaxUpload message.
10 IncomingFile Client -> Server There's a file coming. The file name.
11 FileUpload Client -> Server The file to upload. The file data.
Sent after IncomingFile.
Now all that's required is to implement this application protocol in server and client and you're done.
You also have to decide what to do if a client or server doesn't adhere to the prototol. It can for example send a message that you can't parse, an unknown message ID, a message length that you don't want to support, an out-of-order message (FileUpload before IncomingFile) or a message that isn't conform the messages sent earlier, like a client uploading a larger file than the server said it would accept or an invalid extension. You also have to think about "acknowledgement" or response messages, like the server telling the client "OK, go ahead, send the next message".
All in all, this is a very broad question and not answered easily. I tried to address that in my comment to your question, which got removed. So here you have your answer.
You can learn more about this on the web, for example Beej's Guide to Network Programming as linked to by Giorgi (be sure to read the entire guide) and Stephen Cleary's blog.
First of all, english is my foreign language.
Under C# I am trying to send POST http request to a forvo.com (bank with pronounces of words) to get an audio file (stream) with pronunciation of some word. For example, I want to click button1 and listen pronunciation of word "stack" that I got from forvo.com site. For this I have a code:
using (var wb = new WebClient())
{
var data = new NameValueCollection();
data["LANGUAGE_ID"] = "39";
data["WORD"] = "someword"; //this tag doesnt make effect to request
//var response = wb.UploadValues("http://www.forvo.com/search/", "POST", data);
byte[] responseArray = wb.UploadValues("http://www.forvo.com/search/data", "POST", data);
File.WriteAllBytes(Path.GetDirectoryName(Application.ExecutablePath) + "\\fi2le.html", responseArray);
}
With that actions I am trying to get a link to an audio-file. After looking to a file.html I see these strings
<img src="/_presentation/img/ico_play.gif" alt="data pronunciation" id="play_38630" width="23" height="23" />data and others looking similiar.
What can I do with these js-functions or what is it?
I am a newbie in web-programming, and know nothing about js.
and I know forvo.com has its own API for tasks like mine, but I'm trying make it without API.
Looking at what's going on here, the click on the icon triggers the Javascript Play() function.
This function decodes those base 64-encoded values passed to it, and uses them to find the relevant file to play, and whether you are using a browser capable of understanding an audio element (HTML5) in which to play it, or, if not, it creates a Flash player in which to play it.
The second and third arguments to the Play() function are, as mentioned, the base-64 encoded URIs of the files to play. They can be decoded quite simply as follows (example is for a console application):
static void Main()
{
string base64a = "ODk3NTU5NS8zOS84OTc1NTk1XzM5XzE3MjNfMjI4MjAubXAz";
string base64b = "ODk3NTU5NS8zOS84OTc1NTk1XzM5XzE3MjNfMjI4MjAub2dn";
Console.WriteLine(Encoding.Default.GetString(Convert.FromBase64String(base64a)));
Console.WriteLine(Encoding.Default.GetString(Convert.FromBase64String(base64b)));
Console.ReadKey();
}
Should you run this, you'll find that they convert to:
8975595/39/8975595_39_1723_22820.mp3
8975595/39/8975595_39_1723_22820.ogg
There's some logic in the Play function to determine which one to use, but ultimately, I end up playing the following file:
http://audio.forvo.com/mp3/8975595/39/8975595_39_1723_22820.mp3
This work needs to be done in Javascript, as much is dependent upon the browser or device that is making the request, as highlighted above (e.g. HTML5 capability)
I must add, though, that whilst this hopefully explains what is going on, I doubt it would be of any use as a solution; as a paywall is in existence at 500 requests, I'd have thought that they would be on the lookout for multiple requests from the same IP address over a period of time, and would block any such address that was avoiding payment.
The Client sends a 1481 bytes array.
The server can read all the 1481 bytes message without any problems but by parsing the given messsage from the received binary array i get this exeption:
com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero).
The binary data is the same. I checked that I am using the right version of the proto files. I am a bit at a loss tbh. Any help appreciated.
Code
byte [] data= IOUtils.toByteArray(br1, "ASCII");
System.out.println("SIZE:" + data.length);
AddressBook adb1 = AddressBook.parseFrom(data); System.out.println("Server: Addressbook:" + adb1.getPersonCount()); System.out.println("Server: Addressbook:" + adb1.getPerson(0).getName());
Question:
I need to find a way to correctly parse the received Adressbook msg from the read 1481 bytes arry.
Thanks.
This is the problem:
br1 = new InputStreamReader(s.getInputStream());
That's trying to treat opaque binary data as text. It's not text, it's binary data. So when you convert that Reader into a byte array, you've lost a load of the original data - no wonder it's an invalid protocol buffer.
Just use:
AddressBook adb1 = AddressBook.parseFrom(s.getInputStream());
and avoid the lossy text conversion. That's assuming you haven't got something equally broken on the C# side, of course.
If you must go via text, you should use base64 encoding on both sides.
Now it works I had same mistake by Serializing and Sending the Protocol Buffers Message
I have to interface with a slightly archaic system that doesn't use webservices. In order to send data to this system, I need to post an XML document into a form on the other system's website. This XML document can get very large so I would like to compress it.
The other system sits on IIS and I use C# my end. I could of course implement something that compresses the data before posting it, but that requires the other system to change so it can decompress the data. I would like to avoid changing the other system as I don't own it.
I have heard vague things about enabling compression / http 1.1 in IIS and the browser but I have no idea how to translate that to my program. Basically, is there some property I can set in my program that will make my program automatically compress the data that it is sending to IIS and for IIS to seamlessly decompress it so the receiving app doesn't even know the difference?
Here is some sample code to show roughly what I am doing;
private static void demo()
{
Stream myRequestStream = null;
Stream myResponseStream = null;
HttpWebRequest myWebRequest = (HttpWebRequest)System.Net
.WebRequest.Create("http://example.com");
byte[] bytMessage = null;
bytMessage = Encoding.ASCII.GetBytes("data=xyz");
myWebRequest.ContentLength = bytMessage.Length;
myWebRequest.Method = "POST";
// Set the content type as form so that the data
// will be posted as form
myWebRequest.ContentType = "application/x-www-form-urlencoded";
//Get Stream object
myRequestStream = myWebRequest.GetRequestStream();
//Writes a sequence of bytes to the current stream
myRequestStream.Write(bytMessage, 0, bytMessage.Length);
//Close stream
myRequestStream.Close();
WebResponse myWebResponse = myWebRequest.GetResponse();
myResponseStream = myWebResponse.GetResponseStream();
}
"data=xyz" will actually be "data=[a several MB XML document]".
I am aware that this question may ultimately fall under the non-programming banner if this is achievable through non-programmatic means so apologies in advance.
I see no way to compress the data on one side and receiving them uncompressed on the other side without actively uncompressing the data..
No idea if this will work since all of the examples I could find were for download, but you could try using gzip to compress the data, then set the Content-Encoding header on the outgoing message to gzip. I believe that the Length should be the length of the zipped message, although you may want to play with making it the length of the unencoded message if that doesn't work.
Good luck.
EDIT I think the issue is whether the ISAPI filter that supports compression is ever/always/configurably invoked on upload. I couldn't find an answer to that so I suspect that the answer is never, but you won't know until you try (or find the answer that eluded me).