I have an exe which simulates a video stream. I connect to it and very occasionally read the expected data, but usually I get only the first 28 bytes and then 65508 bytes of zeros. Assume the video stream is working correctly.
TcpClient tcpClient = new TcpClient ();
int port = 13000;
myIP = IPAddress.Loopback.ToString();
tcpClient.Connect (myIP, port);
NetworkStream netStream = tcpClient.GetStream ();
byte[] bytes = new byte[tcpClient.ReceiveBufferSize];
netStream.Read (bytes, 0, (int)tcpClient.ReceiveBufferSize);
string dataString = Encoding.ASCII.GetString (bytes);
Console.WriteLine("\ndataString: "+dataString.Substring(0,1000));
Console.WriteLine("\nnumber of bytes read: "+bytes.Length);
tcpClient.Close ();
// Closing the tcpClient instance does not close the network stream.
netStream.Close();
How can I make it so that I get the expected output every time?
TCP represents a (bi-directional) stream of data. You're supposed to keep reading from it in a loop, and parsing the data as you need them. It doesn't have a concept of messages - ten writes on one side can result in a single read on the other side just as easily as one write on one side can result in ten reads on the other side.
The contract you have with TCP is as follows:
If there is data in the receive buffer, Read returns immediately, filling the buffer you provided with as much data as is available, up to the length of the buffer. The number of bytes read is the return value of Read.
If there is no data in the receive buffer, Read will block until there's at least a single byte of data. Then it follows as in the first case.
If the other sides shuts down the socket, Read will return zero.
So to get TCP working, you need a loop. How exactly you form the loop depends on what you're trying to do. If you're really working with data that is logically a stream (e.g. audio data), just keep reading as fast as you can and process whatever data you get as it comes in. If you need to send messages, you need to implement a message protocol. If you need a one-off message, you can just keep reading until Read returns zero.
Your case can be handled with the first approach - keep reading until the stream closes, and push the received data forward. Assuming the data is actually a UTF8 stream of text, the basic receiver would look something like this:
using (var client = new TcpClient())
{
tcpClient.Connect(myIP, port);
var stream = client.GetStream();
var buffer = new byte[4096]; // Adapt the size based on what you want to do with the data
var charBuffer = new char[4096];
var decoder = Encoding.UTF8.GetDecoder();
int bytesRead;
while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) != 0)
{
var expectedChars = decoder.GetCharCount(buffer, 0, bytesRead);
if (charBuffer.Length < expectedChars) charBuffer = new char[expectedChars];
var charCount = decoder.GetChars(buffer, 0, bytesRead, charBuffer, 0);
Console.Write(new string(charBuffer, 0, charCount));
}
client.Close();
}
Note that this does no error handling, so don't use it as is in any production code. You would probably also want to use Async methods if you're expecting more than a few simultaneous connections. It's just to illustrate the basic way one would handle a stream of data being received over TCP.
If you want some more insight into dealing with TCP, I have a few very simple examples at https://github.com/Luaancz/Networking. I haven't found any good tutorials or code samples for C#, so if this isn't enough, you'll probably have to dig deeper into the documentation around sockets, TCP and all that.
Or just use an existing networking library, rather than trying to write your own :) TCP is still very low level.
Related
I want to send a string converted in byte but i don't know how to do that. I already tried to modify buffer and byteread but the server is just broken.
private void Server(string ip, int port)
{
BackgroundWorker worker = new BackgroundWorker();
worker.DoWork += delegate (object s, DoWorkEventArgs args)
{
//---listen at the specified IP and port no.---
IPAddress localAdd = IPAddress.Parse(ip);
TcpListener listener = new TcpListener(localAdd, port);
Console.WriteLine("Listening...");
listener.Start();
while (true)
{
//---incoming client connected---
TcpClient client = listener.AcceptTcpClient();
//---get the incoming data through a network stream---
NetworkStream nwStream = client.GetStream();
byte[] buffer = new byte[client.ReceiveBufferSize];
//---read incoming stream---
int bytesRead = nwStream.Read(buffer, 0, client.ReceiveBufferSize);
//---convert the data received into a string---
string dataReceived = Encoding.ASCII.GetString(buffer, 0, bytesRead);
Console.WriteLine("Received : " + dataReceived);
//---write back the text to the client---
Console.WriteLine("Sending back : " + dataReceived);
nwStream.Write(buffer, 0, bytesRead); // How do i send back a string ??
client.Close();
}
};
worker.RunWorkerAsync();
}
You already are sending back the bytes that represent the same string (via ASCII) that you received in the first Read, then closing the socket. Note: if you intended to send back the bytes for a different string value: Encoding.ASCII or Encoding.Utf8 are your friends; the process of converting between string data and byte data (for transmission or storage) is exactly what an Encoding does. You're already using this with GetString(...); the reverse operation is GetBytes(...) (with various overloads for using pooled buffers etc).
If this isn't working as intended, we'd need a lot more detail. In particular, TCP is a stream protocol; data isn't terminated inherently (like it would be in UDP packets, for example), so there is no guarantee whatsoever that you've received a logical unit of string data in your Read - all we know is that one Read has completed. This could return zero bytes (indicating EOF, i.e. the other end didn't send anything and closed their outbound socket), one byte, or a few thousand bytes - but: except for the EOF case, that tells us nothing about whether we have an entire "message". In fact, if you were using UTF8 rather than ASCII, we might not even have an entire character - we could have partial/incomplete character data.
So if the "other end" is trying to send us a paragraph of data, we might have received the first word-and-a-half; you then send back the same word-and-a-half, and terminate the connection. We have never seen the rest of the data. Whether this is what you want is unclear, but seems unlikely. That's assuming this actually is a text-based protocol!
For this reason, usually any network protocol includes framing details to tell us how to identify logical units of data in the stream. For a text protocol this might be looking for sentinel characters like CR/LF/NUL; for a binary protocol, this usually takes the form of some header that tells us the number of bytes in the data frame in a well-defined format ("big endian int32", "varint", etc), followed by that same number of bytes. The caller then buffers appropriately so that they're processing only entire frames.
So: in theory you're already sending something back (unless it was EOF), but without the protocol details: we can't say whether than means anything.
I have an asynchronous TCP socket server in C# that I've made using the TcpListener/TcpClient wrappers for Socket. I'm pretty new to networking in general so I was unaware of how TCP treats data sent, and how it doesn't preserve the message boundaries. After a little research I think I've come up with a solid solution but I'm wondering if anyone has more advice for me.
Currently the data I'm sending is a byte[] of a serialized class object using protobuf (Google's data exchange library) https://code.google.com/p/protobuf/
After I serialize my data, and before it's sent, I've decided to append a 4 byte Int32 at the beginning of the byte array. My idea is that when the packet is sent to the server, it will parse out the Int32 and then wait until it's received that number of bytes before doing anything with the data, otherwise just calling BeginRead again.
Here is the code it's ran through before I write the data, and it seems to work fine, but I'm open to any performance suggestions:
public byte[] PackByteArrayForSending(byte[] data)
{
// [0-3] Packet Length
// [3-*] original data
// Get Int32 of the packet length
byte[] packetLength = BitConverter.GetBytes(data.Length);
// Allocate a new byte[] destination array the size of the original data length plus the packet length array size
byte[] dest = new byte[packetLength.Length + data.Length];
// Copy the packetLength array to the dest array
Buffer.BlockCopy(packetLength, 0, dest, 0, packetLength.Length);
// Copy the data array to the dest array
Buffer.BlockCopy(data, 0, dest, packetLength.Length, data.Length);
return dest;
}
I'm a little stuck on the server end. I have it reading the packetLength variable by using Buffer.BlockCopy to copy the first 4 bytes and then BitConverter.ToInt32 to read the length I should be getting. I'm not sure if I should constantly read the incoming data into a client-specific Stream object, or just use a while loop. Here's an example of the code I have on the server end so far:
NetworkStream networkStream = client.NetworkStream;
int bytesRead = networkStream.EndRead(ar);
if (bytesRead == 0)
{
Console.WriteLine("Got 0 bytes from {0}, marking as OFFLINE.", client.User.Username);
RemoveClient(client);
}
Console.WriteLine("Received {0} bytes.", bytesRead);
// Allocate a new byte array the size of the data that needs to be deseralized.
byte[] data = new byte[bytesRead];
// Copy the byte array into the toDeserialize buffer
Buffer.BlockCopy(
client.Buffer,
0,
data,
0,
bytesRead);
// Read the first Int32 tp get the packet length and then use the byte[4] to get the packetLength
byte[] packetLengthBytes = new byte[4];
Buffer.BlockCopy(
data,
0,
packetLengthBytes,
0,
packetLengthBytes.Length);
int packetLength = BitConverter.ToInt32(packetLengthBytes, 0);
// Now what do you recommend?
// If not all data is received, call
// networkStream.BeginRead(client.Buffer, 0, client.Buffer.Length, ReadCallback, client);
// and preserve the initial data in the client object
Thanks for your time and advice, I look forward to learning more about this subject.
TCP guarantees that the bytes stuffed into the stream at one end will fall out the other end in the same order and without loss or duplication. Expect nothing more, certainly not support for entities larger than a byte.
Typically I have prepended a header with a message length, type and subtype. There is often a correlation id provided to match requests to responses.
The basic pattern is to get bytes and append them to a buffer. If the data in the buffer is sufficient to contain a message header then extract the message length. If the data in the buffer is sufficient to contain the message then remove the message from the buffer and process it. Repeat with any remaining data until there are no complete messages to process. Depending on your application this may be the point to wait on a read or check the stream for additional data. Some applications may need to keep reading the stream from a separate thread to avoid throttling the sender.
Note that you cannot assume that you have a complete message length field. You may have three bytes left after processing a message and cannot extract an int.
Depending on your messages it may be more efficient to use a circular buffer rather than shuffling any remaining bytes each time a message is processed.
I created a client in Java and I want simply to send some data to my server that is in C#.
The problem is that if I write in the client for example hello, I get only the first letter. In the byte array there is only one element.
I guess there is some problem with the server side because in my server in JAVA works everything fine so the client in JAVA works fine.
Does anybody see any problem?
Thank you in advance.
You're thinking about TCP the wrong way, you don't simply "Receive" once and get the result of one "Send".
TCP is a "streaming" protocol and does not automatically separate into "packets". You may even get data of 2 sends in one receive.
A common pattern is to prefix one message with its length, so you can call receive until you get the amount of bytes requested. To make Receive return immediately if no data is in the buffer, set your socket to non-blocking.
Here's another good article on the topic.
Now, your provided code should work either way because there is next to no latency on local networks. Have you checked if your Java part buffers steam / can you manually flush them?
As Damon Gant said, TCP is a streaming protocol. I suggest you create your own protocol. I wouldn't send strings. If you're doing anything non-trivial this is really the best way to go.
Typically I include a magic number, checksum, packet body length in bytes, and protocol version in my protocol headers. The magic number makes it easier to delineate packets in a stream (very useful for debugging your custom protocol stream.) Having a checksum helps ensure you're parsing things correctly. A checksum doesn't help much with integrity over TCP as the TCP protocol already has a checksum. The packet body length helps you detect when you have all the bytes for your packet. The protocol version can help you know how to interpret the packet body's bytes.
Upon receiving data, place all bytes into a separate buffer and scan for your protocol header. If you can parse your header, check to see that packet's bytes are all present. If so, parse the packet. Repeat this process till you find an incomplete packet, or the buffer is empty.
For each packet you want to send, I'd create a class. When you want to send a packet, create and serialize the proper class, and prepend your protocol header for that class's bytes.
You could use Java's serializer, but if you've many client's connecting to a single server, you probably don't want to use Java for the server. This makes things difficult because now you need to implement a java serializer in another language. Because of this its typically better to either convert your packets into bytes by hand (tedious but simple,) OR you could write your own serializer using reflection. I'd suggest the latter for bigger projects.
problem is prabably in java side because your listener works fine.
I copy pasted your listener code in a test application.
Than I created another test applicationand send hello word and I listened it completely.
public static void sender()
{
TcpClient client = new TcpClient();
IPEndPoint serverEndPoint = new IPEndPoint(IPAddress.Parse("192.168.2.236"), 30000);
client.Connect(serverEndPoint);
NetworkStream clientStream = client.GetStream();
ASCIIEncoding encoder = new ASCIIEncoding();
byte[] buffer = encoder.GetBytes("Hello Server!");
clientStream.Write(buffer, 0, buffer.Length);
clientStream.Flush();
}
Connection accepted from 192.168.2.236:22811
Recieved...
Hello Server!
Btw, this might be better listener.
public void listener()
{
TcpListener tcpListener = new TcpListener(IPAddress.Any, 30000);
tcpListener.Start();
TcpClient tcpClient = tcpListener.AcceptTcpClient();
NetworkStream clientStream = tcpClient.GetStream();
byte[] message = new byte[4096];
int bytesRead;
while (true)
{
bytesRead = 0;
try
{
//blocks until a client sends a message
bytesRead = clientStream.Read(message, 0, 4096);
}
catch
{
//a socket error has occured
break;
}
if (bytesRead == 0)
{
//the client has disconnected from the server
break;
}
//message has successfully been received
ASCIIEncoding encoder = new ASCIIEncoding();
Console.Write(encoder.GetString(message, 0, bytesRead));
}
tcpClient.Close();
}
Problem just started on client side. Here is my code where I receive TCP/IP message. On my local PC this listener receives many K no problem. I tried to increase buffer size but on client site they still report issues related to it.. Still get's only first 1K (1024 bytes)
public void Start()
{
//Define TCP listener
tcpListener = new TcpListener(IPAddress.Any, IDLocal.LocalSession.PortNumber);
try
{
//Starting TCP listenere
tcpListener.Start();
while (true)
{
var clientSocket = tcpListener.AcceptSocket();
if (clientSocket.Connected)
{
var netStream = new NetworkStream(clientSocket);
// Check to see if this NetworkStream is readable.
if (netStream.CanRead)
{
var myReadBuffer = new byte[1024];
var myCompleteMessage = new StringBuilder();
// Incoming message may be larger than the buffer size.
do
{
var numberOfBytesRead = netStream.Read(myReadBuffer, 0, myReadBuffer.Length);
myCompleteMessage.AppendFormat("{0}", Encoding.ASCII.GetString(myReadBuffer, 0, numberOfBytesRead));
} while (netStream.DataAvailable);
//All we do is response with "OK" message
var sendBytes = Encoding.ASCII.GetBytes("OK");
netStream.Write(sendBytes, 0, sendBytes.Length);
clientSocket.Close();
netStream.Dispose();
// Raise event with message we received
DataReceived(myCompleteMessage.ToString());
}
}
}
}
catch (Exception e)
{
//If we catch network related exception - send event up
IDListenerException(e.Message);
}
}
I don't see any problem with the code you posted to extract the message into a string, so I 'm guessing that something else is afoot.
TCP isn't required to send all data you queue to it in one go. This means it can send as few as it want to at a time, and it can choose to split your data into pieces at will. In particular, it is guaranteed to split your data if they don't fit into one packet. Typically, the maximum packet size (aka MTU) is 1532 bytes IIRC.
Therefore there's a real possibility that the data is sent, but as more than one packet. The delay between reception of the first and second packet could mean that when the first one arrives your code happily reads everything it contains and then stops (no more data) before the second packet has had time to arrive.
You can test this hypothesis by either observing network traffic, or allowing your app to pull more messages from the wire and see if it finally does get all the data you sent (albeit in pieces).
Ultimately the underlying issue is TCP's fundamental stream-based (and not message-based) nature; even if you get this code to work correctly, there is no guarantee that it will continue working in the future because it makes assumptions about stuff that TCP does not guarantee.
To be safe, you will need to incorporate a message-based structure (e.g. prepending each piece of data with exactly 4 bytes that hold its length; then, you can just keep reading forever until you have received that many bytes).
I'm trying to write a simple SNPP (Simple Network Paging Protocol) client using sockets. Everything seems to be working well, except for a small inconsistency between servers.
When I send a command, I need to read the reply, which is usually a single chunk of data. However, Sprint's SNPP server sends replies in two parts. The first chunk of data is the first digit of the status code. The second chunk is the remainder. For example, when I attempt to receive the "220 Gateway ready" reply, it arrives like this:
2
I have to send another empty command to retrieve the rest:
20 Gateway ready
For the moment, I'm using:
byte[] buffer = new byte[256];
socket.Receive(buffer);
How can I make sure that I receive all of the available data after issuing a command without allocating a separate buffer for each chunk of data?
For chunked responses I would recommend you reading data like this:
using (var resultStream = new MemoryStream())
{
const int CHUNK_SIZE = 2 * 1024; // 2KB, could be anything that fits your needs
byte[] buffer = new byte[CHUNK_SIZE];
int bytesReceived;
while ((bytesReceived = socket.Receive(buffer, buffer.Length, SocketFlags.None)) > 0)
{
byte[] actual = new byte[bytesReceived];
Buffer.BlockCopy(buffer, 0, actual, 0, bytesReceived);
resultStream.Write(actual, 0, actual.Length);
}
// Do something with the resultStream, like resultStream.ToArray() ...
}
Try to check Socket.Available property to determine if do you need to call Receive again.
I think I understand your question. This overload of Socket.Receive allows you to pass an integer to specify an offset position to start placing data. If you got 1 byte, on the first call as in your example, you can call this overload with an offset of 1 and use the same buffer.
Socket.Receive returns an integer that is the number of bytes that were received. You could check if this is 1 and call Receive again.
byte[] buffer = new byte[256];
int len = socket.Receive(buffer);
if (len == 1)
socket.Receive(buffer, 1, buffer.Length - 1, SocketFlags.None);
All,
When a web-server is sending data in "chunks" it precedes each chunk with it's length (as a string indicating the hex value).
chunk-size[;chunk-extensions]
chunk-data
eg: to chunk 15 bytes:
F;
123456789ABCDEF
The biggest issue with receiving HTTP data from a Socket is determining how much data to read: if you have received all the available data and call the Recieve method again, then the Recieve method will block until the remote socket sends more data, which for a HTTP/1.0 connection will never happen.
You need to implement a reader that wraps around the socket; it should receive data into a buffer, and provide a "ReadLine" method that reads the contents of the buffer until it reads 13 followed by 10 (CRLF)