I have a huge problem. Trying to create an app that has to have two parts: server and client side. Those two parts have to communicate somehow and exchange objects. I have decides to use Sockets because i'm not familiar with WCF, and i can test both parts on same computer (just put them to listen at 127.0.0.1 address).
Now, when i try to send some "custom" serializable object from client i got "OutOfMemory" exception at server side! I read about Sockets, ways to send/receive objects, i have tried some code i found on net but no luck! I have no idea what's wrong with my code.
Here's my code:
This is test class defined in code of both sides:
[Serializable]
class MyClass
{
public string msg="default";
}
Client-side sending code (works fine):
private void cmdSendData_Click(object sender, System.EventArgs e)
{
try
{
MyClass test = new MyClass();
NetworkStream ns = new NetworkStream(m_socWorker); //m_socWorker is socket
BinaryWriter bw = new BinaryWriter(ns);
MemoryStream ms = new MemoryStream();
BinaryFormatter bf = new BinaryFormatter();
bf.Serialize(ms, test);
bw.Write(ms.ToArray());
MessageBox.Show("Length is: " + ms.ToArray().Length); //length is 152!
ns.Close();
}
catch(System.Net.Sockets.SocketException se)
{
MessageBox.Show (se.Message );
}
}
Server-side code (the one that cause problems):
public void OnDataReceived(IAsyncResult asyn)
{
try
{
CSocketPacket theSockId = (CSocketPacket)asyn.AsyncState ;
NetworkStream ns = new NetworkStream(m_socWorker);
byte[] buffer = new byte[1024];
ns.Read(buffer, 0, buffer.Length);
BinaryFormatter bin = new BinaryFormatter();
MemoryStream mem = new MemoryStream(buffer.Length);
mem.Write(buffer, 0, buffer.Length);
mem.Seek(0, SeekOrigin.Begin);
MyClass tst = (MyClass)bin.Deserialize(mem); //ERROR IS THROWN HERE!
MessageBox.Show(tst.msg);
theSockId.thisSocket.EndReceive(asyn);
WaitForData(m_socWorker);
}
catch (ObjectDisposedException )
{
System.Diagnostics.Debugger.Log(0,"1","\nOnDataReceived: Socket has been closed\n");
}
catch(SocketException se)
{
MessageBox.Show (se.Message );
}
}
So, i got exception when i try to deserialize. Have no idea what's wrong.
I have threatened my code "if you continue causing problems i'll report you to StackOverflow guys" so here i'm :)
There is some very odd code there that:
assumes we read 1024 bytes without checking
copies the 1024 buffer
assumes the serialized data is 1024 bytes, no more no less
deserializes from that
IMO there is your error; you should be reading the correct number of bytes from the stream (usually in a loop). Generally, you would be looping, checking the return value from Read until either we have read the amount of data we wanted, or we get EOF (return <= 0).
Or better; use serializers that do this for you... For example, protobuf-net has SerializeWithLengthPrefix and DeserializeWithLengthPrefix that handle all the length issues for you.
Since you mention "custom" serialization - if you are implementing ISerializable it is also possible that the problem is in there - but we can't see that without code. Besides, the current buffer/stream is so broken (sorry, but it is) that I doubt it is getting that far anyway.
First, while I'm not certain if this is the cause of your issue directly, you have a serious issue with your reading logic.
You create a 1024 byte buffer and read into it without checking to see how much was actually read; if the incoming buffer only has 56 bytes, you'll only read 56 bytes (unless you use a blocking read on the socket itself, which could time out). Likewise, your buffer could have 2000 bytes in it, which means you'd have 976 bytes left in the buffer that won't get processed until you receive more data. That could be an entire object, and the client could be waiting on a response to it before it sends any more.
You then take that buffer and copy it again into a MemoryStream. Rather than doing this, just take the overload of the MemoryStream constructor that takes a buffer. You won't be copying the data that way.
You call EndReceive after you've processed the incoming data; while this may not actually cause an error, it's not in the spirit of the Begin/End old-style async pattern. You should call EndXXX at the beginning of your callback to get the result.
I realize that this is not a direct answer to your question, but you really need to reconsider your decision not to use WCF.
I was in the same boat as you a couple of months ago; I had not used WCF before, and I hadn't bothered to look at how things work in it. It was a very large black box to me, and I had done socket-based communication on other platforms, so it was a known quantity. Looking back, my choice to take the plunge into WCF was the best decision I could have made. Once you've wrapped your head around some of the concepts (bindings, contracts, and how to use the various attributes), development of the service is simple and you don't have to spend your time writing plumbing.
NetTcpBinding provides a TCP-based binding that can support long-lived connections and sessions (which is how I'm using it), and even takes care of keep-alive messages to keep the connection open via Reliable Sessions. It's as simple as turning on a flag. If you need something more interopable (meaning cross-platform), you can write your own binding that does this and keep your code as-is.
Look at some of the TCP WCF examples; it won't take you terribly long to get something up and running, and once you've reached that point, modification is as simple as adding a function to your interface, then a corresponding function on your service class.
Related
I have a program that uses BinaryReader and BinaryWriter to send string, int and byte[] messages over the network.
The order and content of the messages is significant for the flow of the execution on both server and client, but the duration is never long.
Now I'm trying to encrypt everything.
This is the new wrapper that I have around the actual code:
using encReaderStream = CryptoStream(clientStream, myAES.CreateDecryptor(), CryptoStreamMode.Read),\
encWriterStream = CryptoStream(clientStream, myAES.CreateEncryptor(), CryptoStreamMode.Write),\
enc_reader = BinaryReader(encReaderStream),\
enc_writer = BinaryWriter(encWriterStream):
....
This example is boo code, but it should be intuitively equivalent to C# in this instance.
What happens now is that the server gets the first encrypted message correctly with enc_reader.ReadString(), and answers with enc_writer.Write("Accepted"). But the client never gets the answer.
I have tested if the order of the messages is significant, and it is. If I instead start by sending a string from the server, then the client gets it, but if I continue to send messages, I will soon get into the same situation.
I have some idea that CryptoStream might be responsible for not cooperating properly with BinaryReader/BinaryWriter, but I don't know how to sort out this mess in a good way.
My server has a lot of functions that just expects a BinaryReader and a BinaryWriter, and it would be super convenient if they could work like before.
EDIT:
I've also replicated the situation in a small C# project here, using the the mentioned implementation of AES with CTR mode.
You can't flush a block cipher in the middle of a block because those bytes are not ready from an algorithmic standpoint. They are not determined.
Probably, the best fix is to use a stream cipher. .NET has poor built-in support for that. Pull in a library that implements AES in counter mode.
Be aware that without using authenticated encryption (and it does not look like you are going it) attackers can edit the data although they cannot read it. Use AES-GCM to mitigate.
Later we found out that the CTR mode library you are using is broken. Use this:
public int InputBlockSize { get { return 1; } }
public int OutputBlockSize { get { return 1; } }
I am making a .NET application which is supposed to communicate with a Java application over the sockets.
Here is the .NET code:
string str = " MSG1234";
TcpClient client = new TcpClient();
client.Connect("127.0.0.1", 8080);
byte[] msg = Encoding.UTF8.GetBytes(str);
client.Client.Send(msg);
StreamWriter sw = new StreamWriter(client.GetStream());
StreamReader sr = new StreamReader(client.GetStream());
while (true)
{
Console.WriteLine(sr.Read());
}
The problem is that the Java application wouldn't send the answer. The application is a 3rd party piece of software and I can't change it, however I have downloaded a decompiler and found the relevant piece of code in the Java class:
try {
while (this.is.available() <= 0);
body = new byte[this.is.available()];
this.is.readFully(body);
System.out.println("Message received");
} catch (Exception e) {
System.out.println("Error: " + body);
}
"is" in this case is a DataInputStream. Here is what happens:
1. I create a socket and send the message
2. I wait for response, nothing happens
3. I close the app manually - it causes the socket to expire
4. Suddenly the whole message appears in the Java application's log. That means the exception happened and there actually is something in the body.
Can you perhaps point me out where the error could be? I believe the this.is.readFully(body) line shouldn't be there, but I'm not sure. Perhaps I should send an EOF from the C# code manually, but I wasn't able to find out how.
Another, minor problem with that code is, that the first two characters of the message are stripped away, that's why I have included the two space in front of the actual message.
Thank you in advance.
EDIT
So I have tried to fill the socket with some random data and I've finally got the answer:
for (int i = 0; i < 600; i++)
{
sw.Write("some long random string");
sw.Flush();
}
This not a good solution though, because the message I send has to be exact.
Then I have tried to close the socket after relevant data has been sent by soc.Disconnect(false); which causes the Java application's log to fill with proper debug information.
Is there any way to send EOL to the socket so the Java app would stop listening and would start sending the data?
Thank you.
EDIT 2
I have tried to create a Java client to connect to the server, the same thing has happened.
Then I have created a dummy server to listen on the same port as the Java app I'm connecting too, it has also behaved the way as the Java app should and it was working.
Now I feel that my only chance is to send EOT or EOF command to the stream, but I have no idea how to do it on .NET and I wasn't able to find the answer on the internet either.
If the Java application is from a third party, chances are that you're doing something wrong. The DataInputStream.readFully(byte[]) function block the application until it has read the number of bytes that the byte array can hold, so the snippet of code you have is from the read operation.
I also see that you use the Socket.Send(byte[]) function to communicate with the Java application, I recommend you to use something higher level like StreamWriter or BinaryWriter, more less like this:
StreamWriter = new BinaryWriter(client.GetStream());
StreamWriter.Write(msg);
StreamWriter.Flush();
Can you try to flush both the StreamWriter (so it writes its buffer to the stream object) and the actual NetworkStream you got from client.GetStream() (so it sends a packet, despite the packet not being "full" yet)?
NetworkStream ns = client.GetStream();
StreamWriter sw = new StreamWriter(ns);
StreamReader sr = new StreamReader(ns);
// ...
sw.Flush();
ns.Flush();
Most likely, you are not flushing all that you need to flush. Have you tried looking at the network communication with e.g. wireshark - is the data actually going out? If not, the problem is in your .NET code.
I am Sending Serialized large Image Object over UDP Socket.When I write all received bytes in Memory stream and pass the memory stream object for deserialization it throws an exception No assembly ID for object type 'ImagePacket'.
Receiver End Code:
ImageStream = new MemoryStream();
while (AccumulatingBytes <= TotalSizeOfComplexObject)
{
byte[] Recievedbytes = UdpListener.Receive(ref RemoteEndPoint);
ImageStream.Write(Recievedbytes, 0, Recievedbytes.Length);
AccumulatingBytes += Recievedbytes.Length;
}
ImageStream.Position = 0;
imagecontainer = (ImageContainer)bformater.Deserialize(ImageStream);//Here the Code Segment Breaks and Exception thrown
I suspect the problem here is simply: you are using UDP like it is TCP. UDP is packet based, but a: doesn't guarantee that the packets will arrive in order, and b: doesn't guarantee that packets won't be dropped or duplicated.
I fully expect you have some out of order. If you are sending multiple messages, it is also possible some were dropped, and you've included a few from the next message.
To use the network the way your code wants to use it: use TCP. Otherwise, the responsibility for making sense of out-of-order, dropped and duplicated packets is entirely yours. This could be, for example, by adding a sequence number to the packet, and keeping track of what has been received - re-ordering them as necessary, dropping duplicates, and re-requesting any that died en-route. Basically, re-writing everything that TCP adds! Unless you have a very specific scenario, there's a good chance that the TCP stack (with NIC and OS level support) will do a better job of this than you will.
I am trying to interface an ancient network camera to my computer and I am stuck at a very fundamental problem -- detecting the end of stream.
I am using TcpClient to communicate with the camera and I can actually see it transmitting the command data, no problems here.
List<int> incoming = new List<int>();
TcpClient clientSocket = new TcpClient();
clientSocket.Connect(txtHost.Text, Int32.Parse(txtPort.Text));
NetworkStream serverStream = clientSocket.GetStream();
serverStream.Flush();
byte[] command = System.Text.Encoding.ASCII.GetBytes("i640*480M");
serverStream.Write(command, 0, command.Length);
Reading back the response is where the problem begins though. I initially thought something simple like the following bit of code would have worked:
while (serverStream.DataAvailable)
{
incoming.Add(serverStream.ReadByte());
}
But it didn't, so I had a go another version this time utilising ReadByte(). The description states:
Reads a byte from the stream and
advances the position within the
stream by one byte, or returns -1 if
at the end of the stream.
so I thought I could implement something along the lines of:
Boolean run = true;
int rec;
while (run)
{
rec = serverStream.ReadByte();
if (rec == -1)
{
run = false;
//b = (byte)'X';
}
else
{
incoming.Add(rec);
}
}
Nope, still doesn't work. I can actually see data coming in and after a certain point (which is not always the same, otherwise I could have simply read that many bytes every time) I start getting 0 as the value for the rest of the elements and it doesn't halt until I manually stop the execution. Here's what it looks like:
So my question is, am I missing something fundamental here? How can I detect the end of the stream?
Many thanks,
H.
What you're missing is how you're thinking of a TCP data stream. It is an open connection, like an open phone line - someone on the other end may or may not be talking (DataAvailable), and just because they paused to take a breath (DataAvailable==false) it doesn't mean they're actually DONE with their current statement. A moment later they could start talking again (DataAvailable==true)
You need to have some kind of defined rules for the communication protocol ABOVE TCP, which is really just a transport layer. So for instance perhaps the camera will send you a special character sequence when it's current image transmission is complete, and so you need to examine every character sent and determine if that sequence has been sent to you, and then act appropriately.
Well you can't exactly says EOS on a network communication ( unless the other party drop the connection ) usually the protocol itself contains something to signal that the message is complete ( sometimes a new line, for example ). So you read the stream and feed a buffer, and you extract complete message by applying these strategies.
Can anyone point out the flaw in this code? I'm retrieving some HTML with TcpClient. NetworkStream.Read() never seems to finish when talking to an IIS server. If I go use the Fiddler proxy instead, it works fine, but when talking directly to the target server the .read() loop won't exit until the connection exceptions out with an error like "the remote server has closed the connection".
internal TcpClient Client { get; set; }
/// bunch of other code here...
try
{
NetworkStream ns = Client.GetStream();
StreamWriter sw = new StreamWriter(ns);
sw.Write(request);
sw.Flush();
byte[] buffer = new byte[1024];
int read=0;
try
{
while ((read = ns.Read(buffer, 0, buffer.Length)) > 0)
{
response.AppendFormat("{0}", Encoding.ASCII.GetString(buffer, 0, read));
}
}
catch //(SocketException se)
{
}
finally
{
Close();
}
Update
In the debugger, I can see the entire response coming through immediately and being appended to my StringBuilder (response). It just appears that the connection isn't being closed when the server is done sending the response, or my code isn't detecting it.
Conclusion
As has been said here, it's best to take advantage of the offerings of the protocol (in the case of HTTP, the Content-Length header) to determine when a transaction is complete. However, I've found that not all pages have content-length set. So, I'm now using a hybrid solution:
For ALL transactions, set the request's Connection header to "close", so that the server is discouraged from keeping the socket open. This improves the chances that the server will close the connection when it is through responding to your request.
If Content-Length is set, use it to determine when a request is complete.
Else, set the NetworkStream's RequestTimeout property to a large, but reasonable, value like 1 second. Then, loop on NetworkStream.Read() until either a) the timeout occurs, or b) you read fewer bytes than you asked for.
Thanks to everyone for their excellent and detailed responses.
Contrary to what the documentation for NetworkStream.Read implies, the stream obtained from a TcpClient does not simply return 0 for the number of bytes read when there is no data available - it blocks.
If you look at the documentation for TcpClient, you will see this line:
The TcpClient class provides simple methods for connecting, sending, and receiving stream data over a network in synchronous blocking mode.
Now my guess is that if your Read call is blocking, it's because the server has decided not to send any data back. This is probably because the initial request is not getting through properly.
My first suggestion would be to eliminate the StreamWriter as a possible cause (i.e. buffering/encoding nuances), and write directly to the stream using the NetworkStream.Write method. If that works, make sure that you're using the correct parameters for the StreamWriter.
My second suggestion would be not to depend on the result of a Read call to break the loop. The NetworkStream class has a DataAvailable property that is designed for this. The correct way to write a receive loop is:
NetworkStream netStream = client.GetStream();
int read = 0;
byte[] buffer = new byte[1024];
StringBuilder response = new StringBuilder();
do
{
read = netStream.Read(buffer, 0, buffer.Length);
response.Append(Encoding.ASCII.GetString(buffer, 0, read));
}
while (netStream.DataAvailable);
Read the response until you reach a double CRLF. What you now have is the Response headers.
Parse the headers to read the Content-Length header which will be the count of bytes left in the response.
Here is a regular expression that can catch the Content-Length header.
David's Updated Regex
Content-Length: (?<1>\d+)\r\n
Content-Length
Note
If the server does not properly set this header I would not use it.
Not sure if this is helpful or not but with HTTP 1.1 the underlying connection to the server might not be closed so maybe the stream doesn't get closed either? The idea being that you can reuse the connection to send a new request. I think you have to use the content-length. Alternatively use the WebClient or WebRequest classes instead.
I may be wrong, but it looks like your call to Write is writing (under the hood) to the stream ns (via StreamWriter). Later, you're reading from the same stream (ns). I don't quite understand why are you doing this?
Anyway, you may need to use Seek on the stream, to move to the location where you want to start reading. I'd guess that it seeks to the end after writing. But as I said, I'm not really sure if this is a useful answer!
Two Suggestions...
Have you tried using the DataAvailable property of NetworkStream? It should return true if there is data to be read from the stream.
while (ns.DataAvailable)
{
//Do stuff here
}
Another option would be to change the ReadTimeOut to a low value so you don't end up blocking for a long time. It can be done like this:
ns.ReadTimeOut=100;