Develop Reference

C# best way to implement TCP Client Server Application

I want to extend my experience with the .NET framework and want to build a client/server application.
Actually, the client/server is a small Point Of Sale system but first, I want to focus on the communication between server and client.
In the future, I want to make it a WPF application but for now, I simply started with a console application.
2 functionalities:
client(s) receive(s) a dataset and every 15/30min an update with changed prices/new products
(So the code will be in a Async method with a Thread.sleep for 15/30 mins).
when closing the client application, sending a kind of a report (for example, an xml)
On the internet, I found lots of examples but i can't decide which one is the best/safest/performanced manner of working so i need some advice for which techniques i should implement.
CLIENT/SERVER
I want 1 server application that handles max 6 clients. I read that threads use a lot of mb and maybe a better way will be tasks with async/await functionallity.
Example with ASYNC/AWAIT
http://bsmadhu.wordpress.com/2012/09/29/simplify-asynchronous-programming-with-c-5-asyncawait/
Example with THREADS
mikeadev.net/2012/07/multi-threaded-tcp-server-in-csharp/
Example with SOCKETS
codereview.stackexchange.com/questions/5306/tcp-socket-server
This seems to be a great example of sockets, however, the revisioned code isn't working completely because not all the classes are included
msdn.microsoft.com/en-us/library/fx6588te(v=vs.110).aspx
This example of MSDN has a lot more with Buffersize and a signal for the end of a message. I don't know if this just an "old way" to do this because in my previous examples, they just send a string from the client to the server and that's it.
.NET FRAMEWORK REMOTING/ WCF
I found also something about the remoting part of .NET and WCF but don' know if I need to implement this because i think the example with Async/Await isn't bad.
SERIALIZED OBJECTS / DATASET / XML
What is the best way to send data between it? Juse an XML serializer or just binary?
Example with Dataset -> XML
stackoverflow.com/questions/8384014/convert-dataset-to-xml
Example with Remoting
akadia.com/services/dotnet_dataset_remoting.html
If I should use the Async/Await method, is it right to something like this in the serverapplication:
while(true)
{
string input = Console.ReadLine();
if(input == "products")
SendProductToClients(port);
if(input == "rapport")
{
string Example = Console.ReadLine();
}
}

Here are several things anyone writing a client/server application should consider:
Application layer packets may span multiple TCP packets.
Multiple application layer packets may be contained within a single TCP packet.
Encryption.
Authentication.
Lost and unresponsive clients.
Data serialization format.
Thread based or asynchronous socket readers.
Retrieving packets properly requires a wrapper protocol around your data. The protocol can be very simple. For example, it may be as simple as an integer that specifies the payload length. The snippet I have provided below was taken directly from the open source client/server application framework project DotNetOpenServer available on GitHub. Note this code is used by both the client and the server:
private byte[] buffer = new byte[8192];
private int payloadLength;
private int payloadPosition;
private MemoryStream packet = new MemoryStream();
private PacketReadTypes readState;
private Stream stream;
private void ReadCallback(IAsyncResult ar)
{
try
{
int available = stream.EndRead(ar);
int position = 0;
while (available > 0)
{
int lengthToRead;
if (readState == PacketReadTypes.Header)
{
lengthToRead = (int)packet.Position + available >= SessionLayerProtocol.HEADER_LENGTH ?
SessionLayerProtocol.HEADER_LENGTH - (int)packet.Position :
available;
packet.Write(buffer, position, lengthToRead);
position += lengthToRead;
available -= lengthToRead;
if (packet.Position >= SessionLayerProtocol.HEADER_LENGTH)
readState = PacketReadTypes.HeaderComplete;
}
if (readState == PacketReadTypes.HeaderComplete)
{
packet.Seek(0, SeekOrigin.Begin);
BinaryReader br = new BinaryReader(packet, Encoding.UTF8);
ushort protocolId = br.ReadUInt16();
if (protocolId != SessionLayerProtocol.PROTOCAL_IDENTIFIER)
throw new Exception(ErrorTypes.INVALID_PROTOCOL);
payloadLength = br.ReadInt32();
readState = PacketReadTypes.Payload;
}
if (readState == PacketReadTypes.Payload)
{
lengthToRead = available >= payloadLength - payloadPosition ?
payloadLength - payloadPosition :
available;
packet.Write(buffer, position, lengthToRead);
position += lengthToRead;
available -= lengthToRead;
payloadPosition += lengthToRead;
if (packet.Position >= SessionLayerProtocol.HEADER_LENGTH + payloadLength)
{
if (Logger.LogPackets)
Log(Level.Debug, "RECV: " + ToHexString(packet.ToArray(), 0, (int)packet.Length));
MemoryStream handlerMS = new MemoryStream(packet.ToArray());
handlerMS.Seek(SessionLayerProtocol.HEADER_LENGTH, SeekOrigin.Begin);
BinaryReader br = new BinaryReader(handlerMS, Encoding.UTF8);
if (!ThreadPool.QueueUserWorkItem(OnPacketReceivedThreadPoolCallback, br))
throw new Exception(ErrorTypes.NO_MORE_THREADS_AVAILABLE);
Reset();
}
}
}
stream.BeginRead(buffer, 0, buffer.Length, new AsyncCallback(ReadCallback), null);
}
catch (ObjectDisposedException)
{
Close();
}
catch (Exception ex)
{
ConnectionLost(ex);
}
}
private void Reset()
{
readState = PacketReadTypes.Header;
packet = new MemoryStream();
payloadLength = 0;
payloadPosition = 0;
}
If you're transmitting point of sale information, it should be encrypted. I suggest TLS which is easily enabled on through .Net. The code is very simple and there are quite a few samples out there so for brevity I'm not going to show it here. If you are interested, you can find an example implementation in DotNetOpenServer.
All connections should be authenticated. There are many ways to accomplish this. I've use Windows Authentication (NTLM) as well as Basic. Although NTLM is powerful as well as automatic it is limited to specific platforms. Basic authentication simply passes a username and password after the socket has been encrypted. Basic authentication can still, however; authenticate the username/password combination against the local server or domain controller essentially impersonating NTLM. The latter method enables developers to easily create non-Windows client applications that run on iOS, Mac, Unix/Linux flavors as well as Java platforms (although some Java implementations support NTLM). Your server implementation should never allow application data to be transferred until after the session has been authenticated.
There are only a few things we can count on: taxes, networks failing and client applications hanging. It's just the nature of things. Your server should implement a method to clean up both lost and hung client sessions. I've accomplished this in many client/server frameworks through a keep-alive (AKA heartbeat) protocol. On the server side I implement a timer that is reset every time a client sends a packet, any packet. If the server doesn't receive a packet within the timeout, the session is closed. The keep-alive protocol is used to send packets when other application layer protocols are idle. Since your application only sends XML once every 15 minutes sending a keep-alive packet once a minute would able the server side to issue an alert to the administrator when a connection is lost prior to the 15 minute interval possibly enabling the IT department to resolve a network issue in a more timely fashion.
Next, data format. In your case XML is great. XML enables you to change up the payload however you want whenever you want. If you really need speed, then binary will always trump the bloated nature of string represented data.
Finally, as #NSFW already stated, threads or asynchronous doesn't really matter in your case. I've written servers that scale to 10000 connections based on threads as well as asynchronous callbacks. It's all really the same thing when it comes down to it. As #NSFW said, most of us are using asynchronous callbacks now and the latest server implementation I've written follows that model as well.

Threads are not terribly expensive, considering the amount of RAM available on modern systems, so I don't think it's helpful to optimize for a low thread count. Especially if we're talking about a difference between 1 thread and 2-5 threads. (With hundreds or thousands of threads, the cost of a thread starts to matter.)
But you do want to optimize for minimal blocking of whatever threads you do have. So for example instead of using Thread.Sleep to do work on 15 minute intervals, just set a timer, let the thread return, and trust the system to invoke your code 15 minutes later. And instead of blocking operations for reading or writing information over the network, use non-blocking operations.
The async/await pattern is the new hotness for asynchronous programming on .Net, and it is a big improvement over the Begin/End pattern that dates back to .Net 1.0. Code written with async/await is still using threads, it is just using features of C# and .Net to hide a lot of the complexity of threads from you - and for the most part, it hides the stuff that should be hidden, so that you can focus your attention on your application's features rather than the details of multi-threaded programming.
So my advice is to use the async/await approach for all of your IO (network and disk) and use timers for periodic chores like sending those updates you mentioned.
And about serialization...
One of the biggest advantages of XML over binary formats is that you can save your XML transmissions to disk and open them up using readily-available tools to confirm that the payload really contains the data that you thought would be in there. So I tend to avoid binary formats unless bandwidth is scarce - and even then, it's useful to develop most of the app using a text-friendly format like XML, and then switch to binary after the basic mechanism of sending and receiving data have been fleshed out.
So my vote is for XML.
And regarding your code example, well ther's no async/await in it...
But first, note that a typical simple TCP server will have a small loop that listens for incoming connections and starts a thread to hanadle each new connection. The code for the connection thread will then listen for incoming data, process it, and send an appropriate response. So the listen-for-new-connections code and the handle-a-single-connection code are completely separate.
So anyway, the connection thread code might look similar to what you wrote, but instead of just calling ReadLine you'd do something like "string line = await ReadLine();" The await keyword is approximately where your code lets one thread exit (after invoking ReadLine) and then resumes on another thread (when the result of ReadLine is available). Except that awaitable methods should have a name that ends with Async, for example ReadLineAsync. Reading a line of text from the network is not a bad idea, but you'll have to write ReadLineAsync yourself, building upon the existing network API.
I hope this helps.

Streaming data between C# applications

I'm trying to share data between two applications: the first gets the data continuously from a sensor, all I need to do is to transmit these data while I'm receiving them to another (WPF) application (and draw a graph).
The data is received through an EventHandler, and then it's transmitted through a socket, like this:
static TcpClient client = new TcpClient("localhost", 8181);
static double w, dba;
static void Write()
{
try
{
Stream s = client.GetStream();
StreamReader sr = new StreamReader(s);
StreamWriter sw = new StreamWriter(s);
sw.AutoFlush = true;
while (true)
{
String line = "W:" + w + "DB/A:" + dba;
sw.Write(line);
Console.WriteLine(sr.ReadLine());
}
s.Close();
}
finally
{
client.Close();
}
}
The thing is, do I need to put it in a separate Thread? (As I tried to, unsuccessfully) Because like this, while the eventHandler keeps being triggered and producing data (storing it into the two variables), the sw seems unable to proceed.
Sorry if the question is a bit vague, it's my first attempt with a distributed app so I'm a bit confused as well.. any advice or help would be very appreciated!
TIA

Use memory mapped files, which allows you to directly share memory between applications. It will be significantly faster than most other approaches.
Look at the section called Non-Persisted Memory-Mapped Files, which details how to share a segment of memory via a string name. I use this in an application I worked on for debugging purposes, and it's fast enough that my C# application could read my native application's memory in real-time.
If you're transmitting the data via this method, consider how you would transmit data in general. Determine a polling frequency (say every 100ms) and have application A write the data to the memory-mapped file. Then have application B read that memory mapped file and store it locally in a collection. This is your transmission.
So A basically writes and rewrites the same structure into the memory mapped file and B reads it at a given polling rate and stores what it polls in a collection.

If both apps are .net, why not use WCF for inter process communication : much more robust than using files. See this link WCF Basic Interprocess Communication

UDP sending data, something i don´t quite understand

Okay from my knowledge UDP works like this:
You have data you want to send, you say to the UDP client, hey send this data.
The UDP client then says, sure why not, and sends the data to the selected IP and Port.
If it get´s through or in the right order is another story, it have sent the data, you didn´t ask for anything else.
Now from this perspective, it´s pretty much impossible to send data and assemble it.
for example, i have a 1mb image, and i send it.
So i send divide it in 60kb files (or something to fit the packages), and send them one by one from first to last.
So in theory, if all get´s added, the image should be exactly the same.
But, that theory breaks as there is no law that tells the packages if it can arrive faster or slower than another, so it may only be possible if you make some kind of wait timer, and hope for the best that the arrive in the order they are sent.
Anyway, what i want to understand is, why does this work:
void Sending(object sender, NAudio.Wave.WaveInEventArgs e)
{
if (connect == true && MuteMic.Checked == false)
{
udpSend.Send(e.Buffer, e.BytesRecorded, otherPartyIP.Address.ToString(), 1500);
}
}
Recieving:
while (connect == true)
{
byte[] byteData = udpReceive.Receive(ref remoteEP);
waveProvider.AddSamples(byteData, 0, byteData.Length);
}
So this is basically, it sends the audio buffer through udp.
The receiving par just adds the udp data received in a buffer and plays it.
Now, this works.
And i wonder.. why?
How can this work, how come the data is sent in the right order and added so it appears as a constant audio stream?
Cause if i would to this with an image, i would probably get all the data.
But they would be in a random order probably, and i can only solve that by marking packages and stuff like that. And then there is simply no reason for it, and TCP takes over.
So if someone can please explain this, i just don´t get it.
Here is a code example that is when sending an image, and well, it works. But it seems to work better when the entire byte array isn´t sent, meanign some part of the image is corrupted (not sure why, probably something to do with how the size of the byte array are).
Send:
using (var udpcap = new UdpClient(0))
{
udpcap.Client.SendBufferSize *= 16;
bsize = ms.Length;
var buff = new byte[7000];
int c = 0;
int size = 7000;
for (int i = 0; i < ms.Length; i += size)
{
c = Math.Min(size, (int)ms.Length - i);
Array.Copy(ms.GetBuffer(), i, buff, 0, c);
udpcap.Send(buff, c, adress.Address.ToString(), 1700);
}
Receive:
using (var udpcap = new UdpClient(1700))
{
udpcap.Client.SendBufferSize *= 16;
var databyte = new byte[1619200];
int i = 0;
for (int q = 0; q < 11; ++q)
{
byte[] data = udpcap.Receive(ref adress);
Array.Copy(data, 0, databyte, i, data.Length);
i += data.Length;
}
var newImage = Image.FromStream(new MemoryStream(databyte));
gmp.DrawImage(newImage,0,0);
}

You should be using TCP. You write: it´s pretty much impossible to send data and assemble it. for example, i have a 1mb image, and i send it. So i send divide it in 60kb files (or something to fit the packages), and send them one by one from first to last. ... But, that theory breaks as there is no law that tells the packages if it can arrive faster or slower than another, so it may only be possible if you make some kind of wait timer, and hope for the best that the arrive in the order they are sent. That's exactly what TCP does: ensure that all the pieces of a stream of data are received in the order they were sent, with no omissions, duplications, or modifications. If you really want to re-implement that yourself, you should be reading RFC 793 - it talks at length about how to build a reliable data stream atop an unreliable packet service.
But really, just use TCP.

You're missing a lot of helpful details from your question, but based on the level of understanding presented I'll attempt to answer at a similar level:
You're absolutely right, in general the UDP protocol doesn't guarantee order of delivery or even delivery at all. Your local host is going to send the packets (i.e. parts of your request message) in the order it receives them from the sending application, and from there its up to network components to decide how your request gets delivered. In local networks however (within a handful of hops of the original requester) there aren't really a lot of directions for the packets to go. As such they will likely just flow in line and never see a hiccup.
On the greater internet however, there is likely a wide variety of routing choices available to each router between your requesting host and the destination. Every router along the way can make a choice on which direction parts of your message take. Assuming all paths are equal (which they aren't) and guaranteed reliability of every network segment between the 2 hosts its likely to see similar results as within network (with added latency). Unfortunately neither of the posed conditions can be considered reliable (different paths on the internet perform differently depending on client and server, and no single path on the internet should ever be considered to be reliable (that's why it's a "net").
These are of course based on general observations from my own experience in network support and admin roles. Members of other StackExchange sites may be able to provide better feedback.

problem with C# when reading from serial

i have a problem with a serial port reader in C#.
if i send 5555 through the serial port the program prints out 555.
here is the program
public static void Main()
{
byte[] buffer = new byte[256];
string buff;
using (SerialPort sp = new SerialPort("COM2", 6200))
{
sp.Open();
//read directly
sp.Read(buffer, 0, (int)buffer.Length);
//read using a Stream
sp.BaseStream.Read(buffer, 0, (int)buffer.Length);
string sir = System.Text.Encoding.Default.GetString(buffer);
Console.WriteLine(sir);

Both your computer's and the other device's UART may have a hardware buffer which passes data in respect to the actual hardware control enabled for the connection. Hence you have to take care of:
hardware control flow setup;
timing of data reading / writing;
Bear in mind you are working with a real-time hardware device that has its own timing which needs to be respected by your application. Communicating with a hardware device is a process. In other words, a one-shot read may not be enough to retrieve all input you are expecting on the logical level.
Update: Google for “SerialPort tutorial C#” and study few of them, like this one.

You need to use the int returned from the "Read" methods. The value returned will tell you how many bytes were actually read. You will probably need to loop and call "Read" multiple times until you have read the number of bytes you need.
Update: This other question has some sample code that shows how to read multiple times until you have enough data to process.

SerialPort data loss - C#

I'm working on a SerialPort app and one very simple part of it is giving me issues. I simply want to read a constant stream of data from the port and write it out to a binary file as it comes in. The problem seems to be speed: my code has worked fine on my 9600 baud test device, but when carried over to the 115200bps live device, I seem to be losing data. What happens is after a variable period of time, I miss 1 byte which throws off the rest of the data. I've tried a few things:
private void serialPort1_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
bwLogger.Write((byte)serialPort1.ReadByte());
}
or
private void serialPort1_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
byte[] inc = new byte[serialPort1.BytesToRead];
serialPort1.Read(inc, 0, inc.Length);
bwLogger.Write(inc);
}
and a few variations. I can't use ReadLine() as I am working with a constant stream of data (right?). I've tried fiddling with the buffer size (both serialPort1.ReadBufferSize and the hardware FIFO buffer). Ideally, for usability purposes, I'd handle this on the software side and not make the user have to change Windows driver settings.
Any ideas?

If the problem seems to be that you can't process the data fast enough, what you could try would be to double-buffer your data.
1) Allow one thread to read the serial port into one buffer. This may involve copying data off the port into the buffer (i'm not intimately familiar with .NET).
2) When you are ready to handle the incoming data, (on a different thread) make your program read into the 2nd buffer, and while this is happening you should write the first buffer to disk.
3) When the first buffer is written to disk, swap it back to the serial port buffer, and write the 2nd buffer to disk. Repeat process, continually swapping the buffers.

You might try enabling handshaking, using the Handshake property of the SerialPort object.
You'll have to set it on both the sender or receiver. however: if you're overflowing the receiver's UART's buffer (very small, 16 bytes IIRC), there's probably no other way. If you can't enable handshaking on the sender, you'll probably have to stay at 9600 or below.

I'd try the following:
Set the Buffer-Size to at least 230K Bytes
Set the Incoming Threshold to 16K, 32K or 65K
Write this fixed blocks of data to the file
I'm not sure if this might help, but it should at least take the pressure of the framework to fire the event that often.

I would check the number of bytes read which is returned by the Read(Byte>[], Int32, Int32) method and make sure it matches what you expect.
Make sure you are listening for SerialErrorReceivedEventHandler ErrorReceived events on the port object. An RXOver error would indicate your buffer is full.
Check the thread safety on your output buffer. If the write to the output buffer is not thread safe, a second write may corrupt the first write.

Is your bwLogger a BinaryWriter class? You might try using it with a BufferedStream to make the disk I/O nonblocking.
Also, if your packets have a known ending character, you can set the SerialPort.NewLine property to enable you to use ReadLine/WriteLine, although I don't think that would make much of a performance difference.

The machines I've been working with recently all send a stop code (in my case ASCII code 3 or 4). If you also have this feature, you can make use of ReadTo(string) off your SerialPort object.