This is take II, i posted the other week and my question was put on hold, i adjusted my text but could not get a review, and the system closed the original post.
Logic
Server Side: Read only - Server Opens pipe then at regular interval checks if there is content (i.e. not at end of stream) and reads info. This check has to be poll based as only during the poll is there a valid context to pass on the data..
Client Side: Write-only - Open pipe, write to pipe, close (client.exe called many times, has short life span, code below is test code), e.g. some other script will "call client.exe with info"
Can this work flow be handled in pipes ? e.g. snippet of client code shown only the first client message is seen by the "server"
If pipes can do this after coding tips as most examples are for client - servers having similar life cycles.
Code snippets
for (int i = 0; i < 10; i++)
{
//Client - simulate exe starting and ending
var client = new NamedPipeClientStream(".", "PipesOfPiece", PipeDirection.Out, PipeOptions.WriteThrough);
client.Connect();
StreamWriter writer = new StreamWriter(client);
Console.WriteLine("Client about to send message");
writer.WriteLine("Called from client i = {0}", i);
writer.Close();
client.Close();
Thread.Sleep(5000);
}
// server snippet
var server = new NamedPipeServerStream("PipesOfPiece", PipeDirection.In);
server.WaitForConnection(); <= can this we optional with code below
StreamReader reader = new StreamReader(server);
while (true)
{
// simulate start of poll code
if (server.IsConnected)
{
if (!reader.EndOfStream)
{
var line = reader.ReadToEnd();
Console.WriteLine("Server: {0}", line);
}
} // End of poll code
Thread.Sleep(1000);
}
// server snippet
var server = new NamedPipeServerStream("PipesOfPiece", PipeDirection.In);
server.WaitForConnection(); <= can this we optional with code below
StreamReader reader = new StreamReader(server);
while (true)
{
// simulate start of poll code
if (server.IsConnected)
{
if (!reader.EndOfStream)
{
var line = reader.ReadToEnd();
Console.WriteLine("Server: {0}", line);
}
} // End of poll code
Thread.Sleep(1000);
}
So I am rusty on my pipes, I am hoping that a pipe can be opened, written to then read, and the waitforconnect() is there for the cases where you want this and is optional. I suppose it all triggers around who owns the pipe, i.e. if the server opens a pipe and is waiting for someone to write for it, why does it need to wait for a connect ? (I am hoping the server is the owner so when it ends, the pipe disappears)
Without a good, minimal, complete code example that reliably reproduces whatever specific problem you are having, it is impossible to provide specific advice as to how to fix that problem. However, I can at least try to answer some of your questions about how named pipes can be used, and provide a code example to illustrate some of the concepts.
First, some rules and observations:
A pipe instance can be used for only one connection. Note that pipes inherit Stream, and streams have a very specific paradigm: you open one, read to the end, and then you're done with the stream. Some streams, like FileStream, are seekable but even there you are only ever dealing with a single resource (i.e. the original file…you can't reconnect a FileStream to a different file), and network streams aren't even seekable.
A pipe must be connected before you perform I/O on it.
You may have multiple instances of pipes with the same name (if you initialize them correctly…by default, you may only have one pipe of any given name).
Clients trying to connect to a named pipe will wait until such a pipe exists. It does not need to exist at the time the client initiates the connection.
Only one client can connect to any given instance of a pipe. Any given instance of a server pipe can only ever handle a single client during its entire lifetime (see the very first point above).
So, what about your questions?
Can this work flow be handled in pipes ?
If I understand the work-flow correctly, yes. But you need to be careful to implement it correctly.
As I understand it, you want for your server to only attempt to read from clients periodically. At the same time, you want for a client to be able to write to a pipe at any time. This can be done, but it won't be straightforward.
Note per the above that you cannot open a single server pipe, and then let multiple clients periodically connect and disconnect from that pipe. Once the first client has connected, the pipe is no longer usable. It's a stream, and that first client's disconnection causes the stream to reach its end. It's done.
Note also that while a client can attempt to connect to a pipe that doesn't exist yet, it will wait until it can. So if you want your clients to not have to wait until the polling interval has expired, you'll need to maintain a server pipe available to connect to at all times.
But you've already said that you won't be able to handle data read from the server pipe at arbitrary points in time, but rather only during your polling interval.
Because pipes don't inherently support this specific scenario, IMHO the right way to implement it is to separate the behaviors into two different components. Maintain a simple pipe server that opens a pipe, waits for a client to connect, reads whatever the client has sent, closes the pipe, and then starts over.
Then have an intermediary class that can act as the go-between for the server I/O and whatever component you have that ultimately receives the data. This intermediary will retain a copy of the data after it's been received (the pipe code will deliver it to the intermediary as soon as it's received, regardless of the polling interval); the data will then later be retrieved by the polling component on its next polling interval (i.e. when the "context" as you put it is in fact available to which to deliver the data).
I am hoping that a pipe can be opened, written to then read, and the waitforconnect() is there for the cases where you want this and is optional
Unfortunately, your hope doesn't match the reality. Pipes can be bidirectional; i.e. "written to then read". But WaitForConnect() is not optional. The server must wait for a connection before attempting to read from the pipe, and for that pipe instance it will only ever be able to receive data from a single client.
I am hoping the server is the owner so when it ends, the pipe disappears
The server process is the one that actually creates the pipe. So yes, in that sense it is the owner. And yes, when the server process is terminated, any pipes it's created are destroyed.
Below, please find a simple code example that illustrates the use of multiple and concurrent servers and clients. You can adjust the numbers of each with the declared constants at the top of the example.
When running it, note that if more clients are active than servers, the additional clients will simply wait until a server pipe is available to connect to. Once one is, they will connect and proceed normally. If there are at least as many server pipe instances as there are clients trying to connect, all of the clients are serviced concurrently.
// NOTE: as a sample program, contrary to normal and correct
// programming practices error-handling has been omitted, and
// non-awaited async methods have been declared as void.
class Program
{
private const string _kserverName = "TestSO33093954NamedPipeClients";
private const int _kmaxServerCount = 3;
private const int _kmaxClientCount = 3;
static void Main(string[] args)
{
StartServers(_kmaxServerCount);
StartClients(_kmaxClientCount);
Console.WriteLine("Clients are being started. Press return to exit program.");
Console.ReadLine();
}
private static async void StartClients(int clientCount)
{
for (int i = 0; i < clientCount; i++)
{
RunClient(i);
await Task.Delay(300);
}
}
private static async void RunClient(int instance)
{
NamedPipeClientStream client = new NamedPipeClientStream(
".", _kserverName, PipeDirection.InOut, PipeOptions.Asynchronous);
client.Connect();
ReadClient(client);
using (StreamWriter writer = new StreamWriter(client))
{
writer.AutoFlush = true;
for (int i = 0; i < 5; i++)
{
string text =
string.Format("Instance #{0}, iteration #{1}", instance, i);
Console.WriteLine("Client send: " + text);
await writer.WriteLineAsync(text);
await Task.Delay(1000);
}
client.WaitForPipeDrain();
}
}
private static async void ReadClient(Stream stream)
{
using (TextReader reader = new StreamReader(stream))
{
string line;
while ((line = await reader.ReadLineAsync()) != null)
{
Console.WriteLine("Client recv: " + line);
}
}
}
private static void StartServers(int maxServerInstances)
{
for (int i = 0; i < maxServerInstances; i++)
{
RunServer(maxServerInstances);
}
}
private static async void RunServer(int maxServerInstances)
{
while (true)
{
using (NamedPipeServerStream server = new NamedPipeServerStream(
_kserverName, PipeDirection.InOut, maxServerInstances,
PipeTransmissionMode.Byte, PipeOptions.Asynchronous))
{
await server.WaitForConnectionAsync();
byte[] buffer = new byte[1024];
int bytesRead;
Decoder decoder = Encoding.UTF8.GetDecoder();
while ((bytesRead =
await server.ReadAsync(buffer, 0, buffer.Length)) > 0)
{
int cch = decoder.GetCharCount(buffer, 0, bytesRead);
char[] rgch = new char[cch];
decoder.GetChars(buffer, 0, bytesRead, rgch, 0);
Console.Write("Server recv: " + new string(rgch));
await server.WriteAsync(buffer, 0, bytesRead);
}
}
}
}
}
static class PipeExtensions
{
// As I am not running with .NET 4.6 yet, I need this little helper extension
// to wrap the APM-based asynchronous connection-waiting with the await-friendly
// Task-based syntax. Anyone using .NET 4.6 will have this in the framework already
public static Task WaitForConnectionAsync(this NamedPipeServerStream server)
{
return Task.Factory.FromAsync(
server.BeginWaitForConnection, server.EndWaitForConnection, null);
}
}
Related
I'm writing an application for windows phone and I need to communicate with a server and transmit data. The SERVER is written in C++ and I cannot modify it. The CLIENT is what I have to write. The Server is designed such that the client connect to it and transmit data. The connection remains open for all the transmission. By writing my code in C# I am able to receive data from the server but after the first receive, the data that I read in the buffer are alway the same. So I need a way to flush the input buffer so I can receive the new data (data are sent continuously). I'm using the class defined in here:
http://msdn.microsoft.com/en-us/library/windowsphone/develop/hh202858%28v=vs.105%29.aspx
thanks a lot !!
I used this code for Receiving in the SocketClient.cs :
public string Receive()
{
string response = "Operation Timeout";
// We are receiving over an established socket connection
if (_socket != null)
{
// Create SocketAsyncEventArgs context object
SocketAsyncEventArgs socketEventArg = new SocketAsyncEventArgs();
socketEventArg.RemoteEndPoint = _socket.RemoteEndPoint;
// Setup the buffer to receive the data
socketEventArg.SetBuffer(new Byte[MAX_BUFFER_SIZE], 0, MAX_BUFFER_SIZE);
// Inline event handler for the Completed event.
// Note: This even handler was implemented inline in order to make
// this method self-contained.
socketEventArg.Completed += new EventHandler<SocketAsyncEventArgs>(delegate(object s, SocketAsyncEventArgs e)
{
if (e.SocketError == SocketError.Success)
{
// *********************************************
// THIS part of the code was added to receive
// a vector of 3 double
Double[] OdomD = new Double[3];
for (int i = 0; i < 3; i++)
{
OdomD[i] = BitConverter.ToDouble(e.Buffer, 8 * i);
}
// *********************************************
}
else
{
response = e.SocketError.ToString();
}
_clientDone.Set();
});
// Sets the state of the event to nonsignaled, causing threads to block
_clientDone.Reset();
// Make an asynchronous Receive request over the socket
_socket.ReceiveAsync(socketEventArg);
// Block the UI thread for a maximum of TIMEOUT_MILLISECONDS milliseconds.
// If no response comes back within this time then proceed
_clientDone.WaitOne(TIMEOUT_MILLISECONDS);
}
else
{
response = "Socket is not initialized";
}
return response;
}
The Connect() method is exactly the same reported in the link above. So when the application start, the Connect() method is called as follow:
SocketClient client = new SocketClient();
// Attempt to connect to server for receiving data
Log(String.Format("Connecting to server '{0}' over port {1} (data) ...", txtRemoteHost.Text, 4444), true);
result = client.Connect(txtRemoteHost.Text, 4444);
Log(result, false);
That is done just once at the beginning, then I need receive this array of 3 double that is updated every second. So I use:
Log("Requesting Receive ...", true);
result = client.Receive();
Log(result, false);
The problem is that also if I debug the code and stop the execution inside Receive(), I always read the same value, that is the first value sent by the server. What I'm expecting is that every time I call client.Receive(), I get the new value, but this is not appening.
I had a similar problem by writing the same client in Matlab environment. I solved the problem by using the function flushinput(t) before to read the input buffer. In this way I was able to read always the last data sent by the server. I'm lookin for a function similar to that one ..
The size of the input buffer is fixed equal to the data that I'm expecting to receive, in that case is 24 bytes ( 3* sizeof(double) ) ..
Thanks a lot for you time !!
oleksii is right, you should call client.Receive() in a loop. You can choose to start a thread that covers the receive section of your code. Also note that client.Receive() will keep trying to receive from the buffer, and it will get stuck if there is no data available.
The main question was **how to clear the input buffer? ** or am I wrong?=!
Nevertheless; since you don't have a fixed buffer denoted as seen from you posted code and receive it via the SocketAsyncEventArgs, you could clear it with:
Array.Clear(e.Buffer, 0, e.Buffer.Length);
I need to connect a server (with ip and port) and create a read-loop that will get messages from the server as XML. sometimes there are no messages from the server.
I tried to create a connection (works fine) and read messages, I get the first message from the server and when I'm trying to read another one - it get stuck. I think that maybe there are no messages right now but I need that the loop will continue until there will be messages... it doesn't even go to "catch" or "finally", just do nothing..
public class Connection
{
public Connection()
{
Socket server = null;
try
{
string p = string.Empty;
using (var client = new TcpClient(myIPAddress, myPort))
using (var stream = client.GetStream())
using (var reader = new StreamReader(stream))
{
while (p != null)
{
try
{
p = reader.ReadLine();
}
catch (Exception e)
{
//
}
}
}
}
catch (Exception e)
{
//
}
finally {
server.Close();
}
}
}
The loop is continuing, waiting for data. The issue here seems to be simply that ReadLine() is a blocking call. You mention that there might not be a message yet; well, ReadLine() is going to block until one of two conditions is met:
it can successfully read some data, terminated by a newline (or EOF, i.e. a message without a newline followed by socket closure) - in which case it returns the line of data
no more data is received and the stream is closed, in which case it returns null
So basically, ReadLine() is going to wait until either a message comes in, or the socket is closed. That is simply the behaviour of ReadLine(). If that is problematic, you could work closer to the socket, and check NetworkStream.DataAvailable but: note that only tells you if some data is currently available; it doesn't mean "this is an entire message", nor can it be used to tell if more messages will arrive. The main use of DataAvailable is to decide between sync and async access. Plus if you work close to the socket you'll have to do all your own buffering and encoding/decoding.
It looks to me like ReadLine() is working successfully. The only thing I would do here is re-phrase it a bit:
string line;
while((line = reader.ReadLine()) != null) {
// line is meaningful; do something
}
One last thought: xml is not always trivially split into messages simply on a "per-line" basis. You might want to consider some other form of framing, but that may well mean working closer to the socket, rather than a StreamReader.
You have to wait till data arrives at the stream, you could try using follwing,
if(reader.EndOfStream)
continue;
I have a client/server infrastructure. At present they use a TcpClient and TcpListener to send a receive data between all the clients and server.
What I currently do is when data is received (on it's own thread), it is put in a queue for another thread to process in order to free the socket so it is ready and open to receive new data.
// Enter the listening loop.
while (true)
{
Debug.WriteLine("Waiting for a connection... ");
// Perform a blocking call to accept requests.
using (client = server.AcceptTcpClient())
{
data = new List<byte>();
// Get a stream object for reading and writing
using (NetworkStream stream = client.GetStream())
{
// Loop to receive all the data sent by the client.
int length;
while ((length = stream.Read(bytes, 0, bytes.Length)) != 0)
{
var copy = new byte[length];
Array.Copy(bytes, 0, copy, 0, length);
data.AddRange(copy);
}
}
}
receivedQueue.Add(data);
}
However I wanted to find out if there is a better way to do this. For example if there are 10 clients and they all want to send data to the server at the same time, one will get through while all the others will fail.Or if one client has a slow connection and hogs the socket all other communication will halt.
Is there not some way to be able to receive data from all clients at the same time and add the received data in the queue for processing when it has finished downloading?
So here is an answer that will get you started - which is more beginner level than my blog post.
.Net has an async pattern that revolves around a Begin* and End* call. For instance - BeginReceive and EndReceive. They nearly always have their non-async counterpart (in this case Receive); and achieve the exact same goal.
The most important thing to remember is that the socket ones do more than just make the call async - they expose something called IOCP (IO Completion Ports, Linux/Mono has these two but I forget the name) which is extremely important to use on a server; the crux of what IOCP does is that your application doesn't consume a thread while it waits for data.
How to Use The Begin/End Pattern
Every Begin* method will have exactly 2 more arguments in comparisson to it's non-async counterpart. The first is an AsyncCallback, the second is an object. What these two mean is, "here is a method to call when you are done" and "here is some data I need inside that method." The method that gets called always has the same signature, inside this method you call the End* counterpart to get what would have been the result if you had done it synchronously. So for example:
private void BeginReceiveBuffer()
{
_socket.BeginReceive(buffer, 0, buffer.Length, BufferEndReceive, buffer);
}
private void EndReceiveBuffer(IAsyncResult state)
{
var buffer = (byte[])state.AsyncState; // This is the last parameter.
var length = _socket.EndReceive(state); // This is the return value of the method call.
DataReceived(buffer, 0, length); // Do something with the data.
}
What happens here is .Net starts waiting for data from the socket, as soon as it gets data it calls EndReceiveBuffer and passes through the 'custom data' (in this case buffer) to it via state.AsyncResult. When you call EndReceive it will give you back the length of the data that was received (or throw an exception if something failed).
Better Pattern for Sockets
This form will give you central error handling - it can be used anywhere where the async pattern wraps a stream-like 'thing' (e.g. TCP arrives in the order it was sent, so it could be seen as a Stream object).
private Socket _socket;
private ArraySegment<byte> _buffer;
public void StartReceive()
{
ReceiveAsyncLoop(null);
}
// Note that this method is not guaranteed (in fact
// unlikely) to remain on a single thread across
// async invocations.
private void ReceiveAsyncLoop(IAsyncResult result)
{
try
{
// This only gets called once - via StartReceive()
if (result != null)
{
int numberOfBytesRead = _socket.EndReceive(result);
if(numberOfBytesRead == 0)
{
OnDisconnected(null); // 'null' being the exception. The client disconnected normally in this case.
return;
}
var newSegment = new ArraySegment<byte>(_buffer.Array, _buffer.Offset, numberOfBytesRead);
// This method needs its own error handling. Don't let it throw exceptions unless you
// want to disconnect the client.
OnDataReceived(newSegment);
}
// Because of this method call, it's as though we are creating a 'while' loop.
// However this is called an async loop, but you can see it the same way.
_socket.BeginReceive(_buffer.Array, _buffer.Offset, _buffer.Count, SocketFlags.None, ReceiveAsyncLoop, null);
}
catch (Exception ex)
{
// Socket error handling here.
}
}
Accepting Multiple Connections
What you generally do is write a class that contains your socket etc. (as well as your async loop) and create one for each client. So for instance:
public class InboundConnection
{
private Socket _socket;
private ArraySegment<byte> _buffer;
public InboundConnection(Socket clientSocket)
{
_socket = clientSocket;
_buffer = new ArraySegment<byte>(new byte[4096], 0, 4096);
StartReceive(); // Start the read async loop.
}
private void StartReceive() ...
private void ReceiveAsyncLoop() ...
private void OnDataReceived() ...
}
Each client connection should be tracked by your server class (so that you can disconnect them cleanly when the server shuts down, as well as search/look them up).
You should use asynchronous socket programming to achieve this. Take a look at the example provided by MSDN.
You should use asynchronous method of reading the data, an example is:
// Enter the listening loop.
while (true)
{
Debug.WriteLine("Waiting for a connection... ");
client = server.AcceptTcpClient();
ThreadPool.QueueUserWorkItem(new WaitCallback(HandleTcp), client);
}
private void HandleTcp(object tcpClientObject)
{
TcpClient client = (TcpClient)tcpClientObject;
// Perform a blocking call to accept requests.
data = new List<byte>();
// Get a stream object for reading and writing
using (NetworkStream stream = client.GetStream())
{
// Loop to receive all the data sent by the client.
int length;
while ((length = stream.Read(bytes, 0, bytes.Length)) != 0)
{
var copy = new byte[length];
Array.Copy(bytes, 0, copy, 0, length);
data.AddRange(copy);
}
}
receivedQueue.Add(data);
}
Also you should consider using AutoResetEvent or ManualResetEvent to be notified when new data is added to the collection so the thread that handle the data will know when data is received, and if you are using 4.0 you better switch off to using BlockingCollection instead of Queue.
What I do usually is using a thread pool with several threads.
Upon each new connection I'm running the connection handling (in your case - everything you do in the using clause) in one of the threads from the pool.
By that you achieve both performance since you're allowing several simultaneously accepted connection and you also limiting the number of resources (threads, etc') you allocate for handling incoming connections.
You have a nice example here
Good Luck
I'm developing an application that is running a lot of TcpListener tasks
using c# with .net 3.5 on windows server 2008
the TcpListener task is:
listening
to get an xml message
read an ID
from it
send a command to a
specific camera based on the ID to
take a snapshot and store it in a
folder
This is meant to execute within 1 second so I can take a snapshot with the camera. I've got a delay in executing this task, using the following code:
private Thread listen_thread;
public void start_listen()
{
this.listen_thread = new Thread(new ThreadStart(save_data));
this.listen_thread.Priority = ThreadPriority.Normal;
this.listen_thread.Start();
}
private void save_data()
{
//work to be done
}
Is this the best multi-threading technique to use? This application is running on a on dell poweredge 2900
with 2 quad core prosessor, and I think it could go faster. How might I be able to improve the latency of this code?
thats the code for the tcplistener
Int32 port = controller_port;
try
{
//this server ip
IPAddress localAddr = IPAddress.Parse(this_ip);
server = new TcpListener(localAddr, port);
server.Start();
Byte[] bytes = new Byte[256];
String data = null;
while (true)
{
Console.Write("Waiting for a connection... ");
TcpClient client = server.AcceptTcpClient();
Console.Write("Connected!");
data = null;
NetworkStream stream = client.GetStream();
int i;
string add_data = "";
Console.Write("Waiting...!");
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
try
{
string full_row = "";
data = System.Text.Encoding.ASCII.GetString(bytes, 0, i);
Console.Write("Received from controller: " + data);
add_data += data;
}
catch{}
}
}
}
then i divide it to get the ID .
It's not clear at all
how you receive requests
upon what you parallelize the application
It seems that you listen on different sockets and assign a thread on each of them.
I think this is a poor choice, because you can't control the parallelism degree.
You could instead have just one thread that nondeterministically listens on all sockets and assigns the connection to a thread pool, configured to simultaneously run at most n threads, where n is the parallelism degree of your machine (if you have a dual quadcore, n is equal to 8).
These methods have strikingly wrong names. start_listen() doesn't start listening. save_data() cannot possibly save any data until at least a connection is established. Which requires the client code to start up first and make the connection. Sure, one second is quickly gone with that. Starting a thread doesn't take more than a fraction of a millisecond when there's a core available.
Focus on the code that's missing in your snippet.
Using this tutorial i came up with the code below. My client is ran frequently. Its activated via clicks and possibly can be launched twice at the same moment in certain circumstance. I am worried one client may close while another client opens which causes the pipe to be closed in that slim few milliseconds. Whats the best way to keep the pipe open?
static public void ThreadStartServer()
{
while (true)
{
using (NamedPipeServerStream pipeStream = new NamedPipeServerStream("mytestpipe"))
{
Console.WriteLine("[Server] Pipe created {0}", pipeStream.GetHashCode());
pipeStream.WaitForConnection();
Console.WriteLine("[Server] Pipe connection established");
using (StreamReader sr = new StreamReader(pipeStream))
{
string temp;
while ((temp = sr.ReadLine()) != null)
{
Console.WriteLine("{0}: {1}", DateTime.Now, temp);
}
}
}
}
Make your pipe server multi-threaded, with one thread dedicated to listening. See these answers: