Develop Reference

BeginWaitForConnection and Generic.List concurrency

I'm using a NamedPipeServerStream and BeginWaitForConnection that gets called for every client connecting to the stream. The callback it takes manipulates a shared static List.
I'm wondering if the fact that BeginWaitForConnection is asynchronous and may be running multiple callbacks in parallel will raise a concurrency issue for that List. I've tried running it a few times and it seems to have worked ok, but I'm not sure if it's thread-safe. Should I be using ConcurrentBag instead, or lock(files) {...} around my FetchFile code? I'm no stranger to asynchronous concepts and multi-threading, but concurrency is fairly new to me, so any insight here is much appreciated.
PipeListener is the entry point here.
static List<string> files = new List<string>();
static void PipeListener()
{
NamedPipeServerStream server = new NamedPipeServerStream("MyPipe", PipeDirection.In, -1,
PipeTransmissionMode.Byte, PipeOptions.Asynchronous);
server.BeginWaitForConnection(FetchFile, server);
}
static void FetchFile(IAsyncResult ar)
{
PipeListener();
NamedPipeServerStream server = ar.AsyncState as NamedPipeServerStream;
server.EndWaitForConnection(ar);
StreamReader reader = new StreamReader(server);
while (!reader.EndOfStream)
files.Add(reader.ReadLine());
server.Dispose();
}

In the callback for BeginWaitForConnection you already start a new BeginWaitForConnection call. This means that concurrent invocations are possible and you need to protect shared state.
Note, that you probably should use await instead of the obsolete APM. Also don't forget to manage your resources with using.

.Net Socket Read function blocking issue

I have client-server application in C#.Net and for that I am using Tcp Socket. I have used following function to aggressive close of socket object.
void CloseSocket(Socket socket)
{
if(socket != null)
{
socket.ShutDown(ocketShutdown.Both);
socket.Close();
}
}
In Normal Condition this function works perfectly and my method returns with 0 bytes returned from Read function.
But whenever client process terminated by taskmanager server program blocks into read function of network stream.
How can I workaround this read blocking function ? I don't want to use AsyncRead function because whole project uses blocking strategy so write now I can't change it to Async pattern.
Thanks, in advance.

I'm assuming that what you are saying is that when the connection isn't closed cleanly by the client, the server can end up blocking at Read indefinitely, even if the client has actually terminated abruptly. If so: yes, that happens. So if you want to use the synchronous read methods, you should use timeouts, in particular ReceiveTimeout. If you have a multi-message protocol, it may be worthwhile adding some kind of heartbeat message periodically, to allow you to correctly identify true zombies from idle connections (for example: if you are sending a heartbeat every minute, and you haven't seen any activity on a connection for 3 minutes, hit it with a shovel).

**you can try this may help you**
public void close()
{
if(clientSocket != null )
{
sendCommand("QUIT");
}
cleanup();
}
private void cleanup()
{
if(clientSocket!=null)
{
clientSocket.Close();
clientSocket = null;
}
logined = false;
}

.Net Remoting uses only one connection ?

Does .Net Remoting opens multiple connections or just one ?
let say I have a server and a client. if the client creates multiple SingleCall objects. then for every object is there going to be a new connection or there going to be a single connection for every object ?
I can't find the answer anywhere.

Long answer is: it depends on many things.
Short answer is yes. Yes Yes.
Let's make an experiment and see the most common default situation.
We have 2 console apps and 1 common class library which is referred by both console apps.
The 1st console app has the role of the client and the 2nd one has that of the server.
First, here's what the commonly depended upon class library contains:
public interface IFactory {
string Hello(string name);
}
Now for some server code. Here's the start up:
private static TcpChannel channel;
static void Main(string[] args) {
BinaryClientFormatterSinkProvider clientProv = new BinaryClientFormatterSinkProvider();
BinaryServerFormatterSinkProvider serverProv = new BinaryServerFormatterSinkProvider();
serverProv.TypeFilterLevel = TypeFilterLevel.Full;
channel = new TcpChannel(
properties: new Hashtable {
{ #"port", 2013 }
},
clientSinkProvider: clientProv,
serverSinkProvider: serverProv
);
ChannelServices.RegisterChannel(channel, false);
RemotingConfiguration.RegisterWellKnownServiceType(typeof(Factory), "Factory.rem", WellKnownObjectMode.SingleCall);
Console.WriteLine("Server started...");
Console.WriteLine("Press any key to stop...");
Console.ReadKey(intercept: true);
}
We just mentioned a class called Factory.
RemotingConfiguration.RegisterWellKnownServiceType(typeof(Factory), "Factory.rem", WellKnownObjectMode.SingleCall);
You guessed it. It's the IFactory implementation:
private sealed class Factory : MarshalByRefObject, IFactory {
#region IFactory Members
string IFactory.Hello(string name) {
return #"Hello " + name + #" !";
}
#endregion
}
Now for some client:
static void Main(string[] args) {
Console.WriteLine("Press any key to connect...");
Console.ReadKey(intercept: true);
IFactory factory = Activator.GetObject(typeof(IFactory), #"tcp://127.0.0.1:2013/Factory.rem") as IFactory;
EventWaitHandle signal = new EventWaitHandle(initialState: false, mode: EventResetMode.ManualReset);
ThreadStart action = () => {
signal.WaitOne();
var result = factory.Hello("Eduard");
Console.WriteLine(result);
};
foreach (var i in Enumerable.Range(0, 99))
new Thread(action) { IsBackground = true }.Start();
Console.WriteLine("Press any key to bombard server...");
Console.ReadKey(intercept: true);
signal.Set();
Console.ReadKey(intercept: true);
}
You already know all of these things, I'm sure. We obtain a transparent proxy to the SingleCall service on the other side (they're both on the same machine and we're using TCP port 2013):
IFactory factory = Activator.GetObject(typeof(IFactory), #"tcp://127.0.0.1:2013/Factory.rem") as IFactory;
Then, for "simulataneous-ness" reasons we create 100 threads, start them (which can take some time), but "hold them in a leash" (a signal is an essential means of synchronization of the OS) until we "pull the trigger":
EventWaitHandle signal = new EventWaitHandle(initialState: false, mode: EventResetMode.ManualReset);
ThreadStart action = () => {
signal.WaitOne();
var result = factory.Hello("Eduard");
Console.WriteLine(result);
};
foreach (var i in Enumerable.Range(0, 99))
new Thread(action) { IsBackground = true }.Start();
So although all 100 threads have been created AND started, they are all waiting in the following invocation:
signal.WaitOne();
That way we can get the to start at the same time better, otherwise the creation and starting of threads itself would have made their actual execution more or less sequential.
We ask the user to decide when to "bombard the server" with 100 Hello invocations:
Console.WriteLine("Press any key to bombard server...");
Console.ReadKey(intercept: true);
signal.Set();
And this is what happens:
1) We start the server console app and let it run in peace:
2) We start the client console app, "make the connection" by pressing any key (which is only a logical connection since it just creates a transparent proxy) but we postpone the "bombardment":
3) We start up Mark Russinovich's Process Explorer and use it to discover the client process in the process list, and while doing that we open it's properties window and select the TCP/IP tab:
4) We hit any key in the client console app, and .. TA DAA !!
You get a lot of connections in Process Explorer.
Are they a hundred ? Sometimes yes, sometimes no.
It's a connection pool, that's for sure.
After a short while (5 to 10 seconds) of idleness they close down which is a very good thing (because the .NET Remoting stack is implemented that way).
I hope this experiment generally answered your question.
In more specific cases and in a more rigorous sense you should check out the documentation and read about the various channels that you might use in your .NET Remoting apps (there are loads of them out there, what you've seen here is just the regular TcpChannel officially provided by Microsoft, it depends on what your .NET Remoting configuration says, on whether you're hosting the server in IIS or not, etc).

The number of network connections depends on remoting channel you use. The default TcpChannel opens as many network connections as many threads in your program try to access the server during one point of time.
For single-threaded applications, TcpChannel uses one network connection.
As an opposite example, a third party remoting channel IiopChannel uses multiplexing and thus allows to have just a few network connections for many hundreds of active threads.

can a c# lock serialize access to an external resouce?

i have a multi-threaded application which wants to send a sequence of data to an external device via a serial port. the sequence of data is a typical cmd - response protocol (ie: a given thread sends a sequence of bytes then waits to read a response which is typically an ack and then it might send another sequence).
what we are are looking to do is declare a sequence of code has exclusive access to this resource until it is done and if another thread wants access to the same external resouce, it waits.
this seems like what LOCK does, but all the examples that i have seen show lock being used to protect a specific block of code, not to serialize access to a resource.
so programatically can i have
Object serialPortLock = new Object();
and in different parts of my program use a construct that looks like:
Lock (serialPortLock)
{
// my turn to do something that is not the same as what
// someone else wants to do but it acts on the same resource
}
the c# documentation talks about using Mutex as a more robust version of Lock. is that whats required here?

Yes, your pattern is correct as long as your program is the only software accessing the serial port.
You have not posted your entire code. If the class that contains serialPortLock has multiple instances, then you MUST make serialPortLock a static. This is normally best practice.
class MySerialPort
{
static object synchLock = new object();
public void DoSomething()
{
lock (synchLock)
{
// whatever
}
}
}

Locking should work fine in the case you've suggested as long as you are locking around all access to any of the object instances that point at the external resource.

Reading from MSMQ slows down when there is a lot of messages queued

Short introduction
I have a SEDA based system, and used MSMQ for communication (event triggering) between the different applications/services.
One of these services gets messages by file, so I have a file listener that reads the file content and inserts this into a queue (or actually 4 different queues, but that's not very important for the first question).
Server is Windows Server 2008
First question - read slows down
My application that reads these messages at the other side normally reads about 20 messages from the queue per second, but when the service that posts messages start queuing some thousand messages, the read goes down, and the read application only reads 2-4 messages per second. When there is no posting to the queue, the read application can again read up to 20 messages per second.
The code in the reading application is pretty simple, developed in C#, I use the Read(TimeSpan timeout) function in System.Messaging.
Q: Why does the read slows down when there is a lot of messages posted to the queue?
Second question - limitations of TPS
An additional question is about the read itself. It seems there is no difference in how many messages I can read per second if I use 1 or 5 threads to read from the queue. I've also tried implementing a "round robin solution" where the post service are posting to a random set of 4 queues, and the read application had one thread listening to each of these queues, but there is still only 20 TPS even if I read from 1 queue with 1 thread, 1 queue with 4 threads or 4 queues (with one thread per queue).
I know the processing in the thread takes about 50 ms, so 20 TPS is quite correct if there is only one message processed at the time, but the clue with multi threading should be that messages are handled in parallel and not sequential.
There is about 110 different queues on the server.
Q: Why can't I get more than 20 messages out of my queue at the time even with multi threading and the use of several queues?
This is the code running today:
// There are 4 BackgroundWorkers running this function
void bw_DoWork(object sender, DoWorkEventArgs e)
{
using(var mq = new MessageQueue(".\\content"))
{
mq.Formatter = new BinaryMessageFormatter();
// ShouldIRun is a bool set to false by OnStop()
while(ShouldIRun)
{
try
{
using(var msg = mq.Receive(new TimeSpan(0,0,2))
{
ProcessMessageBody(msg.Body); // This takes 50 ms to complete
}
}
catch(MessageQueueException mqe)
{
// This occurs every time TimeSpan in Receive() is reached
if(mqe.MessageQueueErrorCode == MessageQueueErrorCode.IOTimeout)
continue;
}
}
}
But even if there are 4 threads, it seems all waits for the function to enter the "Receive" point again. I've also tried using 4 different queues (content1, content2, content3 and content4), but still i get 1 message processed every 50 ms.
Does this have anything to do with the TimeSpan in Receive(), and/or is it possible to omit this?
Another question is if the use of private queues, instad of public will solve anything?

Performance issues.
You don't mention if all the code is running on the server or if you have clients remotely accessing the queues on the server. From the speed, I'll assume the latter.
Also, are the queues transactional?
How large are the messages?
If you want to read a message from a queue, your application does not connect to the queue itself. Everything goes between the local queue manager and the remote queue manager. The queue manager is the only process that writes to, and reads from queues. Therefore having multiple queues or a single queue won't necessarily perform any differently.
The MSMQ queue manager is therefore going to be a bottleneck at some point as there is only so much work it can do at the same time. Your first question shows this - when you put a high load on the queue manager putting messages IN, your ability to take messages OUT slows down. I'd recommend looking at performance monitor to see if MQSVC.EXE is maxed out, for example.

Why are you using timespan? - that is a bad thing and here is why.
When developing services and queue you need to program in a theadsafe manner. Each item in the queue will spawn a new thread. Using timespan is forcing each of the threads to use a single timer event thread. These events are having to wait for their turn at the event thread.
The norm is 1 thread per queue events - This is generally your System.Messaging.ReceiveCompletedEventArgs event. Another thread is your onStart event...
20 threads or 20 reads per second is probably correct. Generally when thread pooling you can only spawn 36 threads at a time in .net.
My advice is drop the timer event an make your queue simply process the data.
do something more like this;
namespace MessageService
{
public partial class MessageService : ServiceBase
{
public MessageService()
{
InitializeComponent();
}
private string MessageDirectory = ConfigurationManager.AppSettings["MessageDirectory"];
private string MessageQueue = ConfigurationManager.AppSettings["MessageQueue"];
private System.Messaging.MessageQueue messageQueue = null;
private ManualResetEvent manualResetEvent = new ManualResetEvent(true);
protected override void OnStart(string[] args)
{
// Create directories if needed
if (!System.IO.Directory.Exists(MessageDirectory))
System.IO.Directory.CreateDirectory(MessageDirectory);
// Create new message queue instance
messageQueue = new System.Messaging.MessageQueue(MessageQueue);
try
{
// Set formatter to allow ASCII text
messageQueue.Formatter = new System.Messaging.ActiveXMessageFormatter();
// Assign event handler when message is received
messageQueue.ReceiveCompleted +=
new System.Messaging.ReceiveCompletedEventHandler(messageQueue_ReceiveCompleted);
// Start listening
messageQueue.BeginReceive();
}
catch (Exception e)
{
}
}
protected override void OnStop()
{
//Make process synchronous before closing the queue
manualResetEvent.WaitOne();
// Clean up
if (this.messageQueue != null)
{
this.messageQueue.Close();
this.messageQueue = null;
}
}
public void messageQueue_ReceiveCompleted(object sender, System.Messaging.ReceiveCompletedEventArgs e)
{
manualResetEvent.Reset();
System.Messaging.Message completeMessage = null;
System.IO.FileStream fileStream = null;
System.IO.StreamWriter streamWriter = null;
string fileName = null;
byte[] bytes = new byte[2500000];
string xmlstr = string.Empty;
try
{
// Receive the message
completeMessage = this.messageQueue.EndReceive(e.AsyncResult);
completeMessage.BodyStream.Read(bytes, 0, bytes.Length);
System.Text.ASCIIEncoding ascii = new System.Text.ASCIIEncoding();
long len = completeMessage.BodyStream.Length;
int intlen = Convert.ToInt32(len);
xmlstr = ascii.GetString(bytes, 0, intlen);
}
catch (Exception ex0)
{
//Error converting message to string
}
}