Develop Reference

C# How to make a TCP server raise an event/callback when there is new data from the client

So basically I want my server to raise an event (or a callback) when a connected client sends data. I can't come up with a solution to this problem and can't find anything online after days of searching.
What I've thought of was making an asynchronous foreach loop that looped through all the connected users, and check if there is any data to be read on each one (using TcpClient.Avaliable, but a network stream could also check this) but an infinite loop like this without any stop would be bad practice and use an insane amount of resources (from what I understand at least, I am new to threading and networking).
There is logic I need to be executed whenever the server gets data from a client (in this case a message, because it's a chat application), basically broadcast it to every other user, but I just can't find out how to detect if any user has sent data so that it raises an event to broadcast the message, log the message, etc...
Please be "soft" with the explanations as I am new to threading/networking and ty in advance.
As per request here is my code, take note that it is prototype-y and a bit unfinished, but I'm sure it gets the point across:
//Properties
public List<User> ConnectedUsers { get; private set; } = new List<User>();
public TcpListener listener { get; set; }
public bool IsListeningForConnections { get; set; }
public int DisconnectionCheckInterval { get; set; } //in seconds
//Events
public event EventHandler<ServerEventArgs> UserConnected;
public event EventHandler<ServerEventArgs> MessageReceived;
public NetworkManager()
{
listener = new TcpListener(IPAddress.Parse("192.168.1.86"), 6000); //binds // TODO: Change to: user input / prop file
DisconnectionCheckInterval = 10;
IsListeningForConnections = false;
}
public async void StartListeningForConnections()
{
IsListeningForConnections = true;
listener.Start();
while (IsListeningForConnections)
{
User newUser = new User();
newUser.TcpClient = await listener.AcceptTcpClientAsync();
OnUserConnected(newUser); // raises/triggers the event
}
}
public void StartListeningForDisconnections()
{
System.Timers.Timer disconnectionIntervalTimer = new System.Timers.Timer(DisconnectionCheckInterval * 1000);
//TODO: setup event
//disconnectionIntervalTimer.Elasped += ;
disconnectionIntervalTimer.AutoReset = true;
disconnectionIntervalTimer.Enabled = true;
//disconnectionIntervalTimer.Stop();
//disconnectionIntervalTimer.Dispose();
}
public async void StartListeningForData()
{
//??????????
}
public async void SendData(string data, TcpClient recipient)
{
try
{
byte[] buffer = Encoding.ASCII.GetBytes(data);
NetworkStream stream = recipient.GetStream();
await stream.WriteAsync(buffer, 0, buffer.Length); //await
Array.Clear(buffer, 0, buffer.Length);
}
catch { } //TODO: handle exception when message couldn't be sent (user disconnected)
}
public string ReceiveData(TcpClient sender)
{
try
{
NetworkStream stream = sender.GetStream();
byte[] buffer = new byte[1024];
stream.Read(buffer, 0, buffer.Length);
return Encoding.ASCII.GetString(buffer).Trim('\0');
}
catch
{
return null; //TODO: handle exception when message couldn't be read (user disconnected)
}
}
protected virtual void OnUserConnected(User user)
{
ConnectedUsers.Add(user);
UserConnected?.Invoke(this, new ServerEventArgs() { User = user });
}
protected virtual void OnMessageReceived(User user, Message message) //needs trigger
{
MessageReceived?.Invoke(this, new ServerEventArgs() { User = user, Message = message });
}
basically a different class will call all the 3 classes that start with "StartListeningForX", then one of the 3 corresponding events are raised when one of the checks goes through (disconnection/connection/new message), and process that data, I just can't get my hands on how to call an event when a new message arrives for each user.

What I've thought of was making an asynchronous foreach loop that looped through all the connected users, and check if there is any data to be read on each one (using TcpClient.Avaliable, but a network stream could also check this) but an infinite loop like this without any stop would be bad practice and use an insane amount of resources
The standard practice is to have an "infinite" loop for each connected client, so that there is always a read going on every socket. I put "infinite" in quotes because it will actually eventually stop; either by reading 0 bytes (indicating end of stream) or by receiving an exception (indicating a broken connection).
I am new to threading/networking
It's funny how often I see developers trying to learn networking and threading at the same time. Let me be clear: threading and TCP/IP sockets are both extremely complicated and take quite a bit of time to learn all the sharp corners. Trying to learn both of these topics at once is insane. I strongly recommend choosing one of them to learn about (I'd recommend threading first), and only after that one is mastered, proceed to the other.

RabbitMQ
If you have access to the client side code, I'd consider using something like RabbitMQ, or a similar queue service. This allows to link the different apps together through a message broker or queue, and get messages/events real time.
There are functions you can call on event received.

.NET Client - Waiting for an MQTT response before proceeding to the next request

I have a MQTT calls inside a loop and in each iteration, it should return a response from the subscriber so that I could use the value being forwarded after I published. But the problem is I don't know how would I do it.
I hope you have an idea there or maybe if I'm just not implementing it right, may you guide me through this. Thanks.
Here's my code:
// MyClientMgr
class MyClientMgr{
public long CurrentOutput { get; set; }
public void GetCurrentOutput(MyObjectParameters parameters, MqttClient client)
{
MyMessageObject msg = new MyMessageObject
{
Action = MyEnum.GetOutput,
Data = JsonConvert.SerializeObject(parameters)
}
mq_GetCurrentOutput(msg, client);
}
private void mq_GetCurrentOutput(MyMessageObject msg, MqttClient client)
{
string msgStr = JsonConvert.SerializeObject(msg);
client.Publish("getOutput", Encoding.UTF8.GetBytes(msgStr),
MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE, false);
client.MqttMsgPublishReceived += (sender, e) =>{
MyObjectOutput output = JsonConvert.DeserializeObject<MyObjectOutput>(Encoding.UTF8.GetString(e.Message));
CurrentOutput = output;
};
}
}
// MyServerMgr
class MyServerMgr
{
public void InitSubscriptions()
{
mq_GetOutput();
}
private void mq_GetOutput()
{
MqttClient clientSubscribe = new MqttClient(host);
string clientId = Guid.NewGuid().ToString();
clientSubscribe.Connect(clientId);
clientSubscribe.Subscribe(new string[] { "getOutput" }, new byte[] { MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE });
MqttClient clientPublish = new MqttClient(host);
string clientIdPub = Guid.NewGuid().ToString();
clientPublish.Connect(clientIdPub);
clientSubscribe.MqttMsgPublishReceived += (sender, e) => {
MyMessageObj msg = JsonConvert.DeserializeObject<MyMessageObj>(Encoding.UTF8.GetString(e.Message));
var output = msg.Output;
clientPublish.Publish("getOutput", Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(output)), MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE, false);
}
}
}
// MyCallerClass
class MyCallerClass
{
var host = "test.mqtt.org";
var myServer = new MyServerMgr(host);
var myClient = new MyClientMgr();
myServer.InitSubscriptions();
MqttClient client = new MqttClient(host);
for(int i = 0; i < 10; i++)
{
long output = 0;
MyObjectParameters parameters = {};
myClient.GetCurrentOutput(parameters, client) // here I call the method from my client manager
// to publish the getting of the output and assigned
// below for use, but the problem is the value doesn't
// being passed to the output variable because it is not
// yet returned by the server.
// Is there a way I could wait the process to
// get the response before assigning the output?
output = myClient.CurrentOutput; // output here will always be null
// because the response is not yet forwarded by the server
}
}
I have a loop in my caller class to call the mqtt publish for getting the output, but I have no idea how to get the output before it was assigned, I want to wait for the response first before going to the next.
I've already tried doing a while loop inside like this:
while(output == 0)
{
output = myClient.CurrentOutput;
}
Yes, I can get the output here, but it will slow down the process that much. And sometimes it will fail.
Please help me. Thanks.

It looks like you are trying to do synchronous communication over an asynchronous protocol (MQTT).
By this I mean you want to send a message and then wait for a response, this is not how MQTT works as there is no concept of a reply to a message at the protocol level.
I'm not that familiar with C# so I'll just give an abstract description of possible solution.
My suggestion would be to use a publishing thread, wait/pulse (Look at the Monitor class) to have this block after each publish and have the message handler call pulse when it has received the response.
If the response doesn't contain a wait to identify the original request you will also need a state machine variable to record which request is in progress.
You may want to look at putting a time out on the wait in case the other end does not respond for some reasons.

You can use AutoResetEvent class that has WaitOne() and Set() methods. Using WaitOne() after publish will wait until the message is published and using Set() under client_MqttMsgPublishReceived event will release the wait when the subscriber received the message he subscribed for.

Read event argument after wait

I have a hardware component that I use with TCP communication. The class that control it has 3 jobs: Trigger the device, listen to incoming messages and raise an event when a message is received:
public class Hardware
{
public event Action<string> OnHardwareMessage;
private NetworkStream stream = new NetworkStream();
public Hardware()
{
Task.Factory.StartNew(() => { Listen(); });
}
private void Listen()
{
//listen to TCP port and raise an event when a message is received
byte[] bytes = new byte[1024];
int bytesRead = stream.Read(bytes, 0, bytes.Length);
string response = Encoding.ASCII.GetString(bytes, 0, bytesRead);
if (OnHardwareMessage != null)
OnHardwareMessage(response);
}
public void Trigger()
{
//Trigger the hardware component
//the response usually takes up to 5 seconds to arrive
}
}
This class is used in a loop inside a view-model:
public class MainViewModel
{
private static EventWaitHandle hardwareWaiter =
new EventWaitHandle(false, EventResetMode.AutoReset);
private Hardware hardware = new Hardware();
//what i'm doing now is holding a field for incoming event results
private string hardwareResult;
public MainViewModel()
{
hardware.OnHardwareMessage += hardware_OnHardwareMessage;
while (true)
{
hardware.Trigger();
if (hardwareWaiter.WaitOne(10000))
{
//is it possible to read the event argument here?
//do something with the event argument
someObservableCollection.Add(hardwareResult);
//clear the value
hardwareResult = string.Empty;
}
else
{
throw new Exception("Hardware did not respond on time");
}
}
}
//event listener for hardware events
private void hardware_OnHardwareMessage(string message)
{
hardwareResult = message;
hardwareWaiter.Set();
}
}
What I do is trigger the device and wait for up to 10 seconds for a response. What I'm doing now is holding a class scoped field, assign the message received inside the event listener, read it in the inside the loop and clear it.
My question is if there's any built in mechanism that could let me read the event argument directly after the EventWaitHandle was signaled (outside the event listener).

I would not base this on events. Maybe your listener should expose a BlockingCollection that represents the incoming stream of messages. Readers can then take from that collection with a timeout. If the timeout hits no message is taken and lost (this is a race condition that you have right now).
If you insist on using events use a closure to thread the message state:
string hardwareResult = null;
Action handler = (...) =>
{
hardwareResult = message;
hardwareWaiter.Set();
};
hardware.OnHardwareMessage += handler;
That way you only need a local variable which is scoped more tightly than a field.
There's the usual TCP bug as well: You assume to read an entire message. You could read one byte at a time instead.

Task.Start() does not return until completed (Big Updated)

I have a WinForms C# application (Web Crawler) which connects to a WCF self hosted Service, which in turn connects to an MS SQL Database.
The WebCrawler Starts (logs the user in etc) LoginActual();
Receives a URL to process and
starts a BackGroundWorker which actually starts the Crawler RunCrawler();
RunCrawler() gets the HTML from the URL to process and gets all the URLs within the html. The Urls are added to alOutUrls, which is an ArrayList. The URLs are then sent to SendURls(); If the URL being Processed returned an Error such as 404 then SendError(); is called.
If alOutUrls has more than 100 urls in it then it adds the URLs to a Queue for processing in a Task. And starts the RunQueue method as a new task.
Else
It iterates through alOutUrls in a foreach sending the Raptor Service API (RAPI) the URL and additional information.
RunQueue() Iterates through the Queue in a While statement sending the Raptor Service API (RAPI) the URL taken from the Queue and other additional information.
However, The following problems occur all the time!
Time Outs on the Send from the Client to the Service Host
The RunQueue() method does not seem to run in the background but blocks the application until it is completed; which defeats the problem.
HUGE UPDATE
A massive simplification of the code is here - which still has the same problems.
private ArrayList alOutUrls = new ArrayList();
private void RunCrawler()
{
while(true)
{
int ErrorCode = 0;
string cu = alUrls[0].ToString();
string html = LoadUrlIfNotContentType(cu, out ErrorCode);
if (!string.IsNullOrEmpty(html))
{
alOutUrls = GetUrls(html, cu);
SendUrls(cu, ErrorCode); // Send the URLS
}
else
SendError(cu, ErrorCode); // Send the StatusCode of the Url 404, 500 etc
alUrls.RemoveAt(0);
alUrls.AddRange(RAPI.SendUrls());
}
}
private Task tRunQueue = null;
private Queue<string> bigqueue = new Queue<string>();
private bool IsContentObject = false;
private void SendUrls(string cu, int ErrorCode)
{
foreach (string u in alOutUrls)
{
bigqueue.Enqueue(u);
}
if(tRunQueue == null)
{
tRunQueue = new Task(() => RunQueue(cu, IsContentObject, ErrorCode));
tRunQueue.Start();
}
}
private void RunQueue(string cu, bool IsContentObject, int ErrorCode)
{
while (bgwCrawler.CancellationPending != true)
{
if (bigqueue.Count > 0)
{
string url = bigqueue.Dequeue();
string urlHash = Hashing.HashString(url.ToLowerInvariant().Trim().ToString());
RAPI.ReceiveUrls(url, urlHash, cu, IsContentObject, ErrorCode);
}
}
}
So to reiterate the problems are:
Time outs on the send from the client to the service host.
The RunQueue() method does not run in the background.
Any insights would be helpful, this is a very important program
designed to protect children from sexual exploitation and murder.

Found the problem:
Application.DoEvents();
RunQueue is supposed to be running in the background. But by calling Application.DoEvents() you are hijacking the UI processing. I don't know the exact effect of calling that from a background thread, but it can't be good.

SocketAsyncEventArgs.UserToken not being updated?

I currently have a client-server application that involves a Silverlight client and a .NET server. The .NET portion uses the Tcp classes provided in System.Net.Sockets namespace whereas the Silverlight client uses raw sockets. I'm porting into this from code that currently uses the HttpListener classes because it doesn't suit my needs. The Http classes, though, have on the SL side the ability to use Begin* and End* style asynchronous methods that allow me to specify a handler once the operation has completed. I'm having trouble getting this to work with the new system. My current strategy is to include the handler as part of the UserToken. However, it seems that this token is not getting updated.
Here is some redacted code. I am able to get the two sides to talk to each other, but it seems the correct UserToken is not being sent.
public class ClientUserToken
{
public Handler Handler { get; set; }
public string Test { get; set; }
public ClientUserToken(Handler handler, string test)
{
Handler = handler;
Test = test;
}
}
public class SocketClient
{
private Socket _clientSocket;
private string _ipAddress;
private int _port;
private void OpenSocket()
{
var endPoint = new DnsEndPoint(_ipAddress, _port);
SocketAsyncEventArgs args = new SocketAsyncEventArgs();
args.UserToken = new ClientUserToken(null, "foo");
args.RemoteEndPoint = endPoint;
args.Completed += OnSocketCompleted;
_clientSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
_clientSocket.ConnectAsync(args);
}
void OnSocketCompleted(object sender, SocketAsyncEventArgs e)
{
switch (e.LastOperation)
{
case SocketAsyncOperation.Connect: ProcessConnect(e); break;
case SocketAsyncOperation.Receive: ProcessReceive(e); break;
case SocketAsyncOperation.Send: ProcessSend(e); break; // this never gets called
}
}
void ProcessConnect(SocketAsyncEventArgs e)
{
if (e.SocketError == SocketError.Success)
{
byte[] response = new byte[1024];
e.SetBuffer(response, 0, response.Length);
_clientSocket.ReceiveAsync(e);
}
}
void ProcessReceive(SocketAsyncEventArgs e)
{
if (e.SocketError == SocketError.Success && e.BytesTransferred > 0)
{
var userToken = e.UserToken as ClientUserToken; // this token is always the one set in ProcessConnect
// process the data
if (!_clientSocket.ReceiveAsync(e))
{
ProcessReceive(e);
}
}
}
// this is never called, for some reason
void ProcessSend(SocketAsyncEventArgs e)
{
if (e.SocketError == SocketError.Success)
{
var userToken = e.UserToken as ClientUserToken;
if (!_clientSocket.ReceiveAsync(e))
{
ProcessReceive(e);
}
}
}
// this is the public API that users use to actually send data across
public void SendToServer(byte[] data, int len, Handler handler)
{
SocketAsyncEventArgs args = new SocketAsyncEventArgs();
args.UserToken = new ClientUserToken(handler, "bar");
args.SetBuffer(data, 0, len);
if (!_clientSocket.SendAsync(args))
{
ProcessReceive(args);
}
}
}
As the comments above suggest, in ProcessReceive, userToken.Test is always "foo" and userToken.Handler is always null.
I have so far not been able to break into ProcessSend so I can't see what SocketAsyncEventArgs it actually sent. Anyone have a clue why this event isn't firing (Completed after a Send)? Am I screwing up my SendToServer function?
I realize there may be other existing problems with synchronization and such, but I don't think that's the issue here. One thing I did try was setting up a ManualResetEvent to ensure that no one sends data to the server before the connection has been completed (ProcessConnect) but that did not solve the issue, either.
Any help will be appreciated!
EDIT: So the reason this is happening is because when I call ReceiveAsync in the ProcessConnect function, it is being used when the server is sending back the response for my data. Hence, UserToken "foo" is present in the handler. The next time the server sends data, the ReceiveAsync uses the args with the UserToken "bar". So it is kind of out of sync, for the duplex communication bit. I can't ensure that the SocketAsyncEventArgs that I sent from the client-side is the same one that is used on the response. It seems like the only solution is to have the SL client open two sockets--one for server-initiated data and the other for client-initiated requests. However, this means I'm not taking advantage of the duplex nature.

This model won't work because I'm creating a new SocketAsyncEventArgs on each send, which means that the data can come back on any of these args. I've been moving towards a model with a pool of SocketAsyncEventArgs and each client can only have one request/response at a time.