Here is my code I'm having an issue with:
// read the file in chunks of 5KB
var buffer = new byte[1024*5];
int bytesRead = 0;
do
{
bytesRead = host.Receive(buffer, buffer.Length, 0);
output.Write(buffer, 0, bytesRead);
}
while (bytesRead == buffer.Length);
So here's my problem: I want to read in data as long as the buffer is full. However, even if there is more data, the buffer isn't guaranteed to be filled when sending. This causes the Receive to prematurely exit. If I change the while condition to bytesRead > 0 then it reaches the end of data and Receive blocks until more data is available (which it won't be). How do I solve this?
I think you need to think about how the protocol works and make a more robust solution. Either you go with blocking and then you could either wait until you have no more data to read and then do something else and read again when you know you have more data to read.
Or you add threading and have a seperate thread that just reads and then it doesn't matter if it blocks since it will be on a seperate thread.
Another solution that might be simpler is to use asynchronous reads.
You can read more about it here: https://msdn.microsoft.com/en-us/library/bbx2eya8(v=vs.110).aspx
A simple example from the site above is to start with the read:
private static void Receive(Socket client) {
try {
// Create the state object.
StateObject state = new StateObject();
state.workSocket = client;
// Begin receiving the data from the remote device.
client.BeginReceive( state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReceiveCallback), state);
} catch (Exception e) {
Console.WriteLine(e.ToString());
}
}
Then you will get a callback to ReceiveCallback when there is data to be handled:
private static void ReceiveCallback( IAsyncResult ar ) {
try {
// Retrieve the state object and the client socket
// from the asynchronous state object.
StateObject state = (StateObject) ar.AsyncState;
Socket client = state.workSocket;
// Read data from the remote device.
int bytesRead = client.EndReceive(ar);
if (bytesRead > 0) {
// There might be more data, so store the data received so far.
state.sb.Append(Encoding.ASCII.GetString(state.buffer,0,bytesRead));
// Get the rest of the data.
client.BeginReceive(state.buffer,0,StateObject.BufferSize,0,
new AsyncCallback(ReceiveCallback), state);
} else {
// All the data has arrived; put it in response.
if (state.sb.Length > 1) {
response = state.sb.ToString();
}
// Signal that all bytes have been received.
receiveDone.Set();
}
} catch (Exception e) {
Console.WriteLine(e.ToString());
}
}
I hope this works if not we need to know more about what protocol you use and if there is some reason you need blocking reads so we can give more specific information and help.
Related
I have written a web service that's an Asp.Net MVC application hosted in IIS. The data from the web service isn't retrieved from a database but from another server that is accessed via TCP. We'll call this the Data Server. There can be multiple Data Servers that the web service connects to.
For the sake of discussion, a user authenticates by specifying a username and password then a "Session" with a Socket is created with the appropriate Data Server. Assuming everything is good - we keep the Socket alive and pass the user a token that identifies the Session they belong to.
For each Socket I need to prevent traffic from interrupting each other. I assume that the best way to do that is to run the network traffic on a Socket in a serialized manner. I have achieved this by using a lock. My code to execute a query on this Data Server follows. The problem I'm having is that once in a while it appears that I'm getting two queries that collide and one may hang. I see a query come in but it looks like it gets stuck at the lock. Is the lock mechanism safe for IIS async calls? Is there instrumentation I can put in to make sure this is actually the bug? I've been looking for a while but I can't find guidance for this particular scenario.
private async Task<string> query(string request, AbstractPermission context = null, bool bUpdateDateTime = true)
{
try
{
string reply = "";
isErrorMsg = false;
//this lock prevents the keep alive thread from coming in here while we're on the socket
lock (socketLock)
{
sendDone.Reset();
receiveDone.Reset();
// Send test data to the remote device.
Send(Socket, request);
sendDone.WaitOne();
// Receive the response from the remote device.
Receive(Socket);
receiveDone.WaitOne();
reply = QueryResponse;
} //done reading - let's unlock
if (bUpdateDateTime)
{
this.LastUsed = DateTime.Now;
}
return QueryResponse;
}
catch (Exception e)
{
throw e;
}
}
private void Send(Socket client, String data)
{
byte[] byteData = Encoding.ASCII.GetBytes(data);
client.BeginSend(byteData, 0, byteData.Length, 0,
new AsyncCallback(SendCallback), client);
}
private void SendCallback(IAsyncResult ar)
{
try
{
// Retrieve the socket from the state object.
Socket client = (Socket)ar.AsyncState;
// Complete sending the data to the remote device.
int bytesSent = client.EndSend(ar);
// Signal that all bytes have been sent.
sendDone.Set();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}
private void Receive(Socket client)
{
try
{
// Create the state object.
StateObject state = new StateObject();
state.workSocket = client;
state.PostInitialRead = false;
// Begin receiving the data from the remote device.
client.BeginReceive(state.buffer, 0, StateObject.BufferSize, SocketFlags.None,
new AsyncCallback(ReceiveCallback), state);
}
catch (Exception e)
{
LogError(e);
}
}
private void ReceiveCallback(IAsyncResult ar)
{
try
{
// Retrieve the state object and the client socket
// from the asynchronous state object.
StateObject state = (StateObject)ar.AsyncState;
Socket client = state.workSocket;
bool PostInitialRead = state.PostInitialRead;
// Read data from the remote device.
int bytesRead = client.EndReceive(ar);
//
//
var thisBatch = Encoding.ASCII.GetString(state.buffer, 0, bytesRead);
var endIdx = thisBatch.IndexOf('\x04');
if (!PostInitialRead)
{
if (bytesRead == 0)
throw new ApplicationException("Timeout waiting for response");
if (endIdx != -1)
{
thisBatch = thisBatch.Substring(0, endIdx);
}
if (state.buffer[0] != 0)
{
thisBatch = thisBatch.Substring(1, state.buffer[0]);
isErrorMsg = true;
}
else if (state.buffer[1] != 0)
{
thisBatch = thisBatch.Substring(2);
isErrorMsg = true;
}
else
{
thisBatch = thisBatch.Substring(2);
}
state.sb.Append(thisBatch);
state.PostInitialRead = true;
}
else
{
state.ms.Write(state.buffer, 0, endIdx!=-1?endIdx:bytesRead);
}
if (endIdx != -1)
{
// Got everything
state.sb.Append(Encoding.ASCII.GetString(state.ms.ToArray()));
QueryResponse = state.sb.ToString();
receiveDone.Set();
return;
}
else
{
// Get the rest of the data.
client.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReceiveCallback), state);
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}
1: Is hosting something like this within IIS completely absurd?
No. See, dotnetcore and modern development uses SignalR based on WebSockets for stuff like that - and it is hosted in IIS, so it is NOT completely absurd.
2: We can not really answer that. The IO part is trivial - IIS can handle this. But without details on the data server - no idea. YOu generally want to avoid locks as much as possible, but that does not mean totally. MRSV (multiple Reader, Single Writer), copy on write etc. can help minimizing writes, but at the end you will need SOME locking. It is NOT fun debugging this, but that is what people doing that get paid big bucks for.
General applicatiosn avoid all that by offloading the locking to the database at the backend - which spends a LOT of time by many people optimizing locking on data. If you can not do that (remember, we do not know at all what your data server does internally) - welcome to the hard programming.
Your best chance for debugging is trying to find a repro case and then - while it is stuck - attach a debugger. Stuff like that is NOTORIOUSLY hard to debug - but again, this is like the topmost level of regular programming (leaving out certain hardware and very special tasks).
I have a C# TPC Socket with the following code:
public static void ReadCallback(IAsyncResult ar)
{
String content = String.Empty;
// Retrieve the state object and the handler socket
// from the asynchronous state object.
StateObject state = (StateObject)ar.AsyncState;
Socket handler = state.workSocket;
// Read data from the client socket.
int bytesRead = handler.EndReceive(ar);
if (bytesRead > 0)
{
// There might be more data, so store the data received so far.
state.sb.Append(Encoding.ASCII.GetString(
state.buffer, 0, bytesRead));
// HERE WE NEED TO MAKE SURE THAT THE MESSAGE IS COMPLETE, IF NOT THEN READ MORE DATA
if (bytesRead == ***something***)
{
//Do something HERE...
}
else
{
// Not all data received. Get more.
handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReadCallback), state);
}
}
}
The data that will be sent to the TCP Socket will be formed like this:
So the size can be different for every message. The first 10 bytes and the last 4 bytes are always fixed, but the payload is dynamic.
So I have to implement a way of validating the size of the message by intercepting the 4 bytes of the PAYLOAD SIZE position, that way is just matter of doing a sum of 2 + 4 + 4 + payload size + 4 so i can enter the if statement where I will do some other stuff there.
Any advice or clue on the best way to do this?
You can do it like this:
static void ReadCallback(IAsyncResult ar)
{
//put received bytes into queue for further processing
//initiate BeginReceive
}
On another thread:
static void ProcessMessages()
{
while(true)
{
//read messages from queue
//check message frames
}
}
You can check how it can be implemented in simplsockets, method ProcessReceivedMessage.
I'm trying to develop a project where I can have multiple clients making their own server requests for the purpose of stress testing. I'm having a lot of difficulty figuring out how I can manipulate custom control properties when I make a new thread and want that thread to do the work. I have upwards of 100 controls; so ideally 100 individual clients. The problem is my controls are part of the GUI and I don't know how to allow that thread in question get access from that relative control.
Here's what I have:
// Custom project.class to get access to a custom base of properties created within the tool itself.
List<custom_control_project.custom_control_widget> controlList = new List<custom_control_project.custom_control_widget>();
private async void btnStart_Click(object sender, EventArgs e)
{
...// control property initializations
foreach (var control in controlList)
{
if (control.Enabled)
{
Thread thread = new Thread(() => StartClient());
thread.Start();
// Loop until worker thread activates.
while (!thread.IsAlive);
... // Ideally the GUI updates would happen from these threads. Simple updates to labels based on status code responses and expected xml parameters received.
}
}
My StartClient() is largely based off Microsofts asynchronous socket client example here: http://msdn.microsoft.com/en-us/library/bew39x2a%28v=vs.110%29.aspx
I was running these clients asynchronously but the program was not my end result. I have made a few changes, including resetting the ManualResetEvents. However, when I run my application, all the controls still run one after the other, and I'd like them to be independent. Do I have the right approach by making new threads with the StartClient()?
Referencing microsofts example, the part I'm most interested in is the ReceiveCallback(IAsyncResult ar) method:
public class StateObject
{
// Client socket.
public Socket workSocket = null;
// Size of receive buffer.
public const int BufferSize = 1024;
// Receive buffer.
public byte[] buffer = new byte[BufferSize];
// Received data string.
public StringBuilder sb = new StringBuilder();
}
// ManualResetEvent instances signal completion.
private ManualResetEvent connectDone =
new ManualResetEvent(false);
private ManualResetEvent sendDone =
new ManualResetEvent(false);
private ManualResetEvent receiveDone =
new ManualResetEvent(false);
// The response from the remote device.
private String response = String.Empty;
public void StartClient()
{
// Connect to a remote device.
try
{
// Establish the remote endpoint for the socket.
// The name of the
// remote device is "host.contoso.com".
//IPHostEntry ipHostInfo = Dns.Resolve("host.contoso.com");
//IPAddress ipAddress = ipHostInfo.AddressList[0];
//IPEndPoint remoteEP = new IPEndPoint(ipAddress, port);
// Create a TCP/IP socket.
Socket client = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);
// Connect to the remote endpoint.
client.BeginConnect(asyncServerHolder, asyncPortHolder,
new AsyncCallback(ConnectCallback), client);
connectDone.WaitOne();
// Send test data to the remote device.
Send(client, (Upload)); //POST HTTP string + xml parameters
sendDone.WaitOne();
// Receive the response from the remote device.
Receive(client);
receiveDone.WaitOne();
// Write the response to the console.
//MessageBox.Show("Response received : " + response);
// Release the socket.
client.Shutdown(SocketShutdown.Both);
client.Close();
// Reset all manual events for next instance.
connectDone.Reset();
sendDone.Reset();
receiveDone.Reset();
}
catch (Exception)
{
//MessageBox.Show(e.ToString());
}
}
private void ConnectCallback(IAsyncResult ar)
{
try
{
// Retrieve the socket from the state object.
Socket client = (Socket)ar.AsyncState;
// Complete the connection.
client.EndConnect(ar);
//MessageBox.Show("Socket connected to " + client.RemoteEndPoint.ToString());
// Signal that the connection has been made.
connectDone.Set();
}
catch (Exception)
{
//MessageBox.Show(e.ToString());
}
}
private void Receive(Socket client)
{
try
{
// Create the state object.
StateObject state = new StateObject();
state.workSocket = client;
// Begin receiving the data from the remote device.
client.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReceiveCallback), state);
}
catch (Exception)
{
//MessageBox.Show(e.ToString());
}
}
private void ReceiveCallback(IAsyncResult ar)
{
try
{
// Retrieve the state object and the client socket
// from the asynchronous state object.
StateObject state = (StateObject)ar.AsyncState;
Socket client = state.workSocket;
// Read data from the remote device.
int bytesRead = client.EndReceive(ar);
if (bytesRead > 0)
{
// There might be more data, so store the data received so far.
state.sb.Append(Encoding.ASCII.GetString(state.buffer, 0, bytesRead));
// Get the rest of the data.
client.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReceiveCallback), state);
}
else
{
// All the data has arrived; put it in response.
if (state.sb.Length > 1)
{
response1 = state.sb.ToString();
///
///
/// **THIS IS WHERE I WANT ALL THE PROCESSING TO BE DONE**
/// **AFTER THE RESPONSE IS COMPLETE!!**
///
///
}
// Signal that all bytes have been received.
receiveDone.Set();
}
}
catch (Exception)
{
//MessageBox.Show(e.ToString());
}
}
private void Send(Socket client, String data)
{
// Convert the string data to byte data using ASCII encoding.
byte[] byteData = Encoding.ASCII.GetBytes(data);
// Begin sending the data to the remote device.
client.BeginSend(byteData, 0, byteData.Length, 0,
new AsyncCallback(SendCallback), client);
}
private void SendCallback(IAsyncResult ar)
{
try
{
// Retrieve the socket from the state object.
Socket client = (Socket)ar.AsyncState;
// Complete sending the data to the remote device.
int bytesSent = client.EndSend(ar);
//MessageBox.Show("Sent " + bytesSent + " bytes to server.");
// Signal that all bytes have been sent.
sendDone.Set();
}
catch (Exception)
{
//MessageBox.Show(e.ToString());
}
}
How am I able to get the relative control from my foreach statement passed into the method and then amend my GUI (invoking somehow?) based on the outcome? Also, is it even possible to make each client independent, and have multiple client to server to client requests simultaneously? That is my main objective.
If this is a far fetched or very discouraged way of attempting this, please say so. Without diving into too much programming jargon (for understanding purposes, I'm fairly new to programming), how would you go about doing this?
Thanks in advance!
Well, you can sort of manipulate any control from a different thread by using the BeginInvoke function on the control, passing the action you want to execute. The action will be executed on the UI thread.
But your main problem here is that you're failing to separate the concerns. You UI code can for sure make actions happen, but these actions should be distinct from any UI code. You should design your actions in such a way to make them reusable. In other words, if you decide to rewrite your UI from scratch, you should still be able to reuse your actions as-is.
To make this possible, your actions should not reference any UI, they even should not be aware of the existence of any UI. This will make your code more manageable. So extract all that stuff in a different class, and then use something like events for instance to communicate back with the UI. The UI code then would make the BeginInvoke call.
I implemented a client - server application uses TCP protocol.
The client side is implemented in c++, cannot be changed. I am writing server side in c#.
(In this case the client sends to the server the data and the server handles the data)
The communication flows well in most of cases.
In case of receiving a large amount of data, I have the following problem: The window size is full and I get only partial data. I follow the communication by WireShark, which mark the last received package as [TCP Window full] and the next receive as [TCP zero window].
I understand the issue here is the limitation of the stream size.
Trying to increase my Socket.ReceiveBufferSize solve the problem, but I do NOT want to use it, since the amount of data is very dynamic and I have no way to know it before.
I am looking for a way to EMPTY the socket from the received data which I have already read.
Does anyone know a way to do it?
Please see my code below:
public static void AcceptCallBack(IAsyncResult asyncResult)
{
allDone.Set();
Socket listener = (Socket)asyncResult.AsyncState;
Socket handler = listener.EndAccept(asyncResult);
StateObject state = new StateObject();
state.workSocket = handler;
handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, SocketFlags.None, new AsyncCallback(ReadCallback), state);
}
public static void ReadCallback(IAsyncResult asyncResult)
{
string content = String.Empty;
StateObject state = (StateObject)asyncResult.AsyncState;
Socket handler = state.workSocket;
if (handler.Connected)
{
int bytesRead = handler.EndReceive(asyncResult);
//Is there a way delete here the read bytes from socket???
if (bytesRead > 0)
{
state.sb.Append(Encoding.ASCII.GetString(state.buffer, 0, bytesRead));
content = state.sb.ToString();
XmlDocument doc = new XmlDocument();
if (/*end of expected data*/)
{
}
else
handler.BeginReceive(state.buffer, 0, StateObject.BufferSize, SocketFlags.None, new AsyncCallback(ReadCallback), state);
}
else
{
handler.Shutdown(SocketShutdown.Both);
handler.Close();
allDone.Reset();
Console.WriteLine("waiting for re-connection...");
listener.BeginAccept(new AsyncCallback(AcceptCallBack), listener);
}
}
else
{
handler.Shutdown(SocketShutdown.Both);
handler.Close();
}
}
EDIT:
A more concise explanation of what I was trying to do here and with answers
I'm using c# asynchronous sockets to receive data from a source.
My problem is how and where to store received data if there are more to be received?
When a string is received, I can just use a string builder to receive and store just like this from msdn:
private void ReceiveCallback_onQuery(IAsyncResult ar)
{
try
{
// Retrieve the state object and the client socket
// from the asynchronous state object.
StateObject state = (StateObject)ar.AsyncState;
Socket client = state.workSocket;
// Read data from the remote device.
int bytesRead = client.EndReceive(ar);
if (bytesRead > 0)
{
// There might be more data, so store the data received so far.
dataReceived += state.buffer; //Does not work (dataReceived is byte[] type)
// Get the rest of the data.
client.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReceiveCallback_onQuery), state);
}
else
{
// All the data has arrived; put it in response.
if (dataReceived > 1)
{
response_onQueryHistory = ByteArrayToObject(dataReceived)
}
// Signal that all bytes have been received.
receiveDoneQuery.Set();
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}
I dont want to convert my received data into a string as in my case, I am receiving a complex object.
The data sent was serialized and I'm also fine with deserializing.
My question how to 'continually' receive data from the socket without using the string builder to store it.
Thanks!
This depends on how the complex thing is serialized prior to being pushed down the wire in the broken down bytes, you will receive those bytes and use the same algorithm/technique used to serialize the thing to deserialize it back to its original state.
For a more specific answer I'd ask that you be more specific yourself.
My problem is how and where to store received data if there are more to be received?
Can use Buffer.BlockCopy and queue it up, for eg,
int rbytes = client.EndReceive(ar);
if (rbytes > state.buffer)
{
byte[] bytesReceived = new byte[rbytes];
Buffer.BlockCopy(state.buffer, 0, bytesReceived, 0, rbytes);
state.myQueue.Enqueue(bytesReceived);
client.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReceiveCallback_onQuery), state)
}