I've received some C# code from a colleague for interacting with a cRIO device connected over ethernet. I'm trying to improve the code quality to make it a bit more comprehensible for future users, however I'm struggling a little bit to extract some relevant information from the API documentation. My main question is whether there would be problems caused in leaving a NetworkVariableManager in the Connected state?
Right now the code uses a class which looks something like
public class RIOVar<T>
{
public readonly string location;
public RIOVar(string location)
{
this.location = location;
}
public T Get()
{
using(NetworkVariableReader<T> reader = new NetworkVariableReader<T>(location) )
{
reader.Connect();
return reader.ReadData().GetValue()
}
}
public void Write(T value)
{
using(NetworkVariableWriter<T> writer = new NetworkVariableWriter<T>(location) )
{
writer.Connect();
writer.WriteValue(value);
}
}
}
The actual class does a lot more than this, but the part that actually communicates with the cRIO basically boils down to these two methods and the location data member.
What I'm wondering about is whether it would be better to instead have reader and writer as class members and Connect them in the constructor (at the point that they are constructed the connection should be posible), but what I don't know is if this would have some adverse effect on the way the computer and RIO communicate with each other (maybe a connected manager uses some resource or the program must maintain some sort of register...?) and therefore the approach here of having the manager connected only for the read/write operation is better design.
Keeping a variable connected keeps its backing resources in memory:
threads
sockets
data buffers
These resources are listed in the online help, but it's unclear to me if that list is complete:
NationalInstruments.NetworkVariable uses multiple threads to implement the reading and writing infrastructure. When reading or writing in a tight loop insert a Sleep call, passing 0, to allow a context switch to occur thereby giving the network variable threads time to execute.
... snip ...
NationalInstruments.NetworkVariable shares resources such as sockets and data buffers among connections that refer to the same network variable in the same program.
In my opinion, I'd expect better runtime performance by connecting/disconnecting as infrequently as possible. For example, when the network is reachable, connect; when it isn't, disconnect.
Related
I have a project which is a Web API project, my project is accessed by multiple users (i mean a really-really lot of users). When my project being accessed from frontend (web page using HTML 5), and user doing something like updating or retrieving data, the backend app (web API) will write a single log file (a .log file but the content is JSON).
The problem is, when being accessed by multiple users, the frontend became unresponsive (always loading). The problem is in writing process of the log file (single log file being accessed by a really-really lot of users). I heard that using a multi threading technique can solve the problem, but i don't know which method. So, maybe anyone can help me please.
Here is my code (sorry if typo, i use my smartphone and mobile version of stack overflow):
public static void JsonInputLogging<T>(T m, string methodName)
{
MemoryStream ms = new MemoryStream();
DataContractJsonSerializer ser = new
DataContractJsonSerializer(typeof(T));
ser.WriteObject(ms, m);
string jsonString = Encoding.UTF8.GetString(ms.ToArray());
ms.Close();
logging("MethodName: " + methodName + Environment.NewLine + jsonString.ToString());
}
public static void logging (string message)
{
string pathLogFile = "D:\jsoninput.log";
FileInfo jsonInputFile = new FileInfo(pathLogFile);
if (File.Exists(jsonInputFile.ToString()))
{
long fileLength = jsonInputFile.Length;
if (fileLength > 1000000)
{
File.Move(pathLogFile, pathLogFile.Replace(*some new path*);
}
}
File.AppendAllText(pathLogFile, *some text*);
}
You have to understand some internals here first. For each [x] users, ASP.Net will use a single worker process. One worker process holds multiple threads. If you're using multiple instances on the cloud, it's even worse because then you also have multiple server instances (I assume this ain't the case).
A few problems here:
You have multiple users and therefore multiple threads.
Multiple threads can deadlock each other writing the files.
You have multiple appdomains and therefore multiple processes.
Multiple processes can lock out each other
Opening and locking files
File.Open has a few flags for locking. You can basically lock files exclusively per process, which is a good idea in this case. A two-step approach with Exists and Open won't help, because in between another worker process might do something. Bascially the idea is to call Open with write-exclusive access and if it fails, try again with another filename.
This basically solves the issue with multiple processes.
Writing from multiple threads
File access is single threaded. Instead of writing your stuff to a file, you might want to use a separate thread to do the file access, and multiple threads that tell the thing to write.
If you have more log requests than you can handle, you're in the wrong zone either way. In that case, the best way to handle it for logging IMO is to simply drop the data. In other words, make the logger somewhat lossy to make life better for your users. You can use the queue for that as well.
I usually use a ConcurrentQueue for this and a separate thread that works away all the logged data.
This is basically how to do this:
// Starts the worker thread that gets rid of the queue:
internal void Start()
{
loggingWorker = new Thread(LogHandler)
{
Name = "Logging worker thread",
IsBackground = true,
Priority = ThreadPriority.BelowNormal
};
loggingWorker.Start();
}
We also need something to do the actual work and some variables that are shared:
private Thread loggingWorker = null;
private int loggingWorkerState = 0;
private ManualResetEventSlim waiter = new ManualResetEventSlim();
private ConcurrentQueue<Tuple<LogMessageHandler, string>> queue =
new ConcurrentQueue<Tuple<LogMessageHandler, string>>();
private void LogHandler(object o)
{
Interlocked.Exchange(ref loggingWorkerState, 1);
while (Interlocked.CompareExchange(ref loggingWorkerState, 1, 1) == 1)
{
waiter.Wait(TimeSpan.FromSeconds(10.0));
waiter.Reset();
Tuple<LogMessageHandler, string> item;
while (queue.TryDequeue(out item))
{
writeToFile(item.Item1, item.Item2);
}
}
}
Basically this code enables you to work away all the items from a single thread using a queue that's shared across threads. Note that ConcurrentQueue doesn't use locks for TryDequeue, so clients won't feel any pain because of this.
Last thing that's needed is to add stuff to the queue. That's the easy part:
public void Add(LogMessageHandler l, string msg)
{
if (queue.Count < MaxLogQueueSize)
{
queue.Enqueue(new Tuple<LogMessageHandler, string>(l, msg));
waiter.Set();
}
}
This code will be called from multiple threads. It's not 100% correct because Count and Enqueue don't necessarily have to be called in a consistent way - but for our intents and purposes it's good enough. It also doesn't lock in the Enqueue and the waiter will ensure that the stuff is removed by the other thread.
Wrap all this in a singleton pattern, add some more logic to it, and your problem should be solved.
That can be problematic, since every client request handled by new thread by default anyway. You need some "root" object that is known across the project (don't think you can achieve this in static class), so you can lock on it before you access the log file. However, note that it will basically serialize the requests, and probably will have a very bad effect on performance.
No multi-threading does not solve your problem. How are multiple threads supposed to write to the same file at the same time? You would need to care about data consistency and I don't think that's the actual problem here.
What you search is asynchronous programming. The reason your GUI becomes unresponsive is, that it waits for the tasks to complete. If you know, the logger is your bottleneck then use async to your advantage. Fire the log method and forget about the outcome, just write the file.
Actually I don't really think your logger is the problem. Are you sure there is no other logic which blocks you?
I want to extend my experience with the .NET framework and want to build a client/server application.
Actually, the client/server is a small Point Of Sale system but first, I want to focus on the communication between server and client.
In the future, I want to make it a WPF application but for now, I simply started with a console application.
2 functionalities:
client(s) receive(s) a dataset and every 15/30min an update with changed prices/new products
(So the code will be in a Async method with a Thread.sleep for 15/30 mins).
when closing the client application, sending a kind of a report (for example, an xml)
On the internet, I found lots of examples but i can't decide which one is the best/safest/performanced manner of working so i need some advice for which techniques i should implement.
CLIENT/SERVER
I want 1 server application that handles max 6 clients. I read that threads use a lot of mb and maybe a better way will be tasks with async/await functionallity.
Example with ASYNC/AWAIT
http://bsmadhu.wordpress.com/2012/09/29/simplify-asynchronous-programming-with-c-5-asyncawait/
Example with THREADS
mikeadev.net/2012/07/multi-threaded-tcp-server-in-csharp/
Example with SOCKETS
codereview.stackexchange.com/questions/5306/tcp-socket-server
This seems to be a great example of sockets, however, the revisioned code isn't working completely because not all the classes are included
msdn.microsoft.com/en-us/library/fx6588te(v=vs.110).aspx
This example of MSDN has a lot more with Buffersize and a signal for the end of a message. I don't know if this just an "old way" to do this because in my previous examples, they just send a string from the client to the server and that's it.
.NET FRAMEWORK REMOTING/ WCF
I found also something about the remoting part of .NET and WCF but don' know if I need to implement this because i think the example with Async/Await isn't bad.
SERIALIZED OBJECTS / DATASET / XML
What is the best way to send data between it? Juse an XML serializer or just binary?
Example with Dataset -> XML
stackoverflow.com/questions/8384014/convert-dataset-to-xml
Example with Remoting
akadia.com/services/dotnet_dataset_remoting.html
If I should use the Async/Await method, is it right to something like this in the serverapplication:
while(true)
{
string input = Console.ReadLine();
if(input == "products")
SendProductToClients(port);
if(input == "rapport")
{
string Example = Console.ReadLine();
}
}
Here are several things anyone writing a client/server application should consider:
Application layer packets may span multiple TCP packets.
Multiple application layer packets may be contained within a single TCP packet.
Encryption.
Authentication.
Lost and unresponsive clients.
Data serialization format.
Thread based or asynchronous socket readers.
Retrieving packets properly requires a wrapper protocol around your data. The protocol can be very simple. For example, it may be as simple as an integer that specifies the payload length. The snippet I have provided below was taken directly from the open source client/server application framework project DotNetOpenServer available on GitHub. Note this code is used by both the client and the server:
private byte[] buffer = new byte[8192];
private int payloadLength;
private int payloadPosition;
private MemoryStream packet = new MemoryStream();
private PacketReadTypes readState;
private Stream stream;
private void ReadCallback(IAsyncResult ar)
{
try
{
int available = stream.EndRead(ar);
int position = 0;
while (available > 0)
{
int lengthToRead;
if (readState == PacketReadTypes.Header)
{
lengthToRead = (int)packet.Position + available >= SessionLayerProtocol.HEADER_LENGTH ?
SessionLayerProtocol.HEADER_LENGTH - (int)packet.Position :
available;
packet.Write(buffer, position, lengthToRead);
position += lengthToRead;
available -= lengthToRead;
if (packet.Position >= SessionLayerProtocol.HEADER_LENGTH)
readState = PacketReadTypes.HeaderComplete;
}
if (readState == PacketReadTypes.HeaderComplete)
{
packet.Seek(0, SeekOrigin.Begin);
BinaryReader br = new BinaryReader(packet, Encoding.UTF8);
ushort protocolId = br.ReadUInt16();
if (protocolId != SessionLayerProtocol.PROTOCAL_IDENTIFIER)
throw new Exception(ErrorTypes.INVALID_PROTOCOL);
payloadLength = br.ReadInt32();
readState = PacketReadTypes.Payload;
}
if (readState == PacketReadTypes.Payload)
{
lengthToRead = available >= payloadLength - payloadPosition ?
payloadLength - payloadPosition :
available;
packet.Write(buffer, position, lengthToRead);
position += lengthToRead;
available -= lengthToRead;
payloadPosition += lengthToRead;
if (packet.Position >= SessionLayerProtocol.HEADER_LENGTH + payloadLength)
{
if (Logger.LogPackets)
Log(Level.Debug, "RECV: " + ToHexString(packet.ToArray(), 0, (int)packet.Length));
MemoryStream handlerMS = new MemoryStream(packet.ToArray());
handlerMS.Seek(SessionLayerProtocol.HEADER_LENGTH, SeekOrigin.Begin);
BinaryReader br = new BinaryReader(handlerMS, Encoding.UTF8);
if (!ThreadPool.QueueUserWorkItem(OnPacketReceivedThreadPoolCallback, br))
throw new Exception(ErrorTypes.NO_MORE_THREADS_AVAILABLE);
Reset();
}
}
}
stream.BeginRead(buffer, 0, buffer.Length, new AsyncCallback(ReadCallback), null);
}
catch (ObjectDisposedException)
{
Close();
}
catch (Exception ex)
{
ConnectionLost(ex);
}
}
private void Reset()
{
readState = PacketReadTypes.Header;
packet = new MemoryStream();
payloadLength = 0;
payloadPosition = 0;
}
If you're transmitting point of sale information, it should be encrypted. I suggest TLS which is easily enabled on through .Net. The code is very simple and there are quite a few samples out there so for brevity I'm not going to show it here. If you are interested, you can find an example implementation in DotNetOpenServer.
All connections should be authenticated. There are many ways to accomplish this. I've use Windows Authentication (NTLM) as well as Basic. Although NTLM is powerful as well as automatic it is limited to specific platforms. Basic authentication simply passes a username and password after the socket has been encrypted. Basic authentication can still, however; authenticate the username/password combination against the local server or domain controller essentially impersonating NTLM. The latter method enables developers to easily create non-Windows client applications that run on iOS, Mac, Unix/Linux flavors as well as Java platforms (although some Java implementations support NTLM). Your server implementation should never allow application data to be transferred until after the session has been authenticated.
There are only a few things we can count on: taxes, networks failing and client applications hanging. It's just the nature of things. Your server should implement a method to clean up both lost and hung client sessions. I've accomplished this in many client/server frameworks through a keep-alive (AKA heartbeat) protocol. On the server side I implement a timer that is reset every time a client sends a packet, any packet. If the server doesn't receive a packet within the timeout, the session is closed. The keep-alive protocol is used to send packets when other application layer protocols are idle. Since your application only sends XML once every 15 minutes sending a keep-alive packet once a minute would able the server side to issue an alert to the administrator when a connection is lost prior to the 15 minute interval possibly enabling the IT department to resolve a network issue in a more timely fashion.
Next, data format. In your case XML is great. XML enables you to change up the payload however you want whenever you want. If you really need speed, then binary will always trump the bloated nature of string represented data.
Finally, as #NSFW already stated, threads or asynchronous doesn't really matter in your case. I've written servers that scale to 10000 connections based on threads as well as asynchronous callbacks. It's all really the same thing when it comes down to it. As #NSFW said, most of us are using asynchronous callbacks now and the latest server implementation I've written follows that model as well.
Threads are not terribly expensive, considering the amount of RAM available on modern systems, so I don't think it's helpful to optimize for a low thread count. Especially if we're talking about a difference between 1 thread and 2-5 threads. (With hundreds or thousands of threads, the cost of a thread starts to matter.)
But you do want to optimize for minimal blocking of whatever threads you do have. So for example instead of using Thread.Sleep to do work on 15 minute intervals, just set a timer, let the thread return, and trust the system to invoke your code 15 minutes later. And instead of blocking operations for reading or writing information over the network, use non-blocking operations.
The async/await pattern is the new hotness for asynchronous programming on .Net, and it is a big improvement over the Begin/End pattern that dates back to .Net 1.0. Code written with async/await is still using threads, it is just using features of C# and .Net to hide a lot of the complexity of threads from you - and for the most part, it hides the stuff that should be hidden, so that you can focus your attention on your application's features rather than the details of multi-threaded programming.
So my advice is to use the async/await approach for all of your IO (network and disk) and use timers for periodic chores like sending those updates you mentioned.
And about serialization...
One of the biggest advantages of XML over binary formats is that you can save your XML transmissions to disk and open them up using readily-available tools to confirm that the payload really contains the data that you thought would be in there. So I tend to avoid binary formats unless bandwidth is scarce - and even then, it's useful to develop most of the app using a text-friendly format like XML, and then switch to binary after the basic mechanism of sending and receiving data have been fleshed out.
So my vote is for XML.
And regarding your code example, well ther's no async/await in it...
But first, note that a typical simple TCP server will have a small loop that listens for incoming connections and starts a thread to hanadle each new connection. The code for the connection thread will then listen for incoming data, process it, and send an appropriate response. So the listen-for-new-connections code and the handle-a-single-connection code are completely separate.
So anyway, the connection thread code might look similar to what you wrote, but instead of just calling ReadLine you'd do something like "string line = await ReadLine();" The await keyword is approximately where your code lets one thread exit (after invoking ReadLine) and then resumes on another thread (when the result of ReadLine is available). Except that awaitable methods should have a name that ends with Async, for example ReadLineAsync. Reading a line of text from the network is not a bad idea, but you'll have to write ReadLineAsync yourself, building upon the existing network API.
I hope this helps.
I was trying to develop a multicast receiver program and socket initialization was done as shown below:
public void initializeThread()
{
statuscheckthread = new Thread(SetSocketOptions);
statuscheckthread.IsBackground = true;
}
private void Form1_Load(object sender, EventArgs e)
{
rxsock = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
iep = new IPEndPoint(IPAddress.Any, 9191);
rxsock.Bind(iep);
ep = (EndPoint)iep;
initializeThread();
statuscheckthread.Start();
}
public void SetSocketOptions()
{
initializeThread(); //re-initializes thread thus making it not alive
while (true)
{
if (NetworkInterface.GetIsNetworkAvailable())
{
bool sockOptnSet = false;
while (!sockOptnSet)
{
try
{
rxsock.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.AddMembership, new MulticastOption(IPAddress.Parse("224.50.50.50")));
rxsock.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.MulticastTimeToLive, 64);
sockOptnSet = true;
}
catch
{
//Catch exception here
}
}
}
break; // Break out from loop once socket options are set
}
}
When my PC is not connected to a network, SetSocketOption method was throwing exception and even after network is connected,
I was unable to receive data because socket options are not set.
To avoid this I used a thread which runs in the background checking
for network availability and once network is available, it sets the socket options.
It works properly in some PC's but in some others, NetworkInterface.GetIsNetworkAvailable()
returned true before network got connected
(while network was being identified) .
So, to make sure Socket options are set, I used a bool variable sockOptnSet
which is set as
true if all the statements in the try block is executed as shown inside the method public void SetSocketOptions()
This program works fine in all PC's I tried, but I am doubtful about how much I can rely on this to work.
My questions are:
1) Is this good practice?
2) If not, what are the possible errors or problems it may cause? And how can I implement it in a better way?
Is this a good practice?
No, not a good practice. The vast majority of exceptions, including your first one, fall in the category of vexing exceptions. Software is supposed to work, worked well when you tested it, but doesn't on the user's machine. Something went wrong but you do not know what and there isn't anything meaningful that you can do about it. Trying to keep your program going is not useful, it cannot do the job it is supposed to do. In your case, there's no hope that the socket is ever going to receive data when there is no network. And, as you found out, trying to work around the problem just begets more problems. That's normal.
If this is bad practice, how can I implement it in a better way?
You need help from a human. The user is going to have to setup the machine to provide a working network connection. This requires a user interface, you must have a way to tell a human what he needs to do to solve your problem. You can make that as intricate or as simple as you desire. Just an error message, a verbatim copy of the Exception.Message can be enough. Writing an event handler for the AppDomain.CurrentDomain.UnhandledException event is a very good (and required) strategy. Microsoft spent an enormous amount of effort to make exception messages as clear and helpful as possible, even localizing them for you in the user's native language, you want to take advantage of that. Even if the exception message is mystifying, a quick Google query on the message text returns hundreds of hits. With this event handler in place, you don't have to do anything special. Your program automatically terminates and your user knows what to do about it.
You can certainly make it more intricate, you discovered that SetSocketOption() is liable to fail right after the network becomes available but works when you wait long enough. So this is actually an error condition that you can work around, just by waiting long enough. Whether you should write the code to handle this is something that you have to decide for yourself. It is something you write when you have enough experience with the way your program behaves, you never write it up front. Usually as a result from feedback from the users of your program.
Some good advice in the comments, lets' expand on it.
Firstly, I would put all this socket code in to its' own class, outside of the form. This makes it its' own entity and semantically easier to understand. This class could have a property Initialised, which is initially set to false. The first thing you do in your form is call an Initialise method on this class which attempts to set socket options and catches the relevant exceptions if the network is not available. If it is available, we set our Initialised property to true.
If not available, we set a single timeout (see System.Threading.Timer) that calls this same function (potentially with a retry count) after 'x' seconds. Once again we'll find ourselves back in this Initialise function, perhaps with a retry count mentioned at the beginning. Once again, if it is available, we're good - if not, set the timer again. Eventually, after 'x' retries if we're not initialised we can throw an exception or set some other failure property to indicate that we can't proceed.
Your Form class can periodically check (or hook in to an event) to determine whether the socket is now ready for communication. In case of failure you can gracefully quit out, or because our class is nice and abstracted, attempt to start the whole process again.
In my c# application multiple clients will access the same server, to process one client ata a time below code is written.In the code i used Moniter class and also the queue class.will this code affect the performance.if i use Monitor class, then shall i remove queue class from the code.
Sometimes my remote server machine where my application running as service is totally down.is the below code is the reasond behind, coz all the clients go in a queue, when i check the netstatus -an command using command prompt, for 8 clients it shows 50 connections are holding in Time-wait...
Below is my code where client acces the server ...
if (Id == "")
{
System.Threading.Monitor.Enter(this);
try
{
if (Request.AcceptTypes == null)
{
queue.Enqueue(Request.QueryString["sessionid"].Value);
string que = "";
que = queue.Dequeue();
TypeController.session_id = que;
langStr = SessionDatabase.Language;
filter = new AllThingzFilter(SessionDatabase, parameters, langStr);
TypeController.session_id = "";
filter.Execute();
Request.Clear();
return filter.XML;
}
else
{
TypeController.session_id = "";
filter = new AllThingzFilter(SessionDatabase, parameters, langStr);
filter.Execute();
}
}
finally
{
System.Threading.Monitor.Exit(this);
}
}
Locking this is pretty wrong, it won't work at all if every thread uses a different instance of whatever class this code lives in. It isn't clear from the snippet if that's the case but fix that first. Create a separate object just to store the lock and make it static or give it the same scope as the shared object you are trying to protect (also not clear).
You might still have trouble since this sounds like a deadlock rather than a race. Deadlocks are pretty easy to troubleshoot with the debugger since the code got stuck and is not executing at all. Debug + Break All, then Debug + Windows + Threads. Locate the worker threads in the thread list. Double click one to select it and use Debug + Call Stack to see where it got stuck. Repeat for other threads. Look back through the stack trace to see where one of them acquired a lock and compare to other threads to see what lock they are blocking on.
That could still be tricky if the deadlock is intricate and involves multiple interleaved locks. In which case logging might help. Really hard to diagnose mandelbugs might require a rewrite that cuts back on the amount of threading.
i have a multi-threaded application which wants to send a sequence of data to an external device via a serial port. the sequence of data is a typical cmd - response protocol (ie: a given thread sends a sequence of bytes then waits to read a response which is typically an ack and then it might send another sequence).
what we are are looking to do is declare a sequence of code has exclusive access to this resource until it is done and if another thread wants access to the same external resouce, it waits.
this seems like what LOCK does, but all the examples that i have seen show lock being used to protect a specific block of code, not to serialize access to a resource.
so programatically can i have
Object serialPortLock = new Object();
and in different parts of my program use a construct that looks like:
Lock (serialPortLock)
{
// my turn to do something that is not the same as what
// someone else wants to do but it acts on the same resource
}
the c# documentation talks about using Mutex as a more robust version of Lock. is that whats required here?
Yes, your pattern is correct as long as your program is the only software accessing the serial port.
You have not posted your entire code. If the class that contains serialPortLock has multiple instances, then you MUST make serialPortLock a static. This is normally best practice.
class MySerialPort
{
static object synchLock = new object();
public void DoSomething()
{
lock (synchLock)
{
// whatever
}
}
}
Locking should work fine in the case you've suggested as long as you are locking around all access to any of the object instances that point at the external resource.