While trying to read data from a TCP socket using ReceiveAsync.Completed, I experience a few ms delay (1-5ms, once in a while) between the time the data arrives to my application and the time it arrives to the machine. I use Wireshark to compare the timestamps, PSH bit is turned on for those messages as well and the application is not busy or blocking anything. What am I missing ?
Thanks.
With such a small delay it could come from anywhere, the garbage collector kicking in could delay you that much easily.
Wireshark isn't operating on the same level as your application. It could be that it's getting the data at different times than you just because the system is supplying it a little faster. Or maybe their times are off. Maybe your times are off. With such a small difference its hard to tell.
I bet you're not doing anything wrong.
Related
I've got quite an abstract question. I'm working on a project that requires constant device communication. I'm integrating multiple devices onto an external processing unit with a touchpanel to execute certain methods. I.e. the "start videocall" button on the touchpanel activates a relay, turns a display-device, camera-device and microphone-device on, etc.
On the flipside, I'm also trying to monitor these devices. What status do they currently have? Are they enabled/disabled ? What input is the display device currently on?
So far, I've come up with two solutions to prevent a bottleneck in the communication where I'm constantly polling (i.e. every two to five seconds to keep an acurate and up-to-date status) the on-state and input-state of the display-device.
Make use of threading so I can enqueue the different commands and execute them async. By also reading the response async, all communication should be nicely spaced out but I'd have a very "busy" communication line, taking it's toll on the processing unit.
With the help of events have the display-device notify the processor of it's changed status. This would take a lot of stress off of the communication line, but I feel like this is very easily disrupted. If the device doesn't throw it's events correctly (or the events are missed out on) the monitored state does not correspond with the actual state.
I'm curious if there are other ways of going about this issue. As of now, I'm leaning towards the second one because it stresses the processing unit a whole lot less, I just feel like I should be building in a lot of safeguards to prevent an inacurate representation of the actual device-states.
The project runs in C# on .Net 3.5.
Polling works, but it isn't fun or optimal. Reactive is best but as you've mentioned there may be a hiccup insuring your still listening to to the device and not just standing by for nothing. In this situation it makes since to optimize both processes. Poll when you're waiting or haven't heard a response in so long and listen when your polling returns good info, passing the polling.
That said, you shouldn't worry about taxing the unit too much with polling on various threads. This sounds like a purpose device so as long as you're not running it hot or stressing it to max all the time then using your resources are perfectly fine.
I'm working on a Unity game that receives OSC messages from the Muse EEG headset. I've tried two 3rd party C# libraries to handle the OSC communication, UnityOSC and unity-OSC-receiver. Both implement the OSC communication with an underlying System.Net.Sockets.UdpClient. Everything is running smoothly on Windows, but on OSX, after a while, I just stop receiving messages every time. No exceptions or error messages, no indication of what went wrong at all, just silence.
My application roughly works as follows:
Start a thread that spawns a process that runs Muse-IO. This makes the headset start sending messages. After starting the process, this thread is just chilling on process.WaitforExit()
Another thread runs a while loop - not in MonoBehavior.Update(), that's not fast enough - that keeps receiving and processing OSC messages. In both libraries, this essentially boils down to calling UdpClient.Receive()
Game uses the processed messages in the normal Unity update cycle.
Some 120 to 140 seconds after the connection is initialized, the stream of messages just stops, and so far I haven't been able to figure out why. The connection indicator light on the headset stays on, but nothing indicates it's actually still sending data.
Things I've ruled out:
It's not because the number of messages or the size of the messages. If I modify the command to the headset to only send some categories of messages, cutting the total in half (from about 600/s to 300/s), the timeout still happens at the same time.
It's not the OSC library. I get the exact same results with both OSC libraries.
It's not the firewall. The firewall is off.
It's probably not the port being used by something else. I tried different ports with the same result.
It doesn't appear to be Muse's OSX driver. When I use their GUI to visualize the incoming data, it keeps receiving data for as long as I want.
I suspect that Mono, Unity or OSX might be shutting down (garbage-collecting?) the Muse-IO process or thread, because the time before the problem occurs seems to be pretty much constant regardless of what I try. But I'm unsure how to further diagnose, let alone fix this now. Any clues, suggestions or amazing solutions would be most welcome.
I found the cause.
After spawning the I/O process, the thread would do
print("Process started!");
process.PriorityClass = ProcessPriorityClass.High;
process.WaitforExit();
In hindsight, that print statement is really poorly placed, oh well. It worked fine on Windows. Changing process priority only requires admin privilege if you're increasing it to Realtime, according to the docs. Not so on Mac though. apparently setting it to High also requires elevated rights on OSX. The resulting exception was silent/undetected/uncaught because it happens outside the main thread.
Then, several minutes later, it seems the thread is garbage collected, including its child process, even though that's still running. That delay really threw me off, making me look for the cause in all the wrong places.
Lessons learned:
Be more careful with possible exceptions when multithreading,
Don't mess with process priority if you don't absolutely have to,
And never trust the docs.
I have written my network code for my app using Sockets. I've tested the library on a windows PC and it is very fast. Whether it's many small "packets" (by packets I mean send operations, I am using TCP which is streaming) of data in a small amount of time, or a few large ones, it works perfectly.
I moved the code into a test app for the iPhone. Ran the test, great speeds again. about 5 MB sent over wifi between two phones in about 3 seconds.
Im using synchronous Socket.Send() operations in a threadpool thread, and using ReceiveAsync() for receiving. (I've also tried the BeginReceive() style, but it behaves the same.
However, once I move the code into my app, I start to encounter problems. The general problem is that the receiving app doesnt seem to receive consistently. I could send several "packets" of data from the host phone, and it can be anywhere from instantly, to a few seconds to 10 minutes before they appear on the receiving end. I've been on this problem for 2 weeks now (evenings/weekends) and i've spent days testing it to try understand exactly what I can do to reproduce it, but its never the same twice.
At the moment, im putting it down to threadpool threads being exhausted. I've used
ThreadPool.SetMaxThreads()
to increase the thread count drastically, but it doesnt make any difference. Its like as if the completed callback in SocketAsyncEventArgs cannot get a thread to operate on, so it just sits there. I've gone through my code and refactored anything that was un-necessarily using threads with a loop performing periodic tasks and changed them to timers, but the problem remains.
I have literally no idea where to turn with this one. Im hoping its maybe a bug in Monotouch (not that Im trying to blame those guys!).
Im not sure what code to post, as the network code has been tested on its own and operates fine. I've tested it with 1,000,000 send/receives to check if there was some kind of leak but found no problems.
It seems like the data is getting to the recipient, but the callback is somehow getting severely delayed in getting called, sometimes by several minutes.
Can anyone point me in a direction of why this might be happening?
Thank you.
My problem with this was caused by having a GKSession also initialized. I hope this is a bug in Monotouch/Mono that can be fixed, as I do need both network features enabled. As soon as i disabled the GKSession, the socket code flows freely.
I'm trying to create a website similar to BidCactus and LanceLivre.
The specific part I'm having trouble with is the seconds aspect of the timer.
When an auction starts, a timer of 15 seconds starts counting down, and every time a person bids, the timer is reset and the price of the item is increased by 0,01$.
I've tried using SignalR for this bit, and while it does work well during trials runs in the office, it's just not good enough for real world usage where seconds count. I would get HTTP 503 errors when too many users were bidding and idling on the site.
How can I make the timer on the clients end shows the correct remaining time?
Would HTTP GETting that information with AJAX every second allow me to properly display the missing time? That's a request each second!
And not only that, but when a user requests that GET, I calculate remaining seconds, but until the user see's that response, that time is no longer useful as a second or more might pass between processing and returning. Do you see my conundrum?
Any suggestions on how to approach this problem?
There are a couple problems with the solution you described:
It is extremely wasteful. There is already a fairly high accuracy clock built into every computer on the Internet.
The Internet always has latency. By the time the packet reaches the client, it will be old.
The Internet is a variable-latency network, so the time update packets you get could be as high or higher than one second behind for one packet, and as low as 20ms behind for another packet.
It takes complicated algorithms to deal with #2 and #3.
If you actually need second-level accuracy
There is existing Internet-standard software that solves it - the Network Time Protocol.
Use a real NTP client (not the one built into Windows - it only guarantees it will be accurate to within a couple seconds) to synchronize your server with national standard NTP servers, and build a real NTP client into your application. Sync the time on your server regularly, and sync the time on the client regularly (possibly each time they log in/connect? Maybe every hour?). Then simply use the system clock for time calculations.
Don't try to sync the client's system time - they may not have access to do so, and certainly not from the browser. Instead, you can get a reference time relative to the system time, and simply add the difference as an offset on client-side calculations.
If you don't actually need second-level accuracy
You might not really need to guarantee accuracy to within a second.
If you make this decision, you can simplify things a bit. Simply transmit a relative finish time to the client for each auction, rather than an absolute time. Re-request it on the client side every so often (e.g. every minute). Their global system time may be out of sync, but the second-hand on their clock should pretty accurately tick down seconds.
If you want to make this a little more slick, you could try to determine the (relative) latency for each call to the server. Keep track of how much time has passed between calls to the server, and the time-left value from the previous call. Compare them. Then, calculate whichever is smaller, and base your new time off that calculation.
I'd be careful when engineering such a solution, though. If you get the calculations wrong, or are dealing with inaccurate system clocks, you could break your whole syncing model, or unintentionally cause the client to prefer the higest latency call. Make sure you account for all cases if you write the "slick" version of this code :)
One way to get really good real-time communication is to open a connection from the browser to a special tcp/ip socket server that you write on the server. This is how a lot of chat packages on the web work.
Duplex sockets allow you to push data both directions. Because the connection is already open, you can send quite a bit of very fast data across.
In the past, you needed to use Adobe Flash to accomplish this. I'm not sure if browsers have advanced enough to handle this without a plugin (eg, websockets?)
Another approach worth looking at is long polling. In concept, a connection is made to the server that just doesn't die, and it gives you the opportunity on the server to trickle bits of realtime data down to the clients.
Just some pointers. I have written web software using JavaScript <-> Flash <-> Python/PHP, and was please with how it worked.
Good luck.
I know this is probably the canonical "It depends..." question but I'd appreciate any pointers as to where to start looking.
I have a client/server app talking over ethernet. In one computer I run the server and a client and on another just the client. One runs Vista and one runs XP. After an uptime of about 3 weeks the entire computer freezes and nothing works, not mouse, not keyboard, nothing -just power off. Every ten seconds the server sends a ping message to see if the clients are alive, other than that just a few small messages go back and forth every day.
I'm trying to find out if it's me causing it or something else. I've started a session and after a few days I thought I'd check for strange increases in memory use but beyond that I have very few ideas.
Some thoughts to consider:
You know the computer doesn't respond, but that doesn't mean it's hung. Does it respond to a ping?
Maybe the disk activity light is on all the time?
You say "no keyboard" - do you mean no caps lock or num lock lights?
Although the .NET application may be the only one you're running at the time, that does not imply it is the cause of the problem. Some background job could be doing it.
For example, I notice that Retrospect backup, when it is creating a snapshot, freezes the entire system for 10-15 minutes. I mean, no caps lock, the clock in the task bar doesn't update, no CTRL-ALT-DEL, can't type into an "Answer" text box in SO, nothing. It had nothing to do with what I was doing at the time, which was answering a question on SO.
After it came back, SO asked if I was a human. My feelings were hurt. ;-)
You could attach a kernel debugger to the OS. That way you should be able to inspect the state of the OS and your process even if the OS is completely unresponsive. (Unfortunately, it's a lot harder than just hitting "break" in VS. I suggest reading John Robbin's "Debugging Applications for .NET and Windows" before trying that.)
You could also try to create memory dumps of your application in regular intervals. You might have to do a little scripting for that, though. (usually, you'd create a dump with a keystroke, using a tool like userdump or adplus, but if the OS is not responding to keystrokes, that won't work.) That way, you know what state your process is in during or shortly before a hang.
This page: http://blogs.msdn.com/debuggingtoolbox/default.aspx is a good starting point for scripting WinDbg. (If you don't know what to do with a memory dump, I'd again suggest John Robbin's excellent book on debugging!)
Other than that, I can only think of standard debugging tricks: does the problem occur on every PC? Does it happen if there are no client requests? Does it happen sooner if there are more client requests? Does it happen sooner if there is less available physical memory? Try removing parts of your application (maybe on a separate server for testing) and see if the problem still occurs, and so on. Try running it in a VM so you can see if it uses the CPU, harddisk, or network during those "hangs".
This isn't going to be the answer, but I'd advise starting by checking your OS event logs and running a perfmon to keep track of memory, cpu usage etc.
Which computer freezes, the server or client? And what OSes are they running respectively?
As Daniel L noted, tight polling loops can really kill the CPU. If you can, change your code to use event handlers, it's a much more robust solution.
Finally, are you certain there's not a hardware problem on the freezing computer?