I'm writing an application in C# that allows people to track the amount of time they spend on tasks. It can be used by a single person to track their own personal time, but it will also be able to work in, for example, a company - like, if they want to track the amount of time spend on some project.
The data being stored by this program is pretty simple - a collection of all the tasks and each "block" of time that was spent on it (including date, start/stop time, and length of time spent).
For the multiuser functionality, my plan was to have a single server that the clients send updates to the tracked time. I don't think the clients will need a continuous connection as the updates would typically be pretty far apart.
Additionally, as both the server and the client will store a copy of the data, either of them can ask for a copy from the other if there's a data loss on either. Femaref has informed me that this is a poor idea, so I've removed it.
So, my question is, how should I approach this? I've seen some C# client/server tutorials, but those seem to be geared towards continuous connections.
Your best bet is to track the data separately. First Allow users to track there own time, and just store that in a local db (you can use something like csharp-sqlite ), then when the user connects sync what data you want to keep on server.
For data that you want to track sever side your just going to want the app to sign in and say its starting a task and then sign out when its stopping a task(then have the server side hit the db functions)(your going to want to keep the user data, and the server data separate, so you know what you can trust, and what implications there are for using what data ) .
Obviously, your going to want to handle situations where a task goes on longer then expected. For example someone forgets to say there done with the task(like there computer just crashes)(you can do this by having your app just say its still working on a task every so often).
The best way I have found to get around issues that are caused by trusting peoples input is to just tie into something like your local A.D or LDAP and allow management control(because in the end they are the ones that sort out any messes that come from people having the wrong hours) thats all handled server side. If you don't have A.D or LDAP, you might have to consider implementing some kind of RSA key mechanism for authentication and authority chains.
For talking to the server side process on the client, I suggest something like SOAP (SOAP using C#). That way you can move your server language to what ever makes your feel all warm and fuzzy.
This is a bit of a broad question so its hard to cover everything, but it should give you some leads in the right direction.
Related
This is more of a programming strategy and direction question, than the actual code itself.
I am programming in C-Sharp.
I have an application that remotely starts processes on many different clients on the network, could be up to 1000 clients in theory.
It then monitors the status of the remote processes by reading a log file on each client.
I currently do this by running one thread that loops through all of the clients in a list, and reading the log file. It works fine for 10 or 20 machines, but 1000 would probably be untenable.
There are several problems with this approach:
First, if the thread doesn’t finish reading all of the client statuses before it’s called again, the client statuses at the end of the list might not be read and updated.
Secondly, if any client in the list goes offline during this period, the updating hangs, until that client is back online again.
So I require a different approach, and have thought up a few possible ways to resolve this.
Spawn a separate thread for each client, to read their log file and update its progress.
a. However, I’m not sure if having 1000 threads running on my machine is something that would be acceptable.
Test the connect for each machine first, before trying to read the file, and if it cannot connect, then just ignore it for that iteration and move on to the next client in the list.
a. This still has the same problem of not getting through the list before the next call, and causes more delay and it tries to test the connection via a port first. With 1000 clients, this would be noticeable.
Have each client send the data to the machine running the application whenever there is an update.
a. This could create a lot of chatter with 1000 machines trying to send data repeatedly.
So I’m trying to figure if there is another more efficient and reliable method, that I haven’t considered, or which one of these would be the best.
Right now I’m leaning towards having the clients send updates to the application, instead of having the application pulling the data.
Looking for thoughts, concerns, ideas and recommendations.
In my opinion, you are doing this (Monitoring) the wrong way. Instead of keeping all logs in a text file, you'd better preserve them in a central data repository that can be of any kind. With respect to the fact that you are monitoring the performance of those system, your design and the mechanism behind it must not impact the performance of the target systems negatively, and with this design the disk and CPU would be involved so much in certain cases that can result in a performance issue itself.
I recommend you to create a log repository server using a fast in-memory database like Redis, and send logged data directly to that server. Keep in mind that this database must be running on a different virtual machine. You can then tune Redis to store received data on physical Disk once a particular number of indexes are reached or a particular interval elapses. The in-memory feature here is advantageous as you may need to query information a lot in a monitoring application like this. On the other hand, the performance of Redis is so high that it efficiently passes processing millions of indexes.
The blueprint for you is that:
1- Centralize all log data in a single repository.
2- Configure clients to send monitored information to the centralized repository.
3- Read the data from the centralized repository by the main server (monitoring system) when required.
I'm not trying to advertise for a particular tool here as I'm only sharing my own experience. There's many more tools that you can use for this purpose such as ElasticSearch.
I'm trying to create a website similar to BidCactus and LanceLivre.
The specific part I'm having trouble with is the seconds aspect of the timer.
When an auction starts, a timer of 15 seconds starts counting down, and every time a person bids, the timer is reset and the price of the item is increased by 0,01$.
I've tried using SignalR for this bit, and while it does work well during trials runs in the office, it's just not good enough for real world usage where seconds count. I would get HTTP 503 errors when too many users were bidding and idling on the site.
How can I make the timer on the clients end shows the correct remaining time?
Would HTTP GETting that information with AJAX every second allow me to properly display the missing time? That's a request each second!
And not only that, but when a user requests that GET, I calculate remaining seconds, but until the user see's that response, that time is no longer useful as a second or more might pass between processing and returning. Do you see my conundrum?
Any suggestions on how to approach this problem?
There are a couple problems with the solution you described:
It is extremely wasteful. There is already a fairly high accuracy clock built into every computer on the Internet.
The Internet always has latency. By the time the packet reaches the client, it will be old.
The Internet is a variable-latency network, so the time update packets you get could be as high or higher than one second behind for one packet, and as low as 20ms behind for another packet.
It takes complicated algorithms to deal with #2 and #3.
If you actually need second-level accuracy
There is existing Internet-standard software that solves it - the Network Time Protocol.
Use a real NTP client (not the one built into Windows - it only guarantees it will be accurate to within a couple seconds) to synchronize your server with national standard NTP servers, and build a real NTP client into your application. Sync the time on your server regularly, and sync the time on the client regularly (possibly each time they log in/connect? Maybe every hour?). Then simply use the system clock for time calculations.
Don't try to sync the client's system time - they may not have access to do so, and certainly not from the browser. Instead, you can get a reference time relative to the system time, and simply add the difference as an offset on client-side calculations.
If you don't actually need second-level accuracy
You might not really need to guarantee accuracy to within a second.
If you make this decision, you can simplify things a bit. Simply transmit a relative finish time to the client for each auction, rather than an absolute time. Re-request it on the client side every so often (e.g. every minute). Their global system time may be out of sync, but the second-hand on their clock should pretty accurately tick down seconds.
If you want to make this a little more slick, you could try to determine the (relative) latency for each call to the server. Keep track of how much time has passed between calls to the server, and the time-left value from the previous call. Compare them. Then, calculate whichever is smaller, and base your new time off that calculation.
I'd be careful when engineering such a solution, though. If you get the calculations wrong, or are dealing with inaccurate system clocks, you could break your whole syncing model, or unintentionally cause the client to prefer the higest latency call. Make sure you account for all cases if you write the "slick" version of this code :)
One way to get really good real-time communication is to open a connection from the browser to a special tcp/ip socket server that you write on the server. This is how a lot of chat packages on the web work.
Duplex sockets allow you to push data both directions. Because the connection is already open, you can send quite a bit of very fast data across.
In the past, you needed to use Adobe Flash to accomplish this. I'm not sure if browsers have advanced enough to handle this without a plugin (eg, websockets?)
Another approach worth looking at is long polling. In concept, a connection is made to the server that just doesn't die, and it gives you the opportunity on the server to trickle bits of realtime data down to the clients.
Just some pointers. I have written web software using JavaScript <-> Flash <-> Python/PHP, and was please with how it worked.
Good luck.
As part of my constant learning curve into what you can do to make apps scale better, I am currently trying to get a direction to go with queuing, i.e. job queuing or workload processing whichever phrase you like.
In the distant past I used IBM MQ/Series - it worked for a financial app but quite heavy if I remember.
I know of MSMQ, and I have also heard of quite a few others.
But first, here is my context
I have a C#/.NET back-end web app which serves data etc to a Javascript (mostly jQuery etc) front-end via AJAX calls etc. I have a situation where a certain action involves uploading some files, setting up a few record entries in the database, emailing some users etc. So of course I don't want to make this process "online"/"real-time" due to the possible time delay and I am sure the overheads on the webserver/database etc.
So given the type of "messages" that I need to queue and process, what would be (I shouldn't just say easy here I guess!) a good start point? should I run with MSMQ and/or the SQL 2008 service broker stuff, or something like ZeroMQ - or should I simply create my own lightweight workload queue service?
I realise again without seeing the full picture it is hard to make full recommendations, however any start points gratefully received!
David
Don't try to make your own, please! There are so many things to take into account that you will spend more time on it than the rest of your project most probably.
I'd say go for MSMQ, it's very easy to use with WCF, the queues are transactional, have a retry mechanism, etc, and you benefit from the MSMQ UI to see the messages, move them and so on.
It has become apparent that where I work needs, internally, a "notification system". The issue being that we are very spread out throughout multiple buildings and the bulk of the work force regularly keeps there email closed for hours at a time.
I need to create a simple way to be able to push out a message and have it "pop up" on everyones computer(or a single computer).
My first thought was to write a windows service that calls a winform/wpf app that resides on each computer that simply pops up with the message. Not sure how viable an idea that is but this is just brain-storming.
A different route, I thought, could be an app that resides in the systray on each computer that polls a db table and using the Query Notifications could pop up a message each time a new row is added. Then simply create an insanely basic app for writing a row to that table.
So, what I am asking is if any one else has walked this path. If so, how?
What things did you take into
consideration?
Are either of my ideas valid starting
points or are "egg and my face in
perfect alignment"?
Is there a different way that is even
simpler?
Thanks
Some simple requirements --> Must be "One Way" as I cannot give our user base a "chat" system. Must be, somewhat, hidden so as to discourage users shutting it off. A la system tray or service.
Wouldn't net send save you reinventing the wheel?
I've never done this but I've worked in a call-centre that did use something similar and they're insanely useful. I remember once when everyone got a message saying "does anyone know Mandarin? HELP ME!!" Brilliant. (Luckily someone did.)
Anyway your ideas are perfectly fine. Personally I'd do this as a client/server application. A windows forms or WPF application that sits in the systray could link to a server using a TCP/IP duplex connection using WCF. Perhaps get the client to register to certain groups depending on the department of the PC it's sitting on.
Then when someone wants to send a message they select which group it needs to go to (or all groups), the message hits the server which sends out to all connected clients, and the WPF app on the computer receives the message and pops it up. You don't even need a database except to store the users/groups, and the message history if you need to.
This might be a ridiculous answer but have you considered implementing a chat system? It's simple to implement and well tested.
Here are some possibilities:
http://messenger.softros.com/
http://en.wikipedia.org/wiki/Instant_messaging#User_base
Article on building your own:
http://www.computerworld.com/s/article/9002059/How_to_build_your_own_corporate_IM_system_
The easiest way to do this is to have a simple client on each machine polling a central service for alerts. Each alert should have a unique id so each client can deal with idempotency (you don't want the central service keeping tabs on which clients have "popped up").
I also recommend having a reasonably short lifespan for each alert, so the client only needs to know a very short list of alerts it has displayed and so if a machine was re-started, only a small history of alerts would be displayed.
With 300 subscribers, you'll want the polling to involve a nice long gap too - you don't really want 300 checks every 10 seconds - so you'll have to balance the technical desire for long gaps between checks with the business requirement to get an alert within a certain timeframe.
You could easily achieve this with a NET/TCP WCF service being polled by either a WINFORM / WPF application that is added as a start up program, or a windows service that then spawns a UI to display the notification.
I did something like this a long time ago to coordinate smoke breaks. I simply sent a broadcast packet out on the LAN at a specific port. Worked relatively well, although since anybody could broadcast and everybody would get a popup, it got abused a lot.
I would recommend you SPARK. We have same problem in my firm and finally decided to save time and do not reinventing the wheel and use existing (freeware) solution. SPARK does the job for us.
"Spark is an Open Source, cross-platform IM client optimized for businesses and organizations. It features built-in support for group chat, telephony integration, and strong security. It also offers a great end-user experience with features like in-line spell checking, group chat room bookmarks, and tabbed conversations."
If you cannot use / install existing IMs you might thing about implementing simple "chat" protocol in your app.
It is quite easy do that base on sockets and many articles available.
For example:
http://www.codeproject.com/KB/IP/TCPIPChat.aspx
http://www.codeproject.com/KB/miscctrl/SimpleMessenger.aspx?display=Print
If you need something advanced (eg. receive historical notification, users status management etc) you can consider using openSource Jabber API:
Eg http://www.codeproject.com/KB/gadgets/googletalk.aspx
I have a series of systems on a LAN running a synchronized display routine. For example, think of a chorus line. The program they ran is fixed. I have each "client" download the entire routine, and then contact the central "server" at fixed points in the routine for synchronization. The routine itself is mundane with, perhaps, 20 possible instructions.
Each client runs the same routine, but they can be doing completely different things at any one time. One part of the chorus line can be kicking left, another part kicking right, but all in time with each other. Clients can join and drop out at any time, but they're all assigned a part. If no-one is there to run the part, it just doesn't get run.
This is all coded in C# .Net.
The client display is a Windows Forms application. The server accepts TCP connections, and then services them round-robin fashion, keeping a master clock of what's going on. The clients send a signal that says "I've reached sync-point 32" (or 19, or 5, or whatever) and waits for the server to acknowledge and then moves on. Or the server can say "No, you need to start at sync-point 15".
This all works great. There is a minor bit of delay between the first and last clients to hit a sync-point, but it's hardly noticeable. Ran for months.
Then the Specification changed.
Now the clients need to respond to near real-time instructions from the server -- it's no longer a pre-set dance program. The server is going to be sending instructions out and the dance program is made up on the fly. I get the fun job of re-designing the protocol, the servicing loops, and the programming instructions.
My toolkit includes anything in a standard .Net 3.5 toolbox. Installing new software is a pain in the arse, since so many systems (clients) can be involved.
I'm looking for suggestions on keeping the clients synced (some sort of latching system? UDP? Broadcast?), distribution of the "dance program", anything that might make this easier than a traditional Client/Server TCP arrangement.
Keep in mind that there are time/speed limitations going on as well. I could put the dance program in a network database, but I'd have to shove instructions in fairly quickly and there'd be a lot of readers using a rather thick protocol (DBI, SqlClient, etc..) to get a small bit of text. That seems overly complex. And I still need something to keep them all displaying in sync.
Suggestions? Opinions? Wild-ass speculation? Code examples?
PS: Answers may not get marked as "correct" (since this isn't a "correct" answer), but +1 votes for good suggestions for sure.
I did something similar (quite a while back) with synchronizing a bank of 4 displays, each run by a single system, receiving messages from a central server.
The architecture we finally settled on after a fair amount of testing involved having one "master" machine. In your case, this would be having one of your 20 clients that acts as the master, and have it connect to the server via TCP.
The server then would send the entire series of commands for the series through to that one machine.
That machine then used UDP to broadcast real-time instructions to each of the other machines (the 19 other clients on its LAN) to keep their displays up to date. We used UDP for a couple reasons here - there was lower overhead involved, which helped keep the total resource usage down. Also, since you're updating in real-time, if one or two "frames" was out of sync, it was never noticable, at least not noticeable enough for our purposes (having a human sitting and interacting with the system).
The key point to this working smoothly, though, is having an intelligent communication means between the main server and the "master" machine - you want to keep the bandwidth as low as possible. In a case like yours, I'd probably come up with a single binary blob that had the current instruction set for the 20 machines, in its smallest form. (Maybe something like 20 bytes, or 40 bytes if you need it, etc). The "master" machine would then worry about translating this out to the other 19 machines and itself.
There are some nice things about this - the server has a much easier time transmitting to one machine in the cluster instead of every machine in the cluster. This let us, for example, have one single, centralized server "drive" multiple clusters efficiently, without having ridiculous hardware requirements anywhere. It also keeps the client code very, very simple. It just has to listen for a UDP datagram and do whatever it says - in your case, it sounds like it would have one of 20 commands, so the client becomes very, very simple.
The "master" server is the trickiest. In our implementation, we actually had the same client code on it as the other 19 (as a separate proces) and one "translation" process that took the blob, broke it into 20 pieces, and transmitted them. It was fairly simple to write, and worked very well.