I have a winforms app and i want to keep track of every time a user clicks certain buttons, etc as well as other actions. What is the best way for me to keep track of this information and then put it together so i can run metrics on most used features, etc.
This is a winforms app and I have users around the world.
There are 2 big issues your design has to be sure to address
Privacy (what Don alluded to) - You must be crystal clear what information you are collecting and sending up to your central server, ideally the users should be able to inspect the exact data you are sending back to the central server (if they wish to) - For any public software there should be a very easy way to opt out.
Performance and Scalability - You have to estimate how much data you really need on the server side and look at all sort of tricks that aggregate and compress the data client side (as well as have hard limits on the amount of traffic you will be sending and how often you will be sending it)
As to the client side implementation, I would recommend investigating Sqlite.net or another embedded DB. Using an embedded DB to store this information on the client will give you lots of flexibility with aggregations and will have the advantage of being transactional, fast and simple to implement. Sqlite has a very generous public domain license so from a legal perspective its really easy to use in public apps.
Try doing a google scholar search. There are some interesting ideas by Ben Liblit and co-authors, and another series of ideas by Alex Orso and co-authors (disclaimer - I'm one of Alex Orso's co-authors) based on taking a sample of runtime information from each user, and putting it together in an interesting way.
http://theory.stanford.edu/~aiken/publications/papers/pldi03b.pdf
and
http://www.cs.umd.edu/class/fall2006/cmsc838p/Ramss/remoteClassJournal.pdf
are two (not necessarily the best) examples of such papers/ideas.
I'd try something like this:
// execute this method once all forms have been created
public static void HookButtons()
{
foreach( Form f in Application.OpenForms )
{
EnumerateControls( f.Controls );
}
}
public static void EnumerateControls( ICollection controls )
{
foreach( Control ctrl in controls )
{
if( ctrl.Controls.Count > 0 )
{
EnumerateControls( ctrl.Controls );
}
if( ctrl is ButtonBase )
{
ctrl.MouseClick +=new MouseEventHandler( ctrl_MouseClick );
}
}
}
static void ctrl_MouseClick( object sender, MouseEventArgs e )
{
ButtonBase clicked = ((ButtonBase)sender);
// do something with the click information here
}
Be careful how you handle this. Some companies have gotten user backlash from collecting too much information or not being clear what was collected. The safest way is to ask the user before enabling any "phone home" features. Allowing the user to see the actual data before you send it seems good, too.
I've wondered if there's some way to piggyback on the one-click deployment call that happens whenever a one-click app starts up and checks for updates. I haven't investigated yet, though.
As for collecting the actual numbers, perhaps the user settings are the easiest place. If you're not familiar with them, just check out the project properties and go to the Settings tab.
Related
is there any need to handle locks in terms of threading in any inventory application.
like as i think asp.net is not thread safe.
lets say that there is a product available and its quantity available is 1 and number of user partially trying to book that particular product are 40. so which is going to get that product. or what happens.
not sure even if the question is reliable or not.
http://blogs.msdn.com/b/benchr/archive/2008/09/03/does-asp-net-magically-handle-thread-safety-for-you.aspx
i am not sure on this please help.
Well, technically, you're not even talking about ASP.NET here, but rather Entity Framework or whatever else you're using to communicate with SQL Server or whatever else persistent data store you're using. Relational databases will typically row-lock, so that as one client is updating the row, the row cannot be read by another client, but you can still run into concurrency issues.
You can handle this situation one of two ways: pessimistic concurrency or optimistic concurrency. With pessimistic concurrency you create locks and any other thread trying to read/write the same data is simply turned away in the mean time. In a multi-threaded environment, it's far more common to use optimistic concurrency, since this allows a bit of play room for failover.
With optimistic concurrency, you version the data. As a simplistic example, let's say that I'm looking for the current stock of widgets in my dbo.Widgets table. I'd have a column like Version which might initially be set to "1" and 100 widgets in my Stock column. Client one wants to buy a widget, so I read the row and note the version, 1. Now, I want to update the column so I do an update to set Stock to 99 and Version to 2, but I include in my where clause Version = 1. But, between the time the row was initially read and the update was sent, another client bought a widget and updated the version of the row to 2. The first client's update fails, because Version is no longer 1. So the application then reads the row fresh and tries to update it again, subtracting 1 from Stock and incrementing Version by 1. Rinse and repeat. Generally, you'll want to have some upward limit of attempts before you'll just give up and return an error to the user, but in most scenarios, you might have one collision and then the next one goes through fine. Your server would have to be getting slammed with people eagerly trying to buy widgets before it would be a real problem.
Now of course, this is a highly simplistic approach, and honestly, not something you really have to manage yourself. Entity Framework, for example, will handle concurrency for you automatically as long as you have a rowversion column:
[Timestamp]
public byte[] RowVersion { get; set; }
See http://www.asp.net/mvc/tutorials/getting-started-with-ef-using-mvc/handling-concurrency-with-the-entity-framework-in-an-asp-net-mvc-application for the full guide to setting it up.
ASP.NET certainly is not Thread Safe. The article you link too is fine as a start, but doesn't tell all the story by a long way. In your case, you likely load the Product List into memory at first request for it, or at Application Startup or some other trigger.
When a Request wants to work with a product you grab the appropriate member of this preloaded list. (Believe me this is better than having every request loading the product or product list from the database.) However, now if you have 40 simultaneous requests for the same product they will all be accessing the same object, and new nasty things can happen, like ending up with -39 stock.
You can address this in a many ways ways, but they boild down to two:
Protect the data somehow
Do what Amazon does
Protect the data
There are numerous ways of doing this. One would be to use a critical section via the Lock keyword on C#. For an example, something like this in the Product Class:
private object lockableThing; // Created in the ctor
public bool ReduceStockLevelForSale(int qtySold)
{
bool success = false;
if (this.quantityOnHand >= qtySold)
{
lock (lockableThing)
{
if (this.quantityOnHand >= qtySold)
{
this.quantityOnHand -= qtySold;
success = true;
}
}
}
return success;
}
The double check on the quantity on hand is deliberate and required. There are any number of ways of doing the equivalent. Books have been written about this sort of thing.
Do what Amazon does
As long as at some point in the Order Taking sequence, Amazon thinks it has enough on hand (or maybe even any) it will let you place the order. It doesn't reduce the stock level while the order is being confirmed. Once the order has been confirmed, it has a back-end process (i.e. NOT run by the Web Site) which checks order by order that the order can be fulfilled, and only reduces the On Hand level if it can. If it can't be, they put the order on hold and send you an email saying 'Sorry! We don't have enough of Product X!' and giving you some options.
Discussion
Amazon's is the best way, because if you decrement the stock from the Web Site at what point do you do it? Probably not until the order is confirmed. If the stock has gone, what do you then do? Also, you are going to have to have some functionality to send the 'Sorry!' email: what happens when the last one (or two or three) items of that product can't be found, don't physically exist or are broken? You send a 'Sorry!' email.
However, this does assume that you are in control of the full order to dispatch cycle which is not always the case. If you aren't in control of the full cycle, you need to adjust to what you are in control of, and then pick a method.
Now that Stack Overflow uses redis, do they handle cache invalidation the same way? i.e. a list of identities hashed to a query string + name (I guess the name is some kind of purpose or object type name).
Perhaps they then retrieve individual items that are missing from the cache directly by id (which bypasses a bunch of database indexes and uses the more efficient clustered index instead perhaps). That'd be smart (the rehydration that Jeff mentions?).
Right now, I'm struggling to find a way to pivot all of this in a succinct way. Are there any examples of this kind of thing that I could use to help clarify my thinking prior to doing a first cut myself?
Also, I'm wondering where the cutoff is between using a .net cache (System.Runtime.Caching or System.Web.Caching) and going out and using redis. Or is Redis just hands down faster?
Here's the original SO question from 2009:
https://meta.stackexchange.com/questions/6435/how-does-stackoverflow-handle-cache-invalidation
A couple of other links:
https://meta.stackexchange.com/questions/69164/does-stackoverflow-use-caching-and-if-so-how/69172#69172
https://meta.stackexchange.com/questions/110320/stack-overflow-db-performance-and-redis-cache
I honestly can't decide if this is a SO question or a MSO question, but:
Going off to another system is never faster than querying local memory (as long as it is keyed); simple answer: we use both! So we use:
local memory
else check redis, and update local memory
else fetch from source, and update redis and local memory
This then, as you say, causes an issue of cache invalidation - although actually that isn't critical in most places. But for this - redis events (pub/sub) allow an easy way to broadcast keys that are changing to all nodes, so they can drop their local copy - meaning: next time it is needed we'll pick up the new copy from redis. Hence we broadcast the key-names that are changing against a single event channel name.
Tools: redis on ubuntu server; BookSleeve as a redis wrapper; protobuf-net and GZipStream (enabled / disabled automatically depending on size) for packaging data.
So: the redis pub/sub events are used to invalidate the cache for a given key from one node (the one that knows the state has changed) immediately (pretty much) to all nodes.
Regarding distinct processes (from comments, "do you use any kind of shared memory model for multiple distinct processes feeding off the same data?"): no, we don't do that. Each web-tier box is only really hosting one process (of any given tier), with multi-tenancy within that, so inside the same process we might have 70 sites. For legacy reasons (i.e. "it works and doesn't need fixing") we primarily use the http cache with the site-identity as part of the key.
For the few massively data-intensive parts of the system, we have mechanisms to persist to disk so that the in-memory model can be passed between successive app-domains as the web naturally recycles (or is re-deployed), but that is unrelated to redis.
Here's a related example that shows the broad flavour only of how this might work - spin up a number of instances of the following, and then type some key names in:
static class Program
{
static void Main()
{
const string channelInvalidate = "cache/invalidate";
using(var pub = new RedisConnection("127.0.0.1"))
using(var sub = new RedisSubscriberConnection("127.0.0.1"))
{
pub.Open();
sub.Open();
sub.Subscribe(channelInvalidate, (channel, data) =>
{
string key = Encoding.UTF8.GetString(data);
Console.WriteLine("Invalidated {0}", key);
});
Console.WriteLine(
"Enter a key to invalidate, or an empty line to exit");
string line;
do
{
line = Console.ReadLine();
if(!string.IsNullOrEmpty(line))
{
pub.Publish(channelInvalidate, line);
}
} while (!string.IsNullOrEmpty(line));
}
}
}
What you should see is that when you type a key-name, it is shown immediately in all the running instances, which would then dump their local copy of that key. Obviously in real use the two connections would need to be put somewhere and kept open, so would not be in using statements. We use an almost-a-singleton for this.
I come from a PHP background and have used Wordpress quite a lot, I love how their plugin architecture works and the ability to hook events to event names. One of the best parts I like about it is being able to *add_filter()* to any database value just before it gets shown to the end user. My question is multi-part on how to replicate the whole plugin architecture in a C#.NET environment?
Part 1:
To create plug-ins I have researched the MEF framework would probably be the best (Managed Extensibility Framework -http://mef.codeplex.com/). This is designed specifically to take the grunt work out by giving you the ability to monitor directories for new plug-ins, tracking dependencies and other normal things. MEF ships with .NET 3.5+
Part 2
Hooking events? I can't seem to find much information about replicating a global channel based event system. From what I have upto yet I need a publish/subscribe pattern (which isn't that hard to make as you just create some concrete objects and give them events). The hard part is giving each event a 'channel' name and for all the events in the whole system to be part of a global collection (Mediator pattern).
To replicate: (http://codex.wordpress.org/Function_Reference/add_filter)
Example 1
// Add's my button to the end of the content
add_filter('the_content', 'my_plugin_button');
function my_plugin_button( $content ) {
// Add's my button to the end of the content
return $content . "<a href='#'>My button</a>";
}
OR
Example 2
// Add a new admin menu item by hooking in
add_action('admin_menu', 'my_plugin_menu');
function my_plugin_menu() {
add_options_page('My Plugin Options', 'My Plugin', 'manage_options', 'my-unique-identifier', 'my_plugin_options');
}
I hope your all with me upto yet? I have managed to replicate the functionality I need in Javascript and even jQuery has their .on() event function... same thing but channel or list based...
My 2 examples:
http://jsfiddle.net/AaronLayton/U3ucS/53/
http://jsfiddle.net/AaronLayton/eyNre/33/
Can anyone point me in the correct direction or is this the totaly wrong approach for c#?
I think NServiceBus can help you a lot with these issues. Udi Dahan which is the author of NServiceBus has also written a lot of articles about Domain Event pattern, which is a publish/subscribe mechanism.
Know it's been a long time since you posted this and you probably built something already. However I have been thinking about something like this myself. There are 2 options - really forget WordPress and try and build something much cleaner - it's a mess at the bottom of WordPress' code :D
Or this:
function the_content()
{
var result = get_the_content();
// do other stuff...if you want to.
execute_filters(ref result);
execute_actions(ref result);
return result;
}
function execute_filters(ref string result, string action_name)
{
var filters = get_pre_filters(action_name);
filters.ForEach(filter =>
{
/// somehow call the method name in filter. PHP is generally global. C# namespaced,
/// so you would need to think about that.
}
}
function execute_actions(ref string result, string action_name)
{
/// and so on....
}
When building something to mimic WordPress, you need to remember many of the issues of WordPress' plugin architecture (in my personal opinion)... It seems to want to run every plugin near enough on every page even if that page has nothing to do with that plugin. I onced installed a plugin that added 60 database queries to each page call, and it wasn't used.
Try and think smart about it when you are building it. Try and add a way to only have the plugins that are going to get used on the page/post of your new setup to be run e.g. in your database have a "Plugins" field on the post/page object with a list of plugins allowed to run on that page. That way you won't need to check all the plugins each time to see if it wants to run.
Anyways. Hope you got something working.
I'm tring to pass "statistics" from one program to another (my first question is here how to pass some "statistics" from c# program to another program?)
To pass statistics I first need to collect it.
I've decided to implement central storage, like StatisticsStorage with one method StatisticsStorage.joinStatistics(string groupName, string indicatorName, callback getValueMethod)
Then for example Thermometer class should look like that (pseudo code):
class Thermometer {
Thermometer(string installationPlace) {
StatisticsStorage.joinStatistics("temperature", installationPlace, this.getThermometerValue);
}
callback double getThermometerValue {
return this.thermometerValue;
}
private double thermometerValue;
//.....
}
StatisticsStorage should call callBack method for all indicators periodically.
Once statistics is collected I can pass it one way or another.
Questions:
do you see any problems with my approach?
how to implement callbacks on c# better? (i'm pretty novice to c#)
There are probably many ways to achieve your desire result.
I would probably publish a WCF service, maybe hosted in a windows service, and you can connect and post stats on that. This will keep a good separation of system concerns and can be reused from other systems etc etc.
I suppose it depends on how in depth you want to go and the requirements in this scenario.
Then again, i could just be over analyzing what you are trying to do :)
One application I work on does only one thing, looking from outside world. Takes a file as input and after ~5 minutes spits out another file.
What happens inside is actually a sequential series of action. The application is, in our opinion, structured well because each action is like a small box, without too many dependencies.
Usually some later actions use some information from previous one and just a few can be executed in parallel - for the sake of simplicity we prefer to the execution sequential.
Now the problem is that the function that executes all this actions is like a batch file: a long list of calls to different functions with different arguments. So, looking in the code it looks like:
main
{
try
{
result1 = Action1(inputFile);
result2 = Action2(inputFile);
result3 = Action3(result2.value);
result4 = Action4(result1.value, inputFile);
... //You get the idea. There is no pattern passed paramteres
resultN = ActionN(parameters);
write output
}
catch
{
something went wrong, display the error
}
}
How would you model the main function of this application so is not just a long list of commands?
Not everything needs to fit to a clever pattern. There are few more elegant ways to express a long series of imperative statements than as, well, a long series of imperative statements.
If there are certain kinds of flexibility you feel you are currently lacking, express them, and we can try to propose solutions.
If there are certain clusters of actions and results that are re-used often, you could pull them out into new functions and build "aggregate" actions from them.
You could look in to dataflow languages and libraries, but I expect the gain to be small.
Not sure if it's the best approach, but you could have an object that would store all the results and you would give it to each method in turn. Every method would read the parameters it needs and write its result there. You could then have a collection of actions (either as delegates or objects implementing an interface) and call them in a loop.
class Results
{
public int Result1 { get; set; }
public string Result2 { get; set; }
…
}
var actions = new Action<Results>[] { Action1, Action2, … };
Results results = new Results();
foreach (var action in actions)
action(results);
You can think of implementing a Sequential Workflow from Windows Workflow
First of all, this solution is far not bad. If the actions are disjunct, I mean there are no global parameters or other hidden dependencies between different actions or between actions and the environment, it's a good solution. Easy to maintain or read, and when you need to expand the functionality, you have just to add new actions, when the "quantity" changes, you have just to add or remove lines from the macro sequence. If there's no need for change frequently the process chain: don't move!
If it's a system, where the implementation of actions don't often changes, but their order and parameters yes, you may design a simple script language, and transform the macro class into that script. This script should be maintained by someone else than you, someone who is familiar with the problem domain in the level of your "actions". So, he/she can assembly the application using script language without your assistance.
One nice approach for that kind of problem splitting is dataflow programming (a.k.a. Flow-based programming). In dataflow programming, there are pre-written components. Components are black boxes (from the view of the application developer), they have consumer (input) and producer (output) ports, which can be connected to form a processing network, which is then the application. If there're a good set of components for a domain, many applications can created without programming new components. Also, components can be built of other components (they called composite components).
Wikipedia (good starting point):
http://en.wikipedia.org/wiki/Dataflow_programming
http://en.wikipedia.org/wiki/Flow-based_programming
JPM's site (book, wiki, everything):
http://jpaulmorrison.com/fbp/
I think, bigger systems must have that split point you describe as "macro". Even games have that point, e.g. FPS games have a 3D engine and a game logic script, or there's SCUMM VM, which is the same.