I want to create an application that will run in the background and track the click actions of the mouse, and when the user clicks on an external WPF or Winforms app, the background app should be able to detect the clicked control and display its ID/name/text.
I'm not sure if this is possible, but since there are automation tools that can perform similar actions, I think it should be possible. For example, with some tools it is possible to get the "window" object using the external app's PID, and then the controls (for example a button) can be accessed by providing the ID of the control. However in my case it is the other way around. I need to get ID/name/text of the control the user has clicked.
I can obtain the mouse position using windows Hook
I can obtain the hwnd and Process ID of the window the user has clicked
So is it possible to obtain the information about the element clicked? Any help and advice is appreciated, thanks in advance.
I tried writing code several different times, but I came to an error with each one.
Basically, I'm trying to make "windows" similar to say Explorer, Paint, MediaPlayer, where you could drag then around, interact with them, minimize and close. Of course, if you clicked on a window, the one below it (they can overlap) shouldn't get affected.
I know how to do this, I have a list of the class I call Window, loop through it, and I only interact with the first window to contain the location of the mouse-click. This way, other windows overlap won't get affected.[1]
Next, I had to make it so that two buttons that are overlapping don't get activated when the user clicks in the "intersection of both buttons." I handled this by using the same method I used above.[2]
But the problem I'm facing now is that, if I hold the left click, but then I decide not to click a button, I drag the mouse away from the button, and release the left click, so that the button-click event won't be activated. But, when I remove the mouse from the boundaries of the button, and say, into another.. the new button get activated. Which it should not.[3]
My set up is like this:
I have a class called Window.
In Window, I have a list of the class called Interface (similar to the Control class in WinForms).
And each Interface has a struct in it that contains 4 bools, if the left/right is currently down, and if they were down in the previous processing. (prevLeft, prevRight, currLeft, currRight)
So, I'm ready to discard that (I have not yet, so I still have the source code), but I need a good structure for making an object-oriented type of application. However, I am not using WinForms. I need help with the structure alone, so no actual code is necessary, description is enough. I need to avoid the 3 problems I mentioned above.
Creating your own Window Manager is not an easy task. I know it because I'm making one too ;)
You can use an existing, though maybe not the best solution, like for example Nuclex.UI, which I personally rejected when I first saw it, but if you're not dead set on making your own WM, I suggest to use that or hybrid WinForms-XNA approach.
But if you're really dead set on implementing a custom Window Manager, you have to understand how any other WM works. Since we're talking about XNA, it means Windows, and that means Windows Explorer, which is a great thing to learn from.
You have to recognize how the simplest things work, and it's really not so hard. The hard part is figuring out what logic is updated when, and how to not spend all the CPU on only UI updates. Let me just give you a few hints on how to solve the problems you mention in your question.
To keep track of all windows, I'm using a Dictionary<string, Window>, where Window is a custom class, and the string is its unique name for rare cases where I have to call windows by name. Think of it as a window GUID or Handle. But you can just make it so that a "Form" can only appear once, and store all references in static variables.
To make WM understand what control you're clicking I use rectangles and check if they contain a Point which is at Cursor coordinates and has {1; 1} pixel size, which is probably about the same way it's done in Windows Explorer. To do that your WM needs to know in which order to update the active windows. Usually you'd want to start from the topmost window and continue towards the end of the list of active windows. For that you can just iterate through the list with a foreach loop.
But that's not all, because every window itself is a Container, which means it contains other controls, some of which may even be Containers themselves, like WinForms Panel class. This means you have to iterate through each of the Windows' Children controls. The update order should make sense too - update from the topmost child to the bottommost, recursively for Container controls, in case they also have Containers in them. This basically means you'd want to implement a recursive GetAllControls() method for your WindowManager class that would iterate through all Containers and return a list of all Controls.
Drawing all those Controls should be done in reverse order of updating them, so you can just GetAllControls().Reverse() and iterate through that in a foreach loop.
Where to draw and what to update depends on all the parent containers the current container has and their combined offset from the top-left corner of the game window. I solve this by storing a ParentContainer reference in all children controls to get the appropriate DrawRectangles and update areas via recursive properties.
When you click somewhere on the screen and a click is registered on a Control, make the WindowManager remember that (bool clickRegistered) and not run any OnClick events on any underlying Controls.
Windows Explorer remembers the control you clicked and will activate its OnRelease event if the cursor is then released in the update area of the very same control. So basically Windows Manager only does something when you release the mouse button. You can make your WindowManager and Controls to handle click events differently, like firing an event right after you press the mouse button, i.e. OnMouseDown. But remember that Microsoft aren't noobs and there's a reason for that behavior in Windows Explorer, and it's because if you accidentally press a mouse button somewhere you didn't intend, you can still fix it by moving the cursor outside the pressed control's update area and not run its action.
At this point you might be thinking "Is it really worth implementing all this?" For me the answer was "maybe", because I was a total noob in both C# and XNA at the time I started, and now I know my game, which was originally supposed to use some Window Manager, is going to benefit from my own WM implementation far more than from ready third-party solutions. And besides, it's a great exercise in logic and programming.
But if you'd like to think of yourself as a game developer, you should think in terms of reaching your goal as quickly as possible, i.e. actually making a game, and not the game engine. So in this case, better make use of existing solutions and start selling your product.
Instead of having the structure with the 4 booleans (similar to xna), how about you make a way to tell where the mouse "is." So in a sense, the mouse is in Window number 5 which is Paint, and the user is holding the mouse down on interface/control number 2 which is a button.
That sounds like it could work.
I have written an application that currently handles clicks from multiple mouse devices.
I used this project and modified to handle mouse clicks as apposed to keyboards.
This is working fine, however now I need to know if there is a way to suppress a click event if it has been handled by my app. The app is a quiz game so the idea is that the quiz master will have (and still be able to use) 1 mouse, and the other contestants will have their own mouse (as buzzers). So when they buzz in, I don't want the mouse click events to fire in the operating system (or at least this application).
The concept is the familiar override of void WndProc(ref Message message), and so I have tried not calling base.WndProc(ref Message) when I don't want the click events to fire, but this has not worked.
Can anybody point me in the right direction here?
Should I be going down the windows hook route? I have looked at this, but I can't seem to work out how I could hook to each mouse device individually.
Any help would be greatly appreciated.
Edit:
This is a Windows Form UI project, and not WPF. So the MultiPoint SDK from Microsoft won't work.
The solution to this lies within not WndProc, but PreFilterMessage(). By intercepting messages before they even reach the form, you can remove them from the message pump causing them to never reach the control that was clicked. This also works for child controls within the form.
I answered this and posted the full source in the following question:
C# Get Mouse handle (GetRawInputDeviceInfo)
Is it possible to programmatically change the process that starts once the user click the LifeChat LX-3000 center button?
I don't have any kind of experience programming with hardware so don't know if it's even possible. The problem is, a client of us has a lot of this headphones and we want to be able to change the default behavior of that button from open live messenger to open our CRM. I try to follow this instructions to modify the registry as doing (not programmatically):
Find HKEY_CLASSES_ROOT\CLSID{F5F545A6-39C4-40B5-814D-B45040A89FB5}\LocalServer32
Remove the (Default) value by double clicking it and deleting the text
Change the value to OurCRMExePath
But this doesn't work. I suppose things changed since the date that web page was wrote. Any idea? I suppose first step would be to know which registry entry does that button modify and then go and change it. But, how can one get to know that?
I'm creating a log system ('outside' the application) that logs all the click on the application. I have setup a PreFilterMessage function wich detects a click from the mouse, but I can't find the control/element that was clicked.
I've tried with Mouse.DirectlyOverbut the element is always null.
I've tried also with VisualTreeHelper.HitTest but I don't have a Visualto make the search from.
I don't have access to the inside of the application: only to the Main method (with the Application.Run(new MainForm()); and my Application.AddMessageFilter(new Logger());).
Does anyone have an idea (or a walk around) on how to get the clicked control in the application (in .Net 3.5)?
I've been using an amazing little application called Snoop for some time now that I think does exactly what you're after, it's open source (C#) and may be of use to you if you can find out how it works. (All WPF developers should get this and no I don't work on Snoop lol)
http://snoopwpf.codeplex.com/