I am trying to figure out what item (for example document, web page tab, window, picture, folder) the user has clicked on. I started by using the following code when I detect a global left mouse click:
System.Drawing.Point MousePoint = System.Windows.Forms.Cursor.Position;
AutomationElement AutomationElement = AutomationElement.FromPoint(new System.Windows.Point(MousePoint.X, MousePoint.Y));
Console.WriteLine(AutomationElement.Current.Name);
This seems to work well in most conditions. However, I need to (if possible) get names of documents/images/folders inside Windows Explorer for example. The value returned when I click a document in the right hand pane of Windows Explorer (not the tree view) is "Name". Is there anyway to get the actual document name? For some reason, clicking sub-folders in the tree view returns the name of the folder, which is what I want.
I also notice that the code seems to display the document/image/folder name when clicked if the Windows Explorer view is set to icons (medium, large or extra large). Is there any reason why other views return "Name" or empty string while medium, large and extra large icons return the actual document/image/folder name? Is it to do with the size of the object clicked? I could really do with a way round this if possible?
I apologise, I am new to UI Automation and just really want a way to find the name of the object (file, folder, document, picture, web page tab etc.) that the user has clicked on. Any help anyone could give would be great.
You need to listen to InvokePattern.InvokedEvent UI automation event.
Example
The following example assumes you have an open instance of windows "Calculator" app. Then if you run the application, when you click on any button in Calculator, we handle the click event and show what button has clicked.
[DllImport("user32.dll")]
private extern static IntPtr FindWindow(string lpClassName, string lpWindowName);
protected override void OnLoad(EventArgs e)
{
base.OnLoad(e);
var proc = System.Diagnostics.Process.GetProcessesByName("Calculator")
.FirstOrDefault();
if (proc != null)
{
var mainHwnd = FindWindow(null, "Calculator");
var mainWndElement = AutomationElement.FromHandle(mainHwnd);
Automation.AddAutomationEventHandler(
InvokePattern.InvokedEvent, mainWndElement,
TreeScope.Subtree, (s1, e1) =>
{
var element = s1 as AutomationElement;
MessageBox.Show($"'{element.Current.Name}' clicked by user!");
});
}
}
protected override void OnFormClosing(FormClosingEventArgs e)
{
Automation.RemoveAllEventHandlers();
base.OnFormClosing(e);
}
You need to add reference to UIAutomationClient and UIAutomationTypes assemblies.
Note
You can easily extend the example and listen to Win32_ProcessStartTrace to detect when Calculator opens. You can see an example to detect when an external process window closes which is using Win32_ProcessStopTrace.
Another option is using SetWinEventHook you can listen to some events from other processes and register a WinEventProc callback method to receive the event when the event raised. Here, you are interested in EVENT_OBJECT_INVOKED event.
Related
I have a lots of problem to distinguish such a simple thing.
I need to know if a form is currently in front of everything, the one which receives key entries.
I have no way to know if it is.
I can check if not minimized. But then it may just be behind other windows, or just not being selected (for example it is openend, desktop is behind, you click on desktop, then you still see the application, but it doesn't receive key inputs).
The property focus is irrevelant for this.
Here is the code
protected override void OnActivated(EventArgs e)
{
base.OnActivated(e);
if (this.Focused)
{
gotFocus = true;
// never reaches tis
}
Check if window is the current active window.
Code:
using System.Runtime.InteropServices; // To use DllImport
...
[DllImport("user32.dll")]
private static extern IntPtr GetForegroundWindow();
if ((IntPtr)GetForegroundWindow() == this.Handle)
{
// Do stuff
}
See: Use GetForegroundWindow result in an if statement to check user's current window
I'm writing an application that shows notifications trough baloontips over notifyicon. There are two kinds of notifications I want to display - normal baloontip and clickable baloontip. I want clickable baloontips to open some url in web browser. The problem is that events stacks over baloontip.
I'm not sure if this explanation says anything, so here's an example:
code:
NotifyIcon ni = new NotifyIcon();
void showClickableNotification(string title, string content, string url)
{
ni.BaloonTipClicked += new EventHandler((sender, e) => ni_BalloonTipClicked(sender, e, url));
ni.ShowBaloonTip(1, title, content, ToolTipIcon.Info);
}
void ni_BalooTipClicked(object sender, EventArgs e, string url)
{
Process.Start(url);
}
every use of showClickableNotification will assign one more url to BallonTipClicked event
I want to clear event after notification will hide, to prevent opening multiple tabs unassociated with current notification.
Also, when normal notification is shown after a clickable one it's click opens all the stacked urls as well.
I tried to assign an empty function for ni.BaloonTipClicked += emptyFunction this, but += operator just adds another event to the pool instead of overwriting it. -= does not work since I'm adding new event every time. I guess I could do some global variable that holds current url and avoid assigning new everytime (-= would work then), but it looks like cheap workaround. Is there any (correct?) way to do it?
I have a program not written by me (closed source) and I need to read the text from what looks like a multi-line text edit.
The program has a TabControl at the highest level with 15 tabs. Within the 15th tab, there is the text edit that I need to get to.
AutomationElement aeEntireApplication = AutomationElement.FromHandle(hwd);
AutomationElementCollection aeEditCollection = aeEntireApplication.FindAll(TreeScope.Descendants, new PropertyCondition(AutomationElement.ClassNameProperty,"Edit"));
foreach (AutomationElement aeEdit in aeEditCollection )
{
object patternObj;
if (edit.TryGetCurrentPattern(TextPattern.Pattern, out patternObj))
{
var textPattern = (TextPattern)patternObj;
Console.WriteLine(textPattern.DocumentRange.GetText(-1).TrimEnd('\r')); // often there is an extra '\r' hanging off the end.
}
}
With this code, it only will print the contents of the Text Edits for the tab I am currently on. Is it possible to get to the contents of tab #15 without having to have that tab open?
It's always worth pointing the Inspect SDK tool at your UI, and checking what's being exposed through the UIA Raw view. The Raw view contains everything being exposed through UIA for the UI. (The view being shown in Inspect can be selected from Inspect's Options menu.) If Inspect doesn't show you the UI that you're interested in, then that UI isn't being exposed by the app (or the UI framework the app's using), and your own UIA client code won't be able to access it either.
I just created a test WinForm app with a TabControl. The TabControl has two tabs, and a TextBox for each tab. Inspect shows me that at any given time, only the UI contained in the active tab page is being exposed through UIA. So you won't be able to use UIA to access the UI on the other tabs.
Thanks,
Guy
One solution that works it to use
internal static extern bool EnumChildWindows(IntPtr hwnd, WindowEnumProc func, IntPtr lParam);
I enumerate through all children of the entire program, then find a button next to the edit which has a unique name. Then I use
uint GW_HWNDPREV = 3;
logTextBoxHandle = GetWindow(hwnd, GW_HWNDPREV);
This gets me the handle to the Text Edit I need.
Not elegant, but it seems to work.
I want to create a windows app that does the following.
When a button is clicked,
Find a running instance of IE (which I was able to get a handle using FindWindow api (user32.dll))
Send message to windows OS to mouse click on the image in IE. I already know that there is an image on the page. -- this is where I need help!! thanks.
How do I get a image object in html from windows app when I have a handle?
I've tried user32.dll (mouse_event(long dwFlags, long dx, long dy, long cButtons, long dwExtraInfo)) but I can't find correct x and y for the image in the page.
please, check the code below; it should be iterating through all opened IE windows using SHDocVw.ShellWindows collection; for each window it iterates through collection of images of the mshtml.HTMLDocument object associated with the window. For each image it dumps its href into the console and tries to click on it using the mshtml.HTMLImg click method. You can also check HTMLImg defenition for the object's position in case you would want to go with sending mouse click through mouse_event procedure. I also added OnClick event handler to image objects; it should be executed every time you click on the image either in code or using mouse.
private void button2_Click(object sender, EventArgs e)
{
SHDocVw.ShellWindows shellWindows = new SHDocVw.ShellWindows();
foreach (SHDocVw.WebBrowser ie in shellWindows)
{
mshtml.HTMLDocument doc = ie.Document as mshtml.HTMLDocument;
if (doc != null)
{
foreach (mshtml.HTMLImg imgElement in doc.images)
{
((HTMLImgEvents_Event)imgElement).onclick += new mshtml. HTMLImgEvents_onclickEventHandler(Form1_onclick);
Console.WriteLine(imgElement.href);
imgElement.click();
}
}
}
}
private bool Form1_onclick()
{
Console.WriteLine("click !!");
return true;
}
hope this helps, regards
I have a C#/.NET app and I want to implement the following behavior:
I have a popup menu. Whenever the user clicks on anything within the application that is not the popup menu, I want the popup menu to close.
However, whenever a user is not in the application I don't want anything to happen.
I'm trying to manage this through the LostFocus event, but I'm having trouble determining whether my application is the active window. The code looks something like this.
private void Button_LostFocus(object sender, System.EventArgs e)
{
if (InActiveWindow()) {
CloseMenu()
}
else {
// not in active window, do nothing
}
}
What I need to know is how to implement the InActiveWindow() method.
You could P/Invoke into GetForegroundWindow(), and compare the HWND returned to the application's form.Handle property.
Once you have the handle, you can also P/Invoke GetAncestor() to get the root owner window. This should be the handle of your application's main, startup window, if this is in your application.
I stumbled upon your question while working on a project and based on Reed Copsey's answer, I wrote this quick code which seems to do the job well.
Here's the code:
Public Class Form1
'''<summary>
'''Returns a handle to the foreground window.
'''</summary>
<Runtime.InteropServices.DllImport("user32.dll", SetLastError:=True)> _
Private Shared Function GetForegroundWindow() As IntPtr
End Function
'''<summary>
'''Gets a value indicating whether this instance is foreground window.
'''</summary>
'''<value>
'''<c>true</c> if this is the foreground window; otherwise, <c>false</c>.
'''</value>
Private ReadOnly Property IsForegroundWindow As Boolean
Get
Dim foreWnd = GetForegroundWindow()
Return ((From f In Me.MdiChildren Select f.Handle).Union(
From f In Me.OwnedForms Select f.Handle).Union(
{Me.Handle})).Contains(foreWnd)
End Get
End Property
End Class
I didn't have too much time to convert it to C# as I'm working on a project with a deadline in 2 days but I believe you can quickly do the conversion.
Here is the C# version of the VB.NET code:
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
[System.Runtime.InteropServices.DllImport("user32.dll")]
private static extern IntPtr GetForegroundWindow();
///<summary>Gets a value indicating whether this instance is foreground window.</summary>
///<value><c>true</c> if this is the foreground window; otherwise, <c>false</c>.</value>
private bool IsForegroundWindow
{
get
{
var foreWnd = GetForegroundWindow();
return ((from f in this.MdiChildren select f.Handle)
.Union(from f in this.OwnedForms select f.Handle)
.Union(new IntPtr[] { this.Handle })).Contains(foreWnd);
}
}
}
It seems like the biggest reason this is tricky is because the popup loses focus before the main form is deactivated, so the active window will always be in the application at the time of this event. Really, you want to know whether it will still be the active window after the event is over.
You could set up some kind of scheme where you remember that a popup is losing focus, set aside the fact that you will need to close it, and in the LostFocus or Deactivate event of the application's main form cancel the note that tells you you need to close it; but the problem is when will you process the note?
I'm thinking it might be easier, at least if the popup is a direct child of the main form (which I suspect in your case it may be) to hook the Focus or maybe even Click event of the main form and use it close the popup if it is open (perhaps by scanning its list of child forms for any that implement an ICloseOnLostFocus interface, so that the popup will have a chance to participate in the decision and do anything else it needs to do).
I wish I knew of a better document explaining what all these events actually mean and how they are sequenced with respect to one another, MSDN leaves a lot to be desired in describing them.