UI Automation FindAll(TreeScope.Descendants) isnt reaching grandchildren - c#

I have a program not written by me (closed source) and I need to read the text from what looks like a multi-line text edit.
The program has a TabControl at the highest level with 15 tabs. Within the 15th tab, there is the text edit that I need to get to.
AutomationElement aeEntireApplication = AutomationElement.FromHandle(hwd);
AutomationElementCollection aeEditCollection = aeEntireApplication.FindAll(TreeScope.Descendants, new PropertyCondition(AutomationElement.ClassNameProperty,"Edit"));
foreach (AutomationElement aeEdit in aeEditCollection )
{
object patternObj;
if (edit.TryGetCurrentPattern(TextPattern.Pattern, out patternObj))
{
var textPattern = (TextPattern)patternObj;
Console.WriteLine(textPattern.DocumentRange.GetText(-1).TrimEnd('\r')); // often there is an extra '\r' hanging off the end.
}
}
With this code, it only will print the contents of the Text Edits for the tab I am currently on. Is it possible to get to the contents of tab #15 without having to have that tab open?

It's always worth pointing the Inspect SDK tool at your UI, and checking what's being exposed through the UIA Raw view. The Raw view contains everything being exposed through UIA for the UI. (The view being shown in Inspect can be selected from Inspect's Options menu.) If Inspect doesn't show you the UI that you're interested in, then that UI isn't being exposed by the app (or the UI framework the app's using), and your own UIA client code won't be able to access it either.
I just created a test WinForm app with a TabControl. The TabControl has two tabs, and a TextBox for each tab. Inspect shows me that at any given time, only the UI contained in the active tab page is being exposed through UIA. So you won't be able to use UIA to access the UI on the other tabs.
Thanks,
Guy

One solution that works it to use
internal static extern bool EnumChildWindows(IntPtr hwnd, WindowEnumProc func, IntPtr lParam);
I enumerate through all children of the entire program, then find a button next to the edit which has a unique name. Then I use
uint GW_HWNDPREV = 3;
logTextBoxHandle = GetWindow(hwnd, GW_HWNDPREV);
This gets me the handle to the Text Edit I need.
Not elegant, but it seems to work.

Related

How to set Word Application.Selection focused

I am building a Word VSTO Add-in. There I have a side drawer with some buttons on it. Now when I click on a button it inserts a predefined text in a document selection position. When it inserts the text I would like to bring back focus to the document itself.
How to do that?
My best try so far:
Word.Selection currentSelection = Application.Selection;
currentSelection.Text = ((Button)sender).Tag.ToString();
Application.ActiveWindow.SetFocus();
Application.ActiveWindow.Activate();
It makes sense to call the Activate method prior calling the SetFocus one:
Word.Selection currentSelection = Application.Selection;
currentSelection.Text = ((Button)sender).Tag.ToString();
Application.ActiveWindow.Activate();
Application.ActiveWindow.SetFocus();
Also you may find Windows API functions helpful for such scenarios. For example, the SetForegroundWindow function which brings the thread that created the specified window into the foreground and activates the window. Keyboard input is directed to the window, and various visual cues are changed for the user. You may find the Win32: Bring a window to top thread helpful.
Try Application.ActiveDocument.Activate.

Getting a console window out of "select mode" to prevent a crash

In my C# application, I am trying to prevent a crash in my application, basically, I am using a Console window as a log display window. I already solved the "Close Button" issue by disabling the close window, and I show/hide the menu with Show and Hide calls, all of that is working just fine.
My final hurtle is if Text Selection is active and the window attempting to be hidden.
I either need to:
A. Kick the window out of select mode. (not sure how I would do this, since selection pauses all output.)
B. Disable the "Edit" menu in the same way I disabled the Close menu, in a hope that it would also disable the mouse selection, but I have yet to find any way to remove the "Edit" menu, and I am not even sure that would prevent mouse selection.
C. This seems the most obvious, disable mouse selection, and this is the one I have currently in my code, but it's not working, so I am not sure what I am missing.
The Code in Question:
[DllImport("user32.dll")]
private static extern bool ShowWindow(IntPtr hWnd, int nCmdShow);
....
[DllImport("Kernel32.dll", ExactSpelling = true)]
private static extern int SetConsoleMode(IntPtr hWnd, int wFlag);
....
public static void HideConsoleLog () {
if (handle != IntPtr.Zero) {
SetConsoleMode(handle, 0x0080); // ENABLE_EXTENDED_FLAGS 0x0080
ShowWindow(handle, (int)WinCntrlOpt.SW_HIDE);
LogVisible = false;
}
}
According to the Documentation:
ENABLE_EXTENDED_FLAGS 0x0080 Required to enable or disable extended
flags. See ENABLE_INSERT_MODE and ENABLE_QUICK_EDIT_MODE.
And...
ENABLE_QUICK_EDIT_MODE 0x0040 This flag enables the user to use the
mouse to select and edit text.
To enable this mode, use ENABLE_QUICK_EDIT_MODE |
ENABLE_EXTENDED_FLAGS. To disable this mode, use ENABLE_EXTENDED_FLAGS
without this flag.
So, it seems I am doing this correctly, and it does run, but it does not prevent mouse selection. I do know that not to long ago, Microsoft tinkered with Consoles, and I don't know if that's broke this or not.
There is a similar question on stack exchange, but that nether gives a working solution, and is in-fact the reverse to what I am trying to do and in C or C++. I can interpolate, but it's basically what I am already trying here.
It seems to be the same suggestion, use ENABLE_EXTENDED_FLAGS without any other flags to disable the option, and that's what I have. But it doesn't work.
So, I am either scratching at the wrong solution or am not doing something else correctly.
The effect is, while the console window is open, you can use the mouse to select text which automatically puts the console in that "select mode" that Microsoft's tinkering added. That's not a problem, but it's when it's left IN that mode, and you select "Show Log" (a check-marked toggle menu option in the main application window) The console window does indeed hide, but then the main window locks up and the application crashes. (my guess is because the window is blocked by that select mode.) It doesn't crash when the 'select mode' is not active.
I would also like to try this from the same documentation:
ENABLE_MOUSE_INPUT 0x0010 If the mouse pointer is within the borders
of the console window and the window has the keyboard focus, mouse
events generated by mouse movement and button presses are placed in
the input buffer. These events are discarded by ReadFile or
ReadConsole, even when this mode is enabled.
Problem is, it says:
When a console is created, all input modes except ENABLE_WINDOW_INPUT
are enabled by default.
So, how do I disable it?
I am fine with any solution that either kicks it out of select mode, or prevents it to begin with, and I have here what I have tried.
When the "Show Log" menu option is selected, the window is going to minimize.
Before the window minimizes, you have to check whether the user has selected any input to prevent the crash. You can check whether the user is selecting anything by using GetConsoleSelectionInfo.
The CONSOLE_SELECTION_INFO out parameter should be equal to 0x00, and if it's not, you need to process the selection. As the documentation from GetConsoleMode/SetConsoleMode shows, you either need to call ReadFile or ReadConsole to discard the selection event that might be going on.
When you've implemented this, it shouldn't crash anymore.

Invoke on click on a button using UI Automation with no InvokePattern or clickablePoint

i'm try to send click message to (or invoke) a button in another application .
i used UISpy.exe and could find the element which i need.
but it has no id,no clickablePoint and no Invoke pattern.
i tried following code:
var processStartInfo = new ProcessStartInfo(#"tdesktop\Program.exe");
var proc = Process.Start(processStartInfo);
Thread.Sleep(3000);
AutomationElement mainWin = AutomationElement.RootElement.FindChildByProcessId(proc.Id);
List<AutomationElement> elmList= GetChildren(mainWin);
//MessageBox.Show(elmList.Count.ToString());
if (elmList.Count == 7)
{
List<AutomationElement> menubar= GetChildren(elmList[6]);
AutomationElement elementNode = menubar[1];
double x = elementNode.GetClickablePoint().X;
double y = elementNode.GetClickablePoint().Y;
win32 w = new win32();
w.move_left_click((UInt32)x, (UInt32)y);
}
it throws an exception in elementNode.GetClickablePoint().X that the Autumation Element has no clickable point.
i tried also TryGetInvokePattern() but still throws execption it has no InvokePattern.
i use VS2012 and .net 4.5
is there any way to make this?
As was already suggested, I'd strongly recommend pointing the Inspect SDK tool to the UI you're interested in. The tool (inspect.exe) be found in places like "C:\Program Files (x86)\Windows Kits\8.1\bin\x64".
By using the tool, you can see how the UI element of interest is being exposed programmatically. For example, is it exposed as a single UIA element, or it is part of a larger UIA element which represents a set of UI elements shown visually, and what UIA patterns does it support? As a test, I just pointed Inspect to an arrow shape in Paint. The results are shown below.
So I can tell that the arrow is exposed programmatically as a single UIA element, and that it supports the Invoke pattern. This means I can programmatically invoke the button through UIA. (And I can call the pattern methods on the element from inside Inspect’s Action menu, which is pretty handy.) If the UIA element didn't support any patterns that would allow me to programmatically control it, I can find its BoundingRectangle property through UIA, and simulate a mouse click in the middle of that to invoke it. (And I'm assuming the button's not obscured when I simulate the mouse click.)
But if I look at another group of elements shown visually on the screen, using Inspect I can learn that the whole set is exposed through UIA as a single UIA element. So in the image of Inspect shown below, I can learn that I'm not able to programmatically invoke a specific color in that group.
So in that case, I'd probably have to assume I know the size and layout of the UI elements shown visually in that group, and simulate a mouse click at some point which I think is appropriate based on the color I want to invoke.
By using Inspect, I can get a good understanding of what my options are. Ideally a single element shown visually on the screen will be exposed through UIA as a single element that I can control through whatever patterns are relevant, (for example, Invoke, Toggle, SelectionItem etc). But if useful patterns aren't supported, then I could consider simulating mouse clicks instead, based on whatever ClickablePoint or BoundingRectangle data's exposed.
Thanks,
Guy
A menu bar doesn't expose the InvokePattern (see UI Automation Support for the MenuBar Control Type). However, a menu item can be Invoked (see UI Automation Support for the MenuItem Control Type).
The following code illustrates how to generate a list of menu items:
AutomationElementCollection items = menubar.FindAll(
TreeScope.Children,
new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.MenuItem));

Microsoft UI Automation/Get The Item That The User Clicks

I am trying to figure out what item (for example document, web page tab, window, picture, folder) the user has clicked on. I started by using the following code when I detect a global left mouse click:
System.Drawing.Point MousePoint = System.Windows.Forms.Cursor.Position;
AutomationElement AutomationElement = AutomationElement.FromPoint(new System.Windows.Point(MousePoint.X, MousePoint.Y));
Console.WriteLine(AutomationElement.Current.Name);
This seems to work well in most conditions. However, I need to (if possible) get names of documents/images/folders inside Windows Explorer for example. The value returned when I click a document in the right hand pane of Windows Explorer (not the tree view) is "Name". Is there anyway to get the actual document name? For some reason, clicking sub-folders in the tree view returns the name of the folder, which is what I want.
I also notice that the code seems to display the document/image/folder name when clicked if the Windows Explorer view is set to icons (medium, large or extra large). Is there any reason why other views return "Name" or empty string while medium, large and extra large icons return the actual document/image/folder name? Is it to do with the size of the object clicked? I could really do with a way round this if possible?
I apologise, I am new to UI Automation and just really want a way to find the name of the object (file, folder, document, picture, web page tab etc.) that the user has clicked on. Any help anyone could give would be great.
You need to listen to InvokePattern.InvokedEvent UI automation event.
Example
The following example assumes you have an open instance of windows "Calculator" app. Then if you run the application, when you click on any button in Calculator, we handle the click event and show what button has clicked.
[DllImport("user32.dll")]
private extern static IntPtr FindWindow(string lpClassName, string lpWindowName);
protected override void OnLoad(EventArgs e)
{
base.OnLoad(e);
var proc = System.Diagnostics.Process.GetProcessesByName("Calculator")
.FirstOrDefault();
if (proc != null)
{
var mainHwnd = FindWindow(null, "Calculator");
var mainWndElement = AutomationElement.FromHandle(mainHwnd);
Automation.AddAutomationEventHandler(
InvokePattern.InvokedEvent, mainWndElement,
TreeScope.Subtree, (s1, e1) =>
{
var element = s1 as AutomationElement;
MessageBox.Show($"'{element.Current.Name}' clicked by user!");
});
}
}
protected override void OnFormClosing(FormClosingEventArgs e)
{
Automation.RemoveAllEventHandlers();
base.OnFormClosing(e);
}
You need to add reference to UIAutomationClient and UIAutomationTypes assemblies.
Note
You can easily extend the example and listen to Win32_ProcessStartTrace to detect when Calculator opens. You can see an example to detect when an external process window closes which is using Win32_ProcessStopTrace.
Another option is using SetWinEventHook you can listen to some events from other processes and register a WinEventProc callback method to receive the event when the event raised. Here, you are interested in EVENT_OBJECT_INVOKED event.

Windows API Call CB_SELECTSTRING

We have a C# 2012 WCF service that is interacting with a desktop application that is built in VB6 no less and are having issues with a drop down selection using Windows API calls.
The way the application works when you are actually working within it is that you select an entry in a drop down box first. Then enter information in text boxes below it. You select another entry in the drop down box, the fields reset to blank text boxes. The code behind on the downstream application associates the text box information with the selection in the drop down. That is way that particular app works.
Our problem is automating this process. We use Win32 API calls to set information in the drop down using the following:
[DllImport("user32.dll")]
public static extern int SendMessage(IntPtr hWnd, int uMsg, [Out] int wParam, string lParam);
// Actual call to the downstream app.
SendMessage(hWnd, (int)CB_SELECTSTRING, 0, strValue);
The problem is when we try to set the second value in the drop down box. The text boxes are not being reset. We have tried using the following calls as well:
SendMessage(hWnd, (int)CB_SELECTSTRING, -1, strValue);
SendMessage(hWnd, (int)CB_SETCURSEL, 0, strValue);
SendMessage(hWnd, (int)CB_SETCURSEL, -1, strValue);
None of these messages are accomplishing the task. We need the call to simulate the Selected Index Changed event. Are we not using the right piece here? Or do we need to send a message to the parent to fire off that event?
I looked through our solution and did not find CBN_SELCHANGE anywhere or any API call that would send that to the parent.
Any ideas here folks?
I am not sure that the notification CBN_SELCHANGE should be fired in this case. Once you change something in the combobox programmatically, it will be easier for you to change whatever else on the form from the next line of your code. Doing same thing threw the notification only complicates the code. Think as if you are writing the code of this form.
Once you are able to access the hWnd of the combobox, you maybe should update your entry fields as well using the same SendMessage.
Other approach - send the CBN_SELCHANGE yourself. There is nothing against that. This is simply a message.
The CB_SETCURSEL is not using the LPARAM. You should pass 0 there.
You should also recheck how .Net marshalls your string. I am not sure this will be so simple. To verify this build your own small dll, check definition to DllImport("myUser32.dll") and put a breakpoint in the debugger at the entry of your dummy SendMessage. You will see what comes to the native code.

Categories

Resources