I am working with MS UIAutomation on windows 7 (64bit). I want to know if it is possible to make the UIAutomation element persist, if so what could be the starting point to do it.
The only thing I came up with is RuntimeID of an element, but the problem is that some elements may not have an id.
A simple scenario I want to achieve is that I find out the UIAutomation element of a control on the screen by using the AutomationElement.FromPoint(), now I want to persist it, so that next time I can easily access it by "Loading" it back to a UIAutomation Element.
No, you cannot persist the UIAutomation element from the automation client. You will need to store sufficient information about the element (the AutomationID and context are the best, if available) to find it again the next time.
The element is a reference to the object in (probably) some other process, and it can only live so long as that other process lets it. Elements are likely to go away when their windows close, the app closes, etc. Elements such as menus and flyouts are likely to be very short lived, and may no longer exist by the time the client receives a notification.
Related
I'm currently using a delegate to poll the current active control from the currently focused external form every few milliseconds, which works badly.
The title of the question is somewhat confusing, as this does not actually have anything to do with Application.OpenForms. That property just exposes a collection of all the currently open forms. If a new form is opened that had not previously been opened, it will be added to the collection. But the operating system allows only one active window at a time on the desktop, so what you really want is to get notified when the active window changes. Polling will work for this, but it isn't a good solution.
You will either need to install a global hook (WH_CBT is likely the one you want), or take advantage of the WinEvents infrastructure (via the SetWinEventHook function) intended for accessibility tools. Both of these will provide you with a notification when the active window changes. Of course, their scope won't be limited to the application that you are testing. You'll get notification when the active window on the desktop changes to any other window. It will be up to you to filter that down to the application you care about.
I don't have time at the moment to translate any of the required code to C#/.NET, but you can probably find it somewhere online, once you know what to search for. Additionally, WinEvents is already wrapped for you in the System.Windows.Automation namespace.
In my WPF application, I have multiple classes each of which implements a message loop. All of these message loops then run in parallel when I create objects of these classes. These objects may show a dialog box at times. The main application thread must keep an eye on these objects and make sure none of them is stuck with the dialog (and press Cancel (Escape key) if it determines such cases). What is the correct way to determine the active foreground window of a thread? I know there is a GetForegroundWindow() API, but it appears as if it works at system level and not thread level.
There's no such thing as the per-thread active foreground window. So what you are explicitly asking for does not have an answer.
Probably the right way to go here, using Win32 at least, is to enumerate top-level windows with EnumWindows. Then use GetWindowThreadProcessId to identify that the window is associated with one of your threads. Finally use GetClassName to identify that the window is a file dialog. Then feel free to do whatever dastardly thing it is you want to do to the window!
On the other hand, this sounds like a perfect candidate for UIAutomation. You are automating testing of UI. UIAutomation will be able to find these file dialog windows and press buttons on them.
So, I have a [.NET] program that implements a NotifyIcon that persists in the Notification Tray in Windows for the entire time that the program runs. When I first execute the program, and for a small time afterwards, the NotifyIcon and associated ContextMenuStrip work an absolute charm.
However, after some time of the machine being in a lower power state or just idle for a few hours, when trying to access the Context Menu, it can take up to 2 seconds for the object to draw.
Is there some persistence need to define to get the menu to respond faster? Have I enabled an incorrect event that is firing a cleanup of resources?
Edit
It would seem that I have 3 options to see if I can get this to work:
1. Insert a SecureString object into the ContextMenuStrip object; this assumes that by having the SecureString inside the ContextMenuStrip, the whole object will be treated the same and will not be page swapped
2. Create a Timer that touches the ContextMenuStrip on interval in some manner to keep it in memory
3. Alter Process.MinWorkingSet when the app has completed initial launch to try (read: hope) that the ContextMenuStrip object is kept alive and not something else
I really am not a fan of items 2 or 3; 3 especially. I am going to try item 1 and see if it works.
Thanks for the responses so far!
From my testing it would appear that option 1 outlined in the questions edit has done the trick ... for the most part.
As has been stated, and of which I already understood, C# is a managed language; making memory management difficult.
However, by adding a new SecureString() object to one of the ContextMenuStrip menu items' Tag attributes, the menu object will persist in RAM.
There are, sadly, two primary caveats to this approach:
1. If the machines RAM gets hit HARD, the SecureString object is either forcibly discarded or decoupled from the encompassing object and the page fault/swap is performed. I am unsure as to whether it re-couples together again post scenario, but it seems to do ok.
2. If the machine is put to sleep or hibernate, the whole app gets page swapped and the SecureString is most definitely discarded. To aid in these situations, I have a SessionSwitchEventHandler that detects if the machine has been unlocked and creates a new SecureString() object in a menu items' Tag attribute and re-associates the ContextMenuStrip object to the NotifyIcon.ContextMenuStrip
Although it's not the best solution, it does work and is better then creating a timer to touch the menu [shudders] or, worse, playing with the Process.MinWorkerSet [takes a shower at the thought]
Thanks again to those who took part in the question and helped think further to find a solution.
im stuck in my problem, i have invoke a process and now i want its focused elements.
for example. i run a process www.google.com that opens up in Google chrome
now lets say i want to search "hello" the way of doing this as far as i know is to get the keyboard focus element that will be a textbox of google.com .so how to do that through my application i.e programmatically(getting keyboard focused element of chrome process)
Process[] pr = Process.GetProcessesByName("chrome");
var target = pr[0];
Your question a is a bit vague, so I'll try to give several pointers based on several different interpretations:
1) If the process you're referring to is always a web-site, you should skip the screen-scraping techniques, like #vulkanino mentioned, and simply fetch the contents of the web-site directly. If you must get the website as parsed by a real browser (rather than the raw HTML itself), you can use screen-scraping, but remember that not all computers use the same browser.
2) If you want to screen-scrape information from a running process, you (unfortunately) don't have a simple API like the one you expected, where the Process pr object contains a list of elements you can traverse. This is because the process might have been written in .NET/WinForms, .NET/WPF, C++/MFC or any of a dozen other UI frameworks that work very differently.
What I would suggest is using a screen-reader accessibility framework, such as Microsoft's UI Automation, which is able to extract text from the UI of many common applications. This functionality exists to help read out application UI for the hard of sight, but can be used for screen-scraping as well.
I have a windows form application which needs to be the TopMost. I've set my form to be the TopMost and my application works as I'd like it to except for in one case.
There is a 3rd party application (referred to as player.exe) that displays SWF movie files on a portion of the screen that popup on top of my application.
Using Process Monitor I determined that player.exe application calls
flash.exe <PositionX> <PositionY> <Width> <Height> <MovieFile>
in my case:
flash.exe 901 96 379 261 somemovie.swf
Since flash.exe is being spawned in a new process after my form has been set to the TopMost it is appearing on top of my application.
First thing I did was make my application minimize the player.exe main application window hoping that this would prevent the Flash from appearing also. But, unfortunately it doesn't... even with the window minimized whenever the flash movie starts it shows up at the pixel location (901,96). I then tried creating a timer to keep setting the form.TopMost property to true every 10ms. This sort of works but you still see a very quick blip of the swf file.
Is there some type of Windows API call which can be used to temporarily prevent player.exe from spawning child processes which are visible? I admit it sounds a little far fetched. But, curious if anyone else has had a similar problem.
Addendum:
This addendum is to provide a reply to some of the suggestions layed out in Mathew's post below.
For the emergency situation described in the comments, I would look at possible solutions along these lines:
1) How does the third party application normally get started and
stopped? Am I permitted to close it
the same way? If it is a service, the
Service Control Manager can stop it.
If it is a regular application,
sending an escape keystroke (with
SendInput() perhaps) or WM_CLOSE
message to its main window may work.
Easiest way to close the app is to CTRL-ALT-DEL, then kill process. -OR-
The proper way is to Hold ESC while clicking the left mouse button... then input your username and password, navigate some menu's to stop the player.
There is no PAUSE command... believe it or not.
I don't think using WM_CLOSE will help since minimizing the application doesn't. Would that kill the process also? If not, how do you reopen it.
2) If I can't close it nicely, am I permitted to kill it? If so,
TerminateProcess() should work.
I can't kill the process for two reasons. 1) Upon relaunch you need to supply username/password credentials... There may be a way to get around this since it doesn't prompt when the machine is rebooted but... 2) Whenever I kill the process in task manager it doesn't die gracefully and asks if you want to send an error report.
3) If I absolutely have to leave the other process running, I would try
to see if I can programmatically
invoke fast user switching to take me
to a different session (in which there
will be no competing topmost windows).
I don't know where in the API to start
with this one. (Peter Ruderman
suggests SwitchDesktop() for this
purpose in his answer.)
I got really excited by this idea... I found this article on CodeProject which provides a lot of the API Wrapper methods. I stopped implementing it because I think that in order for desktop's to work you must have explorer.exe running (which I do not).
EDIT2: On second thought... maybe explorer.exe isn't needed. I'll give it a try and report back.
Edit3: Was unable to get the code in that article working. Will have to put this on hold for a moment.
Answer Summary
As one might have expected, there is no simple answer to this problem. The best solution would be to problematically switch to a different desktop when you need to guarantee nothing will appear over it. I was unable to find a simple C# implementation of desktop switching that worked and I had a looming doubt that I would just be opening a whole new set of worms once it was implemented. Therefore, I decided not to implement the desktop switching. I did find a C++ Implementation that works well. Please post working C# virtual desktop implementations for others.
Setting the TopMost property (or adding the WS_EX_TOPMOST style to a window) does not make it unique in the system. Any number of topmost windows may be created by any number of applications; the only guarantee is that all topmost windows will be drawn 'above' all non-topmost windows. If there are two or more topmost windows, the Z-order still applies. From your description, I suspect that flash.exe is also creating a topmost window.
Aside from periodically forcing your window to the top of the Z-order, I think there is little you can do. Be warned, however, that this approach is dangerous: if two or more windows are simultaneously trying to force themselves to the top of the Z-order, the result will be a flickering mess that the user will likely have to use the task manager to escape.
I recommend that your program not attempt to meddle with other processes on the computer (unless that is its explicit purpose, e.g. a task manager clone). The computer belongs to the user, and he may not value your program more highly than all others.
Addendum:
For the emergency situation described in the comments, I would look at possible solutions along these lines:
How does the third party application normally get started and stopped? Am I permitted to close it the same way? If it is a service, the Service Control Manager can stop it. If it is a regular application, sending an escape keystroke (with SendInput() perhaps) or WM_CLOSE message to its main window may work.
If I can't close it nicely, am I permitted to kill it? If so, TerminateProcess() should work.
If I absolutely have to leave the other process running, I would try to see if I can programmatically invoke fast user switching to take me to a different session (in which there will be no competing topmost windows). I don't know where in the API to start with this one. (Peter Ruderman suggests SwitchDesktop() for this purpose in his answer.)
You can use the Process class to start flash.exe directly - and use an appropriate ProcessStartInfo settings to show the window in a hidden state - or with a WindowStyle of hidden or minimized.
You could also consider using the SetWindowsHookEx API to intercept the process start API calls, and when the process is flash.exe run some code to restore you window to top-most status.
Matthew's answer is excellent, but I suspect you may be asking the wrong question. Why does your application need to be topmost? If you're trying to create a kiosk or some such, then topmost is not the way to go.
Edit: After reading your response to Matthew's comment, I'd suggest creating a new desktop and switching to it before displaying your alert. (See CreateDesktop and SwitchDesktop in MSDN.)