I'd like to be able to do following actions within another application:
to change Tabs
to Copy text from within a TextBox
to click on a Button
to enter text into a TextBox
to select DropDownList element
Right now I'm using separate methods such as:
mouse_event() to change mouse coordinates and click on a button
another mouse_event(LeftMouseClick followed by the RightMouseClick) to copy a text within a TextBox
Clipboard.GetText(System.Windows.Forms.TextDataFormat.Text) to Copy what's inside the clipboard
SendInput (for each key) - enter the text into a TextBox
Disadvantages of this approach are:
(not crucial) PC becomes unusable (you can't work while script is running)
I have to know exact pixels (read - position/coordinates) of EVERY element within an app
slow execution time (each key has to be typed separately)
I'm looking forward to create an application which can click on a TextBox/Button/List without the need of having exact coordinates of these elements.
Is such task possible with C# WinForms? My current approach works but it has it's flaws.
Any advice?
Read about Mutex. If you plan on scale-ability, read about network communication (for example, the TCP protocol. TcpListner, and TcpClient).
This sounds like a job for UI Automation. I have only used it to get text from another application, but it has functions to activate controls by name or to navigate the control tree if there is no name.
You can get text and interact with controls using AutomationElements that you find using patterns or navigating the control tree.
There is a complete framework for doing exactly this kind of thing, and it's called the UI Automation Framework
Here's some examples on how to use it.
And you can also apply this technique to generic windows's using the UI Spy to determine the automation elements.
Related
In BluePrism it is possible to spy buttons, fields etc. of any windows application and then click those buttons automatically. For example, if you look at the screenshot you can see the attributes of the calculator's button "1" as spied in the Win32-mode (there are also HTML-, Accessibility-, Region- and UI Automation-modes) by which BluePrism will be able to identify and click the according button. Furthermore, BluePrim is compatible with C# and I wonder whether (and how) I could also get those attributes via C# instead or at least use the spied attributes as can be seen on the screenshot to press the same button with C# instead?
EDIT:
It would be highly appreciated if someone could provide the C# code for pressing the calculator's button "1" to get me started (I'm a C# newbie :-) Thanks!
You do not need C# for that, you can just use a Reader stage and action Get AA Attribute or Get Window Attribute.
EDIT:
But if you really want to do it using .NET only, have a look at UI Automation or Active Accessibility Automation (older one)
This seems like a nice example.
You will also find a tool called Inspect.exe very handy to spy elements without using BP. This
You can use the Application Model that you just spied, put it in a navigate stage in your object, then select Click Mouse Centre as the action. This will result in the button you have spied being clicked. Generally, to use Sendkeys is with C# syntax, i.e. use the root element (root element and top of list in Application Modeller tree) and select Global SendKeys as the action inside a navigate stage. In the Text input field you can enter your C# sendkey code, like for example for Ctrl + Alt + Delete use: "^%{DELETE}". I think this is what you may be looking for? Otherwise, if I understand correctly, you might be looking to use a C# code-stage inside BP to pass sendkeys to an application attached to a BP process, which would possibly involve referencing each and every attribute of the host application in C# and then using (for example) the C# Enum properties of each keyboard key, etc... this would be a massively redundant exercise since BP already provides the functionality inside the navigate stage as explained above.
So, I am trying to use UI Automation(Specically White library) to simpulate multiple automations at the same time. I would not like my mouse to be taken over. Is there anyway to do this? Basically, I want an instance of my UI Automation to use a virtual mouse specifically for that program, instead of my main mouse.
Not really clear on what you mean. The White library uses UI Automation elements to interact with desktop applications. Once you capture the element you can send any sort of command to it you like such as click and it won't actually use your existing mouse to click on it. Sounds like you are automating your mouse instead of automating the application.
How does one go about reading another applications listview and sub-listviews? This program has a main listview with a few other mini listviews once an item is clicked on the main view.
I want to create an application that can read the all of the listviews, how do I go about doing that? What is the best way to achieve this?
There are a number of ways to do this.
The supported way is to use an automation API like UIAutomation which is very simple to program. If the other application is using the standard Windows list view control then it should be accessible through UIAutomation. If the other application is using non-windowed controls (e.g. WPF, Qt) then you are dependent on the application implementing support for UIAutomation.
The other commonly attempted approach is to send windows messages to the list view control. This first of all involves enumerating the child windows of the application's main window in order to find the list view window. You can then send the control messages to obtain its contents. This sounds easy, but is actually rather tricky. The reason is that the messages require you to supply pointers to structures that are meaningful in the other process address space. This involves calls to OpenProcess, VirtualAlloc, WriteProcessMemory, ReadProcessMemory etc. It's quite tricky to get it right.
In my view, you should choose the automation API.
We are currently looking to create a text-editor in WPF (.NET 4.0) which will allow writers within our team to create movie scripts. In short, the functionality should ressemble that of FinalDraft or Adobe Story (i.e.: contextual positioning of text depending on the cursor's position and user intentions)
We are currently looking at two different solutions design-wise:
One WPF control which will act as the container, and multiple small text-editing controls which will represent rows within the script. This will allow us to position the controls using their margin, while also making binding easy. The challenge here would be the handling of multi-line selections. I was thinking of using a Listbox as the container, and each listbox item would be a custom control containing a textbox. This would require the instantiation of controls depending on the user's action. Everything would be skinned to give the impression that the user is working on a blank page.
One big textbox capable of displaying custom XML data. The challenge here would be to determine where exactly the cursor is located (i.e.: is the cursor on top of an actor's name, etc.) and positioning the text appropriately (i.e.: actor names are centered and in caps, etc.)
I recently tried implementing the first solution, but having to re-implement the whole selection behavior that is built-in in basic text boxes is non-trivial and requires a lot of work. As for the second solution, binding to my business objects will be much harder than simply instantiating multiple controls with different bindings.
Do you have any other solution in mind ?
I needed a text editor for a application once. We had a big xml file for settings and the user should be able edit those.
Turns out , if your file is large enough (+ 10000 lines) the rich text box is getting pritty slow.
As for building a gui mask : only if your user wirtes some short options like text. But is i understand you want your useres to write creativ text. This "mask" gui - "lot of small places" will make them feel like they are in the 80ties.
I suggest: Dont write the Programm , only write a Plugin to an exitings editor. Some are free like:
http://www.codeproject.com/Articles/42490/Using-AvalonEdit-WPF-Text-Editor
or an add in for word - people love Word and know Word
http://www.codeproject.com/Articles/8837/Writing-a-Word-Add-in-Part-I
And for binding data and the like: Once the text is written, the user just need to press the save button and you can parse the input for information. I would not do it on the fly as it can get pretty slow. Also you say that the information is linked so only if all the data is written you can make use of it.
I need to create a dialog that allows user to choose between several rather complex actions. I really like the usability of the windows 7 file replace dialog and I think it would suit my needs very well. Here's a screenshot for reference:
Is it possible to use the controls that were used for windows dialog? If not, how would you recommend creating UI similar to this dialog?
Seems like this could easily be done with a window containing a few labels for the text and three custom controls - with some images for the arrows and file icons - each of which changes their background image when the user mouses over them - and fires an event the window picks up on when they are clicked. Fairly standard WinForms stuff.
Is there a particular part of the process you need some extra help with? Like, for example, the mouse over?