I want to automate simple Windows Application using MSAA. I am not getting Idea that how to find the control (say TextBox and Button) and to get value of them(say get value of TextBox and click in case of button).
Why do you want to automate using MSAA? Microsoft's current way is UI Automation.
From Wikipedia (emphasis mine):
UIA is similar to Microsoft Active Accessibility (MSAA) in that it
provides a means for exposing and collecting information about user
interface elements and controls to support user interface
accessibility and software test automation. However, UIA is a newer
technology that provides a much richer object model than MSAA...
With UI Automation you can automate by creating a native or .NET application or use a 3rd party tool like RIATest that relies on UI Automation.
(Disclamer: I am affiliated with RIATest).
The following assumes you'll be developing your automation program in C/C++, as I'm not sure it would be easy to use MSAA from .NET (it might be possible though!)
If you don't want to use C++, you might investigate using MS UI Automation from .NET.
To get started developing an automation client app using MSAA and C++, I suggest:
Install the Windows SDK for Windows 7 and .NET Framework 4
That will install the Inspect.exe tool - find it under:
Programs > Microsoft Windows SDK v7.1 > Tools > Inspect Objects
Launch Inspect.exe and switch the upper left dropdown menu from "UI Automation" to "MSAA"
Launch your application under test and mouse over its UI elements - the MSAA properties of the elements should display in Inspect.exe
Write your automation code :)
An important choice will be to decide whether to use event-driven techniques to locate your elements, or simply start at the top level (the desktop), find your app among the child elements of the desktop, and drill down into your app's element tree from there.
There's a good event-driven MSAA tutorial on codeproject.com at: http://www.codeproject.com/Articles/18290/Introduction-to-Microsoft-Active-Accessibility
Tutorials on tree-based navigation in MSAA seem a little more scarce. I would start with the MS docs - I recommend the first one:
Active Accessibility Client Interfaces and Functions
Active Accessibility User Interface Services
Active Accessibility Start Page
Related
Is it possible to write a code that will copy text values from a window belonging to another application?
I have an application that gives me live results(only texts), every 5 minutes, and I cannot copy paste them every 5 min.
Maybe.
It depends on how the target application is exposing its text to the OS.
If the application is using a private 2D/drawing library to render text by itself to an in-memory or in-VRAM buffer, then no. You'll need to grab a screenshot and perform OCR on it - or you could inject your own code into the target process and intercept those 2D/drawing library calls to get the text being rendered.
If the application is using the Windows-provided GDI then there are ways of intercepting those calls to get the text. I believe Direct2D and DirectWrite also offer straightforward ways of intercepting/profiling their calls as well.
If the application is using a GUI framework or platform like WinForms or WPF then there are ways of inspecting the rendered view's object-model to extract data and text - this is how various "Spy" utilities work. "Spy++" (spyxx.exe included in Visual Studio and the Windows SDK) can inspect native Win32 hWnd windows and "Snoop" is a very powerful tool for inspecting WPF applications (Visual Studio's built-in Visual Inspector does the same thing).
Additionally, often GUI frameworks and platforms will support the OS' built-in Accessibility platform and will expose on-screen data as machine-readable structured data for use by screen-readers for the blind and visually impaired as well as automation software. Windows' built-in platform is called Active Accessibility and Windows UI Automation. There are premade tools you can download to inspect Active Accessibility data.
If it's a HTML application (e.g. Windows HTA, Electron app, Chrome desktop app, etc) then that's another topic.
EDIT 1 2017/01/29 - At the moment our choice has gone to electron in combination with EdgeJS and all the power of NodeJS and the packages/resources on npm there.. There is no better alternative for us to create cross platform apps for all three plyers (macOS, linux and windows). Thanks everyone for their input.
ORIGINAL Question:
TL;DR - Backend code written in C# (.Net/Mono). Writing a desktop application that needs low level access to the system (cross platform OSX, Linux, Windows 7 to 10). In search of a one-stop cross platform UI tool/framework that can interface with the status of the backend-code, either by polling/messaging or calling into the code of the backend code. This "backend" includes for example the scanning & listing of process info, taking screenshots, etc.. Sending network information (which websites were visited etc..) It's in essence an exam monitor for students when they're doing an online exam in classrooms.**
Options we are exploring:
electron (formerly known as atom shell) looks nice, but we have no idea whether it can interop/call into/kickstart a C#/Mono application. Posted this as well on their official forum for feedback, and whether it is a good use case for electron. In an ideal world, we'd like to use electron fully, but unfortunately I think we need platform specific logic (which now we handle in C# w/ .net & mono).
Eto.Forms is what we work with now, but is limiting in it's UI's capabilites - as sometimes it still renders differently (sizing of windows) on different platforms. It also has a different look & feel that's tied to the platform of the user - which is a good feature, but it makes it difficult to make dedicated controls to visualise status, which would be easier in html/css.
Another route, is a mix between Eto.Forms (or something else) to bootstrap a native UI that hosts a browser in it's window that cross platform and not dependent on the locally installed browsers, such as Awesomium. This would still require Eto.Forms, or any other bootstrapper to kickstart the application, but all other navigation would go through the Awesomium rendered html.
Thanks,
Yves
AvaloniaUI (A cross platform XAML Framework for .NET Framework, .NET Core and Mono) would be a perfect choice if you are familiar with WPF.
There are many other frameworks to develop cross-platform desktop apps. You can checkout my github repo electron-alternatives for more cross-platform desktop development options.
Xamarin Forms is my framework of choice. https://visualstudio.microsoft.com/xamarin/ It's made for mobile but was extended to include desktop as well.
I'm attempting to write a application in C# utilizing the Microsoft UI Automation framework. The application to be controlled uses a component called "FarPoint Spread". It's easy enough to automate the rest of the application (e.g. its WinForms controls), but I can't seem to do anything with the FarPoint Spread control.
I'm still very green when it comes to programming. I've got just the basics of C# and UI automation down. Just need someone to point me to the information I need to have to automate this Spread component (if that's even possible). Thanks!
The problem is the grid rendering in Spread is written probably using custom code, and it's not equipped with what UI Automation expects.
I have tested the Sample Explorer provided by GrapeCity here (http://spread.grapecity.com/Pages/Demos/) with UI Automation's Inspect tool, and once you get to the data grid itself, there's nothing shown in Inspect.
There's also this topic in GrapeCity's forums:
Is Spread even supported in coded UI test which clearly states (2 years ago) it's not supported (as you known VS Coded UI also uses UI Automation as the underlying technology).
So, I guess it's not possible to automate Spread using UI automation (maybe they have their own system, you'd have to contact them).
I use UI Spy and Spy++ to view windows text of some windows, but none offers a result. As people said, some windows are built up and executed by some components, say they are embedded inside those components. Except the two tools to view the window handles, text, could you introduce me other tools or ways to get a window text or handle without using hooks ?
Edit: I also use Spy Object to retrieve windows objects information and search from ancestors till I can meet the right object but this tool to the particular windows I am watching is incapable to do the job because it also reports incorrect parents positions in ancestor hierarchy, I try out the MAAS and UI-Automation without luck, Thank you for you help.
Edit3: I don't like their windows'ideas of internal designs (I mean only they-designers- know how to untie the knots)
Usually the Accessibility APIs are used, the ones used for text-to-speech screen readers for blind users. In your question you mentioned MAAS, did you mean MSAA (Microsoft Active Accessibility)? If not, that's where I'd look next.
http://white.codeplex.com/ is a great framework which does such things.
White is a framework for automating rich client applications based on Win32, WinForms, WPF, Silverlight and SWT (Java) platforms. It is .NET based and does not require the use of any proprietary scripting languages. Tests/automation programs using White can be written with whatever .NET language, IDE and tools you are already using. White provides a consistent object-oriented API, hiding the complexity of Microsoft's UIAutomation library (on which White is based) and windows messages. Only stable versions of White are released, so all releases are production-ready.
I want to know how can I build UIs like skype using standard .Net/C#. Is it possible at all?
Thanks
You can use Windows Presentation Foundation to build more stylish GUIs than Windows Forms. It's pretty difficult to move from Forms to WPF. You usually need a good design tool, like Expression Blend.
AFAIK skype was built using Qt4, it's rather easy to build custom gui widgets, check
C++ GUI Programming with Qt4, 2nd Edition and this tutorial.
P.S. check this to see how to build qt4 on windows using MSVC 2008.
It might be possible to build such an UI using Windows Forms, but only with a lot of custom control code or a really good component suite. But as Chris said, WPF should be the tool of your choice when you want to use .NET.
Qt4 is a really powerful C++ framework, also powering apps like Google Earth. The Qt SDK, inclduing everything you need to get started (Compiler, IDE, documentation), can be obtained here. It's licensed (among others) under the LGPL.
Windows Presentation Foundation will give you the most flexibility, but it can be hard to use. An alternative to this is Sliverlight - Version 3 is going to be capable of being used outside of the browser (similar to Adobe Air), and in some ways provides an easier development experience. You might want to consider using some third party controls to give you access to nice functionality that you wouldn't get out of box.
To my acknowledge Skype is build with Delphi an a set of third party tools, so it for sure can be done without WPF.
Take a look at the components from DevExpress or TMS.