We are looking to write automated tests using coded ui test framework. We are looking to test the ui components in isolation without launching the application in a separate process.
For example if we have a pop-up dialog in the application to capture data from user, we would like to launch only the specific dialog and validate different use cases rather than running the whole application.
We tried to test by launching the dialog as part of the test initialize() but it is not able to find the controls... but the same test works fine if I launch the dialog separately.
Have anyone tried this or have advise to get this working ?
Coded UI Framework is a very powerfull framework yet has lots (and I mean LOTS) of problems.
I wouldn't recommend it to do what you are trying to accomplish.
Besides, testing "components in isolation" is unit testing, and from my experience this is not a best practice for Coded UI Test at all.
Coded UI test will help you in testing cross-application processes from end to end, closest to the user as it gets since its simulating the users inputs though keystrokes and mouse clicks.
Also, as UI tends to change quite a lot during development, and Coded UI relies on that, I suggest you use it mainly for regression testing for windows you know that are not going to change anytime soon.
This way you'll keep you maintainance low and productivity high.
Hope this helps.
Coded UI is intended for checking the functionality of applications (and also of web pages). Coded UI is not for testing fragments of UI separate from their application. However, it would be possible to create a test harness application that contained one or more UI components allowing them to be tested in isolation from the real application.
A test harness could easily have a window for each component being tested. The window would include the component being tested plus a some other simple controls. These simple controls could expose internal values of the component being tested and also be used to pass values into the component.
I think what your attempting to do is viable, however ONLY for your own custom controls. I think you should tackle this in this order.
A second instance of VS to capture what it actually sees about your control when your test spawns your control.
Update your search criteria, e.g. process name, window title
Potentially add properties to your TestInitialize spawner so it creates your control with useful ID's, names, or titles. (not sure which tech stack your control is in)
Run tests again
Related
I've started to run into caching problems with the application I'm writing tests for and at the moment I can't see a clean solution. I'm hoping someone might be able to offer some pointers.
Some of my tests require a fair amount of test data setup before the actual UI test can be executed. To ensure the tests are atomic and can be executed in parallel, the setup data required for each test is created before the UI test is run. This is typically done via a sequence of api calls. To create this data via the UI could take upwards of 5-10 minutes. Only after all api calls have returned 200 O.K will the browser open and the 'actual' UI test start.
This approach had been working well for me until I started to notice caching issues. By that I mean data I created via the api was often not available when the UI test needed to interact with it (e.g. in a dropdown). Only after a few F5 refreshes would the data appear.
Because there is no way to 'poll' the data to confirm its existence in a dropdown, the only hacky solution is keep pressing F5 until it appears. It's not clean at all though as a refresh will remove any data you have added up to that point, and if it's a single page application, may navigate you to a different tab from where you were.
Is there anything either me or the developers can do to solve this particular problem?
Many thanks,
I have made a rather complex .NET 4.0 (C#) Windows Forms application using Visual Studio 2013. The question is quite general though, and should be applicable for other versions of .NET and VS as well.
On startup the system reads config file, parses file folders and reads file content, reads data from database, performs a web request and adds data to a lot of controls on the main startup form.
I want to avoid having a splash screen with "waiting-hourglass", the goal is to make the application startup fast and show the main form immediately.
My solution has been to use backgroundworker for some of the startup tasks, making the application visible and responsive while data are fetched. The user can then choose to navigate away from the startup form and start doing other tasks without having to wait for all the startup procedures to be completed.
Is use of backgroundworker suitable for this?
What other methods should be considered instead of, or in addition to, backgroundworker to enable fast startup for an application with a lot of startup procedures?
In my applications I use a splash screen. However, I do not show a waiting-hourglass. Instead it shows a status line where current action is written, e.g. "Read config file", "Connect to database", "Perform web request", etc.
Of course, the application does not start faster but the user does not have the feeling of a hanging program and it appears faster.
In any case it depends if early access availablity makes sense for the user. A good way would also be to just preload the first page / form / tab before the user can see the interface (Splashscreen or loading bar before that).
When the first bits are loaded you could asynchroniously cache more data and only allow the user switching pages / tabs when the caching of these components is completed (you will have to display a "still loading" message or grey out other tabs while doing this to not confuse the user).
You can also just load addditional data if the user chooses to use the page / tab / feature to reduce loading unneccesary information but this will lead to waiting while using the application - it`s up to you.
Technically, as BackgroundWorker is explicitly labeled as obsolete in .NET 4.5 you should see if the introduced await/async would be a more elegant solution for you (See MSDN Asynchronous Programming with Async and Await Introduction)
MSDN says:
The async-based approach to asynchronous programming is preferable to
existing approaches in almost every case. In particular, this approach
is better than BackgroundWorker for IO-bound operations because the
code is simpler and you don't have to guard against race conditions.
See a comparison thread Background Worker vs Await/Async
See a well commented example of backgroundworker code to load GUI data if you choose to use that technique
Rather an advice than an answer:
Is use of backgroundworker suitable for this? - yes.
What other methods should be considered instead of, or in addition to, backgroundworker to enable fast startup for an application with a lot of startup procedures? - consider on-demand a.k.a. lazy loading of data. That is, load the data only when they are actually needed rather than query everything at once possibly many of them without ever being used or looked at. If this is not possible as of your UI setup, consider refining your UI and rethink whether everything should be displayed as is. For example, use separate windows or expanders to display details and query the data when they are made visible. This does not only save you time on app startup but also makes sure that you display any data in an up-to-date manner.
This is probably a bit crazy/nuts, but is there any way to perform WPF Automation Testing without using the MS Automation Peer framework?
There are several times when I actually need the UIElement/FrameworkElement and would like to access that directly. I was looking into Snoop to try and deal with it, but it looked like it was a bit painful to "inject" into a process.
I've tried connecting to the process that is launched (we're currently using SpecFlow), but any attempt to marshall the Process or Window thread (I've looked for the actual Window's proc ID and tried connecting) results in a null from the HwndSource. I was trying to also create an instance of the Application inside my unit test, but it looks like NUnit has issues with launching the Application in the background (even on a STA Thread), because the second a test starts trying to load, the Application starts blocking.
Has anyone had any success in doing something else? My only thoughts to "solve" the problem now would be to create a special AutomationPeer that I could reflect to and then get the FrameworkElement from, or something like that.
Any thoughts?
No, there's nothing that can marshal WPF elements (especially underlying native elements such as graphic resources, images, etc.) across processes, and UI Automation works by definition across processes.
If you own the target application, you can dramatically increase its "UIAutomation-ness" if you create AutomationPeers specific for your app.
Otherwise, the snoop way (process injection) is the only way (but has a lot of drawbacks, including security issues).
If this is the case:
any way to perform WPF Automation Testing without using the MS Automation Peer framework?
you can use the Telerik's Testing Studio, it's rich and with some coding you'll get use to it. Another plus is the good documentation and the help that can be provided by them.
To be honest it took me a while, but since the Framework is free I finally decide to use it over the MS alternative.
I have a moderately sized, highly tangled, messy, big ball-of-mud project that I want to refactor. It is about 50k lines of code and I am the only maintainer. One class is 10k LOC long with line widths of 400 characters. Features are still being added, albeit slowly. I feel good about the refactoring part. Ripping apart code is very comfortable to me. However, getting a set of tests around the application to ensure that I don't break it seems daunting. Huge object graphs and mocking out all the databases will be real buggers. I probably could get away with doing massive refactoring without breaking things horribly. But, prudence dictates some level of tests to ensure at least some level of safety. At the same time, I don't want to spend any more than a minimal amount of time getting a set of "behavior lock-down" tests around the code. I fully intend to add a full set of unit tests once I get things a bit decoupled. A rewrite is a non-starter.
Does anyone have any pointers or techniques? Has anyone done this sort of thing before?
Mindset
Generally it can be tough going trying to write automated tests (even high level ones) for applications that were not build with testability in mind.
The best thing is going to be make sure you are disciplined in writing tests as you refactor (which it sounds like you are intending to be). This will slowly turn that ball of code, into an elegant dancing unicorn of well encapsulated testable classes.
Suggestions
Start with creating some manual high level tests (e.g. user goes to page one, clicks on the red button, then a textbox appears..) to have a starting point. Depending on the technology the app is built in there are a few frameworks out there that can help automate these high level (often UI driven) tests:
For web apps Selenium is a great choice, for WPF apps you can use the UI Automation framework, for other application, while its a bit rudimentary, AutoIt can be a life saver.
Here how I do it in c++ (and c-).
Create a directory to house the tests, cd to this directory
Create a directory to house mock objects, say mock-objs.
Create a makefile to compile all object files of interest.
Add necessary include directories or mock .h files to make all object file compile.
Congratulate yourself you are 90% done.
Add a test harness of your choice (e.g. cppunit, atf-tests, google test ..).
Add a null test case - just start, log and declare success.
Add necessary libraries and/or mock .c/.cpp files until link is successful and the very first test passes. Note: all functions in these .c/.cpp mock files should contain only a primitive to fail the test when called.
Congratulate yourself you are 99% done.
Add a primitive event scheduler: say just list of callbacks - so you can post request and receive response from event callback.
Add a primitive timer: say timer wheel or even timer list if you need just a few timers.
Write advance-time function: (a) process all queued events (b)increment current time to the next tick, (c)expire all timers waiting for this tick, (d)process all queued events over and over again until no events left, (e) if end the time advance is not reached go to step(b).
Congratulate yourself: now you can add tests with relative ease: add a test case, modify mock functions as required, repeat.
When doing Acceptance Testings to the GUI, do tests to the GUI actually show the GUI (in a fraction of a second, I suppose) or are they just mock implementations? I'm talking about frameworks as WindowLicker, White, NUnitForms, etc.
I'm going to assume you mean automated testing. My interpretation of acceptance testing for a GUI involves people actually using the GUI to see if it does what it's supposed to do, in which case you obviously need to show the GUI.
In the test cases I've written for GUI components you usually create a genuine UI without mockups, but don't make it visible. Usually that is sufficient, and mockups of the higher level GUI components can be hard. However a lot of GUI components need to be part of a component tree to function This means that you generally need to run on a machine which has a display, even if you don't use it. There are some functions where GUI components behave differently when they are visible, but I've been able to work round that, with a single exception involving JOGL.
Last time i used it, NunitForms would show the actual Form being tested on screen. You could make it render the actual form off screen if you prefer. I'm not sure about the others you mentioned, i have never used them.
I don't know about the others you mentioned but WindowLicker definitely shows the gui. I don't know if it's possible to run it without doing this, I never tried. It also takes longer than 'a fraction of a second'. You can watch it move the mouse pointer around, click on items, 'typing' text, etc.