I have NUnit selenium tests that i have embedded into Unit test and i am trying to run this using TFS Build 2015. All the tests run perfectly fine locally but not from the TFS Server. I enabled code coverage and found out that "Module unittests.dll" works fine and most of the code is covered but for "Seleniumtest.exe" 0 lines covered.
I have read a lot on this problem and came across the idea of enabling the TFS Agent to run in interactive mode but i do not know how to do this from the TFS Server and most of the documentation online is not good enough.Thanks
If you're using the Hosted agent queue, you can't. If you have your own on-premise build agent, you just need to run the agent from a command prompt (as opposed to as a service). That's all that "interactive" means.
You can also use the Test Agent tasks to deploy a test agent to a target machine and invoke the tests on them. This will also allow you to run UI tests even if you're using the hosted agent queue -- you'll be passing off the running of the tests to an on-premise test agent.
On a side note, you should generally not run UI tests as part of a build process, but rather as a step in a release pipeline after the application has been released to a QA environment. UI tests are slow and brittle, and you don't want your CI process to be bogged down with running UI tests.
Related
We have a custom automation framework in place. So for running scripts we have a windows application for trigger some test case. Test cases are simple C# methods and we invoke them using Reflection.
Now My requirement is to integrate this framework with MTM. and would like to have continuous integration in process. This means I want if some build comes then some of the test case will automatically start running to secure our code.
How can I achieve this?
More specifically my requirement is I pushed some code related to some PBI, So the testcase related to that requirement should automatically run.
Generally the CI (continuous integration) is against the build process in TFS.
If you mean run the tests using custom test framework, then you need to create a Test Adapter accordingly, so that VS and the Build server can pick up the tests. Here is a good article for your reference on how to do this: Authoring a new Visual studio unit test adapter
Actually you can run the tests in build process, with the CI set, the new build will be triggered once a new change checked in, then the tests will run automatically. Refer to below articles for details:
vNext build - Testing in Continuous Integration and Continuous
Deployment Workflows
Xaml build - TFS, Automated Testing and Continuous Integration
If you still want to use MTM, you can reference below articles:
Automate a test case in Microsoft Test Manager
Continuous Delivery with TFS: Running Automated Web Tests with
MTM
I was wondering if it would be possible to either write or deploy an existing test runner along side my MVC application. The tests could be referenced from the test Runner application and deployed separately from the main application.
The aim is to run the tests remotely on client site as a sort of diagnostics tool.
It would be great if we could have a little more information when things go wrong.
Is this possible?
I would like to test ASP.NET MVC webpage by MSTest using selenium driver. I suppose I need to start the ISS and start web application somehow, get the url of the web app (localhost at some port) and then start test.
There are many examples how to use selenium driver but they all work with already configured urls.
My expectation is following:
Anyone who just clone the git repository with this web app and open solution in Visual studio, click run all test. The web app is started and test are executed.
It should also work as part of continual integration (for example with Teamcity) where these unittest are executed using MSTest on different computers.
It would be nice if this solution works also on computers without Visual Studio Installed (just nuget, MSBuild, MSTest and various SDKs).
There are a few things you should be able to do here.
1) If you have Visual Studio, you can write a powershell script that will spin up the local site you need to run the tests against. This may help you.
You'll then need to also create a script that will run your selenium scripts after the site starts. I've only done this with Microsoft WebTests, but I've seen the similarities with Selenium. I'd look into running your selenium scripts from a powershell script, then combine these two, and have a batch file which new users can just click and to run the tests.
2) You can certainly run these scripts from TeamCity, there just may be some additional configuration like installing necessary tools on the build server. TeamCity has build in tools to run these tests though, so look to utilize that.
I have been trying for a while and I am not able to run Load Tests with my Build Agent. I have a solution with different projects, running Coded UI Tests, Web Performance Tests; all of them running fine on the Build Agent. The problem is that I can't run Load tests.
I started thinking that it was not possible, but I found this at Visual Studio online documentation:
The build agent can run:
Code Coverage
Coded UI Tests (Requires a build server that is running in interactive
mode)
Database Test Data Generation
Database Unit Tests
Generic Tests
Load Tests
Unit Tests
Ordered Tests
Test Impact Analysis
Web Tests
So I guess it is possible somehow. Does anybody know how to do it?
I'm running some automated Continuous Integration tests through a TFS build. The tests are written using the Microsoft UI Automation Framework on MSTest, and are configured to run locally on the build agent, without any controller or agent.
Now, the tests themselves execute well. The Startup method of the test suite launches the compiled application's executable and the process shows up in the server's task manager. However, since this is done through the Team Foundation Build Service, the UI isn't actually shown, as services are not allowed to interact with the active desktop.
Is there a way to force the application's UI to appear on the active desktop? I'm aware that I can install a test controller and agent that allow tests interaction with the desktop, however this creates additional overhead and some technical problems, as we're using VS2012 against TFS2010, which goes directly to the VS2010 MSTest and thus to the VS2010 test controller & agent, which I cannot configure through VS2012.
I ended up setting up a test controller a test agent. Apparently, with VS2010 it's not possible to install a test agent on the same server with the build controller, so I went for a separate machine altogether.
The tests run nicely now!
With what account are you logged on to the build server?
Only if you are logged on with the build account you will see the interaction with the desktop/UI.